kaslo/synapse
1
0
Fork 0
Code Wiki Activity
18,887 Commits 1,047 Branches 805 Tags
7d33ba70df99eeed84b622279006cc6038ab014f
Commit Graph

149 Commits

Author SHA1 Message Date
Will Hunt
7d33ba70df Merge remote-tracking branch 'origin/release-v1.36' into hs/hacked-together-event-cache 2021-06-16 11:32:27 +01:00
Andrew Morgan
4d6e5a5e99 Use a database table to hold the users that should have full presence sent to them, instead of something in-memory (#9823) 2021-05-18 14:13:45 +01:00
Will Hunt
89ad2f60ca Merge remote-tracking branch 'origin/develop' into hs/hacked-together-event-cache 2021-05-05 14:27:02 +01:00
Richard van der Hoff
3ff2251754 Improved validation for received requests (#9817) 
* Simplify `start_listening` callpath

* Correctly check the size of uploaded files
 2021-04-23 19:20:44 +01:00
Richard van der Hoff
84936e2264 Kill off _PushHTTPChannel. (#9878) 
First of all, a fixup to `FakeChannel` which is needed to make it work with the default HTTP channel implementation.

Secondly, it looks like we no longer need `_PushHTTPChannel`, because as of #8013, the producer that gets attached to the `HTTPChannel` is now an `IPushProducer`. This is good, because it means we can remove a whole load of test-specific boilerplate which causes variation between tests and production.
 2021-04-23 18:40:57 +01:00
Richard van der Hoff
59d24c5bef pass a reactor into SynapseSite (#9874) 2021-04-23 17:06:47 +01:00
Will Hunt
f98d0985d3 Merge remote-tracking branch 'origin/develop' into hs/hacked-together-event-cache 2021-04-20 16:10:16 +01:00
Jonathan de Jong
495b214f4f Fix (final) Bugbear violations (#9838) 2021-04-20 11:50:49 +01:00
Will Hunt
82fcd3cce8 Merge remote-tracking branch 'origin/develop' into hs/hacked-together-event-cache 2021-04-20 00:34:28 +01:00
Erik Johnston
00a6db9676 Move some replication processing out of generic_worker (#9796) 
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
 2021-04-14 17:06:06 +01:00
Jonathan de Jong
4b965c862d Remove redundant "coding: utf-8" lines (#9786) 
Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
 2021-04-14 15:34:27 +01:00
Will Hunt
4be9d0918e Merge remote-tracking branch 'origin/develop' into hs/hacked-together-event-cache 2021-04-13 15:03:34 +01:00
Dan Callahan
1d5f0e3529 Bump black configuration to target py36 (#9781) 
Signed-off-by: Dan Callahan <danc@element.io>
 2021-04-13 10:41:34 +01:00
Patrick Cloke
0b3112123d Use mock from the stdlib. (#9772) 2021-04-09 13:44:38 -04:00
Will Hunt
5f631512b5 Merge remote-tracking branch 'origin/develop' into hs/hacked-together-event-cache 2021-04-09 14:39:26 +01:00
Jonathan de Jong
2ca4e349e9 Bugbear: Add Mutable Parameter fixes (#9682) 
Part of #9366

Adds in fixes for B006 and B008, both relating to mutable parameter lint errors.

Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
 2021-04-08 22:38:54 +01:00
Andrew Morgan
5fdff97719 Fix CI by ignore type for None module import (#9709) 2021-03-29 14:42:38 +01:00
Erik Johnston
b5efcb577e Make it possible to use dmypy (#9692) 
Running `dmypy run` will do a `mypy` check while spinning up a daemon
that makes rerunning `dmypy run` a lot faster.

`dmypy` doesn't support `follow_imports = silent` and has
`local_partial_types` enabled, so this PR enables those options and
fixes the issues that were newly raised. Note that `local_partial_types`
will be enabled by default in upcoming mypy releases.
 2021-03-26 16:49:46 +00:00
Will Hunt
45b1c58898 Merge tag 'v1.30.0rc1' into hs/hacked-together-event-cache 
Synapse 1.30.0rc1 (2021-03-16)
==============================

Note that this release deprecates the ability for appservices to
call `POST /_matrix/client/r0/register`  without the body parameter `type`. Appservice
developers should use a `type` value of `m.login.application_service` as
per [the spec](https://matrix.org/docs/spec/application_service/r0.1.2#server-admin-style-permissions).
In future releases, calling this endpoint with an access token - but without a `m.login.application_service`
type - will fail.

Features
--------

- Add prometheus metrics for number of users successfully registering and logging in. ([\#9510](https://github.com/matrix-org/synapse/issues/9510), [\#9511](https://github.com/matrix-org/synapse/issues/9511), [\#9573](https://github.com/matrix-org/synapse/issues/9573))
- Add `synapse_federation_last_sent_pdu_time` and `synapse_federation_last_received_pdu_time` prometheus metrics, which monitor federation delays by reporting the timestamps of messages sent and received to a set of remote servers. ([\#9540](https://github.com/matrix-org/synapse/issues/9540))
- Add support for generating JSON Web Tokens dynamically for use as OIDC client secrets. ([\#9549](https://github.com/matrix-org/synapse/issues/9549))
- Optimise handling of incomplete room history for incoming federation. ([\#9601](https://github.com/matrix-org/synapse/issues/9601))
- Finalise support for allowing clients to pick an SSO Identity Provider ([MSC2858](https://github.com/matrix-org/matrix-doc/pull/2858)). ([\#9617](https://github.com/matrix-org/synapse/issues/9617))
- Tell spam checker modules about the SSO IdP a user registered through if one was used. ([\#9626](https://github.com/matrix-org/synapse/issues/9626))

Bugfixes
--------

- Fix long-standing bug when generating thumbnails for some images with transparency: `TypeError: cannot unpack non-iterable int object`. ([\#9473](https://github.com/matrix-org/synapse/issues/9473))
- Purge chain cover indexes for events that were purged prior to Synapse v1.29.0. ([\#9542](https://github.com/matrix-org/synapse/issues/9542), [\#9583](https://github.com/matrix-org/synapse/issues/9583))
- Fix bug where federation requests were not correctly retried on 5xx responses. ([\#9567](https://github.com/matrix-org/synapse/issues/9567))
- Fix re-activating an account via the admin API when local passwords are disabled. ([\#9587](https://github.com/matrix-org/synapse/issues/9587))
- Fix a bug introduced in Synapse 1.20 which caused incoming federation transactions to stack up, causing slow recovery from outages. ([\#9597](https://github.com/matrix-org/synapse/issues/9597))
- Fix a bug introduced in v1.28.0 where the OpenID Connect callback endpoint could error with a `MacaroonInitException`. ([\#9620](https://github.com/matrix-org/synapse/issues/9620))
- Fix Internal Server Error on `GET /_synapse/client/saml2/authn_response` request. ([\#9623](https://github.com/matrix-org/synapse/issues/9623))

Updates to the Docker image
---------------------------

- Use jemalloc if available in docker. ([\#8553](https://github.com/matrix-org/synapse/issues/8553))

Improved Documentation
----------------------

- Add relayd entry to reverse proxy example configurations. ([\#9508](https://github.com/matrix-org/synapse/issues/9508))
- Improve the SAML2 upgrade notes for 1.27.0. ([\#9550](https://github.com/matrix-org/synapse/issues/9550))
- Link to the "List user's media" admin API from the media admin API docs. ([\#9571](https://github.com/matrix-org/synapse/issues/9571))
- Clarify the spam checker modules documentation example to mention that `parse_config` is a required method. ([\#9580](https://github.com/matrix-org/synapse/issues/9580))
- Clarify the sample configuration for `stats` settings. ([\#9604](https://github.com/matrix-org/synapse/issues/9604))

Deprecations and Removals
-------------------------

- The `synapse_federation_last_sent_pdu_age` and `synapse_federation_last_received_pdu_age` prometheus metrics have been removed. They are replaced by `synapse_federation_last_sent_pdu_time` and `synapse_federation_last_received_pdu_time`. ([\#9540](https://github.com/matrix-org/synapse/issues/9540))
- Registering an Application Service user without using the `m.login.application_service` login type will be unsupported in an upcoming Synapse release. ([\#9559](https://github.com/matrix-org/synapse/issues/9559))

Internal Changes
----------------

- Add tests to ResponseCache. ([\#9458](https://github.com/matrix-org/synapse/issues/9458))
- Add type hints to purge room and server notice admin API. ([\#9520](https://github.com/matrix-org/synapse/issues/9520))
- Add extra logging to ObservableDeferred when callbacks throw exceptions. ([\#9523](https://github.com/matrix-org/synapse/issues/9523))
- Fix incorrect type hints. ([\#9528](https://github.com/matrix-org/synapse/issues/9528), [\#9543](https://github.com/matrix-org/synapse/issues/9543), [\#9591](https://github.com/matrix-org/synapse/issues/9591), [\#9608](https://github.com/matrix-org/synapse/issues/9608), [\#9618](https://github.com/matrix-org/synapse/issues/9618))
- Add an additional test for purging a room. ([\#9541](https://github.com/matrix-org/synapse/issues/9541))
- Add a `.git-blame-ignore-revs` file with the hashes of auto-formatting. ([\#9560](https://github.com/matrix-org/synapse/issues/9560))
- Increase the threshold before which outbound federation to a server goes into "catch up" mode, which is expensive for the remote server to handle. ([\#9561](https://github.com/matrix-org/synapse/issues/9561))
- Fix spurious errors reported by the `config-lint.sh` script. ([\#9562](https://github.com/matrix-org/synapse/issues/9562))
- Fix type hints and tests for BlacklistingAgentWrapper and BlacklistingReactorWrapper. ([\#9563](https://github.com/matrix-org/synapse/issues/9563))
- Do not have mypy ignore type hints from unpaddedbase64. ([\#9568](https://github.com/matrix-org/synapse/issues/9568))
- Improve efficiency of calculating the auth chain in large rooms. ([\#9576](https://github.com/matrix-org/synapse/issues/9576))
- Convert `synapse.types.Requester` to an `attrs` class. ([\#9586](https://github.com/matrix-org/synapse/issues/9586))
- Add logging for redis connection setup. ([\#9590](https://github.com/matrix-org/synapse/issues/9590))
- Improve logging when processing incoming transactions. ([\#9596](https://github.com/matrix-org/synapse/issues/9596))
- Remove unused `stats.retention` setting, and emit a warning if stats are disabled. ([\#9604](https://github.com/matrix-org/synapse/issues/9604))
- Prevent attempting to bundle aggregations for state events in /context APIs. ([\#9619](https://github.com/matrix-org/synapse/issues/9619))
 2021-03-16 15:40:21 +00:00
Patrick Cloke
d29b71aa50 Fix remaining mypy issues due to Twisted upgrade. (#9608) 2021-03-15 11:14:39 -04:00
Patrick Cloke
55da8df078 Fix additional type hints from Twisted 21.2.0. (#9591) 2021-03-12 11:37:57 -05:00
Patrick Cloke
7fdc6cefb3 Fix additional type hints. (#9543) 
Type hint fixes due to Twisted 21.2.0 adding type hints.
 2021-03-09 07:41:32 -05:00
Will Hunt
4a3260092d Merge remote-tracking branch 'origin/develop' into hs/hacked-together-event-cache 2021-02-25 19:32:18 +00:00
Erik Johnston
2927921942 Clean up ShardedWorkerHandlingConfig (#9466) 
* Split ShardedWorkerHandlingConfig

This is so that we have a type level understanding of when it is safe to
call `get_instance(..)` (as opposed to `should_handle(..)`).

* Remove special cases in ShardedWorkerHandlingConfig.

`ShardedWorkerHandlingConfig` tried to handle the various different ways
it was possible to configure federation senders and pushers. This led to
special cases that weren't hit during testing.

To fix this the handling of the different cases is moved from there and
`generic_worker` into the worker config class. This allows us to have
the logic in one place and allows the rest of the code to ignore the
different cases.
 2021-02-24 13:23:18 +00:00
Will Hunt
52d3e5c796 Add Del/Expire commands to test 2021-02-22 23:58:39 +00:00
Eric Eastwood
0a00b7ff14 Update black, and run auto formatting over the codebase (#9381) 
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
 2021-02-16 22:32:34 +00:00
Erik Johnston
dd8da8c5f6 Precompute joined hosts and store in Redis (#9198) 2021-01-26 13:57:31 +00:00
Richard van der Hoff
c9dd47d668 lint 2020-12-15 22:35:50 +00:00
Richard van der Hoff
394516ad1b Remove spurious "SynapseRequest" result from `make_request" 
This was never used, so let's get rid of it.
 2020-12-15 22:35:40 +00:00
Richard van der Hoff
7eebe4b3fc Replace request.code with channel.code 
The two are equivalent, but really we want to check the HTTP result that got
returned to the channel, not the code that the Request object *intended* to
return to the channel.
 2020-12-15 22:32:12 +00:00
Patrick Cloke
344ab0b53a Default to blacklisting reserved IP ranges and add a whitelist. (#8870) 
This defaults `ip_range_blacklist` to reserved IP ranges and also adds an
`ip_range_whitelist` setting to override it.
 2020-12-09 13:56:06 -05:00
Patrick Cloke
96358cb424 Add authentication to replication endpoints. (#8853) 
Authentication is done by checking a shared secret provided
in the Synapse configuration file.
 2020-12-04 10:56:28 -05:00
Patrick Cloke
b774c555d8 Add additional validation to pusher URLs. (#8865) 
Pusher URLs now must end in `/_matrix/push/v1/notify` per the
specification.
 2020-12-04 10:51:56 -05:00
Richard van der Hoff
ed5172852a Merge pull request #8858 from matrix-org/rav/sso_uia 
UIA: offer only available auth flows
 2020-12-02 20:06:53 +00:00
Richard van der Hoff
7ea85302f3 fix up various test cases 
A few test cases were relying on being able to mount non-client servlets on the
test resource. it's better to give them their own Resources.
 2020-12-02 16:30:01 +00:00
Patrick Cloke
30fba62108 Apply an IP range blacklist to push and key revocation requests. (#8821) 
Replaces the `federation_ip_range_blacklist` configuration setting with an
`ip_range_blacklist` setting with wider scope. It now applies to:

* Federation
* Identity servers
* Push notifications
* Checking key validitity for third-party invite events

The old `federation_ip_range_blacklist` setting is still honored if present, but
with reduced scope (it only applies to federation and identity servers).
 2020-12-02 11:09:24 -05:00
Jonathan de Jong
ca60822b34 Simplify the way the HomeServer object caches its internal attributes. (#8565) 
Changes `@cache_in_self` to use underscore-prefixed attributes.
 2020-11-30 13:28:44 -05:00
Richard van der Hoff
be8fa65d0b Remove redundant calls to render() 2020-11-16 18:24:08 +00:00
Richard van der Hoff
129ae841e5 Make make_request actually render the request 
remove the stubbing out of `request.process`, so that `requestReceived` also renders the request via the appropriate resource.

Replace render() with a stub for now.
 2020-11-16 18:24:00 +00:00
Richard van der Hoff
bebfb9a97b Merge branch 'develop' into rav/pass_site_to_make_request 2020-11-16 15:22:40 +00:00
Richard van der Hoff
791d7cd6f0 Rename create_test_json_resource to create_test_resource (#8759) 
The root resource isn't necessarily a JsonResource, so rename this method
accordingly, and update a couple of test classes to use the method rather than
directly manipulating self.resource.
 2020-11-16 14:45:52 +00:00
Richard van der Hoff
cfd895a22e use global make_request() directly where we have a custom Resource 
Where we want to render a request against a specific Resource, call the global
make_request() function rather than the one in HomeserverTestCase, allowing us
to pass in an appropriate `Site`.
 2020-11-15 23:09:03 +00:00
Erik Johnston
46f4be94b4 Fix race for concurrent downloads of remote media. (#8682) 
Fixes #6755
 2020-10-30 10:55:24 +00:00
Erik Johnston
f21e24ffc2 Add ability for access tokens to belong to one user but grant access to another user. (#8616) 
We do it this way round so that only the "owner" can delete the access token (i.e. `/logout/all` by the "owner" also deletes that token, but `/logout/all` by the "target user" doesn't).

A future PR will add an API for creating such a token.

When the target user and authenticated entity are different the `Processed request` log line will be logged with a: `{@admin:server as @bob:server} ...`. I'm not convinced by that format (especially since it adds spaces in there, making it harder to use `cut -d ' '` to chop off the start of log lines). Suggestions welcome.
 2020-10-29 15:58:44 +00:00
Erik Johnston
9a7e0d2ea6 Don't require hiredis to run unit tests (#8680) 2020-10-29 11:17:35 +00:00
Dan Callahan
aff1eb7c67 Tell Black to format code for Python 3.5 (#8664) 
This allows trailing commas in multi-line arg lists.

Minor, but we might as well keep our formatting current with regard to
our minimum supported Python version.

Signed-off-by: Dan Callahan <danc@element.io>
 2020-10-27 23:26:36 +00:00
Jonathan de Jong
6b5a115c0a Solidify the HomeServer constructor. (#8515) 
This implements a more standard API for instantiating a homeserver and
moves some of the dependency injection into the test suite.

More concretely this stops using `setattr` on all `kwargs` passed to `HomeServer`.
 2020-10-15 15:29:13 -04:00
Richard van der Hoff
4433d01519 Merge pull request #8537 from matrix-org/rav/simplify_locally_reject_invite 
Simplify `_locally_reject_invite`
 2020-10-15 10:20:19 +01:00
Erik Johnston
1264c8ac89 Add basic tests for sync/pagination with vector clock tokens. (#8488) 
These are tests for #8439
 2020-10-14 13:53:20 +01:00
Richard van der Hoff
a34b17e492 Simplify _locally_reject_invite 
Update `EventCreationHandler.create_event` to accept an auth_events param, and
use it in `_locally_reject_invite` instead of reinventing the wheel.
 2020-10-13 23:58:48 +01:00