1
0

Compare commits

..

228 Commits

Author SHA1 Message Date
Half-Shot
059274de05 Merge remote-tracking branch 'origin/rav/state_stream_limit_assertion' into hs/sssh-testing-redis-things 2020-04-28 18:36:16 +01:00
Richard van der Hoff
8b6468cc15 changelog 2020-04-28 17:54:46 +01:00
Richard van der Hoff
3778424b0e Fix AssertionErrors being thrown by EventsStream
Part of the problem was that there was an off-by-one error in the assertion,
but also the limit logic was too simple. Fix it all up and add some tests.
2020-04-28 17:54:46 +01:00
Richard van der Hoff
b21490b656 Rework TestReplicationDataHandler
This wasn't very easy to work with: the mock wrapping was largely superfluous,
and it's useful to be able to inspect the received rows, and clear out the
received list.
2020-04-28 17:54:46 +01:00
Richard van der Hoff
dc3e7e16fb Factor out functions for injecting events into database
I want to add some more flexibility to the tools for injecting events into the
database, and I don't want to clutter up HomeserverTestCase with them, so let's
factor them out to a new file.
2020-04-28 17:43:35 +01:00
Erik Johnston
fce663889b Add some replication tests (#7278)
Specifically some tests for the typing stream, which means we test streams that fetch missing updates via HTTP (rather than via the DB).

We also shuffle things around a bit so that we create two separate `HomeServer` objects, rather than trying to insert a slaved store into places.

Note: `test_typing.py` is heavily inspired by `test_receipts.py`
2020-04-28 17:42:03 +01:00
Andrew Morgan
ce207aa0ff Fix typo 'datbases' in ConfigError 2020-04-28 16:53:10 +01:00
Richard van der Hoff
fb8ff79efd Fix collation for postgres for unit tests (#7359)
When running the UTs against a postgres deatbase, we need to set the collation
correctly.
2020-04-28 14:21:48 +01:00
Erik Johnston
38919b521e Run replication streamers on workers (#7146)
Currently we never write to streams from workers, but that will change soon
2020-04-28 13:34:12 +01:00
Richard van der Hoff
07337fe30b Fix incorrect metrics reporting for renew_attestations (#7344)
We need to wait for the renewals to finish, so that the metrics are correctly
reported.
2020-04-27 22:20:10 +01:00
Brendan Abolivier
036fab5d8a Document monitoring workers (#7357)
It doesn't seem to be documented anywhere and means that you suddenly start losing metrics without any obvious reason when you go from monolith to workers (e.g. #7312).
2020-04-27 21:36:47 +02:00
Erik Johnston
0877107001 Add test 2020-04-27 15:29:37 +01:00
lub
aa2492907f Add some explanation to application_services.md (#7091)
Signed-off-by: Simon Körner <git@lubiland.de>
2020-04-27 15:03:09 +01:00
Brendan Abolivier
cc9eceb00d Don't crash when one of the configuration files is empty (#7341)
If the admin adds a `.yaml` file that's either empty or doesn't parse into a dict to a config directory (e.g. `conf.d` for debs installs), stuff like https://github.com/matrix-org/synapse/issues/7322 would happen. This PR checks that the file is correctly parsed into a dict, or ignores it with a warning if it parses into any other type (including `None` for empty files).

Fixes https://github.com/matrix-org/synapse/issues/7322
2020-04-27 15:01:03 +02:00
Erik Johnston
929dbb0fb3 Newsfile 2020-04-27 12:16:34 +01:00
Erik Johnston
659b6bec35 Don't relay REMOTE_SERVER_UP cmds to same conn.
For direct TCP connections we need the master to relay REMOTE_SERVER_UP
commands to the other connections so that all instances get notified
about it. The old implementation just relayed to all connections,
assuming that sending back to the original sender of the command was
safe. This is not true for redis, where commands sent get echoed back to
the sender, which was causing master to effectively infinite loop
sending and then re-receiving REMOTE_SERVER_UP commands that it sent.

The fix is to ensure that we only relay to *other* connections and not
to the connection we received the notification from.

Fixes #7334.
2020-04-27 12:05:57 +01:00
Erik Johnston
70227045ba Pass connection to 'on_*' replication handler 2020-04-27 10:41:15 +01:00
Patrick Cloke
7bfe0902ce Add documentation to the sample config about the templates for SSO. (#7343) 2020-04-24 15:03:49 -04:00
Patrick Cloke
33bceb7f70 Convert some of the federation handler methods to async/await. (#7338) 2020-04-24 14:36:38 -04:00
Richard van der Hoff
69a1ac00b2 Merge pull request #7337 from matrix-org/rav/fix_update_limit_assertion
Fix assertions being thrown by the EventsStream update function
2020-04-24 14:00:29 +01:00
Richard van der Hoff
3655eafe85 changelog 2020-04-24 13:59:21 +01:00
Richard van der Hoff
ce428a1abe Fix EventsStream raising assertions when it falls behind
Figuring out how to correctly limit updates from this stream without dropping
entries is far more complicated than just counting the number of rows being
returned. We need to consider each query separately and, if any one query hits
the limit, truncate the results from the others.

I think this also fixes some potentially long-standing bugs where events or
state changes could get missed if we hit the limit on either query.
2020-04-24 13:59:21 +01:00
Patrick Cloke
68384d96fd Merge branch 'master' into develop 2020-04-23 12:04:50 -04:00
Patrick Cloke
204664d1ad Merge tag 'v1.12.4'
Synapse v1.12.4

Features:

* Always send users their own device updates. (#7160)
* Add support for handling GET requests for account_data on a worker. (#7311)

Bugfixes:

* Fix a bug that prevented cross-signing with users on worker-mode synapses. (#7255)
* Do not treat display names as globs in push rules. (#7271)
* Fix a bug with cross-signing devices belonging to remote users who did not share a
  room with any user on the local homeserver. (#7289)
2020-04-23 12:03:33 -04:00
Patrick Cloke
ce9b62e13f 1.12.4 2020-04-23 10:59:10 -04:00
Richard van der Hoff
9cbdfb3a2f Make it clear that the limit for an update_function is a target 2020-04-23 15:45:12 +01:00
Richard van der Hoff
23b28266ac Remove 'limit' param from get_repl_stream_updates API
there doesn't seem to be much point in passing this limit all around, since
both sides agree it's meant to be 100.
2020-04-23 15:44:35 +01:00
Brendan Abolivier
2e3b9a0fcb Revert "Revert "Merge pull request #7315 from matrix-org/babolivier/request_token""
This reverts commit 1adf6a5587.
2020-04-23 11:23:53 +02:00
Brendan Abolivier
fb825759e3 Merge branch 'master' into develop 2020-04-23 11:23:33 +02:00
Brendan Abolivier
1adf6a5587 Revert "Merge pull request #7315 from matrix-org/babolivier/request_token"
This reverts commit 6f4319368b, reversing
changes made to 0d775fcc2d.
2020-04-23 11:23:10 +02:00
Brendan Abolivier
6f4319368b Merge pull request #7315 from matrix-org/babolivier/request_token
Config option to inhibit 3PID errors on /requestToken
2020-04-23 10:38:57 +02:00
Richard van der Hoff
71a1abb8a1 Stop the master relaying USER_SYNC for other workers (#7318)
Long story short: if we're handling presence on the current worker, we shouldn't be sending USER_SYNC commands over replication.

In an attempt to figure out what is going on here, I ended up refactoring some bits of the presencehandler code, so the first 4 commits here are non-functional refactors to move this code slightly closer to sanity. (There's still plenty to do here :/). Suggest reviewing individual commits.

Fixes (I hope) #7257.
2020-04-22 22:39:04 +01:00
Brendan Abolivier
69ad7cc13b Config option to inhibit 3PID errors on /requestToken
Adds a request_token_inhibit_errors configuration flag (disabled by
default) which, if enabled, change the behaviour of all /requestToken
endpoints so that they return a 200 and a fake sid if the 3PID was/was
not found associated with an account (depending on the endpoint),
instead of an error.

Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2020-04-22 23:38:42 +02:00
Erik Johnston
841c581c40 Fix replication metrics when using redis (#7325) 2020-04-22 16:26:19 +01:00
Richard van der Hoff
f16beaa969 Merge tag 'v1.12.4rc1' into develop
Synapse 1.12.4rc1 (2020-04-22)
==============================

Features
--------

- Always send users their own device updates. ([\#7160](https://github.com/matrix-org/synapse/issues/7160))
- Add support for handling GET requests for `account_data` on a worker. ([\#7311](https://github.com/matrix-org/synapse/issues/7311))

Bugfixes
--------

- Fix a bug that prevented cross-signing with users on worker-mode synapses. ([\#7255](https://github.com/matrix-org/synapse/issues/7255))
- Do not treat display names as globs in push rules. ([\#7271](https://github.com/matrix-org/synapse/issues/7271))
- Fix a bug with cross-signing devices belonging to remote users who did not share a room with any user on the local homeserver. ([\#7289](https://github.com/matrix-org/synapse/issues/7289))
2020-04-22 14:54:08 +01:00
Richard van der Hoff
ba0aac5e44 formatting for the changelog 2020-04-22 14:50:51 +01:00
Richard van der Hoff
82d8b1dd1f Another go at fixing one-word commands (#7326)
I messed this up last time I tried (#7239 / e13c6c7).
2020-04-22 14:34:31 +01:00
Richard van der Hoff
dc8003f921 1.12.4rc1 2020-04-22 14:29:39 +01:00
Richard van der Hoff
83af1079d6 1.12.4rc1 2020-04-22 14:28:23 +01:00
Richard van der Hoff
7c7618c7e3 fix changelog name 2020-04-22 14:01:11 +01:00
Richard van der Hoff
13683a3a22 Extend StreamChangeCache to support multiple entities per stream ID (#7303)
First some background: StreamChangeCache is used to keep track of what "entities" have 
changed since a given stream ID. So for example, we might use it to keep track of when the last
to-device message for a given user was received [1], and hence whether we need to pull any to-device messages from the database on a sync [2].

Now, it turns out that StreamChangeCache didn't support more than one thing being changed at
a given stream_id (this was part of the problem with #7206). However, it's entirely valid to send
to-device messages to more than one user at a time.

As it turns out, this did in fact work, because *some* methods of StreamChangeCache coped
ok with having multiple things changing on the same stream ID, and it seems we never actually
use the methods which don't work on the stream change caches where we allow multiple
changes at the same stream ID. But that feels horribly fragile, hence: let's update
StreamChangeCache to properly support this, and add some typing and some more tests while
we're at it.

[1]: https://github.com/matrix-org/synapse/blob/release-v1.12.3/synapse/storage/data_stores/main/deviceinbox.py#L301
[2]: https://github.com/matrix-org/synapse/blob/release-v1.12.3/synapse/storage/data_stores/main/deviceinbox.py#L47-L51
2020-04-22 13:45:40 +01:00
Dirk Klimpel
6b6685db9f Extend room admin api with additional attributes (#7225) 2020-04-22 13:38:41 +01:00
Richard van der Hoff
2aa5bf13c8 Merge branch 'release-v1.12.4' into develop 2020-04-22 13:09:23 +01:00
Erik Johnston
51f7eaf908 Add ability to run replication protocol over redis. (#7040)
This is configured via the `redis` config options.
2020-04-22 13:07:41 +01:00
Patrick Cloke
51f358e2fe Do not treat display names as globs for push rules. (#7271) 2020-04-22 13:07:12 +01:00
Michael Kaye
5308239d5d Reduce logging verbosity of URL cache cleanup. (#7295) 2020-04-22 07:45:16 -04:00
Andrew Morgan
f89ad3b6df Query missing cross-signing keys on local sig upload (#7289) 2020-04-22 12:29:36 +01:00
Andrew Morgan
ff5604e7f1 import urllib.parse when using urllib.parse.quote (#7319) 2020-04-22 12:18:51 +01:00
Michael Kaye
336989a57f Reduce federation logging on success (#7321)
Splitting based on the response code means we can avoid double logging here and identical information from line 164 while still logging at info if we don't get a good response and need to retry.
2020-04-22 11:18:18 +01:00
Richard van der Hoff
556566f0b8 Fix changelog file
I updated the PR and forgot to update the changelog.
2020-04-21 13:20:16 +01:00
Richard van der Hoff
974c0d726a Support GET account_data requests on a worker (#7311) 2020-04-21 10:46:30 +01:00
Richard van der Hoff
461f01ad43 Merge branch 'release-v1.12.4' into develop 2020-04-20 17:55:39 +01:00
Richard van der Hoff
d41c8f6d4d Revert "Query missing cross-signing keys on local sig upload"
This was incorrectly merged to the release branch before it was ready.

This reverts commit 72fe2affb6.
2020-04-20 17:54:35 +01:00
Richard van der Hoff
a46ff43319 Merge branch 'release-v1.12.4' into develop 2020-04-20 17:42:27 +01:00
David Baker
40f79f58bf Always send the user updates to their own device list (#7160) 2020-04-20 17:20:38 +01:00
Lars Franke
13917232d5 Fix indention in generated config file (#7300)
Also adjust sample_config.yaml

Signed-off-by: Lars Franke <frcl@mailbox.org>
2020-04-20 16:51:27 +01:00
Patrick Cloke
f5ea8b48bd Reject unknown UI auth sessions (instead of silently generating a new one) (#7268) 2020-04-20 08:54:42 -04:00
Richard van der Hoff
0f8f02bc39 On catchup, process each row with its own stream id (#7286)
Other parts of the code (such as the StreamChangeCache) assume that there will
not be multiple changes with the same stream id.

This code was introduced in #7024, and I hope this fixes #7206.
2020-04-20 11:43:29 +01:00
Patrick Cloke
054c231e58 Use a template for the SSO success page to allow for customization. (#7279) 2020-04-17 13:34:55 -04:00
James
701788a227 Added explicit Python build tools to snap requirements (#7213)
Signed-off-by: James Hebden <james@ec0.io>
2020-04-17 17:28:00 +01:00
Andrew Morgan
72fe2affb6 Query missing cross-signing keys on local sig upload
Add changelog

Save retrieved keys to the db

lint

Fix and de-brittle remote result dict processing

Use query_user_devices instead, assume only master, self_signing key types

Make changelog more useful

Remove very specific exception handling

Wrap get_verify_key_from_cross_signing_key in a try/except

Note that _get_e2e_cross_signing_verify_key can raise a SynapseError

lint

Add comment explaining why this is useful

Only fetch master and self_signing key types

Fix log statements, docstrings

Remove extraneous items from remote query try/except

lint

Factor key retrieval out into a separate function

Send device updates, modeled after SigningKeyEduUpdater._handle_signing_key_updates

Update method docstring
2020-04-17 15:47:49 +01:00
Richard van der Hoff
67ff7b8ba0 Improve type checking in replication.tcp.Stream (#7291)
The general idea here is to get rid of the type: ignore annotations on all of the current_token and update_function assignments, which would have caught #7290.

After a bit of experimentation, it seems like the least-awful way to do this is to pass the offending functions in as parameters to the Stream constructor. Unfortunately that means that the concrete implementations no longer have the same constructor signature as Stream itself, which means that it gets hard to correctly annotate STREAMS_MAP.

I've also introduced a couple of new types, to take out some duplication.
2020-04-17 14:49:55 +01:00
nataraj-hates-MS-for-stealing-github
0d775fcc2d Improve example TURN configuration in documentation (#7284) 2020-04-17 08:04:23 -04:00
Tristan Lins
c07fca9e2f Clarify the comments for media_storage_providers options (#7272) 2020-04-17 07:09:33 -04:00
Patrick Cloke
01294e6b3a Do not treat display names as globs for push rules. (#7271) 2020-04-16 10:52:55 -04:00
Richard van der Hoff
d7d42387f5 Fix 'generator object is not subscriptable' error (#7290)
Some of the query functions return generators rather than lists, so we can't
index into the result. Happily we already have a copy of the results.

(think this was introduced in #7024)
2020-04-16 14:37:06 +01:00
Patrick Cloke
eed7c5b89e Convert auth handler to async/await (#7261) 2020-04-15 12:40:18 -04:00
Patrick Cloke
17a2433b0d Add notes to the changelog about an additional SSO template. (#7259) 2020-04-15 10:36:33 -04:00
Andrew Morgan
a48138784e Allow specifying the value of Accept-Language header for URL previews (#7265) 2020-04-15 13:35:29 +01:00
Richard van der Hoff
6fb63d6426 Add some warnings and notes about slow upgrade (#7275)
Fixes #7144.
2020-04-15 12:40:43 +01:00
Richard van der Hoff
f2049a8d21 Fix a potentially-huge sql query (#7274)
We could end up looking up tens of thousands of events, which could cause large
amounts of data to be logged to the postgres log.
2020-04-15 10:16:35 +01:00
Zay11Zay
f1097e7720 Fix the parameters of a test fixture (#7243) 2020-04-14 15:37:28 -04:00
Andrew Morgan
ac6a84818f Only register devices edu handler on the master process (#7255) 2020-04-14 11:36:24 +01:00
Andrew Morgan
fef82f4e22 Only register devices edu handler on the master process (#7255) 2020-04-14 10:09:58 +01:00
Ryan Hovland
4a0dadafbe Add setting to nginx configuration to allow larger file uploads (#7251) 2020-04-13 17:23:36 +01:00
PeerD
f41b742161 Check on room creation if the user is allowed to publish the room to the room directory. (#7260) 2020-04-13 07:42:32 -04:00
Matthew Hodgson
118b58f0c9 typos 2020-04-11 20:55:18 +01:00
Andrew Morgan
ac978ab3da Default PL100 to enable encryption in a room (#7230) 2020-04-09 18:45:38 +01:00
Patrick Cloke
b85d7652ff Do not allow a deactivated user to login via SSO. (#7240) 2020-04-09 13:28:13 -04:00
Andrew Morgan
967f99b9f8 Unblacklist /upgrade creates a new room (#7228)
This commit was originally merged in #7228 but reverted in #7254 as the
associated sytest was not ready yet. Now that it is, we can merge this
again.
2020-04-09 13:53:35 +01:00
Andrew Morgan
a026bdaab7 Add matrix-synapse-shared-secret-auth as an example password provider (#7248) 2020-04-09 12:49:05 +01:00
Andrew Morgan
e3cd28d8a4 Revert "Unblacklist /upgrade creates a new room (#7228)" (#7254)
This reverts commit 59f0ca8b87.
2020-04-09 12:47:16 +01:00
Andrew Morgan
59f0ca8b87 Unblacklist /upgrade creates a new room (#7228) 2020-04-09 12:44:48 +01:00
Richard van der Hoff
5a709630bf Fix --help commandline argument (#7249)
I don't really remember why this was so complicated; I think it dates
back to the time when we had to instantiate the Config classes before
we could call `add_arguments` - ie before #5597. In any case, I don't
think there's a good reason for it any more, and the impact of it
being complicated is that `--help` doesn't work correctly.
2020-04-09 12:44:37 +01:00
Andrew Morgan
7f7f9968b3 Make it clearer to the user that they haven't overridden onLogin/Register (#7237) 2020-04-09 12:38:38 +01:00
Andrew Morgan
55d46da59a Upgrade jQuery to 3.x on fallback login/registration screens (#7236) 2020-04-09 12:23:30 +01:00
Richard van der Hoff
24722de7c8 Fix bad merge of CHANGES.md 2020-04-08 17:41:46 +01:00
Richard van der Hoff
23f8d285eb Remove redundant checks on daemonize from synctl (#7233)
We pass --daemonize on the commandline, which (since at least #4853) overrides
whatever the config file, so there is no need for it to be set in the config
file.
2020-04-08 11:59:47 +01:00
Richard van der Hoff
cae4121484 Make systemd-with-workers doc official (#7234)
Simplify and update this documentation, and make it part of the core dist.
2020-04-08 11:59:26 +01:00
Brendan Abolivier
c11d24d48c Fix changelog for #7235 2020-04-08 11:59:51 +02:00
Andrew Morgan
29b7e22b93 Add documentation to password_providers config option (#7238) 2020-04-08 00:46:50 +01:00
Richard van der Hoff
f31e65a749 bg update to clear out duplicate outbound_device_list_pokes (#7193)
We seem to have some duplicates, which could do with being cleared out.
2020-04-07 23:06:39 +01:00
Richard van der Hoff
aedeedc206 Merge pull request #7239 from matrix-org/rav/replication_cleanup
Miscellaneous cleanups to replication code
2020-04-07 22:00:55 +01:00
Patrick Cloke
1722b8a527 Convert delete_url_cache_media to async/await. (#7241) 2020-04-07 21:56:34 +01:00
Patrick Cloke
d78cb31588 Add typing information to federation_server. (#7219) 2020-04-07 15:03:23 -04:00
Richard van der Hoff
bd2ea3432b changelog 2020-04-07 17:44:51 +01:00
Richard van der Hoff
e13c6c7a96 Handle one-word replication commands correctly
`REPLICATE` is now a valid command, and it's nice if you can issue it from the
console without remembering to call it `REPLICATE ` with a trailing space.
2020-04-07 17:43:46 +01:00
Richard van der Hoff
c3e4b4edb2 Fix warnings about not calling superclass constructor
Separate `SimpleCommand` from `Command`, so that things which don't want to use
the `data` property don't have to, and thus fix the warnings PyCharm was giving
me about not calling `__init__` in the base class.
2020-04-07 17:40:22 +01:00
Richard van der Hoff
6a519a0ca0 Remove vestigal references to SYNC replication command
We've ripped pretty much all of this out: let's remove the remains.
2020-04-07 17:40:07 +01:00
Brendan Abolivier
ec5ac8e2b1 Fix typo in the login fallback javascript (#7235)
* Fix typo in the login fallback javascript

* Changelog
2020-04-07 18:31:50 +02:00
Richard van der Hoff
2e105c156b Remove sent outbound device list pokes from the database (#7192)
They just get in the way.
2020-04-07 15:19:19 +01:00
Erik Johnston
ce72355d7f Fix race in replication (#7226)
Fixes a race between handling `POSITION` and `RDATA` commands. We do this by simply linearizing handling of them.
2020-04-07 11:01:04 +01:00
Erik Johnston
82498ee901 Move server command handling out of TCP protocol (#7187)
This completes the merging of server and client command processing.
2020-04-07 10:51:07 +01:00
Patrick Cloke
71953139d1 Add information about .well-known to Debian installation. (#7227) 2020-04-06 17:02:44 -04:00
Andrew Morgan
4b0f00ad0c Remove stream before/after debug log lines (#7207) 2020-04-06 12:40:34 +01:00
Andrew Morgan
b21000a44f Improve error responses when a remote server doesn't allow you to access its public rooms list (#6899) 2020-04-06 12:35:30 +01:00
Erik Johnston
5016b162fc Move client command handling out of TCP protocol (#7185)
The aim here is to move the command handling out of the TCP protocol classes and to also merge the client and server command handling (so that we can reuse them for redis protocol). This PR simply moves the client paths to the new `ReplicationCommandHandler`, a future PR will move the server paths too.
2020-04-06 09:58:42 +01:00
Brendan Abolivier
d73bf18d13 Server notices: Dissociate room creation/lookup from invite (#7199)
Fixes #6815

Before figuring out whether we should alert a user on MAU, we call get_notice_room_for_user to get some info on the existing server notices room for this user. This function, if the room doesn't exist, creates it and invites the user in it. This means that, if we decide later that no server notice is needed, the user gets invited in a room with no message in it. This happens at every restart of the server, since the room ID returned by get_notice_room_for_user is cached.

This PR fixes that by moving the inviting bit to a dedicated function, that's only called when the server actually needs to send a notice to the user. A potential issue with this approach is that the room that's created by get_notice_room_for_user doesn't match how that same function looks for an existing room (i.e. it creates a room that doesn't have an invite or a join for the current user in it, so it could lead to a new room being created each time a user syncs), but I'm not sure this is a problem given it's cached until the server restarts, so that function won't run very often.

It also renames get_notice_room_for_user into get_or_create_notice_room_for_user to make what it does clearer.
2020-04-04 17:27:45 +02:00
Patrick Cloke
694d8bed0e Support CAS in UI Auth flows. (#7186) 2020-04-03 15:35:05 -04:00
Martin Milata
b0db928c63 Extend web_client_location to handle absolute URLs (#7006)
Log warning when filesystem path is used.

Signed-off-by: Martin Milata <martin@martinmilata.cz>
2020-04-03 11:57:34 -04:00
Amber Brown
334bfdbc90 Add some benchmarks for LruCache (#6446) 2020-04-03 16:31:52 +01:00
Andrew Morgan
07b88c546d Convert http.HTTPStatus objects to their int equivalent (#7188) 2020-04-03 14:26:07 +01:00
Richard van der Hoff
0f05fd1530 Reduce the number of calls to resource.getrusage (#7183)
Let's just call `getrusage` once on each logcontext change, rather than twice.
2020-04-03 13:21:30 +01:00
Richard van der Hoff
fd4c975b5b Merge pull request #7190 from matrix-org/rav/one_bg_update_at_a_time
Only run one background update at a time
2020-04-03 13:17:30 +01:00
Richard van der Hoff
bae32740da Remove some run_in_background calls in replication code (#7203)
By running this stuff with `run_in_background`, it won't be correctly reported
against the relevant CPU usage stats.

Fixes #7202
2020-04-03 12:29:30 +01:00
Richard van der Hoff
6dd6a3557c Merge branch 'master' into develop 2020-04-03 11:29:43 +01:00
Richard van der Hoff
0cbb4808ed Revert "Revert "Merge pull request #7153 from matrix-org/babolivier/sso_whitelist_login_fallback""
This reverts commit 0122ef1037.
2020-04-03 11:28:49 +01:00
Richard van der Hoff
14a8e71297 Revert "Revert "Improve the UX of the login fallback when using SSO (#7152)""
This reverts commit 8d4cbdeaa9.
2020-04-03 11:28:43 +01:00
Richard van der Hoff
883ac4b1bb Merge tag 'v1.12.3'
Synapse 1.12.3 (2020-04-03)
===========================

- Remove the the pin to Pillow 7.0 which was introduced in Synapse 1.12.2, and
correctly fix the issue with building the Debian packages. ([\#7212](https://github.com/matrix-org/synapse/issues/7212))
2020-04-03 11:25:56 +01:00
Richard van der Hoff
cb40b0cb80 Merge tag 'v1.12.2'
Synapse 1.12.2 (2020-04-02)
===========================

This release fixes [an
issue](https://github.com/matrix-org/synapse/issues/7208) with building the
debian packages.

No other significant changes since 1.12.1.
2020-04-03 11:25:42 +01:00
Richard van der Hoff
0122ef1037 Revert "Merge pull request #7153 from matrix-org/babolivier/sso_whitelist_login_fallback"
This was incorrectly merged to master.

This reverts commit 319c41f573, reversing
changes made to 229eb81498.
2020-04-03 11:17:39 +01:00
Richard van der Hoff
8d4cbdeaa9 Revert "Improve the UX of the login fallback when using SSO (#7152)"
This was incorrectly merged to `master` instead of develop.

This reverts commit 90246344e3.
2020-04-03 11:16:41 +01:00
Richard van der Hoff
553c8a9b6b tweak changelog 2020-04-03 11:00:57 +01:00
Richard van der Hoff
29ce90358c 1.12.3 2020-04-03 10:57:07 +01:00
Richard van der Hoff
fcc2de7a0c Update docstring per review comments 2020-04-03 10:51:32 +01:00
Richard van der Hoff
daa1ac89a0 Fix device list update stream ids going backward (#7158)
Occasionally we could get a federation device list update transaction which
looked like:

```
[
    {'edu_type': 'm.device_list_update', 'content': {'user_id': '@user:test', 'device_id': 'D2', 'prev_id': [], 'stream_id': 12, 'deleted': True}},
    {'edu_type': 'm.device_list_update', 'content': {'user_id': '@user:test', 'device_id': 'D1', 'prev_id': [12], 'stream_id': 11, 'deleted': True}},
    {'edu_type': 'm.device_list_update', 'content': {'user_id': '@user:test', 'device_id': 'D3', 'prev_id': [11], 'stream_id': 13, 'deleted': True}}
]
```

Having `stream_ids` which are lower than `prev_ids` looks odd. It might work
(I'm not actually sure), but in any case it doesn't seem like a reasonable
thing to expect other implementations to support.
2020-04-03 10:40:22 +01:00
Richard van der Hoff
6d7cec7a57 Fix the debian build in a better way. (#7212) 2020-04-03 10:23:36 +01:00
Andrew Morgan
f7d6e849b3 Fix changelog wording 2020-04-02 19:08:06 +01:00
Andrew Morgan
08edefe694 1.12.2 2020-04-02 19:02:45 +01:00
Andrew Morgan
ec56620ff6 Pin Pillow>=4.3.0,<7.1.0 to fix dep issue 2020-04-02 18:58:08 +01:00
Andrew Morgan
b730480abb 1.12.1 2020-04-02 18:57:31 +01:00
Andrew Morgan
61bb834364 Merge branch 'master' into develop
* master:
  1.12.1
  Note where bugs were introduced
  1.12.1rc1
  Newsfile
  Rewrite changelog
  Add changelog
  Only import sqlite3 when type checking
  Fix another instance
  Only setdefault for signatures if device has key_json
  Fix starting workers when federation sending not split out.
  Attempt to clarify Python version requirements (#7161)
  Improve the UX of the login fallback when using SSO (#7152)
  Update the wording of the config comment
  Lint
  Changelog
  Regenerate sample config
  Whitelist the login fallback by default for SSO
2020-04-02 16:24:38 +01:00
Andrew Morgan
84a901cf0c Merge tag 'v1.12.1'
Synapse 1.12.1 (2020-04-02)
===========================

No significant changes since 1.12.1rc1.

Synapse 1.12.1rc1 (2020-03-31)
==============================

Bugfixes
--------

- Fix starting workers when federation sending not split out. ([\#7133](https://github.com/matrix-org/synapse/issues/7133)). Introduced in v1.12.0.
- Avoid importing `sqlite3` when using the postgres backend. Contributed by David Vo. ([\#7155](https://github.com/matrix-org/synapse/issues/7155)). Introduced in v1.12.0rc1.
- Fix a bug which could cause outbound federation traffic to stop working if a client uploaded an incorrect e2e device signature. ([\#7177](https://github.com/matrix-org/synapse/issues/7177)). Introduced in v1.11.0.

* tag 'v1.12.1':
  1.12.1
  Note where bugs were introduced
  1.12.1rc1
  Newsfile
  Rewrite changelog
  Add changelog
  Only import sqlite3 when type checking
  Fix another instance
  Only setdefault for signatures if device has key_json
  Fix starting workers when federation sending not split out.
2020-04-02 16:20:54 +01:00
Richard van der Hoff
af47264b78 review comment 2020-04-02 12:04:55 +01:00
Andrew Morgan
529462b5c0 1.12.1 2020-04-02 11:32:16 +01:00
Patrick Cloke
b9930d24a0 Support SAML in the user interactive authentication workflow. (#7102) 2020-04-01 08:48:00 -04:00
Patrick Cloke
468dcc767b Allow admins to create aliases when they are not in the room (#7191) 2020-04-01 08:27:05 -04:00
siroccal
250f87d0de Update postgres.md (#7119) 2020-04-01 12:44:51 +01:00
Erik Johnston
dfa0782254 Remove connections per replication stream metric. (#7195)
This broke in a recent PR (#7024) and is no longer useful due to all
replication clients implicitly subscribing to all streams, so let's
just remove it.
2020-04-01 10:40:46 +01:00
Richard van der Hoff
b413ab8aa6 changelog 2020-03-31 17:44:36 +01:00
Richard van der Hoff
7b608cf468 Only run one background update at a time 2020-03-31 17:43:58 +01:00
Richard van der Hoff
b4c2234232 Make do_next_background_update return a bool
returning a None or an int that we don't use is confusing.
2020-03-31 17:43:58 +01:00
Richard van der Hoff
51f4d52cb4 Set a logging context while running the bg updates
This mostly just reduces the amount of "running from sentinel context" spam
during unittest setup.
2020-03-31 17:43:58 +01:00
Richard van der Hoff
26d17b9bdc Make has_completed_background_updates async
(Almost) everywhere that uses it is happy with an awaitable.
2020-03-31 17:43:58 +01:00
Richard van der Hoff
cfe8c8ab8e Remove unused start_background_update
This was only used in a unit test, so let's just inline it in the test.
2020-03-31 17:24:06 +01:00
Jostein Kjønigsen
2e826cd80c Improve TURN documentation. (#7167) 2020-03-31 15:50:48 +01:00
Richard van der Hoff
60adcbed91 Fix "'NoneType' has no attribute start|stop" logcontext errors (#7181)
Fixes #7179.
2020-03-31 15:18:41 +01:00
Karlinde
fe1580bfd9 Fill in the 'default' field for user-defined push rules (#6639)
Signed-off-by: Karl Linderhed <git@karlinde.se>
2020-03-31 15:08:56 +01:00
Andrew Morgan
b994e86e35 Only setdefault for signatures if device has key_json (#7177) 2020-03-31 14:51:22 +01:00
Patrick Cloke
0a7b0882c1 Fix use of async/await in media code (#7184) 2020-03-31 09:33:02 -04:00
Richard van der Hoff
62a7289133 Fix a bug which could cause incorrect 'cyclic dependency' error. (#7178)
If there was an exception setting up one of the attributes of the Homeserver
god object, then future attempts to fetch that attribute would raise a
confusing "Cyclic dependency" error. Let's make sure that we clear the
`building` flag so that we just get the original exception.

Ref: #7169
2020-03-31 13:09:16 +01:00
Andrew Morgan
677d0edbac Note where bugs were introduced 2020-03-31 11:58:48 +01:00
Andrew Morgan
3fb9fc40f5 1.12.1rc1 2020-03-31 11:49:43 +01:00
Erik Johnston
5d99bde788 Newsfile 2020-03-31 11:30:34 +01:00
Andrew Morgan
2cf115f0ea Rewrite changelog 2020-03-31 11:30:16 +01:00
Andrew Morgan
2cb38ca871 Add changelog 2020-03-31 11:30:05 +01:00
David Vo
5bd2b27525 Only import sqlite3 when type checking
Fixes: #7127
Signed-off-by: David Vo <david@vovo.id.au>
2020-03-31 11:27:17 +01:00
Andrew Morgan
b5d0b038f4 Fix another instance 2020-03-31 11:26:37 +01:00
Andrew Morgan
b5ecafd157 Only setdefault for signatures if device has key_json 2020-03-31 11:26:29 +01:00
Erik Johnston
db098ec994 Fix starting workers when federation sending not split out. 2020-03-31 11:25:21 +01:00
Richard van der Hoff
7966a1cde9 Rewrite prune_old_outbound_device_pokes for efficiency (#7159)
make sure we clear out all but one update for the user
2020-03-30 19:06:52 +01:00
Andrew Morgan
7042840b32 Transfer alias mappings when joining an upgraded room (#6946) 2020-03-30 17:53:25 +01:00
Andrew Morgan
d9f29f8dae Fix a small typo in the metrics_flags config option. (#7171) 2020-03-30 17:38:21 +01:00
Erik Johnston
4f21c33be3 Remove usage of "conn_id" for presence. (#7128)
* Remove `conn_id` usage for UserSyncCommand.

Each tcp replication connection is assigned a "conn_id", which is used
to give an ID to a remotely connected worker. In a redis world, there
will no longer be a one to one mapping between connection and instance,
so instead we need to replace such usages with an ID generated by the
remote instances and included in the replicaiton commands.

This really only effects UserSyncCommand.

* Add CLEAR_USER_SYNCS command that is sent on shutdown.

This should help with the case where a synchrotron gets restarted
gracefully, rather than rely on 5 minute timeout.
2020-03-30 16:37:24 +01:00
David Baker
07569f25d1 Merge pull request #7160 from matrix-org/dbkr/always_send_own_device_list_updates
Always send the user updates to their own device list
2020-03-30 14:34:28 +01:00
Andrew Morgan
104844c1e1 Add explanatory comment 2020-03-30 14:00:11 +01:00
Richard van der Hoff
6486c96b65 Merge pull request #7157 from matrix-org/rev.outbound_device_pokes_tests
Add tests for outbound device pokes
2020-03-30 13:59:07 +01:00
Richard van der Hoff
e577c5d607 Attempt to clarify Python version requirements (#7161)
In particular, we depend on `typing.TYPE_CHECKING`, which is only present in
3.5.2.

It turns out that Ubuntu Xenial, despite having a package called `python 3
(3.5.1-3)`, actually has python 3.5.2, so I think this is fine.
2020-03-30 13:55:01 +01:00
Patrick Cloke
c5f89fba55 Add developer documentation for running a local CAS server (#7147) 2020-03-30 07:28:42 -04:00
David Baker
7406477525 black 2020-03-30 10:18:33 +01:00
David Baker
9fc588e6dc Just add own user ID to the list we track device changes for 2020-03-30 10:11:26 +01:00
Richard van der Hoff
b7da598a61 Always whitelist the login fallback for SSO (#7153)
That fallback sets the redirect URL to itself (so it can process the login
token then return gracefully to the client). This would make it pointless to
ask the user for confirmation, since the URL the confirmation page would be
showing wouldn't be the client's.
2020-03-27 20:24:52 +00:00
Brendan Abolivier
84f7eaed16 Improve the UX of the login fallback when using SSO (#7152)
* Don't show the login forms if we're currently logging in with a
  password or a token.
* Submit directly the SSO login form, showing only a spinner to the
  user, in order to eliminate from the clunkiness of SSO through this
  fallback.
2020-03-27 20:19:54 +00:00
Dirk Klimpel
fb69690761 Admin API to join users to a room. (#7051) 2020-03-27 19:16:43 +00:00
Dirk Klimpel
8327eb9280 Add options to prevent users from changing their profile. (#7096) 2020-03-27 19:15:23 +00:00
txt-file
ae219fb411 update debian installation instructions to recommend installing virtualenv instead of python3-virtualenv (#6892)
* change debian package from python3-virtualenv to virtualenv

The virtualenv package is needed for the virtualenv command. The
virtualenv package depends on python3-virtualenv (at least since
debian jessie) so there is no need to specify python3-virtualenv
additionally.

Signed-off-by: Vieno Hakkerinen <vieno@hakkerinen.eu>

* Add changelog

Co-authored-by: Andrew Morgan <andrew@amorgan.xyz>
2020-03-27 15:02:00 +00:00
Brendan Abolivier
90246344e3 Improve the UX of the login fallback when using SSO (#7152)
* Don't show the login forms if we're currently logging in with a
  password or a token.
* Submit directly the SSO login form, showing only a spinner to the
  user, in order to eliminate from the clunkiness of SSO through this
  fallback.
2020-03-27 15:44:13 +01:00
Brendan Abolivier
319c41f573 Merge pull request #7153 from matrix-org/babolivier/sso_whitelist_login_fallback
Always whitelist the login fallback for SSO
2020-03-27 15:34:41 +01:00
Brendan Abolivier
63aea691a7 Update the wording of the config comment 2020-03-27 15:09:12 +01:00
Andrew Morgan
12aa5a7fa7 Ensure is_verified on /_matrix/client/r0/room_keys/keys is a boolean (#7150) 2020-03-27 13:30:22 +00:00
David Vo
fbf0782c63 Only import sqlite3 when type checking (#7155)
Fixes: #7127
Signed-off-by: David Vo <david@vovo.id.au>
2020-03-27 13:20:00 +00:00
David Baker
16ee97988a Fix undefined variable & remove debug logging 2020-03-27 12:39:54 +00:00
David Baker
a07e03ce90 black 2020-03-27 12:35:32 +00:00
David Baker
d9965fb8d6 changelog 2020-03-27 12:30:59 +00:00
David Baker
09cc058a4c Always send the user updates to their own device list
This will allow clients to notify users about new devices even if
the user isn't in any rooms (yet).
2020-03-27 12:26:47 +00:00
Richard van der Hoff
665630fcaa Add tests for outbound device pokes 2020-03-27 12:01:37 +00:00
Jason Robinson
7496d3d2f6 Merge pull request #7151 from matrix-org/jaywink/saml-redirect-fix
Allow RedirectResponse in SAML response handler
2020-03-26 22:10:31 +02:00
Patrick Cloke
fa4f12102d Refactor the CAS code (move the logic out of the REST layer to a handler) (#7136) 2020-03-26 15:05:26 -04:00
Jason Robinson
55ca6cf88c Update changelog.d/7151.bugfix
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-03-26 20:35:50 +02:00
Brendan Abolivier
bdf3cdaec8 Lint 2020-03-26 19:06:44 +01:00
Brendan Abolivier
48b37f61ce Changelog 2020-03-26 19:02:59 +01:00
Brendan Abolivier
7083147961 Regenerate sample config 2020-03-26 19:01:54 +01:00
Brendan Abolivier
c2ab0b3066 Whitelist the login fallback by default for SSO 2020-03-26 18:58:58 +01:00
Nektarios Katakis
825fb5d0a5 Don't default to an invalid sqlite config if no database configuration is provided (#6573) 2020-03-26 17:13:14 +00:00
Jason Robinson
060e7dce09 Allow RedirectResponse in SAML response handler
Allow custom SAML handlers to redirect after processing an auth response.

Fixes #7149

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2020-03-26 19:02:35 +02:00
Dirk Klimpel
e8e2ddb60a Allow server admins to define and enforce a password policy (MSC2000). (#7118) 2020-03-26 16:51:13 +00:00
Patrick Cloke
1c1242acba Validate that the session is not modified during UI-Auth (#7068) 2020-03-26 07:39:34 -04:00
Aaron Raimist
6ca5e56fd1 Remove unused captcha_bypass_secret option (#7137)
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2020-03-25 17:49:34 +00:00
Erik Johnston
4cff617df1 Move catchup of replication streams to worker. (#7024)
This changes the replication protocol so that the server does not send down `RDATA` for rows that happened before the client connected. Instead, the server will send a `POSITION` and clients then query the database (or master out of band) to get up to date.
2020-03-25 14:54:01 +00:00
Andrew Morgan
7bab642707 Various cleanups to INSTALL.md (#7141) 2020-03-25 13:56:40 +00:00
Erik Johnston
b1cfaf08af Merge pull request #7133 from matrix-org/erikj/fix_worker_startup
Fix starting workers when federation sending not split out.
2020-03-25 09:42:39 +00:00
Richard van der Hoff
28d9d6e8a9 Remove spurious "name" parameter to default_config
this is never set to anything other than "test", and is a source of unnecessary
boilerplate.
2020-03-24 18:33:49 +00:00
Richard van der Hoff
39230d2171 Clean up some LoggingContext stuff (#7120)
* Pull Sentinel out of LoggingContext

... and drop a few unnecessary references to it

* Factor out LoggingContext.current_context

move `current_context` and `set_context` out to top-level functions.

Mostly this means that I can more easily trace what's actually referring to
LoggingContext, but I think it's generally neater.

* move copy-to-parent into `stop`

this really just makes `start` and `stop` more symetric. It also means that it
behaves correctly if you manually `set_log_context` rather than using the
context manager.

* Replace `LoggingContext.alive` with `finished`

Turn `alive` into `finished` and make it a bit better defined.
2020-03-24 14:45:33 +00:00
Naugrimm
1fcf9c6f95 Fix CAS redirect url (#6634)
Build the same service URL when requesting the CAS ticket and when calling the proxyValidate URL.
2020-03-24 11:59:04 +00:00
Erik Johnston
d6828c129f Newsfile 2020-03-24 10:36:44 +00:00
Erik Johnston
c816072d47 Fix starting workers when federation sending not split out. 2020-03-24 10:35:00 +00:00
Patrick Cloke
190ab593b7 Use the proper error code when a canonical alias that does not exist is used. (#7109) 2020-03-23 15:21:54 -04:00
Kartikaya Gupta (kats)
e341518f92 Update pre-built package name for FreeBSD (#7107). (#7107)
Signed-off-by: Kartikaya Gupta <kats@trevize.staktrace.com>
2020-03-23 15:31:02 +00:00
Richard van der Hoff
a564b92d37 Convert *StreamRow classes to inner classes (#7116)
This just helps keep the rows closer to their streams, so that it's easier to
see what the format of each stream is.
2020-03-23 13:59:11 +00:00
Richard van der Hoff
5126cb1253 Merge branch 'master' into develop 2020-03-23 13:54:29 +00:00
Richard van der Hoff
229eb81498 Merge tag 'v1.12.0'
Synapse 1.12.0 (2020-03-23)
===========================

No significant changes since 1.12.0rc1.

Debian packages and Docker images are rebuilt using the latest versions of
dependency libraries, including Twisted 20.3.0. **Please see security advisory
below**.

Security advisory
-----------------

Synapse may be vulnerable to request-smuggling attacks when it is used with a
reverse-proxy. The vulnerabilties are fixed in Twisted 20.3.0, and are
described in
[CVE-2020-10108](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10108)
and
[CVE-2020-10109](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10109).
For a good introduction to this class of request-smuggling attacks, see
https://portswigger.net/research/http-desync-attacks-request-smuggling-reborn.

We are not aware of these vulnerabilities being exploited in the wild, and
do not believe that they are exploitable with current versions of any reverse
proxies. Nevertheless, we recommend that all Synapse administrators ensure that
they have the latest versions of the Twisted library to ensure that their
installation remains secure.

* Administrators using the [`matrix.org` Docker
  image](https://hub.docker.com/r/matrixdotorg/synapse/) or the [Debian/Ubuntu
  packages from
  `matrix.org`](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#matrixorg-packages)
  should ensure that they have version 1.12.0 installed: these images include
  Twisted 20.3.0.
* Administrators who have [installed Synapse from
  source](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#installing-from-source)
  should upgrade Twisted within their virtualenv by running:
  ```sh
  <path_to_virtualenv>/bin/pip install 'Twisted>=20.3.0'
  ```
* Administrators who have installed Synapse from distribution packages should
  consult the information from their distributions.

The `matrix.org` Synapse instance was not vulnerable to these vulnerabilities.

Advance notice of change to the default `git` branch for Synapse
----------------------------------------------------------------

Currently, the default `git` branch for Synapse is `master`, which tracks the
latest release.

After the release of Synapse 1.13.0, we intend to change this default to
`develop`, which is the development tip. This is more consistent with common
practice and modern `git` usage.

Although we try to keep `develop` in a stable state, there may be occasions
where regressions creep in. Developers and distributors who have scripts which
run builds using the default branch of `Synapse` should therefore consider
pinning their scripts to `master`.

Synapse 1.12.0rc1 (2020-03-19)
==============================

Features
--------

- Changes related to room alias management ([MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432)):
  - Publishing/removing a room from the room directory now requires the user to have a power level capable of modifying the canonical alias, instead of the room aliases. ([\#6965](https://github.com/matrix-org/synapse/issues/6965))
  - Validate the `alt_aliases` property of canonical alias events. ([\#6971](https://github.com/matrix-org/synapse/issues/6971))
  - Users with a power level sufficient to modify the canonical alias of a room can now delete room aliases. ([\#6986](https://github.com/matrix-org/synapse/issues/6986))
  - Implement updated authorization rules and redaction rules for aliases events, from [MSC2261](https://github.com/matrix-org/matrix-doc/pull/2261) and [MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432). ([\#7037](https://github.com/matrix-org/synapse/issues/7037))
  - Stop sending m.room.aliases events during room creation and upgrade. ([\#6941](https://github.com/matrix-org/synapse/issues/6941))
  - Synapse no longer uses room alias events to calculate room names for push notifications. ([\#6966](https://github.com/matrix-org/synapse/issues/6966))
  - The room list endpoint no longer returns a list of aliases. ([\#6970](https://github.com/matrix-org/synapse/issues/6970))
  - Remove special handling of aliases events from [MSC2260](https://github.com/matrix-org/matrix-doc/pull/2260) added in v1.10.0rc1. ([\#7034](https://github.com/matrix-org/synapse/issues/7034))
- Expose the `synctl`, `hash_password` and `generate_config` commands in the snapcraft package. Contributed by @devec0. ([\#6315](https://github.com/matrix-org/synapse/issues/6315))
- Check that server_name is correctly set before running database updates. ([\#6982](https://github.com/matrix-org/synapse/issues/6982))
- Break down monthly active users by `appservice_id` and emit via Prometheus. ([\#7030](https://github.com/matrix-org/synapse/issues/7030))
- Render a configurable and comprehensible error page if something goes wrong during the SAML2 authentication process. ([\#7058](https://github.com/matrix-org/synapse/issues/7058), [\#7067](https://github.com/matrix-org/synapse/issues/7067))
- Add an optional parameter to control whether other sessions are logged out when a user's password is modified. ([\#7085](https://github.com/matrix-org/synapse/issues/7085))
- Add prometheus metrics for the number of active pushers. ([\#7103](https://github.com/matrix-org/synapse/issues/7103), [\#7106](https://github.com/matrix-org/synapse/issues/7106))
- Improve performance when making HTTPS requests to sygnal, sydent, etc, by sharing the SSL context object between connections. ([\#7094](https://github.com/matrix-org/synapse/issues/7094))

Bugfixes
--------

- When a user's profile is updated via the admin API, also generate a displayname/avatar update for that user in each room. ([\#6572](https://github.com/matrix-org/synapse/issues/6572))
- Fix a couple of bugs in email configuration handling. ([\#6962](https://github.com/matrix-org/synapse/issues/6962))
- Fix an issue affecting worker-based deployments where replication would stop working, necessitating a full restart, after joining a large room. ([\#6967](https://github.com/matrix-org/synapse/issues/6967))
- Fix `duplicate key` error which was logged when rejoining a room over federation. ([\#6968](https://github.com/matrix-org/synapse/issues/6968))
- Prevent user from setting 'deactivated' to anything other than a bool on the v2 PUT /users Admin API. ([\#6990](https://github.com/matrix-org/synapse/issues/6990))
- Fix py35-old CI by using native tox package. ([\#7018](https://github.com/matrix-org/synapse/issues/7018))
- Fix a bug causing `org.matrix.dummy_event` to be included in responses from `/sync`. ([\#7035](https://github.com/matrix-org/synapse/issues/7035))
- Fix a bug that renders UTF-8 text files incorrectly when loaded from media. Contributed by @TheStranjer. ([\#7044](https://github.com/matrix-org/synapse/issues/7044))
- Fix a bug that would cause Synapse to respond with an error about event visibility if a client tried to request the state of a room at a given token. ([\#7066](https://github.com/matrix-org/synapse/issues/7066))
- Repair a data-corruption issue which was introduced in Synapse 1.10, and fixed in Synapse 1.11, and which could cause `/sync` to return with 404 errors about missing events and unknown rooms. ([\#7070](https://github.com/matrix-org/synapse/issues/7070))
- Fix a bug causing account validity renewal emails to be sent even if the feature is turned off in some cases. ([\#7074](https://github.com/matrix-org/synapse/issues/7074))

Improved Documentation
----------------------

- Updated CentOS8 install instructions. Contributed by Richard Kellner. ([\#6925](https://github.com/matrix-org/synapse/issues/6925))
- Fix `POSTGRES_INITDB_ARGS` in the `contrib/docker/docker-compose.yml` example docker-compose configuration. ([\#6984](https://github.com/matrix-org/synapse/issues/6984))
- Change date in [INSTALL.md](./INSTALL.md#tls-certificates) for last date of getting TLS certificates to November 2019. ([\#7015](https://github.com/matrix-org/synapse/issues/7015))
- Document that the fallback auth endpoints must be routed to the same worker node as the register endpoints. ([\#7048](https://github.com/matrix-org/synapse/issues/7048))

Deprecations and Removals
-------------------------

- Remove the unused query_auth federation endpoint per [MSC2451](https://github.com/matrix-org/matrix-doc/pull/2451). ([\#7026](https://github.com/matrix-org/synapse/issues/7026))

Internal Changes
----------------

- Add type hints to `logging/context.py`. ([\#6309](https://github.com/matrix-org/synapse/issues/6309))
- Add some clarifications to `README.md` in the database schema directory. ([\#6615](https://github.com/matrix-org/synapse/issues/6615))
- Refactoring work in preparation for changing the event redaction algorithm. ([\#6874](https://github.com/matrix-org/synapse/issues/6874), [\#6875](https://github.com/matrix-org/synapse/issues/6875), [\#6983](https://github.com/matrix-org/synapse/issues/6983), [\#7003](https://github.com/matrix-org/synapse/issues/7003))
- Improve performance of v2 state resolution for large rooms. ([\#6952](https://github.com/matrix-org/synapse/issues/6952), [\#7095](https://github.com/matrix-org/synapse/issues/7095))
- Reduce time spent doing GC, by freezing objects on startup. ([\#6953](https://github.com/matrix-org/synapse/issues/6953))
- Minor perfermance fixes to `get_auth_chain_ids`. ([\#6954](https://github.com/matrix-org/synapse/issues/6954))
- Don't record remote cross-signing keys in the `devices` table. ([\#6956](https://github.com/matrix-org/synapse/issues/6956))
- Use flake8-comprehensions to enforce good hygiene of list/set/dict comprehensions. ([\#6957](https://github.com/matrix-org/synapse/issues/6957))
- Merge worker apps together. ([\#6964](https://github.com/matrix-org/synapse/issues/6964), [\#7002](https://github.com/matrix-org/synapse/issues/7002), [\#7055](https://github.com/matrix-org/synapse/issues/7055), [\#7104](https://github.com/matrix-org/synapse/issues/7104))
- Remove redundant `store_room` call from `FederationHandler._process_received_pdu`. ([\#6979](https://github.com/matrix-org/synapse/issues/6979))
- Update warning for incorrect database collation/ctype to include link to documentation. ([\#6985](https://github.com/matrix-org/synapse/issues/6985))
- Add some type annotations to the database storage classes. ([\#6987](https://github.com/matrix-org/synapse/issues/6987))
- Port `synapse.handlers.presence` to async/await. ([\#6991](https://github.com/matrix-org/synapse/issues/6991), [\#7019](https://github.com/matrix-org/synapse/issues/7019))
- Add some type annotations to the federation base & client classes. ([\#6995](https://github.com/matrix-org/synapse/issues/6995))
- Port `synapse.rest.keys` to async/await. ([\#7020](https://github.com/matrix-org/synapse/issues/7020))
- Add a type check to `is_verified` when processing room keys. ([\#7045](https://github.com/matrix-org/synapse/issues/7045))
- Add type annotations and comments to the auth handler. ([\#7063](https://github.com/matrix-org/synapse/issues/7063))
2020-03-23 13:54:17 +00:00
Richard van der Hoff
b3cee0ce67 Fix processing of groups stream, and use symbolic names for streams (#7117)
`groups` != `receipts`

Introduced in #6964
2020-03-23 11:39:36 +00:00
Dionysis Grigoropoulos
96071eea8f Set Referrer-Policy to no-referrer for media (#7009) 2020-03-23 09:48:28 +00:00
Patrick Cloke
477c4f5b1c Clean-up some auth/login REST code (#7115) 2020-03-20 16:22:47 -04:00
Richard van der Hoff
c165c1233b Improve database configuration docs (#6988)
Attempts to clarify the sample config for databases, and add some stuff about
tcp keepalives to `postgres.md`.
2020-03-20 15:24:22 +00:00
Erik Johnston
fdb1344716 Remove concept of a non-limited stream. (#7011) 2020-03-20 14:40:47 +00:00
Patrick Cloke
caec7d4fa0 Convert some of the media REST code to async/await (#7110) 2020-03-20 07:20:02 -04:00
Patrick Cloke
c2db6599c8 Fix a bug in the federation API which could cause occasional "Failed to get PDU" errors (#7089). 2020-03-19 08:22:56 -04:00
Erik Johnston
a319cb1dd1 Change device list streams to have one row per ID (#7010)
* Add 'device_lists_outbound_pokes' as extra table.

This makes sure we check all the relevant tables to get the current max
stream ID.

Currently not doing so isn't problematic as the max stream ID in
`device_lists_outbound_pokes` is the same as in `device_lists_stream`,
however that will change.

* Change device lists stream to have one row per id.

This will make it possible to process the streams more incrementally,
avoiding having to process large chunks at once.

* Change device list replication to match new semantics.

Instead of sending down batches of user ID/host tuples, send down a row
per entity (user ID or host).

* Newsfile

* Remove handling of multiple rows per ID

* Fix worker handling

* Comments from review
2020-03-19 11:36:53 +00:00
Erik Johnston
6e6476ef07 Comments from review 2020-03-18 10:13:55 +00:00
Richard van der Hoff
4ce50519cd Update postgres.md
fix broken link
2020-03-17 18:08:43 +00:00
Erik Johnston
65a941d1f8 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fixup_devices_stream 2020-03-02 16:55:55 +00:00
Erik Johnston
e53744c737 Fix worker handling 2020-03-02 12:52:28 +00:00
Erik Johnston
f70f44abc7 Remove handling of multiple rows per ID 2020-02-28 11:45:35 +00:00
Erik Johnston
59ad93d2a4 Newsfile 2020-02-28 11:27:37 +00:00
Erik Johnston
9ce4e344a8 Change device list replication to match new semantics.
Instead of sending down batches of user ID/host tuples, send down a row
per entity (user ID or host).
2020-02-28 11:25:34 +00:00
Erik Johnston
f5caa1864e Change device lists stream to have one row per id.
This will make it possible to process the streams more incrementally,
avoiding having to process large chunks at once.
2020-02-28 11:21:25 +00:00
Erik Johnston
c3c6c0e622 Add 'device_lists_outbound_pokes' as extra table.
This makes sure we check all the relevant tables to get the current max
stream ID.

Currently not doing so isn't problematic as the max stream ID in
`device_lists_outbound_pokes` is the same as in `device_lists_stream`,
however that will change.
2020-02-28 11:15:11 +00:00
335 changed files with 10716 additions and 4158 deletions

View File

@@ -5,8 +5,6 @@ Message history can be paginated
Can re-join room if re-invited
/upgrade creates a new room
The only membership state included in an initial sync is for all the senders in the timeline
Local device key changes get to remote servers

View File

@@ -1,12 +1,85 @@
Synapse 1.12.0 (2020-03-23)
Next version
============
* New templates (`sso_auth_confirm.html`, `sso_auth_success.html`, and
`sso_account_deactivated.html`) were added to Synapse. If your Synapse is
configured to use SSO and a custom `sso_redirect_confirm_template_dir`
configuration then these templates will need to be duplicated into that
directory.
* Plugins using the `complete_sso_login` method of `synapse.module_api.ModuleApi`
should update to using the async/await version `complete_sso_login_async` which
includes additional checks. The non-async version is considered deprecated.
Synapse 1.12.4 (2020-04-23)
===========================
No significant changes since 1.12.0rc1.
No significant changes.
Synapse 1.12.4rc1 (2020-04-22)
==============================
Features
--------
- Always send users their own device updates. ([\#7160](https://github.com/matrix-org/synapse/issues/7160))
- Add support for handling GET requests for `account_data` on a worker. ([\#7311](https://github.com/matrix-org/synapse/issues/7311))
Bugfixes
--------
- Fix a bug that prevented cross-signing with users on worker-mode synapses. ([\#7255](https://github.com/matrix-org/synapse/issues/7255))
- Do not treat display names as globs in push rules. ([\#7271](https://github.com/matrix-org/synapse/issues/7271))
- Fix a bug with cross-signing devices belonging to remote users who did not share a room with any user on the local homeserver. ([\#7289](https://github.com/matrix-org/synapse/issues/7289))
Synapse 1.12.3 (2020-04-03)
===========================
- Remove the the pin to Pillow 7.0 which was introduced in Synapse 1.12.2, and
correctly fix the issue with building the Debian packages. ([\#7212](https://github.com/matrix-org/synapse/issues/7212))
Synapse 1.12.2 (2020-04-02)
===========================
This release works around [an issue](https://github.com/matrix-org/synapse/issues/7208) with building the debian packages.
No other significant changes since 1.12.1.
Synapse 1.12.1 (2020-04-02)
===========================
No significant changes since 1.12.1rc1.
Synapse 1.12.1rc1 (2020-03-31)
==============================
Bugfixes
--------
- Fix starting workers when federation sending not split out. ([\#7133](https://github.com/matrix-org/synapse/issues/7133)). Introduced in v1.12.0.
- Avoid importing `sqlite3` when using the postgres backend. Contributed by David Vo. ([\#7155](https://github.com/matrix-org/synapse/issues/7155)). Introduced in v1.12.0rc1.
- Fix a bug which could cause outbound federation traffic to stop working if a client uploaded an incorrect e2e device signature. ([\#7177](https://github.com/matrix-org/synapse/issues/7177)). Introduced in v1.11.0.
Synapse 1.12.0 (2020-03-23)
===========================
Debian packages and Docker images are rebuilt using the latest versions of
dependency libraries, including Twisted 20.3.0. **Please see security advisory
below**.
Potential slow database update during upgrade
---------------------------------------------
Synapse 1.12.0 includes a database update which is run as part of the upgrade,
and which may take some time (several hours in the case of a large
server). Synapse will not respond to HTTP requests while this update is taking
place. For imformation on seeing if you are affected, and workaround if you
are, see the [upgrade notes](UPGRADE.rst#upgrading-to-v1120).
Security advisory
-----------------

View File

@@ -2,7 +2,6 @@
- [Installing Synapse](#installing-synapse)
- [Installing from source](#installing-from-source)
- [Platform-Specific Instructions](#platform-specific-instructions)
- [Troubleshooting Installation](#troubleshooting-installation)
- [Prebuilt packages](#prebuilt-packages)
- [Setting up Synapse](#setting-up-synapse)
- [TLS certificates](#tls-certificates)
@@ -10,6 +9,7 @@
- [Registering a user](#registering-a-user)
- [Setting up a TURN server](#setting-up-a-turn-server)
- [URL previews](#url-previews)
- [Troubleshooting Installation](#troubleshooting-installation)
# Choosing your server name
@@ -36,7 +36,7 @@ that your email address is probably `user@example.com` rather than
System requirements:
- POSIX-compliant system (tested on Linux & OS X)
- Python 3.5, 3.6, 3.7 or 3.8.
- Python 3.5.2 or later, up to Python 3.8.
- At least 1GB of free RAM if you want to join large public rooms like #matrix:matrix.org
Synapse is written in Python but some of the libraries it uses are written in
@@ -70,7 +70,7 @@ pip install -U matrix-synapse
```
Before you can start Synapse, you will need to generate a configuration
file. To do this, run (in your virtualenv, as before)::
file. To do this, run (in your virtualenv, as before):
```
cd ~/synapse
@@ -84,22 +84,24 @@ python -m synapse.app.homeserver \
... substituting an appropriate value for `--server-name`.
This command will generate you a config file that you can then customise, but it will
also generate a set of keys for you. These keys will allow your Home Server to
identify itself to other Home Servers, so don't lose or delete them. It would be
also generate a set of keys for you. These keys will allow your homeserver to
identify itself to other homeserver, so don't lose or delete them. It would be
wise to back them up somewhere safe. (If, for whatever reason, you do need to
change your Home Server's keys, you may find that other Home Servers have the
change your homeserver's keys, you may find that other homeserver have the
old key cached. If you update the signing key, you should change the name of the
key in the `<server name>.signing.key` file (the second word) to something
different. See the
[spec](https://matrix.org/docs/spec/server_server/latest.html#retrieving-server-keys)
for more information on key management.)
for more information on key management).
To actually run your new homeserver, pick a working directory for Synapse to
run (e.g. `~/synapse`), and::
run (e.g. `~/synapse`), and:
cd ~/synapse
source env/bin/activate
synctl start
```
cd ~/synapse
source env/bin/activate
synctl start
```
### Platform-Specific Instructions
@@ -110,7 +112,7 @@ Installing prerequisites on Ubuntu or Debian:
```
sudo apt-get install build-essential python3-dev libffi-dev \
python3-pip python3-setuptools sqlite3 \
libssl-dev python3-virtualenv libjpeg-dev libxslt1-dev
libssl-dev virtualenv libjpeg-dev libxslt1-dev
```
#### ArchLinux
@@ -188,7 +190,7 @@ doas pkg_add python libffi py-pip py-setuptools sqlite3 py-virtualenv \
There is currently no port for OpenBSD. Additionally, OpenBSD's security
settings require a slightly more difficult installation process.
XXX: I suspect this is out of date.
(XXX: I suspect this is out of date)
1. Create a new directory in `/usr/local` called `_synapse`. Also, create a
new user called `_synapse` and set that directory as the new user's home.
@@ -196,7 +198,7 @@ XXX: I suspect this is out of date.
write and execute permissions on the same memory space to be run from
`/usr/local`.
2. `su` to the new `_synapse` user and change to their home directory.
3. Create a new virtualenv: `virtualenv -p python2.7 ~/.synapse`
3. Create a new virtualenv: `virtualenv -p python3 ~/.synapse`
4. Source the virtualenv configuration located at
`/usr/local/_synapse/.synapse/bin/activate`. This is done in `ksh` by
using the `.` command, rather than `bash`'s `source`.
@@ -217,45 +219,6 @@ be found at https://docs.microsoft.com/en-us/windows/wsl/install-win10 for
Windows 10 and https://docs.microsoft.com/en-us/windows/wsl/install-on-server
for Windows Server.
### Troubleshooting Installation
XXX a bunch of this is no longer relevant.
Synapse requires pip 8 or later, so if your OS provides too old a version you
may need to manually upgrade it::
sudo pip install --upgrade pip
Installing may fail with `Could not find any downloads that satisfy the requirement pymacaroons-pynacl (from matrix-synapse==0.12.0)`.
You can fix this by manually upgrading pip and virtualenv::
sudo pip install --upgrade virtualenv
You can next rerun `virtualenv -p python3 synapse` to update the virtual env.
Installing may fail during installing virtualenv with `InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.`
You can fix this by manually installing ndg-httpsclient::
pip install --upgrade ndg-httpsclient
Installing may fail with `mock requires setuptools>=17.1. Aborting installation`.
You can fix this by upgrading setuptools::
pip install --upgrade setuptools
If pip crashes mid-installation for reason (e.g. lost terminal), pip may
refuse to run until you remove the temporary installation directory it
created. To reset the installation::
rm -rf /tmp/pip_install_matrix
pip seems to leak *lots* of memory during installation. For instance, a Linux
host with 512MB of RAM may run out of memory whilst installing Twisted. If this
happens, you will have to individually install the dependencies which are
failing, e.g.::
pip install twisted
## Prebuilt packages
As an alternative to installing from source, prebuilt packages are available
@@ -314,7 +277,7 @@ For `buster` and `sid`, Synapse is available in the Debian repositories and
it should be possible to install it with simply:
```
sudo apt install matrix-synapse
sudo apt install matrix-synapse
```
There is also a version of `matrix-synapse` in `stretch-backports`. Please see
@@ -375,15 +338,17 @@ sudo pip install py-bcrypt
Synapse can be found in the void repositories as 'synapse':
xbps-install -Su
xbps-install -S synapse
```
xbps-install -Su
xbps-install -S synapse
```
### FreeBSD
Synapse can be installed via FreeBSD Ports or Packages contributed by Brendan Molloy from:
- Ports: `cd /usr/ports/net-im/py-matrix-synapse && make install clean`
- Packages: `pkg install py27-matrix-synapse`
- Packages: `pkg install py37-matrix-synapse`
### NixOS
@@ -420,6 +385,7 @@ so, you will need to edit `homeserver.yaml`, as follows:
resources:
- names: [client, federation]
```
* You will also need to uncomment the `tls_certificate_path` and
`tls_private_key_path` lines under the `TLS` section. You can either
point these settings at an existing certificate and key, or you can
@@ -427,15 +393,15 @@ so, you will need to edit `homeserver.yaml`, as follows:
for having Synapse automatically provision and renew federation
certificates through ACME can be found at [ACME.md](docs/ACME.md).
Note that, as pointed out in that document, this feature will not
work with installs set up after November 2019.
work with installs set up after November 2019.
If you are using your own certificate, be sure to use a `.pem` file that
includes the full certificate chain including any intermediate certificates
(for instance, if using certbot, use `fullchain.pem` as your certificate, not
`cert.pem`).
For a more detailed guide to configuring your server for federation, see
[federate.md](docs/federate.md)
[federate.md](docs/federate.md).
## Email
@@ -482,7 +448,7 @@ on your server even if `enable_registration` is `false`.
## Setting up a TURN server
For reliable VoIP calls to be routed via this homeserver, you MUST configure
a TURN server. See [docs/turn-howto.md](docs/turn-howto.md) for details.
a TURN server. See [docs/turn-howto.md](docs/turn-howto.md) for details.
## URL previews
@@ -491,10 +457,24 @@ turn it on you must enable the `url_preview_enabled: True` config parameter
and explicitly specify the IP ranges that Synapse is not allowed to spider for
previewing in the `url_preview_ip_range_blacklist` configuration parameter.
This is critical from a security perspective to stop arbitrary Matrix users
spidering 'internal' URLs on your network. At the very least we recommend that
spidering 'internal' URLs on your network. At the very least we recommend that
your loopback and RFC1918 IP addresses are blacklisted.
This also requires the optional lxml and netaddr python dependencies to be
installed. This in turn requires the libxml2 library to be available - on
This also requires the optional `lxml` and `netaddr` python dependencies to be
installed. This in turn requires the `libxml2` library to be available - on
Debian/Ubuntu this means `apt-get install libxml2-dev`, or equivalent for
your OS.
# Troubleshooting Installation
`pip` seems to leak *lots* of memory during installation. For instance, a Linux
host with 512MB of RAM may run out of memory whilst installing Twisted. If this
happens, you will have to individually install the dependencies which are
failing, e.g.:
```
pip install twisted
```
If you have any other problems, feel free to ask in
[#synapse:matrix.org](https://matrix.to/#/#synapse:matrix.org).

View File

@@ -75,6 +75,71 @@ for example:
wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb
dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
Upgrading to v1.12.0
====================
This version includes a database update which is run as part of the upgrade,
and which may take some time (several hours in the case of a large
server). Synapse will not respond to HTTP requests while this update is taking
place.
This is only likely to be a problem in the case of a server which is
participating in many rooms.
0. As with all upgrades, it is recommended that you have a recent backup of
your database which can be used for recovery in the event of any problems.
1. As an initial check to see if you will be affected, you can try running the
following query from the `psql` or `sqlite3` console. It is safe to run it
while Synapse is still running.
.. code:: sql
SELECT MAX(q.v) FROM (
SELECT (
SELECT ej.json AS v
FROM state_events se INNER JOIN event_json ej USING (event_id)
WHERE se.room_id=rooms.room_id AND se.type='m.room.create' AND se.state_key=''
LIMIT 1
) FROM rooms WHERE rooms.room_version IS NULL
) q;
This query will take about the same amount of time as the upgrade process: ie,
if it takes 5 minutes, then it is likely that Synapse will be unresponsive for
5 minutes during the upgrade.
If you consider an outage of this duration to be acceptable, no further
action is necessary and you can simply start Synapse 1.12.0.
If you would prefer to reduce the downtime, continue with the steps below.
2. The easiest workaround for this issue is to manually
create a new index before upgrading. On PostgreSQL, his can be done as follows:
.. code:: sql
CREATE INDEX CONCURRENTLY tmp_upgrade_1_12_0_index
ON state_events(room_id) WHERE type = 'm.room.create';
The above query may take some time, but is also safe to run while Synapse is
running.
We assume that no SQLite users have databases large enough to be
affected. If you *are* affected, you can run a similar query, omitting the
``CONCURRENTLY`` keyword. Note however that this operation may in itself cause
Synapse to stop running for some time. Synapse admins are reminded that
`SQLite is not recommended for use outside a test
environment <https://github.com/matrix-org/synapse/blob/master/README.rst#using-postgresql>`_.
3. Once the index has been created, the ``SELECT`` query in step 1 above should
complete quickly. It is therefore safe to upgrade to Synapse 1.12.0.
4. Once Synapse 1.12.0 has successfully started and is responding to HTTP
requests, the temporary index can be removed:
.. code:: sql
DROP INDEX tmp_upgrade_1_12_0_index;
Upgrading to v1.10.0
====================

1
changelog.d/6446.misc Normal file
View File

@@ -0,0 +1 @@
Add benchmarks for LruCache.

1
changelog.d/6573.bugfix Normal file
View File

@@ -0,0 +1 @@
Don't attempt to use an invalid sqlite config if no database configuration is provided. Contributed by @nekatak.

1
changelog.d/6634.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix single-sign on with CAS systems: pass the same service URL when requesting the CAS ticket and when calling the `proxyValidate` URL. Contributed by @Naugrimm.

1
changelog.d/6639.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix missing field `default` when fetching user-defined push rules.

1
changelog.d/6892.doc Normal file
View File

@@ -0,0 +1 @@
Update Debian installation instructions to recommend installing the `virtualenv` package instead of `python3-virtualenv`.

1
changelog.d/6899.bugfix Normal file
View File

@@ -0,0 +1 @@
Improve error responses when accessing remote public room lists.

1
changelog.d/6946.bugfix Normal file
View File

@@ -0,0 +1 @@
Transfer alias mappings on room upgrade.

1
changelog.d/6988.doc Normal file
View File

@@ -0,0 +1 @@
Improve the documentation for database configuration.

1
changelog.d/7006.feature Normal file
View File

@@ -0,0 +1 @@
Extend the `web_client_location` option to accept an absolute URL to use as a redirect. Adds a warning when running the web client on the same hostname as homeserver. Contributed by Martin Milata.

1
changelog.d/7009.feature Normal file
View File

@@ -0,0 +1 @@
Set `Referrer-Policy` header to `no-referrer` on media downloads.

1
changelog.d/7010.misc Normal file
View File

@@ -0,0 +1 @@
Change device list streams to have one row per ID.

1
changelog.d/7011.misc Normal file
View File

@@ -0,0 +1 @@
Remove concept of a non-limited stream.

1
changelog.d/7024.misc Normal file
View File

@@ -0,0 +1 @@
Move catchup of replication streams logic to worker.

1
changelog.d/7040.feature Normal file
View File

@@ -0,0 +1 @@
Add support for running replication over Redis when using workers.

1
changelog.d/7051.feature Normal file
View File

@@ -0,0 +1 @@
Admin API `POST /_synapse/admin/v1/join/<roomIdOrAlias>` to join users to a room like `auto_join_rooms` for creation of users.

1
changelog.d/7068.bugfix Normal file
View File

@@ -0,0 +1 @@
Ensure that a user inteactive authentication session is tied to a single request.

1
changelog.d/7089.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix a bug in the federation API which could cause occasional "Failed to get PDU" errors.

1
changelog.d/7091.doc Normal file
View File

@@ -0,0 +1 @@
Improve the documentation of application service configuration files.

1
changelog.d/7096.feature Normal file
View File

@@ -0,0 +1 @@
Add options to prevent users from changing their profile or associated 3PIDs.

1
changelog.d/7102.feature Normal file
View File

@@ -0,0 +1 @@
Support SSO in the user interactive authentication workflow.

1
changelog.d/7107.doc Normal file
View File

@@ -0,0 +1 @@
Update pre-built package name for FreeBSD.

1
changelog.d/7109.bugfix Normal file
View File

@@ -0,0 +1 @@
Return the proper error (M_BAD_ALIAS) when a non-existant canonical alias is provided.

1
changelog.d/7110.misc Normal file
View File

@@ -0,0 +1 @@
Convert some of synapse.rest.media to async/await.

1
changelog.d/7115.misc Normal file
View File

@@ -0,0 +1 @@
De-duplicate / remove unused REST code for login and auth.

1
changelog.d/7116.misc Normal file
View File

@@ -0,0 +1 @@
Convert `*StreamRow` classes to inner classes.

1
changelog.d/7117.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix a bug which meant that groups updates were not correctly replicated between workers.

1
changelog.d/7118.feature Normal file
View File

@@ -0,0 +1 @@
Allow server admins to define and enforce a password policy (MSC2000).

1
changelog.d/7119.doc Normal file
View File

@@ -0,0 +1 @@
Update postgres docs with login troubleshooting information.

1
changelog.d/7120.misc Normal file
View File

@@ -0,0 +1 @@
Clean up some LoggingContext code.

1
changelog.d/7128.misc Normal file
View File

@@ -0,0 +1 @@
Add explicit `instance_id` for USER_SYNC commands and remove implicit `conn_id` usage.

1
changelog.d/7133.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix starting workers when federation sending not split out.

1
changelog.d/7136.misc Normal file
View File

@@ -0,0 +1 @@
Refactored the CAS authentication logic to a separate class.

1
changelog.d/7137.removal Normal file
View File

@@ -0,0 +1 @@
Remove nonfunctional `captcha_bypass_secret` option from `homeserver.yaml`.

1
changelog.d/7141.doc Normal file
View File

@@ -0,0 +1 @@
Clean up INSTALL.md a bit.

1
changelog.d/7146.misc Normal file
View File

@@ -0,0 +1 @@
Run replication streamers on workers.

1
changelog.d/7147.doc Normal file
View File

@@ -0,0 +1 @@
Add documentation for running a local CAS server for testing.

1
changelog.d/7150.bugfix Normal file
View File

@@ -0,0 +1 @@
Ensure `is_verified` is a boolean in responses to `GET /_matrix/client/r0/room_keys/keys`. Also warn the user if they forgot the `version` query param.

1
changelog.d/7151.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix error page being shown when a custom SAML handler attempted to redirect when processing an auth response.

1
changelog.d/7152.feature Normal file
View File

@@ -0,0 +1 @@
Improve the support for SSO authentication on the login fallback page.

1
changelog.d/7153.feature Normal file
View File

@@ -0,0 +1 @@
Always whitelist the login fallback in the SSO configuration if `public_baseurl` is set.

1
changelog.d/7155.bugfix Normal file
View File

@@ -0,0 +1 @@
Avoid importing `sqlite3` when using the postgres backend. Contributed by David Vo.

1
changelog.d/7157.misc Normal file
View File

@@ -0,0 +1 @@
Add tests for outbound device pokes.

1
changelog.d/7158.misc Normal file
View File

@@ -0,0 +1 @@
Fix device list update stream ids going backward.

1
changelog.d/7159.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix excessive CPU usage by `prune_old_outbound_device_pokes` job.

1
changelog.d/7167.doc Normal file
View File

@@ -0,0 +1 @@
Improve README.md by being explicit about public IP recommendation for TURN relaying.

1
changelog.d/7171.doc Normal file
View File

@@ -0,0 +1 @@
Fix a small typo in the `metrics_flags` config option.

1
changelog.d/7177.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix a bug which could cause outbound federation traffic to stop working if a client uploaded an incorrect e2e device signature.

1
changelog.d/7178.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix a bug which could cause incorrect 'cyclic dependency' error.

1
changelog.d/7181.misc Normal file
View File

@@ -0,0 +1 @@
Clean up some LoggingContext code.

1
changelog.d/7183.misc Normal file
View File

@@ -0,0 +1 @@
Clean up some LoggingContext code.

1
changelog.d/7184.misc Normal file
View File

@@ -0,0 +1 @@
Convert some of synapse.rest.media to async/await.

1
changelog.d/7185.misc Normal file
View File

@@ -0,0 +1 @@
Move client command handling out of TCP protocol.

1
changelog.d/7186.feature Normal file
View File

@@ -0,0 +1 @@
Support SSO in the user interactive authentication workflow.

1
changelog.d/7187.misc Normal file
View File

@@ -0,0 +1 @@
Move server command handling out of TCP protocol.

1
changelog.d/7188.misc Normal file
View File

@@ -0,0 +1 @@
Fix consistency of HTTP status codes reported in log lines.

1
changelog.d/7190.misc Normal file
View File

@@ -0,0 +1 @@
Only run one background database update at a time.

1
changelog.d/7191.feature Normal file
View File

@@ -0,0 +1 @@
Admin users are no longer required to be in a room to create an alias for it.

1
changelog.d/7192.misc Normal file
View File

@@ -0,0 +1 @@
Remove sent outbound device list pokes from the database.

1
changelog.d/7193.misc Normal file
View File

@@ -0,0 +1 @@
Add a background database update job to clear out duplicate `device_lists_outbound_pokes`.

1
changelog.d/7195.misc Normal file
View File

@@ -0,0 +1 @@
Move catchup of replication streams logic to worker.

1
changelog.d/7199.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix a bug that could cause a user to be invited to a server notices (aka System Alerts) room without any notice being sent.

1
changelog.d/7203.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix some worker-mode replication handling not being correctly recorded in CPU usage stats.

1
changelog.d/7207.misc Normal file
View File

@@ -0,0 +1 @@
Remove some extraneous debugging log lines.

1
changelog.d/7213.misc Normal file
View File

@@ -0,0 +1 @@
Add explicit Python build tooling as dependencies for the snapcraft build.

1
changelog.d/7219.misc Normal file
View File

@@ -0,0 +1 @@
Add typing information to federation server code.

1
changelog.d/7225.misc Normal file
View File

@@ -0,0 +1 @@
Extend room admin api (`GET /_synapse/admin/v1/rooms`) with additional attributes.

1
changelog.d/7226.misc Normal file
View File

@@ -0,0 +1 @@
Move catchup of replication streams logic to worker.

1
changelog.d/7228.misc Normal file
View File

@@ -0,0 +1 @@
Unblacklist '/upgrade creates a new room' sytest for workers.

1
changelog.d/7230.feature Normal file
View File

@@ -0,0 +1 @@
Require admin privileges to enable room encryption by default. This does not affect existing rooms.

1
changelog.d/7233.misc Normal file
View File

@@ -0,0 +1 @@
Remove redundant checks on `daemonize` from synctl.

1
changelog.d/7234.doc Normal file
View File

@@ -0,0 +1 @@
Update the contributed documentation on managing synapse workers with systemd, and bring it into the core distribution.

1
changelog.d/7235.feature Normal file
View File

@@ -0,0 +1 @@
Improve the support for SSO authentication on the login fallback page.

1
changelog.d/7236.misc Normal file
View File

@@ -0,0 +1 @@
Upgrade jQuery to v3.4.1 on fallback login/registration pages.

1
changelog.d/7237.misc Normal file
View File

@@ -0,0 +1 @@
Change log line that told user to implement onLogin/onRegister fallback js functions to a warning, instead of an info, so it's more visible.

1
changelog.d/7238.doc Normal file
View File

@@ -0,0 +1 @@
Add documentation to the `password_providers` config option. Add known password provider implementations to docs.

1
changelog.d/7239.misc Normal file
View File

@@ -0,0 +1 @@
Move catchup of replication streams logic to worker.

1
changelog.d/7240.bugfix Normal file
View File

@@ -0,0 +1 @@
Do not allow a deactivated user to login via SSO.

1
changelog.d/7241.misc Normal file
View File

@@ -0,0 +1 @@
Convert some of synapse.rest.media to async/await.

1
changelog.d/7243.misc Normal file
View File

@@ -0,0 +1 @@
Correct the parameters of a test fixture. Contributed by Isaiah Singletary.

1
changelog.d/7248.doc Normal file
View File

@@ -0,0 +1 @@
Add documentation to the `password_providers` config option. Add known password provider implementations to docs.

1
changelog.d/7249.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix --help command-line argument.

1
changelog.d/7251.doc Normal file
View File

@@ -0,0 +1 @@
Modify suggested nginx reverse proxy configuration to match Synapse's default file upload size. Contributed by @ProCycleDev.

1
changelog.d/7259.bugfix Normal file
View File

@@ -0,0 +1 @@
Do not allow a deactivated user to login via SSO.

1
changelog.d/7260.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix room publish permissions not being checked on room creation.

1
changelog.d/7261.misc Normal file
View File

@@ -0,0 +1 @@
Convert auth handler to async/await.

1
changelog.d/7265.feature Normal file
View File

@@ -0,0 +1 @@
Add a config option for specifying the value of the Accept-Language HTTP header when generating URL previews.

1
changelog.d/7268.bugfix Normal file
View File

@@ -0,0 +1 @@
Reject unknown session IDs during user interactive authentication instead of silently creating a new session.

1
changelog.d/7272.doc Normal file
View File

@@ -0,0 +1 @@
Documentation of media_storage_providers options updated to avoid misunderstandings. Contributed by Tristan Lins.

1
changelog.d/7274.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix a sql query introduced in Synapse 1.12.0 which could cause large amounts of logging to the postgres slow-query log.

1
changelog.d/7278.misc Normal file
View File

@@ -0,0 +1 @@
Add some unit tests for replication.

1
changelog.d/7279.feature Normal file
View File

@@ -0,0 +1 @@
Support SSO in the user interactive authentication workflow.

1
changelog.d/7286.misc Normal file
View File

@@ -0,0 +1 @@
Move catchup of replication streams logic to worker.

1
changelog.d/7290.misc Normal file
View File

@@ -0,0 +1 @@
Move catchup of replication streams logic to worker.

1
changelog.d/7291.misc Normal file
View File

@@ -0,0 +1 @@
Improve typing annotations in `synapse.replication.tcp.streams.Stream`.

1
changelog.d/7295.misc Normal file
View File

@@ -0,0 +1 @@
Reduce log verbosity of url cache cleanup tasks.

1
changelog.d/7300.misc Normal file
View File

@@ -0,0 +1 @@
Fix sample SAML Service Provider configuration. Contributed by @frcl.

Some files were not shown because too many files have changed in this diff Show More