1
0

Compare commits

...

60 Commits

Author SHA1 Message Date
Erik Johnston
6b2d6fdd33 Add some debugging 2020-05-15 15:33:11 +01:00
Erik Johnston
d263a4de02 Enable moving event persistence off of master 2020-05-14 17:25:42 +01:00
Erik Johnston
66c1dff3ba Use new writers config 2020-05-14 17:25:41 +01:00
Erik Johnston
96b6023e3b Make location of events writer configurable 2020-05-14 17:25:09 +01:00
Erik Johnston
452019064c Allow ReplicationRestResource to be added to workers 2020-05-14 17:25:09 +01:00
Erik Johnston
7c8e09bcf1 Add a worker store for search insertion 2020-05-14 17:25:09 +01:00
Erik Johnston
e7f5ac4ed8 Fix lint 2020-05-14 17:09:58 +01:00
Erik Johnston
208ab7b135 Fix typing and add assertion. 2020-05-14 17:09:58 +01:00
Erik Johnston
41f558ccf7 Newsfile 2020-05-14 17:09:58 +01:00
Erik Johnston
342796d6ac Move push rules ID gen to push rules worker 2020-05-14 17:09:58 +01:00
Erik Johnston
bc3fc3927f Move events ID gens to EventWorkerStore 2020-05-14 17:09:58 +01:00
Erik Johnston
d67a8b5455 Move repliction event stream handling out of slave store 2020-05-14 17:09:58 +01:00
Erik Johnston
4734a7bbe4 Move EventStream handling into default ReplicationDataHandler (#7493)
This is so that the logic can happen on both master and workers when we move event persistence out.
2020-05-14 14:01:39 +01:00
Erik Johnston
1de36407d1 Add instance_map config and route replication calls (#7495) 2020-05-14 14:00:58 +01:00
Richard van der Hoff
dede23ff1e Merge tag 'v1.13.0rc2' into develop
Synapse 1.13.0rc2 (2020-05-14)
==============================

Bugfixes
--------

- Fix a long-standing bug which could cause messages not to be sent over federation, when state events with state keys matching user IDs (such as custom user statuses) were received. ([\#7376](https://github.com/matrix-org/synapse/issues/7376))
- Restore compatibility with non-compliant clients during the user interactive authentication process, fixing a problem introduced in v1.13.0rc1. ([\#7483](https://github.com/matrix-org/synapse/issues/7483))

Internal Changes
----------------

- Fix linting errors in new version of Flake8. ([\#7470](https://github.com/matrix-org/synapse/issues/7470))
2020-05-14 11:46:38 +01:00
Richard van der Hoff
75fbc1a0c6 1.13.0rc2 2020-05-14 11:10:11 +01:00
Richard van der Hoff
623abbeb8d 1.13.0rc2 2020-05-14 11:01:24 +01:00
Richard van der Hoff
a0e063387d Stop get_joined_users corruption from custom statuses (#7376)
Fix a bug where the `get_joined_users` cache could be corrupted by custom
status events (or other state events with a state_key matching the user ID).

The bug was introduced by #2229, but has largely gone unnoticed since then.

Fixes #7099, #7373.
2020-05-14 10:07:54 +01:00
Patrick Cloke
5d64fefd6c Do not validate that the client dict is stable during UI Auth. (#7483)
This backs out some of the validation for the client dictionary and logs if
this changes during a user interactive authentication session instead.
2020-05-13 14:26:44 -04:00
Erik Johnston
1124111a12 Allow censoring of events to happen on workers. (#7492)
This is safe as we can now write to cache invalidation stream on workers, and is required for when we move event persistence off master.
2020-05-13 17:15:40 +01:00
Paul Tötterman
46cb2550bb Fix copypasted comment (#7477)
Signed-off-by: Paul Tötterman <paul.totterman@iki.fi>
2020-05-13 16:55:43 +01:00
Erik Johnston
18c1e52d82 Clean up replication unit tests. (#7490) 2020-05-13 16:01:47 +01:00
Erik Johnston
00ba9c48bf Spelling 2020-05-13 13:38:51 +01:00
Erik Johnston
782e4e64df Shuffle persist event data store functions. (#7440)
The aim here is to get to a stage where we have a `PersistEventStore` that holds all the write methods used during event persistence, so that we can take that class out of the `DataStore` mixin and instansiate it separately. This will allow us to instansiate it on processes other than master, while also ensuring it is only available on processes that are configured to write to events stream.

This is a bit of an architectural change, where we end up with multiple classes per data store (rather than one per data store we have now). We end up having:

1. Storage classes that provide high level APIs that can talk to multiple data stores.
2. Data store modules that consist of classes that must point at the same database instance.
3. Classes in a data store that can be instantiated on processes depending on config.
2020-05-13 13:38:22 +01:00
Patrick Cloke
edd3b0747c Fix new flake8 errors (#7489)
This is a cherry-pick of 1a1da60ad2 (#7470)
to the release-v1.13.0 branch.
2020-05-13 08:24:50 -04:00
Erik Johnston
7ee24c5674 Have all instances correctly respond to REPLICATE command. (#7475)
Before all streams were only written to from master, so only master needed to respond to `REPLICATE` commands.

Before all instances wrote to the cache invalidation stream, but didn't respond to `REPLICATE`. This was a bug, which could lead to missed rows from cache invalidation stream if an instance is restarted, however all the caches would be empty in that case so it wasn't a problem.
2020-05-13 10:27:02 +01:00
Erik Johnston
8ca79613e6 Fix Redis reconnection logic (#7482)
Proactively send out `POSITION` commands (as if we had just received a `REPLICATE`) when we connect to Redis. This is important as other instances won't notice we've connected to issue a `REPLICATE` command (unlike for direct TCP connections). This is only currently an issue if master process reconnects without restarting (if it restarts then it won't have written anything and so other instances probably won't have missed anything).
2020-05-13 09:57:15 +01:00
Patrick Cloke
51fb0fc2e5 Update documentation about SSO mapping providers (#7458) 2020-05-12 10:51:07 -04:00
Erik Johnston
1a1da60ad2 Fix new flake8 errors (#7470) 2020-05-12 11:20:48 +01:00
Patrick Cloke
8c8858e124 Convert federation handler to async/await. (#7459) 2020-05-11 15:12:46 -04:00
Patrick Cloke
be309d99cf Convert search code to async/await. (#7460) 2020-05-11 15:12:39 -04:00
Amber Brown
7cb8b4bc67 Allow configuration of Synapse's cache without using synctl or environment variables (#6391) 2020-05-11 18:45:23 +01:00
Andrew Morgan
a8580c5f19 Remove unused store method get_hosts_in_room (#7448) 2020-05-11 16:55:57 +01:00
Andrew Morgan
5cf758cdd6 Merge branch 'release-v1.13.0' into develop
* release-v1.13.0:
  Don't UPGRADE database rows
  RST indenting
  Put rollback instructions in upgrade notes
  Fix changelog typo
  Oh yeah, RST
  Absolute URL it is then
  Fix upgrade notes link
  Provide summary of upgrade issues in changelog. Fix )
  Move next version notes from changelog to upgrade notes
  Changelog fixes
  1.13.0rc1
  Documentation on setting up redis (#7446)
  Rework UI Auth session validation for registration (#7455)
  Fix errors from malformed log line (#7454)
  Drop support for redis.dbid (#7450)
2020-05-11 16:46:33 +01:00
Andrew Morgan
fa4af2c3af Don't UPGRADE database rows 2020-05-11 16:08:37 +01:00
Andrew Morgan
e0caeedab3 RST indenting 2020-05-11 16:05:23 +01:00
Andrew Morgan
8cced49764 Put rollback instructions in upgrade notes 2020-05-11 16:04:00 +01:00
Andrew Morgan
647a995b96 Fix changelog typo 2020-05-11 15:40:36 +01:00
Andrew Morgan
c3416c888a Oh yeah, RST 2020-05-11 15:37:51 +01:00
Andrew Morgan
b41d7b3969 Absolute URL it is then 2020-05-11 15:36:32 +01:00
Andrew Morgan
3916c655f8 Fix upgrade notes link 2020-05-11 15:34:59 +01:00
Andrew Morgan
1f73d28b6c Provide summary of upgrade issues in changelog. Fix ) 2020-05-11 15:33:53 +01:00
Andrew Morgan
71c9307509 Move next version notes from changelog to upgrade notes 2020-05-11 15:10:08 +01:00
Andrew Morgan
200ad02624 Changelog fixes 2020-05-11 14:56:29 +01:00
Andrew Morgan
20ffaa7209 1.13.0rc1 2020-05-11 14:54:38 +01:00
Neil Johnson
85155654c5 Documentation on setting up redis (#7446) 2020-05-11 13:21:15 +01:00
Patrick Cloke
0ad6d28b0d Rework UI Auth session validation for registration (#7455)
Be less strict about validation of UI authentication sessions during
registration to match client expecations.
2020-05-08 16:08:58 -04:00
Andrew Morgan
67feea8044 Extend spam checker to allow for multiple modules (#7435) 2020-05-08 19:25:48 +01:00
Quentin Gliech
616af44137 Implement OpenID Connect-based login (#7256) 2020-05-08 08:30:40 -04:00
Manuel Stahl
a4a5ec4096 Add room details admin endpoint (#7317) 2020-05-07 15:33:07 -04:00
Richard van der Hoff
aa5aa6f96a Fix errors from malformed log line (#7454) 2020-05-07 19:51:38 +01:00
Richard van der Hoff
da9b2db3af Drop support for redis.dbid (#7450)
Since we only use pubsub, the dbid is irrelevant.
2020-05-07 16:46:15 +01:00
Brendan Abolivier
5bb26b7c4f Merge branch 'release-v1.13.0' into develop 2020-05-07 17:31:19 +02:00
Patrick Cloke
9e0384dd3f Fixes typo (bellow -> below) (#7449) 2020-05-07 09:31:06 -04:00
Patrick Cloke
22246919e3 Add more type hints to SAML handler. (#7445) 2020-05-07 09:30:45 -04:00
Erik Johnston
d7983b63a6 Support any process writing to cache invalidation stream. (#7436) 2020-05-07 13:51:08 +01:00
Brendan Abolivier
d9b8d27494 Add a configuration setting for the dummy event threshold (#7422)
Add dummy_events_threshold which allows configuring the number of forward extremities a room needs for Synapse to send forward extremities in it.
2020-05-07 10:35:23 +01:00
Patrick Cloke
d7c2df2fa3 Improve per-block CPU and DB usage metrics (#7426) 2020-05-06 16:43:39 -04:00
Andrew Morgan
4162c39dcf Port group attestation renewal slow down from matrix-org-hotfixes (#7442) 2020-05-06 20:21:38 +01:00
Richard van der Hoff
e053c86a96 Make redis go faster with hiredis (#7439)
For the record, the reason we need this is as follows:

each RDATA command comes down the redis pipe as a subscription message. txredisapi as written needs at least three reactor ticks to read each subscription message from the tcp buffer. Hence, once the process gets loaded, it starts getting behind, and eventually redis knifes the connection. it then takes ages for the master to work its way through the backlog, before it reconnects again, during which any commands from any workers are dropped.
2020-05-06 17:36:46 +01:00
294 changed files with 5962 additions and 2518 deletions

View File

@@ -1,15 +1,172 @@
Next version
============
Synapse 1.13.0rc2 (2020-05-14)
==============================
* New templates (`sso_auth_confirm.html`, `sso_auth_success.html`, and
`sso_account_deactivated.html`) were added to Synapse. If your Synapse is
configured to use SSO and a custom `sso_redirect_confirm_template_dir`
configuration then these templates will need to be duplicated into that
directory.
Bugfixes
--------
* Plugins using the `complete_sso_login` method of `synapse.module_api.ModuleApi`
should update to using the async/await version `complete_sso_login_async` which
includes additional checks. The non-async version is considered deprecated.
- Fix a long-standing bug which could cause messages not to be sent over federation, when state events with state keys matching user IDs (such as custom user statuses) were received. ([\#7376](https://github.com/matrix-org/synapse/issues/7376))
- Restore compatibility with non-compliant clients during the user interactive authentication process, fixing a problem introduced in v1.13.0rc1. ([\#7483](https://github.com/matrix-org/synapse/issues/7483))
Internal Changes
----------------
- Fix linting errors in new version of Flake8. ([\#7470](https://github.com/matrix-org/synapse/issues/7470))
Synapse 1.13.0rc1 (2020-05-11)
==============================
This release brings some potential changes necessary for certain
configurations of Synapse:
* If your Synapse is configured to use SSO and have a custom
`sso_redirect_confirm_template_dir` configuration option set, you will need
to duplicate the new `sso_auth_confirm.html`, `sso_auth_success.html` and
`sso_account_deactivated.html` templates into that directory.
* Synapse plugins using the `complete_sso_login` method of
`synapse.module_api.ModuleApi` should instead switch to the async/await
version, `complete_sso_login_async`, which includes additional checks. The
former version is now deprecated.
* A bug was introduced in Synapse 1.4.0 which could cause the room directory
to be incomplete or empty if Synapse was upgraded directly from v1.2.1 or
earlier, to versions between v1.4.0 and v1.12.x.
Please review [UPGRADE.rst](UPGRADE.rst) for more details on these changes
and for general upgrade guidance.
Features
--------
- Extend the `web_client_location` option to accept an absolute URL to use as a redirect. Adds a warning when running the web client on the same hostname as homeserver. Contributed by Martin Milata. ([\#7006](https://github.com/matrix-org/synapse/issues/7006))
- Set `Referrer-Policy` header to `no-referrer` on media downloads. ([\#7009](https://github.com/matrix-org/synapse/issues/7009))
- Add support for running replication over Redis when using workers. ([\#7040](https://github.com/matrix-org/synapse/issues/7040), [\#7325](https://github.com/matrix-org/synapse/issues/7325), [\#7352](https://github.com/matrix-org/synapse/issues/7352), [\#7401](https://github.com/matrix-org/synapse/issues/7401), [\#7427](https://github.com/matrix-org/synapse/issues/7427), [\#7439](https://github.com/matrix-org/synapse/issues/7439), [\#7446](https://github.com/matrix-org/synapse/issues/7446), [\#7450](https://github.com/matrix-org/synapse/issues/7450), [\#7454](https://github.com/matrix-org/synapse/issues/7454))
- Admin API `POST /_synapse/admin/v1/join/<roomIdOrAlias>` to join users to a room like `auto_join_rooms` for creation of users. ([\#7051](https://github.com/matrix-org/synapse/issues/7051))
- Add options to prevent users from changing their profile or associated 3PIDs. ([\#7096](https://github.com/matrix-org/synapse/issues/7096))
- Support SSO in the user interactive authentication workflow. ([\#7102](https://github.com/matrix-org/synapse/issues/7102), [\#7186](https://github.com/matrix-org/synapse/issues/7186), [\#7279](https://github.com/matrix-org/synapse/issues/7279), [\#7343](https://github.com/matrix-org/synapse/issues/7343))
- Allow server admins to define and enforce a password policy ([MSC2000](https://github.com/matrix-org/matrix-doc/issues/2000)). ([\#7118](https://github.com/matrix-org/synapse/issues/7118))
- Improve the support for SSO authentication on the login fallback page. ([\#7152](https://github.com/matrix-org/synapse/issues/7152), [\#7235](https://github.com/matrix-org/synapse/issues/7235))
- Always whitelist the login fallback in the SSO configuration if `public_baseurl` is set. ([\#7153](https://github.com/matrix-org/synapse/issues/7153))
- Admin users are no longer required to be in a room to create an alias for it. ([\#7191](https://github.com/matrix-org/synapse/issues/7191))
- Require admin privileges to enable room encryption by default. This does not affect existing rooms. ([\#7230](https://github.com/matrix-org/synapse/issues/7230))
- Add a config option for specifying the value of the Accept-Language HTTP header when generating URL previews. ([\#7265](https://github.com/matrix-org/synapse/issues/7265))
- Allow `/requestToken` endpoints to hide the existence (or lack thereof) of 3PID associations on the homeserver. ([\#7315](https://github.com/matrix-org/synapse/issues/7315))
- Add a configuration setting to tweak the threshold for dummy events. ([\#7422](https://github.com/matrix-org/synapse/issues/7422))
Bugfixes
--------
- Don't attempt to use an invalid sqlite config if no database configuration is provided. Contributed by @nekatak. ([\#6573](https://github.com/matrix-org/synapse/issues/6573))
- Fix single-sign on with CAS systems: pass the same service URL when requesting the CAS ticket and when calling the `proxyValidate` URL. Contributed by @Naugrimm. ([\#6634](https://github.com/matrix-org/synapse/issues/6634))
- Fix missing field `default` when fetching user-defined push rules. ([\#6639](https://github.com/matrix-org/synapse/issues/6639))
- Improve error responses when accessing remote public room lists. ([\#6899](https://github.com/matrix-org/synapse/issues/6899), [\#7368](https://github.com/matrix-org/synapse/issues/7368))
- Transfer alias mappings on room upgrade. ([\#6946](https://github.com/matrix-org/synapse/issues/6946))
- Ensure that a user interactive authentication session is tied to a single request. ([\#7068](https://github.com/matrix-org/synapse/issues/7068), [\#7455](https://github.com/matrix-org/synapse/issues/7455))
- Fix a bug in the federation API which could cause occasional "Failed to get PDU" errors. ([\#7089](https://github.com/matrix-org/synapse/issues/7089))
- Return the proper error (`M_BAD_ALIAS`) when a non-existant canonical alias is provided. ([\#7109](https://github.com/matrix-org/synapse/issues/7109))
- Fix a bug which meant that groups updates were not correctly replicated between workers. ([\#7117](https://github.com/matrix-org/synapse/issues/7117))
- Fix starting workers when federation sending not split out. ([\#7133](https://github.com/matrix-org/synapse/issues/7133))
- Ensure `is_verified` is a boolean in responses to `GET /_matrix/client/r0/room_keys/keys`. Also warn the user if they forgot the `version` query param. ([\#7150](https://github.com/matrix-org/synapse/issues/7150))
- Fix error page being shown when a custom SAML handler attempted to redirect when processing an auth response. ([\#7151](https://github.com/matrix-org/synapse/issues/7151))
- Avoid importing `sqlite3` when using the postgres backend. Contributed by David Vo. ([\#7155](https://github.com/matrix-org/synapse/issues/7155))
- Fix excessive CPU usage by `prune_old_outbound_device_pokes` job. ([\#7159](https://github.com/matrix-org/synapse/issues/7159))
- Fix a bug which could cause outbound federation traffic to stop working if a client uploaded an incorrect e2e device signature. ([\#7177](https://github.com/matrix-org/synapse/issues/7177))
- Fix a bug which could cause incorrect 'cyclic dependency' error. ([\#7178](https://github.com/matrix-org/synapse/issues/7178))
- Fix a bug that could cause a user to be invited to a server notices (aka System Alerts) room without any notice being sent. ([\#7199](https://github.com/matrix-org/synapse/issues/7199))
- Fix some worker-mode replication handling not being correctly recorded in CPU usage stats. ([\#7203](https://github.com/matrix-org/synapse/issues/7203))
- Do not allow a deactivated user to login via SSO. ([\#7240](https://github.com/matrix-org/synapse/issues/7240), [\#7259](https://github.com/matrix-org/synapse/issues/7259))
- Fix --help command-line argument. ([\#7249](https://github.com/matrix-org/synapse/issues/7249))
- Fix room publish permissions not being checked on room creation. ([\#7260](https://github.com/matrix-org/synapse/issues/7260))
- Reject unknown session IDs during user interactive authentication instead of silently creating a new session. ([\#7268](https://github.com/matrix-org/synapse/issues/7268))
- Fix a SQL query introduced in Synapse 1.12.0 which could cause large amounts of logging to the postgres slow-query log. ([\#7274](https://github.com/matrix-org/synapse/issues/7274))
- Persist user interactive authentication sessions across workers and Synapse restarts. ([\#7302](https://github.com/matrix-org/synapse/issues/7302))
- Fixed backwards compatibility logic of the first value of `trusted_third_party_id_servers` being used for `account_threepid_delegates.email`, which occurs when the former, deprecated option is set and the latter is not. ([\#7316](https://github.com/matrix-org/synapse/issues/7316))
- Fix a bug where event updates might not be sent over replication to worker processes after the stream falls behind. ([\#7337](https://github.com/matrix-org/synapse/issues/7337), [\#7358](https://github.com/matrix-org/synapse/issues/7358))
- Fix bad error handling that would cause Synapse to crash if it's provided with a YAML configuration file that's either empty or doesn't parse into a key-value map. ([\#7341](https://github.com/matrix-org/synapse/issues/7341))
- Fix incorrect metrics reporting for `renew_attestations` background task. ([\#7344](https://github.com/matrix-org/synapse/issues/7344))
- Prevent non-federating rooms from appearing in responses to federated `POST /publicRoom` requests when a filter was included. ([\#7367](https://github.com/matrix-org/synapse/issues/7367))
- Fix a bug which would cause the room durectory to be incorrectly populated if Synapse was upgraded directly from v1.2.1 or earlier to v1.4.0 or later. Note that this fix does not apply retrospectively; see the [upgrade notes](UPGRADE.rst#upgrading-to-v1130) for more information. ([\#7387](https://github.com/matrix-org/synapse/issues/7387))
- Fix bug in `EventContext.deserialize`. ([\#7393](https://github.com/matrix-org/synapse/issues/7393))
Improved Documentation
----------------------
- Update Debian installation instructions to recommend installing the `virtualenv` package instead of `python3-virtualenv`. ([\#6892](https://github.com/matrix-org/synapse/issues/6892))
- Improve the documentation for database configuration. ([\#6988](https://github.com/matrix-org/synapse/issues/6988))
- Improve the documentation of application service configuration files. ([\#7091](https://github.com/matrix-org/synapse/issues/7091))
- Update pre-built package name for FreeBSD. ([\#7107](https://github.com/matrix-org/synapse/issues/7107))
- Update postgres docs with login troubleshooting information. ([\#7119](https://github.com/matrix-org/synapse/issues/7119))
- Clean up INSTALL.md a bit. ([\#7141](https://github.com/matrix-org/synapse/issues/7141))
- Add documentation for running a local CAS server for testing. ([\#7147](https://github.com/matrix-org/synapse/issues/7147))
- Improve README.md by being explicit about public IP recommendation for TURN relaying. ([\#7167](https://github.com/matrix-org/synapse/issues/7167))
- Fix a small typo in the `metrics_flags` config option. ([\#7171](https://github.com/matrix-org/synapse/issues/7171))
- Update the contributed documentation on managing synapse workers with systemd, and bring it into the core distribution. ([\#7234](https://github.com/matrix-org/synapse/issues/7234))
- Add documentation to the `password_providers` config option. Add known password provider implementations to docs. ([\#7238](https://github.com/matrix-org/synapse/issues/7238), [\#7248](https://github.com/matrix-org/synapse/issues/7248))
- Modify suggested nginx reverse proxy configuration to match Synapse's default file upload size. Contributed by @ProCycleDev. ([\#7251](https://github.com/matrix-org/synapse/issues/7251))
- Documentation of media_storage_providers options updated to avoid misunderstandings. Contributed by Tristan Lins. ([\#7272](https://github.com/matrix-org/synapse/issues/7272))
- Add documentation on monitoring workers with Prometheus. ([\#7357](https://github.com/matrix-org/synapse/issues/7357))
- Clarify endpoint usage in the users admin api documentation. ([\#7361](https://github.com/matrix-org/synapse/issues/7361))
Deprecations and Removals
-------------------------
- Remove nonfunctional `captcha_bypass_secret` option from `homeserver.yaml`. ([\#7137](https://github.com/matrix-org/synapse/issues/7137))
Internal Changes
----------------
- Add benchmarks for LruCache. ([\#6446](https://github.com/matrix-org/synapse/issues/6446))
- Return total number of users and profile attributes in admin users endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#6881](https://github.com/matrix-org/synapse/issues/6881))
- Change device list streams to have one row per ID. ([\#7010](https://github.com/matrix-org/synapse/issues/7010))
- Remove concept of a non-limited stream. ([\#7011](https://github.com/matrix-org/synapse/issues/7011))
- Move catchup of replication streams logic to worker. ([\#7024](https://github.com/matrix-org/synapse/issues/7024), [\#7195](https://github.com/matrix-org/synapse/issues/7195), [\#7226](https://github.com/matrix-org/synapse/issues/7226), [\#7239](https://github.com/matrix-org/synapse/issues/7239), [\#7286](https://github.com/matrix-org/synapse/issues/7286), [\#7290](https://github.com/matrix-org/synapse/issues/7290), [\#7318](https://github.com/matrix-org/synapse/issues/7318), [\#7326](https://github.com/matrix-org/synapse/issues/7326), [\#7378](https://github.com/matrix-org/synapse/issues/7378), [\#7421](https://github.com/matrix-org/synapse/issues/7421))
- Convert some of synapse.rest.media to async/await. ([\#7110](https://github.com/matrix-org/synapse/issues/7110), [\#7184](https://github.com/matrix-org/synapse/issues/7184), [\#7241](https://github.com/matrix-org/synapse/issues/7241))
- De-duplicate / remove unused REST code for login and auth. ([\#7115](https://github.com/matrix-org/synapse/issues/7115))
- Convert `*StreamRow` classes to inner classes. ([\#7116](https://github.com/matrix-org/synapse/issues/7116))
- Clean up some LoggingContext code. ([\#7120](https://github.com/matrix-org/synapse/issues/7120), [\#7181](https://github.com/matrix-org/synapse/issues/7181), [\#7183](https://github.com/matrix-org/synapse/issues/7183), [\#7408](https://github.com/matrix-org/synapse/issues/7408), [\#7426](https://github.com/matrix-org/synapse/issues/7426))
- Add explicit `instance_id` for USER_SYNC commands and remove implicit `conn_id` usage. ([\#7128](https://github.com/matrix-org/synapse/issues/7128))
- Refactored the CAS authentication logic to a separate class. ([\#7136](https://github.com/matrix-org/synapse/issues/7136))
- Run replication streamers on workers. ([\#7146](https://github.com/matrix-org/synapse/issues/7146))
- Add tests for outbound device pokes. ([\#7157](https://github.com/matrix-org/synapse/issues/7157))
- Fix device list update stream ids going backward. ([\#7158](https://github.com/matrix-org/synapse/issues/7158))
- Use `stream.current_token()` and remove `stream_positions()`. ([\#7172](https://github.com/matrix-org/synapse/issues/7172))
- Move client command handling out of TCP protocol. ([\#7185](https://github.com/matrix-org/synapse/issues/7185))
- Move server command handling out of TCP protocol. ([\#7187](https://github.com/matrix-org/synapse/issues/7187))
- Fix consistency of HTTP status codes reported in log lines. ([\#7188](https://github.com/matrix-org/synapse/issues/7188))
- Only run one background database update at a time. ([\#7190](https://github.com/matrix-org/synapse/issues/7190))
- Remove sent outbound device list pokes from the database. ([\#7192](https://github.com/matrix-org/synapse/issues/7192))
- Add a background database update job to clear out duplicate `device_lists_outbound_pokes`. ([\#7193](https://github.com/matrix-org/synapse/issues/7193))
- Remove some extraneous debugging log lines. ([\#7207](https://github.com/matrix-org/synapse/issues/7207))
- Add explicit Python build tooling as dependencies for the snapcraft build. ([\#7213](https://github.com/matrix-org/synapse/issues/7213))
- Add typing information to federation server code. ([\#7219](https://github.com/matrix-org/synapse/issues/7219))
- Extend room admin api (`GET /_synapse/admin/v1/rooms`) with additional attributes. ([\#7225](https://github.com/matrix-org/synapse/issues/7225))
- Unblacklist '/upgrade creates a new room' sytest for workers. ([\#7228](https://github.com/matrix-org/synapse/issues/7228))
- Remove redundant checks on `daemonize` from synctl. ([\#7233](https://github.com/matrix-org/synapse/issues/7233))
- Upgrade jQuery to v3.4.1 on fallback login/registration pages. ([\#7236](https://github.com/matrix-org/synapse/issues/7236))
- Change log line that told user to implement onLogin/onRegister fallback js functions to a warning, instead of an info, so it's more visible. ([\#7237](https://github.com/matrix-org/synapse/issues/7237))
- Correct the parameters of a test fixture. Contributed by Isaiah Singletary. ([\#7243](https://github.com/matrix-org/synapse/issues/7243))
- Convert auth handler to async/await. ([\#7261](https://github.com/matrix-org/synapse/issues/7261))
- Add some unit tests for replication. ([\#7278](https://github.com/matrix-org/synapse/issues/7278))
- Improve typing annotations in `synapse.replication.tcp.streams.Stream`. ([\#7291](https://github.com/matrix-org/synapse/issues/7291))
- Reduce log verbosity of url cache cleanup tasks. ([\#7295](https://github.com/matrix-org/synapse/issues/7295))
- Fix sample SAML Service Provider configuration. Contributed by @frcl. ([\#7300](https://github.com/matrix-org/synapse/issues/7300))
- Fix StreamChangeCache to work with multiple entities changing on the same stream id. ([\#7303](https://github.com/matrix-org/synapse/issues/7303))
- Fix an incorrect import in IdentityHandler. ([\#7319](https://github.com/matrix-org/synapse/issues/7319))
- Reduce logging verbosity for successful federation requests. ([\#7321](https://github.com/matrix-org/synapse/issues/7321))
- Convert some federation handler code to async/await. ([\#7338](https://github.com/matrix-org/synapse/issues/7338))
- Fix collation for postgres for unit tests. ([\#7359](https://github.com/matrix-org/synapse/issues/7359))
- Convert RegistrationWorkerStore.is_server_admin and dependent code to async/await. ([\#7363](https://github.com/matrix-org/synapse/issues/7363))
- Add an `instance_name` to `RDATA` and `POSITION` replication commands. ([\#7364](https://github.com/matrix-org/synapse/issues/7364))
- Thread through instance name to replication client. ([\#7369](https://github.com/matrix-org/synapse/issues/7369))
- Convert synapse.server_notices to async/await. ([\#7394](https://github.com/matrix-org/synapse/issues/7394))
- Convert synapse.notifier to async/await. ([\#7395](https://github.com/matrix-org/synapse/issues/7395))
- Fix issues with the Python package manifest. ([\#7404](https://github.com/matrix-org/synapse/issues/7404))
- Prevent methods in `synapse.handlers.auth` from polling the homeserver config every request. ([\#7420](https://github.com/matrix-org/synapse/issues/7420))
- Speed up fetching device lists changes when handling `/sync` requests. ([\#7423](https://github.com/matrix-org/synapse/issues/7423))
- Run group attestation renewal in series rather than parallel for performance. ([\#7442](https://github.com/matrix-org/synapse/issues/7442))
Synapse 1.12.4 (2020-04-23)

View File

@@ -78,12 +78,13 @@ for example:
Upgrading to v1.13.0
====================
Incorrect database migration in old synapse versions
----------------------------------------------------
A bug was introduced in Synapse 1.4.0 which could cause the room directory to
be incomplete or empty if Synapse was upgraded directly from v1.2.1 or earlier,
to versions between v1.4.0 and v1.12.x.
be incomplete or empty if Synapse was upgraded directly from v1.2.1 or
earlier, to versions between v1.4.0 and v1.12.x.
This will *not* be a problem for Synapse installations which were:
* created at v1.4.0 or later,
@@ -105,6 +106,42 @@ affected can be repaired as follows:
2. Restart synapse.
New Single Sign-on HTML Templates
---------------------------------
New templates (``sso_auth_confirm.html``, ``sso_auth_success.html``, and
``sso_account_deactivated.html``) were added to Synapse. If your Synapse is
configured to use SSO and a custom ``sso_redirect_confirm_template_dir``
configuration then these templates will need to be copied from
`synapse/res/templates <synapse/res/templates>`_ into that directory.
Synapse SSO Plugins Method Deprecation
--------------------------------------
Plugins using the ``complete_sso_login`` method of
``synapse.module_api.ModuleApi`` should update to using the async/await
version ``complete_sso_login_async`` which includes additional checks. The
non-async version is considered deprecated.
Rolling back to v1.12.4 after a failed upgrade
----------------------------------------------
v1.13.0 includes a lot of large changes. If something problematic occurs, you
may want to roll-back to a previous version of Synapse. Because v1.13.0 also
includes a new database schema version, reverting that version is also required
alongside the generic rollback instructions mentioned above. In short, to roll
back to v1.12.4 you need to:
1. Stop the server
2. Decrease the schema version in the database:
.. code:: sql
UPDATE schema_version SET version = 57;
3. Downgrade Synapse by following the instructions for your installation method
in the "Rolling back to older versions" section above.
Upgrading to v1.12.0
====================

1
changelog.d/6391.feature Normal file
View File

@@ -0,0 +1 @@
Synapse's cache factor can now be configured in `homeserver.yaml` by the `caches.global_factor` setting. Additionally, `caches.per_cache_factors` controls the cache factors for individual caches.

View File

@@ -1 +0,0 @@
Add benchmarks for LruCache.

View File

@@ -1 +0,0 @@
Don't attempt to use an invalid sqlite config if no database configuration is provided. Contributed by @nekatak.

View File

@@ -1 +0,0 @@
Fix single-sign on with CAS systems: pass the same service URL when requesting the CAS ticket and when calling the `proxyValidate` URL. Contributed by @Naugrimm.

View File

@@ -1 +0,0 @@
Fix missing field `default` when fetching user-defined push rules.

View File

@@ -1 +0,0 @@
Return total number of users and profile attributes in admin users endpoint. Contributed by Awesome Technologies Innovationslabor GmbH.

View File

@@ -1 +0,0 @@
Update Debian installation instructions to recommend installing the `virtualenv` package instead of `python3-virtualenv`.

View File

@@ -1 +0,0 @@
Improve error responses when accessing remote public room lists.

View File

@@ -1 +0,0 @@
Transfer alias mappings on room upgrade.

View File

@@ -1 +0,0 @@
Improve the documentation for database configuration.

View File

@@ -1 +0,0 @@
Extend the `web_client_location` option to accept an absolute URL to use as a redirect. Adds a warning when running the web client on the same hostname as homeserver. Contributed by Martin Milata.

View File

@@ -1 +0,0 @@
Set `Referrer-Policy` header to `no-referrer` on media downloads.

View File

@@ -1 +0,0 @@
Change device list streams to have one row per ID.

View File

@@ -1 +0,0 @@
Remove concept of a non-limited stream.

View File

@@ -1 +0,0 @@
Move catchup of replication streams logic to worker.

View File

@@ -1 +0,0 @@
Add support for running replication over Redis when using workers.

View File

@@ -1 +0,0 @@
Admin API `POST /_synapse/admin/v1/join/<roomIdOrAlias>` to join users to a room like `auto_join_rooms` for creation of users.

View File

@@ -1 +0,0 @@
Ensure that a user inteactive authentication session is tied to a single request.

View File

@@ -1 +0,0 @@
Fix a bug in the federation API which could cause occasional "Failed to get PDU" errors.

View File

@@ -1 +0,0 @@
Improve the documentation of application service configuration files.

View File

@@ -1 +0,0 @@
Add options to prevent users from changing their profile or associated 3PIDs.

View File

@@ -1 +0,0 @@
Support SSO in the user interactive authentication workflow.

View File

@@ -1 +0,0 @@
Update pre-built package name for FreeBSD.

View File

@@ -1 +0,0 @@
Return the proper error (M_BAD_ALIAS) when a non-existant canonical alias is provided.

View File

@@ -1 +0,0 @@
Convert some of synapse.rest.media to async/await.

View File

@@ -1 +0,0 @@
De-duplicate / remove unused REST code for login and auth.

View File

@@ -1 +0,0 @@
Convert `*StreamRow` classes to inner classes.

View File

@@ -1 +0,0 @@
Fix a bug which meant that groups updates were not correctly replicated between workers.

View File

@@ -1 +0,0 @@
Allow server admins to define and enforce a password policy (MSC2000).

View File

@@ -1 +0,0 @@
Update postgres docs with login troubleshooting information.

View File

@@ -1 +0,0 @@
Clean up some LoggingContext code.

View File

@@ -1 +0,0 @@
Add explicit `instance_id` for USER_SYNC commands and remove implicit `conn_id` usage.

View File

@@ -1 +0,0 @@
Fix starting workers when federation sending not split out.

View File

@@ -1 +0,0 @@
Refactored the CAS authentication logic to a separate class.

View File

@@ -1 +0,0 @@
Remove nonfunctional `captcha_bypass_secret` option from `homeserver.yaml`.

View File

@@ -1 +0,0 @@
Clean up INSTALL.md a bit.

View File

@@ -1 +0,0 @@
Run replication streamers on workers.

View File

@@ -1 +0,0 @@
Add documentation for running a local CAS server for testing.

View File

@@ -1 +0,0 @@
Ensure `is_verified` is a boolean in responses to `GET /_matrix/client/r0/room_keys/keys`. Also warn the user if they forgot the `version` query param.

View File

@@ -1 +0,0 @@
Fix error page being shown when a custom SAML handler attempted to redirect when processing an auth response.

View File

@@ -1 +0,0 @@
Improve the support for SSO authentication on the login fallback page.

View File

@@ -1 +0,0 @@
Always whitelist the login fallback in the SSO configuration if `public_baseurl` is set.

View File

@@ -1 +0,0 @@
Avoid importing `sqlite3` when using the postgres backend. Contributed by David Vo.

View File

@@ -1 +0,0 @@
Add tests for outbound device pokes.

View File

@@ -1 +0,0 @@
Fix device list update stream ids going backward.

View File

@@ -1 +0,0 @@
Fix excessive CPU usage by `prune_old_outbound_device_pokes` job.

View File

@@ -1 +0,0 @@
Improve README.md by being explicit about public IP recommendation for TURN relaying.

View File

@@ -1 +0,0 @@
Fix a small typo in the `metrics_flags` config option.

View File

@@ -1 +0,0 @@
Use `stream.current_token()` and remove `stream_positions()`.

View File

@@ -1 +0,0 @@
Fix a bug which could cause outbound federation traffic to stop working if a client uploaded an incorrect e2e device signature.

View File

@@ -1 +0,0 @@
Fix a bug which could cause incorrect 'cyclic dependency' error.

View File

@@ -1 +0,0 @@
Clean up some LoggingContext code.

View File

@@ -1 +0,0 @@
Clean up some LoggingContext code.

View File

@@ -1 +0,0 @@
Convert some of synapse.rest.media to async/await.

View File

@@ -1 +0,0 @@
Move client command handling out of TCP protocol.

View File

@@ -1 +0,0 @@
Support SSO in the user interactive authentication workflow.

View File

@@ -1 +0,0 @@
Move server command handling out of TCP protocol.

View File

@@ -1 +0,0 @@
Fix consistency of HTTP status codes reported in log lines.

View File

@@ -1 +0,0 @@
Only run one background database update at a time.

View File

@@ -1 +0,0 @@
Admin users are no longer required to be in a room to create an alias for it.

View File

@@ -1 +0,0 @@
Remove sent outbound device list pokes from the database.

View File

@@ -1 +0,0 @@
Add a background database update job to clear out duplicate `device_lists_outbound_pokes`.

View File

@@ -1 +0,0 @@
Move catchup of replication streams logic to worker.

View File

@@ -1 +0,0 @@
Fix a bug that could cause a user to be invited to a server notices (aka System Alerts) room without any notice being sent.

View File

@@ -1 +0,0 @@
Fix some worker-mode replication handling not being correctly recorded in CPU usage stats.

View File

@@ -1 +0,0 @@
Remove some extraneous debugging log lines.

View File

@@ -1 +0,0 @@
Add explicit Python build tooling as dependencies for the snapcraft build.

View File

@@ -1 +0,0 @@
Add typing annotations in `synapse.federation`.

View File

@@ -1 +0,0 @@
Extend room admin api (`GET /_synapse/admin/v1/rooms`) with additional attributes.

View File

@@ -1 +0,0 @@
Move catchup of replication streams logic to worker.

View File

@@ -1 +0,0 @@
Unblacklist '/upgrade creates a new room' sytest for workers.

View File

@@ -1 +0,0 @@
Require admin privileges to enable room encryption by default. This does not affect existing rooms.

View File

@@ -1 +0,0 @@
Remove redundant checks on `daemonize` from synctl.

View File

@@ -1 +0,0 @@
Update the contributed documentation on managing synapse workers with systemd, and bring it into the core distribution.

View File

@@ -1 +0,0 @@
Improve the support for SSO authentication on the login fallback page.

View File

@@ -1 +0,0 @@
Upgrade jQuery to v3.4.1 on fallback login/registration pages.

View File

@@ -1 +0,0 @@
Change log line that told user to implement onLogin/onRegister fallback js functions to a warning, instead of an info, so it's more visible.

View File

@@ -1 +0,0 @@
Add documentation to the `password_providers` config option. Add known password provider implementations to docs.

View File

@@ -1 +0,0 @@
Move catchup of replication streams logic to worker.

View File

@@ -1 +0,0 @@
Do not allow a deactivated user to login via SSO.

View File

@@ -1 +0,0 @@
Convert some of synapse.rest.media to async/await.

View File

@@ -1 +0,0 @@
Correct the parameters of a test fixture. Contributed by Isaiah Singletary.

View File

@@ -1 +0,0 @@
Add documentation to the `password_providers` config option. Add known password provider implementations to docs.

View File

@@ -1 +0,0 @@
Fix --help command-line argument.

View File

@@ -1 +0,0 @@
Modify suggested nginx reverse proxy configuration to match Synapse's default file upload size. Contributed by @ProCycleDev.

1
changelog.d/7256.feature Normal file
View File

@@ -0,0 +1 @@
Add OpenID Connect login/registration support. Contributed by Quentin Gliech, on behalf of [les Connecteurs](https://connecteu.rs).

View File

@@ -1 +0,0 @@
Do not allow a deactivated user to login via SSO.

View File

@@ -1 +0,0 @@
Fix room publish permissions not being checked on room creation.

View File

@@ -1 +0,0 @@
Convert auth handler to async/await.

View File

@@ -1 +0,0 @@
Add a config option for specifying the value of the Accept-Language HTTP header when generating URL previews.

View File

@@ -1 +0,0 @@
Reject unknown session IDs during user interactive authentication instead of silently creating a new session.

View File

@@ -1 +0,0 @@
Documentation of media_storage_providers options updated to avoid misunderstandings. Contributed by Tristan Lins.

View File

@@ -1 +0,0 @@
Fix a sql query introduced in Synapse 1.12.0 which could cause large amounts of logging to the postgres slow-query log.

View File

@@ -1 +0,0 @@
Add some unit tests for replication.

View File

@@ -1 +0,0 @@
Support SSO in the user interactive authentication workflow.

View File

@@ -1 +0,0 @@
Move catchup of replication streams logic to worker.

View File

@@ -1 +0,0 @@
Move catchup of replication streams logic to worker.

View File

@@ -1 +0,0 @@
Improve typing annotations in `synapse.replication.tcp.streams.Stream`.

Some files were not shown because too many files have changed in this diff Show More