Merge branch 'develop' into madlittlemods/11850-migrate-to-opentelemetry
Conflicts: synapse/handlers/message.py synapse/logging/opentracing.py
This commit is contained in:
Eric Eastwood
@@ -0,0 +1,17 @@
|
||||
version: 2
|
||||
updates:
|
||||
- # "pip" is the correct setting for poetry, per https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#package-ecosystem
|
||||
package-ecosystem: "pip"
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
|
||||
- package-ecosystem: "docker"
|
||||
directory: "/docker"
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
|
||||
- package-ecosystem: "github-actions"
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
+35
-27
@@ -1,17 +1,25 @@
|
||||
Synapse 1.68.0rc2 (2022-09-23)
|
||||
==============================
|
||||
Synapse 1.68.0 (2022-09-27)
|
||||
===========================
|
||||
|
||||
Please note that Synapse will now refuse to start if configured to use a version of SQLite earlier than 3.27.
|
||||
Please note that Synapse will now refuse to start if configured to use a version of SQLite older than 3.27.
|
||||
|
||||
In addition, please note that installing Synapse from a source checkout now requires a recent Rust compiler.
|
||||
Those using packages will not be affected. On most platforms, installing with `pip install matrix-synapse` will not be affected.
|
||||
See the [upgrade notes](https://matrix-org.github.io/synapse/v1.68/upgrade.html#upgrading-to-v1670).
|
||||
|
||||
See the [upgrade notes](https://matrix-org.github.io/synapse/v1.68/upgrade.html#upgrading-to-v1680).
|
||||
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Fix building from packaged sdist. Broke in v1.68.0rc1. ([\#13866](https://github.com/matrix-org/synapse/issues/13866))
|
||||
- Fix packaging to include `Cargo.lock` in `sdist`. ([\#13909](https://github.com/matrix-org/synapse/issues/13909))
|
||||
|
||||
|
||||
Synapse 1.68.0rc2 (2022-09-23)
|
||||
==============================
|
||||
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Fix building from packaged sdist. Broken in v1.68.0rc1. ([\#13866](https://github.com/matrix-org/synapse/issues/13866))
|
||||
|
||||
|
||||
Internal Changes
|
||||
@@ -19,7 +27,7 @@ Internal Changes
|
||||
|
||||
- Fix the release script not publishing binary wheels. ([\#13850](https://github.com/matrix-org/synapse/issues/13850))
|
||||
- Lower minimum supported rustc version to 1.58.1. ([\#13857](https://github.com/matrix-org/synapse/issues/13857))
|
||||
- Lock Rust dependencies versions. ([\#13858](https://github.com/matrix-org/synapse/issues/13858))
|
||||
- Lock Rust dependencies' versions. ([\#13858](https://github.com/matrix-org/synapse/issues/13858))
|
||||
|
||||
|
||||
Synapse 1.68.0rc1 (2022-09-20)
|
||||
@@ -40,7 +48,7 @@ Features
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Fix a bug introduced in Synapse v1.41.0 where the `/hierarchy` API returned non-standard information (a `room_id` field under each entry in `children_state`). ([\#13506](https://github.com/matrix-org/synapse/issues/13506))
|
||||
- Fix a bug introduced in Synapse 1.41.0 where the `/hierarchy` API returned non-standard information (a `room_id` field under each entry in `children_state`). ([\#13506](https://github.com/matrix-org/synapse/issues/13506))
|
||||
- Fix a long-standing bug where previously rejected events could end up in room state because they pass auth checks given the current state of the room. ([\#13723](https://github.com/matrix-org/synapse/issues/13723))
|
||||
- Fix a long-standing bug where Synapse fails to start if a signing key file contains an empty line. ([\#13738](https://github.com/matrix-org/synapse/issues/13738))
|
||||
- Fix a long-standing bug where Synapse would fail to handle malformed user IDs or room aliases gracefully in certain cases. ([\#13746](https://github.com/matrix-org/synapse/issues/13746))
|
||||
@@ -54,10 +62,10 @@ Improved Documentation
|
||||
----------------------
|
||||
|
||||
- Note that `libpq` is required on ARM-based Macs. ([\#13480](https://github.com/matrix-org/synapse/issues/13480))
|
||||
- Fix a mistake in the config manual: the `event_cache_size` _is_ scaled by `caches.global_factor`. The documentation was incorrect since Synapse v1.22.0. ([\#13726](https://github.com/matrix-org/synapse/issues/13726))
|
||||
- Fix a mistake in the config manual introduced in Synapse 1.22.0: the `event_cache_size` _is_ scaled by `caches.global_factor`. ([\#13726](https://github.com/matrix-org/synapse/issues/13726))
|
||||
- Fix a typo in the documentation for the login ratelimiting configuration. ([\#13727](https://github.com/matrix-org/synapse/issues/13727))
|
||||
- Define Synapse's compatability policy for SQLite versions. ([\#13728](https://github.com/matrix-org/synapse/issues/13728))
|
||||
- Add docs for common fix of deleting the `matrix_synapse.egg-info/` directory for fixing Python dependency problems. ([\#13785](https://github.com/matrix-org/synapse/issues/13785))
|
||||
- Add docs for the common fix of deleting the `matrix_synapse.egg-info/` directory for fixing Python dependency problems. ([\#13785](https://github.com/matrix-org/synapse/issues/13785))
|
||||
- Update request log format documentation to mention the format used when the authenticated user is controlling another user. ([\#13794](https://github.com/matrix-org/synapse/issues/13794))
|
||||
|
||||
|
||||
@@ -98,7 +106,7 @@ Internal Changes
|
||||
- Fix a memory leak when running the unit tests. ([\#13798](https://github.com/matrix-org/synapse/issues/13798))
|
||||
- Use partial indices on SQLite. ([\#13802](https://github.com/matrix-org/synapse/issues/13802))
|
||||
- Check that portdb generates the same postgres schema as that in the source tree. ([\#13808](https://github.com/matrix-org/synapse/issues/13808))
|
||||
- Fix Docker build when Rust .so has been build locally first. ([\#13811](https://github.com/matrix-org/synapse/issues/13811))
|
||||
- Fix Docker build when Rust .so has been built locally first. ([\#13811](https://github.com/matrix-org/synapse/issues/13811))
|
||||
- Complement: Initialise the Postgres database directly inside the target image instead of the base Postgres image to fix building using Buildah. ([\#13819](https://github.com/matrix-org/synapse/issues/13819))
|
||||
- Support providing an index predicate clause when doing upserts. ([\#13822](https://github.com/matrix-org/synapse/issues/13822))
|
||||
- Minor speedups to linting in CI. ([\#13827](https://github.com/matrix-org/synapse/issues/13827))
|
||||
@@ -152,7 +160,7 @@ Bugfixes
|
||||
- Fix [MSC3030](https://github.com/matrix-org/matrix-spec-proposals/pull/3030) `/timestamp_to_event` endpoint to return the correct next event when the events have the same timestamp. ([\#13658](https://github.com/matrix-org/synapse/issues/13658))
|
||||
- Fix bug where we wedge media plugins if clients disconnect early. Introduced in v1.22.0. ([\#13660](https://github.com/matrix-org/synapse/issues/13660))
|
||||
- Fix a long-standing bug which meant that keys for unwhitelisted servers were not returned by `/_matrix/key/v2/query`. ([\#13683](https://github.com/matrix-org/synapse/issues/13683))
|
||||
- Fix a bug introduced in Synapse v1.20.0 that would cause the unstable unread counts from [MSC2654](https://github.com/matrix-org/matrix-spec-proposals/pull/2654) to be calculated even if the feature is disabled. ([\#13694](https://github.com/matrix-org/synapse/issues/13694))
|
||||
- Fix a bug introduced in Synapse 1.20.0 that would cause the unstable unread counts from [MSC2654](https://github.com/matrix-org/matrix-spec-proposals/pull/2654) to be calculated even if the feature is disabled. ([\#13694](https://github.com/matrix-org/synapse/issues/13694))
|
||||
|
||||
|
||||
Updates to the Docker image
|
||||
@@ -179,7 +187,7 @@ Deprecations and Removals
|
||||
|
||||
- Drop support for calling `/_matrix/client/v3/rooms/{roomId}/invite` without an `id_access_token`, which was not permitted by the spec. Contributed by @Vetchu. ([\#13241](https://github.com/matrix-org/synapse/issues/13241))
|
||||
- Remove redundant `_get_joined_users_from_context` cache. Contributed by Nick @ Beeper (@fizzadar). ([\#13569](https://github.com/matrix-org/synapse/issues/13569))
|
||||
- Remove the ability to use direct TCP replication with workers. Direct TCP replication was deprecated in Synapse v1.18.0. Workers now require using Redis. ([\#13647](https://github.com/matrix-org/synapse/issues/13647))
|
||||
- Remove the ability to use direct TCP replication with workers. Direct TCP replication was deprecated in Synapse 1.18.0. Workers now require using Redis. ([\#13647](https://github.com/matrix-org/synapse/issues/13647))
|
||||
- Remove support for unstable [private read receipts](https://github.com/matrix-org/matrix-spec-proposals/pull/2285). ([\#13653](https://github.com/matrix-org/synapse/issues/13653), [\#13692](https://github.com/matrix-org/synapse/issues/13692))
|
||||
|
||||
|
||||
@@ -223,7 +231,7 @@ was originally planned for Synapse 1.64, but was later deferred until now. See
|
||||
the [upgrade notes](https://matrix-org.github.io/synapse/v1.66/upgrade.html#upgrading-to-v1660) for more details.
|
||||
|
||||
Deployments with multiple workers should note that the direct TCP replication
|
||||
configuration was deprecated in Synapse v1.18.0 and will be removed in Synapse
|
||||
configuration was deprecated in Synapse 1.18.0 and will be removed in Synapse
|
||||
v1.67.0. In particular, the TCP `replication` [listener](https://matrix-org.github.io/synapse/v1.66/usage/configuration/config_documentation.html#listeners)
|
||||
type (not to be confused with the `replication` resource on the `http` listener
|
||||
type) and the `worker_replication_port` config option will be removed .
|
||||
@@ -353,7 +361,7 @@ Bugfixes
|
||||
--------
|
||||
|
||||
- Update the version of the LDAP3 auth provider module included in the `matrixdotorg/synapse` DockerHub images and the Debian packages hosted on packages.matrix.org to 0.2.2. This version fixes a regression in the module. ([\#13470](https://github.com/matrix-org/synapse/issues/13470))
|
||||
- Fix a bug introduced in Synapse v1.41.0 where the `/hierarchy` API returned non-standard information (a `room_id` field under each entry in `children_state`) (this was reverted in v1.65.0rc2, see changelog notes above). ([\#13365](https://github.com/matrix-org/synapse/issues/13365))
|
||||
- Fix a bug introduced in Synapse 1.41.0 where the `/hierarchy` API returned non-standard information (a `room_id` field under each entry in `children_state`) (this was reverted in v1.65.0rc2, see changelog notes above). ([\#13365](https://github.com/matrix-org/synapse/issues/13365))
|
||||
- Fix a bug introduced in Synapse 0.24.0 that would respond with the wrong error status code to `/joined_members` requests when the requester is not a current member of the room. Contributed by @andrewdoh. ([\#13374](https://github.com/matrix-org/synapse/issues/13374))
|
||||
- Fix bug in handling of typing events for appservices. Contributed by Nick @ Beeper (@fizzadar). ([\#13392](https://github.com/matrix-org/synapse/issues/13392))
|
||||
- Fix a bug introduced in Synapse 1.57.0 where rooms listed in `exclude_rooms_from_sync` in the configuration file would not be properly excluded from incremental syncs. ([\#13408](https://github.com/matrix-org/synapse/issues/13408))
|
||||
@@ -418,7 +426,7 @@ No significant changes since 1.64.0rc2.
|
||||
Deprecation Warning
|
||||
-------------------
|
||||
|
||||
Synapse v1.66.0 will remove the ability to delegate the tasks of verifying email address ownership, and password reset confirmation, to an identity server.
|
||||
Synapse 1.66.0 will remove the ability to delegate the tasks of verifying email address ownership, and password reset confirmation, to an identity server.
|
||||
|
||||
If you require your homeserver to verify e-mail addresses or to support password resets via e-mail, please configure your homeserver with SMTP access so that it can send e-mails on its own behalf.
|
||||
[Consult the configuration documentation for more information.](https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html#email)
|
||||
@@ -427,7 +435,7 @@ If you require your homeserver to verify e-mail addresses or to support password
|
||||
Synapse 1.64.0rc2 (2022-07-29)
|
||||
==============================
|
||||
|
||||
This RC reintroduces support for `account_threepid_delegates.email`, which was removed in 1.64.0rc1. It remains deprecated and will be removed altogether in Synapse v1.66.0. ([\#13406](https://github.com/matrix-org/synapse/issues/13406))
|
||||
This RC reintroduces support for `account_threepid_delegates.email`, which was removed in 1.64.0rc1. It remains deprecated and will be removed altogether in Synapse 1.66.0. ([\#13406](https://github.com/matrix-org/synapse/issues/13406))
|
||||
|
||||
|
||||
Synapse 1.64.0rc1 (2022-07-26)
|
||||
@@ -676,7 +684,7 @@ Bugfixes
|
||||
- Fix a bug introduced in Synapse 1.58 where Synapse would not report full version information when installed from a git checkout. This is a best-effort affair and not guaranteed to be stable. ([\#12973](https://github.com/matrix-org/synapse/issues/12973))
|
||||
- Fix a bug introduced in Synapse 1.60 where Synapse would fail to start if the `sqlite3` module was not available. ([\#12979](https://github.com/matrix-org/synapse/issues/12979))
|
||||
- Fix a bug where non-standard information was required when requesting the `/hierarchy` API over federation. Introduced
|
||||
in Synapse v1.41.0. ([\#12991](https://github.com/matrix-org/synapse/issues/12991))
|
||||
in Synapse 1.41.0. ([\#12991](https://github.com/matrix-org/synapse/issues/12991))
|
||||
- Fix a long-standing bug which meant that rate limiting was not restrictive enough in some cases. ([\#13018](https://github.com/matrix-org/synapse/issues/13018))
|
||||
- Fix a bug introduced in Synapse 1.58 where profile requests for a malformed user ID would ccause an internal error. Synapse now returns 400 Bad Request in this situation. ([\#13041](https://github.com/matrix-org/synapse/issues/13041))
|
||||
- Fix some inconsistencies in the event authentication code. ([\#13087](https://github.com/matrix-org/synapse/issues/13087), [\#13088](https://github.com/matrix-org/synapse/issues/13088))
|
||||
@@ -1269,7 +1277,7 @@ If you have already upgraded to Synapse 1.57.0 without problem, then you have no
|
||||
Updates to the Docker image
|
||||
---------------------------
|
||||
|
||||
- Include version 0.2.0 of the Synapse LDAP Auth Provider module in the Docker image. This matches the version that was present in the Docker image for Synapse v1.56.0. ([\#12512](https://github.com/matrix-org/synapse/issues/12512))
|
||||
- Include version 0.2.0 of the Synapse LDAP Auth Provider module in the Docker image. This matches the version that was present in the Docker image for Synapse 1.56.0. ([\#12512](https://github.com/matrix-org/synapse/issues/12512))
|
||||
|
||||
|
||||
Synapse 1.57.0 (2022-04-19)
|
||||
@@ -1521,10 +1529,10 @@ Features
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Use the proper serialization format for bundled thread aggregations. The bug has existed since Synapse v1.48.0. ([\#12090](https://github.com/matrix-org/synapse/issues/12090))
|
||||
- Use the proper serialization format for bundled thread aggregations. The bug has existed since Synapse 1.48.0. ([\#12090](https://github.com/matrix-org/synapse/issues/12090))
|
||||
- Fix a long-standing bug when redacting events with relations. ([\#12113](https://github.com/matrix-org/synapse/issues/12113), [\#12121](https://github.com/matrix-org/synapse/issues/12121), [\#12130](https://github.com/matrix-org/synapse/issues/12130), [\#12189](https://github.com/matrix-org/synapse/issues/12189))
|
||||
- Fix a bug introduced in Synapse 1.7.2 whereby background updates are never run with the default background batch size. ([\#12157](https://github.com/matrix-org/synapse/issues/12157))
|
||||
- Fix a bug where non-standard information was returned from the `/hierarchy` API. Introduced in Synapse v1.41.0. ([\#12175](https://github.com/matrix-org/synapse/issues/12175))
|
||||
- Fix a bug where non-standard information was returned from the `/hierarchy` API. Introduced in Synapse 1.41.0. ([\#12175](https://github.com/matrix-org/synapse/issues/12175))
|
||||
- Fix a bug introduced in Synapse 1.54.0 that broke background updates on sqlite homeservers while search was disabled. ([\#12215](https://github.com/matrix-org/synapse/issues/12215))
|
||||
- Fix a long-standing bug when a `filter` argument with `event_fields` which did not include the `unsigned` field could result in a 500 error on `/sync`. ([\#12234](https://github.com/matrix-org/synapse/issues/12234))
|
||||
|
||||
@@ -1909,15 +1917,15 @@ Bugfixes
|
||||
- Fix a long-standing issue which could cause Synapse to incorrectly accept data in the unsigned field of events
|
||||
received over federation. ([\#11530](https://github.com/matrix-org/synapse/issues/11530))
|
||||
- Fix a long-standing bug where Synapse wouldn't cache a response indicating that a remote user has no devices. ([\#11587](https://github.com/matrix-org/synapse/issues/11587))
|
||||
- Fix an error that occurs whilst trying to get the federation status of a destination server that was working normally. This admin API was newly introduced in Synapse v1.49.0. ([\#11593](https://github.com/matrix-org/synapse/issues/11593))
|
||||
- Fix an error that occurs whilst trying to get the federation status of a destination server that was working normally. This admin API was newly introduced in Synapse 1.49.0. ([\#11593](https://github.com/matrix-org/synapse/issues/11593))
|
||||
- Fix bundled aggregations not being included in the `/sync` response, per [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675). ([\#11612](https://github.com/matrix-org/synapse/issues/11612), [\#11659](https://github.com/matrix-org/synapse/issues/11659), [\#11791](https://github.com/matrix-org/synapse/issues/11791))
|
||||
- Fix the `/_matrix/client/v1/room/{roomId}/hierarchy` endpoint returning incorrect fields which have been present since Synapse 1.49.0. ([\#11667](https://github.com/matrix-org/synapse/issues/11667))
|
||||
- Fix preview of some GIF URLs (like tenor.com). Contributed by Philippe Daouadi. ([\#11669](https://github.com/matrix-org/synapse/issues/11669))
|
||||
- Fix a bug where only the first 50 rooms from a space were returned from the `/hierarchy` API. This has existed since the introduction of the API in Synapse v1.41.0. ([\#11695](https://github.com/matrix-org/synapse/issues/11695))
|
||||
- Fix a bug introduced in Synapse v1.18.0 where password reset and address validation emails would not be sent if their subject was configured to use the 'app' template variable. Contributed by @br4nnigan. ([\#11710](https://github.com/matrix-org/synapse/issues/11710), [\#11745](https://github.com/matrix-org/synapse/issues/11745))
|
||||
- Fix a bug where only the first 50 rooms from a space were returned from the `/hierarchy` API. This has existed since the introduction of the API in Synapse 1.41.0. ([\#11695](https://github.com/matrix-org/synapse/issues/11695))
|
||||
- Fix a bug introduced in Synapse 1.18.0 where password reset and address validation emails would not be sent if their subject was configured to use the 'app' template variable. Contributed by @br4nnigan. ([\#11710](https://github.com/matrix-org/synapse/issues/11710), [\#11745](https://github.com/matrix-org/synapse/issues/11745))
|
||||
- Make the 'List Rooms' Admin API sort stable. Contributed by Daniël Sonck. ([\#11737](https://github.com/matrix-org/synapse/issues/11737))
|
||||
- Fix a long-standing bug where space hierarchy over federation would only work correctly some of the time. ([\#11775](https://github.com/matrix-org/synapse/issues/11775))
|
||||
- Fix a bug introduced in Synapse v1.46.0 that prevented `on_logged_out` module callbacks from being correctly awaited by Synapse. ([\#11786](https://github.com/matrix-org/synapse/issues/11786))
|
||||
- Fix a bug introduced in Synapse 1.46.0 that prevented `on_logged_out` module callbacks from being correctly awaited by Synapse. ([\#11786](https://github.com/matrix-org/synapse/issues/11786))
|
||||
|
||||
|
||||
Improved Documentation
|
||||
@@ -1997,8 +2005,8 @@ This release candidate fixes a federation-breaking regression introduced in Syna
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Fix a bug introduced in Synapse v1.0.0 whereby some device list updates would not be sent to remote homeservers if there were too many to send at once. ([\#11729](https://github.com/matrix-org/synapse/issues/11729))
|
||||
- Fix a bug introduced in Synapse v1.50.0rc1 whereby outbound federation could fail because too many EDUs were produced for device updates. ([\#11730](https://github.com/matrix-org/synapse/issues/11730))
|
||||
- Fix a bug introduced in Synapse 1.0.0 whereby some device list updates would not be sent to remote homeservers if there were too many to send at once. ([\#11729](https://github.com/matrix-org/synapse/issues/11729))
|
||||
- Fix a bug introduced in Synapse 1.50.0rc1 whereby outbound federation could fail because too many EDUs were produced for device updates. ([\#11730](https://github.com/matrix-org/synapse/issues/11730))
|
||||
|
||||
|
||||
Improved Documentation
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
Speed up creation of DM rooms.
|
||||
@@ -0,0 +1 @@
|
||||
Allow server admins to require a manual approval process before new accounts can be used (using [MSC3866](https://github.com/matrix-org/matrix-spec-proposals/pull/3866)).
|
||||
@@ -0,0 +1 @@
|
||||
Send invite push notifications for invite over federation.
|
||||
@@ -0,0 +1 @@
|
||||
Optimise get rooms for user calls. Contributed by Nick @ Beeper (@fizzadar).
|
||||
@@ -0,0 +1 @@
|
||||
Speed up creation of DM rooms.
|
||||
@@ -0,0 +1 @@
|
||||
Port push rules to using Rust.
|
||||
@@ -0,0 +1 @@
|
||||
Carry IdP Session IDs through user-mapping sessions.
|
||||
@@ -0,0 +1 @@
|
||||
Fix `have_seen_event` cache not being invalidated after we persist an event which causes inefficiency effects like extra `/state` federation calls.
|
||||
@@ -0,0 +1 @@
|
||||
Correct the comments in the complement dockerfile.
|
||||
@@ -0,0 +1 @@
|
||||
Fix unstable MSC3882 endpoint being incorrectly available on stable API versions.
|
||||
@@ -0,0 +1 @@
|
||||
Faster room joins: Fix a bug introduced in 1.66.0 where an error would be logged when syncing after joining a room.
|
||||
@@ -0,0 +1 @@
|
||||
Only pull relevant backfill points from the database based on the current depth and limit (instead of all) every time we want to `/backfill`.
|
||||
@@ -0,0 +1 @@
|
||||
Correctly handle a race with device lists when a remote user leaves during a partial join.
|
||||
@@ -0,0 +1 @@
|
||||
Improve backfill robustness by trying more servers when we get a `4xx` error back.
|
||||
@@ -0,0 +1 @@
|
||||
Faster remote room joins: record _when_ we first partial-join to a room.
|
||||
@@ -0,0 +1 @@
|
||||
Fix a bug introduced in 1.66 where some required fields in the pushrules sent to clients were not present anymore. Contributed by Nico.
|
||||
@@ -0,0 +1 @@
|
||||
Faster remote room joins: correctly handle remote device list updates during a partial join.
|
||||
@@ -0,0 +1 @@
|
||||
Complement image: propagate SIGTERM to all workers.
|
||||
@@ -0,0 +1 @@
|
||||
Emphasize the right reasons when to use `(room_id, event_id)` in a database schema.
|
||||
@@ -0,0 +1 @@
|
||||
Support a `dir` parameter on the `/relations` endpoint per [MSC3715](https://github.com/matrix-org/matrix-doc/pull/3715).
|
||||
@@ -0,0 +1 @@
|
||||
Fix long-standing bug where device updates could cause delays sending out to-device messages over federation.
|
||||
@@ -0,0 +1 @@
|
||||
Update an innaccurate comment in Synapse's upsert database helper.
|
||||
@@ -0,0 +1 @@
|
||||
Update mypy (0.950 -> 0.981) and mypy-zope (0.3.7 -> 0.3.11).
|
||||
@@ -0,0 +1 @@
|
||||
Add instruction to contributing guide for running unit tests in parallel. Contributed by @ashfame.
|
||||
@@ -0,0 +1 @@
|
||||
Update the man page for the `hash_password` script to correct the default number of bcrypt rounds performed.
|
||||
@@ -0,0 +1 @@
|
||||
Clarify that the `auto_join_rooms` config option can also be used with Space aliases.
|
||||
@@ -0,0 +1 @@
|
||||
Experimental support for thread-specific receipts ([MSC3771](https://github.com/matrix-org/matrix-spec-proposals/pull/3771)).
|
||||
@@ -0,0 +1 @@
|
||||
Correctly handle sending local device list updates to remote servers during a partial join.
|
||||
@@ -0,0 +1 @@
|
||||
Exponentially backoff from backfilling the same event over and over.
|
||||
@@ -0,0 +1 @@
|
||||
Experimental support for thread-specific receipts ([MSC3771](https://github.com/matrix-org/matrix-spec-proposals/pull/3771)).
|
||||
@@ -0,0 +1 @@
|
||||
Experimental support for thread-specific receipts ([MSC3771](https://github.com/matrix-org/matrix-spec-proposals/pull/3771)).
|
||||
@@ -0,0 +1 @@
|
||||
Add cache invalidation across workers to module API.
|
||||
@@ -0,0 +1 @@
|
||||
Fix a bug introduced in v1.68.0 where Synapse would require `setuptools_rust` at runtime, even though the package is only required at build time.
|
||||
@@ -0,0 +1 @@
|
||||
Ask mail servers receiving emails from Synapse to not send automatic reply (e.g. out-of-office responses).
|
||||
@@ -0,0 +1 @@
|
||||
Refactor language in user directory `_track_user_joined_room` code to make it more clear that we use both local and remote users.
|
||||
@@ -0,0 +1 @@
|
||||
Fix a performance regression in the `get_users_in_room` database query. Introduced in v1.67.0.
|
||||
@@ -0,0 +1 @@
|
||||
Speed up calculating push actions in large rooms.
|
||||
@@ -0,0 +1 @@
|
||||
Add some cross references to worker documentation.
|
||||
@@ -0,0 +1 @@
|
||||
Enable update notifications from Github's dependabot.
|
||||
@@ -0,0 +1 @@
|
||||
Speed up calculating push actions in large rooms.
|
||||
@@ -0,0 +1 @@
|
||||
Update mypy (0.950 -> 0.981) and mypy-zope (0.3.7 -> 0.3.11).
|
||||
Vendored
+6
@@ -5,6 +5,12 @@ matrix-synapse-py3 (1.69.0~rc1+nmu1) UNRELEASED; urgency=medium
|
||||
|
||||
-- Synapse Packaging team <packages@matrix.org> Mon, 26 Sep 2022 18:05:09 +0100
|
||||
|
||||
matrix-synapse-py3 (1.68.0) stable; urgency=medium
|
||||
|
||||
* New Synapse release 1.68.0.
|
||||
|
||||
-- Synapse Packaging team <packages@matrix.org> Tue, 27 Sep 2022 12:02:09 +0100
|
||||
|
||||
matrix-synapse-py3 (1.68.0~rc2) stable; urgency=medium
|
||||
|
||||
* New Synapse release 1.68.0rc2.
|
||||
|
||||
Vendored
+1
-1
@@ -10,7 +10,7 @@
|
||||
.P
|
||||
\fBhash_password\fR takes a password as an parameter either on the command line or the \fBSTDIN\fR if not supplied\.
|
||||
.P
|
||||
It accepts an YAML file which can be used to specify parameters like the number of rounds for bcrypt and password_config section having the pepper value used for the hashing\. By default \fBbcrypt_rounds\fR is set to \fB10\fR\.
|
||||
It accepts an YAML file which can be used to specify parameters like the number of rounds for bcrypt and password_config section having the pepper value used for the hashing\. By default \fBbcrypt_rounds\fR is set to \fB12\fR\.
|
||||
.P
|
||||
The hashed password is written on the \fBSTDOUT\fR\.
|
||||
.SH "FILES"
|
||||
|
||||
@@ -8,19 +8,15 @@
|
||||
|
||||
ARG SYNAPSE_VERSION=latest
|
||||
|
||||
# first of all, we create a base image with a postgres server and database,
|
||||
# which we can copy into the target image. For repeated rebuilds, this is
|
||||
# much faster than apt installing postgres each time.
|
||||
#
|
||||
# This trick only works because (a) the Synapse image happens to have all the
|
||||
# shared libraries that postgres wants, (b) we use a postgres image based on
|
||||
# the same debian version as Synapse's docker image (so the versions of the
|
||||
# shared libraries match).
|
||||
|
||||
# now build the final image, based on the Synapse image.
|
||||
|
||||
FROM matrixdotorg/synapse-workers:$SYNAPSE_VERSION
|
||||
# copy the postgres installation over from the image we built above
|
||||
# First of all, we copy postgres server from the official postgres image,
|
||||
# since for repeated rebuilds, this is much faster than apt installing
|
||||
# postgres each time.
|
||||
|
||||
# This trick only works because (a) the Synapse image happens to have all the
|
||||
# shared libraries that postgres wants, (b) we use a postgres image based on
|
||||
# the same debian version as Synapse's docker image (so the versions of the
|
||||
# shared libraries match).
|
||||
RUN adduser --system --uid 999 postgres --home /var/lib/postgresql
|
||||
COPY --from=postgres:13-bullseye /usr/lib/postgresql /usr/lib/postgresql
|
||||
COPY --from=postgres:13-bullseye /usr/share/postgresql /usr/share/postgresql
|
||||
@@ -28,7 +24,7 @@ FROM matrixdotorg/synapse-workers:$SYNAPSE_VERSION
|
||||
ENV PATH="${PATH}:/usr/lib/postgresql/13/bin"
|
||||
ENV PGDATA=/var/lib/postgresql/data
|
||||
|
||||
# initialise the database cluster in /var/lib/postgresql
|
||||
# We also initialize the database at build time, rather than runtime, so that it's faster to spin up the image.
|
||||
RUN gosu postgres initdb --locale=C --encoding=UTF-8 --auth-host password
|
||||
|
||||
# Configure a password and create a database for Synapse
|
||||
|
||||
@@ -167,6 +167,12 @@ was broken. They are slower than the linters but will typically catch more error
|
||||
poetry run trial tests
|
||||
```
|
||||
|
||||
You can run unit tests in parallel by specifying `-jX` argument to `trial` where `X` is the number of parallel runners you want. To use 4 cpu cores, you would run them like:
|
||||
|
||||
```sh
|
||||
poetry run trial -j4 tests
|
||||
```
|
||||
|
||||
If you wish to only run *some* unit tests, you may specify
|
||||
another module instead of `tests` - or a test class or a method:
|
||||
|
||||
|
||||
@@ -195,23 +195,24 @@ There are three separate aspects to this:
|
||||
|
||||
## `event_id` global uniqueness
|
||||
|
||||
In room versions `1` and `2` it's possible to end up with two events with the
|
||||
same `event_id` (in the same or different rooms). After room version `3`, that
|
||||
can only happen with a hash collision, which we basically hope will never
|
||||
happen.
|
||||
|
||||
There are several places in Synapse and even Matrix APIs like [`GET
|
||||
`event_id`'s can be considered globally unique although there has been a lot of
|
||||
debate on this topic in places like
|
||||
[MSC2779](https://github.com/matrix-org/matrix-spec-proposals/issues/2779) and
|
||||
[MSC2848](https://github.com/matrix-org/matrix-spec-proposals/pull/2848) which
|
||||
has no resolution yet (as of 2022-09-01). There are several places in Synapse
|
||||
and even in the Matrix APIs like [`GET
|
||||
/_matrix/federation/v1/event/{eventId}`](https://spec.matrix.org/v1.1/server-server-api/#get_matrixfederationv1eventeventid)
|
||||
where we assume that event IDs are globally unique.
|
||||
|
||||
But hash collisions are still possible, and by treating event IDs as room
|
||||
scoped, we can reduce the possibility of a hash collision. When scoping
|
||||
`event_id` in the database schema, it should be also accompanied by `room_id`
|
||||
(`PRIMARY KEY (room_id, event_id)`) and lookups should be done through the pair
|
||||
`(room_id, event_id)`.
|
||||
When scoping `event_id` in a database schema, it is often nice to accompany it
|
||||
with `room_id` (`PRIMARY KEY (room_id, event_id)` and a `FOREIGN KEY(room_id)
|
||||
REFERENCES rooms(room_id)`) which makes flexible lookups easy. For example it
|
||||
makes it very easy to find and clean up everything in a room when it needs to be
|
||||
purged (no need to use sub-`select` query or join from the `events` table).
|
||||
|
||||
A note on collisions: In room versions `1` and `2` it's possible to end up with
|
||||
two events with the same `event_id` (in the same or different rooms). After room
|
||||
version `3`, that can only happen with a hash collision, which we basically hope
|
||||
will never happen (SHA256 has a massive big key space).
|
||||
|
||||
There has been a lot of debate on this in places like
|
||||
https://github.com/matrix-org/matrix-spec-proposals/issues/2779 and
|
||||
[MSC2848](https://github.com/matrix-org/matrix-spec-proposals/pull/2848) which
|
||||
has no resolution yet (as of 2022-09-01).
|
||||
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
worker_app: synapse.app.media_repository
|
||||
worker_name: media_worker
|
||||
|
||||
# The replication listener on the main synapse process.
|
||||
worker_replication_host: 127.0.0.1
|
||||
worker_replication_http_port: 9093
|
||||
|
||||
worker_listeners:
|
||||
- type: http
|
||||
port: 8085
|
||||
resources:
|
||||
- names: [media]
|
||||
|
||||
worker_log_config: /etc/matrix-synapse/media-worker-log.yaml
|
||||
+43
-8
@@ -15,9 +15,8 @@ this document.
|
||||
The website <https://endoflife.date> also offers convenient
|
||||
summaries.
|
||||
|
||||
- If Synapse was installed using [prebuilt
|
||||
packages](setup/installation.md#prebuilt-packages), you will need to follow the
|
||||
normal process for upgrading those packages.
|
||||
- If Synapse was installed using [prebuilt packages](setup/installation.md#prebuilt-packages),
|
||||
you will need to follow the normal process for upgrading those packages.
|
||||
|
||||
- If Synapse was installed using pip then upgrade to the latest
|
||||
version by running:
|
||||
@@ -89,12 +88,48 @@ process, for example:
|
||||
dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
|
||||
```
|
||||
|
||||
# Upgrading to v1.69.0
|
||||
|
||||
## Changes to the receipts replication streams
|
||||
|
||||
Synapse now includes information indicating if a receipt applies to a thread when
|
||||
replicating it to other workers. This is a forwards- and backwards-incompatible
|
||||
change: v1.68 and workers cannot process receipts replicated by v1.69 workers, and
|
||||
vice versa.
|
||||
|
||||
Once all workers are upgraded to v1.69 (or downgraded to v1.68), receipts
|
||||
replication will resume as normal.
|
||||
|
||||
# Upgrading to v1.68.0
|
||||
|
||||
As announced in the upgrade notes for v1.67.0, Synapse now requires a SQLite
|
||||
version of 3.27.0 or higher if SQLite is in use and source checkouts of Synapse
|
||||
now require a recent Rust compiler.
|
||||
Two changes announced in the upgrade notes for v1.67.0 have now landed in v1.68.0.
|
||||
|
||||
## SQLite version requirement
|
||||
|
||||
Synapse now requires a SQLite version of 3.27.0 or higher if SQLite is configured as
|
||||
Synapse's database.
|
||||
|
||||
Installations using
|
||||
|
||||
- Docker images [from `matrixdotorg`](https://hub.docker.com/r/matrixdotorg/synapse),
|
||||
- Debian packages [from Matrix.org](https://packages.matrix.org/), or
|
||||
- a PostgreSQL database
|
||||
|
||||
are not affected.
|
||||
|
||||
## Rust requirement when building from source.
|
||||
|
||||
Building from a source checkout of Synapse now requires a recent Rust compiler
|
||||
(currently Rust 1.58.1, but see also the
|
||||
[Platform Dependency Policy](https://matrix-org.github.io/synapse/latest/deprecation_policy.html)).
|
||||
|
||||
Installations using
|
||||
|
||||
- Docker images [from `matrixdotorg`](https://hub.docker.com/r/matrixdotorg/synapse),
|
||||
- Debian packages [from Matrix.org](https://packages.matrix.org/), or
|
||||
- PyPI wheels via `pip install matrix-synapse` (on supported platforms and architectures)
|
||||
|
||||
will not be affected.
|
||||
|
||||
# Upgrading to v1.67.0
|
||||
|
||||
@@ -128,12 +163,12 @@ The simplest way of installing Rust is via [rustup.rs](https://rustup.rs/)
|
||||
|
||||
## SQLite version requirement in the next release
|
||||
|
||||
From the next major release (v1.68.0) Synapse will require SQLite 3.27.0 or
|
||||
From the next major release (v1.68.0) Synapse will require SQLite 3.27.0 or
|
||||
higher. Synapse v1.67.0 will be the last major release supporting SQLite
|
||||
versions 3.22 to 3.26.
|
||||
|
||||
Those using Docker images or Debian packages from Matrix.org will not be
|
||||
affected. If you have installed from source, you should check the version of
|
||||
affected. If you have installed from source, you should check the version of
|
||||
SQLite used by Python with:
|
||||
|
||||
```shell
|
||||
|
||||
@@ -2230,6 +2230,9 @@ homeserver. If the room already exists, make certain it is a publicly joinable
|
||||
room, i.e. the join rule of the room must be set to 'public'. You can find more options
|
||||
relating to auto-joining rooms below.
|
||||
|
||||
As Spaces are just rooms under the hood, Space aliases may also be
|
||||
used.
|
||||
|
||||
Example configuration:
|
||||
```yaml
|
||||
auto_join_rooms:
|
||||
@@ -2241,7 +2244,7 @@ auto_join_rooms:
|
||||
|
||||
Where `auto_join_rooms` are specified, setting this flag ensures that
|
||||
the rooms exist by creating them when the first user on the
|
||||
homeserver registers.
|
||||
homeserver registers. This option will not create Spaces.
|
||||
|
||||
By default the auto-created rooms are publicly joinable from any federated
|
||||
server. Use the `autocreate_auto_join_rooms_federated` and
|
||||
@@ -2259,7 +2262,7 @@ autocreate_auto_join_rooms: false
|
||||
---
|
||||
### `autocreate_auto_join_rooms_federated`
|
||||
|
||||
Whether the rooms listen in `auto_join_rooms` that are auto-created are available
|
||||
Whether the rooms listed in `auto_join_rooms` that are auto-created are available
|
||||
via federation. Only has an effect if `autocreate_auto_join_rooms` is true.
|
||||
|
||||
Note that whether a room is federated cannot be modified after
|
||||
|
||||
+13
-15
@@ -93,7 +93,6 @@ listener" for the main process; and secondly, you need to enable redis-based
|
||||
replication. Optionally, a shared secret can be used to authenticate HTTP
|
||||
traffic between workers. For example:
|
||||
|
||||
|
||||
```yaml
|
||||
# extend the existing `listeners` section. This defines the ports that the
|
||||
# main process will listen on.
|
||||
@@ -129,7 +128,8 @@ In the config file for each worker, you must specify:
|
||||
* The HTTP replication endpoint that it should talk to on the main synapse process
|
||||
(`worker_replication_host` and `worker_replication_http_port`)
|
||||
* If handling HTTP requests, a `worker_listeners` option with an `http`
|
||||
listener, in the same way as the `listeners` option in the shared config.
|
||||
listener, in the same way as the [`listeners`](usage/configuration/config_documentation.md#listeners)
|
||||
option in the shared config.
|
||||
* If handling the `^/_matrix/client/v3/keys/upload` endpoint, the HTTP URI for
|
||||
the main process (`worker_main_http_uri`).
|
||||
|
||||
@@ -285,8 +285,9 @@ For multiple workers not handling the SSO endpoints properly, see
|
||||
[#7530](https://github.com/matrix-org/synapse/issues/7530) and
|
||||
[#9427](https://github.com/matrix-org/synapse/issues/9427).
|
||||
|
||||
Note that a HTTP listener with `client` and `federation` resources must be
|
||||
configured in the `worker_listeners` option in the worker config.
|
||||
Note that a [HTTP listener](usage/configuration/config_documentation.md#listeners)
|
||||
with `client` and `federation` `resources` must be configured in the `worker_listeners`
|
||||
option in the worker config.
|
||||
|
||||
#### Load balancing
|
||||
|
||||
@@ -326,7 +327,8 @@ effects of bursts of events from that bridge on events sent by normal users.
|
||||
Additionally, the writing of specific streams (such as events) can be moved off
|
||||
of the main process to a particular worker.
|
||||
|
||||
To enable this, the worker must have a HTTP replication listener configured,
|
||||
To enable this, the worker must have a
|
||||
[HTTP `replication` listener](usage/configuration/config_documentation.md#listeners) configured,
|
||||
have a `worker_name` and be listed in the `instance_map` config. The same worker
|
||||
can handle multiple streams, but unless otherwise documented, each stream can only
|
||||
have a single writer.
|
||||
@@ -410,7 +412,7 @@ the stream writer for the `presence` stream:
|
||||
There is also support for moving background tasks to a separate
|
||||
worker. Background tasks are run periodically or started via replication. Exactly
|
||||
which tasks are configured to run depends on your Synapse configuration (e.g. if
|
||||
stats is enabled).
|
||||
stats is enabled). This worker doesn't handle any REST endpoints itself.
|
||||
|
||||
To enable this, the worker must have a `worker_name` and can be configured to run
|
||||
background tasks. For example, to move background tasks to a dedicated worker,
|
||||
@@ -457,8 +459,8 @@ worker application type.
|
||||
#### Notifying Application Services
|
||||
|
||||
You can designate one generic worker to send output traffic to Application Services.
|
||||
|
||||
Specify its name in the shared configuration as follows:
|
||||
Doesn't handle any REST endpoints itself, but you should specify its name in the
|
||||
shared configuration as follows:
|
||||
|
||||
```yaml
|
||||
notify_appservices_from_worker: worker_name
|
||||
@@ -536,16 +538,12 @@ file to stop the main synapse running background jobs related to managing the
|
||||
media repository. Note that doing so will prevent the main process from being
|
||||
able to handle the above endpoints.
|
||||
|
||||
In the `media_repository` worker configuration file, configure the http listener to
|
||||
In the `media_repository` worker configuration file, configure the
|
||||
[HTTP listener](usage/configuration/config_documentation.md#listeners) to
|
||||
expose the `media` resource. For example:
|
||||
|
||||
```yaml
|
||||
worker_listeners:
|
||||
- type: http
|
||||
port: 8085
|
||||
resources:
|
||||
- names:
|
||||
- media
|
||||
{{#include systemd-with-workers/workers/media_worker.yaml}}
|
||||
```
|
||||
|
||||
Note that if running multiple media repositories they must be on the same server
|
||||
|
||||
Generated
+30
-29
@@ -584,11 +584,11 @@ python-versions = "*"
|
||||
|
||||
[[package]]
|
||||
name = "mypy"
|
||||
version = "0.950"
|
||||
version = "0.981"
|
||||
description = "Optional static typing for Python"
|
||||
category = "dev"
|
||||
optional = false
|
||||
python-versions = ">=3.6"
|
||||
python-versions = ">=3.7"
|
||||
|
||||
[package.dependencies]
|
||||
mypy-extensions = ">=0.4.3"
|
||||
@@ -611,14 +611,14 @@ python-versions = "*"
|
||||
|
||||
[[package]]
|
||||
name = "mypy-zope"
|
||||
version = "0.3.7"
|
||||
version = "0.3.11"
|
||||
description = "Plugin for mypy to support zope interfaces"
|
||||
category = "dev"
|
||||
optional = false
|
||||
python-versions = "*"
|
||||
|
||||
[package.dependencies]
|
||||
mypy = "0.950"
|
||||
mypy = "0.981"
|
||||
"zope.interface" = "*"
|
||||
"zope.schema" = "*"
|
||||
|
||||
@@ -2266,37 +2266,38 @@ msgpack = [
|
||||
{file = "msgpack-1.0.3.tar.gz", hash = "sha256:51fdc7fb93615286428ee7758cecc2f374d5ff363bdd884c7ea622a7a327a81e"},
|
||||
]
|
||||
mypy = [
|
||||
{file = "mypy-0.950-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:cf9c261958a769a3bd38c3e133801ebcd284ffb734ea12d01457cb09eacf7d7b"},
|
||||
{file = "mypy-0.950-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:b5b5bd0ffb11b4aba2bb6d31b8643902c48f990cc92fda4e21afac658044f0c0"},
|
||||
{file = "mypy-0.950-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:5e7647df0f8fc947388e6251d728189cfadb3b1e558407f93254e35abc026e22"},
|
||||
{file = "mypy-0.950-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:eaff8156016487c1af5ffa5304c3e3fd183edcb412f3e9c72db349faf3f6e0eb"},
|
||||
{file = "mypy-0.950-cp310-cp310-win_amd64.whl", hash = "sha256:563514c7dc504698fb66bb1cf897657a173a496406f1866afae73ab5b3cdb334"},
|
||||
{file = "mypy-0.950-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:dd4d670eee9610bf61c25c940e9ade2d0ed05eb44227275cce88701fee014b1f"},
|
||||
{file = "mypy-0.950-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:ca75ecf2783395ca3016a5e455cb322ba26b6d33b4b413fcdedfc632e67941dc"},
|
||||
{file = "mypy-0.950-cp36-cp36m-win_amd64.whl", hash = "sha256:6003de687c13196e8a1243a5e4bcce617d79b88f83ee6625437e335d89dfebe2"},
|
||||
{file = "mypy-0.950-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:4c653e4846f287051599ed8f4b3c044b80e540e88feec76b11044ddc5612ffed"},
|
||||
{file = "mypy-0.950-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:e19736af56947addedce4674c0971e5dceef1b5ec7d667fe86bcd2b07f8f9075"},
|
||||
{file = "mypy-0.950-cp37-cp37m-win_amd64.whl", hash = "sha256:ef7beb2a3582eb7a9f37beaf38a28acfd801988cde688760aea9e6cc4832b10b"},
|
||||
{file = "mypy-0.950-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:0112752a6ff07230f9ec2f71b0d3d4e088a910fdce454fdb6553e83ed0eced7d"},
|
||||
{file = "mypy-0.950-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:ee0a36edd332ed2c5208565ae6e3a7afc0eabb53f5327e281f2ef03a6bc7687a"},
|
||||
{file = "mypy-0.950-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:77423570c04aca807508a492037abbd72b12a1fb25a385847d191cd50b2c9605"},
|
||||
{file = "mypy-0.950-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:5ce6a09042b6da16d773d2110e44f169683d8cc8687e79ec6d1181a72cb028d2"},
|
||||
{file = "mypy-0.950-cp38-cp38-win_amd64.whl", hash = "sha256:5b231afd6a6e951381b9ef09a1223b1feabe13625388db48a8690f8daa9b71ff"},
|
||||
{file = "mypy-0.950-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:0384d9f3af49837baa92f559d3fa673e6d2652a16550a9ee07fc08c736f5e6f8"},
|
||||
{file = "mypy-0.950-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:1fdeb0a0f64f2a874a4c1f5271f06e40e1e9779bf55f9567f149466fc7a55038"},
|
||||
{file = "mypy-0.950-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:61504b9a5ae166ba5ecfed9e93357fd51aa693d3d434b582a925338a2ff57fd2"},
|
||||
{file = "mypy-0.950-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:a952b8bc0ae278fc6316e6384f67bb9a396eb30aced6ad034d3a76120ebcc519"},
|
||||
{file = "mypy-0.950-cp39-cp39-win_amd64.whl", hash = "sha256:eaea21d150fb26d7b4856766e7addcf929119dd19fc832b22e71d942835201ef"},
|
||||
{file = "mypy-0.950-py3-none-any.whl", hash = "sha256:a4d9898f46446bfb6405383b57b96737dcfd0a7f25b748e78ef3e8c576bba3cb"},
|
||||
{file = "mypy-0.950.tar.gz", hash = "sha256:1b333cfbca1762ff15808a0ef4f71b5d3eed8528b23ea1c3fb50543c867d68de"},
|
||||
{file = "mypy-0.981-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:4bc460e43b7785f78862dab78674e62ec3cd523485baecfdf81a555ed29ecfa0"},
|
||||
{file = "mypy-0.981-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:756fad8b263b3ba39e4e204ee53042671b660c36c9017412b43af210ddee7b08"},
|
||||
{file = "mypy-0.981-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:a16a0145d6d7d00fbede2da3a3096dcc9ecea091adfa8da48fa6a7b75d35562d"},
|
||||
{file = "mypy-0.981-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ce65f70b14a21fdac84c294cde75e6dbdabbcff22975335e20827b3b94bdbf49"},
|
||||
{file = "mypy-0.981-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:6e35d764784b42c3e256848fb8ed1d4292c9fc0098413adb28d84974c095b279"},
|
||||
{file = "mypy-0.981-cp310-cp310-win_amd64.whl", hash = "sha256:e53773073c864d5f5cec7f3fc72fbbcef65410cde8cc18d4f7242dea60dac52e"},
|
||||
{file = "mypy-0.981-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:6ee196b1d10b8b215e835f438e06965d7a480f6fe016eddbc285f13955cca659"},
|
||||
{file = "mypy-0.981-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8ad21d4c9d3673726cf986ea1d0c9fb66905258709550ddf7944c8f885f208be"},
|
||||
{file = "mypy-0.981-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:d1debb09043e1f5ee845fa1e96d180e89115b30e47c5d3ce53bc967bab53f62d"},
|
||||
{file = "mypy-0.981-cp37-cp37m-win_amd64.whl", hash = "sha256:9f362470a3480165c4c6151786b5379351b790d56952005be18bdbdd4c7ce0ae"},
|
||||
{file = "mypy-0.981-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:c9e0efb95ed6ca1654951bd5ec2f3fa91b295d78bf6527e026529d4aaa1e0c30"},
|
||||
{file = "mypy-0.981-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:e178eaffc3c5cd211a87965c8c0df6da91ed7d258b5fc72b8e047c3771317ddb"},
|
||||
{file = "mypy-0.981-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:06e1eac8d99bd404ed8dd34ca29673c4346e76dd8e612ea507763dccd7e13c7a"},
|
||||
{file = "mypy-0.981-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:fa38f82f53e1e7beb45557ff167c177802ba7b387ad017eab1663d567017c8ee"},
|
||||
{file = "mypy-0.981-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:64e1f6af81c003f85f0dfed52db632817dabb51b65c0318ffbf5ff51995bbb08"},
|
||||
{file = "mypy-0.981-cp38-cp38-win_amd64.whl", hash = "sha256:e1acf62a8c4f7c092462c738aa2c2489e275ed386320c10b2e9bff31f6f7e8d6"},
|
||||
{file = "mypy-0.981-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:b6ede64e52257931315826fdbfc6ea878d89a965580d1a65638ef77cb551f56d"},
|
||||
{file = "mypy-0.981-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:eb3978b191b9fa0488524bb4ffedf2c573340e8c2b4206fc191d44c7093abfb7"},
|
||||
{file = "mypy-0.981-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:77f8fcf7b4b3cc0c74fb33ae54a4cd00bb854d65645c48beccf65fa10b17882c"},
|
||||
{file = "mypy-0.981-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f64d2ce043a209a297df322eb4054dfbaa9de9e8738291706eaafda81ab2b362"},
|
||||
{file = "mypy-0.981-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:2ee3dbc53d4df7e6e3b1c68ac6a971d3a4fb2852bf10a05fda228721dd44fae1"},
|
||||
{file = "mypy-0.981-cp39-cp39-win_amd64.whl", hash = "sha256:8e8e49aa9cc23aa4c926dc200ce32959d3501c4905147a66ce032f05cb5ecb92"},
|
||||
{file = "mypy-0.981-py3-none-any.whl", hash = "sha256:794f385653e2b749387a42afb1e14c2135e18daeb027e0d97162e4b7031210f8"},
|
||||
{file = "mypy-0.981.tar.gz", hash = "sha256:ad77c13037d3402fbeffda07d51e3f228ba078d1c7096a73759c9419ea031bf4"},
|
||||
]
|
||||
mypy-extensions = [
|
||||
{file = "mypy_extensions-0.4.3-py2.py3-none-any.whl", hash = "sha256:090fedd75945a69ae91ce1303b5824f428daf5a028d2f6ab8a299250a846f15d"},
|
||||
{file = "mypy_extensions-0.4.3.tar.gz", hash = "sha256:2d82818f5bb3e369420cb3c4060a7970edba416647068eb4c5343488a6c604a8"},
|
||||
]
|
||||
mypy-zope = [
|
||||
{file = "mypy-zope-0.3.7.tar.gz", hash = "sha256:9da171e78e8ef7ac8922c86af1a62f1b7f3244f121020bd94a2246bc3f33c605"},
|
||||
{file = "mypy_zope-0.3.7-py3-none-any.whl", hash = "sha256:9c7637d066e4d1bafa0651abc091c752009769098043b236446e6725be2bc9c2"},
|
||||
{file = "mypy-zope-0.3.11.tar.gz", hash = "sha256:d4255f9f04d48c79083bbd4e2fea06513a6ac7b8de06f8c4ce563fd85142ca05"},
|
||||
{file = "mypy_zope-0.3.11-py3-none-any.whl", hash = "sha256:ec080a6508d1f7805c8d2054f9fdd13c849742ce96803519e1fdfa3d3cab7140"},
|
||||
]
|
||||
netaddr = [
|
||||
{file = "netaddr-0.8.0-py2.py3-none-any.whl", hash = "sha256:9666d0232c32d2656e5e5f8d735f58fd6c7457ce52fc21c98d45f2af78f990ac"},
|
||||
|
||||
+1
-1
@@ -57,7 +57,7 @@ manifest-path = "rust/Cargo.toml"
|
||||
|
||||
[tool.poetry]
|
||||
name = "matrix-synapse"
|
||||
version = "1.68.0rc2"
|
||||
version = "1.68.0"
|
||||
description = "Homeserver for the Matrix decentralised comms protocol"
|
||||
authors = ["Matrix.org Team and Contributors <packages@matrix.org>"]
|
||||
license = "Apache-2.0"
|
||||
|
||||
+3
-1
@@ -11,7 +11,9 @@ rust-version = "1.58.1"
|
||||
|
||||
[lib]
|
||||
name = "synapse"
|
||||
crate-type = ["cdylib"]
|
||||
# We generate a `cdylib` for Python and a standard `lib` for running
|
||||
# tests/benchmarks.
|
||||
crate-type = ["lib", "cdylib"]
|
||||
|
||||
[package.metadata.maturin]
|
||||
# This is where we tell maturin where to place the built library.
|
||||
|
||||
@@ -0,0 +1,149 @@
|
||||
// Copyright 2022 The Matrix.org Foundation C.I.C.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#![feature(test)]
|
||||
use synapse::push::{
|
||||
evaluator::PushRuleEvaluator, Condition, EventMatchCondition, FilteredPushRules, PushRules,
|
||||
};
|
||||
use test::Bencher;
|
||||
|
||||
extern crate test;
|
||||
|
||||
#[bench]
|
||||
fn bench_match_exact(b: &mut Bencher) {
|
||||
let flattened_keys = [
|
||||
("type".to_string(), "m.text".to_string()),
|
||||
("room_id".to_string(), "!room:server".to_string()),
|
||||
("content.body".to_string(), "test message".to_string()),
|
||||
]
|
||||
.into_iter()
|
||||
.collect();
|
||||
|
||||
let eval = PushRuleEvaluator::py_new(
|
||||
flattened_keys,
|
||||
10,
|
||||
0,
|
||||
Default::default(),
|
||||
Default::default(),
|
||||
true,
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let condition = Condition::Known(synapse::push::KnownCondition::EventMatch(
|
||||
EventMatchCondition {
|
||||
key: "room_id".into(),
|
||||
pattern: Some("!room:server".into()),
|
||||
pattern_type: None,
|
||||
},
|
||||
));
|
||||
|
||||
let matched = eval.match_condition(&condition, None, None).unwrap();
|
||||
assert!(matched, "Didn't match");
|
||||
|
||||
b.iter(|| eval.match_condition(&condition, None, None).unwrap());
|
||||
}
|
||||
|
||||
#[bench]
|
||||
fn bench_match_word(b: &mut Bencher) {
|
||||
let flattened_keys = [
|
||||
("type".to_string(), "m.text".to_string()),
|
||||
("room_id".to_string(), "!room:server".to_string()),
|
||||
("content.body".to_string(), "test message".to_string()),
|
||||
]
|
||||
.into_iter()
|
||||
.collect();
|
||||
|
||||
let eval = PushRuleEvaluator::py_new(
|
||||
flattened_keys,
|
||||
10,
|
||||
0,
|
||||
Default::default(),
|
||||
Default::default(),
|
||||
true,
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let condition = Condition::Known(synapse::push::KnownCondition::EventMatch(
|
||||
EventMatchCondition {
|
||||
key: "content.body".into(),
|
||||
pattern: Some("test".into()),
|
||||
pattern_type: None,
|
||||
},
|
||||
));
|
||||
|
||||
let matched = eval.match_condition(&condition, None, None).unwrap();
|
||||
assert!(matched, "Didn't match");
|
||||
|
||||
b.iter(|| eval.match_condition(&condition, None, None).unwrap());
|
||||
}
|
||||
|
||||
#[bench]
|
||||
fn bench_match_word_miss(b: &mut Bencher) {
|
||||
let flattened_keys = [
|
||||
("type".to_string(), "m.text".to_string()),
|
||||
("room_id".to_string(), "!room:server".to_string()),
|
||||
("content.body".to_string(), "test message".to_string()),
|
||||
]
|
||||
.into_iter()
|
||||
.collect();
|
||||
|
||||
let eval = PushRuleEvaluator::py_new(
|
||||
flattened_keys,
|
||||
10,
|
||||
0,
|
||||
Default::default(),
|
||||
Default::default(),
|
||||
true,
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let condition = Condition::Known(synapse::push::KnownCondition::EventMatch(
|
||||
EventMatchCondition {
|
||||
key: "content.body".into(),
|
||||
pattern: Some("foobar".into()),
|
||||
pattern_type: None,
|
||||
},
|
||||
));
|
||||
|
||||
let matched = eval.match_condition(&condition, None, None).unwrap();
|
||||
assert!(!matched, "Didn't match");
|
||||
|
||||
b.iter(|| eval.match_condition(&condition, None, None).unwrap());
|
||||
}
|
||||
|
||||
#[bench]
|
||||
fn bench_eval_message(b: &mut Bencher) {
|
||||
let flattened_keys = [
|
||||
("type".to_string(), "m.text".to_string()),
|
||||
("room_id".to_string(), "!room:server".to_string()),
|
||||
("content.body".to_string(), "test message".to_string()),
|
||||
]
|
||||
.into_iter()
|
||||
.collect();
|
||||
|
||||
let eval = PushRuleEvaluator::py_new(
|
||||
flattened_keys,
|
||||
10,
|
||||
0,
|
||||
Default::default(),
|
||||
Default::default(),
|
||||
true,
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let rules =
|
||||
FilteredPushRules::py_new(PushRules::new(Vec::new()), Default::default(), false, false);
|
||||
|
||||
b.iter(|| eval.run(&rules, Some("bob"), Some("person")));
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
// Copyright 2022 The Matrix.org Foundation C.I.C.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#![feature(test)]
|
||||
|
||||
use synapse::push::utils::{glob_to_regex, GlobMatchType};
|
||||
use test::Bencher;
|
||||
|
||||
extern crate test;
|
||||
|
||||
#[bench]
|
||||
fn bench_whole(b: &mut Bencher) {
|
||||
b.iter(|| glob_to_regex("test", GlobMatchType::Whole));
|
||||
}
|
||||
|
||||
#[bench]
|
||||
fn bench_word(b: &mut Bencher) {
|
||||
b.iter(|| glob_to_regex("test", GlobMatchType::Word));
|
||||
}
|
||||
|
||||
#[bench]
|
||||
fn bench_whole_wildcard_run(b: &mut Bencher) {
|
||||
b.iter(|| glob_to_regex("test***??*?*?foo", GlobMatchType::Whole));
|
||||
}
|
||||
|
||||
#[bench]
|
||||
fn bench_word_wildcard_run(b: &mut Bencher) {
|
||||
b.iter(|| glob_to_regex("test***??*?*?foo", GlobMatchType::Whole));
|
||||
}
|
||||
+1
-1
@@ -22,7 +22,7 @@ fn main() -> Result<(), std::io::Error> {
|
||||
|
||||
for entry in entries {
|
||||
if entry.is_dir() {
|
||||
dirs.push(entry)
|
||||
dirs.push(entry);
|
||||
} else {
|
||||
paths.push(entry.to_str().expect("valid rust paths").to_string());
|
||||
}
|
||||
|
||||
@@ -262,6 +262,7 @@ pub const BASE_APPEND_UNDERRIDE_RULES: &[PushRule] = &[
|
||||
priority_class: 1,
|
||||
conditions: Cow::Borrowed(&[Condition::Known(KnownCondition::RelationMatch {
|
||||
rel_type: Cow::Borrowed("m.thread"),
|
||||
event_type_pattern: None,
|
||||
sender: None,
|
||||
sender_type: Some(Cow::Borrowed("user_id")),
|
||||
})]),
|
||||
|
||||
@@ -0,0 +1,374 @@
|
||||
// Copyright 2022 The Matrix.org Foundation C.I.C.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
use std::{
|
||||
borrow::Cow,
|
||||
collections::{BTreeMap, BTreeSet},
|
||||
};
|
||||
|
||||
use anyhow::{Context, Error};
|
||||
use lazy_static::lazy_static;
|
||||
use log::warn;
|
||||
use pyo3::prelude::*;
|
||||
use regex::Regex;
|
||||
|
||||
use super::{
|
||||
utils::{get_glob_matcher, get_localpart_from_id, GlobMatchType},
|
||||
Action, Condition, EventMatchCondition, FilteredPushRules, KnownCondition,
|
||||
};
|
||||
|
||||
lazy_static! {
|
||||
/// Used to parse the `is` clause in the room member count condition.
|
||||
static ref INEQUALITY_EXPR: Regex = Regex::new(r"^([=<>]*)([0-9]+)$").expect("valid regex");
|
||||
}
|
||||
|
||||
/// Allows running a set of push rules against a particular event.
|
||||
#[pyclass]
|
||||
pub struct PushRuleEvaluator {
|
||||
/// A mapping of "flattened" keys to string values in the event, e.g.
|
||||
/// includes things like "type" and "content.msgtype".
|
||||
flattened_keys: BTreeMap<String, String>,
|
||||
|
||||
/// The "content.body", if any.
|
||||
body: String,
|
||||
|
||||
/// The number of users in the room.
|
||||
room_member_count: u64,
|
||||
|
||||
/// The `notifications` section of the current power levels in the room.
|
||||
notification_power_levels: BTreeMap<String, i64>,
|
||||
|
||||
/// The relations related to the event as a mapping from relation type to
|
||||
/// set of sender/event type 2-tuples.
|
||||
relations: BTreeMap<String, BTreeSet<(String, String)>>,
|
||||
|
||||
/// Is running "relation" conditions enabled?
|
||||
relation_match_enabled: bool,
|
||||
|
||||
/// The power level of the sender of the event, or None if event is an
|
||||
/// outlier.
|
||||
sender_power_level: Option<i64>,
|
||||
}
|
||||
|
||||
#[pymethods]
|
||||
impl PushRuleEvaluator {
|
||||
/// Create a new `PushRuleEvaluator`. See struct docstring for details.
|
||||
#[new]
|
||||
pub fn py_new(
|
||||
flattened_keys: BTreeMap<String, String>,
|
||||
room_member_count: u64,
|
||||
sender_power_level: Option<i64>,
|
||||
notification_power_levels: BTreeMap<String, i64>,
|
||||
relations: BTreeMap<String, BTreeSet<(String, String)>>,
|
||||
relation_match_enabled: bool,
|
||||
) -> Result<Self, Error> {
|
||||
let body = flattened_keys
|
||||
.get("content.body")
|
||||
.cloned()
|
||||
.unwrap_or_default();
|
||||
|
||||
Ok(PushRuleEvaluator {
|
||||
flattened_keys,
|
||||
body,
|
||||
room_member_count,
|
||||
notification_power_levels,
|
||||
relations,
|
||||
relation_match_enabled,
|
||||
sender_power_level,
|
||||
})
|
||||
}
|
||||
|
||||
/// Run the evaluator with the given push rules, for the given user ID and
|
||||
/// display name of the user.
|
||||
///
|
||||
/// Passing in None will skip evaluating rules matching user ID and display
|
||||
/// name.
|
||||
///
|
||||
/// Returns the set of actions, if any, that match (filtering out any
|
||||
/// `dont_notify` actions).
|
||||
pub fn run(
|
||||
&self,
|
||||
push_rules: &FilteredPushRules,
|
||||
user_id: Option<&str>,
|
||||
display_name: Option<&str>,
|
||||
) -> Vec<Action> {
|
||||
'outer: for (push_rule, enabled) in push_rules.iter() {
|
||||
if !enabled {
|
||||
continue;
|
||||
}
|
||||
|
||||
for condition in push_rule.conditions.iter() {
|
||||
match self.match_condition(condition, user_id, display_name) {
|
||||
Ok(true) => {}
|
||||
Ok(false) => continue 'outer,
|
||||
Err(err) => {
|
||||
warn!("Condition match failed {err}");
|
||||
continue 'outer;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let actions = push_rule
|
||||
.actions
|
||||
.iter()
|
||||
// Filter out "dont_notify" actions, as we don't store them.
|
||||
.filter(|a| **a != Action::DontNotify)
|
||||
.cloned()
|
||||
.collect();
|
||||
|
||||
return actions;
|
||||
}
|
||||
|
||||
Vec::new()
|
||||
}
|
||||
|
||||
/// Check if the given condition matches.
|
||||
fn matches(
|
||||
&self,
|
||||
condition: Condition,
|
||||
user_id: Option<&str>,
|
||||
display_name: Option<&str>,
|
||||
) -> bool {
|
||||
match self.match_condition(&condition, user_id, display_name) {
|
||||
Ok(true) => true,
|
||||
Ok(false) => false,
|
||||
Err(err) => {
|
||||
warn!("Condition match failed {err}");
|
||||
false
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl PushRuleEvaluator {
|
||||
/// Match a given `Condition` for a push rule.
|
||||
pub fn match_condition(
|
||||
&self,
|
||||
condition: &Condition,
|
||||
user_id: Option<&str>,
|
||||
display_name: Option<&str>,
|
||||
) -> Result<bool, Error> {
|
||||
let known_condition = match condition {
|
||||
Condition::Known(known) => known,
|
||||
Condition::Unknown(_) => {
|
||||
return Ok(false);
|
||||
}
|
||||
};
|
||||
|
||||
let result = match known_condition {
|
||||
KnownCondition::EventMatch(event_match) => {
|
||||
self.match_event_match(event_match, user_id)?
|
||||
}
|
||||
KnownCondition::ContainsDisplayName => {
|
||||
if let Some(dn) = display_name {
|
||||
if !dn.is_empty() {
|
||||
get_glob_matcher(dn, GlobMatchType::Word)?.is_match(&self.body)?
|
||||
} else {
|
||||
// We specifically ignore empty display names, as otherwise
|
||||
// they would always match.
|
||||
false
|
||||
}
|
||||
} else {
|
||||
false
|
||||
}
|
||||
}
|
||||
KnownCondition::RoomMemberCount { is } => {
|
||||
if let Some(is) = is {
|
||||
self.match_member_count(is)?
|
||||
} else {
|
||||
false
|
||||
}
|
||||
}
|
||||
KnownCondition::SenderNotificationPermission { key } => {
|
||||
if let Some(sender_power_level) = &self.sender_power_level {
|
||||
let required_level = self
|
||||
.notification_power_levels
|
||||
.get(key.as_ref())
|
||||
.copied()
|
||||
.unwrap_or(50);
|
||||
|
||||
*sender_power_level >= required_level
|
||||
} else {
|
||||
false
|
||||
}
|
||||
}
|
||||
KnownCondition::RelationMatch {
|
||||
rel_type,
|
||||
event_type_pattern,
|
||||
sender,
|
||||
sender_type,
|
||||
} => {
|
||||
self.match_relations(rel_type, sender, sender_type, user_id, event_type_pattern)?
|
||||
}
|
||||
};
|
||||
|
||||
Ok(result)
|
||||
}
|
||||
|
||||
/// Evaluates a relation condition.
|
||||
fn match_relations(
|
||||
&self,
|
||||
rel_type: &str,
|
||||
sender: &Option<Cow<str>>,
|
||||
sender_type: &Option<Cow<str>>,
|
||||
user_id: Option<&str>,
|
||||
event_type_pattern: &Option<Cow<str>>,
|
||||
) -> Result<bool, Error> {
|
||||
// First check if relation matching is enabled...
|
||||
if !self.relation_match_enabled {
|
||||
return Ok(false);
|
||||
}
|
||||
|
||||
// ... and if there are any relations to match against.
|
||||
let relations = if let Some(relations) = self.relations.get(rel_type) {
|
||||
relations
|
||||
} else {
|
||||
return Ok(false);
|
||||
};
|
||||
|
||||
// Extract the sender pattern from the condition
|
||||
let sender_pattern = if let Some(sender) = sender {
|
||||
Some(sender.as_ref())
|
||||
} else if let Some(sender_type) = sender_type {
|
||||
if sender_type == "user_id" {
|
||||
if let Some(user_id) = user_id {
|
||||
Some(user_id)
|
||||
} else {
|
||||
return Ok(false);
|
||||
}
|
||||
} else {
|
||||
warn!("Unrecognized sender_type: {sender_type}");
|
||||
return Ok(false);
|
||||
}
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
let mut sender_compiled_pattern = if let Some(pattern) = sender_pattern {
|
||||
Some(get_glob_matcher(pattern, GlobMatchType::Whole)?)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
let mut type_compiled_pattern = if let Some(pattern) = event_type_pattern {
|
||||
Some(get_glob_matcher(pattern, GlobMatchType::Whole)?)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
for (relation_sender, event_type) in relations {
|
||||
if let Some(pattern) = &mut sender_compiled_pattern {
|
||||
if !pattern.is_match(relation_sender)? {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(pattern) = &mut type_compiled_pattern {
|
||||
if !pattern.is_match(event_type)? {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
return Ok(true);
|
||||
}
|
||||
|
||||
Ok(false)
|
||||
}
|
||||
|
||||
/// Evaluates a `event_match` condition.
|
||||
fn match_event_match(
|
||||
&self,
|
||||
event_match: &EventMatchCondition,
|
||||
user_id: Option<&str>,
|
||||
) -> Result<bool, Error> {
|
||||
let pattern = if let Some(pattern) = &event_match.pattern {
|
||||
pattern
|
||||
} else if let Some(pattern_type) = &event_match.pattern_type {
|
||||
// The `pattern_type` can either be "user_id" or "user_localpart",
|
||||
// either way if we don't have a `user_id` then the condition can't
|
||||
// match.
|
||||
let user_id = if let Some(user_id) = user_id {
|
||||
user_id
|
||||
} else {
|
||||
return Ok(false);
|
||||
};
|
||||
|
||||
match &**pattern_type {
|
||||
"user_id" => user_id,
|
||||
"user_localpart" => get_localpart_from_id(user_id)?,
|
||||
_ => return Ok(false),
|
||||
}
|
||||
} else {
|
||||
return Ok(false);
|
||||
};
|
||||
|
||||
let haystack = if let Some(haystack) = self.flattened_keys.get(&*event_match.key) {
|
||||
haystack
|
||||
} else {
|
||||
return Ok(false);
|
||||
};
|
||||
|
||||
// For the content.body we match against "words", but for everything
|
||||
// else we match against the entire value.
|
||||
let match_type = if event_match.key == "content.body" {
|
||||
GlobMatchType::Word
|
||||
} else {
|
||||
GlobMatchType::Whole
|
||||
};
|
||||
|
||||
let mut compiled_pattern = get_glob_matcher(pattern, match_type)?;
|
||||
compiled_pattern.is_match(haystack)
|
||||
}
|
||||
|
||||
/// Match the member count against an 'is' condition
|
||||
/// The `is` condition can be things like '>2', '==3' or even just '4'.
|
||||
fn match_member_count(&self, is: &str) -> Result<bool, Error> {
|
||||
let captures = INEQUALITY_EXPR.captures(is).context("bad 'is' clause")?;
|
||||
let ineq = captures.get(1).map_or("==", |m| m.as_str());
|
||||
let rhs: u64 = captures
|
||||
.get(2)
|
||||
.context("missing number")?
|
||||
.as_str()
|
||||
.parse()?;
|
||||
|
||||
let matches = match ineq {
|
||||
"" | "==" => self.room_member_count == rhs,
|
||||
"<" => self.room_member_count < rhs,
|
||||
">" => self.room_member_count > rhs,
|
||||
">=" => self.room_member_count >= rhs,
|
||||
"<=" => self.room_member_count <= rhs,
|
||||
_ => false,
|
||||
};
|
||||
|
||||
Ok(matches)
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn push_rule_evaluator() {
|
||||
let mut flattened_keys = BTreeMap::new();
|
||||
flattened_keys.insert("content.body".to_string(), "foo bar bob hello".to_string());
|
||||
let evaluator = PushRuleEvaluator::py_new(
|
||||
flattened_keys,
|
||||
10,
|
||||
Some(0),
|
||||
BTreeMap::new(),
|
||||
BTreeMap::new(),
|
||||
true,
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let result = evaluator.run(&FilteredPushRules::default(), None, Some("bob"));
|
||||
assert_eq!(result.len(), 3);
|
||||
}
|
||||
+20
-8
@@ -42,7 +42,6 @@
|
||||
//!
|
||||
//! The set of "base rules" are the list of rules that every user has by default. A
|
||||
//! user can modify their copy of the push rules in one of three ways:
|
||||
//!
|
||||
//! 1. Adding a new push rule of a certain kind
|
||||
//! 2. Changing the actions of a base rule
|
||||
//! 3. Enabling/disabling a base rule.
|
||||
@@ -58,12 +57,16 @@ use std::collections::{BTreeMap, HashMap, HashSet};
|
||||
use anyhow::{Context, Error};
|
||||
use log::warn;
|
||||
use pyo3::prelude::*;
|
||||
use pythonize::pythonize;
|
||||
use pythonize::{depythonize, pythonize};
|
||||
use serde::de::Error as _;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::Value;
|
||||
|
||||
use self::evaluator::PushRuleEvaluator;
|
||||
|
||||
mod base_rules;
|
||||
pub mod evaluator;
|
||||
pub mod utils;
|
||||
|
||||
/// Called when registering modules with python.
|
||||
pub fn register_module(py: Python<'_>, m: &PyModule) -> PyResult<()> {
|
||||
@@ -71,6 +74,7 @@ pub fn register_module(py: Python<'_>, m: &PyModule) -> PyResult<()> {
|
||||
child_module.add_class::<PushRule>()?;
|
||||
child_module.add_class::<PushRules>()?;
|
||||
child_module.add_class::<FilteredPushRules>()?;
|
||||
child_module.add_class::<PushRuleEvaluator>()?;
|
||||
child_module.add_function(wrap_pyfunction!(get_base_rule_ids, m)?)?;
|
||||
|
||||
m.add_submodule(child_module)?;
|
||||
@@ -274,6 +278,8 @@ pub enum KnownCondition {
|
||||
#[serde(rename = "org.matrix.msc3772.relation_match")]
|
||||
RelationMatch {
|
||||
rel_type: Cow<'static, str>,
|
||||
#[serde(skip_serializing_if = "Option::is_none", rename = "type")]
|
||||
event_type_pattern: Option<Cow<'static, str>>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
sender: Option<Cow<'static, str>>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
@@ -287,20 +293,26 @@ impl IntoPy<PyObject> for Condition {
|
||||
}
|
||||
}
|
||||
|
||||
impl<'source> FromPyObject<'source> for Condition {
|
||||
fn extract(ob: &'source PyAny) -> PyResult<Self> {
|
||||
Ok(depythonize(ob)?)
|
||||
}
|
||||
}
|
||||
|
||||
/// The body of a [`Condition::EventMatch`]
|
||||
#[derive(Serialize, Deserialize, Debug, Clone)]
|
||||
pub struct EventMatchCondition {
|
||||
key: Cow<'static, str>,
|
||||
pub key: Cow<'static, str>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pattern: Option<Cow<'static, str>>,
|
||||
pub pattern: Option<Cow<'static, str>>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pattern_type: Option<Cow<'static, str>>,
|
||||
pub pattern_type: Option<Cow<'static, str>>,
|
||||
}
|
||||
|
||||
/// The collection of push rules for a user.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
#[pyclass(frozen)]
|
||||
struct PushRules {
|
||||
pub struct PushRules {
|
||||
/// Custom push rules that override a base rule.
|
||||
overridden_base_rules: HashMap<Cow<'static, str>, PushRule>,
|
||||
|
||||
@@ -319,7 +331,7 @@ struct PushRules {
|
||||
#[pymethods]
|
||||
impl PushRules {
|
||||
#[new]
|
||||
fn new(rules: Vec<PushRule>) -> PushRules {
|
||||
pub fn new(rules: Vec<PushRule>) -> PushRules {
|
||||
let mut push_rules: PushRules = Default::default();
|
||||
|
||||
for rule in rules {
|
||||
@@ -396,7 +408,7 @@ pub struct FilteredPushRules {
|
||||
#[pymethods]
|
||||
impl FilteredPushRules {
|
||||
#[new]
|
||||
fn py_new(
|
||||
pub fn py_new(
|
||||
push_rules: PushRules,
|
||||
enabled_map: BTreeMap<String, bool>,
|
||||
msc3786_enabled: bool,
|
||||
|
||||
@@ -0,0 +1,215 @@
|
||||
// Copyright 2022 The Matrix.org Foundation C.I.C.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
use anyhow::bail;
|
||||
use anyhow::Context;
|
||||
use anyhow::Error;
|
||||
use lazy_static::lazy_static;
|
||||
use regex;
|
||||
use regex::Regex;
|
||||
use regex::RegexBuilder;
|
||||
|
||||
lazy_static! {
|
||||
/// Matches runs of non-wildcard characters followed by wildcard characters.
|
||||
static ref WILDCARD_RUN: Regex = Regex::new(r"([^\?\*]*)([\?\*]*)").expect("valid regex");
|
||||
}
|
||||
|
||||
/// Extract the localpart from a Matrix style ID
|
||||
pub(crate) fn get_localpart_from_id(id: &str) -> Result<&str, Error> {
|
||||
let (localpart, _) = id
|
||||
.split_once(':')
|
||||
.with_context(|| format!("ID does not contain colon: {id}"))?;
|
||||
|
||||
// We need to strip off the first character, which is the ID type.
|
||||
if localpart.is_empty() {
|
||||
bail!("Invalid ID {id}");
|
||||
}
|
||||
|
||||
Ok(&localpart[1..])
|
||||
}
|
||||
|
||||
/// Used by `glob_to_regex` to specify what to match the regex against.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum GlobMatchType {
|
||||
/// The generated regex will match against the entire input.
|
||||
Whole,
|
||||
/// The generated regex will match against words.
|
||||
Word,
|
||||
}
|
||||
|
||||
/// Convert a "glob" style expression to a regex, anchoring either to the entire
|
||||
/// input or to individual words.
|
||||
pub fn glob_to_regex(glob: &str, match_type: GlobMatchType) -> Result<Regex, Error> {
|
||||
let mut chunks = Vec::new();
|
||||
|
||||
// Patterns with wildcards must be simplified to avoid performance cliffs
|
||||
// - The glob `?**?**?` is equivalent to the glob `???*`
|
||||
// - The glob `???*` is equivalent to the regex `.{3,}`
|
||||
for captures in WILDCARD_RUN.captures_iter(glob) {
|
||||
if let Some(chunk) = captures.get(1) {
|
||||
chunks.push(regex::escape(chunk.as_str()));
|
||||
}
|
||||
|
||||
if let Some(wildcards) = captures.get(2) {
|
||||
if wildcards.as_str() == "" {
|
||||
continue;
|
||||
}
|
||||
|
||||
let question_marks = wildcards.as_str().chars().filter(|c| *c == '?').count();
|
||||
|
||||
if wildcards.as_str().contains('*') {
|
||||
chunks.push(format!(".{{{question_marks},}}"));
|
||||
} else {
|
||||
chunks.push(format!(".{{{question_marks}}}"));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let joined = chunks.join("");
|
||||
|
||||
let regex_str = match match_type {
|
||||
GlobMatchType::Whole => format!(r"\A{joined}\z"),
|
||||
|
||||
// `^|\W` and `\W|$` handle the case where `pattern` starts or ends with a non-word
|
||||
// character.
|
||||
GlobMatchType::Word => format!(r"(?:^|\b|\W){joined}(?:\b|\W|$)"),
|
||||
};
|
||||
|
||||
Ok(RegexBuilder::new(®ex_str)
|
||||
.case_insensitive(true)
|
||||
.build()?)
|
||||
}
|
||||
|
||||
/// Compiles the glob into a `Matcher`.
|
||||
pub fn get_glob_matcher(glob: &str, match_type: GlobMatchType) -> Result<Matcher, Error> {
|
||||
// There are a number of shortcuts we can make if the glob doesn't contain a
|
||||
// wild card.
|
||||
let matcher = if glob.contains(['*', '?']) {
|
||||
let regex = glob_to_regex(glob, match_type)?;
|
||||
Matcher::Regex(regex)
|
||||
} else if match_type == GlobMatchType::Whole {
|
||||
// If there aren't any wildcards and we're matching the whole thing,
|
||||
// then we simply can do a case-insensitive string match.
|
||||
Matcher::Whole(glob.to_lowercase())
|
||||
} else {
|
||||
// Otherwise, if we're matching against words then can first check
|
||||
// if the haystack contains the glob at all.
|
||||
Matcher::Word {
|
||||
word: glob.to_lowercase(),
|
||||
regex: None,
|
||||
}
|
||||
};
|
||||
|
||||
Ok(matcher)
|
||||
}
|
||||
|
||||
/// Matches against a glob
|
||||
pub enum Matcher {
|
||||
/// Plain regex matching.
|
||||
Regex(Regex),
|
||||
|
||||
/// Case-insensitive equality.
|
||||
Whole(String),
|
||||
|
||||
/// Word matching. `regex` is a cache of calling [`glob_to_regex`] on word.
|
||||
Word { word: String, regex: Option<Regex> },
|
||||
}
|
||||
|
||||
impl Matcher {
|
||||
/// Checks if the glob matches the given haystack.
|
||||
pub fn is_match(&mut self, haystack: &str) -> Result<bool, Error> {
|
||||
// We want to to do case-insensitive matching, so we convert to
|
||||
// lowercase first.
|
||||
let haystack = haystack.to_lowercase();
|
||||
|
||||
match self {
|
||||
Matcher::Regex(regex) => Ok(regex.is_match(&haystack)),
|
||||
Matcher::Whole(whole) => Ok(whole == &haystack),
|
||||
Matcher::Word { word, regex } => {
|
||||
// If we're looking for a literal word, then we first check if
|
||||
// the haystack contains the word as a substring.
|
||||
if !haystack.contains(&*word) {
|
||||
return Ok(false);
|
||||
}
|
||||
|
||||
// If it does contain the word as a substring, then we need to
|
||||
// check if it is an actual word by testing it against the regex.
|
||||
let regex = if let Some(regex) = regex {
|
||||
regex
|
||||
} else {
|
||||
let compiled_regex = glob_to_regex(word, GlobMatchType::Word)?;
|
||||
regex.insert(compiled_regex)
|
||||
};
|
||||
|
||||
Ok(regex.is_match(&haystack))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_get_domain_from_id() {
|
||||
get_localpart_from_id("").unwrap_err();
|
||||
get_localpart_from_id(":").unwrap_err();
|
||||
get_localpart_from_id(":asd").unwrap_err();
|
||||
get_localpart_from_id("::as::asad").unwrap_err();
|
||||
|
||||
assert_eq!(get_localpart_from_id("@test:foo").unwrap(), "test");
|
||||
assert_eq!(get_localpart_from_id("@:").unwrap(), "");
|
||||
assert_eq!(get_localpart_from_id("@test:foo:907").unwrap(), "test");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tset_glob() -> Result<(), Error> {
|
||||
assert_eq!(
|
||||
glob_to_regex("simple", GlobMatchType::Whole)?.as_str(),
|
||||
r"\Asimple\z"
|
||||
);
|
||||
assert_eq!(
|
||||
glob_to_regex("simple*", GlobMatchType::Whole)?.as_str(),
|
||||
r"\Asimple.{0,}\z"
|
||||
);
|
||||
assert_eq!(
|
||||
glob_to_regex("simple?", GlobMatchType::Whole)?.as_str(),
|
||||
r"\Asimple.{1}\z"
|
||||
);
|
||||
assert_eq!(
|
||||
glob_to_regex("simple?*?*", GlobMatchType::Whole)?.as_str(),
|
||||
r"\Asimple.{2,}\z"
|
||||
);
|
||||
assert_eq!(
|
||||
glob_to_regex("simple???", GlobMatchType::Whole)?.as_str(),
|
||||
r"\Asimple.{3}\z"
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
glob_to_regex("escape.", GlobMatchType::Whole)?.as_str(),
|
||||
r"\Aescape\.\z"
|
||||
);
|
||||
|
||||
assert!(glob_to_regex("simple", GlobMatchType::Whole)?.is_match("simple"));
|
||||
assert!(!glob_to_regex("simple", GlobMatchType::Whole)?.is_match("simples"));
|
||||
assert!(glob_to_regex("simple*", GlobMatchType::Whole)?.is_match("simples"));
|
||||
assert!(glob_to_regex("simple?", GlobMatchType::Whole)?.is_match("simples"));
|
||||
assert!(glob_to_regex("simple*", GlobMatchType::Whole)?.is_match("simple"));
|
||||
|
||||
assert!(glob_to_regex("simple", GlobMatchType::Word)?.is_match("some simple."));
|
||||
assert!(glob_to_regex("simple", GlobMatchType::Word)?.is_match("simple"));
|
||||
assert!(!glob_to_regex("simple", GlobMatchType::Word)?.is_match("simples"));
|
||||
|
||||
assert!(glob_to_regex("@user:foo", GlobMatchType::Word)?.is_match("Some @user:foo test"));
|
||||
assert!(glob_to_regex("@user:foo", GlobMatchType::Word)?.is_match("@user:foo"));
|
||||
|
||||
Ok(())
|
||||
}
|
||||
@@ -88,10 +88,9 @@ def make_wrapper(factory: Callable[P, R]) -> Callable[P, R]:
|
||||
|
||||
@functools.wraps(factory)
|
||||
def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
|
||||
# type-ignore: should be redundant once we can use https://github.com/python/mypy/pull/12668
|
||||
if "strict" not in kwargs: # type: ignore[attr-defined]
|
||||
if "strict" not in kwargs:
|
||||
raise MissingStrictInConstrainedTypeException(factory.__name__)
|
||||
if not kwargs["strict"]: # type: ignore[index]
|
||||
if not kwargs["strict"]:
|
||||
raise MissingStrictInConstrainedTypeException(factory.__name__)
|
||||
return factory(*args, **kwargs)
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
from typing import Any, Collection, Dict, Mapping, Sequence, Tuple, Union
|
||||
from typing import Any, Collection, Dict, Mapping, Optional, Sequence, Set, Tuple, Union
|
||||
|
||||
from synapse.types import JsonDict
|
||||
|
||||
@@ -35,3 +35,20 @@ class FilteredPushRules:
|
||||
def rules(self) -> Collection[Tuple[PushRule, bool]]: ...
|
||||
|
||||
def get_base_rule_ids() -> Collection[str]: ...
|
||||
|
||||
class PushRuleEvaluator:
|
||||
def __init__(
|
||||
self,
|
||||
flattened_keys: Mapping[str, str],
|
||||
room_member_count: int,
|
||||
sender_power_level: Optional[int],
|
||||
notification_power_levels: Mapping[str, int],
|
||||
relations: Mapping[str, Set[Tuple[str, str]]],
|
||||
relation_match_enabled: bool,
|
||||
): ...
|
||||
def run(
|
||||
self,
|
||||
push_rules: FilteredPushRules,
|
||||
user_id: Optional[str],
|
||||
display_name: Optional[str],
|
||||
) -> Collection[dict]: ...
|
||||
|
||||
@@ -107,7 +107,7 @@ BOOLEAN_COLUMNS = {
|
||||
"redactions": ["have_censored"],
|
||||
"room_stats_state": ["is_federatable"],
|
||||
"local_media_repository": ["safe_from_quarantine"],
|
||||
"users": ["shadow_banned"],
|
||||
"users": ["shadow_banned", "approved"],
|
||||
"e2e_fallback_keys_json": ["used"],
|
||||
"access_tokens": ["used"],
|
||||
"device_lists_changes_in_room": ["converted_to_destinations"],
|
||||
|
||||
@@ -272,3 +272,14 @@ class PublicRoomsFilterFields:
|
||||
|
||||
GENERIC_SEARCH_TERM: Final = "generic_search_term"
|
||||
ROOM_TYPES: Final = "room_types"
|
||||
|
||||
|
||||
class ApprovalNoticeMedium:
|
||||
"""Identifier for the medium this server will use to serve notice of approval for a
|
||||
specific user's registration.
|
||||
|
||||
As defined in https://github.com/matrix-org/matrix-spec-proposals/blob/babolivier/m_not_approved/proposals/3866-user-not-approved-error.md
|
||||
"""
|
||||
|
||||
NONE = "org.matrix.msc3866.none"
|
||||
EMAIL = "org.matrix.msc3866.email"
|
||||
|
||||
@@ -106,6 +106,8 @@ class Codes(str, Enum):
|
||||
# Part of MSC3895.
|
||||
UNABLE_DUE_TO_PARTIAL_STATE = "ORG.MATRIX.MSC3895_UNABLE_DUE_TO_PARTIAL_STATE"
|
||||
|
||||
USER_AWAITING_APPROVAL = "ORG.MATRIX.MSC3866_USER_AWAITING_APPROVAL"
|
||||
|
||||
|
||||
class CodeMessageException(RuntimeError):
|
||||
"""An exception with integer code and message string attributes.
|
||||
@@ -566,6 +568,20 @@ class UnredactedContentDeletedError(SynapseError):
|
||||
return cs_error(self.msg, self.errcode, **extra)
|
||||
|
||||
|
||||
class NotApprovedError(SynapseError):
|
||||
def __init__(
|
||||
self,
|
||||
msg: str,
|
||||
approval_notice_medium: str,
|
||||
):
|
||||
super().__init__(
|
||||
code=403,
|
||||
msg=msg,
|
||||
errcode=Codes.USER_AWAITING_APPROVAL,
|
||||
additional_fields={"approval_notice_medium": approval_notice_medium},
|
||||
)
|
||||
|
||||
|
||||
def cs_error(msg: str, code: str = Codes.UNKNOWN, **kwargs: Any) -> "JsonDict":
|
||||
"""Utility method for constructing an error response for client-server
|
||||
interactions.
|
||||
|
||||
@@ -98,9 +98,7 @@ def register_sighup(func: Callable[P, None], *args: P.args, **kwargs: P.kwargs)
|
||||
func: Function to be called when sent a SIGHUP signal.
|
||||
*args, **kwargs: args and kwargs to be passed to the target function.
|
||||
"""
|
||||
# This type-ignore should be redundant once we use a mypy release with
|
||||
# https://github.com/python/mypy/pull/12668.
|
||||
_sighup_callbacks.append((func, args, kwargs)) # type: ignore[arg-type]
|
||||
_sighup_callbacks.append((func, args, kwargs))
|
||||
|
||||
|
||||
def start_worker_reactor(
|
||||
|
||||
@@ -53,9 +53,9 @@ logger = logging.getLogger("synapse.app.admin_cmd")
|
||||
|
||||
class AdminCmdSlavedStore(
|
||||
SlavedFilteringStore,
|
||||
SlavedDeviceStore,
|
||||
SlavedPushRuleStore,
|
||||
SlavedEventStore,
|
||||
SlavedDeviceStore,
|
||||
TagsWorkerStore,
|
||||
DeviceInboxWorkerStore,
|
||||
AccountDataWorkerStore,
|
||||
|
||||
@@ -51,11 +51,18 @@ import argparse
|
||||
import importlib
|
||||
import itertools
|
||||
import multiprocessing
|
||||
import os
|
||||
import signal
|
||||
import sys
|
||||
from typing import Any, Callable, List
|
||||
from types import FrameType
|
||||
from typing import Any, Callable, List, Optional
|
||||
|
||||
from twisted.internet.main import installReactor
|
||||
|
||||
# a list of the original signal handlers, before we installed our custom ones.
|
||||
# We restore these in our child processes.
|
||||
_original_signal_handlers: dict[int, Any] = {}
|
||||
|
||||
|
||||
class ProxiedReactor:
|
||||
"""
|
||||
@@ -105,6 +112,11 @@ def _worker_entrypoint(
|
||||
|
||||
sys.argv = args
|
||||
|
||||
# reset the custom signal handlers that we installed, so that the children start
|
||||
# from a clean slate.
|
||||
for sig, handler in _original_signal_handlers.items():
|
||||
signal.signal(sig, handler)
|
||||
|
||||
from twisted.internet.epollreactor import EPollReactor
|
||||
|
||||
proxy_reactor._install_real_reactor(EPollReactor())
|
||||
@@ -167,13 +179,29 @@ def main() -> None:
|
||||
update_proc.join()
|
||||
print("===== PREPARED DATABASE =====", file=sys.stderr)
|
||||
|
||||
processes: List[multiprocessing.Process] = []
|
||||
|
||||
# Install signal handlers to propagate signals to all our children, so that they
|
||||
# shut down cleanly. This also inhibits our own exit, but that's good: we want to
|
||||
# wait until the children have exited.
|
||||
def handle_signal(signum: int, frame: Optional[FrameType]) -> None:
|
||||
print(
|
||||
f"complement_fork_starter: Caught signal {signum}. Stopping children.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
for p in processes:
|
||||
if p.pid:
|
||||
os.kill(p.pid, signum)
|
||||
|
||||
for sig in (signal.SIGINT, signal.SIGTERM):
|
||||
_original_signal_handlers[sig] = signal.signal(sig, handle_signal)
|
||||
|
||||
# At this point, we've imported all the main entrypoints for all the workers.
|
||||
# Now we basically just fork() out to create the workers we need.
|
||||
# Because we're using fork(), all the workers get a clone of this launcher's
|
||||
# memory space and don't need to repeat the work of loading the code!
|
||||
# Instead of using fork() directly, we use the multiprocessing library,
|
||||
# which uses fork() on Unix platforms.
|
||||
processes = []
|
||||
for (func, worker_args) in zip(worker_functions, args_by_worker):
|
||||
process = multiprocessing.Process(
|
||||
target=_worker_entrypoint, args=(func, proxy_reactor, worker_args)
|
||||
|
||||
@@ -14,10 +14,25 @@
|
||||
|
||||
from typing import Any
|
||||
|
||||
import attr
|
||||
|
||||
from synapse.config._base import Config
|
||||
from synapse.types import JsonDict
|
||||
|
||||
|
||||
@attr.s(auto_attribs=True, frozen=True, slots=True)
|
||||
class MSC3866Config:
|
||||
"""Configuration for MSC3866 (mandating approval for new users)"""
|
||||
|
||||
# Whether the base support for the approval process is enabled. This includes the
|
||||
# ability for administrators to check and update the approval of users, even if no
|
||||
# approval is currently required.
|
||||
enabled: bool = False
|
||||
# Whether to require that new users are approved by an admin before their account
|
||||
# can be used. Note that this setting is ignored if 'enabled' is false.
|
||||
require_approval_for_new_accounts: bool = False
|
||||
|
||||
|
||||
class ExperimentalConfig(Config):
|
||||
"""Config section for enabling experimental features"""
|
||||
|
||||
@@ -97,6 +112,10 @@ class ExperimentalConfig(Config):
|
||||
# MSC3852: Expose last seen user agent field on /_matrix/client/v3/devices.
|
||||
self.msc3852_enabled: bool = experimental.get("msc3852_enabled", False)
|
||||
|
||||
# MSC3866: M_USER_AWAITING_APPROVAL error code
|
||||
raw_msc3866_config = experimental.get("msc3866", {})
|
||||
self.msc3866 = MSC3866Config(**raw_msc3866_config)
|
||||
|
||||
# MSC3881: Remotely toggle push notifications for another client
|
||||
self.msc3881_enabled: bool = experimental.get("msc3881_enabled", False)
|
||||
|
||||
|
||||
@@ -289,6 +289,10 @@ class _EventInternalMetadata:
|
||||
"""
|
||||
return self._dict.get("historical", False)
|
||||
|
||||
def is_notifiable(self) -> bool:
|
||||
"""Whether this event can trigger a push notification"""
|
||||
return not self.is_outlier() or self.is_out_of_band_membership()
|
||||
|
||||
|
||||
class EventBase(metaclass=abc.ABCMeta):
|
||||
@property
|
||||
|
||||
@@ -646,10 +646,25 @@ class _TransactionQueueManager:
|
||||
|
||||
# We start by fetching device related EDUs, i.e device updates and to
|
||||
# device messages. We have to keep 2 free slots for presence and rr_edus.
|
||||
limit = MAX_EDUS_PER_TRANSACTION - 2
|
||||
device_edu_limit = MAX_EDUS_PER_TRANSACTION - 2
|
||||
|
||||
# We prioritize to-device messages so that existing encryption channels
|
||||
# work. We also keep a few slots spare (by reducing the limit) so that
|
||||
# we can still trickle out some device list updates.
|
||||
(
|
||||
to_device_edus,
|
||||
device_stream_id,
|
||||
) = await self.queue._get_to_device_message_edus(device_edu_limit - 10)
|
||||
|
||||
if to_device_edus:
|
||||
self._device_stream_id = device_stream_id
|
||||
else:
|
||||
self.queue._last_device_stream_id = device_stream_id
|
||||
|
||||
device_edu_limit -= len(to_device_edus)
|
||||
|
||||
device_update_edus, dev_list_id = await self.queue._get_device_update_edus(
|
||||
limit
|
||||
device_edu_limit
|
||||
)
|
||||
|
||||
if device_update_edus:
|
||||
@@ -657,18 +672,6 @@ class _TransactionQueueManager:
|
||||
else:
|
||||
self.queue._last_device_list_stream_id = dev_list_id
|
||||
|
||||
limit -= len(device_update_edus)
|
||||
|
||||
(
|
||||
to_device_edus,
|
||||
device_stream_id,
|
||||
) = await self.queue._get_to_device_message_edus(limit)
|
||||
|
||||
if to_device_edus:
|
||||
self._device_stream_id = device_stream_id
|
||||
else:
|
||||
self.queue._last_device_stream_id = device_stream_id
|
||||
|
||||
pending_edus = device_update_edus + to_device_edus
|
||||
|
||||
# Now add the read receipt EDU.
|
||||
|
||||
@@ -32,6 +32,7 @@ class AdminHandler:
|
||||
self.store = hs.get_datastores().main
|
||||
self._storage_controllers = hs.get_storage_controllers()
|
||||
self._state_storage_controller = self._storage_controllers.state
|
||||
self._msc3866_enabled = hs.config.experimental.msc3866.enabled
|
||||
|
||||
async def get_whois(self, user: UserID) -> JsonDict:
|
||||
connections = []
|
||||
@@ -75,6 +76,10 @@ class AdminHandler:
|
||||
"is_guest",
|
||||
}
|
||||
|
||||
if self._msc3866_enabled:
|
||||
# Only include the approved flag if support for MSC3866 is enabled.
|
||||
user_info_to_return.add("approved")
|
||||
|
||||
# Restrict returned keys to a known set.
|
||||
user_info_dict = {
|
||||
key: value
|
||||
|
||||
@@ -1009,6 +1009,17 @@ class AuthHandler:
|
||||
return res[0]
|
||||
return None
|
||||
|
||||
async def is_user_approved(self, user_id: str) -> bool:
|
||||
"""Checks if a user is approved and therefore can be allowed to log in.
|
||||
|
||||
Args:
|
||||
user_id: the user to check the approval status of.
|
||||
|
||||
Returns:
|
||||
A boolean that is True if the user is approved, False otherwise.
|
||||
"""
|
||||
return await self.store.is_user_approved(user_id)
|
||||
|
||||
async def _find_user_id_and_pwd_hash(
|
||||
self, user_id: str
|
||||
) -> Optional[Tuple[str, str]]:
|
||||
|
||||
+144
-4
@@ -273,11 +273,9 @@ class DeviceWorkerHandler:
|
||||
possibly_left = possibly_changed | possibly_left
|
||||
|
||||
# Double check if we still share rooms with the given user.
|
||||
users_rooms = await self.store.get_rooms_for_users_with_stream_ordering(
|
||||
possibly_left
|
||||
)
|
||||
users_rooms = await self.store.get_rooms_for_users(possibly_left)
|
||||
for changed_user_id, entries in users_rooms.items():
|
||||
if any(e.room_id in room_ids for e in entries):
|
||||
if any(rid in room_ids for rid in entries):
|
||||
possibly_left.discard(changed_user_id)
|
||||
else:
|
||||
possibly_joined.discard(changed_user_id)
|
||||
@@ -309,6 +307,17 @@ class DeviceWorkerHandler:
|
||||
"self_signing_key": self_signing_key,
|
||||
}
|
||||
|
||||
async def handle_room_un_partial_stated(self, room_id: str) -> None:
|
||||
"""Handles sending appropriate device list updates in a room that has
|
||||
gone from partial to full state.
|
||||
"""
|
||||
|
||||
# TODO(faster_joins): worker mode support
|
||||
# https://github.com/matrix-org/synapse/issues/12994
|
||||
logger.error(
|
||||
"Trying handling device list state for partial join: not supported on workers."
|
||||
)
|
||||
|
||||
|
||||
class DeviceHandler(DeviceWorkerHandler):
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
@@ -746,6 +755,95 @@ class DeviceHandler(DeviceWorkerHandler):
|
||||
finally:
|
||||
self._handle_new_device_update_is_processing = False
|
||||
|
||||
async def handle_room_un_partial_stated(self, room_id: str) -> None:
|
||||
"""Handles sending appropriate device list updates in a room that has
|
||||
gone from partial to full state.
|
||||
"""
|
||||
|
||||
# We defer to the device list updater to handle pending remote device
|
||||
# list updates.
|
||||
await self.device_list_updater.handle_room_un_partial_stated(room_id)
|
||||
|
||||
# Replay local updates.
|
||||
(
|
||||
join_event_id,
|
||||
device_lists_stream_id,
|
||||
) = await self.store.get_join_event_id_and_device_lists_stream_id_for_partial_state(
|
||||
room_id
|
||||
)
|
||||
|
||||
# Get the local device list changes that have happened in the room since
|
||||
# we started joining. If there are no updates there's nothing left to do.
|
||||
changes = await self.store.get_device_list_changes_in_room(
|
||||
room_id, device_lists_stream_id
|
||||
)
|
||||
local_changes = {(u, d) for u, d in changes if self.hs.is_mine_id(u)}
|
||||
if not local_changes:
|
||||
return
|
||||
|
||||
# Note: We have persisted the full state at this point, we just haven't
|
||||
# cleared the `partial_room` flag.
|
||||
join_state_ids = await self._state_storage.get_state_ids_for_event(
|
||||
join_event_id, await_full_state=False
|
||||
)
|
||||
current_state_ids = await self.store.get_partial_current_state_ids(room_id)
|
||||
|
||||
# Now we need to work out all servers that might have been in the room
|
||||
# at any point during our join.
|
||||
|
||||
# First we look for any membership states that have changed between the
|
||||
# initial join and now...
|
||||
all_keys = set(join_state_ids)
|
||||
all_keys.update(current_state_ids)
|
||||
|
||||
potentially_changed_hosts = set()
|
||||
for etype, state_key in all_keys:
|
||||
if etype != EventTypes.Member:
|
||||
continue
|
||||
|
||||
prev = join_state_ids.get((etype, state_key))
|
||||
current = current_state_ids.get((etype, state_key))
|
||||
|
||||
if prev != current:
|
||||
potentially_changed_hosts.add(get_domain_from_id(state_key))
|
||||
|
||||
# ... then we add all the hosts that are currently joined to the room...
|
||||
current_hosts_in_room = await self.store.get_current_hosts_in_room(room_id)
|
||||
potentially_changed_hosts.update(current_hosts_in_room)
|
||||
|
||||
# ... and finally we remove any hosts that we were told about, as we
|
||||
# will have sent device list updates to those hosts when they happened.
|
||||
known_hosts_at_join = await self.store.get_partial_state_servers_at_join(
|
||||
room_id
|
||||
)
|
||||
potentially_changed_hosts.difference_update(known_hosts_at_join)
|
||||
|
||||
potentially_changed_hosts.discard(self.server_name)
|
||||
|
||||
if not potentially_changed_hosts:
|
||||
# Nothing to do.
|
||||
return
|
||||
|
||||
logger.info(
|
||||
"Found %d changed hosts to send device list updates to",
|
||||
len(potentially_changed_hosts),
|
||||
)
|
||||
|
||||
for user_id, device_id in local_changes:
|
||||
await self.store.add_device_list_outbound_pokes(
|
||||
user_id=user_id,
|
||||
device_id=device_id,
|
||||
room_id=room_id,
|
||||
stream_id=None,
|
||||
hosts=potentially_changed_hosts,
|
||||
context=None,
|
||||
)
|
||||
|
||||
# Notify things that device lists need to be sent out.
|
||||
self.notifier.notify_replication()
|
||||
for host in potentially_changed_hosts:
|
||||
self.federation_sender.send_device_messages(host, immediate=False)
|
||||
|
||||
|
||||
def _update_device_from_client_ips(
|
||||
device: JsonDict, client_ips: Mapping[Tuple[str, str], Mapping[str, Any]]
|
||||
@@ -836,6 +934,16 @@ class DeviceListUpdater:
|
||||
)
|
||||
return
|
||||
|
||||
# Check if we are partially joining any rooms. If so we need to store
|
||||
# all device list updates so that we can handle them correctly once we
|
||||
# know who is in the room.
|
||||
partial_rooms = await self.store.get_partial_state_rooms_and_servers()
|
||||
if partial_rooms:
|
||||
await self.store.add_remote_device_list_to_pending(
|
||||
user_id,
|
||||
device_id,
|
||||
)
|
||||
|
||||
room_ids = await self.store.get_rooms_for_user(user_id)
|
||||
if not room_ids:
|
||||
# We don't share any rooms with this user. Ignore update, as we
|
||||
@@ -1175,3 +1283,35 @@ class DeviceListUpdater:
|
||||
device_ids.append(verify_key.version)
|
||||
|
||||
return device_ids
|
||||
|
||||
async def handle_room_un_partial_stated(self, room_id: str) -> None:
|
||||
"""Handles sending appropriate device list updates in a room that has
|
||||
gone from partial to full state.
|
||||
"""
|
||||
|
||||
pending_updates = (
|
||||
await self.store.get_pending_remote_device_list_updates_for_room(room_id)
|
||||
)
|
||||
|
||||
for user_id, device_id in pending_updates:
|
||||
logger.info(
|
||||
"Got pending device list update in room %s: %s / %s",
|
||||
room_id,
|
||||
user_id,
|
||||
device_id,
|
||||
)
|
||||
position = await self.store.add_device_change_to_streams(
|
||||
user_id,
|
||||
[device_id],
|
||||
room_ids=[room_id],
|
||||
)
|
||||
|
||||
if not position:
|
||||
# This should only happen if there are no updates, which
|
||||
# shouldn't happen when we've passed in a non-empty set of
|
||||
# device IDs.
|
||||
continue
|
||||
|
||||
self.device_handler.notifier.on_new_event(
|
||||
StreamKeyType.DEVICE_LIST, position, rooms=[room_id]
|
||||
)
|
||||
|
||||
+122
-55
@@ -38,7 +38,7 @@ from signedjson.sign import verify_signed_json
|
||||
from unpaddedbase64 import decode_base64
|
||||
|
||||
from synapse import event_auth
|
||||
from synapse.api.constants import EventContentFields, EventTypes, Membership
|
||||
from synapse.api.constants import MAX_DEPTH, EventContentFields, EventTypes, Membership
|
||||
from synapse.api.errors import (
|
||||
AuthError,
|
||||
CodeMessageException,
|
||||
@@ -149,6 +149,8 @@ class FederationHandler:
|
||||
self.http_client = hs.get_proxied_blacklisted_http_client()
|
||||
self._replication = hs.get_replication_data_handler()
|
||||
self._federation_event_handler = hs.get_federation_event_handler()
|
||||
self._device_handler = hs.get_device_handler()
|
||||
self._bulk_push_rule_evaluator = hs.get_bulk_push_rule_evaluator()
|
||||
|
||||
self._clean_room_for_join_client = ReplicationCleanRoomRestServlet.make_client(
|
||||
hs
|
||||
@@ -209,7 +211,7 @@ class FederationHandler:
|
||||
current_depth: int,
|
||||
limit: int,
|
||||
*,
|
||||
processing_start_time: int,
|
||||
processing_start_time: Optional[int],
|
||||
) -> bool:
|
||||
"""
|
||||
Checks whether the `current_depth` is at or approaching any backfill
|
||||
@@ -221,12 +223,23 @@ class FederationHandler:
|
||||
room_id: The room to backfill in.
|
||||
current_depth: The depth to check at for any upcoming backfill points.
|
||||
limit: The max number of events to request from the remote federated server.
|
||||
processing_start_time: The time when `maybe_backfill` started
|
||||
processing. Only used for timing.
|
||||
processing_start_time: The time when `maybe_backfill` started processing.
|
||||
Only used for timing. If `None`, no timing observation will be made.
|
||||
"""
|
||||
backwards_extremities = [
|
||||
_BackfillPoint(event_id, depth, _BackfillPointType.BACKWARDS_EXTREMITY)
|
||||
for event_id, depth in await self.store.get_backfill_points_in_room(room_id)
|
||||
for event_id, depth in await self.store.get_backfill_points_in_room(
|
||||
room_id=room_id,
|
||||
current_depth=current_depth,
|
||||
# We only need to end up with 5 extremities combined with the
|
||||
# insertion event extremities to make the `/backfill` request
|
||||
# but fetch an order of magnitude more to make sure there is
|
||||
# enough even after we filter them by whether visible in the
|
||||
# history. This isn't fool-proof as all backfill points within
|
||||
# our limit could be filtered out but seems like a good amount
|
||||
# to try with at least.
|
||||
limit=50,
|
||||
)
|
||||
]
|
||||
|
||||
insertion_events_to_be_backfilled: List[_BackfillPoint] = []
|
||||
@@ -234,7 +247,12 @@ class FederationHandler:
|
||||
insertion_events_to_be_backfilled = [
|
||||
_BackfillPoint(event_id, depth, _BackfillPointType.INSERTION_PONT)
|
||||
for event_id, depth in await self.store.get_insertion_event_backward_extremities_in_room(
|
||||
room_id
|
||||
room_id=room_id,
|
||||
current_depth=current_depth,
|
||||
# We only need to end up with 5 extremities combined with
|
||||
# the backfill points to make the `/backfill` request ...
|
||||
# (see the other comment above for more context).
|
||||
limit=50,
|
||||
)
|
||||
]
|
||||
logger.debug(
|
||||
@@ -243,10 +261,6 @@ class FederationHandler:
|
||||
insertion_events_to_be_backfilled,
|
||||
)
|
||||
|
||||
if not backwards_extremities and not insertion_events_to_be_backfilled:
|
||||
logger.debug("Not backfilling as no extremeties found.")
|
||||
return False
|
||||
|
||||
# we now have a list of potential places to backpaginate from. We prefer to
|
||||
# start with the most recent (ie, max depth), so let's sort the list.
|
||||
sorted_backfill_points: List[_BackfillPoint] = sorted(
|
||||
@@ -267,6 +281,33 @@ class FederationHandler:
|
||||
sorted_backfill_points,
|
||||
)
|
||||
|
||||
# If we have no backfill points lower than the `current_depth` then
|
||||
# either we can a) bail or b) still attempt to backfill. We opt to try
|
||||
# backfilling anyway just in case we do get relevant events.
|
||||
if not sorted_backfill_points and current_depth != MAX_DEPTH:
|
||||
logger.debug(
|
||||
"_maybe_backfill_inner: all backfill points are *after* current depth. Trying again with later backfill points."
|
||||
)
|
||||
return await self._maybe_backfill_inner(
|
||||
room_id=room_id,
|
||||
# We use `MAX_DEPTH` so that we find all backfill points next
|
||||
# time (all events are below the `MAX_DEPTH`)
|
||||
current_depth=MAX_DEPTH,
|
||||
limit=limit,
|
||||
# We don't want to start another timing observation from this
|
||||
# nested recursive call. The top-most call can record the time
|
||||
# overall otherwise the smaller one will throw off the results.
|
||||
processing_start_time=None,
|
||||
)
|
||||
|
||||
# Even after recursing with `MAX_DEPTH`, we didn't find any
|
||||
# backward extremities to backfill from.
|
||||
if not sorted_backfill_points:
|
||||
logger.debug(
|
||||
"_maybe_backfill_inner: Not backfilling as no backward extremeties found."
|
||||
)
|
||||
return False
|
||||
|
||||
# If we're approaching an extremity we trigger a backfill, otherwise we
|
||||
# no-op.
|
||||
#
|
||||
@@ -276,47 +317,16 @@ class FederationHandler:
|
||||
# chose more than one times the limit in case of failure, but choosing a
|
||||
# much larger factor will result in triggering a backfill request much
|
||||
# earlier than necessary.
|
||||
#
|
||||
# XXX: shouldn't we do this *after* the filter by depth below? Again, we don't
|
||||
# care about events that have happened after our current position.
|
||||
#
|
||||
max_depth = sorted_backfill_points[0].depth
|
||||
if current_depth - 2 * limit > max_depth:
|
||||
max_depth_of_backfill_points = sorted_backfill_points[0].depth
|
||||
if current_depth - 2 * limit > max_depth_of_backfill_points:
|
||||
logger.debug(
|
||||
"Not backfilling as we don't need to. %d < %d - 2 * %d",
|
||||
max_depth,
|
||||
max_depth_of_backfill_points,
|
||||
current_depth,
|
||||
limit,
|
||||
)
|
||||
return False
|
||||
|
||||
# We ignore extremities that have a greater depth than our current depth
|
||||
# as:
|
||||
# 1. we don't really care about getting events that have happened
|
||||
# after our current position; and
|
||||
# 2. we have likely previously tried and failed to backfill from that
|
||||
# extremity, so to avoid getting "stuck" requesting the same
|
||||
# backfill repeatedly we drop those extremities.
|
||||
#
|
||||
# However, we need to check that the filtered extremities are non-empty.
|
||||
# If they are empty then either we can a) bail or b) still attempt to
|
||||
# backfill. We opt to try backfilling anyway just in case we do get
|
||||
# relevant events.
|
||||
#
|
||||
filtered_sorted_backfill_points = [
|
||||
t for t in sorted_backfill_points if t.depth <= current_depth
|
||||
]
|
||||
if filtered_sorted_backfill_points:
|
||||
logger.debug(
|
||||
"_maybe_backfill_inner: backfill points before current depth: %s",
|
||||
filtered_sorted_backfill_points,
|
||||
)
|
||||
sorted_backfill_points = filtered_sorted_backfill_points
|
||||
else:
|
||||
logger.debug(
|
||||
"_maybe_backfill_inner: all backfill points are *after* current depth. Backfilling anyway."
|
||||
)
|
||||
|
||||
# For performance's sake, we only want to paginate from a particular extremity
|
||||
# if we can actually see the events we'll get. Otherwise, we'd just spend a lot
|
||||
# of resources to get redacted events. We check each extremity in turn and
|
||||
@@ -402,11 +412,22 @@ class FederationHandler:
|
||||
# First we try hosts that are already in the room.
|
||||
# TODO: HEURISTIC ALERT.
|
||||
likely_domains = (
|
||||
await self._storage_controllers.state.get_current_hosts_in_room(room_id)
|
||||
await self._storage_controllers.state.get_current_hosts_in_room_ordered(
|
||||
room_id
|
||||
)
|
||||
)
|
||||
|
||||
async def try_backfill(domains: Collection[str]) -> bool:
|
||||
# TODO: Should we try multiple of these at a time?
|
||||
|
||||
# Number of contacted remote homeservers that have denied our backfill
|
||||
# request with a 4xx code.
|
||||
denied_count = 0
|
||||
|
||||
# Maximum number of contacted remote homeservers that can deny our
|
||||
# backfill request with 4xx codes before we give up.
|
||||
max_denied_count = 5
|
||||
|
||||
for dom in domains:
|
||||
# We don't want to ask our own server for information we don't have
|
||||
if dom == self.server_name:
|
||||
@@ -425,13 +446,33 @@ class FederationHandler:
|
||||
continue
|
||||
except HttpResponseException as e:
|
||||
if 400 <= e.code < 500:
|
||||
raise e.to_synapse_error()
|
||||
logger.warning(
|
||||
"Backfill denied from %s because %s [%d/%d]",
|
||||
dom,
|
||||
e,
|
||||
denied_count,
|
||||
max_denied_count,
|
||||
)
|
||||
denied_count += 1
|
||||
if denied_count >= max_denied_count:
|
||||
return False
|
||||
continue
|
||||
|
||||
logger.info("Failed to backfill from %s because %s", dom, e)
|
||||
continue
|
||||
except CodeMessageException as e:
|
||||
if 400 <= e.code < 500:
|
||||
raise
|
||||
logger.warning(
|
||||
"Backfill denied from %s because %s [%d/%d]",
|
||||
dom,
|
||||
e,
|
||||
denied_count,
|
||||
max_denied_count,
|
||||
)
|
||||
denied_count += 1
|
||||
if denied_count >= max_denied_count:
|
||||
return False
|
||||
continue
|
||||
|
||||
logger.info("Failed to backfill from %s because %s", dom, e)
|
||||
continue
|
||||
@@ -450,10 +491,15 @@ class FederationHandler:
|
||||
|
||||
return False
|
||||
|
||||
processing_end_time = self.clock.time_msec()
|
||||
backfill_processing_before_timer.observe(
|
||||
(processing_end_time - processing_start_time) / 1000
|
||||
)
|
||||
# If we have the `processing_start_time`, then we can make an
|
||||
# observation. We wouldn't have the `processing_start_time` in the case
|
||||
# where `_maybe_backfill_inner` is recursively called to find any
|
||||
# backfill points regardless of `current_depth`.
|
||||
if processing_start_time is not None:
|
||||
processing_end_time = self.clock.time_msec()
|
||||
backfill_processing_before_timer.observe(
|
||||
(processing_end_time - processing_start_time) / 1000
|
||||
)
|
||||
|
||||
success = await try_backfill(likely_domains)
|
||||
if success:
|
||||
@@ -581,7 +627,11 @@ class FederationHandler:
|
||||
# Mark the room as having partial state.
|
||||
# The background process is responsible for unmarking this flag,
|
||||
# even if the join fails.
|
||||
await self.store.store_partial_state_room(room_id, ret.servers_in_room)
|
||||
await self.store.store_partial_state_room(
|
||||
room_id=room_id,
|
||||
servers=ret.servers_in_room,
|
||||
device_lists_stream_id=self.store.get_device_stream_token(),
|
||||
)
|
||||
|
||||
try:
|
||||
max_stream_id = (
|
||||
@@ -606,6 +656,14 @@ class FederationHandler:
|
||||
room_id,
|
||||
)
|
||||
raise LimitExceededError(msg=e.msg, errcode=e.errcode, retry_after_ms=0)
|
||||
else:
|
||||
# Record the join event id for future use (when we finish the full
|
||||
# join). We have to do this after persisting the event to keep foreign
|
||||
# key constraints intact.
|
||||
if ret.partial_state:
|
||||
await self.store.write_partial_state_rooms_join_event_id(
|
||||
room_id, event.event_id
|
||||
)
|
||||
finally:
|
||||
# Always kick off the background process that asynchronously fetches
|
||||
# state for the room.
|
||||
@@ -944,9 +1002,15 @@ class FederationHandler:
|
||||
)
|
||||
|
||||
context = EventContext.for_outlier(self._storage_controllers)
|
||||
await self._federation_event_handler.persist_events_and_notify(
|
||||
event.room_id, [(event, context)]
|
||||
)
|
||||
|
||||
await self._bulk_push_rule_evaluator.action_for_event_by_user(event, context)
|
||||
try:
|
||||
await self._federation_event_handler.persist_events_and_notify(
|
||||
event.room_id, [(event, context)]
|
||||
)
|
||||
except Exception:
|
||||
await self.store.remove_push_actions_from_staging(event.event_id)
|
||||
raise
|
||||
|
||||
return event
|
||||
|
||||
@@ -1612,6 +1676,9 @@ class FederationHandler:
|
||||
# https://github.com/matrix-org/synapse/issues/12994
|
||||
await self.state_handler.update_current_state(room_id)
|
||||
|
||||
logger.info("Handling any pending device list updates")
|
||||
await self._device_handler.handle_room_un_partial_stated(room_id)
|
||||
|
||||
logger.info("Clearing partial-state flag for %s", room_id)
|
||||
success = await self.store.clear_partial_state_room(room_id)
|
||||
if success:
|
||||
|
||||
@@ -2170,6 +2170,7 @@ class FederationEventHandler:
|
||||
if instance != self._instance_name:
|
||||
# Limit the number of events sent over replication. We choose 200
|
||||
# here as that is what we default to in `max_request_body_size(..)`
|
||||
result = {}
|
||||
try:
|
||||
for batch in batch_iter(event_and_contexts, 200):
|
||||
result = await self._send_events(
|
||||
|
||||
+483
-393
File diff suppressed because it is too large
Load Diff
@@ -220,6 +220,7 @@ class RegistrationHandler:
|
||||
by_admin: bool = False,
|
||||
user_agent_ips: Optional[List[Tuple[str, str]]] = None,
|
||||
auth_provider_id: Optional[str] = None,
|
||||
approved: bool = False,
|
||||
) -> str:
|
||||
"""Registers a new client on the server.
|
||||
|
||||
@@ -246,6 +247,8 @@ class RegistrationHandler:
|
||||
user_agent_ips: Tuples of user-agents and IP addresses used
|
||||
during the registration process.
|
||||
auth_provider_id: The SSO IdP the user used, if any.
|
||||
approved: True if the new user should be considered already
|
||||
approved by an administrator.
|
||||
Returns:
|
||||
The registered user_id.
|
||||
Raises:
|
||||
@@ -307,6 +310,7 @@ class RegistrationHandler:
|
||||
user_type=user_type,
|
||||
address=address,
|
||||
shadow_banned=shadow_banned,
|
||||
approved=approved,
|
||||
)
|
||||
|
||||
profile = await self.store.get_profileinfo(localpart)
|
||||
@@ -695,6 +699,7 @@ class RegistrationHandler:
|
||||
user_type: Optional[str] = None,
|
||||
address: Optional[str] = None,
|
||||
shadow_banned: bool = False,
|
||||
approved: bool = False,
|
||||
) -> None:
|
||||
"""Register user in the datastore.
|
||||
|
||||
@@ -713,6 +718,7 @@ class RegistrationHandler:
|
||||
api.constants.UserTypes, or None for a normal user.
|
||||
address: the IP address used to perform the registration.
|
||||
shadow_banned: Whether to shadow-ban the user
|
||||
approved: Whether to mark the user as approved by an administrator
|
||||
"""
|
||||
if self.hs.config.worker.worker_app:
|
||||
await self._register_client(
|
||||
@@ -726,6 +732,7 @@ class RegistrationHandler:
|
||||
user_type=user_type,
|
||||
address=address,
|
||||
shadow_banned=shadow_banned,
|
||||
approved=approved,
|
||||
)
|
||||
else:
|
||||
await self.store.register_user(
|
||||
@@ -738,6 +745,7 @@ class RegistrationHandler:
|
||||
admin=admin,
|
||||
user_type=user_type,
|
||||
shadow_banned=shadow_banned,
|
||||
approved=approved,
|
||||
)
|
||||
|
||||
# Only call the account validity module(s) on the main process, to avoid
|
||||
|
||||
+116
-50
@@ -301,8 +301,7 @@ class RoomCreationHandler:
|
||||
# now send the tombstone
|
||||
await self.event_creation_handler.handle_new_client_event(
|
||||
requester=requester,
|
||||
event=tombstone_event,
|
||||
context=tombstone_context,
|
||||
events_and_context=[(tombstone_event, tombstone_context)],
|
||||
)
|
||||
|
||||
state_filter = StateFilter.from_types(
|
||||
@@ -716,7 +715,7 @@ class RoomCreationHandler:
|
||||
|
||||
if (
|
||||
self._server_notices_mxid is not None
|
||||
and requester.user.to_string() == self._server_notices_mxid
|
||||
and user_id == self._server_notices_mxid
|
||||
):
|
||||
# allow the server notices mxid to create rooms
|
||||
is_requester_admin = True
|
||||
@@ -1042,7 +1041,9 @@ class RoomCreationHandler:
|
||||
creator_join_profile: Optional[JsonDict] = None,
|
||||
ratelimit: bool = True,
|
||||
) -> Tuple[int, str, int]:
|
||||
"""Sends the initial events into a new room.
|
||||
"""Sends the initial events into a new room. Sends the room creation, membership,
|
||||
and power level events into the room sequentially, then creates and batches up the
|
||||
rest of the events to persist as a batch to the DB.
|
||||
|
||||
`power_level_content_override` doesn't apply when initial state has
|
||||
power level state event content.
|
||||
@@ -1053,13 +1054,23 @@ class RoomCreationHandler:
|
||||
"""
|
||||
|
||||
creator_id = creator.user.to_string()
|
||||
|
||||
event_keys = {"room_id": room_id, "sender": creator_id, "state_key": ""}
|
||||
|
||||
depth = 1
|
||||
|
||||
# the last event sent/persisted to the db
|
||||
last_sent_event_id: Optional[str] = None
|
||||
|
||||
def create(etype: str, content: JsonDict, **kwargs: Any) -> JsonDict:
|
||||
# the most recently created event
|
||||
prev_event: List[str] = []
|
||||
# a map of event types, state keys -> event_ids. We collect these mappings this as events are
|
||||
# created (but not persisted to the db) to determine state for future created events
|
||||
# (as this info can't be pulled from the db)
|
||||
state_map: MutableStateMap[str] = {}
|
||||
# current_state_group of last event created. Used for computing event context of
|
||||
# events to be batched
|
||||
current_state_group = None
|
||||
|
||||
def create_event_dict(etype: str, content: JsonDict, **kwargs: Any) -> JsonDict:
|
||||
e = {"type": etype, "content": content}
|
||||
|
||||
e.update(event_keys)
|
||||
@@ -1067,32 +1078,51 @@ class RoomCreationHandler:
|
||||
|
||||
return e
|
||||
|
||||
async def send(etype: str, content: JsonDict, **kwargs: Any) -> int:
|
||||
nonlocal last_sent_event_id
|
||||
async def create_event(
|
||||
etype: str,
|
||||
content: JsonDict,
|
||||
for_batch: bool,
|
||||
**kwargs: Any,
|
||||
) -> Tuple[EventBase, synapse.events.snapshot.EventContext]:
|
||||
nonlocal depth
|
||||
nonlocal prev_event
|
||||
|
||||
event = create(etype, content, **kwargs)
|
||||
logger.debug("Sending %s in new room", etype)
|
||||
# Allow these events to be sent even if the user is shadow-banned to
|
||||
# allow the room creation to complete.
|
||||
(
|
||||
sent_event,
|
||||
last_stream_id,
|
||||
) = await self.event_creation_handler.create_and_send_nonmember_event(
|
||||
event_dict = create_event_dict(etype, content, **kwargs)
|
||||
|
||||
new_event, new_context = await self.event_creation_handler.create_event(
|
||||
creator,
|
||||
event,
|
||||
event_dict,
|
||||
prev_event_ids=prev_event,
|
||||
depth=depth,
|
||||
state_map=state_map,
|
||||
for_batch=for_batch,
|
||||
current_state_group=current_state_group,
|
||||
)
|
||||
depth += 1
|
||||
prev_event = [new_event.event_id]
|
||||
state_map[(new_event.type, new_event.state_key)] = new_event.event_id
|
||||
|
||||
return new_event, new_context
|
||||
|
||||
async def send(
|
||||
event: EventBase,
|
||||
context: synapse.events.snapshot.EventContext,
|
||||
creator: Requester,
|
||||
) -> int:
|
||||
nonlocal last_sent_event_id
|
||||
|
||||
ev = await self.event_creation_handler.handle_new_client_event(
|
||||
requester=creator,
|
||||
events_and_context=[(event, context)],
|
||||
ratelimit=False,
|
||||
ignore_shadow_ban=True,
|
||||
# Note: we don't pass state_event_ids here because this triggers
|
||||
# an additional query per event to look them up from the events table.
|
||||
prev_event_ids=[last_sent_event_id] if last_sent_event_id else [],
|
||||
depth=depth,
|
||||
)
|
||||
|
||||
last_sent_event_id = sent_event.event_id
|
||||
depth += 1
|
||||
last_sent_event_id = ev.event_id
|
||||
|
||||
return last_stream_id
|
||||
# we know it was persisted, so must have a stream ordering
|
||||
assert ev.internal_metadata.stream_ordering
|
||||
return ev.internal_metadata.stream_ordering
|
||||
|
||||
try:
|
||||
config = self._presets_dict[preset_config]
|
||||
@@ -1102,9 +1132,13 @@ class RoomCreationHandler:
|
||||
)
|
||||
|
||||
creation_content.update({"creator": creator_id})
|
||||
await send(etype=EventTypes.Create, content=creation_content)
|
||||
creation_event, creation_context = await create_event(
|
||||
EventTypes.Create, creation_content, False
|
||||
)
|
||||
|
||||
logger.debug("Sending %s in new room", EventTypes.Member)
|
||||
await send(creation_event, creation_context, creator)
|
||||
|
||||
# Room create event must exist at this point
|
||||
assert last_sent_event_id is not None
|
||||
member_event_id, _ = await self.room_member_handler.update_membership(
|
||||
@@ -1118,15 +1152,22 @@ class RoomCreationHandler:
|
||||
prev_event_ids=[last_sent_event_id],
|
||||
depth=depth,
|
||||
)
|
||||
last_sent_event_id = member_event_id
|
||||
prev_event = [member_event_id]
|
||||
|
||||
# update the depth and state map here as the membership event has been created
|
||||
# through a different code path
|
||||
depth += 1
|
||||
state_map[(EventTypes.Member, creator.user.to_string())] = member_event_id
|
||||
|
||||
# We treat the power levels override specially as this needs to be one
|
||||
# of the first events that get sent into a room.
|
||||
pl_content = initial_state.pop((EventTypes.PowerLevels, ""), None)
|
||||
if pl_content is not None:
|
||||
last_sent_stream_id = await send(
|
||||
etype=EventTypes.PowerLevels, content=pl_content
|
||||
power_event, power_context = await create_event(
|
||||
EventTypes.PowerLevels, pl_content, False
|
||||
)
|
||||
current_state_group = power_context._state_group
|
||||
await send(power_event, power_context, creator)
|
||||
else:
|
||||
power_level_content: JsonDict = {
|
||||
"users": {creator_id: 100},
|
||||
@@ -1169,48 +1210,71 @@ class RoomCreationHandler:
|
||||
# apply those.
|
||||
if power_level_content_override:
|
||||
power_level_content.update(power_level_content_override)
|
||||
|
||||
last_sent_stream_id = await send(
|
||||
etype=EventTypes.PowerLevels, content=power_level_content
|
||||
pl_event, pl_context = await create_event(
|
||||
EventTypes.PowerLevels,
|
||||
power_level_content,
|
||||
False,
|
||||
)
|
||||
current_state_group = pl_context._state_group
|
||||
await send(pl_event, pl_context, creator)
|
||||
|
||||
events_to_send = []
|
||||
if room_alias and (EventTypes.CanonicalAlias, "") not in initial_state:
|
||||
last_sent_stream_id = await send(
|
||||
etype=EventTypes.CanonicalAlias,
|
||||
content={"alias": room_alias.to_string()},
|
||||
room_alias_event, room_alias_context = await create_event(
|
||||
EventTypes.CanonicalAlias, {"alias": room_alias.to_string()}, True
|
||||
)
|
||||
current_state_group = room_alias_context._state_group
|
||||
events_to_send.append((room_alias_event, room_alias_context))
|
||||
|
||||
if (EventTypes.JoinRules, "") not in initial_state:
|
||||
last_sent_stream_id = await send(
|
||||
etype=EventTypes.JoinRules, content={"join_rule": config["join_rules"]}
|
||||
join_rules_event, join_rules_context = await create_event(
|
||||
EventTypes.JoinRules,
|
||||
{"join_rule": config["join_rules"]},
|
||||
True,
|
||||
)
|
||||
current_state_group = join_rules_context._state_group
|
||||
events_to_send.append((join_rules_event, join_rules_context))
|
||||
|
||||
if (EventTypes.RoomHistoryVisibility, "") not in initial_state:
|
||||
last_sent_stream_id = await send(
|
||||
etype=EventTypes.RoomHistoryVisibility,
|
||||
content={"history_visibility": config["history_visibility"]},
|
||||
visibility_event, visibility_context = await create_event(
|
||||
EventTypes.RoomHistoryVisibility,
|
||||
{"history_visibility": config["history_visibility"]},
|
||||
True,
|
||||
)
|
||||
current_state_group = visibility_context._state_group
|
||||
events_to_send.append((visibility_event, visibility_context))
|
||||
|
||||
if config["guest_can_join"]:
|
||||
if (EventTypes.GuestAccess, "") not in initial_state:
|
||||
last_sent_stream_id = await send(
|
||||
etype=EventTypes.GuestAccess,
|
||||
content={EventContentFields.GUEST_ACCESS: GuestAccess.CAN_JOIN},
|
||||
guest_access_event, guest_access_context = await create_event(
|
||||
EventTypes.GuestAccess,
|
||||
{EventContentFields.GUEST_ACCESS: GuestAccess.CAN_JOIN},
|
||||
True,
|
||||
)
|
||||
current_state_group = guest_access_context._state_group
|
||||
events_to_send.append((guest_access_event, guest_access_context))
|
||||
|
||||
for (etype, state_key), content in initial_state.items():
|
||||
last_sent_stream_id = await send(
|
||||
etype=etype, state_key=state_key, content=content
|
||||
event, context = await create_event(
|
||||
etype, content, True, state_key=state_key
|
||||
)
|
||||
current_state_group = context._state_group
|
||||
events_to_send.append((event, context))
|
||||
|
||||
if config["encrypted"]:
|
||||
last_sent_stream_id = await send(
|
||||
etype=EventTypes.RoomEncryption,
|
||||
encryption_event, encryption_context = await create_event(
|
||||
EventTypes.RoomEncryption,
|
||||
{"algorithm": RoomEncryptionAlgorithms.DEFAULT},
|
||||
True,
|
||||
state_key="",
|
||||
content={"algorithm": RoomEncryptionAlgorithms.DEFAULT},
|
||||
)
|
||||
events_to_send.append((encryption_event, encryption_context))
|
||||
|
||||
return last_sent_stream_id, last_sent_event_id, depth
|
||||
last_event = await self.event_creation_handler.handle_new_client_event(
|
||||
creator, events_to_send, ignore_shadow_ban=True
|
||||
)
|
||||
assert last_event.internal_metadata.stream_ordering is not None
|
||||
return last_event.internal_metadata.stream_ordering, last_event.event_id, depth
|
||||
|
||||
def _generate_room_id(self) -> str:
|
||||
"""Generates a random room ID.
|
||||
@@ -1476,7 +1540,9 @@ class TimestampLookupHandler:
|
||||
)
|
||||
|
||||
likely_domains = (
|
||||
await self._storage_controllers.state.get_current_hosts_in_room(room_id)
|
||||
await self._storage_controllers.state.get_current_hosts_in_room_ordered(
|
||||
room_id
|
||||
)
|
||||
)
|
||||
|
||||
# Loop through each homeserver candidate until we get a succesful response
|
||||
|
||||
@@ -379,8 +379,7 @@ class RoomBatchHandler:
|
||||
await self.create_requester_for_user_id_from_app_service(
|
||||
event.sender, app_service_requester.app_service
|
||||
),
|
||||
event=event,
|
||||
context=context,
|
||||
events_and_context=[(event, context)],
|
||||
)
|
||||
|
||||
return event_ids
|
||||
|
||||
@@ -432,8 +432,7 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
|
||||
with tracing.start_active_span("handle_new_client_event"):
|
||||
result_event = await self.event_creation_handler.handle_new_client_event(
|
||||
requester,
|
||||
event,
|
||||
context,
|
||||
events_and_context=[(event, context)],
|
||||
extra_users=[target],
|
||||
ratelimit=ratelimit,
|
||||
)
|
||||
@@ -1252,7 +1251,10 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
|
||||
raise SynapseError(403, "This room has been blocked on this server")
|
||||
|
||||
event = await self.event_creation_handler.handle_new_client_event(
|
||||
requester, event, context, extra_users=[target_user], ratelimit=ratelimit
|
||||
requester,
|
||||
events_and_context=[(event, context)],
|
||||
extra_users=[target_user],
|
||||
ratelimit=ratelimit,
|
||||
)
|
||||
|
||||
prev_member_event_id = prev_state_ids.get(
|
||||
@@ -1860,8 +1862,7 @@ class RoomMemberMasterHandler(RoomMemberHandler):
|
||||
|
||||
result_event = await self.event_creation_handler.handle_new_client_event(
|
||||
requester,
|
||||
event,
|
||||
context,
|
||||
events_and_context=[(event, context)],
|
||||
extra_users=[UserID.from_string(target_user)],
|
||||
)
|
||||
# we know it was persisted, so must have a stream ordering
|
||||
|
||||
@@ -187,6 +187,19 @@ class SendEmailHandler:
|
||||
multipart_msg["To"] = email_address
|
||||
multipart_msg["Date"] = email.utils.formatdate()
|
||||
multipart_msg["Message-ID"] = email.utils.make_msgid()
|
||||
# Discourage automatic responses to Synapse's emails.
|
||||
# Per RFC 3834, automatic responses should not be sent if the "Auto-Submitted"
|
||||
# header is present with any value other than "no". See
|
||||
# https://www.rfc-editor.org/rfc/rfc3834.html#section-5.1
|
||||
multipart_msg["Auto-Submitted"] = "auto-generated"
|
||||
# Also include a Microsoft-Exchange specific header:
|
||||
# https://learn.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-oxcmail/ced68690-498a-4567-9d14-5c01f974d8b1
|
||||
# which suggests it can take the value "All" to "suppress all auto-replies",
|
||||
# or a comma separated list of auto-reply classes to suppress.
|
||||
# The following stack overflow question has a little more context:
|
||||
# https://stackoverflow.com/a/25324691/5252017
|
||||
# https://stackoverflow.com/a/61646381/5252017
|
||||
multipart_msg["X-Auto-Response-Suppress"] = "All"
|
||||
multipart_msg.attach(text_part)
|
||||
multipart_msg.attach(html_part)
|
||||
|
||||
|
||||
@@ -147,6 +147,9 @@ class UsernameMappingSession:
|
||||
# A unique identifier for this SSO provider, e.g. "oidc" or "saml".
|
||||
auth_provider_id: str
|
||||
|
||||
# An optional session ID from the IdP.
|
||||
auth_provider_session_id: Optional[str]
|
||||
|
||||
# user ID on the IdP server
|
||||
remote_user_id: str
|
||||
|
||||
@@ -464,6 +467,7 @@ class SsoHandler:
|
||||
client_redirect_url,
|
||||
next_step_url,
|
||||
extra_login_attributes,
|
||||
auth_provider_session_id,
|
||||
)
|
||||
|
||||
user_id = await self._register_mapped_user(
|
||||
@@ -585,6 +589,7 @@ class SsoHandler:
|
||||
client_redirect_url: str,
|
||||
next_step_url: bytes,
|
||||
extra_login_attributes: Optional[JsonDict],
|
||||
auth_provider_session_id: Optional[str],
|
||||
) -> NoReturn:
|
||||
"""Creates a UsernameMappingSession and redirects the browser
|
||||
|
||||
@@ -607,6 +612,8 @@ class SsoHandler:
|
||||
extra_login_attributes: An optional dictionary of extra
|
||||
attributes to be provided to the client in the login response.
|
||||
|
||||
auth_provider_session_id: An optional session ID from the IdP.
|
||||
|
||||
Raises:
|
||||
RedirectException
|
||||
"""
|
||||
@@ -615,6 +622,7 @@ class SsoHandler:
|
||||
now = self._clock.time_msec()
|
||||
session = UsernameMappingSession(
|
||||
auth_provider_id=auth_provider_id,
|
||||
auth_provider_session_id=auth_provider_session_id,
|
||||
remote_user_id=remote_user_id,
|
||||
display_name=attributes.display_name,
|
||||
emails=attributes.emails,
|
||||
@@ -968,6 +976,7 @@ class SsoHandler:
|
||||
session.client_redirect_url,
|
||||
session.extra_login_attributes,
|
||||
new_user=True,
|
||||
auth_provider_session_id=session.auth_provider_session_id,
|
||||
)
|
||||
|
||||
def _expire_old_sessions(self) -> None:
|
||||
|
||||
+23
-13
@@ -1196,7 +1196,9 @@ class SyncHandler:
|
||||
room_id: The partial state room to find the remaining memberships for.
|
||||
members_to_fetch: The memberships to find.
|
||||
events_with_membership_auth: A mapping from user IDs to events whose auth
|
||||
events are known to contain their membership.
|
||||
events would contain their prior membership, if one exists.
|
||||
Note that join events will not cite a prior membership if a user has
|
||||
never been in a room before.
|
||||
found_state_ids: A dict from (type, state_key) -> state_event_id, containing
|
||||
memberships that have been previously found. Entries in
|
||||
`members_to_fetch` that have a membership in `found_state_ids` are
|
||||
@@ -1206,6 +1208,10 @@ class SyncHandler:
|
||||
A dict from ("m.room.member", state_key) -> state_event_id, containing the
|
||||
memberships missing from `found_state_ids`.
|
||||
|
||||
When `events_with_membership_auth` contains a join event for a given user
|
||||
which does not cite a prior membership, no membership is returned for that
|
||||
user.
|
||||
|
||||
Raises:
|
||||
KeyError: if `events_with_membership_auth` does not have an entry for a
|
||||
missing membership. Memberships in `found_state_ids` do not need an
|
||||
@@ -1223,8 +1229,18 @@ class SyncHandler:
|
||||
if (EventTypes.Member, member) in found_state_ids:
|
||||
continue
|
||||
|
||||
missing_members.add(member)
|
||||
event_with_membership_auth = events_with_membership_auth[member]
|
||||
is_join = (
|
||||
event_with_membership_auth.is_state()
|
||||
and event_with_membership_auth.type == EventTypes.Member
|
||||
and event_with_membership_auth.state_key == member
|
||||
and event_with_membership_auth.content.get("membership")
|
||||
== Membership.JOIN
|
||||
)
|
||||
if not is_join:
|
||||
# The event must include the desired membership as an auth event, unless
|
||||
# it's the first join event for a given user.
|
||||
missing_members.add(member)
|
||||
auth_event_ids.update(event_with_membership_auth.auth_event_ids())
|
||||
|
||||
auth_events = await self.store.get_events(auth_event_ids)
|
||||
@@ -1248,7 +1264,7 @@ class SyncHandler:
|
||||
auth_event.type == EventTypes.Member
|
||||
and auth_event.state_key == member
|
||||
):
|
||||
missing_members.remove(member)
|
||||
missing_members.discard(member)
|
||||
additional_state_ids[
|
||||
(EventTypes.Member, member)
|
||||
] = auth_event.event_id
|
||||
@@ -1479,16 +1495,14 @@ class SyncHandler:
|
||||
since_token.device_list_key
|
||||
)
|
||||
if changed_users is not None:
|
||||
result = await self.store.get_rooms_for_users_with_stream_ordering(
|
||||
changed_users
|
||||
)
|
||||
result = await self.store.get_rooms_for_users(changed_users)
|
||||
|
||||
for changed_user_id, entries in result.items():
|
||||
# Check if the changed user shares any rooms with the user,
|
||||
# or if the changed user is the syncing user (as we always
|
||||
# want to include device list updates of their own devices).
|
||||
if user_id == changed_user_id or any(
|
||||
e.room_id in joined_rooms for e in entries
|
||||
rid in joined_rooms for rid in entries
|
||||
):
|
||||
users_that_have_changed.add(changed_user_id)
|
||||
else:
|
||||
@@ -1522,13 +1536,9 @@ class SyncHandler:
|
||||
newly_left_users.update(left_users)
|
||||
|
||||
# Remove any users that we still share a room with.
|
||||
left_users_rooms = (
|
||||
await self.store.get_rooms_for_users_with_stream_ordering(
|
||||
newly_left_users
|
||||
)
|
||||
)
|
||||
left_users_rooms = await self.store.get_rooms_for_users(newly_left_users)
|
||||
for user_id, entries in left_users_rooms.items():
|
||||
if any(e.room_id in joined_rooms for e in entries):
|
||||
if any(rid in joined_rooms for rid in entries):
|
||||
newly_left_users.discard(user_id)
|
||||
|
||||
return DeviceListUpdates(
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
# limitations under the License.
|
||||
|
||||
import logging
|
||||
from typing import TYPE_CHECKING, Any, Dict, List, Optional
|
||||
from typing import TYPE_CHECKING, Any, Dict, List, Optional, Set, Tuple
|
||||
|
||||
import synapse.metrics
|
||||
from synapse.api.constants import EventTypes, HistoryVisibility, JoinRules, Membership
|
||||
@@ -379,7 +379,7 @@ class UserDirectoryHandler(StateDeltasHandler):
|
||||
user_id, event.content.get("displayname"), event.content.get("avatar_url")
|
||||
)
|
||||
|
||||
async def _track_user_joined_room(self, room_id: str, user_id: str) -> None:
|
||||
async def _track_user_joined_room(self, room_id: str, joining_user_id: str) -> None:
|
||||
"""Someone's just joined a room. Update `users_in_public_rooms` or
|
||||
`users_who_share_private_rooms` as appropriate.
|
||||
|
||||
@@ -390,32 +390,44 @@ class UserDirectoryHandler(StateDeltasHandler):
|
||||
room_id
|
||||
)
|
||||
if is_public:
|
||||
await self.store.add_users_in_public_rooms(room_id, (user_id,))
|
||||
await self.store.add_users_in_public_rooms(room_id, (joining_user_id,))
|
||||
else:
|
||||
users_in_room = await self.store.get_users_in_room(room_id)
|
||||
other_users_in_room = [
|
||||
other
|
||||
for other in users_in_room
|
||||
if other != user_id
|
||||
if other != joining_user_id
|
||||
and (
|
||||
# We can't apply any special rules to remote users so
|
||||
# they're always included
|
||||
not self.is_mine_id(other)
|
||||
# Check the special rules whether the local user should be
|
||||
# included in the user directory
|
||||
or await self.store.should_include_local_user_in_dir(other)
|
||||
)
|
||||
]
|
||||
to_insert = set()
|
||||
updates_to_users_who_share_rooms: Set[Tuple[str, str]] = set()
|
||||
|
||||
# First, if they're our user then we need to update for every user
|
||||
if self.is_mine_id(user_id):
|
||||
# First, if the joining user is our local user then we need an
|
||||
# update for every other user in the room.
|
||||
if self.is_mine_id(joining_user_id):
|
||||
for other_user_id in other_users_in_room:
|
||||
to_insert.add((user_id, other_user_id))
|
||||
updates_to_users_who_share_rooms.add(
|
||||
(joining_user_id, other_user_id)
|
||||
)
|
||||
|
||||
# Next we need to update for every local user in the room
|
||||
# Next, we need an update for every other local user in the room
|
||||
# that they now share a room with the joining user.
|
||||
for other_user_id in other_users_in_room:
|
||||
if self.is_mine_id(other_user_id):
|
||||
to_insert.add((other_user_id, user_id))
|
||||
updates_to_users_who_share_rooms.add(
|
||||
(other_user_id, joining_user_id)
|
||||
)
|
||||
|
||||
if to_insert:
|
||||
await self.store.add_users_who_share_private_room(room_id, to_insert)
|
||||
if updates_to_users_who_share_rooms:
|
||||
await self.store.add_users_who_share_private_room(
|
||||
room_id, updates_to_users_who_share_rooms
|
||||
)
|
||||
|
||||
async def _handle_remove_user(self, room_id: str, user_id: str) -> None:
|
||||
"""Called when when someone leaves a room. The user may be local or remote.
|
||||
|
||||
+10
-10
@@ -579,7 +579,7 @@ class LoggingContextFilter(logging.Filter):
|
||||
True to include the record in the log output.
|
||||
"""
|
||||
context = current_context()
|
||||
record.request = self._default_request # type: ignore
|
||||
record.request = self._default_request
|
||||
|
||||
# context should never be None, but if it somehow ends up being, then
|
||||
# we end up in a death spiral of infinite loops, so let's check, for
|
||||
@@ -587,21 +587,21 @@ class LoggingContextFilter(logging.Filter):
|
||||
if context is not None:
|
||||
# Logging is interested in the request ID. Note that for backwards
|
||||
# compatibility this is stored as the "request" on the record.
|
||||
record.request = str(context) # type: ignore
|
||||
record.request = str(context)
|
||||
|
||||
# Add some data from the HTTP request.
|
||||
request = context.request
|
||||
if request is None:
|
||||
return True
|
||||
|
||||
record.ip_address = request.ip_address # type: ignore
|
||||
record.site_tag = request.site_tag # type: ignore
|
||||
record.requester = request.requester # type: ignore
|
||||
record.authenticated_entity = request.authenticated_entity # type: ignore
|
||||
record.method = request.method # type: ignore
|
||||
record.url = request.url # type: ignore
|
||||
record.protocol = request.protocol # type: ignore
|
||||
record.user_agent = request.user_agent # type: ignore
|
||||
record.ip_address = request.ip_address
|
||||
record.site_tag = request.site_tag
|
||||
record.requester = request.requester
|
||||
record.authenticated_entity = request.authenticated_entity
|
||||
record.method = request.method
|
||||
record.url = request.url
|
||||
record.protocol = request.protocol
|
||||
record.user_agent = request.user_agent
|
||||
|
||||
return True
|
||||
|
||||
|
||||
@@ -209,7 +209,10 @@ class _DummyLookup(object):
|
||||
def __init__(self, value: T) -> None:
|
||||
self.value = value
|
||||
|
||||
def __getattribute__(self, name: str) -> T:
|
||||
# type-ignore: Because mypy says "A function returning TypeVar should receive at
|
||||
# least one argument containing the same Typevar" but this is just a dummy
|
||||
# stand-in that doesn't need any input.
|
||||
def __getattribute__(self, name: str) -> T: # type: ignore[type-var]
|
||||
return object.__getattribute__(self, "value")
|
||||
|
||||
|
||||
@@ -949,9 +952,9 @@ def tag_args(func: Callable[P, R]) -> Callable[P, R]:
|
||||
# FIXME: We could update this to handle any type of function by ignoring the
|
||||
# first argument only if it's named `self` or `cls`. This isn't fool-proof
|
||||
# but handles the idiomatic cases.
|
||||
for i, arg in enumerate(args[1:], start=1): # type: ignore[index]
|
||||
for i, arg in enumerate(args[1:], start=1):
|
||||
set_attribute(SynapseTags.FUNC_ARG_PREFIX + argspec.args[i], str(arg))
|
||||
set_attribute(SynapseTags.FUNC_ARGS, str(args[len(argspec.args) :])) # type: ignore[index]
|
||||
set_attribute(SynapseTags.FUNC_ARGS, str(args[len(argspec.args) :]))
|
||||
set_attribute(SynapseTags.FUNC_KWARGS, str(kwargs))
|
||||
yield
|
||||
|
||||
|
||||
@@ -842,6 +842,8 @@ class ModuleApi:
|
||||
however invalidation that needs to go to other workers needs to call `invalidate_cache`
|
||||
on the module API instead.
|
||||
|
||||
Added in Synapse v1.69.0.
|
||||
|
||||
Args:
|
||||
cached_function: The cached function that will be registered to receive invalidation
|
||||
locally and from other workers.
|
||||
@@ -856,6 +858,8 @@ class ModuleApi:
|
||||
"""Invalidate a cache entry of a cached function across workers. The cached function
|
||||
needs to be registered on all workers first with `register_cached_function`.
|
||||
|
||||
Added in Synapse v1.69.0.
|
||||
|
||||
Args:
|
||||
cached_function: The cached function that needs an invalidation
|
||||
keys: keys of the entry to invalidate, usually matching the arguments of the
|
||||
|
||||
@@ -17,6 +17,7 @@ import itertools
|
||||
import logging
|
||||
from typing import (
|
||||
TYPE_CHECKING,
|
||||
Any,
|
||||
Collection,
|
||||
Dict,
|
||||
Iterable,
|
||||
@@ -37,13 +38,11 @@ from synapse.events.snapshot import EventContext
|
||||
from synapse.state import POWER_KEY
|
||||
from synapse.storage.databases.main.roommember import EventIdMembership
|
||||
from synapse.storage.state import StateFilter
|
||||
from synapse.synapse_rust.push import FilteredPushRules, PushRule
|
||||
from synapse.synapse_rust.push import FilteredPushRules, PushRule, PushRuleEvaluator
|
||||
from synapse.util.caches import register_cache
|
||||
from synapse.util.metrics import measure_func
|
||||
from synapse.visibility import filter_event_for_clients_with_state
|
||||
|
||||
from .push_rule_evaluator import PushRuleEvaluatorForEvent
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from synapse.server import HomeServer
|
||||
|
||||
@@ -173,7 +172,11 @@ class BulkPushRuleEvaluator:
|
||||
|
||||
async def _get_power_levels_and_sender_level(
|
||||
self, event: EventBase, context: EventContext
|
||||
) -> Tuple[dict, int]:
|
||||
) -> Tuple[dict, Optional[int]]:
|
||||
# There are no power levels and sender levels possible to get from outlier
|
||||
if event.internal_metadata.is_outlier():
|
||||
return {}, None
|
||||
|
||||
event_types = auth_types_for_event(event.room_version, event)
|
||||
prev_state_ids = await context.get_prev_state_ids(
|
||||
StateFilter.from_types(event_types)
|
||||
@@ -250,8 +253,8 @@ class BulkPushRuleEvaluator:
|
||||
should increment the unread count, and insert the results into the
|
||||
event_push_actions_staging table.
|
||||
"""
|
||||
if event.internal_metadata.is_outlier():
|
||||
# This can happen due to out of band memberships
|
||||
if not event.internal_metadata.is_notifiable():
|
||||
# Push rules for events that aren't notifiable can't be processed by this
|
||||
return
|
||||
|
||||
# Disable counting as unread unless the experimental configuration is
|
||||
@@ -286,11 +289,11 @@ class BulkPushRuleEvaluator:
|
||||
if relation.rel_type == RelationTypes.THREAD:
|
||||
thread_id = relation.parent_id
|
||||
|
||||
evaluator = PushRuleEvaluatorForEvent(
|
||||
event,
|
||||
evaluator = PushRuleEvaluator(
|
||||
_flatten_dict(event),
|
||||
room_member_count,
|
||||
sender_power_level,
|
||||
power_levels,
|
||||
power_levels.get("notifications", {}),
|
||||
relations,
|
||||
self._relations_match_enabled,
|
||||
)
|
||||
@@ -300,20 +303,10 @@ class BulkPushRuleEvaluator:
|
||||
event.room_id, users
|
||||
)
|
||||
|
||||
# This is a check for the case where user joins a room without being
|
||||
# allowed to see history, and then the server receives a delayed event
|
||||
# from before the user joined, which they should not be pushed for
|
||||
uids_with_visibility = await filter_event_for_clients_with_state(
|
||||
self.store, users, event, context
|
||||
)
|
||||
|
||||
for uid, rules in rules_by_user.items():
|
||||
if event.sender == uid:
|
||||
continue
|
||||
|
||||
if uid not in uids_with_visibility:
|
||||
continue
|
||||
|
||||
display_name = None
|
||||
profile = profiles.get(uid)
|
||||
if profile:
|
||||
@@ -334,17 +327,30 @@ class BulkPushRuleEvaluator:
|
||||
# current user, it'll be added to the dict later.
|
||||
actions_by_user[uid] = []
|
||||
|
||||
for rule, enabled in rules.rules():
|
||||
if not enabled:
|
||||
continue
|
||||
actions = evaluator.run(rules, uid, display_name)
|
||||
if "notify" in actions:
|
||||
# Push rules say we should notify the user of this event
|
||||
actions_by_user[uid] = actions
|
||||
|
||||
matches = evaluator.check_conditions(rule.conditions, uid, display_name)
|
||||
if matches:
|
||||
actions = [x for x in rule.actions if x != "dont_notify"]
|
||||
if actions and "notify" in actions:
|
||||
# Push rules say we should notify the user of this event
|
||||
actions_by_user[uid] = actions
|
||||
break
|
||||
# If there aren't any actions then we can skip the rest of the
|
||||
# processing.
|
||||
if not actions_by_user:
|
||||
return
|
||||
|
||||
# This is a check for the case where user joins a room without being
|
||||
# allowed to see history, and then the server receives a delayed event
|
||||
# from before the user joined, which they should not be pushed for
|
||||
#
|
||||
# We do this *after* calculating the push actions as a) its unlikely
|
||||
# that we'll filter anyone out and b) for large rooms its likely that
|
||||
# most users will have push disabled and so the set of users to check is
|
||||
# much smaller.
|
||||
uids_with_visibility = await filter_event_for_clients_with_state(
|
||||
self.store, actions_by_user.keys(), event, context
|
||||
)
|
||||
|
||||
for user_id in set(actions_by_user).difference(uids_with_visibility):
|
||||
actions_by_user.pop(user_id, None)
|
||||
|
||||
# Mark in the DB staging area the push actions for users who should be
|
||||
# notified for this event. (This will then get handled when we persist
|
||||
@@ -361,3 +367,21 @@ MemberMap = Dict[str, Optional[EventIdMembership]]
|
||||
Rule = Dict[str, dict]
|
||||
RulesByUser = Dict[str, List[Rule]]
|
||||
StateGroup = Union[object, int]
|
||||
|
||||
|
||||
def _flatten_dict(
|
||||
d: Union[EventBase, Mapping[str, Any]],
|
||||
prefix: Optional[List[str]] = None,
|
||||
result: Optional[Dict[str, str]] = None,
|
||||
) -> Dict[str, str]:
|
||||
if prefix is None:
|
||||
prefix = []
|
||||
if result is None:
|
||||
result = {}
|
||||
for key, value in d.items():
|
||||
if isinstance(value, str):
|
||||
result[".".join(prefix + [key])] = value.lower()
|
||||
elif isinstance(value, Mapping):
|
||||
_flatten_dict(value, prefix=(prefix + [key]), result=result)
|
||||
|
||||
return result
|
||||
|
||||
@@ -102,10 +102,8 @@ def _rule_to_template(rule: PushRule) -> Optional[Dict[str, Any]]:
|
||||
# with PRIORITY_CLASS_INVERSE_MAP.
|
||||
raise ValueError("Unexpected template_name: %s" % (template_name,))
|
||||
|
||||
if unscoped_rule_id:
|
||||
templaterule["rule_id"] = unscoped_rule_id
|
||||
if rule.default:
|
||||
templaterule["default"] = True
|
||||
templaterule["rule_id"] = unscoped_rule_id
|
||||
templaterule["default"] = rule.default
|
||||
return templaterule
|
||||
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@
|
||||
# limitations under the License.
|
||||
import logging
|
||||
import urllib.parse
|
||||
from typing import TYPE_CHECKING, Any, Dict, Iterable, Optional, Union
|
||||
from typing import TYPE_CHECKING, Any, Dict, Iterable, List, Optional, Union
|
||||
|
||||
from prometheus_client import Counter
|
||||
|
||||
@@ -28,7 +28,7 @@ from synapse.metrics.background_process_metrics import run_as_background_process
|
||||
from synapse.push import Pusher, PusherConfig, PusherConfigException
|
||||
from synapse.storage.databases.main.event_push_actions import HttpPushAction
|
||||
|
||||
from . import push_rule_evaluator, push_tools
|
||||
from . import push_tools
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from synapse.server import HomeServer
|
||||
@@ -56,6 +56,39 @@ http_badges_failed_counter = Counter(
|
||||
)
|
||||
|
||||
|
||||
def tweaks_for_actions(actions: List[Union[str, Dict]]) -> Dict[str, Any]:
|
||||
"""
|
||||
Converts a list of actions into a `tweaks` dict (which can then be passed to
|
||||
the push gateway).
|
||||
|
||||
This function ignores all actions other than `set_tweak` actions, and treats
|
||||
absent `value`s as `True`, which agrees with the only spec-defined treatment
|
||||
of absent `value`s (namely, for `highlight` tweaks).
|
||||
|
||||
Args:
|
||||
actions: list of actions
|
||||
e.g. [
|
||||
{"set_tweak": "a", "value": "AAA"},
|
||||
{"set_tweak": "b", "value": "BBB"},
|
||||
{"set_tweak": "highlight"},
|
||||
"notify"
|
||||
]
|
||||
|
||||
Returns:
|
||||
dictionary of tweaks for those actions
|
||||
e.g. {"a": "AAA", "b": "BBB", "highlight": True}
|
||||
"""
|
||||
tweaks = {}
|
||||
for a in actions:
|
||||
if not isinstance(a, dict):
|
||||
continue
|
||||
if "set_tweak" in a:
|
||||
# value is allowed to be absent in which case the value assumed
|
||||
# should be True.
|
||||
tweaks[a["set_tweak"]] = a.get("value", True)
|
||||
return tweaks
|
||||
|
||||
|
||||
class HttpPusher(Pusher):
|
||||
INITIAL_BACKOFF_SEC = 1 # in seconds because that's what Twisted takes
|
||||
MAX_BACKOFF_SEC = 60 * 60
|
||||
@@ -281,7 +314,7 @@ class HttpPusher(Pusher):
|
||||
if "notify" not in push_action.actions:
|
||||
return True
|
||||
|
||||
tweaks = push_rule_evaluator.tweaks_for_actions(push_action.actions)
|
||||
tweaks = tweaks_for_actions(push_action.actions)
|
||||
badge = await push_tools.get_badge_count(
|
||||
self.hs.get_datastores().main,
|
||||
self.user_id,
|
||||
|
||||
@@ -1,361 +0,0 @@
|
||||
# Copyright 2015, 2016 OpenMarket Ltd
|
||||
# Copyright 2017 New Vector Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
import logging
|
||||
import re
|
||||
from typing import (
|
||||
Any,
|
||||
Dict,
|
||||
List,
|
||||
Mapping,
|
||||
Optional,
|
||||
Pattern,
|
||||
Sequence,
|
||||
Set,
|
||||
Tuple,
|
||||
Union,
|
||||
)
|
||||
|
||||
from matrix_common.regex import glob_to_regex, to_word_pattern
|
||||
|
||||
from synapse.events import EventBase
|
||||
from synapse.types import UserID
|
||||
from synapse.util.caches.lrucache import LruCache
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
GLOB_REGEX = re.compile(r"\\\[(\\\!|)(.*)\\\]")
|
||||
IS_GLOB = re.compile(r"[\?\*\[\]]")
|
||||
INEQUALITY_EXPR = re.compile("^([=<>]*)([0-9]*)$")
|
||||
|
||||
|
||||
def _room_member_count(
|
||||
ev: EventBase, condition: Mapping[str, Any], room_member_count: int
|
||||
) -> bool:
|
||||
return _test_ineq_condition(condition, room_member_count)
|
||||
|
||||
|
||||
def _sender_notification_permission(
|
||||
ev: EventBase,
|
||||
condition: Mapping[str, Any],
|
||||
sender_power_level: int,
|
||||
power_levels: Dict[str, Union[int, Dict[str, int]]],
|
||||
) -> bool:
|
||||
notif_level_key = condition.get("key")
|
||||
if notif_level_key is None:
|
||||
return False
|
||||
|
||||
notif_levels = power_levels.get("notifications", {})
|
||||
assert isinstance(notif_levels, dict)
|
||||
room_notif_level = notif_levels.get(notif_level_key, 50)
|
||||
|
||||
return sender_power_level >= room_notif_level
|
||||
|
||||
|
||||
def _test_ineq_condition(condition: Mapping[str, Any], number: int) -> bool:
|
||||
if "is" not in condition:
|
||||
return False
|
||||
m = INEQUALITY_EXPR.match(condition["is"])
|
||||
if not m:
|
||||
return False
|
||||
ineq = m.group(1)
|
||||
rhs = m.group(2)
|
||||
if not rhs.isdigit():
|
||||
return False
|
||||
rhs_int = int(rhs)
|
||||
|
||||
if ineq == "" or ineq == "==":
|
||||
return number == rhs_int
|
||||
elif ineq == "<":
|
||||
return number < rhs_int
|
||||
elif ineq == ">":
|
||||
return number > rhs_int
|
||||
elif ineq == ">=":
|
||||
return number >= rhs_int
|
||||
elif ineq == "<=":
|
||||
return number <= rhs_int
|
||||
else:
|
||||
return False
|
||||
|
||||
|
||||
def tweaks_for_actions(actions: List[Union[str, Dict]]) -> Dict[str, Any]:
|
||||
"""
|
||||
Converts a list of actions into a `tweaks` dict (which can then be passed to
|
||||
the push gateway).
|
||||
|
||||
This function ignores all actions other than `set_tweak` actions, and treats
|
||||
absent `value`s as `True`, which agrees with the only spec-defined treatment
|
||||
of absent `value`s (namely, for `highlight` tweaks).
|
||||
|
||||
Args:
|
||||
actions: list of actions
|
||||
e.g. [
|
||||
{"set_tweak": "a", "value": "AAA"},
|
||||
{"set_tweak": "b", "value": "BBB"},
|
||||
{"set_tweak": "highlight"},
|
||||
"notify"
|
||||
]
|
||||
|
||||
Returns:
|
||||
dictionary of tweaks for those actions
|
||||
e.g. {"a": "AAA", "b": "BBB", "highlight": True}
|
||||
"""
|
||||
tweaks = {}
|
||||
for a in actions:
|
||||
if not isinstance(a, dict):
|
||||
continue
|
||||
if "set_tweak" in a:
|
||||
# value is allowed to be absent in which case the value assumed
|
||||
# should be True.
|
||||
tweaks[a["set_tweak"]] = a.get("value", True)
|
||||
return tweaks
|
||||
|
||||
|
||||
class PushRuleEvaluatorForEvent:
|
||||
def __init__(
|
||||
self,
|
||||
event: EventBase,
|
||||
room_member_count: int,
|
||||
sender_power_level: int,
|
||||
power_levels: Dict[str, Union[int, Dict[str, int]]],
|
||||
relations: Dict[str, Set[Tuple[str, str]]],
|
||||
relations_match_enabled: bool,
|
||||
):
|
||||
self._event = event
|
||||
self._room_member_count = room_member_count
|
||||
self._sender_power_level = sender_power_level
|
||||
self._power_levels = power_levels
|
||||
self._relations = relations
|
||||
self._relations_match_enabled = relations_match_enabled
|
||||
|
||||
# Maps strings of e.g. 'content.body' -> event["content"]["body"]
|
||||
self._value_cache = _flatten_dict(event)
|
||||
|
||||
# Maps cache keys to final values.
|
||||
self._condition_cache: Dict[str, bool] = {}
|
||||
|
||||
def check_conditions(
|
||||
self, conditions: Sequence[Mapping], uid: str, display_name: Optional[str]
|
||||
) -> bool:
|
||||
"""
|
||||
Returns true if a user's conditions/user ID/display name match the event.
|
||||
|
||||
Args:
|
||||
conditions: The user's conditions to match.
|
||||
uid: The user's MXID.
|
||||
display_name: The display name.
|
||||
|
||||
Returns:
|
||||
True if all conditions match the event, False otherwise.
|
||||
"""
|
||||
for cond in conditions:
|
||||
_cache_key = cond.get("_cache_key", None)
|
||||
if _cache_key:
|
||||
res = self._condition_cache.get(_cache_key, None)
|
||||
if res is False:
|
||||
return False
|
||||
elif res is True:
|
||||
continue
|
||||
|
||||
res = self.matches(cond, uid, display_name)
|
||||
if _cache_key:
|
||||
self._condition_cache[_cache_key] = bool(res)
|
||||
|
||||
if not res:
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def matches(
|
||||
self, condition: Mapping[str, Any], user_id: str, display_name: Optional[str]
|
||||
) -> bool:
|
||||
"""
|
||||
Returns true if a user's condition/user ID/display name match the event.
|
||||
|
||||
Args:
|
||||
condition: The user's condition to match.
|
||||
uid: The user's MXID.
|
||||
display_name: The display name, or None if there is not one.
|
||||
|
||||
Returns:
|
||||
True if the condition matches the event, False otherwise.
|
||||
"""
|
||||
if condition["kind"] == "event_match":
|
||||
return self._event_match(condition, user_id)
|
||||
elif condition["kind"] == "contains_display_name":
|
||||
return self._contains_display_name(display_name)
|
||||
elif condition["kind"] == "room_member_count":
|
||||
return _room_member_count(self._event, condition, self._room_member_count)
|
||||
elif condition["kind"] == "sender_notification_permission":
|
||||
return _sender_notification_permission(
|
||||
self._event, condition, self._sender_power_level, self._power_levels
|
||||
)
|
||||
elif (
|
||||
condition["kind"] == "org.matrix.msc3772.relation_match"
|
||||
and self._relations_match_enabled
|
||||
):
|
||||
return self._relation_match(condition, user_id)
|
||||
else:
|
||||
# XXX This looks incorrect -- we have reached an unknown condition
|
||||
# kind and are unconditionally returning that it matches. Note
|
||||
# that it seems possible to provide a condition to the /pushrules
|
||||
# endpoint with an unknown kind, see _rule_tuple_from_request_object.
|
||||
return True
|
||||
|
||||
def _event_match(self, condition: Mapping, user_id: str) -> bool:
|
||||
"""
|
||||
Check an "event_match" push rule condition.
|
||||
|
||||
Args:
|
||||
condition: The "event_match" push rule condition to match.
|
||||
user_id: The user's MXID.
|
||||
|
||||
Returns:
|
||||
True if the condition matches the event, False otherwise.
|
||||
"""
|
||||
pattern = condition.get("pattern", None)
|
||||
|
||||
if not pattern:
|
||||
pattern_type = condition.get("pattern_type", None)
|
||||
if pattern_type == "user_id":
|
||||
pattern = user_id
|
||||
elif pattern_type == "user_localpart":
|
||||
pattern = UserID.from_string(user_id).localpart
|
||||
|
||||
if not pattern:
|
||||
logger.warning("event_match condition with no pattern")
|
||||
return False
|
||||
|
||||
# XXX: optimisation: cache our pattern regexps
|
||||
if condition["key"] == "content.body":
|
||||
body = self._event.content.get("body", None)
|
||||
if not body or not isinstance(body, str):
|
||||
return False
|
||||
|
||||
return _glob_matches(pattern, body, word_boundary=True)
|
||||
else:
|
||||
haystack = self._value_cache.get(condition["key"], None)
|
||||
if haystack is None:
|
||||
return False
|
||||
|
||||
return _glob_matches(pattern, haystack)
|
||||
|
||||
def _contains_display_name(self, display_name: Optional[str]) -> bool:
|
||||
"""
|
||||
Check an "event_match" push rule condition.
|
||||
|
||||
Args:
|
||||
display_name: The display name, or None if there is not one.
|
||||
|
||||
Returns:
|
||||
True if the display name is found in the event body, False otherwise.
|
||||
"""
|
||||
if not display_name:
|
||||
return False
|
||||
|
||||
body = self._event.content.get("body", None)
|
||||
if not body or not isinstance(body, str):
|
||||
return False
|
||||
|
||||
# Similar to _glob_matches, but do not treat display_name as a glob.
|
||||
r = regex_cache.get((display_name, False, True), None)
|
||||
if not r:
|
||||
r1 = re.escape(display_name)
|
||||
r1 = to_word_pattern(r1)
|
||||
r = re.compile(r1, flags=re.IGNORECASE)
|
||||
regex_cache[(display_name, False, True)] = r
|
||||
|
||||
return bool(r.search(body))
|
||||
|
||||
def _relation_match(self, condition: Mapping, user_id: str) -> bool:
|
||||
"""
|
||||
Check an "relation_match" push rule condition.
|
||||
|
||||
Args:
|
||||
condition: The "event_match" push rule condition to match.
|
||||
user_id: The user's MXID.
|
||||
|
||||
Returns:
|
||||
True if the condition matches the event, False otherwise.
|
||||
"""
|
||||
rel_type = condition.get("rel_type")
|
||||
if not rel_type:
|
||||
logger.warning("relation_match condition missing rel_type")
|
||||
return False
|
||||
|
||||
sender_pattern = condition.get("sender")
|
||||
if sender_pattern is None:
|
||||
sender_type = condition.get("sender_type")
|
||||
if sender_type == "user_id":
|
||||
sender_pattern = user_id
|
||||
type_pattern = condition.get("type")
|
||||
|
||||
# If any other relations matches, return True.
|
||||
for sender, event_type in self._relations.get(rel_type, ()):
|
||||
if sender_pattern and not _glob_matches(sender_pattern, sender):
|
||||
continue
|
||||
if type_pattern and not _glob_matches(type_pattern, event_type):
|
||||
continue
|
||||
# All values must have matched.
|
||||
return True
|
||||
|
||||
# No relations matched.
|
||||
return False
|
||||
|
||||
|
||||
# Caches (string, is_glob, word_boundary) -> regex for push. See _glob_matches
|
||||
regex_cache: LruCache[Tuple[str, bool, bool], Pattern] = LruCache(
|
||||
50000, "regex_push_cache"
|
||||
)
|
||||
|
||||
|
||||
def _glob_matches(glob: str, value: str, word_boundary: bool = False) -> bool:
|
||||
"""Tests if value matches glob.
|
||||
|
||||
Args:
|
||||
glob
|
||||
value: String to test against glob.
|
||||
word_boundary: Whether to match against word boundaries or entire
|
||||
string. Defaults to False.
|
||||
"""
|
||||
|
||||
try:
|
||||
r = regex_cache.get((glob, True, word_boundary), None)
|
||||
if not r:
|
||||
r = glob_to_regex(glob, word_boundary=word_boundary)
|
||||
regex_cache[(glob, True, word_boundary)] = r
|
||||
return bool(r.search(value))
|
||||
except re.error:
|
||||
logger.warning("Failed to parse glob to regex: %r", glob)
|
||||
return False
|
||||
|
||||
|
||||
def _flatten_dict(
|
||||
d: Union[EventBase, Mapping[str, Any]],
|
||||
prefix: Optional[List[str]] = None,
|
||||
result: Optional[Dict[str, str]] = None,
|
||||
) -> Dict[str, str]:
|
||||
if prefix is None:
|
||||
prefix = []
|
||||
if result is None:
|
||||
result = {}
|
||||
for key, value in d.items():
|
||||
if isinstance(value, str):
|
||||
result[".".join(prefix + [key])] = value.lower()
|
||||
elif isinstance(value, Mapping):
|
||||
_flatten_dict(value, prefix=(prefix + [key]), result=result)
|
||||
|
||||
return result
|
||||
@@ -25,6 +25,7 @@ from synapse.replication.http import (
|
||||
push,
|
||||
register,
|
||||
send_event,
|
||||
send_events,
|
||||
state,
|
||||
streams,
|
||||
)
|
||||
@@ -43,6 +44,7 @@ class ReplicationRestResource(JsonResource):
|
||||
|
||||
def register_servlets(self, hs: "HomeServer") -> None:
|
||||
send_event.register_servlets(hs, self)
|
||||
send_events.register_servlets(hs, self)
|
||||
federation.register_servlets(hs, self)
|
||||
presence.register_servlets(hs, self)
|
||||
membership.register_servlets(hs, self)
|
||||
|
||||
@@ -51,6 +51,7 @@ class ReplicationRegisterServlet(ReplicationEndpoint):
|
||||
user_type: Optional[str],
|
||||
address: Optional[str],
|
||||
shadow_banned: bool,
|
||||
approved: bool,
|
||||
) -> JsonDict:
|
||||
"""
|
||||
Args:
|
||||
@@ -68,6 +69,8 @@ class ReplicationRegisterServlet(ReplicationEndpoint):
|
||||
or None for a normal user.
|
||||
address: the IP address used to perform the regitration.
|
||||
shadow_banned: Whether to shadow-ban the user
|
||||
approved: Whether the user should be considered already approved by an
|
||||
administrator.
|
||||
"""
|
||||
return {
|
||||
"password_hash": password_hash,
|
||||
@@ -79,6 +82,7 @@ class ReplicationRegisterServlet(ReplicationEndpoint):
|
||||
"user_type": user_type,
|
||||
"address": address,
|
||||
"shadow_banned": shadow_banned,
|
||||
"approved": approved,
|
||||
}
|
||||
|
||||
async def _handle_request( # type: ignore[override]
|
||||
@@ -99,6 +103,7 @@ class ReplicationRegisterServlet(ReplicationEndpoint):
|
||||
user_type=content["user_type"],
|
||||
address=content["address"],
|
||||
shadow_banned=content["shadow_banned"],
|
||||
approved=content["approved"],
|
||||
)
|
||||
|
||||
return 200, {}
|
||||
|
||||
@@ -141,8 +141,8 @@ class ReplicationSendEventRestServlet(ReplicationEndpoint):
|
||||
"Got event to send with ID: %s into room: %s", event.event_id, event.room_id
|
||||
)
|
||||
|
||||
event = await self.event_creation_handler.persist_and_notify_client_event(
|
||||
requester, event, context, ratelimit=ratelimit, extra_users=extra_users
|
||||
event = await self.event_creation_handler.persist_and_notify_client_events(
|
||||
requester, [(event, context)], ratelimit=ratelimit, extra_users=extra_users
|
||||
)
|
||||
|
||||
return (
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
# Copyright 2022 The Matrix.org Foundation C.I.C.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
import logging
|
||||
from typing import TYPE_CHECKING, List, Tuple
|
||||
|
||||
from twisted.web.server import Request
|
||||
|
||||
from synapse.api.room_versions import KNOWN_ROOM_VERSIONS
|
||||
from synapse.events import EventBase, make_event_from_dict
|
||||
from synapse.events.snapshot import EventContext
|
||||
from synapse.http.server import HttpServer
|
||||
from synapse.http.servlet import parse_json_object_from_request
|
||||
from synapse.replication.http._base import ReplicationEndpoint
|
||||
from synapse.types import JsonDict, Requester, UserID
|
||||
from synapse.util.metrics import Measure
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from synapse.server import HomeServer
|
||||
from synapse.storage.databases.main import DataStore
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ReplicationSendEventsRestServlet(ReplicationEndpoint):
|
||||
"""Handles batches of newly created events on workers, including persisting and
|
||||
notifying.
|
||||
|
||||
The API looks like:
|
||||
|
||||
POST /_synapse/replication/send_events/:txn_id
|
||||
|
||||
{
|
||||
"events": [{
|
||||
"event": { .. serialized event .. },
|
||||
"room_version": .., // "1", "2", "3", etc: the version of the room
|
||||
// containing the event
|
||||
"event_format_version": .., // 1,2,3 etc: the event format version
|
||||
"internal_metadata": { .. serialized internal_metadata .. },
|
||||
"outlier": true|false,
|
||||
"rejected_reason": .., // The event.rejected_reason field
|
||||
"context": { .. serialized event context .. },
|
||||
"requester": { .. serialized requester .. },
|
||||
"ratelimit": true,
|
||||
}]
|
||||
}
|
||||
|
||||
200 OK
|
||||
|
||||
{ "stream_id": 12345, "event_id": "$abcdef..." }
|
||||
|
||||
Responds with a 409 when a `PartialStateConflictError` is raised due to an event
|
||||
context that needs to be recomputed due to the un-partial stating of a room.
|
||||
|
||||
"""
|
||||
|
||||
NAME = "send_events"
|
||||
PATH_ARGS = ()
|
||||
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
super().__init__(hs)
|
||||
|
||||
self.event_creation_handler = hs.get_event_creation_handler()
|
||||
self.store = hs.get_datastores().main
|
||||
self._storage_controllers = hs.get_storage_controllers()
|
||||
self.clock = hs.get_clock()
|
||||
|
||||
@staticmethod
|
||||
async def _serialize_payload( # type: ignore[override]
|
||||
events_and_context: List[Tuple[EventBase, EventContext]],
|
||||
store: "DataStore",
|
||||
requester: Requester,
|
||||
ratelimit: bool,
|
||||
extra_users: List[UserID],
|
||||
) -> JsonDict:
|
||||
"""
|
||||
Args:
|
||||
store
|
||||
requester
|
||||
events_and_ctx
|
||||
ratelimit
|
||||
"""
|
||||
serialized_events = []
|
||||
|
||||
for event, context in events_and_context:
|
||||
serialized_context = await context.serialize(event, store)
|
||||
serialized_event = {
|
||||
"event": event.get_pdu_json(),
|
||||
"room_version": event.room_version.identifier,
|
||||
"event_format_version": event.format_version,
|
||||
"internal_metadata": event.internal_metadata.get_dict(),
|
||||
"outlier": event.internal_metadata.is_outlier(),
|
||||
"rejected_reason": event.rejected_reason,
|
||||
"context": serialized_context,
|
||||
"requester": requester.serialize(),
|
||||
"ratelimit": ratelimit,
|
||||
"extra_users": [u.to_string() for u in extra_users],
|
||||
}
|
||||
serialized_events.append(serialized_event)
|
||||
|
||||
payload = {"events": serialized_events}
|
||||
|
||||
return payload
|
||||
|
||||
async def _handle_request( # type: ignore[override]
|
||||
self, request: Request
|
||||
) -> Tuple[int, JsonDict]:
|
||||
with Measure(self.clock, "repl_send_events_parse"):
|
||||
payload = parse_json_object_from_request(request)
|
||||
events_and_context = []
|
||||
events = payload["events"]
|
||||
|
||||
for event_payload in events:
|
||||
event_dict = event_payload["event"]
|
||||
room_ver = KNOWN_ROOM_VERSIONS[event_payload["room_version"]]
|
||||
internal_metadata = event_payload["internal_metadata"]
|
||||
rejected_reason = event_payload["rejected_reason"]
|
||||
|
||||
event = make_event_from_dict(
|
||||
event_dict, room_ver, internal_metadata, rejected_reason
|
||||
)
|
||||
event.internal_metadata.outlier = event_payload["outlier"]
|
||||
|
||||
requester = Requester.deserialize(
|
||||
self.store, event_payload["requester"]
|
||||
)
|
||||
context = EventContext.deserialize(
|
||||
self._storage_controllers, event_payload["context"]
|
||||
)
|
||||
|
||||
ratelimit = event_payload["ratelimit"]
|
||||
events_and_context.append((event, context))
|
||||
|
||||
extra_users = [
|
||||
UserID.from_string(u) for u in event_payload["extra_users"]
|
||||
]
|
||||
|
||||
logger.info(
|
||||
"Got batch of events to send, last ID of batch is: %s, sending into room: %s",
|
||||
event.event_id,
|
||||
event.room_id,
|
||||
)
|
||||
|
||||
last_event = (
|
||||
await self.event_creation_handler.persist_and_notify_client_events(
|
||||
requester, events_and_context, ratelimit, extra_users
|
||||
)
|
||||
)
|
||||
|
||||
return (
|
||||
200,
|
||||
{
|
||||
"stream_id": last_event.internal_metadata.stream_ordering,
|
||||
"event_id": last_event.event_id,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def register_servlets(hs: "HomeServer", http_server: HttpServer) -> None:
|
||||
ReplicationSendEventsRestServlet(hs).register(http_server)
|
||||
@@ -69,6 +69,7 @@ class UsersRestServletV2(RestServlet):
|
||||
self.store = hs.get_datastores().main
|
||||
self.auth = hs.get_auth()
|
||||
self.admin_handler = hs.get_admin_handler()
|
||||
self._msc3866_enabled = hs.config.experimental.msc3866.enabled
|
||||
|
||||
async def on_GET(self, request: SynapseRequest) -> Tuple[int, JsonDict]:
|
||||
await assert_requester_is_admin(self.auth, request)
|
||||
@@ -95,6 +96,13 @@ class UsersRestServletV2(RestServlet):
|
||||
guests = parse_boolean(request, "guests", default=True)
|
||||
deactivated = parse_boolean(request, "deactivated", default=False)
|
||||
|
||||
# If support for MSC3866 is not enabled, apply no filtering based on the
|
||||
# `approved` column.
|
||||
if self._msc3866_enabled:
|
||||
approved = parse_boolean(request, "approved", default=True)
|
||||
else:
|
||||
approved = True
|
||||
|
||||
order_by = parse_string(
|
||||
request,
|
||||
"order_by",
|
||||
@@ -115,8 +123,22 @@ class UsersRestServletV2(RestServlet):
|
||||
direction = parse_string(request, "dir", default="f", allowed_values=("f", "b"))
|
||||
|
||||
users, total = await self.store.get_users_paginate(
|
||||
start, limit, user_id, name, guests, deactivated, order_by, direction
|
||||
start,
|
||||
limit,
|
||||
user_id,
|
||||
name,
|
||||
guests,
|
||||
deactivated,
|
||||
order_by,
|
||||
direction,
|
||||
approved,
|
||||
)
|
||||
|
||||
# If support for MSC3866 is not enabled, don't show the approval flag.
|
||||
if not self._msc3866_enabled:
|
||||
for user in users:
|
||||
del user["approved"]
|
||||
|
||||
ret = {"users": users, "total": total}
|
||||
if (start + limit) < total:
|
||||
ret["next_token"] = str(start + len(users))
|
||||
@@ -163,6 +185,7 @@ class UserRestServletV2(RestServlet):
|
||||
self.deactivate_account_handler = hs.get_deactivate_account_handler()
|
||||
self.registration_handler = hs.get_registration_handler()
|
||||
self.pusher_pool = hs.get_pusherpool()
|
||||
self._msc3866_enabled = hs.config.experimental.msc3866.enabled
|
||||
|
||||
async def on_GET(
|
||||
self, request: SynapseRequest, user_id: str
|
||||
@@ -239,6 +262,15 @@ class UserRestServletV2(RestServlet):
|
||||
HTTPStatus.BAD_REQUEST, "'deactivated' parameter is not of type boolean"
|
||||
)
|
||||
|
||||
approved: Optional[bool] = None
|
||||
if "approved" in body and self._msc3866_enabled:
|
||||
approved = body["approved"]
|
||||
if not isinstance(approved, bool):
|
||||
raise SynapseError(
|
||||
HTTPStatus.BAD_REQUEST,
|
||||
"'approved' parameter is not of type boolean",
|
||||
)
|
||||
|
||||
# convert List[Dict[str, str]] into List[Tuple[str, str]]
|
||||
if external_ids is not None:
|
||||
new_external_ids = [
|
||||
@@ -343,6 +375,9 @@ class UserRestServletV2(RestServlet):
|
||||
if "user_type" in body:
|
||||
await self.store.set_user_type(target_user, user_type)
|
||||
|
||||
if approved is not None:
|
||||
await self.store.update_user_approval_status(target_user, approved)
|
||||
|
||||
user = await self.admin_handler.get_user(target_user)
|
||||
assert user is not None
|
||||
|
||||
@@ -355,6 +390,10 @@ class UserRestServletV2(RestServlet):
|
||||
if password is not None:
|
||||
password_hash = await self.auth_handler.hash(password)
|
||||
|
||||
new_user_approved = True
|
||||
if self._msc3866_enabled and approved is not None:
|
||||
new_user_approved = approved
|
||||
|
||||
user_id = await self.registration_handler.register_user(
|
||||
localpart=target_user.localpart,
|
||||
password_hash=password_hash,
|
||||
@@ -362,6 +401,7 @@ class UserRestServletV2(RestServlet):
|
||||
default_display_name=displayname,
|
||||
user_type=user_type,
|
||||
by_admin=True,
|
||||
approved=new_user_approved,
|
||||
)
|
||||
|
||||
if threepids is not None:
|
||||
@@ -550,6 +590,7 @@ class UserRegisterServlet(RestServlet):
|
||||
user_type=user_type,
|
||||
default_display_name=displayname,
|
||||
by_admin=True,
|
||||
approved=True,
|
||||
)
|
||||
|
||||
result = await register._create_registration_details(user_id, body)
|
||||
|
||||
@@ -28,7 +28,14 @@ from typing import (
|
||||
|
||||
from typing_extensions import TypedDict
|
||||
|
||||
from synapse.api.errors import Codes, InvalidClientTokenError, LoginError, SynapseError
|
||||
from synapse.api.constants import ApprovalNoticeMedium
|
||||
from synapse.api.errors import (
|
||||
Codes,
|
||||
InvalidClientTokenError,
|
||||
LoginError,
|
||||
NotApprovedError,
|
||||
SynapseError,
|
||||
)
|
||||
from synapse.api.ratelimiting import Ratelimiter
|
||||
from synapse.api.urls import CLIENT_API_PREFIX
|
||||
from synapse.appservice import ApplicationService
|
||||
@@ -55,11 +62,11 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
class LoginResponse(TypedDict, total=False):
|
||||
user_id: str
|
||||
access_token: str
|
||||
access_token: Optional[str]
|
||||
home_server: str
|
||||
expires_in_ms: Optional[int]
|
||||
refresh_token: Optional[str]
|
||||
device_id: str
|
||||
device_id: Optional[str]
|
||||
well_known: Optional[Dict[str, Any]]
|
||||
|
||||
|
||||
@@ -92,6 +99,12 @@ class LoginRestServlet(RestServlet):
|
||||
hs.config.registration.refreshable_access_token_lifetime is not None
|
||||
)
|
||||
|
||||
# Whether we need to check if the user has been approved or not.
|
||||
self._require_approval = (
|
||||
hs.config.experimental.msc3866.enabled
|
||||
and hs.config.experimental.msc3866.require_approval_for_new_accounts
|
||||
)
|
||||
|
||||
self.auth = hs.get_auth()
|
||||
|
||||
self.clock = hs.get_clock()
|
||||
@@ -220,6 +233,14 @@ class LoginRestServlet(RestServlet):
|
||||
except KeyError:
|
||||
raise SynapseError(400, "Missing JSON keys.")
|
||||
|
||||
if self._require_approval:
|
||||
approved = await self.auth_handler.is_user_approved(result["user_id"])
|
||||
if not approved:
|
||||
raise NotApprovedError(
|
||||
msg="This account is pending approval by a server administrator.",
|
||||
approval_notice_medium=ApprovalNoticeMedium.NONE,
|
||||
)
|
||||
|
||||
well_known_data = self._well_known_builder.get_well_known()
|
||||
if well_known_data:
|
||||
result["well_known"] = well_known_data
|
||||
@@ -356,6 +377,16 @@ class LoginRestServlet(RestServlet):
|
||||
errcode=Codes.INVALID_PARAM,
|
||||
)
|
||||
|
||||
if self._require_approval:
|
||||
approved = await self.auth_handler.is_user_approved(user_id)
|
||||
if not approved:
|
||||
# If the user isn't approved (and needs to be) we won't allow them to
|
||||
# actually log in, so we don't want to create a device/access token.
|
||||
return LoginResponse(
|
||||
user_id=user_id,
|
||||
home_server=self.hs.hostname,
|
||||
)
|
||||
|
||||
initial_display_name = login_submission.get("initial_device_display_name")
|
||||
(
|
||||
device_id,
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user