Fix sending receipts

.git-blame-ignore-revs

@@ -6,6 +6,3 @@ aff1eb7c671b0a3813407321d2702ec46c71fa56
 
 # Update black to 20.8b1 (#9381).
 0a00b7ff14890987f09112a2ae696c61001e6cf1
-
-# Convert tests/rest/admin/test_room.py to unix file endings (#7953).
-c4268e3da64f1abb5b31deaeb5769adb6510c0a7

CHANGES.md

@@ -1,145 +1,3 @@
-Synapse 1.60.0 (2022-05-31)
-===========================
-
-This release of Synapse adds a unique index to the `state_group_edges` table, in
-order to prevent accidentally introducing duplicate information (for example,
-because a database backup was restored multiple times). If your Synapse database
-already has duplicate rows in this table, this could fail with an error and
-require manual remediation.
-
-Additionally, the signature of the `check_event_for_spam` module callback has changed.
-The previous signature has been deprecated and remains working for now. Module authors
-should update their modules to use the new signature where possible.
-
-See [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1600)
-for more details.
-
-Bugfixes
---------
-
-- Fix a bug introduced in Synapse 1.60.0rc1 that would break some imports from `synapse.module_api`. ([\#12918](https://github.com/matrix-org/synapse/issues/12918))
-
-
-Synapse 1.60.0rc2 (2022-05-27)
-==============================
-
-Features
---------
-
-- Add an option allowing users to use their password to reauthenticate for privileged actions even though password login is disabled. ([\#12883](https://github.com/matrix-org/synapse/issues/12883))
-
-
-Bugfixes
---------
-
-- Explicitly close `ijson` coroutines once we are done with them, instead of leaving the garbage collector to close them. ([\#12875](https://github.com/matrix-org/synapse/issues/12875))
-
-
-Internal Changes
-----------------
-
-- Improve URL previews by not including the content of media tags in the generated description. ([\#12887](https://github.com/matrix-org/synapse/issues/12887))
-
-
-Synapse 1.60.0rc1 (2022-05-24)
-==============================
-
-Features
---------
-
-- Measure the time taken in spam-checking callbacks and expose those measurements as metrics. ([\#12513](https://github.com/matrix-org/synapse/issues/12513))
-- Add a `default_power_level_content_override` config option to set default room power levels per room preset. ([\#12618](https://github.com/matrix-org/synapse/issues/12618))
-- Add support for [MSC3787: Allowing knocks to restricted rooms](https://github.com/matrix-org/matrix-spec-proposals/pull/3787). ([\#12623](https://github.com/matrix-org/synapse/issues/12623))
-- Send `USER_IP` commands on a different Redis channel, in order to reduce traffic to workers that do not process these commands. ([\#12672](https://github.com/matrix-org/synapse/issues/12672), [\#12809](https://github.com/matrix-org/synapse/issues/12809))
-- Synapse will now reload [cache config](https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html#caching) when it receives a [SIGHUP](https://en.wikipedia.org/wiki/SIGHUP) signal. ([\#12673](https://github.com/matrix-org/synapse/issues/12673))
-- Add a config options to allow for auto-tuning of caches. ([\#12701](https://github.com/matrix-org/synapse/issues/12701))
-- Update [MSC2716](https://github.com/matrix-org/matrix-spec-proposals/pull/2716) implementation to process marker events from the current state to avoid markers being lost in timeline gaps for federated servers which would cause the imported history to be undiscovered. ([\#12718](https://github.com/matrix-org/synapse/issues/12718))
-- Add a `drop_federated_event` callback to `SpamChecker` to disregard inbound federated events before they take up much processing power, in an emergency. ([\#12744](https://github.com/matrix-org/synapse/issues/12744))
-- Implement [MSC3818: Copy room type on upgrade](https://github.com/matrix-org/matrix-spec-proposals/pull/3818). ([\#12786](https://github.com/matrix-org/synapse/issues/12786), [\#12792](https://github.com/matrix-org/synapse/issues/12792))
-- Update to the `check_event_for_spam` module callback. Deprecate the current callback signature, replace it with a new signature that is both less ambiguous (replacing booleans with explicit allow/block) and more powerful (ability to return explicit error codes). ([\#12808](https://github.com/matrix-org/synapse/issues/12808))
-
-
-Bugfixes
---------
-
-- Fix a bug introduced in Synapse 1.7.0 that would prevent events from being sent to clients if there's a retention policy in the room when the support for retention policies is disabled. ([\#12611](https://github.com/matrix-org/synapse/issues/12611))
-- Fix a bug introduced in Synapse 1.57.0 where `/messages` would throw a 500 error when querying for a non-existent room. ([\#12683](https://github.com/matrix-org/synapse/issues/12683))
-- Add a unique index to `state_group_edges` to prevent duplicates being accidentally introduced and the consequential impact to performance. ([\#12687](https://github.com/matrix-org/synapse/issues/12687))
-- Fix a long-standing bug where an empty room would be created when a user with an insufficient power level tried to upgrade a room. ([\#12696](https://github.com/matrix-org/synapse/issues/12696))
-- Fix a bug introduced in Synapse 1.30.0 where empty rooms could be automatically created if a monthly active users limit is set. ([\#12713](https://github.com/matrix-org/synapse/issues/12713))
-- Fix push to dismiss notifications when read on another client. Contributed by @SpiritCroc @ Beeper. ([\#12721](https://github.com/matrix-org/synapse/issues/12721))
-- Fix poor database performance when reading the cache invalidation stream for large servers with lots of workers. ([\#12747](https://github.com/matrix-org/synapse/issues/12747))
-- Delete events from the `federation_inbound_events_staging` table when a room is purged through the admin API. ([\#12770](https://github.com/matrix-org/synapse/issues/12770))
-- Give a meaningful error message when a client tries to create a room with an invalid alias localpart. ([\#12779](https://github.com/matrix-org/synapse/issues/12779))
-- Fix a bug introduced in 1.43.0 where a file (`providers.json`) was never closed. Contributed by @arkamar. ([\#12794](https://github.com/matrix-org/synapse/issues/12794))
-- Fix a long-standing bug where finished log contexts would be re-started when failing to contact remote homeservers. ([\#12803](https://github.com/matrix-org/synapse/issues/12803))
-- Fix a bug, introduced in Synapse 1.21.0, that led to media thumbnails being unusable before the index has been added in the background. ([\#12823](https://github.com/matrix-org/synapse/issues/12823))
-
-
-Updates to the Docker image
----------------------------
-
-- Fix the docker file after a dependency update. ([\#12853](https://github.com/matrix-org/synapse/issues/12853))
-
-
-Improved Documentation
-----------------------
-
-- Fix a typo in the Media Admin API documentation. ([\#12715](https://github.com/matrix-org/synapse/issues/12715))
-- Update the OpenID Connect example for Keycloak to be compatible with newer versions of Keycloak. Contributed by @nhh. ([\#12727](https://github.com/matrix-org/synapse/issues/12727))
-- Fix typo in server listener documentation. ([\#12742](https://github.com/matrix-org/synapse/issues/12742))
-- Link to the configuration manual from the welcome page of the documentation. ([\#12748](https://github.com/matrix-org/synapse/issues/12748))
-- Fix typo in `run_background_tasks_on` option name in configuration manual documentation. ([\#12749](https://github.com/matrix-org/synapse/issues/12749))
-- Add information regarding the `rc_invites` ratelimiting option to the configuration docs. ([\#12759](https://github.com/matrix-org/synapse/issues/12759))
-- Add documentation for cancellation of request processing. ([\#12761](https://github.com/matrix-org/synapse/issues/12761))
-- Recommend using docker to run tests against postgres. ([\#12765](https://github.com/matrix-org/synapse/issues/12765))
-- Add missing user directory endpoint from the generic worker documentation. Contributed by @olmari. ([\#12773](https://github.com/matrix-org/synapse/issues/12773))
-- Add additional info to documentation of config option `cache_autotuning`. ([\#12776](https://github.com/matrix-org/synapse/issues/12776))
-- Update configuration manual documentation to document size-related suffixes. ([\#12777](https://github.com/matrix-org/synapse/issues/12777))
-- Fix invalid YAML syntax in the example documentation for the `url_preview_accept_language` config option. ([\#12785](https://github.com/matrix-org/synapse/issues/12785))
-
-
-Deprecations and Removals
--------------------------
-
-- Require a body in POST requests to `/rooms/{roomId}/receipt/{receiptType}/{eventId}`, as required by the [Matrix specification](https://spec.matrix.org/v1.2/client-server-api/#post_matrixclientv3roomsroomidreceiptreceipttypeeventid). This breaks compatibility with Element Android 1.2.0 and earlier: users of those clients will be unable to send read receipts. ([\#12709](https://github.com/matrix-org/synapse/issues/12709))
-
-
-Internal Changes
-----------------
-
-- Improve event caching mechanism to avoid having multiple copies of an event in memory at a time. ([\#10533](https://github.com/matrix-org/synapse/issues/10533))
-- Preparation for faster-room-join work: return subsets of room state which we already have, immediately. ([\#12498](https://github.com/matrix-org/synapse/issues/12498))
-- Add `@cancellable` decorator, for use on endpoint methods that can be cancelled when clients disconnect. ([\#12586](https://github.com/matrix-org/synapse/issues/12586), [\#12588](https://github.com/matrix-org/synapse/issues/12588), [\#12630](https://github.com/matrix-org/synapse/issues/12630), [\#12694](https://github.com/matrix-org/synapse/issues/12694), [\#12698](https://github.com/matrix-org/synapse/issues/12698), [\#12699](https://github.com/matrix-org/synapse/issues/12699), [\#12700](https://github.com/matrix-org/synapse/issues/12700), [\#12705](https://github.com/matrix-org/synapse/issues/12705))
-- Enable cancellation of `GET /rooms/$room_id/members`, `GET /rooms/$room_id/state` and `GET /rooms/$room_id/state/$event_type/*` requests. ([\#12708](https://github.com/matrix-org/synapse/issues/12708))
-- Improve documentation of the `synapse.push` module. ([\#12676](https://github.com/matrix-org/synapse/issues/12676))
-- Refactor functions to on `PushRuleEvaluatorForEvent`. ([\#12677](https://github.com/matrix-org/synapse/issues/12677))
-- Preparation for database schema simplifications: stop writing to `event_reference_hashes`. ([\#12679](https://github.com/matrix-org/synapse/issues/12679))
-- Remove code which updates unused database column `application_services_state.last_txn`. ([\#12680](https://github.com/matrix-org/synapse/issues/12680))
-- Refactor `EventContext` class. ([\#12689](https://github.com/matrix-org/synapse/issues/12689))
-- Remove an unneeded class in the push code. ([\#12691](https://github.com/matrix-org/synapse/issues/12691))
-- Consolidate parsing of relation information from events. ([\#12693](https://github.com/matrix-org/synapse/issues/12693))
-- Convert namespace class `Codes` into a string enum. ([\#12703](https://github.com/matrix-org/synapse/issues/12703))
-- Optimize private read receipt filtering. ([\#12711](https://github.com/matrix-org/synapse/issues/12711))
-- Drop the logging level of status messages for the URL preview cache expiry job from INFO to DEBUG. ([\#12720](https://github.com/matrix-org/synapse/issues/12720))
-- Downgrade some OIDC errors to warnings in the logs, to reduce the noise of Sentry reports. ([\#12723](https://github.com/matrix-org/synapse/issues/12723))
-- Update configs used by Complement to allow more invites/3PID validations during tests. ([\#12731](https://github.com/matrix-org/synapse/issues/12731))
-- Fix a long-standing bug where the user directory background process would fail to make forward progress if a user included a null codepoint in their display name or avatar. ([\#12762](https://github.com/matrix-org/synapse/issues/12762))
-- Tweak the mypy plugin so that `@cached` can accept `on_invalidate=None`. ([\#12769](https://github.com/matrix-org/synapse/issues/12769))
-- Move methods that call `add_push_rule` to the `PushRuleStore` class. ([\#12772](https://github.com/matrix-org/synapse/issues/12772))
-- Make handling of federation Authorization header (more) compliant with RFC7230. ([\#12774](https://github.com/matrix-org/synapse/issues/12774))
-- Refactor `resolve_state_groups_for_events` to not pull out full state when no state resolution happens. ([\#12775](https://github.com/matrix-org/synapse/issues/12775))
-- Do not keep going if there are 5 back-to-back background update failures. ([\#12781](https://github.com/matrix-org/synapse/issues/12781))
-- Fix federation when using the demo scripts. ([\#12783](https://github.com/matrix-org/synapse/issues/12783))
-- The `hash_password` script now fails when it is called without specifying a config file. Contributed by @jae1911. ([\#12789](https://github.com/matrix-org/synapse/issues/12789))
-- Improve and fix type hints. ([\#12567](https://github.com/matrix-org/synapse/issues/12567), [\#12477](https://github.com/matrix-org/synapse/issues/12477), [\#12717](https://github.com/matrix-org/synapse/issues/12717), [\#12753](https://github.com/matrix-org/synapse/issues/12753), [\#12695](https://github.com/matrix-org/synapse/issues/12695), [\#12734](https://github.com/matrix-org/synapse/issues/12734), [\#12716](https://github.com/matrix-org/synapse/issues/12716), [\#12726](https://github.com/matrix-org/synapse/issues/12726), [\#12790](https://github.com/matrix-org/synapse/issues/12790), [\#12833](https://github.com/matrix-org/synapse/issues/12833))
-- Update EventContext `get_current_event_ids` and `get_prev_event_ids` to accept state filters and update calls where possible. ([\#12791](https://github.com/matrix-org/synapse/issues/12791))
-- Remove Caddy from the Synapse workers image used in Complement. ([\#12818](https://github.com/matrix-org/synapse/issues/12818))
-- Add Complement's shared registration secret to the Complement worker image. This fixes tests that depend on it. ([\#12819](https://github.com/matrix-org/synapse/issues/12819))
-- Support registering Application Services when running with workers under Complement. ([\#12826](https://github.com/matrix-org/synapse/issues/12826))
-- Disable 'faster room join' Complement tests when testing against Synapse with workers. ([\#12842](https://github.com/matrix-org/synapse/issues/12842))
-
-
 Synapse 1.59.1 (2022-05-18)
 ===========================
 

changelog.d/10533.misc

@@ -0,0 +1 @@
+Improve event caching mechanism to avoid having multiple copies of an event in memory at a time.

changelog.d/12477.misc

@@ -0,0 +1 @@
+Add some type hints to datastore.

changelog.d/12498.misc

@@ -0,0 +1 @@
+Preparation for faster-room-join work: return subsets of room state which we already have, immediately.

changelog.d/12513.feature

@@ -0,0 +1 @@
+Measure the time taken in spam-checking callbacks and expose those measurements as metrics.

changelog.d/12567.misc

@@ -0,0 +1 @@
+Replace string literal instances of stream key types with typed constants.

changelog.d/12586.misc

@@ -0,0 +1 @@
+Add `@cancellable` decorator, for use on endpoint methods that can be cancelled when clients disconnect.

changelog.d/12588.misc

@@ -0,0 +1 @@
+Add ability to cancel disconnected requests to `SynapseRequest`.

changelog.d/12618.feature

@@ -0,0 +1 @@
+Add a `default_power_level_content_override` config option to set default room power levels per room preset.

changelog.d/12623.feature

@@ -0,0 +1 @@
+Add support for [MSC3787: Allowing knocks to restricted rooms](https://github.com/matrix-org/matrix-spec-proposals/pull/3787).

changelog.d/12630.misc

@@ -0,0 +1 @@
+Add a helper class for testing request cancellation.

changelog.d/12672.feature

@@ -0,0 +1 @@
+Send `USER_IP` commands on a different Redis channel, in order to reduce traffic to workers that do not process these commands.

changelog.d/12673.feature

@@ -0,0 +1 @@
+Synapse will now reload [cache config](https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html#caching) when it receives a [SIGHUP](https://en.wikipedia.org/wiki/SIGHUP) signal.

changelog.d/12676.misc

@@ -0,0 +1 @@
+Improve documentation of the `synapse.push` module.

changelog.d/12677.misc

@@ -0,0 +1 @@
+Refactor functions to on `PushRuleEvaluatorForEvent`.

changelog.d/12679.misc

@@ -0,0 +1 @@
+Preparation for database schema simplifications: stop writing to `event_reference_hashes`.

changelog.d/12680.misc

@@ -0,0 +1 @@
+Remove code which updates unused database column `application_services_state.last_txn`.

changelog.d/12683.bugfix

@@ -0,0 +1 @@
+Fix a bug introduced in Synapse 1.57.0 where `/messages` would throw a 500 error when querying for a non-existent room.

changelog.d/12687.bugfix

@@ -0,0 +1 @@
+Add a unique index to `state_group_edges` to prevent duplicates being accidentally introduced and the consequential impact to performance.

changelog.d/12689.misc

@@ -0,0 +1 @@
+Refactor `EventContext` class.

changelog.d/12691.misc

@@ -0,0 +1 @@
+Remove an unneeded class in the push code.

changelog.d/12693.misc

@@ -0,0 +1 @@
+Consolidate parsing of relation information from events.

changelog.d/12694.misc

@@ -0,0 +1 @@
+Capture the `Deferred` for request cancellation in `_AsyncResource`.

changelog.d/12695.misc

@@ -0,0 +1 @@
+Fixes an incorrect type hint for `Filter._check_event_relations`.

changelog.d/12696.bugfix

@@ -0,0 +1 @@
+Fix a long-standing bug where an empty room would be created when a user with an insufficient power level tried to upgrade a room.

changelog.d/12698.misc

@@ -0,0 +1 @@
+Respect the `@cancellable` flag for `DirectServe{Html,Json}Resource`s.

changelog.d/12699.misc

@@ -0,0 +1 @@
+Respect the `@cancellable` flag for `RestServlet`s and `BaseFederationServlet`s.

changelog.d/12700.misc

@@ -0,0 +1 @@
+Respect the `@cancellable` flag for `ReplicationEndpoint`s.

changelog.d/12701.feature

@@ -0,0 +1 @@
+Add a config options to allow for auto-tuning of caches.

changelog.d/12703.misc

@@ -0,0 +1 @@
+Convert namespace class `Codes` into a string enum.

changelog.d/12705.misc

@@ -0,0 +1 @@
+Complain if a federation endpoint has the `@cancellable` flag, since some of the wrapper code may not handle cancellation correctly yet.

changelog.d/12708.misc

@@ -0,0 +1 @@
+Enable cancellation of `GET /rooms/$room_id/members`, `GET /rooms/$room_id/state` and `GET /rooms/$room_id/state/$event_type/*` requests.

changelog.d/12709.removal

@@ -0,0 +1 @@
+Require a body in POST requests to `/rooms/{roomId}/receipt/{receiptType}/{eventId}`, as required by the [Matrix specification](https://spec.matrix.org/v1.2/client-server-api/#post_matrixclientv3roomsroomidreceiptreceipttypeeventid). This breaks compatibility with Element Android 1.2.0 and earlier: users of those clients will be unable to send read receipts.

changelog.d/12711.misc

@@ -0,0 +1 @@
+Optimize private read receipt filtering.

changelog.d/12713.bugfix

@@ -0,0 +1 @@
+Fix a bug introduced in Synapse 1.30.0 where empty rooms could be automatically created if a monthly active users limit is set.

changelog.d/12715.doc

@@ -0,0 +1 @@
+Fix a typo in the Media Admin API documentation.

changelog.d/12716.misc

@@ -0,0 +1 @@
+Add type annotations to increase the number of modules passing `disallow-untyped-defs`.

changelog.d/12717.misc

@@ -0,0 +1 @@
+Add some type hints to datastore.

changelog.d/12720.misc

@@ -0,0 +1 @@
+Drop the logging level of status messages for the URL preview cache expiry job from INFO to DEBUG.

changelog.d/12721.bugfix

@@ -0,0 +1 @@
+Fix push to dismiss notifications when read on another client. Contributed by @SpiritCroc @ Beeper.

changelog.d/12723.misc

@@ -0,0 +1 @@
+Downgrade some OIDC errors to warnings in the logs, to reduce the noise of Sentry reports.

changelog.d/12726.misc

@@ -0,0 +1 @@
+Add type annotations to increase the number of modules passing `disallow-untyped-defs`.

changelog.d/12727.doc

@@ -0,0 +1 @@
+Update the OpenID Connect example for Keycloak to be compatible with newer versions of Keycloak. Contributed by @nhh.

changelog.d/12731.misc

@@ -0,0 +1 @@
+Update configs used by Complement to allow more invites/3PID validations during tests.

changelog.d/12734.misc

@@ -0,0 +1 @@
+Tidy up and type-hint the database engine modules.

changelog.d/12742.doc

@@ -0,0 +1 @@
+Fix typo in server listener documentation.

changelog.d/12747.bugfix

@@ -0,0 +1 @@
+Fix poor database performance when reading the cache invalidation stream for large servers with lots of workers.

changelog.d/12748.doc

@@ -0,0 +1 @@
+Link to the configuration manual from the welcome page of the documentation.

changelog.d/12749.doc

@@ -0,0 +1 @@
+Fix typo in 'run_background_tasks_on' option name in configuration manual documentation.

changelog.d/12753.misc

@@ -0,0 +1 @@
+Add some type hints to datastore.

changelog.d/12759.doc

@@ -0,0 +1 @@
+Add information regarding the `rc_invites` ratelimiting option to the configuration docs.

changelog.d/12761.doc

@@ -0,0 +1 @@
+Add documentation for cancellation of request processing.

changelog.d/12762.misc

@@ -0,0 +1 @@
+Fix a long-standing bug where the user directory background process would fail to make forward progress if a user included a null codepoint in their display name or avatar.

changelog.d/12765.doc

@@ -0,0 +1 @@
+Recommend using docker to run tests against postgres.

changelog.d/12769.misc

@@ -0,0 +1 @@
+Tweak the mypy plugin so that `@cached` can accept `on_invalidate=None`.

changelog.d/12770.bugfix

@@ -0,0 +1 @@
+Delete events from the `federation_inbound_events_staging` table when a room is purged through the admin API.

changelog.d/12772.misc

@@ -0,0 +1 @@
+Move methods that call `add_push_rule` to the `PushRuleStore` class.

changelog.d/12773.doc

@@ -0,0 +1 @@
+Add missing user directory endpoint from the generic worker documentation. Contributed by @olmari.

changelog.d/12774.misc

@@ -0,0 +1 @@
+Make handling of federation Authorization header (more) compliant with RFC7230.

changelog.d/12775.misc

@@ -0,0 +1 @@
+Refactor `resolve_state_groups_for_events` to not pull out full state when no state resolution happens.

changelog.d/12776.doc

@@ -0,0 +1,2 @@
+Add additional info to documentation of config option `cache_autotuning`.
+

changelog.d/12777.doc

@@ -0,0 +1,2 @@
+Update configuration manual documentation to document size-related suffixes.
+

changelog.d/12779.bugfix

@@ -0,0 +1 @@
+Give a meaningful error message when a client tries to create a room with an invalid alias localpart.

changelog.d/12781.misc

@@ -0,0 +1 @@
+Do not keep going if there are 5 back-to-back background update failures.

changelog.d/12783.misc

@@ -0,0 +1 @@
+Fix federation when using the demo scripts.

changelog.d/12785.doc

@@ -0,0 +1 @@
+Fix invalid YAML syntax in the example documentation for the `url_preview_accept_language` config option.

changelog.d/12786.feature

@@ -0,0 +1 @@
+Implement [MSC3818: Copy room type on upgrade](https://github.com/matrix-org/matrix-spec-proposals/pull/3818).

changelog.d/12789.misc

@@ -0,0 +1 @@
+The `hash_password` script now fails when it is called without specifying a config file.

changelog.d/12790.misc

@@ -0,0 +1 @@
+Simplify `disallow_untyped_defs` config in `mypy.ini`.

changelog.d/12791.misc

@@ -0,0 +1 @@
+Update EventContext `get_current_event_ids` and `get_prev_event_ids` to accept state filters and update calls where possible.

changelog.d/12792.feature

@@ -0,0 +1 @@
+Implement [MSC3818: Copy room type on upgrade](https://github.com/matrix-org/matrix-spec-proposals/pull/3818).

changelog.d/12794.bugfix

@@ -0,0 +1 @@
+Fix a bug introduced in 1.43.0 where a file (`providers.json`) was never closed. Contributed by @arkamar.

changelog.d/12803.bugfix

@@ -0,0 +1 @@
+Fix a long-standing bug where finished log contexts would be re-started when failing to contact remote homeservers.

changelog.d/12809.feature

@@ -0,0 +1 @@
+Send `USER_IP` commands on a different Redis channel, in order to reduce traffic to workers that do not process these commands.

changelog.d/12811.misc

@@ -0,0 +1 @@
+Reduce the amount of state we pull from the DB.

changelog.d/12828.misc

@@ -0,0 +1 @@
+Pull out less state when handling gaps in room DAG.

debian/changelog

@@ -1,21 +1,3 @@
-matrix-synapse-py3 (1.60.0) stable; urgency=medium
-
-  * New Synapse release 1.60.0.
-
- -- Synapse Packaging team <packages@matrix.org>  Tue, 31 May 2022 13:41:22 +0100
-
-matrix-synapse-py3 (1.60.0~rc2) stable; urgency=medium
-
-  * New Synapse release 1.60.0rc2.
-
- -- Synapse Packaging team <packages@matrix.org>  Fri, 27 May 2022 11:04:55 +0100
-
-matrix-synapse-py3 (1.60.0~rc1) stable; urgency=medium
-
-  * New Synapse release 1.60.0rc1.
-
- -- Synapse Packaging team <packages@matrix.org>  Tue, 24 May 2022 12:05:01 +0100
-
 matrix-synapse-py3 (1.59.1) stable; urgency=medium
 
   * New Synapse release 1.59.1.

docker/Dockerfile

@@ -55,7 +55,7 @@ RUN \
 # NB: In poetry 1.2 `poetry export` will be moved into a plugin; we'll need to also
 # pip install poetry-plugin-export (https://github.com/python-poetry/poetry-plugin-export).
 RUN --mount=type=cache,target=/root/.cache/pip \
-  pip install --user "poetry-core==1.1.0a7" "git+https://github.com/python-poetry/poetry.git@fb13b3a676f476177f7937ffa480ee5cff9a90a5"
+  pip install --user git+https://github.com/python-poetry/poetry.git@fb13b3a676f476177f7937ffa480ee5cff9a90a5
 
 WORKDIR /synapse
 

docker/complement/SynapseWorkers.Dockerfile

@@ -6,6 +6,12 @@
 # https://github.com/matrix-org/synapse/blob/develop/docker/README-testing.md#testing-with-postgresql-and-single-or-multi-process-synapse
 FROM matrixdotorg/synapse-workers
 
+# Download a caddy server to stand in front of nginx and terminate TLS using Complement's
+# custom CA.
+# We include this near the top of the file in order to cache the result.
+RUN curl -OL "https://github.com/caddyserver/caddy/releases/download/v2.3.0/caddy_2.3.0_linux_amd64.tar.gz" && \
+  tar xzf caddy_2.3.0_linux_amd64.tar.gz && rm caddy_2.3.0_linux_amd64.tar.gz && mv caddy /root
+
 # Install postgresql
 RUN apt-get update && \
   DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y postgresql-13
@@ -25,12 +31,16 @@ COPY conf-workers/workers-shared.yaml /conf/workers/shared.yaml
 
 WORKDIR /data
 
+# Copy the caddy config
+COPY conf-workers/caddy.complement.json /root/caddy.json
+
 COPY conf-workers/postgres.supervisord.conf /etc/supervisor/conf.d/postgres.conf
+COPY conf-workers/caddy.supervisord.conf /etc/supervisor/conf.d/caddy.conf
 
 # Copy the entrypoint
 COPY conf-workers/start-complement-synapse-workers.sh /
 
-# Expose nginx's listener ports
+# Expose caddy's listener ports
 EXPOSE 8008 8448
 
 ENTRYPOINT ["/start-complement-synapse-workers.sh"]

docker/complement/conf-workers/caddy.complement.json

@@ -0,0 +1,72 @@
+{
+    "apps": {
+      "http": {
+        "servers": {
+          "srv0": {
+            "listen": [
+              ":8448"
+            ],
+            "routes": [
+              {
+                "match": [
+                  {
+                    "host": [
+                      "{{ server_name }}"
+                    ]
+                  }
+                ],
+                "handle": [
+                  {
+                    "handler": "subroute",
+                    "routes": [
+                      {
+                        "handle": [
+                          {
+                            "handler": "reverse_proxy",
+                            "upstreams": [
+                              {
+                                "dial": "localhost:8008"
+                              }
+                            ]
+                          }
+                        ]
+                      }
+                    ]
+                  }
+                ],
+                "terminal": true
+              }
+            ]
+          }
+        }
+      },
+      "tls": {
+        "automation": {
+          "policies": [
+            {
+              "subjects": [
+                "{{ server_name }}"
+              ],
+              "issuers": [
+                {
+                  "module": "internal"
+                }
+              ],
+              "on_demand": true
+            }
+          ]
+        }
+      },
+      "pki": {
+        "certificate_authorities": {
+          "local": {
+            "name": "Complement CA",
+            "root": {
+              "certificate": "/complement/ca/ca.crt",
+              "private_key": "/complement/ca/ca.key"
+            }
+          }
+        }
+      }
+    }
+  }

docker/complement/conf-workers/caddy.supervisord.conf

@@ -0,0 +1,7 @@
+[program:caddy]
+command=/usr/local/bin/prefix-log /root/caddy run --config /root/caddy.json
+autorestart=unexpected
+stdout_logfile=/dev/stdout
+stdout_logfile_maxbytes=0
+stderr_logfile=/dev/stderr
+stderr_logfile_maxbytes=0

docker/complement/conf-workers/start-complement-synapse-workers.sh

@@ -9,6 +9,9 @@ function log {
     echo "$d $@"
 }
 
+# Replace the server name in the caddy config
+sed -i "s/{{ server_name }}/${SERVER_NAME}/g" /root/caddy.json
+
 # Set the server name of the homeserver
 export SYNAPSE_SERVER_NAME=${SERVER_NAME}
 
@@ -36,26 +39,6 @@ export SYNAPSE_WORKER_TYPES="\
     appservice, \
     pusher"
 
-# Add Complement's appservice registration directory, if there is one
-# (It can be absent when there are no application services in this test!)
-if [ -d /complement/appservice ]; then
-    export SYNAPSE_AS_REGISTRATION_DIR=/complement/appservice
-fi
-
-# Generate a TLS key, then generate a certificate by having Complement's CA sign it
-# Note that both the key and certificate are in PEM format (not DER).
-openssl genrsa -out /conf/server.tls.key 2048
-
-openssl req -new -key /conf/server.tls.key -out /conf/server.tls.csr \
-  -subj "/CN=${SERVER_NAME}"
-
-openssl x509 -req -in /conf/server.tls.csr \
-  -CA /complement/ca/ca.crt -CAkey /complement/ca/ca.key -set_serial 1 \
-  -out /conf/server.tls.crt
-
-export SYNAPSE_TLS_CERT=/conf/server.tls.crt
-export SYNAPSE_TLS_KEY=/conf/server.tls.key
-
 # Run the script that writes the necessary config files and starts supervisord, which in turn
 # starts everything else
 exec /configure_workers_and_start.py

docker/complement/conf-workers/workers-shared.yaml

@@ -5,12 +5,6 @@ enable_registration: true
 enable_registration_without_verification: true
 bcrypt_rounds: 4
 
-## Registration ##
-
-# Needed by Complement to register admin users
-# DO NOT USE in a production configuration! This should be a random secret.
-registration_shared_secret: complement
-
 ## Federation ##
 
 # trust certs signed by Complement's CA

docker/conf-workers/nginx.conf.j2

@@ -9,22 +9,6 @@ server {
     listen 8008;
     listen [::]:8008;
 
-    {% if tls_cert_path is not none and tls_key_path is not none %}
-        listen 8448 ssl;
-        listen [::]:8448 ssl;
-
-        ssl_certificate {{ tls_cert_path }};
-        ssl_certificate_key {{ tls_key_path }};
-
-        # Some directives from cipherlist.eu (fka cipherli.st):
-        ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
-        ssl_prefer_server_ciphers on;
-        ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
-        ssl_ecdh_curve secp384r1; # Requires nginx >= 1.1.0
-        ssl_session_cache shared:SSL:10m;
-        ssl_session_tickets off; # Requires nginx >= 1.5.9
-    {% endif %}
-
     server_name localhost;
 
     # Nginx by default only allows file uploads up to 1M in size

docker/conf-workers/shared.yaml.j2

@@ -6,13 +6,4 @@
 redis:
     enabled: true
 
-{% if appservice_registrations is not none %}
-## Application Services ##
-# A list of application service config files to use.
-app_service_config_files:
-{%- for path in appservice_registrations %}
-  - "{{ path }}"
-{%- endfor %}
-{%- endif %}
-
-{{ shared_worker_config }}
+{{ shared_worker_config }}

docker/configure_workers_and_start.py

@@ -21,11 +21,6 @@
 #   * SYNAPSE_REPORT_STATS: Whether to report stats.
 #   * SYNAPSE_WORKER_TYPES: A comma separated list of worker names as specified in WORKER_CONFIG
 #         below. Leave empty for no workers, or set to '*' for all possible workers.
-#   * SYNAPSE_AS_REGISTRATION_DIR: If specified, a directory in which .yaml and .yml files
-#         will be treated as Application Service registration files.
-#   * SYNAPSE_TLS_CERT: Path to a TLS certificate in PEM format.
-#   * SYNAPSE_TLS_KEY: Path to a TLS key. If this and SYNAPSE_TLS_CERT are specified,
-#         Nginx will be configured to serve TLS on port 8448.
 #
 # NOTE: According to Complement's ENTRYPOINT expectations for a homeserver image (as defined
 # in the project's README), this script may be run multiple times, and functionality should
@@ -34,7 +29,6 @@
 import os
 import subprocess
 import sys
-from pathlib import Path
 from typing import Any, Dict, List, Mapping, MutableMapping, NoReturn, Set
 
 import jinja2
@@ -494,23 +488,11 @@ def generate_worker_files(
     master_log_config = generate_worker_log_config(environ, "master", data_dir)
     shared_config["log_config"] = master_log_config
 
-    # Find application service registrations
-    appservice_registrations = None
-    appservice_registration_dir = os.environ.get("SYNAPSE_AS_REGISTRATION_DIR")
-    if appservice_registration_dir:
-        # Scan for all YAML files that should be application service registrations.
-        appservice_registrations = [
-            str(reg_path.resolve())
-            for reg_path in Path(appservice_registration_dir).iterdir()
-            if reg_path.suffix.lower() in (".yaml", ".yml")
-        ]
-
     # Shared homeserver config
     convert(
         "/conf/shared.yaml.j2",
         "/conf/workers/shared.yaml",
         shared_worker_config=yaml.dump(shared_config),
-        appservice_registrations=appservice_registrations,
     )
 
     # Nginx config
@@ -519,8 +501,6 @@ def generate_worker_files(
         "/etc/nginx/conf.d/matrix-synapse.conf",
         worker_locations=nginx_location_config,
         upstream_directives=nginx_upstream_config,
-        tls_cert_path=os.environ.get("SYNAPSE_TLS_CERT"),
-        tls_key_path=os.environ.get("SYNAPSE_TLS_KEY"),
     )
 
     # Supervisord config

docs/development/contributing_guide.md

@@ -422,8 +422,8 @@ same lightweight approach that the Linux Kernel
 [submitting patches process](
 https://www.kernel.org/doc/html/latest/process/submitting-patches.html#sign-your-work-the-developer-s-certificate-of-origin>),
 [Docker](https://github.com/docker/docker/blob/master/CONTRIBUTING.md), and many other
-projects use: the DCO ([Developer Certificate of Origin](http://developercertificate.org/)).
-This is a simple declaration that you wrote
+projects use: the DCO (Developer Certificate of Origin:
+http://developercertificate.org/). This is a simple declaration that you wrote
 the contribution or otherwise have the right to contribute it to Matrix:
 
 ```

docs/modules/spam_checker_callbacks.md

@@ -12,27 +12,21 @@ The available spam checker callbacks are:
 
 _First introduced in Synapse v1.37.0_
 
-_Changed in Synapse v1.60.0: `synapse.module_api.NOT_SPAM` and `synapse.module_api.errors.Codes` can be returned by this callback. Returning a boolean or a string is now deprecated._ 
-
 ```python
-async def check_event_for_spam(event: "synapse.module_api.EventBase") -> Union["synapse.module_api.NOT_SPAM", "synapse.module_api.errors.Codes", str, bool]
+async def check_event_for_spam(event: "synapse.events.EventBase") -> Union[bool, str]
 ```
 
-Called when receiving an event from a client or via federation. The callback must return one of:
-  - `synapse.module_api.NOT_SPAM`, to allow the operation. Other callbacks may still 
-    decide to reject it.
-  - `synapse.module_api.errors.Codes` to reject the operation with an error code. In case
-    of doubt, `synapse.module_api.errors.Codes.FORBIDDEN` is a good error code.
-  - (deprecated) a non-`Codes` `str` to reject the operation and specify an error message. Note that clients
-    typically will not localize the error message to the user's preferred locale.
-  - (deprecated) `False`, which is the same as returning `synapse.module_api.NOT_SPAM`.
-  - (deprecated) `True`, which is the same as returning `synapse.module_api.errors.Codes.FORBIDDEN`.
+Called when receiving an event from a client or via federation. The callback must return
+either:
+- an error message string, to indicate the event must be rejected because of spam and 
+  give a rejection reason to forward to clients;
+- the boolean `True`, to indicate that the event is spammy, but not provide further details; or
+- the booelan `False`, to indicate that the event is not considered spammy.
 
 If multiple modules implement this callback, they will be considered in order. If a
-callback returns `synapse.module_api.NOT_SPAM`, Synapse falls through to the next one.
-The value of the first callback that does not return `synapse.module_api.NOT_SPAM` will
-be used. If this happens, Synapse will not call any of the subsequent implementations of
-this callback.
+callback returns `False`, Synapse falls through to the next one. The value of the first
+callback that does not return `False` will be used. If this happens, Synapse will not call
+any of the subsequent implementations of this callback.
 
 ### `user_may_join_room`
 
@@ -255,24 +249,6 @@ callback returns `False`, Synapse falls through to the next one. The value of th
 callback that does not return `False` will be used. If this happens, Synapse will not call
 any of the subsequent implementations of this callback.
 
-### `should_drop_federated_event`
-
-_First introduced in Synapse v1.60.0_
-
-```python
-async def should_drop_federated_event(event: "synapse.events.EventBase") -> bool
-```
-
-Called when checking whether a remote server can federate an event with us. **Returning
-`True` from this function will silently drop a federated event and split-brain our view
-of a room's DAG, and thus you shouldn't use this callback unless you know what you are
-doing.**
-
-If multiple modules implement this callback, they will be considered in order. If a
-callback returns `False`, Synapse falls through to the next one. The value of the first
-callback that does not return `False` will be used. If this happens, Synapse will not call
-any of the subsequent implementations of this callback.
-
 ## Example
 
 The example below is a module that implements the spam checker callback

docs/sample_config.yaml

@@ -2216,9 +2216,7 @@ sso:
 
 
 password_config:
-   # Uncomment to disable password login.
-   # Set to `only_for_reauth` to permit reauthentication for users that
-   # have passwords and are already logged in.
+   # Uncomment to disable password login
    #
    #enabled: false
 

docs/upgrade.md

@@ -177,36 +177,7 @@ has queries that can be used to check a database for this problem in advance.
 
 </details>
 
-## New signature for the spam checker callback `check_event_for_spam`
 
-The previous signature has been deprecated.
-
-Whereas `check_event_for_spam` callbacks used to return `Union[str, bool]`, they should now return `Union["synapse.module_api.NOT_SPAM", "synapse.module_api.errors.Codes"]`.
-
-This is part of an ongoing refactoring of the SpamChecker API to make it less ambiguous and more powerful.
-
-If your module implements `check_event_for_spam` as follows:
-
-```python
-async def check_event_for_spam(event):
-    if ...:
-        # Event is spam
-        return True
-    # Event is not spam
-    return False
-```
-
-you should rewrite it as follows:
-
-```python
-async def check_event_for_spam(event):
-    if ...:
-        # Event is spam, mark it as forbidden (you may use some more precise error
-        # code if it is useful).
-        return synapse.module_api.errors.Codes.FORBIDDEN
-    # Event is not spam, mark it as such.
-    return synapse.module_api.NOT_SPAM
-```
 
 # Upgrading to v1.59.0
 

docs/usage/configuration/config_documentation.md

@@ -2930,9 +2930,6 @@ Use this setting to enable password-based logins.
 
 This setting has the following sub-options:
 * `enabled`: Defaults to true.
-   Set to false to disable password authentication.
-   Set to `only_for_reauth` to allow users with existing passwords to use them
-   to log in and reauthenticate, whilst preventing new users from setting passwords.
 * `localdb_enabled`: Set to false to disable authentication against the local password
    database. This is ignored if `enabled` is false, and is only useful
    if you have other `password_providers`. Defaults to true. 

mypy.ini

@@ -41,11 +41,16 @@ exclude = (?x)
    |tests/events/test_utils.py
    |tests/federation/test_federation_catch_up.py
    |tests/federation/test_federation_sender.py
+   |tests/federation/test_federation_server.py
    |tests/federation/transport/test_knocking.py
+   |tests/federation/transport/test_server.py
    |tests/handlers/test_typing.py
    |tests/http/federation/test_matrix_federation_agent.py
    |tests/http/federation/test_srv_resolver.py
+   |tests/http/test_fedclient.py
    |tests/http/test_proxyagent.py
+   |tests/http/test_servlet.py
+   |tests/http/test_site.py
    |tests/logging/__init__.py
    |tests/logging/test_terse_json.py
    |tests/module_api/test_api.py
@@ -54,9 +59,12 @@ exclude = (?x)
    |tests/push/test_push_rule_evaluator.py
    |tests/rest/client/test_transactions.py
    |tests/rest/media/v1/test_media_storage.py
+   |tests/scripts/test_new_matrix_user.py
    |tests/server.py
    |tests/server_notices/test_resource_limits_server_notices.py
    |tests/state/test_v2.py
+   |tests/storage/test_base.py
+   |tests/storage/test_roommember.py
    |tests/test_metrics.py
    |tests/test_server.py
    |tests/test_state.py

pyproject.toml

@@ -54,7 +54,7 @@ skip_gitignore = true
 
 [tool.poetry]
 name = "matrix-synapse"
-version = "1.60.0"
+version = "1.59.1"
 description = "Homeserver for the Matrix decentralised comms protocol"
 authors = ["Matrix.org Team and Contributors <packages@matrix.org>"]
 license = "Apache-2.0"

scripts-dev/complement.sh

@@ -45,8 +45,6 @@ docker build -t matrixdotorg/synapse -f "docker/Dockerfile" .
 
 extra_test_args=()
 
-test_tags="synapse_blacklist,msc2716,msc3030"
-
 # If we're using workers, modify the docker files slightly.
 if [[ -n "$WORKERS" ]]; then
   # Build the workers docker image (from the base Synapse image).
@@ -67,10 +65,6 @@ if [[ -n "$WORKERS" ]]; then
 else
   export COMPLEMENT_BASE_IMAGE=complement-synapse
   COMPLEMENT_DOCKERFILE=Dockerfile
-
-  # We only test faster room joins on monoliths, because they are purposefully
-  # being developed without worker support to start with.
-  test_tags="$test_tags,faster_joins"
 fi
 
 # Build the Complement image from the Synapse image we just built.
@@ -79,5 +73,4 @@ docker build -t $COMPLEMENT_BASE_IMAGE -f "docker/complement/$COMPLEMENT_DOCKERF
 # Run the tests!
 echo "Images built; running complement"
 cd "$COMPLEMENT_DIR"
-
-go test -v -tags $test_tags -count=1 "${extra_test_args[@]}" "$@" ./tests/...
+go test -v -tags synapse_blacklist,msc2716,msc3030,faster_joins -count=1 "${extra_test_args[@]}" "$@" ./tests/...

synapse/api/auth.py

@@ -61,7 +61,6 @@ class Auth:
         self.hs = hs
         self.clock = hs.get_clock()
         self.store = hs.get_datastores().main
-        self.state = hs.get_state_handler()
         self._account_validity_handler = hs.get_account_validity_handler()
 
         self.token_cache: LruCache[str, Tuple[str, bool]] = LruCache(
@@ -81,7 +80,7 @@ class Auth:
         user_id: str,
         current_state: Optional[StateMap[EventBase]] = None,
         allow_departed_users: bool = False,
-    ) -> EventBase:
+    ) -> Tuple[str, Optional[str]]:
         """Check if the user is in the room, or was at some point.
         Args:
             room_id: The room to check.
@@ -99,29 +98,28 @@ class Auth:
         Raises:
             AuthError if the user is/was not in the room.
         Returns:
-            Membership event for the user if the user was in the
-            room. This will be the join event if they are currently joined to
-            the room. This will be the leave event if they have left the room.
+            The current membership of the user in the room and the
+            membership event ID of the user.
         """
-        if current_state:
-            member = current_state.get((EventTypes.Member, user_id), None)
-        else:
-            member = await self.state.get_current_state(
-                room_id=room_id, event_type=EventTypes.Member, state_key=user_id
-            )
 
-        if member:
-            membership = member.membership
+        (
+            membership,
+            member_event_id,
+        ) = await self.store.get_local_current_membership_for_user_in_room(
+            user_id=user_id,
+            room_id=room_id,
+        )
 
+        if membership:
             if membership == Membership.JOIN:
-                return member
+                return membership, member_event_id
 
             # XXX this looks totally bogus. Why do we not allow users who have been banned,
             # or those who were members previously and have been re-invited?
             if allow_departed_users and membership == Membership.LEAVE:
                 forgot = await self.store.did_forget(user_id, room_id)
                 if not forgot:
-                    return member
+                    return membership, member_event_id
 
         raise AuthError(403, "User %s not in room %s" % (user_id, room_id))
 
@@ -602,7 +600,8 @@ class Auth:
         # We currently require the user is a "moderator" in the room. We do this
         # by checking if they would (theoretically) be able to change the
         # m.room.canonical_alias events
-        power_level_event = await self.state.get_current_state(
+
+        power_level_event = await self.store.get_current_state_event(
             room_id, EventTypes.PowerLevels, ""
         )
 
@@ -693,12 +692,11 @@ class Auth:
             #  * The user is a non-guest user, and was ever in the room
             #  * The user is a guest user, and has joined the room
             # else it will throw.
-            member_event = await self.check_user_in_room(
+            return await self.check_user_in_room(
                 room_id, user_id, allow_departed_users=allow_departed_users
             )
-            return member_event.membership, member_event.event_id
         except AuthError:
-            visibility = await self.state.get_current_state(
+            visibility = await self.store.get_current_state_event(
                 room_id, EventTypes.RoomHistoryVisibility, ""
             )
             if (

synapse/api/errors.py

@@ -270,7 +270,9 @@ class UnrecognizedRequestError(SynapseError):
     """An error indicating we don't understand the request you're trying to make"""
 
     def __init__(
-        self, msg: str = "Unrecognized request", errcode: str = Codes.UNRECOGNIZED
+        self,
+        msg: str = "Unrecognized request",
+        errcode: str = Codes.UNRECOGNIZED,
     ):
         super().__init__(400, msg, errcode)
 

synapse/config/auth.py

@@ -29,18 +29,7 @@ class AuthConfig(Config):
         if password_config is None:
             password_config = {}
 
-        passwords_enabled = password_config.get("enabled", True)
-        # 'only_for_reauth' allows users who have previously set a password to use it,
-        # even though passwords would otherwise be disabled.
-        passwords_for_reauth_only = passwords_enabled == "only_for_reauth"
-
-        self.password_enabled_for_login = (
-            passwords_enabled and not passwords_for_reauth_only
-        )
-        self.password_enabled_for_reauth = (
-            passwords_for_reauth_only or passwords_enabled
-        )
-
+        self.password_enabled = password_config.get("enabled", True)
         self.password_localdb_enabled = password_config.get("localdb_enabled", True)
         self.password_pepper = password_config.get("pepper", "")
 
@@ -57,9 +46,7 @@ class AuthConfig(Config):
     def generate_config_section(self, **kwargs: Any) -> str:
         return """\
         password_config:
-           # Uncomment to disable password login.
-           # Set to `only_for_reauth` to permit reauthentication for users that
-           # have passwords and are already logged in.
+           # Uncomment to disable password login
            #
            #enabled: false
 

synapse/events/spamcheck.py

@@ -27,7 +27,6 @@ from typing import (
     Union,
 )
 
-from synapse.api.errors import Codes
 from synapse.rest.media.v1._base import FileInfo
 from synapse.rest.media.v1.media_storage import ReadableFileWrapper
 from synapse.spam_checker_api import RegistrationBehaviour
@@ -41,18 +40,7 @@ if TYPE_CHECKING:
 
 logger = logging.getLogger(__name__)
 
-
 CHECK_EVENT_FOR_SPAM_CALLBACK = Callable[
-    ["synapse.events.EventBase"],
-    Awaitable[
-        Union[
-            str,
-            # Deprecated
-            bool,
-        ]
-    ],
-]
-SHOULD_DROP_FEDERATED_EVENT_CALLBACK = Callable[
     ["synapse.events.EventBase"],
     Awaitable[Union[bool, str]],
 ]
@@ -175,16 +163,11 @@ def load_legacy_spam_checkers(hs: "synapse.server.HomeServer") -> None:
 
 
 class SpamChecker:
-    NOT_SPAM = "NOT_SPAM"
-
     def __init__(self, hs: "synapse.server.HomeServer") -> None:
         self.hs = hs
         self.clock = hs.get_clock()
 
         self._check_event_for_spam_callbacks: List[CHECK_EVENT_FOR_SPAM_CALLBACK] = []
-        self._should_drop_federated_event_callbacks: List[
-            SHOULD_DROP_FEDERATED_EVENT_CALLBACK
-        ] = []
         self._user_may_join_room_callbacks: List[USER_MAY_JOIN_ROOM_CALLBACK] = []
         self._user_may_invite_callbacks: List[USER_MAY_INVITE_CALLBACK] = []
         self._user_may_send_3pid_invite_callbacks: List[
@@ -208,9 +191,6 @@ class SpamChecker:
     def register_callbacks(
         self,
         check_event_for_spam: Optional[CHECK_EVENT_FOR_SPAM_CALLBACK] = None,
-        should_drop_federated_event: Optional[
-            SHOULD_DROP_FEDERATED_EVENT_CALLBACK
-        ] = None,
         user_may_join_room: Optional[USER_MAY_JOIN_ROOM_CALLBACK] = None,
         user_may_invite: Optional[USER_MAY_INVITE_CALLBACK] = None,
         user_may_send_3pid_invite: Optional[USER_MAY_SEND_3PID_INVITE_CALLBACK] = None,
@@ -229,11 +209,6 @@ class SpamChecker:
         if check_event_for_spam is not None:
             self._check_event_for_spam_callbacks.append(check_event_for_spam)
 
-        if should_drop_federated_event is not None:
-            self._should_drop_federated_event_callbacks.append(
-                should_drop_federated_event
-            )
-
         if user_may_join_room is not None:
             self._user_may_join_room_callbacks.append(user_may_join_room)
 
@@ -267,7 +242,9 @@ class SpamChecker:
         if check_media_file_for_spam is not None:
             self._check_media_file_for_spam_callbacks.append(check_media_file_for_spam)
 
-    async def check_event_for_spam(self, event: "synapse.events.EventBase") -> str:
+    async def check_event_for_spam(
+        self, event: "synapse.events.EventBase"
+    ) -> Union[bool, str]:
         """Checks if a given event is considered "spammy" by this server.
 
         If the server considers an event spammy, then it will be rejected if
@@ -278,61 +255,10 @@ class SpamChecker:
             event: the event to be checked
 
         Returns:
-            - `NOT_SPAM` if the event is considered good (non-spammy) and should be let
-                through. Other spamcheck filters may still reject it.
-            - A `Code` if the event is considered spammy and is rejected with a specific
-                error message/code.
-            - A string that isn't `NOT_SPAM` if the event is considered spammy and the
-                string should be used as the client-facing error message. This usage is
-                generally discouraged as it doesn't support internationalization.
+            True or a string if the event is spammy. If a string is returned it
+            will be used as the error message returned to the user.
         """
         for callback in self._check_event_for_spam_callbacks:
-            with Measure(
-                self.clock, "{}.{}".format(callback.__module__, callback.__qualname__)
-            ):
-                res = await delay_cancellation(callback(event))
-                if res is False or res == self.NOT_SPAM:
-                    # This spam-checker accepts the event.
-                    # Other spam-checkers may reject it, though.
-                    continue
-                elif res is True:
-                    # This spam-checker rejects the event with deprecated
-                    # return value `True`
-                    return Codes.FORBIDDEN
-                elif not isinstance(res, str):
-                    # mypy complains that we can't reach this code because of the
-                    # return type in CHECK_EVENT_FOR_SPAM_CALLBACK, but we don't know
-                    # for sure that the module actually returns it.
-                    logger.warning(  # type: ignore[unreachable]
-                        "Module returned invalid value, rejecting message as spam"
-                    )
-                    res = "This message has been rejected as probable spam"
-                else:
-                    # The module rejected the event either with a `Codes`
-                    # or some other `str`. In either case, we stop here.
-                    pass
-
-                return res
-
-        # No spam-checker has rejected the event, let it pass.
-        return self.NOT_SPAM
-
-    async def should_drop_federated_event(
-        self, event: "synapse.events.EventBase"
-    ) -> Union[bool, str]:
-        """Checks if a given federated event is considered "spammy" by this
-        server.
-
-        If the server considers an event spammy, it will be silently dropped,
-        and in doing so will split-brain our view of the room's DAG.
-
-        Args:
-            event: the event to be checked
-
-        Returns:
-            True if the event should be silently dropped
-        """
-        for callback in self._should_drop_federated_event_callbacks:
             with Measure(
                 self.clock, "{}.{}".format(callback.__module__, callback.__qualname__)
             ):

synapse/federation/federation_base.py

@@ -98,9 +98,9 @@ class FederationBase:
                 )
             return redacted_event
 
-        spam_check = await self.spam_checker.check_event_for_spam(pdu)
+        result = await self.spam_checker.check_event_for_spam(pdu)
 
-        if spam_check != self.spam_checker.NOT_SPAM:
+        if result:
             logger.warning("Event contains spam, soft-failing %s", pdu.event_id)
             # we redact (to save disk space) as well as soft-failing (to stop
             # using the event in prev_events).