Fix import order

Adress comments
Merge remote-tracking branch 'origin/develop' into mv/unbind-callback
2022-12-13 10:25:19 +01:00 · 2022-12-12 23:31:09 +01:00 · 2022-12-12 17:48:27 +01:00 · 2022-11-28 12:57:35 +01:00 · 2022-08-03 16:24:44 +02:00 · 2022-08-03 14:52:20 +02:00
89 changed files with 1626 additions and 1976 deletions
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -1,74 +1,3 @@
-Synapse 1.74.0rc1 (2022-12-13)
-==============================
-
-Features
--------
-
- Improve user search for international display names. ([\#14464](https://github.com/matrix-org/synapse/issues/14464))
- Stop using deprecated `keyIds` parameter when calling `/_matrix/key/v2/server`. ([\#14490](https://github.com/matrix-org/synapse/issues/14490), [\#14525](https://github.com/matrix-org/synapse/issues/14525))
- Add new `push.enabled` config option to allow opting out of push notification calculation. ([\#14551](https://github.com/matrix-org/synapse/issues/14551), [\#14619](https://github.com/matrix-org/synapse/issues/14619))
- Advertise support for Matrix 1.5 on `/_matrix/client/versions`. ([\#14576](https://github.com/matrix-org/synapse/issues/14576))
- Improve opentracing and logging for to-device message handling. ([\#14598](https://github.com/matrix-org/synapse/issues/14598))
- Allow selecting "prejoin" events by state keys in addition to event types. ([\#14642](https://github.com/matrix-org/synapse/issues/14642))
-
-
-Bugfixes
--------
-
- Fix a long-standing bug where a device list update might not be sent to clients in certain circumstances. ([\#14435](https://github.com/matrix-org/synapse/issues/14435), [\#14592](https://github.com/matrix-org/synapse/issues/14592), [\#14604](https://github.com/matrix-org/synapse/issues/14604))
- Suppress a spurious warning when `POST /rooms/<room_id>/<membership>/`, `POST /join/<room_id_or_alias`, or the unspecced `PUT /join/<room_id_or_alias>/<txn_id>` receive an empty HTTP request body. ([\#14600](https://github.com/matrix-org/synapse/issues/14600))
- Return spec-compliant JSON errors when unknown endpoints are requested. ([\#14620](https://github.com/matrix-org/synapse/issues/14620), [\#14621](https://github.com/matrix-org/synapse/issues/14621))
- Update html templates to load images over HTTPS. Contributed by @ashfame. ([\#14625](https://github.com/matrix-org/synapse/issues/14625))
- Fix a long-standing bug where the user directory would return 1 more row than requested. ([\#14631](https://github.com/matrix-org/synapse/issues/14631))
- Reject invalid read receipt requests with empty room or event IDs. Contributed by Nick @ Beeper (@fizzadar). ([\#14632](https://github.com/matrix-org/synapse/issues/14632))
- Fix a bug introduced in Synapse 1.67.0 where not specifying a config file or a server URL would lead to the `register_new_matrix_user` script failing. ([\#14637](https://github.com/matrix-org/synapse/issues/14637))
- Fix a long-standing bug where the user directory and room/user stats might be out of sync. ([\#14639](https://github.com/matrix-org/synapse/issues/14639), [\#14643](https://github.com/matrix-org/synapse/issues/14643))
- Fix a bug introduced in Synapse 1.72.0 where the background updates to add non-thread unique indexes on receipts would fail if they were previously interrupted. ([\#14650](https://github.com/matrix-org/synapse/issues/14650))
- Improve validation of field size limits in events. ([\#14664](https://github.com/matrix-org/synapse/issues/14664))
- Fix bugs introduced in Synapse 1.55.0 and 1.69.0 where application services would not be notified of events in the correct rooms, due to stale caches. ([\#14670](https://github.com/matrix-org/synapse/issues/14670))
-
-
-Improved Documentation
----------------------
-
- Update worker settings for `pusher` and `federation_sender` functionality. ([\#14493](https://github.com/matrix-org/synapse/issues/14493))
- Add links to third party package repositories, and point to the bug which highlights Ubuntu's out-of-date packages. ([\#14517](https://github.com/matrix-org/synapse/issues/14517))
- Remove old, incorrect minimum postgres version note and replace with a link to the [Dependency Deprecation Policy](https://matrix-org.github.io/synapse/v1.73/deprecation_policy.html). ([\#14590](https://github.com/matrix-org/synapse/issues/14590))
- Add Single-Sign On setup instructions for Mastodon-based instances. ([\#14594](https://github.com/matrix-org/synapse/issues/14594))
- Change `turn_allow_guests` example value to lowercase `true`. ([\#14634](https://github.com/matrix-org/synapse/issues/14634))
-
-
-Internal Changes
----------------
-
- Optimise push badge count calculations. Contributed by Nick @ Beeper (@fizzadar). ([\#14255](https://github.com/matrix-org/synapse/issues/14255))
- Faster remote room joins: stream the un-partial-stating of rooms over replication. ([\#14473](https://github.com/matrix-org/synapse/issues/14473), [\#14474](https://github.com/matrix-org/synapse/issues/14474))
- Share the `ClientRestResource` for both workers and the main process. ([\#14528](https://github.com/matrix-org/synapse/issues/14528))
- Add `--editable` flag to `complement.sh` which uses an editable install of Synapse for faster turn-around times whilst developing iteratively. ([\#14548](https://github.com/matrix-org/synapse/issues/14548))
- Faster joins: use servers list approximation to send read receipts when in partial state instead of waiting for the full state of the room. ([\#14549](https://github.com/matrix-org/synapse/issues/14549))
- Modernize unit tests configuration related to workers. ([\#14568](https://github.com/matrix-org/synapse/issues/14568))
- Bump jsonschema from 4.17.0 to 4.17.3. ([\#14591](https://github.com/matrix-org/synapse/issues/14591))
- Fix Rust lint CI. ([\#14602](https://github.com/matrix-org/synapse/issues/14602))
- Bump JasonEtco/create-an-issue from 2.5.0 to 2.8.1. ([\#14607](https://github.com/matrix-org/synapse/issues/14607))
- Alter some unit test environment parameters to decrease time spent running tests. ([\#14610](https://github.com/matrix-org/synapse/issues/14610))
- Switch to Go recommended installation method for `gotestfmt` template in CI. ([\#14611](https://github.com/matrix-org/synapse/issues/14611))
- Bump phonenumbers from 8.13.0 to 8.13.1. ([\#14612](https://github.com/matrix-org/synapse/issues/14612))
- Bump types-setuptools from 65.5.0.3 to 65.6.0.1. ([\#14613](https://github.com/matrix-org/synapse/issues/14613))
- Bump twine from 4.0.1 to 4.0.2. ([\#14614](https://github.com/matrix-org/synapse/issues/14614))
- Bump types-requests from 2.28.11.2 to 2.28.11.5. ([\#14615](https://github.com/matrix-org/synapse/issues/14615))
- Bump cryptography from 38.0.3 to 38.0.4. ([\#14616](https://github.com/matrix-org/synapse/issues/14616))
- Remove useless cargo install with apt from Dockerfile. ([\#14636](https://github.com/matrix-org/synapse/issues/14636))
- Bump certifi from 2021.10.8 to 2022.12.7. ([\#14645](https://github.com/matrix-org/synapse/issues/14645))
- Bump flake8-bugbear from 22.10.27 to 22.12.6. ([\#14656](https://github.com/matrix-org/synapse/issues/14656))
- Bump packaging from 21.3 to 22.0. ([\#14657](https://github.com/matrix-org/synapse/issues/14657))
- Bump types-pillow from 9.3.0.1 to 9.3.0.4. ([\#14658](https://github.com/matrix-org/synapse/issues/14658))
- Bump serde from 1.0.148 to 1.0.150. ([\#14659](https://github.com/matrix-org/synapse/issues/14659))
- Bump phonenumbers from 8.13.1 to 8.13.2. ([\#14660](https://github.com/matrix-org/synapse/issues/14660))
- Bump authlib from 1.1.0 to 1.2.0. ([\#14661](https://github.com/matrix-org/synapse/issues/14661))
- Move `StateFilter` to `synapse.types`. ([\#14668](https://github.com/matrix-org/synapse/issues/14668))
- Improve type hints. ([\#14597](https://github.com/matrix-org/synapse/issues/14597), [\#14646](https://github.com/matrix-org/synapse/issues/14646), [\#14671](https://github.com/matrix-org/synapse/issues/14671))
-
-
 Synapse 1.73.0 (2022-12-06)
 ===========================

--- a/changelog.d/13227.feature
+++ b/changelog.d/13227.feature
@@ -0,0 +1 @@
+Add a module callback for unbinding a 3PID.
--- a/changelog.d/14255.misc
+++ b/changelog.d/14255.misc
@@ -0,0 +1 @@
+Optimise push badge count calculations. Contributed by Nick @ Beeper (@fizzadar).
--- a/changelog.d/14435.bugfix
+++ b/changelog.d/14435.bugfix
@@ -0,0 +1 @@
+Fix a long-standing bug where a device list update might not be sent to clients in certain circumstances.
--- a/changelog.d/14464.feature
+++ b/changelog.d/14464.feature
@@ -0,0 +1 @@
+Improve user search for international display names.
--- a/changelog.d/14473.misc
+++ b/changelog.d/14473.misc
@@ -0,0 +1 @@
+Faster remote room joins: stream the un-partial-stating of rooms over replication.
--- a/changelog.d/14474.misc
+++ b/changelog.d/14474.misc
@@ -0,0 +1 @@
+Faster remote room joins: stream the un-partial-stating of rooms over replication.
--- a/changelog.d/14490.feature
+++ b/changelog.d/14490.feature
@@ -0,0 +1 @@
+Stop using deprecated `keyIds` parameter when calling `/_matrix/key/v2/server`.
--- a/changelog.d/14493.doc
+++ b/changelog.d/14493.doc
@@ -0,0 +1 @@
+Update worker settings for `pusher` and `federation_sender` functionality.
--- a/changelog.d/14517.doc
+++ b/changelog.d/14517.doc
@@ -0,0 +1 @@
+Add links to third party package repositories, and point to the bug which highlights Ubuntu's out-of-date packages.
--- a/changelog.d/14525.feature
+++ b/changelog.d/14525.feature
@@ -0,0 +1 @@
+Stop using deprecated `keyIds` parameter when calling `/_matrix/key/v2/server`.
--- a/changelog.d/14528.misc
+++ b/changelog.d/14528.misc
@@ -0,0 +1 @@
+Share the `ClientRestResource` for both workers and the main process.
--- a/changelog.d/14548.misc
+++ b/changelog.d/14548.misc
@@ -0,0 +1 @@
+Add `--editable` flag to `complement.sh` which uses an editable install of Synapse for faster turn-around times whilst developing iteratively.
--- a/changelog.d/14549.misc
+++ b/changelog.d/14549.misc
@@ -0,0 +1 @@
+Faster joins: use servers list approximation to send read receipts when in partial state instead of waiting for the full state of the room.
--- a/changelog.d/14551.feature
+++ b/changelog.d/14551.feature
@@ -0,0 +1 @@
+Add new `push.enabled` config option to allow opting out of push notification calculation.
--- a/changelog.d/14568.misc
+++ b/changelog.d/14568.misc
@@ -0,0 +1 @@
+Modernize unit tests configuration related to workers.
--- a/changelog.d/14576.feature
+++ b/changelog.d/14576.feature
@@ -0,0 +1 @@
+Advertise support for Matrix 1.5 on `/_matrix/client/versions`.
--- a/changelog.d/14590.doc
+++ b/changelog.d/14590.doc
@@ -0,0 +1 @@
+Remove old, incorrect minimum postgres version note and replace with a link to the [Dependency Deprecation Policy](https://matrix-org.github.io/synapse/v1.73/deprecation_policy.html).
--- a/changelog.d/14591.misc
+++ b/changelog.d/14591.misc
@@ -0,0 +1 @@
+Bump jsonschema from 4.17.0 to 4.17.3.
--- a/changelog.d/14592.bugfix
+++ b/changelog.d/14592.bugfix
@@ -0,0 +1 @@
+Fix a long-standing bug where a device list update might not be sent to clients in certain circumstances.
--- a/changelog.d/14594.doc
+++ b/changelog.d/14594.doc
@@ -0,0 +1 @@
+Add Single-Sign On setup instructions for Mastodon-based instances.
--- a/changelog.d/14597.misc
+++ b/changelog.d/14597.misc
@@ -0,0 +1 @@
+Add missing type hints.
--- a/changelog.d/14598.feature
+++ b/changelog.d/14598.feature
@@ -0,0 +1 @@
+Improve opentracing and logging for to-device message handling.
--- a/changelog.d/14600.bugfix
+++ b/changelog.d/14600.bugfix
@@ -0,0 +1 @@
+Suppress a spurious warning when `POST /rooms/<room_id>/<membership>/`, `POST /join/<room_id_or_alias`, or the unspecced `PUT /join/<room_id_or_alias>/<txn_id>` receive an empty HTTP request body.
--- a/changelog.d/14602.misc
+++ b/changelog.d/14602.misc
@@ -0,0 +1 @@
+Fix Rust lint CI.
--- a/changelog.d/14604.bugfix
+++ b/changelog.d/14604.bugfix
@@ -0,0 +1 @@
+Fix a long-standing bug where a device list update might not be sent to clients in certain circumstances.
--- a/changelog.d/14607.misc
+++ b/changelog.d/14607.misc
@@ -0,0 +1 @@
+Bump JasonEtco/create-an-issue from 2.5.0 to 2.8.1.
--- a/changelog.d/14610.misc
+++ b/changelog.d/14610.misc
@@ -0,0 +1 @@
+Alter some unit test environment parameters to decrease time spent running tests.
--- a/changelog.d/14611.misc
+++ b/changelog.d/14611.misc
@@ -0,0 +1 @@
+Switch to Go recommended installation method for `gotestfmt` template in CI.
--- a/changelog.d/14612.misc
+++ b/changelog.d/14612.misc
@@ -0,0 +1 @@
+Bump phonenumbers from 8.13.0 to 8.13.1.
--- a/changelog.d/14613.misc
+++ b/changelog.d/14613.misc
@@ -0,0 +1 @@
+Bump types-setuptools from 65.5.0.3 to 65.6.0.1.
--- a/changelog.d/14614.misc
+++ b/changelog.d/14614.misc
@@ -0,0 +1 @@
+Bump twine from 4.0.1 to 4.0.2.
--- a/changelog.d/14615.misc
+++ b/changelog.d/14615.misc
@@ -0,0 +1 @@
+Bump types-requests from 2.28.11.2 to 2.28.11.5.
--- a/changelog.d/14616.misc
+++ b/changelog.d/14616.misc
@@ -0,0 +1 @@
+Bump cryptography from 38.0.3 to 38.0.4.
--- a/changelog.d/14619.doc
+++ b/changelog.d/14619.doc
@@ -0,0 +1 @@
+Add new `push.enabled` config option to allow opting out of push notification calculation.
--- a/changelog.d/14620.bugfix
+++ b/changelog.d/14620.bugfix
@@ -0,0 +1 @@
+Return spec-compliant JSON errors when unknown endpoints are requested.
--- a/changelog.d/14621.bugfix
+++ b/changelog.d/14621.bugfix
@@ -0,0 +1 @@
+Return spec-compliant JSON errors when unknown endpoints are requested.
--- a/changelog.d/14625.bugfix
+++ b/changelog.d/14625.bugfix
@@ -0,0 +1 @@
+Fix html templates to load images only on HTTPS. Contributed by @ashfame.
--- a/changelog.d/14631.bugfix
+++ b/changelog.d/14631.bugfix
@@ -0,0 +1 @@
+Fix a long-standing bug where the user directory would return 1 more row than requested.
--- a/changelog.d/14632.bugfix
+++ b/changelog.d/14632.bugfix
@@ -0,0 +1 @@
+Reject invalid read receipt requests with empty room or event IDs. Contributed by Nick @ Beeper (@fizzadar).
--- a/changelog.d/14634.doc
+++ b/changelog.d/14634.doc
@@ -0,0 +1 @@
+Change `turn_allow_guests` example value to lowercase `true`.
--- a/changelog.d/14636.misc
+++ b/changelog.d/14636.misc
@@ -0,0 +1 @@
+Remove useless cargo install with apt from Dockerfile.
--- a/changelog.d/14637.bugfix
+++ b/changelog.d/14637.bugfix
@@ -0,0 +1 @@
+Fix a bug introduced in v1.67.0 where not specifying a config file or a server URL would lead to the `register_new_matrix_user` script failing.
--- a/changelog.d/14639.bugfix
+++ b/changelog.d/14639.bugfix
@@ -0,0 +1 @@
+Fix a long-standing bug where the user directory and room/user stats might be out of sync.
--- a/changelog.d/14643.bugfix
+++ b/changelog.d/14643.bugfix
@@ -0,0 +1 @@
+Fix a long-standing bug where the user directory and room/user stats might be out of sync.
--- a/changelog.d/14645.misc
+++ b/changelog.d/14645.misc
@@ -0,0 +1 @@
+Bump certifi from 2021.10.8 to 2022.12.7.
--- a/changelog.d/14646.misc
+++ b/changelog.d/14646.misc
@@ -0,0 +1 @@
+Add missing type hints.
--- a/changelog.d/14650.bugfix
+++ b/changelog.d/14650.bugfix
@@ -0,0 +1,2 @@
+Fix a bug introduced in Synapse 1.72.0 where the background updates to add non-thread unique indexes on receipts would fail if they were previously interrupted.
+
--- a/changelog.d/14656.misc
+++ b/changelog.d/14656.misc
@@ -0,0 +1 @@
+Bump flake8-bugbear from 22.10.27 to 22.12.6.
--- a/changelog.d/14657.misc
+++ b/changelog.d/14657.misc
@@ -0,0 +1 @@
+Bump packaging from 21.3 to 22.0.
--- a/changelog.d/14658.misc
+++ b/changelog.d/14658.misc
@@ -0,0 +1 @@
+Bump types-pillow from 9.3.0.1 to 9.3.0.4.
--- a/changelog.d/14659.misc
+++ b/changelog.d/14659.misc
@@ -0,0 +1 @@
+Bump serde from 1.0.148 to 1.0.150.
--- a/changelog.d/14660.misc
+++ b/changelog.d/14660.misc
@@ -0,0 +1 @@
+Bump phonenumbers from 8.13.1 to 8.13.2.
--- a/changelog.d/14661.misc
+++ b/changelog.d/14661.misc
@@ -0,0 +1 @@
+Bump authlib from 1.1.0 to 1.2.0.
--- a/changelog.d/14662.removal
+++ b/changelog.d/14662.removal
@@ -0,0 +1 @@
+(remove from changelog: unreleased) Revert the deletion of stale devices due to performance issues.
--- a/changelog.d/14668.misc
+++ b/changelog.d/14668.misc
@@ -0,0 +1 @@
+Move `StateFilter` to `synapse.types`.
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,10 +1,9 @@
-matrix-synapse-py3 (1.74.0~rc1) stable; urgency=medium
+matrix-synapse-py3 (1.74.0~rc1) UNRELEASED; urgency=medium

  * New dependency on libicu-dev to provide improved results for user
    search.
-  * New Synapse release 1.74.0rc1.

- -- Synapse Packaging team <packages@matrix.org>  Tue, 13 Dec 2022 13:30:01 +0000
+ -- Synapse Packaging team <packages@matrix.org>  Tue, 06 Dec 2022 15:28:10 +0000

 matrix-synapse-py3 (1.73.0) stable; urgency=medium

--- a/docs/modules/third_party_rules_callbacks.md
+++ b/docs/modules/third_party_rules_callbacks.md
@@ -265,6 +265,33 @@ server_.

 If multiple modules implement this callback, Synapse runs them all in order.

+### `unbind_threepid`
+
+_First introduced in Synapse v1.74.0_
+
+```python
+async def unbind_threepid(
+    user_id: str, medium: str, address: str, identity_server: str
+) -> Tuple[bool, bool]:
+```
+
+Called before a threepid association is removed.
+
+The module is given the Matrix ID of the user to which an association is to be removed,
+as well as the medium (`email` or `msisdn`), address of the third-party identifier and
+the identity server where the threepid was successfully registered.
+
+A module can hence do its own custom unbinding, if for example it did also registered a custom
+binding logic with `on_threepid_bind`.
+
+It should return a tuple of 2 booleans:
+- first one should be `True` on a success calling the identity server, otherwise `False` if
+the identity server doesn't support unbinding (or no identity server found to contact).
+- second one should be `True` if unbind needs to stop there. In this case no other module
+unbind will be called, and the default unbind made to the IS that was used on bind will also be
+skipped. In any case the mapping will be removed from the Synapse 3pid remote table,
+except if an Exception was raised at some point.
+
 ## Example

 The example below is a module that implements the third-party rules callback
--- a/docs/usage/configuration/config_documentation.md
+++ b/docs/usage/configuration/config_documentation.md
@@ -2501,53 +2501,32 @@ Config settings related to the client/server API
 ---
 ### `room_prejoin_state`

-This setting controls the state that is shared with users upon receiving an
-invite to a room, or in reply to a knock on a room. By default, the following
-state events are shared with users:
-
- `m.room.join_rules`
- `m.room.canonical_alias`
- `m.room.avatar`
- `m.room.encryption`
- `m.room.name`
- `m.room.create`
- `m.room.topic`
+Controls for the state that is shared with users who receive an invite
+to a room. By default, the following state event types are shared with users who
+receive invites to the room:
+- m.room.join_rules
+- m.room.canonical_alias
+- m.room.avatar
+- m.room.encryption
+- m.room.name
+- m.room.create
+- m.room.topic

 To change the default behavior, use the following sub-options:
-* `disable_default_event_types`: boolean. Set to `true` to disable the above 
-  defaults. If this is enabled, only the event types listed in
-  `additional_event_types` are shared. Defaults to `false`.
-* `additional_event_types`: A list of additional state events to include in the 
-  events to be shared. By default, this list is empty (so only the default event 
-  types are shared).
-
-  Each entry in this list should be either a single string or a list of two
-  strings. 
-  * A standalone string `t` represents all events with type `t` (i.e.
-    with no restrictions on state keys).
-  * A pair of strings `[t, s]` represents a single event with type `t` and 
-    state key `s`. The same type can appear in two entries with different state
-    keys: in this situation, both state keys are included in prejoin state.
+* `disable_default_event_types`: set to true to disable the above defaults. If this
+   is enabled, only the event types listed in `additional_event_types` are shared.
+   Defaults to false.
+* `additional_event_types`: Additional state event types to share with users when they are invited
+   to a room. By default, this list is empty (so only the default event types are shared).

 Example configuration:
 ```yaml
 room_prejoin_state:
-   disable_default_event_types: false
+   disable_default_event_types: true
   additional_event_types:
-     # Share all events of type `org.example.custom.event.typeA`
-     - org.example.custom.event.typeA
-     # Share only events of type `org.example.custom.event.typeB` whose
-     # state_key is "foo"
-     - ["org.example.custom.event.typeB", "foo"]
-     # Share only events of type `org.example.custom.event.typeC` whose
-     # state_key is "bar" or "baz"
-     - ["org.example.custom.event.typeC", "bar"]
-     - ["org.example.custom.event.typeC", "baz"]
+     - org.example.custom.event.type
+     - m.room.join_rules
 ```
-
-*Changed in Synapse 1.74:* admins can filter the events in prejoin state based
-on their state key.
-
 ---
 ### `track_puppeted_user_ips`

--- a/mypy.ini
+++ b/mypy.ini
@@ -12,7 +12,6 @@ local_partial_types = True
 no_implicit_optional = True
 disallow_untyped_defs = True
 strict_equality = True
-warn_redundant_casts = True

 files =
  docker/,
@@ -89,12 +88,6 @@ disallow_untyped_defs = False
 [mypy-tests.*]
 disallow_untyped_defs = False

-[mypy-tests.config.test_api]
-disallow_untyped_defs = True
-
-[mypy-tests.federation.transport.test_client]
-disallow_untyped_defs = True
-
 [mypy-tests.handlers.test_sso]
 disallow_untyped_defs = True

@@ -107,7 +100,7 @@ disallow_untyped_defs = True
 [mypy-tests.push.test_bulk_push_rule_evaluator]
 disallow_untyped_defs = True

-[mypy-tests.rest.*]
+[mypy-tests.test_server]
 disallow_untyped_defs = True

 [mypy-tests.state.test_profile]
@@ -116,10 +109,10 @@ disallow_untyped_defs = True
 [mypy-tests.storage.*]
 disallow_untyped_defs = True

-[mypy-tests.test_server]
+[mypy-tests.rest.*]
 disallow_untyped_defs = True

-[mypy-tests.types.*]
+[mypy-tests.federation.transport.test_client]
 disallow_untyped_defs = True

 [mypy-tests.util.caches.*]
--- a/poetry.lock
+++ b/poetry.lock
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -57,7 +57,7 @@ manifest-path = "rust/Cargo.toml"

 [tool.poetry]
 name = "matrix-synapse"
-version = "1.74.0rc1"
+version = "1.73.0"
 description = "Homeserver for the Matrix decentralised comms protocol"
 authors = ["Matrix.org Team and Contributors <packages@matrix.org>"]
 license = "Apache-2.0"
@@ -209,7 +209,6 @@ Pympler = { version = "*", optional = true }
 parameterized = { version = ">=0.7.4", optional = true }
 idna = { version = ">=2.5", optional = true }
 pyicu = { version = ">=2.10.2", optional = true }
-uvloop = { version = ">=0.17.0", optional = true }

 [tool.poetry.extras]
 # NB: Packages that should be part of `pip install matrix-synapse[all]` need to be specified
@@ -236,7 +235,6 @@ test = ["parameterized", "idna"]
 # requires libicu's development headers installed on the system (e.g. libicu-dev on
 # Debian-based distributions).
 user-search = ["pyicu"]
-uvloop = ["uvloop"]

 # The duplication here is awful. I hate hate hate hate hate it. However, for now I want
 # to ensure you can still `pip install matrix-synapse[all]` like today. Two motivations:
@@ -270,8 +268,6 @@ all = [
    "pympler",
    # improved user search
    "pyicu",
-    # uvloop
-    "uvloop",
    # omitted:
    #   - test: it's useful to have this separate from dev deps in the olddeps job
    #   - systemd: this is a system-based requirement
--- a/scripts-dev/release.py
+++ b/scripts-dev/release.py
@@ -27,7 +27,7 @@ import time
 import urllib.request
 from os import path
 from tempfile import TemporaryDirectory
-from typing import Any, List, Optional
+from typing import Any, List, Optional, cast

 import attr
 import click
@@ -174,7 +174,9 @@ def _prepare() -> None:
        click.get_current_context().abort()

    # Switch to the release branch.
-    parsed_new_version = version.parse(new_version)
+    # Cast safety: parse() won't return a version.LegacyVersion from our
+    # version string format.
+    parsed_new_version = cast(version.Version, version.parse(new_version))

    # We assume for debian changelogs that we only do RCs or full releases.
    assert not parsed_new_version.is_devrelease
--- a/stubs/synapse/synapse_rust/push.pyi
+++ b/stubs/synapse/synapse_rust/push.pyi
@@ -45,7 +45,7 @@ class PushRuleEvaluator:
        notification_power_levels: Mapping[str, int],
        related_events_flattened: Mapping[str, Mapping[str, str]],
        related_event_match_enabled: bool,
-        room_version_feature_flags: Tuple[str, ...],
+        room_version_feature_flags: list[str],
        msc3931_enabled: bool,
    ): ...
    def run(
--- a/synapse/init.py
+++ b/synapse/init.py
@@ -44,15 +44,8 @@ if strtobool(os.environ.get("SYNAPSE_ASYNC_IO_REACTOR", "0")):

    from twisted.internet import asyncioreactor

-    if bool(os.environ.get("SYNAPSE_UVLOOP", False)):
-        import uvloop
-
-        uvloop.install()
-        print("Using uvloop")
-
    asyncioreactor.install(asyncio.get_event_loop())

-
 # Twisted and canonicaljson will fail to import when this file is executed to
 # get the __version__ during a fresh install. That's OK and subsequent calls to
 # actually start Synapse will import these libraries fine.
--- a/synapse/api/constants.py
+++ b/synapse/api/constants.py
@@ -152,7 +152,6 @@ class EduTypes:

 class RejectedReason:
    AUTH_ERROR: Final = "auth_error"
-    OVERSIZED_EVENT: Final = "oversized_event"


 class RoomCreationPreset:
--- a/synapse/api/errors.py
+++ b/synapse/api/errors.py
@@ -424,17 +424,8 @@ class ResourceLimitError(SynapseError):
 class EventSizeError(SynapseError):
    """An error raised when an event is too big."""

-    def __init__(self, msg: str, unpersistable: bool):
-        """
-        unpersistable:
-            if True, the PDU must not be persisted, not even as a rejected PDU
-            when received over federation.
-            This is notably true when the entire PDU exceeds the size limit for a PDU,
-            (as opposed to an individual key's size limit being exceeded).
-        """
-
+    def __init__(self, msg: str):
        super().__init__(413, msg, Codes.TOO_LARGE)
-        self.unpersistable = unpersistable


 class LoginError(SynapseError):
--- a/synapse/api/room_versions.py
+++ b/synapse/api/room_versions.py
@@ -12,7 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.

-from typing import Callable, Dict, Optional, Tuple
+from typing import Callable, Dict, List, Optional

 import attr

@@ -103,7 +103,7 @@ class RoomVersion:
    # is not enough to mark it "supported": the push rule evaluator also needs to
    # support the flag. Unknown flags are ignored by the evaluator, making conditions
    # fail if used.
-    msc3931_push_features: Tuple[str, ...]  # values from PushRuleRoomFlag
+    msc3931_push_features: List[str]  # values from PushRuleRoomFlag


 class RoomVersions:
@@ -124,7 +124,7 @@ class RoomVersions:
        msc2716_redactions=False,
        msc3787_knock_restricted_join_rule=False,
        msc3667_int_only_power_levels=False,
-        msc3931_push_features=(),
+        msc3931_push_features=[],
    )
    V2 = RoomVersion(
        "2",
@@ -143,7 +143,7 @@ class RoomVersions:
        msc2716_redactions=False,
        msc3787_knock_restricted_join_rule=False,
        msc3667_int_only_power_levels=False,
-        msc3931_push_features=(),
+        msc3931_push_features=[],
    )
    V3 = RoomVersion(
        "3",
@@ -162,7 +162,7 @@ class RoomVersions:
        msc2716_redactions=False,
        msc3787_knock_restricted_join_rule=False,
        msc3667_int_only_power_levels=False,
-        msc3931_push_features=(),
+        msc3931_push_features=[],
    )
    V4 = RoomVersion(
        "4",
@@ -181,7 +181,7 @@ class RoomVersions:
        msc2716_redactions=False,
        msc3787_knock_restricted_join_rule=False,
        msc3667_int_only_power_levels=False,
-        msc3931_push_features=(),
+        msc3931_push_features=[],
    )
    V5 = RoomVersion(
        "5",
@@ -200,7 +200,7 @@ class RoomVersions:
        msc2716_redactions=False,
        msc3787_knock_restricted_join_rule=False,
        msc3667_int_only_power_levels=False,
-        msc3931_push_features=(),
+        msc3931_push_features=[],
    )
    V6 = RoomVersion(
        "6",
@@ -219,7 +219,7 @@ class RoomVersions:
        msc2716_redactions=False,
        msc3787_knock_restricted_join_rule=False,
        msc3667_int_only_power_levels=False,
-        msc3931_push_features=(),
+        msc3931_push_features=[],
    )
    MSC2176 = RoomVersion(
        "org.matrix.msc2176",
@@ -238,7 +238,7 @@ class RoomVersions:
        msc2716_redactions=False,
        msc3787_knock_restricted_join_rule=False,
        msc3667_int_only_power_levels=False,
-        msc3931_push_features=(),
+        msc3931_push_features=[],
    )
    V7 = RoomVersion(
        "7",
@@ -257,7 +257,7 @@ class RoomVersions:
        msc2716_redactions=False,
        msc3787_knock_restricted_join_rule=False,
        msc3667_int_only_power_levels=False,
-        msc3931_push_features=(),
+        msc3931_push_features=[],
    )
    V8 = RoomVersion(
        "8",
@@ -276,7 +276,7 @@ class RoomVersions:
        msc2716_redactions=False,
        msc3787_knock_restricted_join_rule=False,
        msc3667_int_only_power_levels=False,
-        msc3931_push_features=(),
+        msc3931_push_features=[],
    )
    V9 = RoomVersion(
        "9",
@@ -295,7 +295,7 @@ class RoomVersions:
        msc2716_redactions=False,
        msc3787_knock_restricted_join_rule=False,
        msc3667_int_only_power_levels=False,
-        msc3931_push_features=(),
+        msc3931_push_features=[],
    )
    MSC3787 = RoomVersion(
        "org.matrix.msc3787",
@@ -314,7 +314,7 @@ class RoomVersions:
        msc2716_redactions=False,
        msc3787_knock_restricted_join_rule=True,
        msc3667_int_only_power_levels=False,
-        msc3931_push_features=(),
+        msc3931_push_features=[],
    )
    V10 = RoomVersion(
        "10",
@@ -333,7 +333,7 @@ class RoomVersions:
        msc2716_redactions=False,
        msc3787_knock_restricted_join_rule=True,
        msc3667_int_only_power_levels=True,
-        msc3931_push_features=(),
+        msc3931_push_features=[],
    )
    MSC2716v4 = RoomVersion(
        "org.matrix.msc2716v4",
@@ -352,7 +352,7 @@ class RoomVersions:
        msc2716_redactions=True,
        msc3787_knock_restricted_join_rule=False,
        msc3667_int_only_power_levels=False,
-        msc3931_push_features=(),
+        msc3931_push_features=[],
    )
    MSC1767v10 = RoomVersion(
        # MSC1767 (Extensible Events) based on room version "10"
@@ -372,7 +372,7 @@ class RoomVersions:
        msc2716_redactions=False,
        msc3787_knock_restricted_join_rule=True,
        msc3667_int_only_power_levels=True,
-        msc3931_push_features=(PushRuleRoomFlag.EXTENSIBLE_EVENTS,),
+        msc3931_push_features=[PushRuleRoomFlag.EXTENSIBLE_EVENTS],
    )


--- a/synapse/appservice/init.py
+++ b/synapse/appservice/init.py
@@ -245,9 +245,7 @@ class ApplicationService:
            return True

        # likewise with the room's aliases (if it has any)
-        alias_list = await store.get_aliases_for_room(
-            room_id, on_invalidate=cache_context.invalidate
-        )
+        alias_list = await store.get_aliases_for_room(room_id)
        for alias in alias_list:
            if self.is_room_alias_in_namespace(alias):
                return True
@@ -313,9 +311,7 @@ class ApplicationService:
        # Find all the rooms the sender is in
        if self.is_interested_in_user(user_id.to_string()):
            return True
-        room_ids = await store.get_rooms_for_user(
-            user_id.to_string(), on_invalidate=cache_context.invalidate
-        )
+        room_ids = await store.get_rooms_for_user(user_id.to_string())

        # Then find out if the appservice is interested in any of those rooms
        for room_id in room_ids:
--- a/synapse/config/_util.py
+++ b/synapse/config/_util.py
@@ -33,9 +33,6 @@ def validate_config(
        config: the configuration value to be validated
        config_path: the path within the config file. This will be used as a basis
           for the error message.
-
-    Raises:
-        ConfigError, if validation fails.
    """
    try:
        jsonschema.validate(config, json_schema)
--- a/synapse/config/api.py
+++ b/synapse/config/api.py
@@ -13,13 +13,12 @@
 # limitations under the License.

 import logging
-from typing import Any, Iterable, Optional, Tuple
+from typing import Any, Iterable

 from synapse.api.constants import EventTypes
 from synapse.config._base import Config, ConfigError
 from synapse.config._util import validate_config
 from synapse.types import JsonDict
-from synapse.types.state import StateFilter

 logger = logging.getLogger(__name__)

@@ -27,20 +26,16 @@ logger = logging.getLogger(__name__)
 class ApiConfig(Config):
    section = "api"

-    room_prejoin_state: StateFilter
-    track_puppetted_users_ips: bool
-
    def read_config(self, config: JsonDict, **kwargs: Any) -> None:
        validate_config(_MAIN_SCHEMA, config, ())
-        self.room_prejoin_state = StateFilter.from_types(
-            self._get_prejoin_state_entries(config)
-        )
+        self.room_prejoin_state = list(self._get_prejoin_state_types(config))
        self.track_puppeted_user_ips = config.get("track_puppeted_user_ips", False)

-    def _get_prejoin_state_entries(
-        self, config: JsonDict
-    ) -> Iterable[Tuple[str, Optional[str]]]:
-        """Get the event types and state keys to include in the prejoin state."""
+    def _get_prejoin_state_types(self, config: JsonDict) -> Iterable[str]:
+        """Get the event types to include in the prejoin state
+
+        Parses the config and returns an iterable of the event types to be included.
+        """
        room_prejoin_state_config = config.get("room_prejoin_state") or {}

        # backwards-compatibility support for room_invite_state_types
@@ -55,39 +50,33 @@ class ApiConfig(Config):

            logger.warning(_ROOM_INVITE_STATE_TYPES_WARNING)

-            for event_type in config["room_invite_state_types"]:
-                yield event_type, None
+            yield from config["room_invite_state_types"]
            return

        if not room_prejoin_state_config.get("disable_default_event_types"):
-            yield from _DEFAULT_PREJOIN_STATE_TYPES_AND_STATE_KEYS
+            yield from _DEFAULT_PREJOIN_STATE_TYPES

-        for entry in room_prejoin_state_config.get("additional_event_types", []):
-            if isinstance(entry, str):
-                yield entry, None
-            else:
-                yield entry
+        yield from room_prejoin_state_config.get("additional_event_types", [])


 _ROOM_INVITE_STATE_TYPES_WARNING = """\
 WARNING: The 'room_invite_state_types' configuration setting is now deprecated,
 and replaced with 'room_prejoin_state'. New features may not work correctly
-unless 'room_invite_state_types' is removed. See the config documentation at
-    https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html#room_prejoin_state
-for details of 'room_prejoin_state'.
+unless 'room_invite_state_types' is removed. See the sample configuration file for
+details of 'room_prejoin_state'.
 --------------------------------------------------------------------------------
 """

-_DEFAULT_PREJOIN_STATE_TYPES_AND_STATE_KEYS = [
-    (EventTypes.JoinRules, ""),
-    (EventTypes.CanonicalAlias, ""),
-    (EventTypes.RoomAvatar, ""),
-    (EventTypes.RoomEncryption, ""),
-    (EventTypes.Name, ""),
+_DEFAULT_PREJOIN_STATE_TYPES = [
+    EventTypes.JoinRules,
+    EventTypes.CanonicalAlias,
+    EventTypes.RoomAvatar,
+    EventTypes.RoomEncryption,
+    EventTypes.Name,
    # Per MSC1772.
-    (EventTypes.Create, ""),
+    EventTypes.Create,
    # Per MSC3173.
-    (EventTypes.Topic, ""),
+    EventTypes.Topic,
 ]


@@ -101,17 +90,7 @@ _ROOM_PREJOIN_STATE_CONFIG_SCHEMA = {
                "disable_default_event_types": {"type": "boolean"},
                "additional_event_types": {
                    "type": "array",
-                    "items": {
-                        "oneOf": [
-                            {"type": "string"},
-                            {
-                                "type": "array",
-                                "items": {"type": "string"},
-                                "minItems": 2,
-                                "maxItems": 2,
-                            },
-                        ],
-                    },
+                    "items": {"type": "string"},
                },
            },
        },
--- a/synapse/event_auth.py
+++ b/synapse/event_auth.py
@@ -52,7 +52,6 @@ from synapse.api.room_versions import (
    KNOWN_ROOM_VERSIONS,
    EventFormatVersions,
    RoomVersion,
-    RoomVersions,
 )
 from synapse.storage.databases.main.events_worker import EventRedactBehaviour
 from synapse.types import MutableStateMap, StateMap, UserID, get_domain_from_id
@@ -342,80 +341,19 @@ def check_state_dependent_auth_rules(
    logger.debug("Allowing! %s", event)


-# Set of room versions where Synapse did not apply event key size limits
-# in bytes, but rather in codepoints.
-# In these room versions, we are more lenient with event size validation.
-LENIENT_EVENT_BYTE_LIMITS_ROOM_VERSIONS = {
-    RoomVersions.V1,
-    RoomVersions.V2,
-    RoomVersions.V3,
-    RoomVersions.V4,
-    RoomVersions.V5,
-    RoomVersions.V6,
-    RoomVersions.MSC2176,
-    RoomVersions.V7,
-    RoomVersions.V8,
-    RoomVersions.V9,
-    RoomVersions.MSC3787,
-    RoomVersions.V10,
-    RoomVersions.MSC2716v4,
-    RoomVersions.MSC1767v10,
-}
-
-
 def _check_size_limits(event: "EventBase") -> None:
-    """
-    Checks the size limits in a PDU.
-
-    The entire size limit of the PDU is checked first.
-    Then the size of fields is checked, first in codepoints and then in bytes.
-
-    The codepoint size limits are only for Synapse compatibility.
-
-    Raises:
-        EventSizeError:
-            when a size limit has been violated.
-
-            unpersistable=True if Synapse never would have accepted the event and
-                the PDU must NOT be persisted.
-
-            unpersistable=False if a prior version of Synapse would have accepted the
-                event and so the PDU must be persisted as rejected to avoid
-                breaking the room.
-    """
-
-    # Whole PDU check
-    if len(encode_canonical_json(event.get_pdu_json())) > MAX_PDU_SIZE:
-        raise EventSizeError("event too large", unpersistable=True)
-
-    # Codepoint size check: Synapse always enforced these limits, so apply
-    # them strictly.
    if len(event.user_id) > 255:
-        raise EventSizeError("'user_id' too large", unpersistable=True)
+        raise EventSizeError("'user_id' too large")
    if len(event.room_id) > 255:
-        raise EventSizeError("'room_id' too large", unpersistable=True)
+        raise EventSizeError("'room_id' too large")
    if event.is_state() and len(event.state_key) > 255:
-        raise EventSizeError("'state_key' too large", unpersistable=True)
+        raise EventSizeError("'state_key' too large")
    if len(event.type) > 255:
-        raise EventSizeError("'type' too large", unpersistable=True)
+        raise EventSizeError("'type' too large")
    if len(event.event_id) > 255:
-        raise EventSizeError("'event_id' too large", unpersistable=True)
-
-    strict_byte_limits = (
-        event.room_version not in LENIENT_EVENT_BYTE_LIMITS_ROOM_VERSIONS
-    )
-
-    # Byte size check: if these fail, then be lenient to avoid breaking rooms.
-    if len(event.user_id.encode("utf-8")) > 255:
-        raise EventSizeError("'user_id' too large", unpersistable=strict_byte_limits)
-    if len(event.room_id.encode("utf-8")) > 255:
-        raise EventSizeError("'room_id' too large", unpersistable=strict_byte_limits)
-    if event.is_state() and len(event.state_key.encode("utf-8")) > 255:
-        raise EventSizeError("'state_key' too large", unpersistable=strict_byte_limits)
-    if len(event.type.encode("utf-8")) > 255:
-        raise EventSizeError("'type' too large", unpersistable=strict_byte_limits)
-    if len(event.event_id.encode("utf-8")) > 255:
-        raise EventSizeError("'event_id' too large", unpersistable=strict_byte_limits)
+        raise EventSizeError("'event_id' too large")
+    if len(encode_canonical_json(event.get_pdu_json())) > MAX_PDU_SIZE:
+        raise EventSizeError("event too large")


 def _check_create(event: "EventBase") -> None:
--- a/synapse/events/third_party_rules.py
+++ b/synapse/events/third_party_rules.py
@@ -45,6 +45,7 @@ CHECK_CAN_DEACTIVATE_USER_CALLBACK = Callable[[str, bool], Awaitable[bool]]
 ON_PROFILE_UPDATE_CALLBACK = Callable[[str, ProfileInfo, bool, bool], Awaitable]
 ON_USER_DEACTIVATION_STATUS_CHANGED_CALLBACK = Callable[[str, bool, bool], Awaitable]
 ON_THREEPID_BIND_CALLBACK = Callable[[str, str, str], Awaitable]
+UNBIND_THREEPID_CALLBACK = Callable[[str, str, str, str], Awaitable[Tuple[bool, bool]]]


 def load_legacy_third_party_event_rules(hs: "HomeServer") -> None:
@@ -174,6 +175,7 @@ class ThirdPartyEventRules:
            ON_USER_DEACTIVATION_STATUS_CHANGED_CALLBACK
        ] = []
        self._on_threepid_bind_callbacks: List[ON_THREEPID_BIND_CALLBACK] = []
+        self._unbind_threepid_callbacks: List[UNBIND_THREEPID_CALLBACK] = []

    def register_third_party_rules_callbacks(
        self,
@@ -193,6 +195,7 @@ class ThirdPartyEventRules:
            ON_USER_DEACTIVATION_STATUS_CHANGED_CALLBACK
        ] = None,
        on_threepid_bind: Optional[ON_THREEPID_BIND_CALLBACK] = None,
+        unbind_threepid: Optional[UNBIND_THREEPID_CALLBACK] = None,
    ) -> None:
        """Register callbacks from modules for each hook."""
        if check_event_allowed is not None:
@@ -230,6 +233,9 @@ class ThirdPartyEventRules:
        if on_threepid_bind is not None:
            self._on_threepid_bind_callbacks.append(on_threepid_bind)

+        if unbind_threepid is not None:
+            self._unbind_threepid_callbacks.append(unbind_threepid)
+
    async def check_event_allowed(
        self, event: EventBase, context: EventContext
    ) -> Tuple[bool, Optional[dict]]:
@@ -523,3 +529,41 @@ class ThirdPartyEventRules:
                logger.exception(
                    "Failed to run module API callback %s: %s", callback, e
                )
+
+    async def unbind_threepid(
+        self, user_id: str, medium: str, address: str, identity_server: str
+    ) -> Tuple[bool, bool]:
+        """Called before a threepid association is removed.
+
+        Note that this callback is called before an association is deleted on the
+        local homeserver.
+
+        Args:
+            user_id: the user being associated with the threepid.
+            medium: the threepid's medium.
+            address: the threepid's address.
+            identity_server: the identity server where the threepid was successfully registered.
+
+        Returns:
+            A tuple of 2 booleans reporting if a changed happened for the first, and if unbind
+            needs to stop there for the second (True value). In this case no other module unbind will be
+            called, and the default unbind made to the IS that was used on bind will also be skipped.
+            In any case the mapping will be removed from the Synapse 3pid remote table, except if an Exception
+            was raised at some point.
+        """
+
+        global_changed = False
+        for callback in self._unbind_threepid_callbacks:
+            try:
+                (changed, stop) = await callback(
+                    user_id, medium, address, identity_server
+                )
+                global_changed |= changed
+                if stop:
+                    return global_changed, True
+            except Exception as e:
+                logger.exception(
+                    "Failed to run module API callback %s: %s", callback, e
+                )
+
+        return global_changed, False
--- a/synapse/events/utils.py
+++ b/synapse/events/utils.py
@@ -28,14 +28,8 @@ from typing import (
 )

 import attr
-from canonicaljson import encode_canonical_json

-from synapse.api.constants import (
-    MAX_PDU_SIZE,
-    EventContentFields,
-    EventTypes,
-    RelationTypes,
-)
+from synapse.api.constants import EventContentFields, EventTypes, RelationTypes
 from synapse.api.errors import Codes, SynapseError
 from synapse.api.room_versions import RoomVersion
 from synapse.types import JsonDict
@@ -680,27 +674,3 @@ def validate_canonicaljson(value: Any) -> None:
    elif not isinstance(value, (bool, str)) and value is not None:
        # Other potential JSON values (bool, None, str) are safe.
        raise SynapseError(400, "Unknown JSON value", Codes.BAD_JSON)
-
-
-def maybe_upsert_event_field(
-    event: EventBase, container: JsonDict, key: str, value: object
-) -> bool:
-    """Upsert an event field, but only if this doesn't make the event too large.
-
-    Returns true iff the upsert took place.
-    """
-    if key in container:
-        old_value: object = container[key]
-        container[key] = value
-        # NB: here and below, we assume that passing a non-None `time_now` argument to
-        # get_pdu_json doesn't increase the size of the encoded result.
-        upsert_okay = len(encode_canonical_json(event.get_pdu_json())) <= MAX_PDU_SIZE
-        if not upsert_okay:
-            container[key] = old_value
-    else:
-        container[key] = value
-        upsert_okay = len(encode_canonical_json(event.get_pdu_json())) <= MAX_PDU_SIZE
-        if not upsert_okay:
-            del container[key]
-
-    return upsert_okay
--- a/synapse/handlers/federation_event.py
+++ b/synapse/handlers/federation_event.py
@@ -43,7 +43,6 @@ from synapse.api.constants import (
 from synapse.api.errors import (
    AuthError,
    Codes,
-    EventSizeError,
    FederationError,
    FederationPullAttemptBackoffError,
    HttpResponseException,
@@ -1737,15 +1736,6 @@ class FederationEventHandler:
                except AuthError as e:
                    logger.warning("Rejecting %r because %s", event, e)
                    context.rejected = RejectedReason.AUTH_ERROR
-                except EventSizeError as e:
-                    if e.unpersistable:
-                        # This event is completely unpersistable.
-                        raise e
-                    # Otherwise, we are somewhat lenient and just persist the event
-                    # as rejected, for moderate compatibility with older Synapse
-                    # versions.
-                    logger.warning("While validating received event %r: %s", event, e)
-                    context.rejected = RejectedReason.OVERSIZED_EVENT

            events_and_contexts_to_persist.append((event, context))

@@ -1791,16 +1781,6 @@ class FederationEventHandler:
            # TODO: use a different rejected reason here?
            context.rejected = RejectedReason.AUTH_ERROR
            return
-        except EventSizeError as e:
-            if e.unpersistable:
-                # This event is completely unpersistable.
-                raise e
-            # Otherwise, we are somewhat lenient and just persist the event
-            # as rejected, for moderate compatibility with older Synapse
-            # versions.
-            logger.warning("While validating received event %r: %s", event, e)
-            context.rejected = RejectedReason.OVERSIZED_EVENT
-            return

        # next, check that we have all of the event's auth events.
        #
--- a/synapse/handlers/identity.py
+++ b/synapse/handlers/identity.py
@@ -275,49 +275,64 @@ class IdentityHandler:
            server doesn't support unbinding
        """

-        if not valid_id_server_location(id_server):
-            raise SynapseError(
-                400,
-                "id_server must be a valid hostname with optional port and path components",
-            )
+        medium = threepid["medium"]
+        address = threepid["address"]

-        url = "https://%s/_matrix/identity/v2/3pid/unbind" % (id_server,)
-        url_bytes = b"/_matrix/identity/v2/3pid/unbind"
-
-        content = {
-            "mxid": mxid,
-            "threepid": {"medium": threepid["medium"], "address": threepid["address"]},
-        }
-
-        # we abuse the federation http client to sign the request, but we have to send it
-        # using the normal http client since we don't want the SRV lookup and want normal
-        # 'browser-like' HTTPS.
-        auth_headers = self.federation_http_client.build_auth_headers(
-            destination=None,
-            method=b"POST",
-            url_bytes=url_bytes,
-            content=content,
-            destination_is=id_server.encode("ascii"),
+        (changed, stop,) = await self.hs.get_third_party_event_rules().unbind_threepid(
+            mxid, medium, address, id_server
        )
-        headers = {b"Authorization": auth_headers}

-        try:
-            # Use the blacklisting http client as this call is only to identity servers
-            # provided by a client
-            await self.blacklisting_http_client.post_json_get_json(
-                url, content, headers
+        # If a module wants to take over unbind it will return stop = True,
+        # in this case we should just purge the table from the 3pid record
+        if not stop:
+            if not valid_id_server_location(id_server):
+                raise SynapseError(
+                    400,
+                    "id_server must be a valid hostname with optional port and path components",
+                )
+
+            url = "https://%s/_matrix/identity/v2/3pid/unbind" % (id_server,)
+            url_bytes = b"/_matrix/identity/v2/3pid/unbind"
+
+            content = {
+                "mxid": mxid,
+                "threepid": {
+                    "medium": threepid["medium"],
+                    "address": threepid["address"],
+                },
+            }
+
+            # we abuse the federation http client to sign the request, but we have to send it
+            # using the normal http client since we don't want the SRV lookup and want normal
+            # 'browser-like' HTTPS.
+            auth_headers = self.federation_http_client.build_auth_headers(
+                destination=None,
+                method=b"POST",
+                url_bytes=url_bytes,
+                content=content,
+                destination_is=id_server.encode("ascii"),
            )
-            changed = True
-        except HttpResponseException as e:
-            changed = False
-            if e.code in (400, 404, 501):
-                # The remote server probably doesn't support unbinding (yet)
-                logger.warning("Received %d response while unbinding threepid", e.code)
-            else:
-                logger.error("Failed to unbind threepid on identity server: %s", e)
-                raise SynapseError(500, "Failed to contact identity server")
-        except RequestTimedOutError:
-            raise SynapseError(500, "Timed out contacting identity server")
+            headers = {b"Authorization": auth_headers}
+
+            try:
+                # Use the blacklisting http client as this call is only to identity servers
+                # provided by a client
+                await self.blacklisting_http_client.post_json_get_json(
+                    url, content, headers
+                )
+                changed &= True
+            except HttpResponseException as e:
+                changed &= False
+                if e.code in (400, 404, 501):
+                    # The remote server probably doesn't support unbinding (yet)
+                    logger.warning(
+                        "Received %d response while unbinding threepid", e.code
+                    )
+                else:
+                    logger.error("Failed to unbind threepid on identity server: %s", e)
+                    raise SynapseError(500, "Failed to contact identity server")
+            except RequestTimedOutError:
+                raise SynapseError(500, "Timed out contacting identity server")

        await self.store.remove_user_bound_threepid(
            user_id=mxid,
--- a/synapse/handlers/message.py
+++ b/synapse/handlers/message.py
@@ -50,7 +50,6 @@ from synapse.event_auth import validate_event_for_room_version
 from synapse.events import EventBase, relation_from_event
 from synapse.events.builder import EventBuilder
 from synapse.events.snapshot import EventContext
-from synapse.events.utils import maybe_upsert_event_field
 from synapse.events.validator import EventValidator
 from synapse.handlers.directory import DirectoryHandler
 from synapse.logging import opentracing
@@ -1740,15 +1739,12 @@ class EventCreationHandler:

            if event.type == EventTypes.Member:
                if event.content["membership"] == Membership.INVITE:
-                    maybe_upsert_event_field(
-                        event,
-                        event.unsigned,
-                        "invite_room_state",
-                        await self.store.get_stripped_room_state_from_event_context(
-                            context,
-                            self.room_prejoin_state_types,
-                            membership_user_id=event.sender,
-                        ),
+                    event.unsigned[
+                        "invite_room_state"
+                    ] = await self.store.get_stripped_room_state_from_event_context(
+                        context,
+                        self.room_prejoin_state_types,
+                        membership_user_id=event.sender,
                    )

                    invitee = UserID.from_string(event.state_key)
@@ -1766,14 +1762,11 @@ class EventCreationHandler:
                        event.signatures.update(returned_invite.signatures)

                if event.content["membership"] == Membership.KNOCK:
-                    maybe_upsert_event_field(
-                        event,
-                        event.unsigned,
-                        "knock_room_state",
-                        await self.store.get_stripped_room_state_from_event_context(
-                            context,
-                            self.room_prejoin_state_types,
-                        ),
+                    event.unsigned[
+                        "knock_room_state"
+                    ] = await self.store.get_stripped_room_state_from_event_context(
+                        context,
+                        self.room_prejoin_state_types,
                    )

            if event.type == EventTypes.Redaction:
--- a/synapse/module_api/init.py
+++ b/synapse/module_api/init.py
@@ -68,6 +68,7 @@ from synapse.events.third_party_rules import (
    ON_PROFILE_UPDATE_CALLBACK,
    ON_THREEPID_BIND_CALLBACK,
    ON_USER_DEACTIVATION_STATUS_CHANGED_CALLBACK,
+    UNBIND_THREEPID_CALLBACK,
 )
 from synapse.handlers.account_data import ON_ACCOUNT_DATA_UPDATED_CALLBACK
 from synapse.handlers.account_validity import (
@@ -319,6 +320,7 @@ class ModuleApi:
            ON_USER_DEACTIVATION_STATUS_CHANGED_CALLBACK
        ] = None,
        on_threepid_bind: Optional[ON_THREEPID_BIND_CALLBACK] = None,
+        unbind_threepid: Optional[UNBIND_THREEPID_CALLBACK] = None,
    ) -> None:
        """Registers callbacks for third party event rules capabilities.

@@ -335,6 +337,7 @@ class ModuleApi:
            on_profile_update=on_profile_update,
            on_user_deactivation_status_changed=on_user_deactivation_status_changed,
            on_threepid_bind=on_threepid_bind,
+            unbind_threepid=unbind_threepid,
        )

    def register_presence_router_callbacks(
--- a/synapse/push/bulk_push_rule_evaluator.py
+++ b/synapse/push/bulk_push_rule_evaluator.py
@@ -342,6 +342,10 @@ class BulkPushRuleEvaluator:
            for user_id, level in notification_levels.items():
                notification_levels[user_id] = int(level)

+        room_version_features = event.room_version.msc3931_push_features
+        if not room_version_features:
+            room_version_features = []
+
        evaluator = PushRuleEvaluator(
            _flatten_dict(event, room_version=event.room_version),
            room_member_count,
@@ -349,7 +353,7 @@ class BulkPushRuleEvaluator:
            notification_levels,
            related_events,
            self._related_event_match_enabled,
-            event.room_version.msc3931_push_features,
+            room_version_features,
            self.hs.config.experimental.msc1767_enabled,  # MSC3931 flag
        )

--- a/synapse/storage/database.py
+++ b/synapse/storage/database.py
@@ -667,8 +667,7 @@ class DatabasePool:
                )
        # also check variables referenced in func's closure
        if inspect.isfunction(func):
-            # Keep the cast for now---it helps PyCharm to understand what `func` is.
-            f = cast(types.FunctionType, func)  # type: ignore[redundant-cast]
+            f = cast(types.FunctionType, func)
            if f.__closure__:
                for i, cell in enumerate(f.__closure__):
                    if inspect.isgenerator(cell.cell_contents):
--- a/synapse/storage/databases/main/events_worker.py
+++ b/synapse/storage/databases/main/events_worker.py
@@ -16,11 +16,11 @@ import logging
 import threading
 import weakref
 from enum import Enum, auto
-from itertools import chain
 from typing import (
    TYPE_CHECKING,
    Any,
    Collection,
+    Container,
    Dict,
    Iterable,
    List,
@@ -76,7 +76,6 @@ from synapse.storage.util.id_generators import (
 )
 from synapse.storage.util.sequence import build_sequence_generator
 from synapse.types import JsonDict, get_domain_from_id
-from synapse.types.state import StateFilter
 from synapse.util import unwrapFirstError
 from synapse.util.async_helpers import ObservableDeferred, delay_cancellation
 from synapse.util.caches.descriptors import cached, cachedList
@@ -880,7 +879,7 @@ class EventsWorkerStore(SQLBaseStore):
    async def get_stripped_room_state_from_event_context(
        self,
        context: EventContext,
-        state_keys_to_include: StateFilter,
+        state_types_to_include: Container[str],
        membership_user_id: Optional[str] = None,
    ) -> List[JsonDict]:
        """
@@ -893,7 +892,7 @@ class EventsWorkerStore(SQLBaseStore):

        Args:
            context: The event context to retrieve state of the room from.
-            state_keys_to_include: The state events to include, for each event type.
+            state_types_to_include: The type of state events to include.
            membership_user_id: An optional user ID to include the stripped membership state
                events of. This is useful when generating the stripped state of a room for
                invites. We want to send membership events of the inviter, so that the
@@ -902,25 +901,21 @@ class EventsWorkerStore(SQLBaseStore):
        Returns:
            A list of dictionaries, each representing a stripped state event from the room.
        """
-        if membership_user_id:
-            types = chain(
-                state_keys_to_include.to_types(),
-                [(EventTypes.Member, membership_user_id)],
-            )
-            filter = StateFilter.from_types(types)
-        else:
-            filter = state_keys_to_include
-        selected_state_ids = await context.get_current_state_ids(filter)
+        current_state_ids = await context.get_current_state_ids()

        # We know this event is not an outlier, so this must be
        # non-None.
-        assert selected_state_ids is not None
+        assert current_state_ids is not None

-        # Confusingly, get_current_state_events may return events that are discarded by
-        # the filter, if they're in context._state_delta_due_to_event. Strip these away.
-        selected_state_ids = filter.filter_state(selected_state_ids)
+        # The state to include
+        state_to_include_ids = [
+            e_id
+            for k, e_id in current_state_ids.items()
+            if k[0] in state_types_to_include
+            or (membership_user_id and k == (EventTypes.Member, membership_user_id))
+        ]

-        state_to_include = await self.get_events(selected_state_ids.values())
+        state_to_include = await self.get_events(state_to_include_ids)

        return [
            {
--- a/synapse/storage/engines/postgres.py
+++ b/synapse/storage/engines/postgres.py
@@ -77,7 +77,7 @@ class PostgresEngine(
        # docs: The number is formed by converting the major, minor, and
        # revision numbers into two-decimal-digit numbers and appending them
        # together. For example, version 8.1.5 will be returned as 80105
-        self._version = db_conn.server_version
+        self._version = cast(int, db_conn.server_version)
        allow_unsafe_locale = self.config.get("allow_unsafe_locale", False)

        # Are we on a supported PostgreSQL version?
--- a/synapse/types/state.py
+++ b/synapse/types/state.py
@@ -118,15 +118,6 @@ class StateFilter:
            )
        )

-    def to_types(self) -> Iterable[Tuple[str, Optional[str]]]:
-        """The inverse to `from_types`."""
-        for (event_type, state_keys) in self.types.items():
-            if state_keys is None:
-                yield event_type, None
-            else:
-                for state_key in state_keys:
-                    yield event_type, state_key
-
    @staticmethod
    def from_lazy_load_member_list(members: Iterable[str]) -> "StateFilter":
        """Creates a filter that returns all non-member events, plus the member
@@ -352,15 +343,6 @@ class StateFilter:
            for s in state_keys
        ]

-    def wildcard_types(self) -> List[str]:
-        """Returns a list of event types which require us to fetch all state keys.
-        This will be empty unless `has_wildcards` returns True.
-
-        Returns:
-            A list of event types.
-        """
-        return [t for t, state_keys in self.types.items() if state_keys is None]
-
    def get_member_split(self) -> Tuple["StateFilter", "StateFilter"]:
        """Return the filter split into two: one which assumes it's exclusively
        matching against member state, and one which assumes it's matching
--- a/tests/config/test_api.py
+++ b/tests/config/test_api.py
@@ -1,145 +0,0 @@
-from unittest import TestCase as StdlibTestCase
-
-import yaml
-
-from synapse.config import ConfigError
-from synapse.config.api import ApiConfig
-from synapse.types.state import StateFilter
-
-DEFAULT_PREJOIN_STATE_PAIRS = {
-    ("m.room.join_rules", ""),
-    ("m.room.canonical_alias", ""),
-    ("m.room.avatar", ""),
-    ("m.room.encryption", ""),
-    ("m.room.name", ""),
-    ("m.room.create", ""),
-    ("m.room.topic", ""),
-}
-
-
-class TestRoomPrejoinState(StdlibTestCase):
-    def read_config(self, source: str) -> ApiConfig:
-        config = ApiConfig()
-        config.read_config(yaml.safe_load(source))
-        return config
-
-    def test_no_prejoin_state(self) -> None:
-        config = self.read_config("foo: bar")
-        self.assertFalse(config.room_prejoin_state.has_wildcards())
-        self.assertEqual(
-            set(config.room_prejoin_state.concrete_types()), DEFAULT_PREJOIN_STATE_PAIRS
-        )
-
-    def test_disable_default_event_types(self) -> None:
-        config = self.read_config(
-            """
-room_prejoin_state:
-    disable_default_event_types: true
-        """
-        )
-        self.assertEqual(config.room_prejoin_state, StateFilter.none())
-
-    def test_event_without_state_key(self) -> None:
-        config = self.read_config(
-            """
-room_prejoin_state:
-    disable_default_event_types: true
-    additional_event_types:
-        - foo
-        """
-        )
-        self.assertEqual(config.room_prejoin_state.wildcard_types(), ["foo"])
-        self.assertEqual(config.room_prejoin_state.concrete_types(), [])
-
-    def test_event_with_specific_state_key(self) -> None:
-        config = self.read_config(
-            """
-room_prejoin_state:
-    disable_default_event_types: true
-    additional_event_types:
-        - [foo, bar]
-        """
-        )
-        self.assertFalse(config.room_prejoin_state.has_wildcards())
-        self.assertEqual(
-            set(config.room_prejoin_state.concrete_types()),
-            {("foo", "bar")},
-        )
-
-    def test_repeated_event_with_specific_state_key(self) -> None:
-        config = self.read_config(
-            """
-room_prejoin_state:
-    disable_default_event_types: true
-    additional_event_types:
-        - [foo, bar]
-        - [foo, baz]
-        """
-        )
-        self.assertFalse(config.room_prejoin_state.has_wildcards())
-        self.assertEqual(
-            set(config.room_prejoin_state.concrete_types()),
-            {("foo", "bar"), ("foo", "baz")},
-        )
-
-    def test_no_specific_state_key_overrides_specific_state_key(self) -> None:
-        config = self.read_config(
-            """
-room_prejoin_state:
-    disable_default_event_types: true
-    additional_event_types:
-        - [foo, bar]
-        - foo
-        """
-        )
-        self.assertEqual(config.room_prejoin_state.wildcard_types(), ["foo"])
-        self.assertEqual(config.room_prejoin_state.concrete_types(), [])
-
-        config = self.read_config(
-            """
-room_prejoin_state:
-    disable_default_event_types: true
-    additional_event_types:
-        - foo
-        - [foo, bar]
-        """
-        )
-        self.assertEqual(config.room_prejoin_state.wildcard_types(), ["foo"])
-        self.assertEqual(config.room_prejoin_state.concrete_types(), [])
-
-    def test_bad_event_type_entry_raises(self) -> None:
-        with self.assertRaises(ConfigError):
-            self.read_config(
-                """
-room_prejoin_state:
-    additional_event_types:
-        - []
-            """
-            )
-
-        with self.assertRaises(ConfigError):
-            self.read_config(
-                """
-room_prejoin_state:
-    additional_event_types:
-        - [a]
-            """
-            )
-
-        with self.assertRaises(ConfigError):
-            self.read_config(
-                """
-room_prejoin_state:
-    additional_event_types:
-        - [a, b, c]
-            """
-            )
-
-        with self.assertRaises(ConfigError):
-            self.read_config(
-                """
-room_prejoin_state:
-    additional_event_types:
-        - [true, 1.23]
-            """
-            )
--- a/tests/events/test_utils.py
+++ b/tests/events/test_utils.py
@@ -12,20 +12,19 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.

-import unittest as stdlib_unittest
-
 from synapse.api.constants import EventContentFields
 from synapse.api.room_versions import RoomVersions
 from synapse.events import make_event_from_dict
 from synapse.events.utils import (
    SerializeEventConfig,
    copy_and_fixup_power_levels_contents,
-    maybe_upsert_event_field,
    prune_event,
    serialize_event,
 )
 from synapse.util.frozenutils import freeze

+from tests import unittest
+

 def MockEvent(**kwargs):
    if "event_id" not in kwargs:
@@ -35,31 +34,7 @@ def MockEvent(**kwargs):
    return make_event_from_dict(kwargs)


-class TestMaybeUpsertEventField(stdlib_unittest.TestCase):
-    def test_update_okay(self) -> None:
-        event = make_event_from_dict({"event_id": "$1234"})
-        success = maybe_upsert_event_field(event, event.unsigned, "key", "value")
-        self.assertTrue(success)
-        self.assertEqual(event.unsigned["key"], "value")
-
-    def test_update_not_okay(self) -> None:
-        event = make_event_from_dict({"event_id": "$1234"})
-        LARGE_STRING = "a" * 100_000
-        success = maybe_upsert_event_field(event, event.unsigned, "key", LARGE_STRING)
-        self.assertFalse(success)
-        self.assertNotIn("key", event.unsigned)
-
-    def test_update_not_okay_leaves_original_value(self) -> None:
-        event = make_event_from_dict(
-            {"event_id": "$1234", "unsigned": {"key": "value"}}
-        )
-        LARGE_STRING = "a" * 100_000
-        success = maybe_upsert_event_field(event, event.unsigned, "key", LARGE_STRING)
-        self.assertFalse(success)
-        self.assertEqual(event.unsigned["key"], "value")
-
-
-class PruneEventTestCase(stdlib_unittest.TestCase):
+class PruneEventTestCase(unittest.TestCase):
    def run_test(self, evdict, matchdict, **kwargs):
        """
        Asserts that a new event constructed with `evdict` will look like
@@ -416,7 +391,7 @@ class PruneEventTestCase(stdlib_unittest.TestCase):
        )


-class SerializeEventTestCase(stdlib_unittest.TestCase):
+class SerializeEventTestCase(unittest.TestCase):
    def serialize(self, ev, fields):
        return serialize_event(
            ev, 1479807801915, config=SerializeEventConfig(only_event_fields=fields)
@@ -538,7 +513,7 @@ class SerializeEventTestCase(stdlib_unittest.TestCase):
            )


-class CopyPowerLevelsContentTestCase(stdlib_unittest.TestCase):
+class CopyPowerLevelsContentTestCase(unittest.TestCase):
    def setUp(self) -> None:
        self.test_content = {
            "ban": 50,
--- a/tests/rest/client/test_third_party_rules.py
+++ b/tests/rest/client/test_third_party_rules.py
@@ -15,6 +15,7 @@ import threading
 from typing import TYPE_CHECKING, Any, Dict, Optional, Tuple, Union
 from unittest.mock import Mock

+from twisted.internet import defer
 from twisted.test.proto_helpers import MemoryReactor

 from synapse.api.constants import EventTypes, LoginType, Membership
@@ -931,3 +932,62 @@ class ThirdPartyRulesTestCase(unittest.FederatingHomeserverTestCase):

        # Check that the mock was called with the right parameters
        self.assertEqual(args, (user_id, "email", "foo@example.com"))
+
+    def test_unbind_threepid(self) -> None:
+        """Tests that the unbind_threepid module callback is called correctly before
+        removing a 3PID mapping.
+        """
+
+        # Register an admin user.
+        self.register_user("admin", "password", admin=True)
+        admin_tok = self.login("admin", "password")
+
+        # Also register a normal user we can modify.
+        user_id = self.register_user("user", "password")
+
+        # Add a 3PID to the user.
+        channel = self.make_request(
+            "PUT",
+            "/_synapse/admin/v2/users/%s" % user_id,
+            {
+                "threepids": [
+                    {
+                        "medium": "email",
+                        "address": "foo@example.com",
+                    },
+                ],
+            },
+            access_token=admin_tok,
+        )
+        self.assertEqual(channel.code, 200, channel.json_body)
+
+        # Add the mapping to the remote 3pid assoc table
+        defer.ensureDeferred(
+            self.hs.get_module_api().store_remote_3pid_association(
+                user_id, "email", "foo@example.com", "identityserver.org"
+            )
+        )
+
+        # Register a mocked callback with stop = True since we don't want to actually
+        # call identityserver.org
+        threepid_unbind_mock = Mock(return_value=make_awaitable((True, True)))
+        third_party_rules = self.hs.get_third_party_event_rules()
+        third_party_rules._unbind_threepid_callbacks.append(threepid_unbind_mock)
+
+        # Deactivate the account, this should remove the 3pid mapping
+        # and call the module handler.
+        channel = self.make_request(
+            "POST",
+            "/_synapse/admin/v1/deactivate/%s" % user_id,
+            access_token=admin_tok,
+        )
+        self.assertEqual(channel.code, 200, channel.json_body)
+
+        # Check that the mock was called once.
+        threepid_unbind_mock.assert_called_once()
+        args = threepid_unbind_mock.call_args[0]
+
+        # Check that the mock was called with the right parameters
+        self.assertEqual(
+            args, (user_id, "email", "foo@example.com", "identityserver.org")
+        )
--- a/tests/storage/test_state.py
+++ b/tests/storage/test_state.py
@@ -26,7 +26,7 @@ from synapse.types import JsonDict, RoomID, StateMap, UserID
 from synapse.types.state import StateFilter
 from synapse.util import Clock

-from tests.unittest import HomeserverTestCase
+from tests.unittest import HomeserverTestCase, TestCase

 logger = logging.getLogger(__name__)

@@ -494,3 +494,624 @@ class StateStoreTestCase(HomeserverTestCase):

        self.assertEqual(is_all, True)
        self.assertDictEqual({(e5.type, e5.state_key): e5.event_id}, state_dict)
+
+
+class StateFilterDifferenceTestCase(TestCase):
+    def assert_difference(
+        self, minuend: StateFilter, subtrahend: StateFilter, expected: StateFilter
+    ) -> None:
+        self.assertEqual(
+            minuend.approx_difference(subtrahend),
+            expected,
+            f"StateFilter difference not correct:\n\n\t{minuend!r}\nminus\n\t{subtrahend!r}\nwas\n\t{minuend.approx_difference(subtrahend)}\nexpected\n\t{expected}",
+        )
+
+    def test_state_filter_difference_no_include_other_minus_no_include_other(
+        self,
+    ) -> None:
+        """
+        Tests the StateFilter.approx_difference method
+        where, in a.approx_difference(b), both a and b do not have the
+        include_others flag set.
+        """
+        # (wildcard on state keys) - (wildcard on state keys):
+        self.assert_difference(
+            StateFilter.freeze(
+                {EventTypes.Member: None, EventTypes.Create: None},
+                include_others=False,
+            ),
+            StateFilter.freeze(
+                {EventTypes.Member: None, EventTypes.CanonicalAlias: None},
+                include_others=False,
+            ),
+            StateFilter.freeze({EventTypes.Create: None}, include_others=False),
+        )
+
+        # (wildcard on state keys) - (specific state keys)
+        # This one is an over-approximation because we can't represent
+        # 'all state keys except a few named examples'
+        self.assert_difference(
+            StateFilter.freeze({EventTypes.Member: None}, include_others=False),
+            StateFilter.freeze(
+                {EventTypes.Member: {"@wombat:spqr"}},
+                include_others=False,
+            ),
+            StateFilter.freeze({EventTypes.Member: None}, include_others=False),
+        )
+
+        # (wildcard on state keys) - (no state keys)
+        self.assert_difference(
+            StateFilter.freeze(
+                {EventTypes.Member: None},
+                include_others=False,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: set(),
+                },
+                include_others=False,
+            ),
+            StateFilter.freeze(
+                {EventTypes.Member: None},
+                include_others=False,
+            ),
+        )
+
+        # (specific state keys) - (wildcard on state keys):
+        self.assert_difference(
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
+                    EventTypes.CanonicalAlias: {""},
+                },
+                include_others=False,
+            ),
+            StateFilter.freeze(
+                {EventTypes.Member: None},
+                include_others=False,
+            ),
+            StateFilter.freeze(
+                {EventTypes.CanonicalAlias: {""}},
+                include_others=False,
+            ),
+        )
+
+        # (specific state keys) - (specific state keys)
+        self.assert_difference(
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
+                    EventTypes.CanonicalAlias: {""},
+                },
+                include_others=False,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr"},
+                },
+                include_others=False,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@spqr:spqr"},
+                    EventTypes.CanonicalAlias: {""},
+                },
+                include_others=False,
+            ),
+        )
+
+        # (specific state keys) - (no state keys)
+        self.assert_difference(
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
+                    EventTypes.CanonicalAlias: {""},
+                },
+                include_others=False,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: set(),
+                },
+                include_others=False,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
+                    EventTypes.CanonicalAlias: {""},
+                },
+                include_others=False,
+            ),
+        )
+
+    def test_state_filter_difference_include_other_minus_no_include_other(self) -> None:
+        """
+        Tests the StateFilter.approx_difference method
+        where, in a.approx_difference(b), only a has the include_others flag set.
+        """
+        # (wildcard on state keys) - (wildcard on state keys):
+        self.assert_difference(
+            StateFilter.freeze(
+                {EventTypes.Member: None, EventTypes.Create: None},
+                include_others=True,
+            ),
+            StateFilter.freeze(
+                {EventTypes.Member: None, EventTypes.CanonicalAlias: None},
+                include_others=False,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Create: None,
+                    EventTypes.Member: set(),
+                    EventTypes.CanonicalAlias: set(),
+                },
+                include_others=True,
+            ),
+        )
+
+        # (wildcard on state keys) - (specific state keys)
+        # This one is an over-approximation because we can't represent
+        # 'all state keys except a few named examples'
+        # This also shows that the resultant state filter is normalised.
+        self.assert_difference(
+            StateFilter.freeze({EventTypes.Member: None}, include_others=True),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr"},
+                    EventTypes.Create: {""},
+                },
+                include_others=False,
+            ),
+            StateFilter(types=frozendict(), include_others=True),
+        )
+
+        # (wildcard on state keys) - (no state keys)
+        self.assert_difference(
+            StateFilter.freeze(
+                {EventTypes.Member: None},
+                include_others=True,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: set(),
+                },
+                include_others=False,
+            ),
+            StateFilter(
+                types=frozendict(),
+                include_others=True,
+            ),
+        )
+
+        # (specific state keys) - (wildcard on state keys):
+        self.assert_difference(
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
+                    EventTypes.CanonicalAlias: {""},
+                },
+                include_others=True,
+            ),
+            StateFilter.freeze(
+                {EventTypes.Member: None},
+                include_others=False,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.CanonicalAlias: {""},
+                    EventTypes.Member: set(),
+                },
+                include_others=True,
+            ),
+        )
+
+        # (specific state keys) - (specific state keys)
+        self.assert_difference(
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
+                    EventTypes.CanonicalAlias: {""},
+                },
+                include_others=True,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr"},
+                },
+                include_others=False,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@spqr:spqr"},
+                    EventTypes.CanonicalAlias: {""},
+                },
+                include_others=True,
+            ),
+        )
+
+        # (specific state keys) - (no state keys)
+        self.assert_difference(
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
+                    EventTypes.CanonicalAlias: {""},
+                },
+                include_others=True,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: set(),
+                },
+                include_others=False,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
+                    EventTypes.CanonicalAlias: {""},
+                },
+                include_others=True,
+            ),
+        )
+
+    def test_state_filter_difference_include_other_minus_include_other(self) -> None:
+        """
+        Tests the StateFilter.approx_difference method
+        where, in a.approx_difference(b), both a and b have the include_others
+        flag set.
+        """
+        # (wildcard on state keys) - (wildcard on state keys):
+        self.assert_difference(
+            StateFilter.freeze(
+                {EventTypes.Member: None, EventTypes.Create: None},
+                include_others=True,
+            ),
+            StateFilter.freeze(
+                {EventTypes.Member: None, EventTypes.CanonicalAlias: None},
+                include_others=True,
+            ),
+            StateFilter(types=frozendict(), include_others=False),
+        )
+
+        # (wildcard on state keys) - (specific state keys)
+        # This one is an over-approximation because we can't represent
+        # 'all state keys except a few named examples'
+        self.assert_difference(
+            StateFilter.freeze({EventTypes.Member: None}, include_others=True),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr"},
+                    EventTypes.CanonicalAlias: {""},
+                },
+                include_others=True,
+            ),
+            StateFilter.freeze(
+                {EventTypes.Member: None, EventTypes.CanonicalAlias: None},
+                include_others=False,
+            ),
+        )
+
+        # (wildcard on state keys) - (no state keys)
+        self.assert_difference(
+            StateFilter.freeze(
+                {EventTypes.Member: None},
+                include_others=True,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: set(),
+                },
+                include_others=True,
+            ),
+            StateFilter.freeze(
+                {EventTypes.Member: None},
+                include_others=False,
+            ),
+        )
+
+        # (specific state keys) - (wildcard on state keys):
+        self.assert_difference(
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
+                    EventTypes.CanonicalAlias: {""},
+                },
+                include_others=True,
+            ),
+            StateFilter.freeze(
+                {EventTypes.Member: None},
+                include_others=True,
+            ),
+            StateFilter(
+                types=frozendict(),
+                include_others=False,
+            ),
+        )
+
+        # (specific state keys) - (specific state keys)
+        # This one is an over-approximation because we can't represent
+        # 'all state keys except a few named examples'
+        self.assert_difference(
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
+                    EventTypes.CanonicalAlias: {""},
+                    EventTypes.Create: {""},
+                },
+                include_others=True,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr"},
+                    EventTypes.Create: set(),
+                },
+                include_others=True,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@spqr:spqr"},
+                    EventTypes.Create: {""},
+                },
+                include_others=False,
+            ),
+        )
+
+        # (specific state keys) - (no state keys)
+        self.assert_difference(
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
+                    EventTypes.CanonicalAlias: {""},
+                },
+                include_others=True,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: set(),
+                },
+                include_others=True,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
+                },
+                include_others=False,
+            ),
+        )
+
+    def test_state_filter_difference_no_include_other_minus_include_other(self) -> None:
+        """
+        Tests the StateFilter.approx_difference method
+        where, in a.approx_difference(b), only b has the include_others flag set.
+        """
+        # (wildcard on state keys) - (wildcard on state keys):
+        self.assert_difference(
+            StateFilter.freeze(
+                {EventTypes.Member: None, EventTypes.Create: None},
+                include_others=False,
+            ),
+            StateFilter.freeze(
+                {EventTypes.Member: None, EventTypes.CanonicalAlias: None},
+                include_others=True,
+            ),
+            StateFilter(types=frozendict(), include_others=False),
+        )
+
+        # (wildcard on state keys) - (specific state keys)
+        # This one is an over-approximation because we can't represent
+        # 'all state keys except a few named examples'
+        self.assert_difference(
+            StateFilter.freeze({EventTypes.Member: None}, include_others=False),
+            StateFilter.freeze(
+                {EventTypes.Member: {"@wombat:spqr"}},
+                include_others=True,
+            ),
+            StateFilter.freeze({EventTypes.Member: None}, include_others=False),
+        )
+
+        # (wildcard on state keys) - (no state keys)
+        self.assert_difference(
+            StateFilter.freeze(
+                {EventTypes.Member: None},
+                include_others=False,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: set(),
+                },
+                include_others=True,
+            ),
+            StateFilter.freeze(
+                {EventTypes.Member: None},
+                include_others=False,
+            ),
+        )
+
+        # (specific state keys) - (wildcard on state keys):
+        self.assert_difference(
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
+                    EventTypes.CanonicalAlias: {""},
+                },
+                include_others=False,
+            ),
+            StateFilter.freeze(
+                {EventTypes.Member: None},
+                include_others=True,
+            ),
+            StateFilter(
+                types=frozendict(),
+                include_others=False,
+            ),
+        )
+
+        # (specific state keys) - (specific state keys)
+        # This one is an over-approximation because we can't represent
+        # 'all state keys except a few named examples'
+        self.assert_difference(
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
+                    EventTypes.CanonicalAlias: {""},
+                },
+                include_others=False,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr"},
+                },
+                include_others=True,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@spqr:spqr"},
+                },
+                include_others=False,
+            ),
+        )
+
+        # (specific state keys) - (no state keys)
+        self.assert_difference(
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
+                    EventTypes.CanonicalAlias: {""},
+                },
+                include_others=False,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: set(),
+                },
+                include_others=True,
+            ),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
+                },
+                include_others=False,
+            ),
+        )
+
+    def test_state_filter_difference_simple_cases(self) -> None:
+        """
+        Tests some very simple cases of the StateFilter approx_difference,
+        that are not explicitly tested by the more in-depth tests.
+        """
+
+        self.assert_difference(StateFilter.all(), StateFilter.all(), StateFilter.none())
+
+        self.assert_difference(
+            StateFilter.all(),
+            StateFilter.none(),
+            StateFilter.all(),
+        )
+
+
+class StateFilterTestCase(TestCase):
+    def test_return_expanded(self) -> None:
+        """
+        Tests the behaviour of the return_expanded() function that expands
+        StateFilters to include more state types (for the sake of cache hit rate).
+        """
+
+        self.assertEqual(StateFilter.all().return_expanded(), StateFilter.all())
+
+        self.assertEqual(StateFilter.none().return_expanded(), StateFilter.none())
+
+        # Concrete-only state filters stay the same
+        # (Case: mixed filter)
+        self.assertEqual(
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:test", "@alicia:test"},
+                    "some.other.state.type": {""},
+                },
+                include_others=False,
+            ).return_expanded(),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:test", "@alicia:test"},
+                    "some.other.state.type": {""},
+                },
+                include_others=False,
+            ),
+        )
+
+        # Concrete-only state filters stay the same
+        # (Case: non-member-only filter)
+        self.assertEqual(
+            StateFilter.freeze(
+                {"some.other.state.type": {""}}, include_others=False
+            ).return_expanded(),
+            StateFilter.freeze({"some.other.state.type": {""}}, include_others=False),
+        )
+
+        # Concrete-only state filters stay the same
+        # (Case: member-only filter)
+        self.assertEqual(
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:test", "@alicia:test"},
+                },
+                include_others=False,
+            ).return_expanded(),
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:test", "@alicia:test"},
+                },
+                include_others=False,
+            ),
+        )
+
+        # Wildcard member-only state filters stay the same
+        self.assertEqual(
+            StateFilter.freeze(
+                {EventTypes.Member: None},
+                include_others=False,
+            ).return_expanded(),
+            StateFilter.freeze(
+                {EventTypes.Member: None},
+                include_others=False,
+            ),
+        )
+
+        # If there is a wildcard in the non-member portion of the filter,
+        # it's expanded to include ALL non-member events.
+        # (Case: mixed filter)
+        self.assertEqual(
+            StateFilter.freeze(
+                {
+                    EventTypes.Member: {"@wombat:test", "@alicia:test"},
+                    "some.other.state.type": None,
+                },
+                include_others=False,
+            ).return_expanded(),
+            StateFilter.freeze(
+                {EventTypes.Member: {"@wombat:test", "@alicia:test"}},
+                include_others=True,
+            ),
+        )
+
+        # If there is a wildcard in the non-member portion of the filter,
+        # it's expanded to include ALL non-member events.
+        # (Case: non-member-only filter)
+        self.assertEqual(
+            StateFilter.freeze(
+                {
+                    "some.other.state.type": None,
+                },
+                include_others=False,
+            ).return_expanded(),
+            StateFilter.freeze({EventTypes.Member: set()}, include_others=True),
+        )
+        self.assertEqual(
+            StateFilter.freeze(
+                {
+                    "some.other.state.type": None,
+                    "yet.another.state.type": {"wombat"},
+                },
+                include_others=False,
+            ).return_expanded(),
+            StateFilter.freeze({EventTypes.Member: set()}, include_others=True),
+        )
--- a/tests/types/init.py
+++ b/tests/types/init.py
--- a/tests/types/test_state.py
+++ b/tests/types/test_state.py
@@ -1,627 +0,0 @@
-from frozendict import frozendict
-
-from synapse.api.constants import EventTypes
-from synapse.types.state import StateFilter
-
-from tests.unittest import TestCase
-
-
-class StateFilterDifferenceTestCase(TestCase):
-    def assert_difference(
-        self, minuend: StateFilter, subtrahend: StateFilter, expected: StateFilter
-    ) -> None:
-        self.assertEqual(
-            minuend.approx_difference(subtrahend),
-            expected,
-            f"StateFilter difference not correct:\n\n\t{minuend!r}\nminus\n\t{subtrahend!r}\nwas\n\t{minuend.approx_difference(subtrahend)}\nexpected\n\t{expected}",
-        )
-
-    def test_state_filter_difference_no_include_other_minus_no_include_other(
-        self,
-    ) -> None:
-        """
-        Tests the StateFilter.approx_difference method
-        where, in a.approx_difference(b), both a and b do not have the
-        include_others flag set.
-        """
-        # (wildcard on state keys) - (wildcard on state keys):
-        self.assert_difference(
-            StateFilter.freeze(
-                {EventTypes.Member: None, EventTypes.Create: None},
-                include_others=False,
-            ),
-            StateFilter.freeze(
-                {EventTypes.Member: None, EventTypes.CanonicalAlias: None},
-                include_others=False,
-            ),
-            StateFilter.freeze({EventTypes.Create: None}, include_others=False),
-        )
-
-        # (wildcard on state keys) - (specific state keys)
-        # This one is an over-approximation because we can't represent
-        # 'all state keys except a few named examples'
-        self.assert_difference(
-            StateFilter.freeze({EventTypes.Member: None}, include_others=False),
-            StateFilter.freeze(
-                {EventTypes.Member: {"@wombat:spqr"}},
-                include_others=False,
-            ),
-            StateFilter.freeze({EventTypes.Member: None}, include_others=False),
-        )
-
-        # (wildcard on state keys) - (no state keys)
-        self.assert_difference(
-            StateFilter.freeze(
-                {EventTypes.Member: None},
-                include_others=False,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: set(),
-                },
-                include_others=False,
-            ),
-            StateFilter.freeze(
-                {EventTypes.Member: None},
-                include_others=False,
-            ),
-        )
-
-        # (specific state keys) - (wildcard on state keys):
-        self.assert_difference(
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
-                    EventTypes.CanonicalAlias: {""},
-                },
-                include_others=False,
-            ),
-            StateFilter.freeze(
-                {EventTypes.Member: None},
-                include_others=False,
-            ),
-            StateFilter.freeze(
-                {EventTypes.CanonicalAlias: {""}},
-                include_others=False,
-            ),
-        )
-
-        # (specific state keys) - (specific state keys)
-        self.assert_difference(
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
-                    EventTypes.CanonicalAlias: {""},
-                },
-                include_others=False,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr"},
-                },
-                include_others=False,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@spqr:spqr"},
-                    EventTypes.CanonicalAlias: {""},
-                },
-                include_others=False,
-            ),
-        )
-
-        # (specific state keys) - (no state keys)
-        self.assert_difference(
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
-                    EventTypes.CanonicalAlias: {""},
-                },
-                include_others=False,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: set(),
-                },
-                include_others=False,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
-                    EventTypes.CanonicalAlias: {""},
-                },
-                include_others=False,
-            ),
-        )
-
-    def test_state_filter_difference_include_other_minus_no_include_other(self) -> None:
-        """
-        Tests the StateFilter.approx_difference method
-        where, in a.approx_difference(b), only a has the include_others flag set.
-        """
-        # (wildcard on state keys) - (wildcard on state keys):
-        self.assert_difference(
-            StateFilter.freeze(
-                {EventTypes.Member: None, EventTypes.Create: None},
-                include_others=True,
-            ),
-            StateFilter.freeze(
-                {EventTypes.Member: None, EventTypes.CanonicalAlias: None},
-                include_others=False,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Create: None,
-                    EventTypes.Member: set(),
-                    EventTypes.CanonicalAlias: set(),
-                },
-                include_others=True,
-            ),
-        )
-
-        # (wildcard on state keys) - (specific state keys)
-        # This one is an over-approximation because we can't represent
-        # 'all state keys except a few named examples'
-        # This also shows that the resultant state filter is normalised.
-        self.assert_difference(
-            StateFilter.freeze({EventTypes.Member: None}, include_others=True),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr"},
-                    EventTypes.Create: {""},
-                },
-                include_others=False,
-            ),
-            StateFilter(types=frozendict(), include_others=True),
-        )
-
-        # (wildcard on state keys) - (no state keys)
-        self.assert_difference(
-            StateFilter.freeze(
-                {EventTypes.Member: None},
-                include_others=True,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: set(),
-                },
-                include_others=False,
-            ),
-            StateFilter(
-                types=frozendict(),
-                include_others=True,
-            ),
-        )
-
-        # (specific state keys) - (wildcard on state keys):
-        self.assert_difference(
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
-                    EventTypes.CanonicalAlias: {""},
-                },
-                include_others=True,
-            ),
-            StateFilter.freeze(
-                {EventTypes.Member: None},
-                include_others=False,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.CanonicalAlias: {""},
-                    EventTypes.Member: set(),
-                },
-                include_others=True,
-            ),
-        )
-
-        # (specific state keys) - (specific state keys)
-        self.assert_difference(
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
-                    EventTypes.CanonicalAlias: {""},
-                },
-                include_others=True,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr"},
-                },
-                include_others=False,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@spqr:spqr"},
-                    EventTypes.CanonicalAlias: {""},
-                },
-                include_others=True,
-            ),
-        )
-
-        # (specific state keys) - (no state keys)
-        self.assert_difference(
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
-                    EventTypes.CanonicalAlias: {""},
-                },
-                include_others=True,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: set(),
-                },
-                include_others=False,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
-                    EventTypes.CanonicalAlias: {""},
-                },
-                include_others=True,
-            ),
-        )
-
-    def test_state_filter_difference_include_other_minus_include_other(self) -> None:
-        """
-        Tests the StateFilter.approx_difference method
-        where, in a.approx_difference(b), both a and b have the include_others
-        flag set.
-        """
-        # (wildcard on state keys) - (wildcard on state keys):
-        self.assert_difference(
-            StateFilter.freeze(
-                {EventTypes.Member: None, EventTypes.Create: None},
-                include_others=True,
-            ),
-            StateFilter.freeze(
-                {EventTypes.Member: None, EventTypes.CanonicalAlias: None},
-                include_others=True,
-            ),
-            StateFilter(types=frozendict(), include_others=False),
-        )
-
-        # (wildcard on state keys) - (specific state keys)
-        # This one is an over-approximation because we can't represent
-        # 'all state keys except a few named examples'
-        self.assert_difference(
-            StateFilter.freeze({EventTypes.Member: None}, include_others=True),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr"},
-                    EventTypes.CanonicalAlias: {""},
-                },
-                include_others=True,
-            ),
-            StateFilter.freeze(
-                {EventTypes.Member: None, EventTypes.CanonicalAlias: None},
-                include_others=False,
-            ),
-        )
-
-        # (wildcard on state keys) - (no state keys)
-        self.assert_difference(
-            StateFilter.freeze(
-                {EventTypes.Member: None},
-                include_others=True,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: set(),
-                },
-                include_others=True,
-            ),
-            StateFilter.freeze(
-                {EventTypes.Member: None},
-                include_others=False,
-            ),
-        )
-
-        # (specific state keys) - (wildcard on state keys):
-        self.assert_difference(
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
-                    EventTypes.CanonicalAlias: {""},
-                },
-                include_others=True,
-            ),
-            StateFilter.freeze(
-                {EventTypes.Member: None},
-                include_others=True,
-            ),
-            StateFilter(
-                types=frozendict(),
-                include_others=False,
-            ),
-        )
-
-        # (specific state keys) - (specific state keys)
-        # This one is an over-approximation because we can't represent
-        # 'all state keys except a few named examples'
-        self.assert_difference(
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
-                    EventTypes.CanonicalAlias: {""},
-                    EventTypes.Create: {""},
-                },
-                include_others=True,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr"},
-                    EventTypes.Create: set(),
-                },
-                include_others=True,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@spqr:spqr"},
-                    EventTypes.Create: {""},
-                },
-                include_others=False,
-            ),
-        )
-
-        # (specific state keys) - (no state keys)
-        self.assert_difference(
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
-                    EventTypes.CanonicalAlias: {""},
-                },
-                include_others=True,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: set(),
-                },
-                include_others=True,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
-                },
-                include_others=False,
-            ),
-        )
-
-    def test_state_filter_difference_no_include_other_minus_include_other(self) -> None:
-        """
-        Tests the StateFilter.approx_difference method
-        where, in a.approx_difference(b), only b has the include_others flag set.
-        """
-        # (wildcard on state keys) - (wildcard on state keys):
-        self.assert_difference(
-            StateFilter.freeze(
-                {EventTypes.Member: None, EventTypes.Create: None},
-                include_others=False,
-            ),
-            StateFilter.freeze(
-                {EventTypes.Member: None, EventTypes.CanonicalAlias: None},
-                include_others=True,
-            ),
-            StateFilter(types=frozendict(), include_others=False),
-        )
-
-        # (wildcard on state keys) - (specific state keys)
-        # This one is an over-approximation because we can't represent
-        # 'all state keys except a few named examples'
-        self.assert_difference(
-            StateFilter.freeze({EventTypes.Member: None}, include_others=False),
-            StateFilter.freeze(
-                {EventTypes.Member: {"@wombat:spqr"}},
-                include_others=True,
-            ),
-            StateFilter.freeze({EventTypes.Member: None}, include_others=False),
-        )
-
-        # (wildcard on state keys) - (no state keys)
-        self.assert_difference(
-            StateFilter.freeze(
-                {EventTypes.Member: None},
-                include_others=False,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: set(),
-                },
-                include_others=True,
-            ),
-            StateFilter.freeze(
-                {EventTypes.Member: None},
-                include_others=False,
-            ),
-        )
-
-        # (specific state keys) - (wildcard on state keys):
-        self.assert_difference(
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
-                    EventTypes.CanonicalAlias: {""},
-                },
-                include_others=False,
-            ),
-            StateFilter.freeze(
-                {EventTypes.Member: None},
-                include_others=True,
-            ),
-            StateFilter(
-                types=frozendict(),
-                include_others=False,
-            ),
-        )
-
-        # (specific state keys) - (specific state keys)
-        # This one is an over-approximation because we can't represent
-        # 'all state keys except a few named examples'
-        self.assert_difference(
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
-                    EventTypes.CanonicalAlias: {""},
-                },
-                include_others=False,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr"},
-                },
-                include_others=True,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@spqr:spqr"},
-                },
-                include_others=False,
-            ),
-        )
-
-        # (specific state keys) - (no state keys)
-        self.assert_difference(
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
-                    EventTypes.CanonicalAlias: {""},
-                },
-                include_others=False,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: set(),
-                },
-                include_others=True,
-            ),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:spqr", "@spqr:spqr"},
-                },
-                include_others=False,
-            ),
-        )
-
-    def test_state_filter_difference_simple_cases(self) -> None:
-        """
-        Tests some very simple cases of the StateFilter approx_difference,
-        that are not explicitly tested by the more in-depth tests.
-        """
-
-        self.assert_difference(StateFilter.all(), StateFilter.all(), StateFilter.none())
-
-        self.assert_difference(
-            StateFilter.all(),
-            StateFilter.none(),
-            StateFilter.all(),
-        )
-
-
-class StateFilterTestCase(TestCase):
-    def test_return_expanded(self) -> None:
-        """
-        Tests the behaviour of the return_expanded() function that expands
-        StateFilters to include more state types (for the sake of cache hit rate).
-        """
-
-        self.assertEqual(StateFilter.all().return_expanded(), StateFilter.all())
-
-        self.assertEqual(StateFilter.none().return_expanded(), StateFilter.none())
-
-        # Concrete-only state filters stay the same
-        # (Case: mixed filter)
-        self.assertEqual(
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:test", "@alicia:test"},
-                    "some.other.state.type": {""},
-                },
-                include_others=False,
-            ).return_expanded(),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:test", "@alicia:test"},
-                    "some.other.state.type": {""},
-                },
-                include_others=False,
-            ),
-        )
-
-        # Concrete-only state filters stay the same
-        # (Case: non-member-only filter)
-        self.assertEqual(
-            StateFilter.freeze(
-                {"some.other.state.type": {""}}, include_others=False
-            ).return_expanded(),
-            StateFilter.freeze({"some.other.state.type": {""}}, include_others=False),
-        )
-
-        # Concrete-only state filters stay the same
-        # (Case: member-only filter)
-        self.assertEqual(
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:test", "@alicia:test"},
-                },
-                include_others=False,
-            ).return_expanded(),
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:test", "@alicia:test"},
-                },
-                include_others=False,
-            ),
-        )
-
-        # Wildcard member-only state filters stay the same
-        self.assertEqual(
-            StateFilter.freeze(
-                {EventTypes.Member: None},
-                include_others=False,
-            ).return_expanded(),
-            StateFilter.freeze(
-                {EventTypes.Member: None},
-                include_others=False,
-            ),
-        )
-
-        # If there is a wildcard in the non-member portion of the filter,
-        # it's expanded to include ALL non-member events.
-        # (Case: mixed filter)
-        self.assertEqual(
-            StateFilter.freeze(
-                {
-                    EventTypes.Member: {"@wombat:test", "@alicia:test"},
-                    "some.other.state.type": None,
-                },
-                include_others=False,
-            ).return_expanded(),
-            StateFilter.freeze(
-                {EventTypes.Member: {"@wombat:test", "@alicia:test"}},
-                include_others=True,
-            ),
-        )
-
-        # If there is a wildcard in the non-member portion of the filter,
-        # it's expanded to include ALL non-member events.
-        # (Case: non-member-only filter)
-        self.assertEqual(
-            StateFilter.freeze(
-                {
-                    "some.other.state.type": None,
-                },
-                include_others=False,
-            ).return_expanded(),
-            StateFilter.freeze({EventTypes.Member: set()}, include_others=True),
-        )
-        self.assertEqual(
-            StateFilter.freeze(
-                {
-                    "some.other.state.type": None,
-                    "yet.another.state.type": {"wombat"},
-                },
-                include_others=False,
-            ).return_expanded(),
-            StateFilter.freeze({EventTypes.Member: set()}, include_others=True),
-        )
Author	SHA1	Message	Date
Mathieu Velten	4dc7b444bd	Fix import order	2022-12-13 10:25:19 +01:00
Mathieu Velten	6073c0ecb1	Adress comments	2022-12-12 23:31:09 +01:00
Mathieu Velten	4ccade636e	Merge remote-tracking branch 'origin/develop' into mv/unbind-callback	2022-12-12 17:48:27 +01:00
Mathieu Velten	5ff0ba261c	Merge remote-tracking branch 'origin/develop' into mv/unbind-callback	2022-11-28 12:57:35 +01:00
Mathieu Velten	9b4c0e79d8	Adress comments	2022-08-03 16:24:44 +02:00
Mathieu Velten	97f991ed2e	Apply suggestions from code review Co-authored-by: Brendan Abolivier <babolivier@matrix.org>	2022-08-03 14:52:20 +02:00
Mathieu Velten	0274a7f2f5	Add 3pid unbind callback to module API	2022-08-03 14:52:20 +02:00
				`@@ -0,0 +1 @@`
				`Add a module callback for unbinding a 3PID.`
				`@@ -0,0 +1 @@`
				`Optimise push badge count calculations. Contributed by Nick @ Beeper (@fizzadar).`
				`@@ -0,0 +1 @@`
				`Fix a long-standing bug where a device list update might not be sent to clients in certain circumstances.`
				`@@ -0,0 +1 @@`
				`Improve user search for international display names.`
				`@@ -0,0 +1 @@`
				`Faster remote room joins: stream the un-partial-stating of rooms over replication.`
				`@@ -0,0 +1 @@`
				Stop using deprecated `keyIds` parameter when calling `/_matrix/key/v2/server`.
				`@@ -0,0 +1 @@`
				Update worker settings for `pusher` and `federation_sender` functionality.
				`@@ -0,0 +1 @@`
				`Add links to third party package repositories, and point to the bug which highlights Ubuntu's out-of-date packages.`
				`@@ -0,0 +1 @@`
				Share the `ClientRestResource` for both workers and the main process.