Cut out regex splits to get a full idea

resolve linter arguments
lints
2022-05-27 15:13:21 +01:00 · 2022-05-27 14:31:51 +01:00 · 2022-05-27 14:31:51 +01:00 · 2022-05-27 14:31:51 +01:00 · 2022-05-27 13:20:43 +01:00 · 2022-05-27 10:45:51 +01:00
118 changed files with 1092 additions and 4437 deletions
@@ -310,6 +310,16 @@ jobs:
    needs: linting-done
    runs-on: ubuntu-latest

+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          # GHA requires all matrix configurations to have at least one value. We don't want to set one here though.
+          - _: monolith
+
+          # Test with workers
+          - workers: workers
+
    steps:
      # The path is set via a file given by $GITHUB_PATH. We need both Go 1.17 and GOPATH on the path to run Complement.
      # See https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#adding-a-system-path
@@ -356,7 +366,7 @@ jobs:

      - run: |
          set -o pipefail
-          COMPLEMENT_DIR=`pwd`/complement synapse/scripts-dev/complement.sh -json 2>&1 | gotestfmt
+          WORKERS=${{ matrix.workers && 1 }} COMPLEMENT_DIR=`pwd`/complement synapse/scripts-dev/complement.sh -json ${{ matrix.regex && format('-run Test[{0}]', matrix.regex) || '' }} 2>&1 | gotestfmt
        shell: bash
        name: Run Complement Tests

@@ -0,0 +1 @@
+Remove support for the non-standard groups/communities feature from Synapse.
@@ -0,0 +1 @@
+Remove support for the non-standard groups/communities feature from Synapse.
@@ -0,0 +1 @@
+Experimental support for [MSC3772](https://github.com/matrix-org/matrix-spec-proposals/pull/3772): Push rule for mutually related events.
@@ -0,0 +1 @@
+Always send an `access_token` in `/thirdparty/` requests to appservices, as required by the [Matrix specification](https://spec.matrix.org/v1.1/application-service-api/#third-party-networks).
@@ -0,0 +1 @@
+Test Synapse against Complement with workers.
@@ -0,0 +1 @@
+Fix a bug where we did not correctly handle invalid device list updates over federation. Contributed by Carl Bordum Hansen.
@@ -0,0 +1 @@
+Fix bug where servers using a Postgres database would fail to backfill from an insertion event when MSC2716 is enabled (`experimental_features.msc2716_enabled`).
@@ -0,0 +1 @@
+Remove `dont_notify` from the `.m.rule.room.server_acl` rule.
@@ -0,0 +1 @@
+Remove the unstable `/hierarchy` endpoint from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946).
@@ -0,0 +1 @@
+Pull out less state when handling gaps in room DAG.
@@ -0,0 +1 @@
+Clean-up the push rules datastore.
@@ -0,0 +1 @@
+Fix [MSC3787](https://github.com/matrix-org/matrix-spec-proposals/pull/3787) rooms being omitted from room directory, room summary and space hierarchy responses.
@@ -0,0 +1 @@
+Experimental support for [MSC3772](https://github.com/matrix-org/matrix-spec-proposals/pull/3772): Push rule for mutually related events.
@@ -0,0 +1 @@
+Correct a type annotation in the URL preview source code.
@@ -0,0 +1 @@
+Fix typos in documentation.
@@ -0,0 +1 @@
+Update `pyjwt` dependency to [2.4.0](https://github.com/jpadilla/pyjwt/releases/tag/2.4.0).
@@ -0,0 +1 @@
+Enable the `/account/whoami` endpoint on synapse worker processes. Contributed by Nick @ Beeper.
@@ -0,0 +1 @@
+Fix documentation incorrectly stating the `sendToDevice` endpoint can be directed at generic workers. Contributed by Nick @ Beeper.
@@ -0,0 +1 @@
+Enable the `batch_send` endpoint on synapse worker processes. Contributed by Nick @ Beeper.
@@ -0,0 +1 @@
+Don't generate empty AS transactions when the AS is flagged as down. Contributed by Nick @ Beeper.
@@ -0,0 +1 @@
+Fix up the variable `state_store` naming.
@@ -0,0 +1 @@
+Fix a bug introduced in Synapse 1.54 which could sometimes cause exceptions when handling federated traffic.
@@ -0,0 +1 @@
+Avoid running queries which will never result in deletions.
@@ -0,0 +1 @@
+CI only, please ignore.
@@ -26,6 +26,9 @@ COPY conf-workers/workers-shared.yaml /conf/workers/shared.yaml
 WORKDIR /data

 COPY conf-workers/postgres.supervisord.conf /etc/supervisor/conf.d/postgres.conf
+COPY conf-workers/synapse_forking.supervisord.conf.j2 /conf/
+
+COPY conf/log_config.yaml.j2 /conf/

 # Copy the entrypoint
 COPY conf-workers/start-complement-synapse-workers.sh /
@@ -0,0 +1,26 @@
+[program:synapse_forking]
+# TODO prefix-log will be no good. We'll have to hack around ourselves.
+command=/usr/local/bin/prefix-log /usr/local/bin/python -m synapse.app._complement_fork_starter /data/homeserver.yaml \
+{%- for worker_config in worker_configs %}
+    -- \
+    {{ worker_config.app }}
+    --config-path="{{ worker_config.config_path }}" \
+    --config-path=/conf/workers/shared.yaml \
+    --config-path=/conf/workers/{{ worker_config.name }}.yaml \
+{%- endfor %}
+    -- \
+    synapse.app.homeserver \
+    --config-path="{{ main_config_path }}" \
+    --config-path=/conf/workers/shared.yaml
+
+autorestart=unexpected
+priority=500
+exitcodes=0
+stdout_logfile=/dev/stdout
+stdout_logfile_maxbytes=0
+stderr_logfile=/dev/stderr
+stderr_logfile_maxbytes=0
+
+# Required because the forking launcher creates subprocesses but doesn't
+# handle signals for us.
+stopasgroup=true
@@ -2,7 +2,11 @@ version: 1

 formatters:
  precise:
-   format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
+    {% if worker_name %}
+    format: '{{ worker_name }} | %(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
+    {% else %}
+    format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
+    {% endif %}

 filters:
  context:
@@ -28,17 +28,17 @@ stderr_logfile_maxbytes=0
 username=redis
 autorestart=true

-[program:synapse_main]
-command=/usr/local/bin/prefix-log /usr/local/bin/python -m synapse.app.homeserver --config-path="{{ main_config_path }}" --config-path=/conf/workers/shared.yaml
-priority=10
-# Log startup failures to supervisord's stdout/err
-# Regular synapse logs will still go in the configured data directory
-stdout_logfile=/dev/stdout
-stdout_logfile_maxbytes=0
-stderr_logfile=/dev/stderr
-stderr_logfile_maxbytes=0
-autorestart=unexpected
-exitcodes=0
+## [program:synapse_main]
+## command=/usr/local/bin/prefix-log /usr/local/bin/python -m synapse.app.homeserver --config-path="{{ main_config_path }}" --config-path=/conf/workers/shared.yaml
+## priority=10
+## # Log startup failures to supervisord's stdout/err
+## # Regular synapse logs will still go in the configured data directory
+## stdout_logfile=/dev/stdout
+## stdout_logfile_maxbytes=0
+## stderr_logfile=/dev/stderr
+## stderr_logfile_maxbytes=0
+## autorestart=unexpected
+## exitcodes=0

 # Additional process blocks
-{{ worker_config }}
+{{ worker_config }}
@@ -158,6 +158,7 @@ WORKERS_CONFIG: Dict[str, Dict[str, Any]] = {
            "^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/(join|invite|leave|ban|unban|kick)$",
            "^/_matrix/client/(api/v1|r0|v3|unstable)/join/",
            "^/_matrix/client/(api/v1|r0|v3|unstable)/profile/",
+            "^/_matrix/client/(v1|unstable/org.matrix.msc2716)/rooms/.*/batch_send",
        ],
        "shared_extra_conf": {},
        "worker_extra_conf": "",
@@ -400,6 +401,8 @@ def generate_worker_files(
    # which exists even if no workers do.
    healthcheck_urls = ["http://localhost:8080/health"]

+    worker_configs: List[Dict[str, Any]] = []
+
    # For each worker type specified by the user, create config values
    for worker_type in worker_types:
        worker_type = worker_type.strip()
@@ -437,6 +440,8 @@ def generate_worker_files(
        # Enable the worker in supervisord
        supervisord_config += SUPERVISORD_PROCESS_CONFIG_BLOCK.format_map(worker_config)

+        worker_configs.append(worker_config)
+
        # Add nginx location blocks for this worker's endpoints (if any are defined)
        for pattern in worker_config["endpoint_patterns"]:
            # Determine whether we need to load-balance this worker
@@ -529,7 +534,15 @@ def generate_worker_files(
        "/conf/supervisord.conf.j2",
        "/etc/supervisor/supervisord.conf",
        main_config_path=config_path,
-        worker_config=supervisord_config,
+        # worker_config=supervisord_config,
+        worker_config="",
+    )
+
+    convert(
+        "/conf/synapse_forking.supervisord.conf.j2",
+        "/etc/supervisor/conf.d/synapse_forking.supervisor.conf",
+        worker_configs=worker_configs,
+        main_config_path=config_path,
    )

    # healthcheck config
@@ -561,7 +574,7 @@ def generate_worker_log_config(
    # Render and write the file
    log_config_filepath = "/conf/workers/{name}.log.config".format(name=worker_name)
    convert(
-        "/conf/log.config",
+        "/conf/log_config.yaml.j2",
        log_config_filepath,
        worker_name=worker_name,
        **extra_log_template_args,
@@ -117,7 +117,7 @@ In this example, we define three jobs:
 Note that this example is tailored to show different configurations and
 features slightly more jobs than it's probably necessary (in practice, a
 server admin would probably consider it better to replace the two last
-jobs with one that runs once a day and handles rooms which which
+jobs with one that runs once a day and handles rooms which
 policy's `max_lifetime` is greater than 3 days).

 Keep in mind, when configuring these jobs, that a purge job can become
@@ -2521,16 +2521,6 @@ push:
 #        "events_default": 1


-# Uncomment to allow non-server-admin users to create groups on this server
-#
-#enable_group_creation: true
-
-# If enabled, non server admins can only create groups with local parts
-# starting with this prefix
-#
-#group_creation_prefix: "unofficial_"
-
-

 # User Directory configuration
 #
@@ -43,7 +43,7 @@ loggers:
 The above logging config will set Synapse as 'INFO' logging level by default,
 with the SQL layer at 'WARNING', and will log to a file, stored as JSON.

-It is also possible to figure Synapse to log to a remote endpoint by using the
+It is also possible to configure Synapse to log to a remote endpoint by using the
 `synapse.logging.RemoteHandler` class included with Synapse. It takes the
 following arguments:

@@ -3145,25 +3145,6 @@ Example configuration:
 encryption_enabled_by_default_for_room_type: invite
 ```
 ---
-Config option: `enable_group_creation`
-
-Set to true to allow non-server-admin users to create groups on this server
-
-Example configuration:
-```yaml
-enable_group_creation: true
-```
---
-Config option: `group_creation_prefix`
-
-If enabled/present, non-server admins can only create groups with local parts
-starting with this prefix.
-
-Example configuration:
-```yaml
-group_creation_prefix: "unofficial_"
-```
---
 Config option: `user_directory`

 This setting defines options related to the user directory. 
@@ -1,6 +1,6 @@
 # Scaling synapse via workers

-For small instances it recommended to run Synapse in the default monolith mode.
+For small instances it is recommended to run Synapse in the default monolith mode.
 For larger instances where performance is a concern it can be helpful to split
 out functionality into multiple separate python processes. These processes are
 called 'workers', and are (eventually) intended to scale horizontally
@@ -193,7 +193,7 @@ information.
    ^/_matrix/federation/v1/user/devices/
    ^/_matrix/federation/v1/get_groups_publicised$
    ^/_matrix/key/v2/query
-    ^/_matrix/federation/(v1|unstable/org.matrix.msc2946)/hierarchy/
+    ^/_matrix/federation/v1/hierarchy/

    # Inbound federation transaction request
    ^/_matrix/federation/v1/send/
@@ -205,9 +205,11 @@ information.
    ^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/context/.*$
    ^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/members$
    ^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/state$
-    ^/_matrix/client/(v1|unstable/org.matrix.msc2946)/rooms/.*/hierarchy$
+    ^/_matrix/client/v1/rooms/.*/hierarchy$
+    ^/_matrix/client/unstable/org.matrix.msc2716/rooms/.*/batch_send$
    ^/_matrix/client/unstable/im.nheko.summary/rooms/.*/summary$
    ^/_matrix/client/(r0|v3|unstable)/account/3pid$
+    ^/_matrix/client/(r0|v3|unstable)/account/whoami$
    ^/_matrix/client/(r0|v3|unstable)/devices$
    ^/_matrix/client/versions$
    ^/_matrix/client/(api/v1|r0|v3|unstable)/voip/turnServer$
@@ -237,9 +239,6 @@ information.
    ^/_matrix/client/(api/v1|r0|v3|unstable)/join/
    ^/_matrix/client/(api/v1|r0|v3|unstable)/profile/

-    # Device requests
-    ^/_matrix/client/(r0|v3|unstable)/sendToDevice/
-
    # Account data requests
    ^/_matrix/client/(r0|v3|unstable)/.*/tags
    ^/_matrix/client/(r0|v3|unstable)/.*/account_data
@@ -813,7 +813,7 @@ python-versions = ">=3.5"

 [[package]]
 name = "pyjwt"
-version = "2.3.0"
+version = "2.4.0"
 description = "JSON Web Token implementation in Python"
 category = "main"
 optional = false
@@ -2264,8 +2264,8 @@ pygments = [
    {file = "Pygments-2.11.2.tar.gz", hash = "sha256:4e426f72023d88d03b2fa258de560726ce890ff3b630f88c21cbb8b2503b8c6a"},
 ]
 pyjwt = [
-    {file = "PyJWT-2.3.0-py3-none-any.whl", hash = "sha256:e0c4bb8d9f0af0c7f5b1ec4c5036309617d03d56932877f2f7a0beeb5318322f"},
-    {file = "PyJWT-2.3.0.tar.gz", hash = "sha256:b888b4d56f06f6dcd777210c334e69c737be74755d3e5e9ee3fe67dc18a0ee41"},
+    {file = "PyJWT-2.4.0-py3-none-any.whl", hash = "sha256:72d1d253f32dbd4f5c88eaf1fdc62f3a19f676ccbadb9dbc5d07e951b2b26daf"},
+    {file = "PyJWT-2.4.0.tar.gz", hash = "sha256:d42908208c699b3b973cbeb01a969ba6a96c821eefb1c5bfe4c390c01d67abba"},
 ]
 pymacaroons = [
    {file = "pymacaroons-0.13.0-py2.py3-none-any.whl", hash = "sha256:3e14dff6a262fdbf1a15e769ce635a8aea72e6f8f91e408f9a97166c53b91907"},
@@ -45,7 +45,7 @@ docker build -t matrixdotorg/synapse -f "docker/Dockerfile" .

 extra_test_args=()

-test_tags="synapse_blacklist,msc2716,msc3030"
+test_tags="synapse_blacklist,msc2716,msc3030,msc3787"

 # If we're using workers, modify the docker files slightly.
 if [[ -n "$WORKERS" ]]; then
@@ -31,11 +31,6 @@ MAX_ALIAS_LENGTH = 255
 # the maximum length for a user id is 255 characters
 MAX_USERID_LENGTH = 255

-# The maximum length for a group id is 255 characters
-MAX_GROUPID_LENGTH = 255
-MAX_GROUP_CATEGORYID_LENGTH = 255
-MAX_GROUP_ROLEID_LENGTH = 255
-

 class Membership:

@@ -106,7 +106,7 @@ def register_sighup(func: Callable[P, None], *args: P.args, **kwargs: P.kwargs)
 def start_worker_reactor(
    appname: str,
    config: HomeServerConfig,
-    run_command: Callable[[], None] = reactor.run,
+    run_command: Callable[[], None] = lambda: reactor.run(),
 ) -> None:
    """Run the reactor in the main process

@@ -141,7 +141,7 @@ def start_reactor(
    daemonize: bool,
    print_pidfile: bool,
    logger: logging.Logger,
-    run_command: Callable[[], None] = reactor.run,
+    run_command: Callable[[], None] = lambda: reactor.run(),
 ) -> None:
    """Run the reactor in the main process

@@ -0,0 +1,140 @@
+# Copyright 2022 The Matrix.org Foundation C.I.C.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+#
+#
+# This script is intended for test purposes only (within Complement).
+# It spawns multiple workers, whilst only going through the code loading process
+# once.
+#
+# TODO more docs
+#      Each worker is specified as an argument group (each argument group is
+#      separated by '--').
+#     ........
+#
+# Usage:
+#   python -m synapse.app._complement_fork_starter \
+#     synapse.app.homeserver [args..] -- \
+#     synapse.app.generic_worker [args..] -- \
+#   ...
+#     synapse.app.generic_worker [args..]
+import importlib
+import itertools
+import multiprocessing
+import sys
+from typing import Any, Callable, List
+
+from twisted.internet.main import installReactor
+
+
+class ProxiedReactor:
+    """
+    Global state is horrible. Use this proxy reactor so we can 'reinstall'
+    the reactor by changing the target of the proxy.
+    """
+
+    def __init__(self) -> None:
+        self.___reactor_target: Any = None
+
+    def ___install(self, new_reactor: Any) -> None:
+        self.___reactor_target = new_reactor
+
+    def __getattr__(self, attr_name: str) -> Any:
+        if attr_name == "___install":
+            return self.___install
+        return getattr(self.___reactor_target, attr_name)
+
+
+def _worker_entrypoint(
+    func: Callable[[], None], proxy_reactor: ProxiedReactor, args: List[str]
+) -> None:
+    sys.argv = args
+
+    from twisted.internet.epollreactor import EPollReactor
+
+    proxy_reactor.___install(EPollReactor())
+    func()
+
+
+def main() -> None:
+    # Split up the arguments into each workers' arguments
+    # Strip out any newlines.
+    # HACK
+    db_config_path = sys.argv[1]
+    args = [arg.replace("\n", "") for arg in sys.argv[2:]]
+    args_by_worker: List[List[str]] = [
+        list(args)
+        for cond, args in itertools.groupby(args, lambda ele: ele != "--")
+        if cond and args
+    ]
+
+    # Prevent Twisted from installing a shared reactor that all the workers will
+    # pick up.
+    proxy_reactor = ProxiedReactor()
+    installReactor(proxy_reactor)
+
+    # Import the entrypoints for all the workers
+    worker_functions = []
+    for worker_args in args_by_worker:
+        worker_module = importlib.import_module(worker_args[0])
+        worker_functions.append(worker_module.main)
+
+    # At this point, we've imported all the main entrypoints for all the workers.
+    # Now we basically just fork() out to create the workers we need.
+    # Because we're using fork(), all the workers get a clone of this launcher's
+    # memory space and don't need to repeat the work of loading the code!
+    # Instead of using fork() directly, we use the multiprocessing library,
+    # which *can* use fork() on Unix platforms.
+    # Now we fork our process!
+
+    # TODO Can we do this better?
+    # We need to prepare the database first as otherwise all the workers will
+    # try to create a schema version table and some will crash out.
+    # HACK
+    from synapse._scripts import update_synapse_database
+
+    update_proc = multiprocessing.Process(
+        target=_worker_entrypoint,
+        args=(
+            update_synapse_database.main,
+            proxy_reactor,
+            [
+                "update_synapse_database",
+                "--database-config",
+                db_config_path,
+                "--run-background-updates",
+            ],
+        ),
+    )
+    print("===== PREPARING DATABASE =====", file=sys.stderr)
+    update_proc.start()
+    print("JNG UPROC", file=sys.stderr)
+    update_proc.join()
+    print("===== PREPARED DATABASE =====", file=sys.stderr)
+
+    processes = []
+    for (func, worker_args) in zip(worker_functions, args_by_worker):
+        process = multiprocessing.Process(
+            target=_worker_entrypoint, args=(func, proxy_reactor, worker_args)
+        )
+        process.start()
+        processes.append(process)
+
+    # Be a good parent and wait for our children to die before exiting.
+    for process in processes:
+        process.join()
+
+
+if __name__ == "__main__":
+    main()
@@ -17,6 +17,11 @@ import sys
 from synapse.app.generic_worker import start
 from synapse.util.logcontext import LoggingContext

-if __name__ == "__main__":
+
+def main() -> None:
    with LoggingContext("main"):
        start(sys.argv[1:])
+
+
+if __name__ == "__main__":
+    main()
@@ -17,6 +17,11 @@ import sys
 from synapse.app.generic_worker import start
 from synapse.util.logcontext import LoggingContext

-if __name__ == "__main__":
+
+def main() -> None:
    with LoggingContext("main"):
        start(sys.argv[1:])
+
+
+if __name__ == "__main__":
+    main()
@@ -17,6 +17,11 @@ import sys
 from synapse.app.generic_worker import start
 from synapse.util.logcontext import LoggingContext

-if __name__ == "__main__":
+
+def main() -> None:
    with LoggingContext("main"):
        start(sys.argv[1:])
+
+
+if __name__ == "__main__":
+    main()
@@ -17,6 +17,11 @@ import sys
 from synapse.app.generic_worker import start
 from synapse.util.logcontext import LoggingContext

-if __name__ == "__main__":
+
+def main() -> None:
    with LoggingContext("main"):
        start(sys.argv[1:])
+
+
+if __name__ == "__main__":
+    main()
@@ -17,6 +17,11 @@ import sys
 from synapse.app.generic_worker import start
 from synapse.util.logcontext import LoggingContext

-if __name__ == "__main__":
+
+def main() -> None:
    with LoggingContext("main"):
        start(sys.argv[1:])
+
+
+if __name__ == "__main__":
+    main()
@@ -17,6 +17,11 @@ import sys
 from synapse.app.generic_worker import start
 from synapse.util.logcontext import LoggingContext

-if __name__ == "__main__":
+
+def main() -> None:
    with LoggingContext("main"):
        start(sys.argv[1:])
+
+
+if __name__ == "__main__":
+    main()
@@ -69,7 +69,6 @@ from synapse.rest.admin import register_servlets_for_media_repo
 from synapse.rest.client import (
    account_data,
    events,
-    groups,
    initial_sync,
    login,
    presence,
@@ -78,6 +77,7 @@ from synapse.rest.client import (
    read_marker,
    receipts,
    room,
+    room_batch,
    room_keys,
    sendtodevice,
    sync,
@@ -87,7 +87,7 @@ from synapse.rest.client import (
    voip,
 )
 from synapse.rest.client._base import client_patterns
-from synapse.rest.client.account import ThreepidRestServlet
+from synapse.rest.client.account import ThreepidRestServlet, WhoamiRestServlet
 from synapse.rest.client.devices import DevicesRestServlet
 from synapse.rest.client.keys import (
    KeyChangesServlet,
@@ -289,6 +289,7 @@ class GenericWorkerServer(HomeServer):
                    RegistrationTokenValidityRestServlet(self).register(resource)
                    login.register_servlets(self, resource)
                    ThreepidRestServlet(self).register(resource)
+                    WhoamiRestServlet(self).register(resource)
                    DevicesRestServlet(self).register(resource)

                    # Read-only
@@ -308,6 +309,7 @@ class GenericWorkerServer(HomeServer):
                    room.register_servlets(self, resource, is_worker=True)
                    room.register_deprecated_servlets(self, resource)
                    initial_sync.register_servlets(self, resource)
+                    room_batch.register_servlets(self, resource)
                    room_keys.register_servlets(self, resource)
                    tags.register_servlets(self, resource)
                    account_data.register_servlets(self, resource)
@@ -320,9 +322,6 @@ class GenericWorkerServer(HomeServer):

                    presence.register_servlets(self, resource)

-                    if self.config.experimental.groups_enabled:
-                        groups.register_servlets(self, resource)
-
                    resources.update({CLIENT_API_PREFIX: resource})

                    resources.update(build_synapse_client_resource_tree(self))
@@ -17,6 +17,11 @@ import sys
 from synapse.app.generic_worker import start
 from synapse.util.logcontext import LoggingContext

-if __name__ == "__main__":
+
+def main() -> None:
    with LoggingContext("main"):
        start(sys.argv[1:])
+
+
+if __name__ == "__main__":
+    main()
@@ -17,6 +17,11 @@ import sys
 from synapse.app.generic_worker import start
 from synapse.util.logcontext import LoggingContext

-if __name__ == "__main__":
+
+def main() -> None:
    with LoggingContext("main"):
        start(sys.argv[1:])
+
+
+if __name__ == "__main__":
+    main()
@@ -17,6 +17,11 @@ import sys
 from synapse.app.generic_worker import start
 from synapse.util.logcontext import LoggingContext

-if __name__ == "__main__":
+
+def main() -> None:
    with LoggingContext("main"):
        start(sys.argv[1:])
+
+
+if __name__ == "__main__":
+    main()
@@ -17,6 +17,11 @@ import sys
 from synapse.app.generic_worker import start
 from synapse.util.logcontext import LoggingContext

-if __name__ == "__main__":
+
+def main() -> None:
    with LoggingContext("main"):
        start(sys.argv[1:])
+
+
+if __name__ == "__main__":
+    main()
@@ -23,13 +23,7 @@ from netaddr import IPSet

 from synapse.api.constants import EventTypes
 from synapse.events import EventBase
-from synapse.types import (
-    DeviceListUpdates,
-    GroupID,
-    JsonDict,
-    UserID,
-    get_domain_from_id,
-)
+from synapse.types import DeviceListUpdates, JsonDict, UserID
 from synapse.util.caches.descriptors import _CacheContext, cached

 if TYPE_CHECKING:
@@ -55,7 +49,6 @@ class ApplicationServiceState(Enum):
@attr.s(slots=True, frozen=True, auto_attribs=True)
 class Namespace:
    exclusive: bool
-    group_id: Optional[str]
    regex: Pattern[str]


@@ -141,30 +134,13 @@ class ApplicationService:
                exclusive = regex_obj.get("exclusive")
                if not isinstance(exclusive, bool):
                    raise ValueError("Expected bool for 'exclusive' in ns '%s'" % ns)
-                group_id = regex_obj.get("group_id")
-                if group_id:
-                    if not isinstance(group_id, str):
-                        raise ValueError(
-                            "Expected string for 'group_id' in ns '%s'" % ns
-                        )
-                    try:
-                        GroupID.from_string(group_id)
-                    except Exception:
-                        raise ValueError(
-                            "Expected valid group ID for 'group_id' in ns '%s'" % ns
-                        )
-
-                    if get_domain_from_id(group_id) != self.server_name:
-                        raise ValueError(
-                            "Expected 'group_id' to be this host in ns '%s'" % ns
-                        )

                regex = regex_obj.get("regex")
                if not isinstance(regex, str):
                    raise ValueError("Expected string for 'regex' in ns '%s'" % ns)

                # Pre-compile regex.
-                result[ns].append(Namespace(exclusive, group_id, re.compile(regex)))
+                result[ns].append(Namespace(exclusive, re.compile(regex)))

        return result

@@ -369,21 +345,6 @@ class ApplicationService:
            if namespace.exclusive
        ]

-    def get_groups_for_user(self, user_id: str) -> Iterable[str]:
-        """Get the groups that this user is associated with by this AS
-
-        Args:
-            user_id: The ID of the user.
-
-        Returns:
-            An iterable that yields group_id strings.
-        """
-        return (
-            namespace.group_id
-            for namespace in self.namespaces[ApplicationService.NS_USERS]
-            if namespace.group_id and namespace.regex.match(user_id)
-        )
-
    def is_rate_limited(self) -> bool:
        return self.rate_limited

@@ -14,7 +14,7 @@
 # limitations under the License.
 import logging
 import urllib.parse
-from typing import TYPE_CHECKING, Dict, Iterable, List, Optional, Tuple
+from typing import TYPE_CHECKING, Any, Dict, Iterable, List, Mapping, Optional, Tuple

 from prometheus_client import Counter
 from typing_extensions import TypeGuard
@@ -155,6 +155,9 @@ class ApplicationServiceApi(SimpleHttpClient):
        if service.url is None:
            return []

+        # This is required by the configuration.
+        assert service.hs_token is not None
+
        uri = "%s%s/thirdparty/%s/%s" % (
            service.url,
            APP_SERVICE_PREFIX,
@@ -162,7 +165,11 @@ class ApplicationServiceApi(SimpleHttpClient):
            urllib.parse.quote(protocol),
        )
        try:
-            response = await self.get_json(uri, fields)
+            args: Mapping[Any, Any] = {
+                **fields,
+                b"access_token": service.hs_token,
+            }
+            response = await self.get_json(uri, args=args)
            if not isinstance(response, list):
                logger.warning(
                    "query_3pe to %s returned an invalid response %r", uri, response
@@ -190,13 +197,15 @@ class ApplicationServiceApi(SimpleHttpClient):
            return {}

        async def _get() -> Optional[JsonDict]:
+            # This is required by the configuration.
+            assert service.hs_token is not None
            uri = "%s%s/thirdparty/protocol/%s" % (
                service.url,
                APP_SERVICE_PREFIX,
                urllib.parse.quote(protocol),
            )
            try:
-                info = await self.get_json(uri)
+                info = await self.get_json(uri, {"access_token": service.hs_token})

                if not _is_valid_3pe_metadata(info):
                    logger.warning(
@@ -384,6 +384,11 @@ class _TransactionController:
            device_list_summary: The device list summary to include in the transaction.
        """
        try:
+            service_is_up = await self._is_service_up(service)
+            # Don't create empty txns when in recovery mode (ephemeral events are dropped)
+            if not service_is_up and not events:
+                return
+
            txn = await self.store.create_appservice_txn(
                service=service,
                events=events,
@@ -393,7 +398,6 @@ class _TransactionController:
                unused_fallback_keys=unused_fallback_keys or {},
                device_list_summary=device_list_summary or DeviceListUpdates(),
            )
-            service_is_up = await self._is_service_up(service)
            if service_is_up:
                sent = await txn.send(self.as_api)
                if sent:
@@ -32,7 +32,6 @@ from synapse.config import (
    emailconfig,
    experimental,
    federation,
-    groups,
    jwt,
    key,
    logger,
@@ -107,7 +106,6 @@ class RootConfig:
    push: push.PushConfig
    spamchecker: spam_checker.SpamCheckerConfig
    room: room.RoomConfig
-    groups: groups.GroupsConfig
    userdirectory: user_directory.UserDirectoryConfig
    consent: consent.ConsentConfig
    stats: stats.StatsConfig
@@ -73,9 +73,6 @@ class ExperimentalConfig(Config):
        # MSC3720 (Account status endpoint)
        self.msc3720_enabled: bool = experimental.get("msc3720_enabled", False)

-        # The deprecated groups feature.
-        self.groups_enabled: bool = experimental.get("groups_enabled", False)
-
        # MSC2654: Unread counts
        self.msc2654_enabled: bool = experimental.get("msc2654_enabled", False)

@@ -84,3 +81,6 @@ class ExperimentalConfig(Config):

        # MSC3786 (Add a default push rule to ignore m.room.server_acl events)
        self.msc3786_enabled: bool = experimental.get("msc3786_enabled", False)
+
+        # MSC3772: A push rule for mutual relations.
+        self.msc3772_enabled: bool = experimental.get("msc3772_enabled", False)
@@ -1,39 +0,0 @@
-# Copyright 2017 New Vector Ltd
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from typing import Any
-
-from synapse.types import JsonDict
-
-from ._base import Config
-
-
-class GroupsConfig(Config):
-    section = "groups"
-
-    def read_config(self, config: JsonDict, **kwargs: Any) -> None:
-        self.enable_group_creation = config.get("enable_group_creation", False)
-        self.group_creation_prefix = config.get("group_creation_prefix", "")
-
-    def generate_config_section(self, **kwargs: Any) -> str:
-        return """\
-        # Uncomment to allow non-server-admin users to create groups on this server
-        #
-        #enable_group_creation: true
-
-        # If enabled, non server admins can only create groups with local parts
-        # starting with this prefix
-        #
-        #group_creation_prefix: "unofficial_"
-        """
@@ -25,7 +25,6 @@ from .database import DatabaseConfig
 from .emailconfig import EmailConfig
 from .experimental import ExperimentalConfig
 from .federation import FederationConfig
-from .groups import GroupsConfig
 from .jwt import JWTConfig
 from .key import KeyConfig
 from .logger import LoggingConfig
@@ -89,7 +88,6 @@ class HomeServerConfig(RootConfig):
        PushConfig,
        SpamCheckerConfig,
        RoomConfig,
-        GroupsConfig,
        UserDirectoryConfig,
        ConsentConfig,
        StatsConfig,
@@ -223,7 +223,7 @@ class PerDestinationQueue:
        """Marks that the destination has new data to send, without starting a
        new transaction.

-        If a transaction loop is already in progress then a new transcation will
+        If a transaction loop is already in progress then a new transaction will
        be attempted when the current one finishes.
        """

@@ -49,11 +49,6 @@ from synapse.types import JsonDict

 logger = logging.getLogger(__name__)

-# Send join responses can be huge, so we set a separate limit here. The response
-# is parsed in a streaming manner, which helps alleviate the issue of memory
-# usage a bit.
-MAX_RESPONSE_SIZE_SEND_JOIN = 500 * 1024 * 1024
-

 class TransportLayerClient:
    """Sends federation HTTP requests to other servers"""
@@ -349,7 +344,6 @@ class TransportLayerClient:
            path=path,
            data=content,
            parser=SendJoinParser(room_version, v1_api=True),
-            max_response_size=MAX_RESPONSE_SIZE_SEND_JOIN,
        )

    async def send_join_v2(
@@ -372,7 +366,6 @@ class TransportLayerClient:
            args=query_params,
            data=content,
            parser=SendJoinParser(room_version, v1_api=False),
-            max_response_size=MAX_RESPONSE_SIZE_SEND_JOIN,
        )

    async def send_leave_v1(
@@ -1360,6 +1353,11 @@ class SendJoinParser(ByteParser[SendJoinResponse]):

    CONTENT_TYPE = "application/json"

+    # /send_join responses can be huge, so we override the size limit here. The response
+    # is parsed in a streaming manner, which helps alleviate the issue of memory
+    # usage a bit.
+    MAX_RESPONSE_SIZE = 500 * 1024 * 1024
+
    def __init__(self, room_version: RoomVersion, v1_api: bool):
        self._response = SendJoinResponse([], [], event_dict={})
        self._room_version = room_version
@@ -1427,6 +1425,9 @@ class _StateParser(ByteParser[StateRequestResponse]):

    CONTENT_TYPE = "application/json"

+    # As with /send_join, /state responses can be huge.
+    MAX_RESPONSE_SIZE = 500 * 1024 * 1024
+
    def __init__(self, room_version: RoomVersion):
        self._response = StateRequestResponse([], [])
        self._room_version = room_version
@@ -27,10 +27,6 @@ from synapse.federation.transport.server.federation import (
    FederationAccountStatusServlet,
    FederationTimestampLookupServlet,
 )
-from synapse.federation.transport.server.groups_local import GROUP_LOCAL_SERVLET_CLASSES
-from synapse.federation.transport.server.groups_server import (
-    GROUP_SERVER_SERVLET_CLASSES,
-)
 from synapse.http.server import HttpServer, JsonResource
 from synapse.http.servlet import (
    parse_boolean_from_args,
@@ -199,38 +195,6 @@ class PublicRoomList(BaseFederationServlet):
        return 200, data


-class FederationGroupsRenewAttestaionServlet(BaseFederationServlet):
-    """A group or user's server renews their attestation"""
-
-    PATH = "/groups/(?P<group_id>[^/]*)/renew_attestation/(?P<user_id>[^/]*)"
-
-    def __init__(
-        self,
-        hs: "HomeServer",
-        authenticator: Authenticator,
-        ratelimiter: FederationRateLimiter,
-        server_name: str,
-    ):
-        super().__init__(hs, authenticator, ratelimiter, server_name)
-        self.handler = hs.get_groups_attestation_renewer()
-
-    async def on_POST(
-        self,
-        origin: str,
-        content: JsonDict,
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        user_id: str,
-    ) -> Tuple[int, JsonDict]:
-        # We don't need to check auth here as we check the attestation signatures
-
-        new_content = await self.handler.on_renew_attestation(
-            group_id, user_id, content
-        )
-
-        return 200, new_content
-
-
 class OpenIdUserInfo(BaseFederationServlet):
    """
    Exchange a bearer token for information about a user.
@@ -292,16 +256,9 @@ class OpenIdUserInfo(BaseFederationServlet):
 SERVLET_GROUPS: Dict[str, Iterable[Type[BaseFederationServlet]]] = {
    "federation": FEDERATION_SERVLET_CLASSES,
    "room_list": (PublicRoomList,),
-    "group_server": GROUP_SERVER_SERVLET_CLASSES,
-    "group_local": GROUP_LOCAL_SERVLET_CLASSES,
-    "group_attestation": (FederationGroupsRenewAttestaionServlet,),
    "openid": (OpenIdUserInfo,),
 }

-DEFAULT_SERVLET_GROUPS = ("federation", "room_list", "openid")
-
-GROUP_SERVLET_GROUPS = ("group_server", "group_local", "group_attestation")
-

 def register_servlets(
    hs: "HomeServer",
@@ -324,10 +281,7 @@ def register_servlets(
            Defaults to ``DEFAULT_SERVLET_GROUPS``.
    """
    if not servlet_groups:
-        servlet_groups = DEFAULT_SERVLET_GROUPS
-        # Only allow the groups servlets if the deprecated groups feature is enabled.
-        if hs.config.experimental.groups_enabled:
-            servlet_groups = servlet_groups + GROUP_SERVLET_GROUPS
+        servlet_groups = SERVLET_GROUPS.keys()

    for servlet_group in servlet_groups:
        # Skip unknown servlet groups.
@@ -650,10 +650,6 @@ class FederationRoomHierarchyServlet(BaseFederationServlet):
        )


-class FederationRoomHierarchyUnstableServlet(FederationRoomHierarchyServlet):
-    PREFIX = FEDERATION_UNSTABLE_PREFIX + "/org.matrix.msc2946"
-
-
 class RoomComplexityServlet(BaseFederationServlet):
    """
    Indicates to other servers how complex (and therefore likely
@@ -752,7 +748,6 @@ FEDERATION_SERVLET_CLASSES: Tuple[Type[BaseFederationServlet], ...] = (
    FederationVersionServlet,
    RoomComplexityServlet,
    FederationRoomHierarchyServlet,
-    FederationRoomHierarchyUnstableServlet,
    FederationV1SendKnockServlet,
    FederationMakeKnockServlet,
    FederationAccountStatusServlet,
@@ -1,115 +0,0 @@
-#  Copyright 2021 The Matrix.org Foundation C.I.C.
-#
-#  Licensed under the Apache License, Version 2.0 (the "License");
-#  you may not use this file except in compliance with the License.
-#  You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-#  Unless required by applicable law or agreed to in writing, software
-#  distributed under the License is distributed on an "AS IS" BASIS,
-#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-#  See the License for the specific language governing permissions and
-#  limitations under the License.
-from typing import TYPE_CHECKING, Dict, List, Tuple, Type
-
-from synapse.api.errors import SynapseError
-from synapse.federation.transport.server._base import (
-    Authenticator,
-    BaseFederationServlet,
-)
-from synapse.handlers.groups_local import GroupsLocalHandler
-from synapse.types import JsonDict, get_domain_from_id
-from synapse.util.ratelimitutils import FederationRateLimiter
-
-if TYPE_CHECKING:
-    from synapse.server import HomeServer
-
-
-class BaseGroupsLocalServlet(BaseFederationServlet):
-    """Abstract base class for federation servlet classes which provides a groups local handler.
-
-    See BaseFederationServlet for more information.
-    """
-
-    def __init__(
-        self,
-        hs: "HomeServer",
-        authenticator: Authenticator,
-        ratelimiter: FederationRateLimiter,
-        server_name: str,
-    ):
-        super().__init__(hs, authenticator, ratelimiter, server_name)
-        self.handler = hs.get_groups_local_handler()
-
-
-class FederationGroupsLocalInviteServlet(BaseGroupsLocalServlet):
-    """A group server has invited a local user"""
-
-    PATH = "/groups/local/(?P<group_id>[^/]*)/users/(?P<user_id>[^/]*)/invite"
-
-    async def on_POST(
-        self,
-        origin: str,
-        content: JsonDict,
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        user_id: str,
-    ) -> Tuple[int, JsonDict]:
-        if get_domain_from_id(group_id) != origin:
-            raise SynapseError(403, "group_id doesn't match origin")
-
-        assert isinstance(
-            self.handler, GroupsLocalHandler
-        ), "Workers cannot handle group invites."
-
-        new_content = await self.handler.on_invite(group_id, user_id, content)
-
-        return 200, new_content
-
-
-class FederationGroupsRemoveLocalUserServlet(BaseGroupsLocalServlet):
-    """A group server has removed a local user"""
-
-    PATH = "/groups/local/(?P<group_id>[^/]*)/users/(?P<user_id>[^/]*)/remove"
-
-    async def on_POST(
-        self,
-        origin: str,
-        content: JsonDict,
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        user_id: str,
-    ) -> Tuple[int, None]:
-        if get_domain_from_id(group_id) != origin:
-            raise SynapseError(403, "user_id doesn't match origin")
-
-        assert isinstance(
-            self.handler, GroupsLocalHandler
-        ), "Workers cannot handle group removals."
-
-        await self.handler.user_removed_from_group(group_id, user_id, content)
-
-        return 200, None
-
-
-class FederationGroupsBulkPublicisedServlet(BaseGroupsLocalServlet):
-    """Get roles in a group"""
-
-    PATH = "/get_groups_publicised"
-
-    async def on_POST(
-        self, origin: str, content: JsonDict, query: Dict[bytes, List[bytes]]
-    ) -> Tuple[int, JsonDict]:
-        resp = await self.handler.bulk_get_publicised_groups(
-            content["user_ids"], proxy=False
-        )
-
-        return 200, resp
-
-
-GROUP_LOCAL_SERVLET_CLASSES: Tuple[Type[BaseFederationServlet], ...] = (
-    FederationGroupsLocalInviteServlet,
-    FederationGroupsRemoveLocalUserServlet,
-    FederationGroupsBulkPublicisedServlet,
-)
@@ -1,755 +0,0 @@
-#  Copyright 2021 The Matrix.org Foundation C.I.C.
-#
-#  Licensed under the Apache License, Version 2.0 (the "License");
-#  you may not use this file except in compliance with the License.
-#  You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-#  Unless required by applicable law or agreed to in writing, software
-#  distributed under the License is distributed on an "AS IS" BASIS,
-#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-#  See the License for the specific language governing permissions and
-#  limitations under the License.
-from typing import TYPE_CHECKING, Dict, List, Tuple, Type
-
-from typing_extensions import Literal
-
-from synapse.api.constants import MAX_GROUP_CATEGORYID_LENGTH, MAX_GROUP_ROLEID_LENGTH
-from synapse.api.errors import Codes, SynapseError
-from synapse.federation.transport.server._base import (
-    Authenticator,
-    BaseFederationServlet,
-)
-from synapse.http.servlet import parse_string_from_args
-from synapse.types import JsonDict, get_domain_from_id
-from synapse.util.ratelimitutils import FederationRateLimiter
-
-if TYPE_CHECKING:
-    from synapse.server import HomeServer
-
-
-class BaseGroupsServerServlet(BaseFederationServlet):
-    """Abstract base class for federation servlet classes which provides a groups server handler.
-
-    See BaseFederationServlet for more information.
-    """
-
-    def __init__(
-        self,
-        hs: "HomeServer",
-        authenticator: Authenticator,
-        ratelimiter: FederationRateLimiter,
-        server_name: str,
-    ):
-        super().__init__(hs, authenticator, ratelimiter, server_name)
-        self.handler = hs.get_groups_server_handler()
-
-
-class FederationGroupsProfileServlet(BaseGroupsServerServlet):
-    """Get/set the basic profile of a group on behalf of a user"""
-
-    PATH = "/groups/(?P<group_id>[^/]*)/profile"
-
-    async def on_GET(
-        self,
-        origin: str,
-        content: Literal[None],
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        new_content = await self.handler.get_group_profile(group_id, requester_user_id)
-
-        return 200, new_content
-
-    async def on_POST(
-        self,
-        origin: str,
-        content: JsonDict,
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        new_content = await self.handler.update_group_profile(
-            group_id, requester_user_id, content
-        )
-
-        return 200, new_content
-
-
-class FederationGroupsSummaryServlet(BaseGroupsServerServlet):
-    PATH = "/groups/(?P<group_id>[^/]*)/summary"
-
-    async def on_GET(
-        self,
-        origin: str,
-        content: Literal[None],
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        new_content = await self.handler.get_group_summary(group_id, requester_user_id)
-
-        return 200, new_content
-
-
-class FederationGroupsRoomsServlet(BaseGroupsServerServlet):
-    """Get the rooms in a group on behalf of a user"""
-
-    PATH = "/groups/(?P<group_id>[^/]*)/rooms"
-
-    async def on_GET(
-        self,
-        origin: str,
-        content: Literal[None],
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        new_content = await self.handler.get_rooms_in_group(group_id, requester_user_id)
-
-        return 200, new_content
-
-
-class FederationGroupsAddRoomsServlet(BaseGroupsServerServlet):
-    """Add/remove room from group"""
-
-    PATH = "/groups/(?P<group_id>[^/]*)/room/(?P<room_id>[^/]*)"
-
-    async def on_POST(
-        self,
-        origin: str,
-        content: JsonDict,
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        room_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        new_content = await self.handler.add_room_to_group(
-            group_id, requester_user_id, room_id, content
-        )
-
-        return 200, new_content
-
-    async def on_DELETE(
-        self,
-        origin: str,
-        content: Literal[None],
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        room_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        new_content = await self.handler.remove_room_from_group(
-            group_id, requester_user_id, room_id
-        )
-
-        return 200, new_content
-
-
-class FederationGroupsAddRoomsConfigServlet(BaseGroupsServerServlet):
-    """Update room config in group"""
-
-    PATH = (
-        "/groups/(?P<group_id>[^/]*)/room/(?P<room_id>[^/]*)"
-        "/config/(?P<config_key>[^/]*)"
-    )
-
-    async def on_POST(
-        self,
-        origin: str,
-        content: JsonDict,
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        room_id: str,
-        config_key: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        result = await self.handler.update_room_in_group(
-            group_id, requester_user_id, room_id, config_key, content
-        )
-
-        return 200, result
-
-
-class FederationGroupsUsersServlet(BaseGroupsServerServlet):
-    """Get the users in a group on behalf of a user"""
-
-    PATH = "/groups/(?P<group_id>[^/]*)/users"
-
-    async def on_GET(
-        self,
-        origin: str,
-        content: Literal[None],
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        new_content = await self.handler.get_users_in_group(group_id, requester_user_id)
-
-        return 200, new_content
-
-
-class FederationGroupsInvitedUsersServlet(BaseGroupsServerServlet):
-    """Get the users that have been invited to a group"""
-
-    PATH = "/groups/(?P<group_id>[^/]*)/invited_users"
-
-    async def on_GET(
-        self,
-        origin: str,
-        content: Literal[None],
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        new_content = await self.handler.get_invited_users_in_group(
-            group_id, requester_user_id
-        )
-
-        return 200, new_content
-
-
-class FederationGroupsInviteServlet(BaseGroupsServerServlet):
-    """Ask a group server to invite someone to the group"""
-
-    PATH = "/groups/(?P<group_id>[^/]*)/users/(?P<user_id>[^/]*)/invite"
-
-    async def on_POST(
-        self,
-        origin: str,
-        content: JsonDict,
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        user_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        new_content = await self.handler.invite_to_group(
-            group_id, user_id, requester_user_id, content
-        )
-
-        return 200, new_content
-
-
-class FederationGroupsAcceptInviteServlet(BaseGroupsServerServlet):
-    """Accept an invitation from the group server"""
-
-    PATH = "/groups/(?P<group_id>[^/]*)/users/(?P<user_id>[^/]*)/accept_invite"
-
-    async def on_POST(
-        self,
-        origin: str,
-        content: JsonDict,
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        user_id: str,
-    ) -> Tuple[int, JsonDict]:
-        if get_domain_from_id(user_id) != origin:
-            raise SynapseError(403, "user_id doesn't match origin")
-
-        new_content = await self.handler.accept_invite(group_id, user_id, content)
-
-        return 200, new_content
-
-
-class FederationGroupsJoinServlet(BaseGroupsServerServlet):
-    """Attempt to join a group"""
-
-    PATH = "/groups/(?P<group_id>[^/]*)/users/(?P<user_id>[^/]*)/join"
-
-    async def on_POST(
-        self,
-        origin: str,
-        content: JsonDict,
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        user_id: str,
-    ) -> Tuple[int, JsonDict]:
-        if get_domain_from_id(user_id) != origin:
-            raise SynapseError(403, "user_id doesn't match origin")
-
-        new_content = await self.handler.join_group(group_id, user_id, content)
-
-        return 200, new_content
-
-
-class FederationGroupsRemoveUserServlet(BaseGroupsServerServlet):
-    """Leave or kick a user from the group"""
-
-    PATH = "/groups/(?P<group_id>[^/]*)/users/(?P<user_id>[^/]*)/remove"
-
-    async def on_POST(
-        self,
-        origin: str,
-        content: JsonDict,
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        user_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        new_content = await self.handler.remove_user_from_group(
-            group_id, user_id, requester_user_id, content
-        )
-
-        return 200, new_content
-
-
-class FederationGroupsSummaryRoomsServlet(BaseGroupsServerServlet):
-    """Add/remove a room from the group summary, with optional category.
-
-    Matches both:
-        - /groups/:group/summary/rooms/:room_id
-        - /groups/:group/summary/categories/:category/rooms/:room_id
-    """
-
-    PATH = (
-        "/groups/(?P<group_id>[^/]*)/summary"
-        "(/categories/(?P<category_id>[^/]+))?"
-        "/rooms/(?P<room_id>[^/]*)"
-    )
-
-    async def on_POST(
-        self,
-        origin: str,
-        content: JsonDict,
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        category_id: str,
-        room_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        if category_id == "":
-            raise SynapseError(
-                400, "category_id cannot be empty string", Codes.INVALID_PARAM
-            )
-
-        if len(category_id) > MAX_GROUP_CATEGORYID_LENGTH:
-            raise SynapseError(
-                400,
-                "category_id may not be longer than %s characters"
-                % (MAX_GROUP_CATEGORYID_LENGTH,),
-                Codes.INVALID_PARAM,
-            )
-
-        resp = await self.handler.update_group_summary_room(
-            group_id,
-            requester_user_id,
-            room_id=room_id,
-            category_id=category_id,
-            content=content,
-        )
-
-        return 200, resp
-
-    async def on_DELETE(
-        self,
-        origin: str,
-        content: Literal[None],
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        category_id: str,
-        room_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        if category_id == "":
-            raise SynapseError(400, "category_id cannot be empty string")
-
-        resp = await self.handler.delete_group_summary_room(
-            group_id, requester_user_id, room_id=room_id, category_id=category_id
-        )
-
-        return 200, resp
-
-
-class FederationGroupsCategoriesServlet(BaseGroupsServerServlet):
-    """Get all categories for a group"""
-
-    PATH = "/groups/(?P<group_id>[^/]*)/categories/?"
-
-    async def on_GET(
-        self,
-        origin: str,
-        content: Literal[None],
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        resp = await self.handler.get_group_categories(group_id, requester_user_id)
-
-        return 200, resp
-
-
-class FederationGroupsCategoryServlet(BaseGroupsServerServlet):
-    """Add/remove/get a category in a group"""
-
-    PATH = "/groups/(?P<group_id>[^/]*)/categories/(?P<category_id>[^/]+)"
-
-    async def on_GET(
-        self,
-        origin: str,
-        content: Literal[None],
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        category_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        resp = await self.handler.get_group_category(
-            group_id, requester_user_id, category_id
-        )
-
-        return 200, resp
-
-    async def on_POST(
-        self,
-        origin: str,
-        content: JsonDict,
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        category_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        if category_id == "":
-            raise SynapseError(400, "category_id cannot be empty string")
-
-        if len(category_id) > MAX_GROUP_CATEGORYID_LENGTH:
-            raise SynapseError(
-                400,
-                "category_id may not be longer than %s characters"
-                % (MAX_GROUP_CATEGORYID_LENGTH,),
-                Codes.INVALID_PARAM,
-            )
-
-        resp = await self.handler.upsert_group_category(
-            group_id, requester_user_id, category_id, content
-        )
-
-        return 200, resp
-
-    async def on_DELETE(
-        self,
-        origin: str,
-        content: Literal[None],
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        category_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        if category_id == "":
-            raise SynapseError(400, "category_id cannot be empty string")
-
-        resp = await self.handler.delete_group_category(
-            group_id, requester_user_id, category_id
-        )
-
-        return 200, resp
-
-
-class FederationGroupsRolesServlet(BaseGroupsServerServlet):
-    """Get roles in a group"""
-
-    PATH = "/groups/(?P<group_id>[^/]*)/roles/?"
-
-    async def on_GET(
-        self,
-        origin: str,
-        content: Literal[None],
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        resp = await self.handler.get_group_roles(group_id, requester_user_id)
-
-        return 200, resp
-
-
-class FederationGroupsRoleServlet(BaseGroupsServerServlet):
-    """Add/remove/get a role in a group"""
-
-    PATH = "/groups/(?P<group_id>[^/]*)/roles/(?P<role_id>[^/]+)"
-
-    async def on_GET(
-        self,
-        origin: str,
-        content: Literal[None],
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        role_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        resp = await self.handler.get_group_role(group_id, requester_user_id, role_id)
-
-        return 200, resp
-
-    async def on_POST(
-        self,
-        origin: str,
-        content: JsonDict,
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        role_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        if role_id == "":
-            raise SynapseError(
-                400, "role_id cannot be empty string", Codes.INVALID_PARAM
-            )
-
-        if len(role_id) > MAX_GROUP_ROLEID_LENGTH:
-            raise SynapseError(
-                400,
-                "role_id may not be longer than %s characters"
-                % (MAX_GROUP_ROLEID_LENGTH,),
-                Codes.INVALID_PARAM,
-            )
-
-        resp = await self.handler.update_group_role(
-            group_id, requester_user_id, role_id, content
-        )
-
-        return 200, resp
-
-    async def on_DELETE(
-        self,
-        origin: str,
-        content: Literal[None],
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        role_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        if role_id == "":
-            raise SynapseError(400, "role_id cannot be empty string")
-
-        resp = await self.handler.delete_group_role(
-            group_id, requester_user_id, role_id
-        )
-
-        return 200, resp
-
-
-class FederationGroupsSummaryUsersServlet(BaseGroupsServerServlet):
-    """Add/remove a user from the group summary, with optional role.
-
-    Matches both:
-        - /groups/:group/summary/users/:user_id
-        - /groups/:group/summary/roles/:role/users/:user_id
-    """
-
-    PATH = (
-        "/groups/(?P<group_id>[^/]*)/summary"
-        "(/roles/(?P<role_id>[^/]+))?"
-        "/users/(?P<user_id>[^/]*)"
-    )
-
-    async def on_POST(
-        self,
-        origin: str,
-        content: JsonDict,
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        role_id: str,
-        user_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        if role_id == "":
-            raise SynapseError(400, "role_id cannot be empty string")
-
-        if len(role_id) > MAX_GROUP_ROLEID_LENGTH:
-            raise SynapseError(
-                400,
-                "role_id may not be longer than %s characters"
-                % (MAX_GROUP_ROLEID_LENGTH,),
-                Codes.INVALID_PARAM,
-            )
-
-        resp = await self.handler.update_group_summary_user(
-            group_id,
-            requester_user_id,
-            user_id=user_id,
-            role_id=role_id,
-            content=content,
-        )
-
-        return 200, resp
-
-    async def on_DELETE(
-        self,
-        origin: str,
-        content: Literal[None],
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-        role_id: str,
-        user_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        if role_id == "":
-            raise SynapseError(400, "role_id cannot be empty string")
-
-        resp = await self.handler.delete_group_summary_user(
-            group_id, requester_user_id, user_id=user_id, role_id=role_id
-        )
-
-        return 200, resp
-
-
-class FederationGroupsSettingJoinPolicyServlet(BaseGroupsServerServlet):
-    """Sets whether a group is joinable without an invite or knock"""
-
-    PATH = "/groups/(?P<group_id>[^/]*)/settings/m.join_policy"
-
-    async def on_PUT(
-        self,
-        origin: str,
-        content: JsonDict,
-        query: Dict[bytes, List[bytes]],
-        group_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester_user_id = parse_string_from_args(
-            query, "requester_user_id", required=True
-        )
-        if get_domain_from_id(requester_user_id) != origin:
-            raise SynapseError(403, "requester_user_id doesn't match origin")
-
-        new_content = await self.handler.set_group_join_policy(
-            group_id, requester_user_id, content
-        )
-
-        return 200, new_content
-
-
-GROUP_SERVER_SERVLET_CLASSES: Tuple[Type[BaseFederationServlet], ...] = (
-    FederationGroupsProfileServlet,
-    FederationGroupsSummaryServlet,
-    FederationGroupsRoomsServlet,
-    FederationGroupsUsersServlet,
-    FederationGroupsInvitedUsersServlet,
-    FederationGroupsInviteServlet,
-    FederationGroupsAcceptInviteServlet,
-    FederationGroupsJoinServlet,
-    FederationGroupsRemoveUserServlet,
-    FederationGroupsSummaryRoomsServlet,
-    FederationGroupsCategoriesServlet,
-    FederationGroupsCategoryServlet,
-    FederationGroupsRolesServlet,
-    FederationGroupsRoleServlet,
-    FederationGroupsSummaryUsersServlet,
-    FederationGroupsAddRoomsServlet,
-    FederationGroupsAddRoomsConfigServlet,
-    FederationGroupsSettingJoinPolicyServlet,
-)
@@ -1,218 +0,0 @@
-# Copyright 2017 Vector Creations Ltd
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Attestations ensure that users and groups can't lie about their memberships.
-
-When a user joins a group the HS and GS swap attestations, which allow them
-both to independently prove to third parties their membership.These
-attestations have a validity period so need to be periodically renewed.
-
-If a user leaves (or gets kicked out of) a group, either side can still use
-their attestation to "prove" their membership, until the attestation expires.
-Therefore attestations shouldn't be relied on to prove membership in important
-cases, but can for less important situations, e.g. showing a users membership
-of groups on their profile, showing flairs, etc.
-
-An attestation is a signed blob of json that looks like:
-
-    {
-        "user_id": "@foo:a.example.com",
-        "group_id": "+bar:b.example.com",
-        "valid_until_ms": 1507994728530,
-        "signatures":{"matrix.org":{"ed25519:auto":"..."}}
-    }
-"""
-
-import logging
-import random
-from typing import TYPE_CHECKING, Optional, Tuple
-
-from signedjson.sign import sign_json
-
-from twisted.internet.defer import Deferred
-
-from synapse.api.errors import HttpResponseException, RequestSendFailed, SynapseError
-from synapse.metrics.background_process_metrics import run_as_background_process
-from synapse.types import JsonDict, get_domain_from_id
-
-if TYPE_CHECKING:
-    from synapse.server import HomeServer
-
-logger = logging.getLogger(__name__)
-
-
-# Default validity duration for new attestations we create
-DEFAULT_ATTESTATION_LENGTH_MS = 3 * 24 * 60 * 60 * 1000
-
-# We add some jitter to the validity duration of attestations so that if we
-# add lots of users at once we don't need to renew them all at once.
-# The jitter is a multiplier picked randomly between the first and second number
-DEFAULT_ATTESTATION_JITTER = (0.9, 1.3)
-
-# Start trying to update our attestations when they come this close to expiring
-UPDATE_ATTESTATION_TIME_MS = 1 * 24 * 60 * 60 * 1000
-
-
-class GroupAttestationSigning:
-    """Creates and verifies group attestations."""
-
-    def __init__(self, hs: "HomeServer"):
-        self.keyring = hs.get_keyring()
-        self.clock = hs.get_clock()
-        self.server_name = hs.hostname
-        self.signing_key = hs.signing_key
-
-    async def verify_attestation(
-        self,
-        attestation: JsonDict,
-        group_id: str,
-        user_id: str,
-        server_name: Optional[str] = None,
-    ) -> None:
-        """Verifies that the given attestation matches the given parameters.
-
-        An optional server_name can be supplied to explicitly set which server's
-        signature is expected. Otherwise assumes that either the group_id or user_id
-        is local and uses the other's server as the one to check.
-        """
-
-        if not server_name:
-            if get_domain_from_id(group_id) == self.server_name:
-                server_name = get_domain_from_id(user_id)
-            elif get_domain_from_id(user_id) == self.server_name:
-                server_name = get_domain_from_id(group_id)
-            else:
-                raise Exception("Expected either group_id or user_id to be local")
-
-        if user_id != attestation["user_id"]:
-            raise SynapseError(400, "Attestation has incorrect user_id")
-
-        if group_id != attestation["group_id"]:
-            raise SynapseError(400, "Attestation has incorrect group_id")
-        valid_until_ms = attestation["valid_until_ms"]
-
-        # TODO: We also want to check that *new* attestations that people give
-        # us to store are valid for at least a little while.
-        now = self.clock.time_msec()
-        if valid_until_ms < now:
-            raise SynapseError(400, "Attestation expired")
-
-        assert server_name is not None
-        await self.keyring.verify_json_for_server(
-            server_name,
-            attestation,
-            now,
-        )
-
-    def create_attestation(self, group_id: str, user_id: str) -> JsonDict:
-        """Create an attestation for the group_id and user_id with default
-        validity length.
-        """
-        validity_period = DEFAULT_ATTESTATION_LENGTH_MS * random.uniform(
-            *DEFAULT_ATTESTATION_JITTER
-        )
-        valid_until_ms = int(self.clock.time_msec() + validity_period)
-
-        return sign_json(
-            {
-                "group_id": group_id,
-                "user_id": user_id,
-                "valid_until_ms": valid_until_ms,
-            },
-            self.server_name,
-            self.signing_key,
-        )
-
-
-class GroupAttestionRenewer:
-    """Responsible for sending and receiving attestation updates."""
-
-    def __init__(self, hs: "HomeServer"):
-        self.clock = hs.get_clock()
-        self.store = hs.get_datastores().main
-        self.assestations = hs.get_groups_attestation_signing()
-        self.transport_client = hs.get_federation_transport_client()
-        self.is_mine_id = hs.is_mine_id
-        self.attestations = hs.get_groups_attestation_signing()
-
-        if not hs.config.worker.worker_app:
-            self._renew_attestations_loop = self.clock.looping_call(
-                self._start_renew_attestations, 30 * 60 * 1000
-            )
-
-    async def on_renew_attestation(
-        self, group_id: str, user_id: str, content: JsonDict
-    ) -> JsonDict:
-        """When a remote updates an attestation"""
-        attestation = content["attestation"]
-
-        if not self.is_mine_id(group_id) and not self.is_mine_id(user_id):
-            raise SynapseError(400, "Neither user not group are on this server")
-
-        await self.attestations.verify_attestation(
-            attestation, user_id=user_id, group_id=group_id
-        )
-
-        await self.store.update_remote_attestion(group_id, user_id, attestation)
-
-        return {}
-
-    def _start_renew_attestations(self) -> "Deferred[None]":
-        return run_as_background_process("renew_attestations", self._renew_attestations)
-
-    async def _renew_attestations(self) -> None:
-        """Called periodically to check if we need to update any of our attestations"""
-
-        now = self.clock.time_msec()
-
-        rows = await self.store.get_attestations_need_renewals(
-            now + UPDATE_ATTESTATION_TIME_MS
-        )
-
-        async def _renew_attestation(group_user: Tuple[str, str]) -> None:
-            group_id, user_id = group_user
-            try:
-                if not self.is_mine_id(group_id):
-                    destination = get_domain_from_id(group_id)
-                elif not self.is_mine_id(user_id):
-                    destination = get_domain_from_id(user_id)
-                else:
-                    logger.warning(
-                        "Incorrectly trying to do attestations for user: %r in %r",
-                        user_id,
-                        group_id,
-                    )
-                    await self.store.remove_attestation_renewal(group_id, user_id)
-                    return
-
-                attestation = self.attestations.create_attestation(group_id, user_id)
-
-                await self.transport_client.renew_group_attestation(
-                    destination, group_id, user_id, content={"attestation": attestation}
-                )
-
-                await self.store.update_attestation_renewal(
-                    group_id, user_id, attestation
-                )
-            except (RequestSendFailed, HttpResponseException) as e:
-                logger.warning(
-                    "Failed to renew attestation of %r in %r: %s", user_id, group_id, e
-                )
-            except Exception:
-                logger.exception(
-                    "Error renewing attestation of %r in %r", user_id, group_id
-                )
-
-        for row in rows:
-            await _renew_attestation((row["group_id"], row["user_id"]))
@@ -31,7 +31,7 @@ class AdminHandler:
    def __init__(self, hs: "HomeServer"):
        self.store = hs.get_datastores().main
        self.storage = hs.get_storage()
-        self.state_store = self.storage.state
+        self.state_storage = self.storage.state

    async def get_whois(self, user: UserID) -> JsonDict:
        connections = []
@@ -233,7 +233,7 @@ class AdminHandler:
            for event_id in extremities:
                if not event_to_unseen_prevs[event_id]:
                    continue
-                state = await self.state_store.get_state_for_event(event_id)
+                state = await self.state_storage.get_state_for_event(event_id)
                writer.write_state(room_id, event_id, state)

        return writer.finished()
@@ -70,7 +70,7 @@ class DeviceWorkerHandler:
        self.store = hs.get_datastores().main
        self.notifier = hs.get_notifier()
        self.state = hs.get_state_handler()
-        self.state_store = hs.get_storage().state
+        self.state_storage = hs.get_storage().state
        self._auth_handler = hs.get_auth_handler()
        self.server_name = hs.hostname

@@ -203,7 +203,9 @@ class DeviceWorkerHandler:
                continue

            # mapping from event_id -> state_dict
-            prev_state_ids = await self.state_store.get_state_ids_for_events(event_ids)
+            prev_state_ids = await self.state_storage.get_state_ids_for_events(
+                event_ids
+            )

            # Check if we've joined the room? If so we just blindly add all the users to
            # the "possibly changed" users.
@@ -763,6 +765,10 @@ class DeviceListUpdater:
        device_id = edu_content.pop("device_id")
        stream_id = str(edu_content.pop("stream_id"))  # They may come as ints
        prev_ids = edu_content.pop("prev_id", [])
+        if not isinstance(prev_ids, list):
+            raise SynapseError(
+                400, "Device list update had an invalid 'prev_ids' field"
+            )
        prev_ids = [str(p) for p in prev_ids]  # They may come as ints

        if get_domain_from_id(user_id) != origin:
@@ -126,7 +126,7 @@ class FederationHandler:

        self.store = hs.get_datastores().main
        self.storage = hs.get_storage()
-        self.state_store = self.storage.state
+        self.state_storage = self.storage.state
        self.federation_client = hs.get_federation_client()
        self.state_handler = hs.get_state_handler()
        self.server_name = hs.hostname
@@ -1027,7 +1027,9 @@ class FederationHandler:
        if event.internal_metadata.outlier:
            raise NotFoundError("State not known at event %s" % (event_id,))

-        state_groups = await self.state_store.get_state_groups_ids(room_id, [event_id])
+        state_groups = await self.state_storage.get_state_groups_ids(
+            room_id, [event_id]
+        )

        # get_state_groups_ids should return exactly one result
        assert len(state_groups) == 1
@@ -99,7 +99,7 @@ class FederationEventHandler:
    def __init__(self, hs: "HomeServer"):
        self._store = hs.get_datastores().main
        self._storage = hs.get_storage()
-        self._state_store = self._storage.state
+        self._state_storage = self._storage.state

        self._state_handler = hs.get_state_handler()
        self._event_creation_handler = hs.get_event_creation_handler()
@@ -274,7 +274,7 @@ class FederationEventHandler:
                    affected=pdu.event_id,
                )

-        await self._process_received_pdu(origin, pdu, state=None)
+        await self._process_received_pdu(origin, pdu, state_ids=None)

    async def on_send_membership_event(
        self, origin: str, event: EventBase
@@ -463,7 +463,9 @@ class FederationEventHandler:
        with nested_logging_context(suffix=event.event_id):
            context = await self._state_handler.compute_event_context(
                event,
-                old_state=state,
+                state_ids_before_event={
+                    (e.type, e.state_key): e.event_id for e in state
+                },
                partial_state=partial_state,
            )

@@ -512,12 +514,12 @@ class FederationEventHandler:
            #
            # This is the same operation as we do when we receive a regular event
            # over federation.
-            state = await self._resolve_state_at_missing_prevs(destination, event)
+            state_ids = await self._resolve_state_at_missing_prevs(destination, event)

            # build a new state group for it if need be
            context = await self._state_handler.compute_event_context(
                event,
-                old_state=state,
+                state_ids_before_event=state_ids,
            )
            if context.partial_state:
                # this can happen if some or all of the event's prev_events still have
@@ -533,7 +535,7 @@ class FederationEventHandler:
                )
                return
            await self._store.update_state_for_partial_state_event(event, context)
-            self._state_store.notify_event_un_partial_stated(event.event_id)
+            self._state_storage.notify_event_un_partial_stated(event.event_id)

    async def backfill(
        self, dest: str, room_id: str, limit: int, extremities: Collection[str]
@@ -767,11 +769,12 @@ class FederationEventHandler:
            return

        try:
-            state = await self._resolve_state_at_missing_prevs(origin, event)
+            state_ids = await self._resolve_state_at_missing_prevs(origin, event)
            # TODO(faster_joins): make sure that _resolve_state_at_missing_prevs does
            #   not return partial state
+
            await self._process_received_pdu(
-                origin, event, state=state, backfilled=backfilled
+                origin, event, state_ids=state_ids, backfilled=backfilled
            )
        except FederationError as e:
            if e.code == 403:
@@ -781,7 +784,7 @@ class FederationEventHandler:

    async def _resolve_state_at_missing_prevs(
        self, dest: str, event: EventBase
-    ) -> Optional[Iterable[EventBase]]:
+    ) -> Optional[StateMap[str]]:
        """Calculate the state at an event with missing prev_events.

        This is used when we have pulled a batch of events from a remote server, and
@@ -808,8 +811,8 @@ class FederationEventHandler:
            event: an event to check for missing prevs.

        Returns:
-            if we already had all the prev events, `None`. Otherwise, returns a list of
-            the events in the state at `event`.
+            if we already had all the prev events, `None`. Otherwise, returns
+            the event ids of the state at `event`.
        """
        room_id = event.room_id
        event_id = event.event_id
@@ -829,10 +832,10 @@ class FederationEventHandler:
        )
        # Calculate the state after each of the previous events, and
        # resolve them to find the correct state at the current event.
-        event_map = {event_id: event}
+
        try:
            # Get the state of the events we know about
-            ours = await self._state_store.get_state_groups_ids(room_id, seen)
+            ours = await self._state_storage.get_state_groups_ids(room_id, seen)

            # state_maps is a list of mappings from (type, state_key) to event_id
            state_maps: List[StateMap[str]] = list(ours.values())
@@ -849,40 +852,23 @@ class FederationEventHandler:
                    # note that if any of the missing prevs share missing state or
                    # auth events, the requests to fetch those events are deduped
                    # by the get_pdu_cache in federation_client.
-                    remote_state = await self._get_state_after_missing_prev_event(
-                        dest, room_id, p
+                    remote_state_map = (
+                        await self._get_state_ids_after_missing_prev_event(
+                            dest, room_id, p
+                        )
                    )

-                    remote_state_map = {
-                        (x.type, x.state_key): x.event_id for x in remote_state
-                    }
                    state_maps.append(remote_state_map)

-                    for x in remote_state:
-                        event_map[x.event_id] = x
-
            room_version = await self._store.get_room_version_id(room_id)
            state_map = await self._state_resolution_handler.resolve_events_with_store(
                room_id,
                room_version,
                state_maps,
-                event_map,
+                event_map={event_id: event},
                state_res_store=StateResolutionStore(self._store),
            )

-            # We need to give _process_received_pdu the actual state events
-            # rather than event ids, so generate that now.
-
-            # First though we need to fetch all the events that are in
-            # state_map, so we can build up the state below.
-            evs = await self._store.get_events(
-                list(state_map.values()),
-                get_prev_content=False,
-                redact_behaviour=EventRedactBehaviour.as_is,
-            )
-            event_map.update(evs)
-
-            state = [event_map[e] for e in state_map.values()]
        except Exception:
            logger.warning(
                "Error attempting to resolve state at missing prev_events",
@@ -894,14 +880,14 @@ class FederationEventHandler:
                "We can't get valid state history.",
                affected=event_id,
            )
-        return state
+        return state_map

-    async def _get_state_after_missing_prev_event(
+    async def _get_state_ids_after_missing_prev_event(
        self,
        destination: str,
        room_id: str,
        event_id: str,
-    ) -> List[EventBase]:
+    ) -> StateMap[str]:
        """Requests all of the room state at a given event from a remote homeserver.

        Args:
@@ -910,7 +896,7 @@ class FederationEventHandler:
            event_id: The id of the event we want the state at.

        Returns:
-            A list of events in the state, including the event itself
+            The event ids of the state *after* the given event.
        """
        (
            state_event_ids,
@@ -925,19 +911,17 @@ class FederationEventHandler:
            len(auth_event_ids),
        )

-        # start by just trying to fetch the events from the store
+        # Start by checking events we already have in the DB
        desired_events = set(state_event_ids)
        desired_events.add(event_id)
        logger.debug("Fetching %i events from cache/store", len(desired_events))
-        fetched_events = await self._store.get_events(
-            desired_events, allow_rejected=True
-        )
+        have_events = await self._store.have_seen_events(room_id, desired_events)

-        missing_desired_events = desired_events - fetched_events.keys()
+        missing_desired_events = desired_events - have_events
        logger.debug(
            "We are missing %i events (got %i)",
            len(missing_desired_events),
-            len(fetched_events),
+            len(have_events),
        )

        # We probably won't need most of the auth events, so let's just check which
@@ -948,7 +932,7 @@ class FederationEventHandler:
        #   already have a bunch of the state events. It would be nice if the
        #   federation api gave us a way of finding out which we actually need.

-        missing_auth_events = set(auth_event_ids) - fetched_events.keys()
+        missing_auth_events = set(auth_event_ids) - have_events
        missing_auth_events.difference_update(
            await self._store.have_seen_events(room_id, missing_auth_events)
        )
@@ -974,47 +958,51 @@ class FederationEventHandler:
                destination=destination, room_id=room_id, event_ids=missing_events
            )

-        # we need to make sure we re-load from the database to get the rejected
-        # state correct.
-        fetched_events.update(
-            await self._store.get_events(missing_desired_events, allow_rejected=True)
-        )
+        # We now need to fill out the state map, which involves fetching the
+        # type and state key for each event ID in the state.
+        state_map = {}

-        # check for events which were in the wrong room.
-        #
-        # this can happen if a remote server claims that the state or
-        # auth_events at an event in room A are actually events in room B
+        event_metadata = await self._store.get_metadata_for_events(state_event_ids)
+        for state_event_id, metadata in event_metadata.items():
+            if metadata.room_id != room_id:
+                # This is a bogus situation, but since we may only discover it a long time
+                # after it happened, we try our best to carry on, by just omitting the
+                # bad events from the returned state set.
+                #
+                # This can happen if a remote server claims that the state or
+                # auth_events at an event in room A are actually events in room B
+                logger.warning(
+                    "Remote server %s claims event %s in room %s is an auth/state "
+                    "event in room %s",
+                    destination,
+                    state_event_id,
+                    metadata.room_id,
+                    room_id,
+                )
+                continue

-        bad_events = [
-            (event_id, event.room_id)
-            for event_id, event in fetched_events.items()
-            if event.room_id != room_id
-        ]
+            if metadata.state_key is None:
+                logger.warning(
+                    "Remote server gave us non-state event in state: %s", state_event_id
+                )
+                continue

-        for bad_event_id, bad_room_id in bad_events:
-            # This is a bogus situation, but since we may only discover it a long time
-            # after it happened, we try our best to carry on, by just omitting the
-            # bad events from the returned state set.
-            logger.warning(
-                "Remote server %s claims event %s in room %s is an auth/state "
-                "event in room %s",
-                destination,
-                bad_event_id,
-                bad_room_id,
-                room_id,
-            )
-
-            del fetched_events[bad_event_id]
+            state_map[(metadata.event_type, metadata.state_key)] = state_event_id

        # if we couldn't get the prev event in question, that's a problem.
-        remote_event = fetched_events.get(event_id)
+        remote_event = await self._store.get_event(
+            event_id,
+            allow_none=True,
+            allow_rejected=True,
+            redact_behaviour=EventRedactBehaviour.as_is,
+        )
        if not remote_event:
            raise Exception("Unable to get missing prev_event %s" % (event_id,))

        # missing state at that event is a warning, not a blocker
        # XXX: this doesn't sound right? it means that we'll end up with incomplete
        #   state.
-        failed_to_fetch = desired_events - fetched_events.keys()
+        failed_to_fetch = desired_events - event_metadata.keys()
        if failed_to_fetch:
            logger.warning(
                "Failed to fetch missing state events for %s %s",
@@ -1022,14 +1010,12 @@ class FederationEventHandler:
                failed_to_fetch,
            )

-        remote_state = [
-            fetched_events[e_id] for e_id in state_event_ids if e_id in fetched_events
-        ]
-
        if remote_event.is_state() and remote_event.rejected_reason is None:
-            remote_state.append(remote_event)
+            state_map[
+                (remote_event.type, remote_event.state_key)
+            ] = remote_event.event_id

-        return remote_state
+        return state_map

    async def _get_state_and_persist(
        self, destination: str, room_id: str, event_id: str
@@ -1056,7 +1042,7 @@ class FederationEventHandler:
        self,
        origin: str,
        event: EventBase,
-        state: Optional[Iterable[EventBase]],
+        state_ids: Optional[StateMap[str]],
        backfilled: bool = False,
    ) -> None:
        """Called when we have a new non-outlier event.
@@ -1078,7 +1064,7 @@ class FederationEventHandler:

            event: event to be persisted

-            state: Normally None, but if we are handling a gap in the graph
+            state_ids: Normally None, but if we are handling a gap in the graph
                (ie, we are missing one or more prev_events), the resolved state at the
                event

@@ -1090,7 +1076,8 @@ class FederationEventHandler:

        try:
            context = await self._state_handler.compute_event_context(
-                event, old_state=state
+                event,
+                state_ids_before_event=state_ids,
            )
            context = await self._check_event_auth(
                origin,
@@ -1107,7 +1094,7 @@ class FederationEventHandler:
            # For new (non-backfilled and non-outlier) events we check if the event
            # passes auth based on the current state. If it doesn't then we
            # "soft-fail" the event.
-            await self._check_for_soft_fail(event, state, origin=origin)
+            await self._check_for_soft_fail(event, state_ids, origin=origin)

        await self._run_push_actions_and_persist_event(event, context, backfilled)

@@ -1589,7 +1576,7 @@ class FederationEventHandler:
    async def _check_for_soft_fail(
        self,
        event: EventBase,
-        state: Optional[Iterable[EventBase]],
+        state_ids: Optional[StateMap[str]],
        origin: str,
    ) -> None:
        """Checks if we should soft fail the event; if so, marks the event as
@@ -1597,7 +1584,7 @@ class FederationEventHandler:

        Args:
            event
-            state: The state at the event if we don't have all the event's prev events
+            state_ids: The state at the event if we don't have all the event's prev events
            origin: The host the event originates from.
        """
        extrem_ids_list = await self._store.get_latest_event_ids_in_room(event.room_id)
@@ -1613,7 +1600,7 @@ class FederationEventHandler:
        room_version_obj = KNOWN_ROOM_VERSIONS[room_version]

        # Calculate the "current state".
-        if state is not None:
+        if state_ids is not None:
            # If we're explicitly given the state then we won't have all the
            # prev events, and so we have a gap in the graph. In this case
            # we want to be a little careful as we might have been down for
@@ -1626,17 +1613,20 @@ class FederationEventHandler:
            # given state at the event. This should correctly handle cases
            # like bans, especially with state res v2.

-            state_sets_d = await self._state_store.get_state_groups(
+            state_sets_d = await self._state_storage.get_state_groups_ids(
                event.room_id, extrem_ids
            )
-            state_sets: List[Iterable[EventBase]] = list(state_sets_d.values())
-            state_sets.append(state)
-            current_states = await self._state_handler.resolve_events(
-                room_version, state_sets, event
+            state_sets: List[StateMap[str]] = list(state_sets_d.values())
+            state_sets.append(state_ids)
+            current_state_ids = (
+                await self._state_resolution_handler.resolve_events_with_store(
+                    event.room_id,
+                    room_version,
+                    state_sets,
+                    event_map=None,
+                    state_res_store=StateResolutionStore(self._store),
+                )
            )
-            current_state_ids: StateMap[str] = {
-                k: e.event_id for k, e in current_states.items()
-            }
        else:
            current_state_ids = await self._state_handler.get_current_state_ids(
                event.room_id, latest_event_ids=extrem_ids
@@ -1895,7 +1885,7 @@ class FederationEventHandler:

        # create a new state group as a delta from the existing one.
        prev_group = context.state_group
-        state_group = await self._state_store.store_state_group(
+        state_group = await self._state_storage.store_state_group(
            event.event_id,
            event.room_id,
            prev_group=prev_group,
@@ -1,503 +0,0 @@
-# Copyright 2017 Vector Creations Ltd
-# Copyright 2018 New Vector Ltd
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import logging
-from typing import TYPE_CHECKING, Any, Awaitable, Callable, Dict, Iterable, List, Set
-
-from synapse.api.errors import HttpResponseException, RequestSendFailed, SynapseError
-from synapse.types import GroupID, JsonDict, get_domain_from_id
-
-if TYPE_CHECKING:
-    from synapse.server import HomeServer
-
-logger = logging.getLogger(__name__)
-
-
-def _create_rerouter(func_name: str) -> Callable[..., Awaitable[JsonDict]]:
-    """Returns an async function that looks at the group id and calls the function
-    on federation or the local group server if the group is local
-    """
-
-    async def f(
-        self: "GroupsLocalWorkerHandler", group_id: str, *args: Any, **kwargs: Any
-    ) -> JsonDict:
-        if not GroupID.is_valid(group_id):
-            raise SynapseError(400, "%s is not a legal group ID" % (group_id,))
-
-        if self.is_mine_id(group_id):
-            return await getattr(self.groups_server_handler, func_name)(
-                group_id, *args, **kwargs
-            )
-        else:
-            destination = get_domain_from_id(group_id)
-
-            try:
-                return await getattr(self.transport_client, func_name)(
-                    destination, group_id, *args, **kwargs
-                )
-            except HttpResponseException as e:
-                # Capture errors returned by the remote homeserver and
-                # re-throw specific errors as SynapseErrors. This is so
-                # when the remote end responds with things like 403 Not
-                # In Group, we can communicate that to the client instead
-                # of a 500.
-                raise e.to_synapse_error()
-            except RequestSendFailed:
-                raise SynapseError(502, "Failed to contact group server")
-
-    return f
-
-
-class GroupsLocalWorkerHandler:
-    def __init__(self, hs: "HomeServer"):
-        self.hs = hs
-        self.store = hs.get_datastores().main
-        self.room_list_handler = hs.get_room_list_handler()
-        self.groups_server_handler = hs.get_groups_server_handler()
-        self.transport_client = hs.get_federation_transport_client()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.keyring = hs.get_keyring()
-        self.is_mine_id = hs.is_mine_id
-        self.signing_key = hs.signing_key
-        self.server_name = hs.hostname
-        self.notifier = hs.get_notifier()
-        self.attestations = hs.get_groups_attestation_signing()
-
-        self.profile_handler = hs.get_profile_handler()
-
-    # The following functions merely route the query to the local groups server
-    # or federation depending on if the group is local or remote
-
-    get_group_profile = _create_rerouter("get_group_profile")
-    get_rooms_in_group = _create_rerouter("get_rooms_in_group")
-    get_invited_users_in_group = _create_rerouter("get_invited_users_in_group")
-    get_group_category = _create_rerouter("get_group_category")
-    get_group_categories = _create_rerouter("get_group_categories")
-    get_group_role = _create_rerouter("get_group_role")
-    get_group_roles = _create_rerouter("get_group_roles")
-
-    async def get_group_summary(
-        self, group_id: str, requester_user_id: str
-    ) -> JsonDict:
-        """Get the group summary for a group.
-
-        If the group is remote we check that the users have valid attestations.
-        """
-        if self.is_mine_id(group_id):
-            res = await self.groups_server_handler.get_group_summary(
-                group_id, requester_user_id
-            )
-        else:
-            try:
-                res = await self.transport_client.get_group_summary(
-                    get_domain_from_id(group_id), group_id, requester_user_id
-                )
-            except HttpResponseException as e:
-                raise e.to_synapse_error()
-            except RequestSendFailed:
-                raise SynapseError(502, "Failed to contact group server")
-
-            group_server_name = get_domain_from_id(group_id)
-
-            # Loop through the users and validate the attestations.
-            chunk = res["users_section"]["users"]
-            valid_users = []
-            for entry in chunk:
-                g_user_id = entry["user_id"]
-                attestation = entry.pop("attestation", {})
-                try:
-                    if get_domain_from_id(g_user_id) != group_server_name:
-                        await self.attestations.verify_attestation(
-                            attestation,
-                            group_id=group_id,
-                            user_id=g_user_id,
-                            server_name=get_domain_from_id(g_user_id),
-                        )
-                    valid_users.append(entry)
-                except Exception as e:
-                    logger.info("Failed to verify user is in group: %s", e)
-
-            res["users_section"]["users"] = valid_users
-
-            res["users_section"]["users"].sort(key=lambda e: e.get("order", 0))
-            res["rooms_section"]["rooms"].sort(key=lambda e: e.get("order", 0))
-
-        # Add `is_publicised` flag to indicate whether the user has publicised their
-        # membership of the group on their profile
-        result = await self.store.get_publicised_groups_for_user(requester_user_id)
-        is_publicised = group_id in result
-
-        res.setdefault("user", {})["is_publicised"] = is_publicised
-
-        return res
-
-    async def get_users_in_group(
-        self, group_id: str, requester_user_id: str
-    ) -> JsonDict:
-        """Get users in a group"""
-        if self.is_mine_id(group_id):
-            return await self.groups_server_handler.get_users_in_group(
-                group_id, requester_user_id
-            )
-
-        group_server_name = get_domain_from_id(group_id)
-
-        try:
-            res = await self.transport_client.get_users_in_group(
-                get_domain_from_id(group_id), group_id, requester_user_id
-            )
-        except HttpResponseException as e:
-            raise e.to_synapse_error()
-        except RequestSendFailed:
-            raise SynapseError(502, "Failed to contact group server")
-
-        chunk = res["chunk"]
-        valid_entries = []
-        for entry in chunk:
-            g_user_id = entry["user_id"]
-            attestation = entry.pop("attestation", {})
-            try:
-                if get_domain_from_id(g_user_id) != group_server_name:
-                    await self.attestations.verify_attestation(
-                        attestation,
-                        group_id=group_id,
-                        user_id=g_user_id,
-                        server_name=get_domain_from_id(g_user_id),
-                    )
-                valid_entries.append(entry)
-            except Exception as e:
-                logger.info("Failed to verify user is in group: %s", e)
-
-        res["chunk"] = valid_entries
-
-        return res
-
-    async def get_joined_groups(self, user_id: str) -> JsonDict:
-        group_ids = await self.store.get_joined_groups(user_id)
-        return {"groups": group_ids}
-
-    async def get_publicised_groups_for_user(self, user_id: str) -> JsonDict:
-        if self.hs.is_mine_id(user_id):
-            result = await self.store.get_publicised_groups_for_user(user_id)
-
-            # Check AS associated groups for this user - this depends on the
-            # RegExps in the AS registration file (under `users`)
-            for app_service in self.store.get_app_services():
-                result.extend(app_service.get_groups_for_user(user_id))
-
-            return {"groups": result}
-        else:
-            try:
-                bulk_result = await self.transport_client.bulk_get_publicised_groups(
-                    get_domain_from_id(user_id), [user_id]
-                )
-            except HttpResponseException as e:
-                raise e.to_synapse_error()
-            except RequestSendFailed:
-                raise SynapseError(502, "Failed to contact group server")
-
-            result = bulk_result.get("users", {}).get(user_id)
-            # TODO: Verify attestations
-            return {"groups": result}
-
-    async def bulk_get_publicised_groups(
-        self, user_ids: Iterable[str], proxy: bool = True
-    ) -> JsonDict:
-        destinations: Dict[str, Set[str]] = {}
-        local_users = set()
-
-        for user_id in user_ids:
-            if self.hs.is_mine_id(user_id):
-                local_users.add(user_id)
-            else:
-                destinations.setdefault(get_domain_from_id(user_id), set()).add(user_id)
-
-        if not proxy and destinations:
-            raise SynapseError(400, "Some user_ids are not local")
-
-        results = {}
-        failed_results: List[str] = []
-        for destination, dest_user_ids in destinations.items():
-            try:
-                r = await self.transport_client.bulk_get_publicised_groups(
-                    destination, list(dest_user_ids)
-                )
-                results.update(r["users"])
-            except Exception:
-                failed_results.extend(dest_user_ids)
-
-        for uid in local_users:
-            results[uid] = await self.store.get_publicised_groups_for_user(uid)
-
-            # Check AS associated groups for this user - this depends on the
-            # RegExps in the AS registration file (under `users`)
-            for app_service in self.store.get_app_services():
-                results[uid].extend(app_service.get_groups_for_user(uid))
-
-        return {"users": results}
-
-
-class GroupsLocalHandler(GroupsLocalWorkerHandler):
-    def __init__(self, hs: "HomeServer"):
-        super().__init__(hs)
-
-        # Ensure attestations get renewed
-        hs.get_groups_attestation_renewer()
-
-    # The following functions merely route the query to the local groups server
-    # or federation depending on if the group is local or remote
-
-    update_group_profile = _create_rerouter("update_group_profile")
-
-    add_room_to_group = _create_rerouter("add_room_to_group")
-    update_room_in_group = _create_rerouter("update_room_in_group")
-    remove_room_from_group = _create_rerouter("remove_room_from_group")
-
-    update_group_summary_room = _create_rerouter("update_group_summary_room")
-    delete_group_summary_room = _create_rerouter("delete_group_summary_room")
-
-    update_group_category = _create_rerouter("update_group_category")
-    delete_group_category = _create_rerouter("delete_group_category")
-
-    update_group_summary_user = _create_rerouter("update_group_summary_user")
-    delete_group_summary_user = _create_rerouter("delete_group_summary_user")
-
-    update_group_role = _create_rerouter("update_group_role")
-    delete_group_role = _create_rerouter("delete_group_role")
-
-    set_group_join_policy = _create_rerouter("set_group_join_policy")
-
-    async def create_group(
-        self, group_id: str, user_id: str, content: JsonDict
-    ) -> JsonDict:
-        """Create a group"""
-
-        logger.info("Asking to create group with ID: %r", group_id)
-
-        if self.is_mine_id(group_id):
-            res = await self.groups_server_handler.create_group(
-                group_id, user_id, content
-            )
-            local_attestation = None
-            remote_attestation = None
-        else:
-            raise SynapseError(400, "Unable to create remote groups")
-
-        is_publicised = content.get("publicise", False)
-        token = await self.store.register_user_group_membership(
-            group_id,
-            user_id,
-            membership="join",
-            is_admin=True,
-            local_attestation=local_attestation,
-            remote_attestation=remote_attestation,
-            is_publicised=is_publicised,
-        )
-        self.notifier.on_new_event("groups_key", token, users=[user_id])
-
-        return res
-
-    async def join_group(
-        self, group_id: str, user_id: str, content: JsonDict
-    ) -> JsonDict:
-        """Request to join a group"""
-        if self.is_mine_id(group_id):
-            await self.groups_server_handler.join_group(group_id, user_id, content)
-            local_attestation = None
-            remote_attestation = None
-        else:
-            local_attestation = self.attestations.create_attestation(group_id, user_id)
-            content["attestation"] = local_attestation
-
-            try:
-                res = await self.transport_client.join_group(
-                    get_domain_from_id(group_id), group_id, user_id, content
-                )
-            except HttpResponseException as e:
-                raise e.to_synapse_error()
-            except RequestSendFailed:
-                raise SynapseError(502, "Failed to contact group server")
-
-            remote_attestation = res["attestation"]
-
-            await self.attestations.verify_attestation(
-                remote_attestation,
-                group_id=group_id,
-                user_id=user_id,
-                server_name=get_domain_from_id(group_id),
-            )
-
-        # TODO: Check that the group is public and we're being added publicly
-        is_publicised = content.get("publicise", False)
-
-        token = await self.store.register_user_group_membership(
-            group_id,
-            user_id,
-            membership="join",
-            is_admin=False,
-            local_attestation=local_attestation,
-            remote_attestation=remote_attestation,
-            is_publicised=is_publicised,
-        )
-        self.notifier.on_new_event("groups_key", token, users=[user_id])
-
-        return {}
-
-    async def accept_invite(
-        self, group_id: str, user_id: str, content: JsonDict
-    ) -> JsonDict:
-        """Accept an invite to a group"""
-        if self.is_mine_id(group_id):
-            await self.groups_server_handler.accept_invite(group_id, user_id, content)
-            local_attestation = None
-            remote_attestation = None
-        else:
-            local_attestation = self.attestations.create_attestation(group_id, user_id)
-            content["attestation"] = local_attestation
-
-            try:
-                res = await self.transport_client.accept_group_invite(
-                    get_domain_from_id(group_id), group_id, user_id, content
-                )
-            except HttpResponseException as e:
-                raise e.to_synapse_error()
-            except RequestSendFailed:
-                raise SynapseError(502, "Failed to contact group server")
-
-            remote_attestation = res["attestation"]
-
-            await self.attestations.verify_attestation(
-                remote_attestation,
-                group_id=group_id,
-                user_id=user_id,
-                server_name=get_domain_from_id(group_id),
-            )
-
-        # TODO: Check that the group is public and we're being added publicly
-        is_publicised = content.get("publicise", False)
-
-        token = await self.store.register_user_group_membership(
-            group_id,
-            user_id,
-            membership="join",
-            is_admin=False,
-            local_attestation=local_attestation,
-            remote_attestation=remote_attestation,
-            is_publicised=is_publicised,
-        )
-        self.notifier.on_new_event("groups_key", token, users=[user_id])
-
-        return {}
-
-    async def invite(
-        self, group_id: str, user_id: str, requester_user_id: str, config: JsonDict
-    ) -> JsonDict:
-        """Invite a user to a group"""
-        content = {"requester_user_id": requester_user_id, "config": config}
-        if self.is_mine_id(group_id):
-            res = await self.groups_server_handler.invite_to_group(
-                group_id, user_id, requester_user_id, content
-            )
-        else:
-            try:
-                res = await self.transport_client.invite_to_group(
-                    get_domain_from_id(group_id),
-                    group_id,
-                    user_id,
-                    requester_user_id,
-                    content,
-                )
-            except HttpResponseException as e:
-                raise e.to_synapse_error()
-            except RequestSendFailed:
-                raise SynapseError(502, "Failed to contact group server")
-
-        return res
-
-    async def on_invite(
-        self, group_id: str, user_id: str, content: JsonDict
-    ) -> JsonDict:
-        """One of our users were invited to a group"""
-        # TODO: Support auto join and rejection
-
-        if not self.is_mine_id(user_id):
-            raise SynapseError(400, "User not on this server")
-
-        local_profile = {}
-        if "profile" in content:
-            if "name" in content["profile"]:
-                local_profile["name"] = content["profile"]["name"]
-            if "avatar_url" in content["profile"]:
-                local_profile["avatar_url"] = content["profile"]["avatar_url"]
-
-        token = await self.store.register_user_group_membership(
-            group_id,
-            user_id,
-            membership="invite",
-            content={"profile": local_profile, "inviter": content["inviter"]},
-        )
-        self.notifier.on_new_event("groups_key", token, users=[user_id])
-        try:
-            user_profile = await self.profile_handler.get_profile(user_id)
-        except Exception as e:
-            logger.warning("No profile for user %s: %s", user_id, e)
-            user_profile = {}
-
-        return {"state": "invite", "user_profile": user_profile}
-
-    async def remove_user_from_group(
-        self, group_id: str, user_id: str, requester_user_id: str, content: JsonDict
-    ) -> JsonDict:
-        """Remove a user from a group"""
-        if user_id == requester_user_id:
-            token = await self.store.register_user_group_membership(
-                group_id, user_id, membership="leave"
-            )
-            self.notifier.on_new_event("groups_key", token, users=[user_id])
-
-            # TODO: Should probably remember that we tried to leave so that we can
-            # retry if the group server is currently down.
-
-        if self.is_mine_id(group_id):
-            res = await self.groups_server_handler.remove_user_from_group(
-                group_id, user_id, requester_user_id, content
-            )
-        else:
-            content["requester_user_id"] = requester_user_id
-            try:
-                res = await self.transport_client.remove_user_from_group(
-                    get_domain_from_id(group_id),
-                    group_id,
-                    requester_user_id,
-                    user_id,
-                    content,
-                )
-            except HttpResponseException as e:
-                raise e.to_synapse_error()
-            except RequestSendFailed:
-                raise SynapseError(502, "Failed to contact group server")
-
-        return res
-
-    async def user_removed_from_group(
-        self, group_id: str, user_id: str, content: JsonDict
-    ) -> None:
-        """One of our users was removed/kicked from a group"""
-        # TODO: Check if user in group
-        token = await self.store.register_user_group_membership(
-            group_id, user_id, membership="leave"
-        )
-        self.notifier.on_new_event("groups_key", token, users=[user_id])
@@ -68,7 +68,7 @@ class InitialSyncHandler:
        ] = ResponseCache(hs.get_clock(), "initial_sync_cache")
        self._event_serializer = hs.get_event_client_serializer()
        self.storage = hs.get_storage()
-        self.state_store = self.storage.state
+        self.state_storage = self.storage.state

    async def snapshot_all_rooms(
        self,
@@ -198,7 +198,7 @@ class InitialSyncHandler:
                        event.stream_ordering,
                    )
                    deferred_room_state = run_in_background(
-                        self.state_store.get_state_for_events, [event.event_id]
+                        self.state_storage.get_state_for_events, [event.event_id]
                    ).addCallback(
                        lambda states: cast(StateMap[EventBase], states[event.event_id])
                    )
@@ -355,7 +355,7 @@ class InitialSyncHandler:
        member_event_id: str,
        is_peeking: bool,
    ) -> JsonDict:
-        room_state = await self.state_store.get_state_for_event(member_event_id)
+        room_state = await self.state_storage.get_state_for_event(member_event_id)

        limit = pagin_config.limit if pagin_config else None
        if limit is None:
@@ -55,7 +55,14 @@ from synapse.metrics.background_process_metrics import run_as_background_process
 from synapse.replication.http.send_event import ReplicationSendEventRestServlet
 from synapse.storage.databases.main.events_worker import EventRedactBehaviour
 from synapse.storage.state import StateFilter
-from synapse.types import Requester, RoomAlias, StreamToken, UserID, create_requester
+from synapse.types import (
+    MutableStateMap,
+    Requester,
+    RoomAlias,
+    StreamToken,
+    UserID,
+    create_requester,
+)
 from synapse.util import json_decoder, json_encoder, log_failure, unwrapFirstError
 from synapse.util.async_helpers import Linearizer, gather_results
 from synapse.util.caches.expiringcache import ExpiringCache
@@ -78,7 +85,7 @@ class MessageHandler:
        self.state = hs.get_state_handler()
        self.store = hs.get_datastores().main
        self.storage = hs.get_storage()
-        self.state_store = self.storage.state
+        self.state_storage = self.storage.state
        self._event_serializer = hs.get_event_client_serializer()
        self._ephemeral_events_enabled = hs.config.server.enable_ephemeral_messages

@@ -125,7 +132,7 @@ class MessageHandler:
            assert (
                membership_event_id is not None
            ), "check_user_in_room_or_world_readable returned invalid data"
-            room_state = await self.state_store.get_state_for_events(
+            room_state = await self.state_storage.get_state_for_events(
                [membership_event_id], StateFilter.from_types([key])
            )
            data = room_state[membership_event_id].get(key)
@@ -186,7 +193,7 @@ class MessageHandler:

            # check whether the user is in the room at that time to determine
            # whether they should be treated as peeking.
-            state_map = await self.state_store.get_state_for_event(
+            state_map = await self.state_storage.get_state_for_event(
                last_event.event_id,
                StateFilter.from_types([(EventTypes.Member, user_id)]),
            )
@@ -207,7 +214,7 @@ class MessageHandler:
            )

            if visible_events:
-                room_state_events = await self.state_store.get_state_for_events(
+                room_state_events = await self.state_storage.get_state_for_events(
                    [last_event.event_id], state_filter=state_filter
                )
                room_state: Mapping[Any, EventBase] = room_state_events[
@@ -237,7 +244,7 @@ class MessageHandler:
                assert (
                    membership_event_id is not None
                ), "check_user_in_room_or_world_readable returned invalid data"
-                room_state_events = await self.state_store.get_state_for_events(
+                room_state_events = await self.state_storage.get_state_for_events(
                    [membership_event_id], state_filter=state_filter
                )
                room_state = room_state_events[membership_event_id]
@@ -1022,8 +1029,35 @@ class EventCreationHandler:
            #
            # TODO(faster_joins): figure out how this works, and make sure that the
            #   old state is complete.
-            old_state = await self.store.get_events_as_list(state_event_ids)
-            context = await self.state.compute_event_context(event, old_state=old_state)
+            metadata = await self.store.get_metadata_for_events(state_event_ids)
+
+            state_map_for_event: MutableStateMap[str] = {}
+            for state_id in state_event_ids:
+                data = metadata.get(state_id)
+                if data is None:
+                    # We're trying to persist a new historical batch of events
+                    # with the given state, e.g. via
+                    # `RoomBatchSendEventRestServlet`. The state can be inferred
+                    # by Synapse or set directly by the client.
+                    #
+                    # Either way, we should have persisted all the state before
+                    # getting here.
+                    raise Exception(
+                        f"State event {state_id} not found in DB,"
+                        " Synapse should have persisted it before using it."
+                    )
+
+                if data.state_key is None:
+                    raise Exception(
+                        f"Trying to set non-state event {state_id} as state"
+                    )
+
+                state_map_for_event[(data.event_type, data.state_key)] = state_id
+
+            context = await self.state.compute_event_context(
+                event,
+                state_ids_before_event=state_map_for_event,
+            )
        else:
            context = await self.state.compute_event_context(event)

@@ -130,7 +130,7 @@ class PaginationHandler:
        self.auth = hs.get_auth()
        self.store = hs.get_datastores().main
        self.storage = hs.get_storage()
-        self.state_store = self.storage.state
+        self.state_storage = self.storage.state
        self.clock = hs.get_clock()
        self._server_name = hs.hostname
        self._room_shutdown_handler = hs.get_room_shutdown_handler()
@@ -539,7 +539,7 @@ class PaginationHandler:
                (EventTypes.Member, event.sender) for event in events
            )

-            state_ids = await self.state_store.get_state_ids_for_event(
+            state_ids = await self.state_storage.get_state_ids_for_event(
                events[0].event_id, state_filter=state_filter
            )

@@ -1193,7 +1193,7 @@ class RoomContextHandler:
        self.auth = hs.get_auth()
        self.store = hs.get_datastores().main
        self.storage = hs.get_storage()
-        self.state_store = self.storage.state
+        self.state_storage = self.storage.state
        self._relations_handler = hs.get_relations_handler()

    async def get_event_context(
@@ -1293,7 +1293,7 @@ class RoomContextHandler:
        # first? Shouldn't we be consistent with /sync?
        # https://github.com/matrix-org/matrix-doc/issues/687

-        state = await self.state_store.get_state_for_events(
+        state = await self.state_storage.get_state_for_events(
            [last_event_id], state_filter=state_filter
        )

@@ -17,7 +17,7 @@ class RoomBatchHandler:
    def __init__(self, hs: "HomeServer"):
        self.hs = hs
        self.store = hs.get_datastores().main
-        self.state_store = hs.get_storage().state
+        self.state_storage = hs.get_storage().state
        self.event_creation_handler = hs.get_event_creation_handler()
        self.room_member_handler = hs.get_room_member_handler()
        self.auth = hs.get_auth()
@@ -141,7 +141,7 @@ class RoomBatchHandler:
        ) = await self.store.get_max_depth_of(event_ids)
        # mapping from (type, state_key) -> state_event_id
        assert most_recent_event_id is not None
-        prev_state_map = await self.state_store.get_state_ids_for_event(
+        prev_state_map = await self.state_storage.get_state_ids_for_event(
            most_recent_event_id
        )
        # List of state event ID's
@@ -1081,17 +1081,6 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
        # Transfer alias mappings in the room directory
        await self.store.update_aliases_for_room(old_room_id, room_id)

-        # Check if any groups we own contain the predecessor room
-        local_group_ids = await self.store.get_local_groups_for_room(old_room_id)
-        for group_id in local_group_ids:
-            # Add new the new room to those groups
-            await self.store.add_room_to_group(
-                group_id, room_id, old_room is not None and old_room["is_public"]
-            )
-
-            # Remove the old room from those groups
-            await self.store.remove_room_from_group(group_id, old_room_id)
-
    async def copy_user_state_on_room_upgrade(
        self, old_room_id: str, new_room_id: str, user_ids: Iterable[str]
    ) -> None:
@@ -662,7 +662,8 @@ class RoomSummaryHandler:
        # The API doesn't return the room version so assume that a
        # join rule of knock is valid.
        if (
-            room.get("join_rules") in (JoinRules.PUBLIC, JoinRules.KNOCK)
+            room.get("join_rule")
+            in (JoinRules.PUBLIC, JoinRules.KNOCK, JoinRules.KNOCK_RESTRICTED)
            or room.get("world_readable") is True
        ):
            return True
@@ -713,9 +714,6 @@ class RoomSummaryHandler:
            "canonical_alias": stats["canonical_alias"],
            "num_joined_members": stats["joined_members"],
            "avatar_url": stats["avatar"],
-            # plural join_rules is a documentation error but kept for historical
-            # purposes. Should match /publicRooms.
-            "join_rules": stats["join_rules"],
            "join_rule": stats["join_rules"],
            "world_readable": (
                stats["history_visibility"] == HistoryVisibility.WORLD_READABLE
@@ -56,7 +56,7 @@ class SearchHandler:
        self._event_serializer = hs.get_event_client_serializer()
        self._relations_handler = hs.get_relations_handler()
        self.storage = hs.get_storage()
-        self.state_store = self.storage.state
+        self.state_storage = self.storage.state
        self.auth = hs.get_auth()

    async def get_old_rooms_from_upgraded_room(self, room_id: str) -> Iterable[str]:
@@ -677,7 +677,7 @@ class SearchHandler:
                    [(EventTypes.Member, sender) for sender in senders]
                )

-                state = await self.state_store.get_state_for_event(
+                state = await self.state_storage.get_state_for_event(
                    last_event_id, state_filter
                )

@@ -166,16 +166,6 @@ class KnockedSyncResult:
        return True


-@attr.s(slots=True, frozen=True, auto_attribs=True)
-class GroupsSyncResult:
-    join: JsonDict
-    invite: JsonDict
-    leave: JsonDict
-
-    def __bool__(self) -> bool:
-        return bool(self.join or self.invite or self.leave)
-
-
@attr.s(slots=True, auto_attribs=True)
 class _RoomChanges:
    """The set of room entries to include in the sync, plus the set of joined
@@ -206,7 +196,6 @@ class SyncResult:
            for this device
        device_unused_fallback_key_types: List of key types that have an unused fallback
            key
-        groups: Group updates, if any
    """

    next_batch: StreamToken
@@ -220,7 +209,6 @@ class SyncResult:
    device_lists: DeviceListUpdates
    device_one_time_keys_count: JsonDict
    device_unused_fallback_key_types: List[str]
-    groups: Optional[GroupsSyncResult]

    def __bool__(self) -> bool:
        """Make the result appear empty if there are no updates. This is used
@@ -236,7 +224,6 @@ class SyncResult:
            or self.account_data
            or self.to_device
            or self.device_lists
-            or self.groups
        )


@@ -252,7 +239,7 @@ class SyncHandler:
        self.state = hs.get_state_handler()
        self.auth = hs.get_auth()
        self.storage = hs.get_storage()
-        self.state_store = self.storage.state
+        self.state_storage = self.storage.state

        # TODO: flush cache entries on subsequent sync request.
        #    Once we get the next /sync request (ie, one with the same access token
@@ -643,7 +630,7 @@ class SyncHandler:
            event: event of interest
            state_filter: The state filter used to fetch state from the database.
        """
-        state_ids = await self.state_store.get_state_ids_for_event(
+        state_ids = await self.state_storage.get_state_ids_for_event(
            event.event_id, state_filter=state_filter or StateFilter.all()
        )
        if event.is_state():
@@ -723,7 +710,7 @@ class SyncHandler:
            return None

        last_event = last_events[-1]
-        state_ids = await self.state_store.get_state_ids_for_event(
+        state_ids = await self.state_storage.get_state_ids_for_event(
            last_event.event_id,
            state_filter=StateFilter.from_types(
                [(EventTypes.Name, ""), (EventTypes.CanonicalAlias, "")]
@@ -901,11 +888,13 @@ class SyncHandler:

            if full_state:
                if batch:
-                    current_state_ids = await self.state_store.get_state_ids_for_event(
-                        batch.events[-1].event_id, state_filter=state_filter
+                    current_state_ids = (
+                        await self.state_storage.get_state_ids_for_event(
+                            batch.events[-1].event_id, state_filter=state_filter
+                        )
                    )

-                    state_ids = await self.state_store.get_state_ids_for_event(
+                    state_ids = await self.state_storage.get_state_ids_for_event(
                        batch.events[0].event_id, state_filter=state_filter
                    )

@@ -926,7 +915,7 @@ class SyncHandler:
            elif batch.limited:
                if batch:
                    state_at_timeline_start = (
-                        await self.state_store.get_state_ids_for_event(
+                        await self.state_storage.get_state_ids_for_event(
                            batch.events[0].event_id, state_filter=state_filter
                        )
                    )
@@ -960,8 +949,10 @@ class SyncHandler:
                )

                if batch:
-                    current_state_ids = await self.state_store.get_state_ids_for_event(
-                        batch.events[-1].event_id, state_filter=state_filter
+                    current_state_ids = (
+                        await self.state_storage.get_state_ids_for_event(
+                            batch.events[-1].event_id, state_filter=state_filter
+                        )
                    )
                else:
                    # Its not clear how we get here, but empirically we do
@@ -991,7 +982,7 @@ class SyncHandler:
                        # So we fish out all the member events corresponding to the
                        # timeline here, and then dedupe any redundant ones below.

-                        state_ids = await self.state_store.get_state_ids_for_event(
+                        state_ids = await self.state_storage.get_state_ids_for_event(
                            batch.events[0].event_id,
                            # we only want members!
                            state_filter=StateFilter.from_types(
@@ -1157,10 +1148,6 @@ class SyncHandler:
                await self.store.get_e2e_unused_fallback_key_types(user_id, device_id)
            )

-        if self.hs_config.experimental.groups_enabled:
-            logger.debug("Fetching group data")
-            await self._generate_sync_entry_for_groups(sync_result_builder)
-
        num_events = 0

        # debug for https://github.com/matrix-org/synapse/issues/9424
@@ -1184,57 +1171,11 @@ class SyncHandler:
            archived=sync_result_builder.archived,
            to_device=sync_result_builder.to_device,
            device_lists=device_lists,
-            groups=sync_result_builder.groups,
            device_one_time_keys_count=one_time_key_counts,
            device_unused_fallback_key_types=unused_fallback_key_types,
            next_batch=sync_result_builder.now_token,
        )

-    @measure_func("_generate_sync_entry_for_groups")
-    async def _generate_sync_entry_for_groups(
-        self, sync_result_builder: "SyncResultBuilder"
-    ) -> None:
-        user_id = sync_result_builder.sync_config.user.to_string()
-        since_token = sync_result_builder.since_token
-        now_token = sync_result_builder.now_token
-
-        if since_token and since_token.groups_key:
-            results = await self.store.get_groups_changes_for_user(
-                user_id, since_token.groups_key, now_token.groups_key
-            )
-        else:
-            results = await self.store.get_all_groups_for_user(
-                user_id, now_token.groups_key
-            )
-
-        invited = {}
-        joined = {}
-        left = {}
-        for result in results:
-            membership = result["membership"]
-            group_id = result["group_id"]
-            gtype = result["type"]
-            content = result["content"]
-
-            if membership == "join":
-                if gtype == "membership":
-                    # TODO: Add profile
-                    content.pop("membership", None)
-                    joined[group_id] = content["content"]
-                else:
-                    joined.setdefault(group_id, {})[gtype] = content
-            elif membership == "invite":
-                if gtype == "membership":
-                    content.pop("membership", None)
-                    invited[group_id] = content["content"]
-            else:
-                if gtype == "membership":
-                    left[group_id] = content["content"]
-
-        sync_result_builder.groups = GroupsSyncResult(
-            join=joined, invite=invited, leave=left
-        )
-
    @measure_func("_generate_sync_entry_for_device_list")
    async def _generate_sync_entry_for_device_list(
        self,
@@ -2333,7 +2274,6 @@ class SyncResultBuilder:
        invited
        knocked
        archived
-        groups
        to_device
    """

@@ -2349,7 +2289,6 @@ class SyncResultBuilder:
    invited: List[InvitedSyncResult] = attr.Factory(list)
    knocked: List[KnockedSyncResult] = attr.Factory(list)
    archived: List[ArchivedSyncResult] = attr.Factory(list)
-    groups: Optional[GroupsSyncResult] = None
    to_device: List[JsonDict] = attr.Factory(list)

    def calculate_user_changes(self) -> Tuple[Set[str], Set[str]]:
@@ -92,9 +92,6 @@ incoming_responses_counter = Counter(
    "synapse_http_matrixfederationclient_responses", "", ["method", "code"]
 )

-# a federation response can be rather large (eg a big state_ids is 50M or so), so we
-# need a generous limit here.
-MAX_RESPONSE_SIZE = 100 * 1024 * 1024

 MAX_LONG_RETRIES = 10
 MAX_SHORT_RETRIES = 3
@@ -116,6 +113,11 @@ class ByteParser(ByteWriteable, Generic[T], abc.ABC):
    the content type doesn't match we fail the request.
    """

+    # a federation response can be rather large (eg a big state_ids is 50M or so), so we
+    # need a generous limit here.
+    MAX_RESPONSE_SIZE: int = 100 * 1024 * 1024
+    """The largest response this parser will accept."""
+
    @abc.abstractmethod
    def finish(self) -> T:
        """Called when response has finished streaming and the parser should
@@ -203,7 +205,6 @@ async def _handle_response(
    response: IResponse,
    start_ms: int,
    parser: ByteParser[T],
-    max_response_size: Optional[int] = None,
 ) -> T:
    """
    Reads the body of a response with a timeout and sends it to a parser
@@ -215,15 +216,12 @@ async def _handle_response(
        response: response to the request
        start_ms: Timestamp when request was made
        parser: The parser for the response
-        max_response_size: The maximum size to read from the response, if None
-            uses the default.

    Returns:
        The parsed response
    """

-    if max_response_size is None:
-        max_response_size = MAX_RESPONSE_SIZE
+    max_response_size = parser.MAX_RESPONSE_SIZE

    try:
        check_content_type_is(response.headers, parser.CONTENT_TYPE)
@@ -240,7 +238,7 @@ async def _handle_response(
            "{%s} [%s] JSON response exceeded max size %i - %s %s",
            request.txn_id,
            request.destination,
-            MAX_RESPONSE_SIZE,
+            max_response_size,
            request.method,
            request.uri.decode("ascii"),
        )
@@ -772,7 +770,6 @@ class MatrixFederationHttpClient:
        backoff_on_404: bool = False,
        try_trailing_slash_on_400: bool = False,
        parser: Literal[None] = None,
-        max_response_size: Optional[int] = None,
    ) -> Union[JsonDict, list]:
        ...

@@ -790,7 +787,6 @@ class MatrixFederationHttpClient:
        backoff_on_404: bool = False,
        try_trailing_slash_on_400: bool = False,
        parser: Optional[ByteParser[T]] = None,
-        max_response_size: Optional[int] = None,
    ) -> T:
        ...

@@ -807,7 +803,6 @@ class MatrixFederationHttpClient:
        backoff_on_404: bool = False,
        try_trailing_slash_on_400: bool = False,
        parser: Optional[ByteParser] = None,
-        max_response_size: Optional[int] = None,
    ):
        """Sends the specified json data using PUT

@@ -843,8 +838,6 @@ class MatrixFederationHttpClient:
                enabled.
            parser: The parser to use to decode the response. Defaults to
                parsing as JSON.
-            max_response_size: The maximum size to read from the response, if None
-                uses the default.

        Returns:
            Succeeds when we get a 2xx HTTP response. The
@@ -895,7 +888,6 @@ class MatrixFederationHttpClient:
            response,
            start_ms,
            parser=parser,
-            max_response_size=max_response_size,
        )

        return body
@@ -984,7 +976,6 @@ class MatrixFederationHttpClient:
        ignore_backoff: bool = False,
        try_trailing_slash_on_400: bool = False,
        parser: Literal[None] = None,
-        max_response_size: Optional[int] = None,
    ) -> Union[JsonDict, list]:
        ...

@@ -999,7 +990,6 @@ class MatrixFederationHttpClient:
        ignore_backoff: bool = ...,
        try_trailing_slash_on_400: bool = ...,
        parser: ByteParser[T] = ...,
-        max_response_size: Optional[int] = ...,
    ) -> T:
        ...

@@ -1013,7 +1003,6 @@ class MatrixFederationHttpClient:
        ignore_backoff: bool = False,
        try_trailing_slash_on_400: bool = False,
        parser: Optional[ByteParser] = None,
-        max_response_size: Optional[int] = None,
    ):
        """GETs some json from the given host homeserver and path

@@ -1043,9 +1032,6 @@ class MatrixFederationHttpClient:
            parser: The parser to use to decode the response. Defaults to
                parsing as JSON.

-            max_response_size: The maximum size to read from the response. If None,
-                uses the default.
-
        Returns:
            Succeeds when we get a 2xx HTTP response. The
            result will be the decoded JSON body.
@@ -1090,7 +1076,6 @@ class MatrixFederationHttpClient:
            response,
            start_ms,
            parser=parser,
-            max_response_size=max_response_size,
        )

        return body
@@ -139,6 +139,7 @@ BASE_APPEND_CONTENT_RULES: List[Dict[str, Any]] = [
            {
                "kind": "event_match",
                "key": "content.body",
+                # Match the localpart of the requester's MXID.
                "pattern_type": "user_localpart",
            }
        ],
@@ -191,6 +192,7 @@ BASE_APPEND_OVERRIDE_RULES: List[Dict[str, Any]] = [
                "pattern": "invite",
                "_cache_key": "_invite_member",
            },
+            # Match the requester's MXID.
            {"kind": "event_match", "key": "state_key", "pattern_type": "user_id"},
        ],
        "actions": [
@@ -290,7 +292,7 @@ BASE_APPEND_OVERRIDE_RULES: List[Dict[str, Any]] = [
                "_cache_key": "_room_server_acl",
            }
        ],
-        "actions": ["dont_notify"],
+        "actions": [],
    },
 ]

@@ -350,6 +352,18 @@ BASE_APPEND_UNDERRIDE_RULES: List[Dict[str, Any]] = [
            {"set_tweak": "highlight", "value": False},
        ],
    },
+    {
+        "rule_id": "global/underride/.org.matrix.msc3772.thread_reply",
+        "conditions": [
+            {
+                "kind": "org.matrix.msc3772.relation_match",
+                "rel_type": "m.thread",
+                # Match the requester's MXID.
+                "sender_type": "user_id",
+            }
+        ],
+        "actions": ["notify", {"set_tweak": "highlight", "value": False}],
+    },
    {
        "rule_id": "global/underride/.m.rule.message",
        "conditions": [
@@ -13,8 +13,9 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.

+import itertools
 import logging
-from typing import TYPE_CHECKING, Any, Dict, List, Optional, Set, Tuple, Union
+from typing import TYPE_CHECKING, Any, Dict, Iterable, List, Optional, Set, Tuple, Union

 import attr
 from prometheus_client import Counter
@@ -121,6 +122,9 @@ class BulkPushRuleEvaluator:
            resizable=False,
        )

+        # Whether to support MSC3772 is supported.
+        self._relations_match_enabled = self.hs.config.experimental.msc3772_enabled
+
    async def _get_rules_for_event(
        self, event: EventBase, context: EventContext
    ) -> Dict[str, List[Dict[str, Any]]]:
@@ -192,6 +196,60 @@ class BulkPushRuleEvaluator:

        return pl_event.content if pl_event else {}, sender_level

+    async def _get_mutual_relations(
+        self, event: EventBase, rules: Iterable[Dict[str, Any]]
+    ) -> Dict[str, Set[Tuple[str, str]]]:
+        """
+        Fetch event metadata for events which related to the same event as the given event.
+
+        If the given event has no relation information, returns an empty dictionary.
+
+        Args:
+            event_id: The event ID which is targeted by relations.
+            rules: The push rules which will be processed for this event.
+
+        Returns:
+            A dictionary of relation type to:
+                A set of tuples of:
+                    The sender
+                    The event type
+        """
+
+        # If the experimental feature is not enabled, skip fetching relations.
+        if not self._relations_match_enabled:
+            return {}
+
+        # If the event does not have a relation, then cannot have any mutual
+        # relations.
+        relation = relation_from_event(event)
+        if not relation:
+            return {}
+
+        # Pre-filter to figure out which relation types are interesting.
+        rel_types = set()
+        for rule in rules:
+            # Skip disabled rules.
+            if "enabled" in rule and not rule["enabled"]:
+                continue
+
+            for condition in rule["conditions"]:
+                if condition["kind"] != "org.matrix.msc3772.relation_match":
+                    continue
+
+                # rel_type is required.
+                rel_type = condition.get("rel_type")
+                if rel_type:
+                    rel_types.add(rel_type)
+
+        # If no valid rules were found, no mutual relations.
+        if not rel_types:
+            return {}
+
+        # If any valid rules were found, fetch the mutual relations.
+        return await self.store.get_mutual_event_relations(
+            relation.parent_id, rel_types
+        )
+
    @measure_func("action_for_event_by_user")
    async def action_for_event_by_user(
        self, event: EventBase, context: EventContext
@@ -216,8 +274,17 @@ class BulkPushRuleEvaluator:
            sender_power_level,
        ) = await self._get_power_levels_and_sender_level(event, context)

+        relations = await self._get_mutual_relations(
+            event, itertools.chain(*rules_by_user.values())
+        )
+
        evaluator = PushRuleEvaluatorForEvent(
-            event, len(room_members), sender_power_level, power_levels
+            event,
+            len(room_members),
+            sender_power_level,
+            power_levels,
+            relations,
+            self._relations_match_enabled,
        )

        # If the event is not a state event check if any users ignore the sender.
@@ -48,6 +48,10 @@ def format_push_rules_for_user(
            elif pattern_type == "user_localpart":
                c["pattern"] = user.localpart

+            sender_type = c.pop("sender_type", None)
+            if sender_type == "user_id":
+                c["sender"] = user.to_string()
+
        rulearray = rules["global"][template_name]

        template_rule = _rule_to_template(r)
@@ -114,7 +114,7 @@ class Mailer:

        self.send_email_handler = hs.get_send_email_handler()
        self.store = self.hs.get_datastores().main
-        self.state_store = self.hs.get_storage().state
+        self.state_storage = self.hs.get_storage().state
        self.macaroon_gen = self.hs.get_macaroon_generator()
        self.state_handler = self.hs.get_state_handler()
        self.storage = hs.get_storage()
@@ -494,7 +494,7 @@ class Mailer:
            )
        else:
            # Attempt to check the historical state for the room.
-            historical_state = await self.state_store.get_state_for_event(
+            historical_state = await self.state_storage.get_state_for_event(
                event.event_id, StateFilter.from_types((type_state_key,))
            )
            sender_state_event = historical_state.get(type_state_key)
@@ -767,7 +767,7 @@ class Mailer:
                member_event_ids.append(sender_state_event_id)
            else:
                # Attempt to check the historical state for the room.
-                historical_state = await self.state_store.get_state_for_event(
+                historical_state = await self.state_storage.get_state_for_event(
                    event_id, StateFilter.from_types((type_state_key,))
                )
                sender_state_event = historical_state.get(type_state_key)
@@ -15,7 +15,7 @@

 import logging
 import re
-from typing import Any, Dict, List, Mapping, Optional, Pattern, Tuple, Union
+from typing import Any, Dict, List, Mapping, Optional, Pattern, Set, Tuple, Union

 from matrix_common.regex import glob_to_regex, to_word_pattern

@@ -120,11 +120,15 @@ class PushRuleEvaluatorForEvent:
        room_member_count: int,
        sender_power_level: int,
        power_levels: Dict[str, Union[int, Dict[str, int]]],
+        relations: Dict[str, Set[Tuple[str, str]]],
+        relations_match_enabled: bool,
    ):
        self._event = event
        self._room_member_count = room_member_count
        self._sender_power_level = sender_power_level
        self._power_levels = power_levels
+        self._relations = relations
+        self._relations_match_enabled = relations_match_enabled

        # Maps strings of e.g. 'content.body' -> event["content"]["body"]
        self._value_cache = _flatten_dict(event)
@@ -188,7 +192,16 @@ class PushRuleEvaluatorForEvent:
            return _sender_notification_permission(
                self._event, condition, self._sender_power_level, self._power_levels
            )
+        elif (
+            condition["kind"] == "org.matrix.msc3772.relation_match"
+            and self._relations_match_enabled
+        ):
+            return self._relation_match(condition, user_id)
        else:
+            # XXX This looks incorrect -- we have reached an unknown condition
+            #     kind and are unconditionally returning that it matches. Note
+            #     that it seems possible to provide a condition to the /pushrules
+            #     endpoint with an unknown kind, see _rule_tuple_from_request_object.
            return True

    def _event_match(self, condition: dict, user_id: str) -> bool:
@@ -256,6 +269,41 @@ class PushRuleEvaluatorForEvent:

        return bool(r.search(body))

+    def _relation_match(self, condition: dict, user_id: str) -> bool:
+        """
+        Check an "relation_match" push rule condition.
+
+        Args:
+            condition: The "event_match" push rule condition to match.
+            user_id: The user's MXID.
+
+        Returns:
+             True if the condition matches the event, False otherwise.
+        """
+        rel_type = condition.get("rel_type")
+        if not rel_type:
+            logger.warning("relation_match condition missing rel_type")
+            return False
+
+        sender_pattern = condition.get("sender")
+        if sender_pattern is None:
+            sender_type = condition.get("sender_type")
+            if sender_type == "user_id":
+                sender_pattern = user_id
+        type_pattern = condition.get("type")
+
+        # If any other relations matches, return True.
+        for sender, event_type in self._relations.get(rel_type, ()):
+            if sender_pattern and not _glob_matches(sender_pattern, sender):
+                continue
+            if type_pattern and not _glob_matches(type_pattern, event_type):
+                continue
+            # All values must have matched.
+            return True
+
+        # No relations matched.
+        return False
+

 # Caches (string, is_glob, word_boundary) -> regex for push. See _glob_matches
 regex_cache: LruCache[Tuple[str, bool, bool], Pattern] = LruCache(
@@ -26,7 +26,6 @@ from synapse.rest.client import (
    directory,
    events,
    filter,
-    groups,
    initial_sync,
    keys,
    knock,
@@ -118,8 +117,6 @@ class ClientRestResource(JsonResource):
        thirdparty.register_servlets(hs, client_resource)
        sendtodevice.register_servlets(hs, client_resource)
        user_directory.register_servlets(hs, client_resource)
-        if hs.config.experimental.groups_enabled:
-            groups.register_servlets(hs, client_resource)
        room_upgrade_rest_servlet.register_servlets(hs, client_resource)
        room_batch.register_servlets(hs, client_resource)
        capabilities.register_servlets(hs, client_resource)
@@ -47,7 +47,6 @@ from synapse.rest.admin.federation import (
    DestinationRestServlet,
    ListDestinationsRestServlet,
 )
-from synapse.rest.admin.groups import DeleteGroupAdminRestServlet
 from synapse.rest.admin.media import ListMediaInRoom, register_servlets_for_media_repo
 from synapse.rest.admin.registration_tokens import (
    ListRegistrationTokensRestServlet,
@@ -293,8 +292,6 @@ def register_servlets_for_client_rest_resource(
    ResetPasswordRestServlet(hs).register(http_server)
    SearchUsersRestServlet(hs).register(http_server)
    UserRegisterServlet(hs).register(http_server)
-    if hs.config.experimental.groups_enabled:
-        DeleteGroupAdminRestServlet(hs).register(http_server)
    AccountValidityRenewServlet(hs).register(http_server)

    # Load the media repo ones if we're using them. Otherwise load the servlets which
@@ -1,50 +0,0 @@
-# Copyright 2019 The Matrix.org Foundation C.I.C.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-import logging
-from http import HTTPStatus
-from typing import TYPE_CHECKING, Tuple
-
-from synapse.api.errors import SynapseError
-from synapse.http.servlet import RestServlet
-from synapse.http.site import SynapseRequest
-from synapse.rest.admin._base import admin_patterns, assert_user_is_admin
-from synapse.types import JsonDict
-
-if TYPE_CHECKING:
-    from synapse.server import HomeServer
-
-logger = logging.getLogger(__name__)
-
-
-class DeleteGroupAdminRestServlet(RestServlet):
-    """Allows deleting of local groups"""
-
-    PATTERNS = admin_patterns("/delete_group/(?P<group_id>[^/]*)$")
-
-    def __init__(self, hs: "HomeServer"):
-        self.group_server = hs.get_groups_server_handler()
-        self.is_mine_id = hs.is_mine_id
-        self.auth = hs.get_auth()
-
-    async def on_POST(
-        self, request: SynapseRequest, group_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        await assert_user_is_admin(self.auth, requester.user)
-
-        if not self.is_mine_id(group_id):
-            raise SynapseError(HTTPStatus.BAD_REQUEST, "Can only delete local groups")
-
-        await self.group_server.delete_group(group_id, requester.user.to_string())
-        return HTTPStatus.OK, {}
@@ -1,962 +0,0 @@
-# Copyright 2017 Vector Creations Ltd
-# Copyright 2018 New Vector Ltd
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import logging
-from functools import wraps
-from typing import TYPE_CHECKING, Any, Awaitable, Callable, Optional, Tuple
-
-from twisted.web.server import Request
-
-from synapse.api.constants import (
-    MAX_GROUP_CATEGORYID_LENGTH,
-    MAX_GROUP_ROLEID_LENGTH,
-    MAX_GROUPID_LENGTH,
-)
-from synapse.api.errors import Codes, SynapseError
-from synapse.handlers.groups_local import GroupsLocalHandler
-from synapse.http.server import HttpServer
-from synapse.http.servlet import (
-    RestServlet,
-    assert_params_in_dict,
-    parse_json_object_from_request,
-)
-from synapse.http.site import SynapseRequest
-from synapse.types import GroupID, JsonDict
-
-from ._base import client_patterns
-
-if TYPE_CHECKING:
-    from synapse.server import HomeServer
-
-logger = logging.getLogger(__name__)
-
-
-def _validate_group_id(
-    f: Callable[..., Awaitable[Tuple[int, JsonDict]]]
-) -> Callable[..., Awaitable[Tuple[int, JsonDict]]]:
-    """Wrapper to validate the form of the group ID.
-
-    Can be applied to any on_FOO methods that accepts a group ID as a URL parameter.
-    """
-
-    @wraps(f)
-    def wrapper(
-        self: RestServlet, request: Request, group_id: str, *args: Any, **kwargs: Any
-    ) -> Awaitable[Tuple[int, JsonDict]]:
-        if not GroupID.is_valid(group_id):
-            raise SynapseError(400, "%s is not a legal group ID" % (group_id,))
-
-        return f(self, request, group_id, *args, **kwargs)
-
-    return wrapper
-
-
-class GroupServlet(RestServlet):
-    """Get the group profile"""
-
-    PATTERNS = client_patterns("/groups/(?P<group_id>[^/]*)/profile$")
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    @_validate_group_id
-    async def on_GET(
-        self, request: SynapseRequest, group_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request, allow_guest=True)
-        requester_user_id = requester.user.to_string()
-
-        group_description = await self.groups_handler.get_group_profile(
-            group_id, requester_user_id
-        )
-
-        return 200, group_description
-
-    @_validate_group_id
-    async def on_POST(
-        self, request: SynapseRequest, group_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        content = parse_json_object_from_request(request)
-        assert_params_in_dict(
-            content, ("name", "avatar_url", "short_description", "long_description")
-        )
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot create group profiles."
-        await self.groups_handler.update_group_profile(
-            group_id, requester_user_id, content
-        )
-
-        return 200, {}
-
-
-class GroupSummaryServlet(RestServlet):
-    """Get the full group summary"""
-
-    PATTERNS = client_patterns("/groups/(?P<group_id>[^/]*)/summary$")
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    @_validate_group_id
-    async def on_GET(
-        self, request: SynapseRequest, group_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request, allow_guest=True)
-        requester_user_id = requester.user.to_string()
-
-        get_group_summary = await self.groups_handler.get_group_summary(
-            group_id, requester_user_id
-        )
-
-        return 200, get_group_summary
-
-
-class GroupSummaryRoomsCatServlet(RestServlet):
-    """Update/delete a rooms entry in the summary.
-
-    Matches both:
-        - /groups/:group/summary/rooms/:room_id
-        - /groups/:group/summary/categories/:category/rooms/:room_id
-    """
-
-    PATTERNS = client_patterns(
-        "/groups/(?P<group_id>[^/]*)/summary"
-        "(/categories/(?P<category_id>[^/]+))?"
-        "/rooms/(?P<room_id>[^/]*)$"
-    )
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    @_validate_group_id
-    async def on_PUT(
-        self,
-        request: SynapseRequest,
-        group_id: str,
-        category_id: Optional[str],
-        room_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        if category_id == "":
-            raise SynapseError(400, "category_id cannot be empty", Codes.INVALID_PARAM)
-
-        if category_id and len(category_id) > MAX_GROUP_CATEGORYID_LENGTH:
-            raise SynapseError(
-                400,
-                "category_id may not be longer than %s characters"
-                % (MAX_GROUP_CATEGORYID_LENGTH,),
-                Codes.INVALID_PARAM,
-            )
-
-        content = parse_json_object_from_request(request)
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot modify group summaries."
-        resp = await self.groups_handler.update_group_summary_room(
-            group_id,
-            requester_user_id,
-            room_id=room_id,
-            category_id=category_id,
-            content=content,
-        )
-
-        return 200, resp
-
-    @_validate_group_id
-    async def on_DELETE(
-        self, request: SynapseRequest, group_id: str, category_id: str, room_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot modify group profiles."
-        resp = await self.groups_handler.delete_group_summary_room(
-            group_id, requester_user_id, room_id=room_id, category_id=category_id
-        )
-
-        return 200, resp
-
-
-class GroupCategoryServlet(RestServlet):
-    """Get/add/update/delete a group category"""
-
-    PATTERNS = client_patterns(
-        "/groups/(?P<group_id>[^/]*)/categories/(?P<category_id>[^/]+)$"
-    )
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    @_validate_group_id
-    async def on_GET(
-        self, request: SynapseRequest, group_id: str, category_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request, allow_guest=True)
-        requester_user_id = requester.user.to_string()
-
-        category = await self.groups_handler.get_group_category(
-            group_id, requester_user_id, category_id=category_id
-        )
-
-        return 200, category
-
-    @_validate_group_id
-    async def on_PUT(
-        self, request: SynapseRequest, group_id: str, category_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        if not category_id:
-            raise SynapseError(400, "category_id cannot be empty", Codes.INVALID_PARAM)
-
-        if len(category_id) > MAX_GROUP_CATEGORYID_LENGTH:
-            raise SynapseError(
-                400,
-                "category_id may not be longer than %s characters"
-                % (MAX_GROUP_CATEGORYID_LENGTH,),
-                Codes.INVALID_PARAM,
-            )
-
-        content = parse_json_object_from_request(request)
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot modify group categories."
-        resp = await self.groups_handler.update_group_category(
-            group_id, requester_user_id, category_id=category_id, content=content
-        )
-
-        return 200, resp
-
-    @_validate_group_id
-    async def on_DELETE(
-        self, request: SynapseRequest, group_id: str, category_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot modify group categories."
-        resp = await self.groups_handler.delete_group_category(
-            group_id, requester_user_id, category_id=category_id
-        )
-
-        return 200, resp
-
-
-class GroupCategoriesServlet(RestServlet):
-    """Get all group categories"""
-
-    PATTERNS = client_patterns("/groups/(?P<group_id>[^/]*)/categories/$")
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    @_validate_group_id
-    async def on_GET(
-        self, request: SynapseRequest, group_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request, allow_guest=True)
-        requester_user_id = requester.user.to_string()
-
-        category = await self.groups_handler.get_group_categories(
-            group_id, requester_user_id
-        )
-
-        return 200, category
-
-
-class GroupRoleServlet(RestServlet):
-    """Get/add/update/delete a group role"""
-
-    PATTERNS = client_patterns("/groups/(?P<group_id>[^/]*)/roles/(?P<role_id>[^/]+)$")
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    @_validate_group_id
-    async def on_GET(
-        self, request: SynapseRequest, group_id: str, role_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request, allow_guest=True)
-        requester_user_id = requester.user.to_string()
-
-        category = await self.groups_handler.get_group_role(
-            group_id, requester_user_id, role_id=role_id
-        )
-
-        return 200, category
-
-    @_validate_group_id
-    async def on_PUT(
-        self, request: SynapseRequest, group_id: str, role_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        if not role_id:
-            raise SynapseError(400, "role_id cannot be empty", Codes.INVALID_PARAM)
-
-        if len(role_id) > MAX_GROUP_ROLEID_LENGTH:
-            raise SynapseError(
-                400,
-                "role_id may not be longer than %s characters"
-                % (MAX_GROUP_ROLEID_LENGTH,),
-                Codes.INVALID_PARAM,
-            )
-
-        content = parse_json_object_from_request(request)
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot modify group roles."
-        resp = await self.groups_handler.update_group_role(
-            group_id, requester_user_id, role_id=role_id, content=content
-        )
-
-        return 200, resp
-
-    @_validate_group_id
-    async def on_DELETE(
-        self, request: SynapseRequest, group_id: str, role_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot modify group roles."
-        resp = await self.groups_handler.delete_group_role(
-            group_id, requester_user_id, role_id=role_id
-        )
-
-        return 200, resp
-
-
-class GroupRolesServlet(RestServlet):
-    """Get all group roles"""
-
-    PATTERNS = client_patterns("/groups/(?P<group_id>[^/]*)/roles/$")
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    @_validate_group_id
-    async def on_GET(
-        self, request: SynapseRequest, group_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request, allow_guest=True)
-        requester_user_id = requester.user.to_string()
-
-        category = await self.groups_handler.get_group_roles(
-            group_id, requester_user_id
-        )
-
-        return 200, category
-
-
-class GroupSummaryUsersRoleServlet(RestServlet):
-    """Update/delete a user's entry in the summary.
-
-    Matches both:
-        - /groups/:group/summary/users/:room_id
-        - /groups/:group/summary/roles/:role/users/:user_id
-    """
-
-    PATTERNS = client_patterns(
-        "/groups/(?P<group_id>[^/]*)/summary"
-        "(/roles/(?P<role_id>[^/]+))?"
-        "/users/(?P<user_id>[^/]*)$"
-    )
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    @_validate_group_id
-    async def on_PUT(
-        self,
-        request: SynapseRequest,
-        group_id: str,
-        role_id: Optional[str],
-        user_id: str,
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        if role_id == "":
-            raise SynapseError(400, "role_id cannot be empty", Codes.INVALID_PARAM)
-
-        if role_id and len(role_id) > MAX_GROUP_ROLEID_LENGTH:
-            raise SynapseError(
-                400,
-                "role_id may not be longer than %s characters"
-                % (MAX_GROUP_ROLEID_LENGTH,),
-                Codes.INVALID_PARAM,
-            )
-
-        content = parse_json_object_from_request(request)
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot modify group summaries."
-        resp = await self.groups_handler.update_group_summary_user(
-            group_id,
-            requester_user_id,
-            user_id=user_id,
-            role_id=role_id,
-            content=content,
-        )
-
-        return 200, resp
-
-    @_validate_group_id
-    async def on_DELETE(
-        self, request: SynapseRequest, group_id: str, role_id: str, user_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot modify group summaries."
-        resp = await self.groups_handler.delete_group_summary_user(
-            group_id, requester_user_id, user_id=user_id, role_id=role_id
-        )
-
-        return 200, resp
-
-
-class GroupRoomServlet(RestServlet):
-    """Get all rooms in a group"""
-
-    PATTERNS = client_patterns("/groups/(?P<group_id>[^/]*)/rooms$")
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    @_validate_group_id
-    async def on_GET(
-        self, request: SynapseRequest, group_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request, allow_guest=True)
-        requester_user_id = requester.user.to_string()
-
-        result = await self.groups_handler.get_rooms_in_group(
-            group_id, requester_user_id
-        )
-
-        return 200, result
-
-
-class GroupUsersServlet(RestServlet):
-    """Get all users in a group"""
-
-    PATTERNS = client_patterns("/groups/(?P<group_id>[^/]*)/users$")
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    @_validate_group_id
-    async def on_GET(
-        self, request: SynapseRequest, group_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request, allow_guest=True)
-        requester_user_id = requester.user.to_string()
-
-        result = await self.groups_handler.get_users_in_group(
-            group_id, requester_user_id
-        )
-
-        return 200, result
-
-
-class GroupInvitedUsersServlet(RestServlet):
-    """Get users invited to a group"""
-
-    PATTERNS = client_patterns("/groups/(?P<group_id>[^/]*)/invited_users$")
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    @_validate_group_id
-    async def on_GET(
-        self, request: SynapseRequest, group_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        result = await self.groups_handler.get_invited_users_in_group(
-            group_id, requester_user_id
-        )
-
-        return 200, result
-
-
-class GroupSettingJoinPolicyServlet(RestServlet):
-    """Set group join policy"""
-
-    PATTERNS = client_patterns("/groups/(?P<group_id>[^/]*)/settings/m.join_policy$")
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    @_validate_group_id
-    async def on_PUT(
-        self, request: SynapseRequest, group_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        content = parse_json_object_from_request(request)
-
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot modify group join policy."
-        result = await self.groups_handler.set_group_join_policy(
-            group_id, requester_user_id, content
-        )
-
-        return 200, result
-
-
-class GroupCreateServlet(RestServlet):
-    """Create a group"""
-
-    PATTERNS = client_patterns("/create_group$")
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-        self.server_name = hs.hostname
-
-    async def on_POST(self, request: SynapseRequest) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        # TODO: Create group on remote server
-        content = parse_json_object_from_request(request)
-        localpart = content.pop("localpart")
-        group_id = GroupID(localpart, self.server_name).to_string()
-
-        if not localpart:
-            raise SynapseError(400, "Group ID cannot be empty", Codes.INVALID_PARAM)
-
-        if len(group_id) > MAX_GROUPID_LENGTH:
-            raise SynapseError(
-                400,
-                "Group ID may not be longer than %s characters" % (MAX_GROUPID_LENGTH,),
-                Codes.INVALID_PARAM,
-            )
-
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot create groups."
-        result = await self.groups_handler.create_group(
-            group_id, requester_user_id, content
-        )
-
-        return 200, result
-
-
-class GroupAdminRoomsServlet(RestServlet):
-    """Add a room to the group"""
-
-    PATTERNS = client_patterns(
-        "/groups/(?P<group_id>[^/]*)/admin/rooms/(?P<room_id>[^/]*)$"
-    )
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    @_validate_group_id
-    async def on_PUT(
-        self, request: SynapseRequest, group_id: str, room_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        content = parse_json_object_from_request(request)
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot modify rooms in a group."
-        result = await self.groups_handler.add_room_to_group(
-            group_id, requester_user_id, room_id, content
-        )
-
-        return 200, result
-
-    @_validate_group_id
-    async def on_DELETE(
-        self, request: SynapseRequest, group_id: str, room_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot modify group categories."
-        result = await self.groups_handler.remove_room_from_group(
-            group_id, requester_user_id, room_id
-        )
-
-        return 200, result
-
-
-class GroupAdminRoomsConfigServlet(RestServlet):
-    """Update the config of a room in a group"""
-
-    PATTERNS = client_patterns(
-        "/groups/(?P<group_id>[^/]*)/admin/rooms/(?P<room_id>[^/]*)"
-        "/config/(?P<config_key>[^/]*)$"
-    )
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    @_validate_group_id
-    async def on_PUT(
-        self, request: SynapseRequest, group_id: str, room_id: str, config_key: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        content = parse_json_object_from_request(request)
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot modify group categories."
-        result = await self.groups_handler.update_room_in_group(
-            group_id, requester_user_id, room_id, config_key, content
-        )
-
-        return 200, result
-
-
-class GroupAdminUsersInviteServlet(RestServlet):
-    """Invite a user to the group"""
-
-    PATTERNS = client_patterns(
-        "/groups/(?P<group_id>[^/]*)/admin/users/invite/(?P<user_id>[^/]*)$"
-    )
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-        self.store = hs.get_datastores().main
-        self.is_mine_id = hs.is_mine_id
-
-    @_validate_group_id
-    async def on_PUT(
-        self, request: SynapseRequest, group_id: str, user_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        content = parse_json_object_from_request(request)
-        config = content.get("config", {})
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot invite users to a group."
-        result = await self.groups_handler.invite(
-            group_id, user_id, requester_user_id, config
-        )
-
-        return 200, result
-
-
-class GroupAdminUsersKickServlet(RestServlet):
-    """Kick a user from the group"""
-
-    PATTERNS = client_patterns(
-        "/groups/(?P<group_id>[^/]*)/admin/users/remove/(?P<user_id>[^/]*)$"
-    )
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    @_validate_group_id
-    async def on_PUT(
-        self, request: SynapseRequest, group_id: str, user_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        content = parse_json_object_from_request(request)
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot kick users from a group."
-        result = await self.groups_handler.remove_user_from_group(
-            group_id, user_id, requester_user_id, content
-        )
-
-        return 200, result
-
-
-class GroupSelfLeaveServlet(RestServlet):
-    """Leave a joined group"""
-
-    PATTERNS = client_patterns("/groups/(?P<group_id>[^/]*)/self/leave$")
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    @_validate_group_id
-    async def on_PUT(
-        self, request: SynapseRequest, group_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        content = parse_json_object_from_request(request)
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot leave a group for a users."
-        result = await self.groups_handler.remove_user_from_group(
-            group_id, requester_user_id, requester_user_id, content
-        )
-
-        return 200, result
-
-
-class GroupSelfJoinServlet(RestServlet):
-    """Attempt to join a group, or knock"""
-
-    PATTERNS = client_patterns("/groups/(?P<group_id>[^/]*)/self/join$")
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    @_validate_group_id
-    async def on_PUT(
-        self, request: SynapseRequest, group_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        content = parse_json_object_from_request(request)
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot join a user to a group."
-        result = await self.groups_handler.join_group(
-            group_id, requester_user_id, content
-        )
-
-        return 200, result
-
-
-class GroupSelfAcceptInviteServlet(RestServlet):
-    """Accept a group invite"""
-
-    PATTERNS = client_patterns("/groups/(?P<group_id>[^/]*)/self/accept_invite$")
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    @_validate_group_id
-    async def on_PUT(
-        self, request: SynapseRequest, group_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        content = parse_json_object_from_request(request)
-        assert isinstance(
-            self.groups_handler, GroupsLocalHandler
-        ), "Workers cannot accept an invite to a group."
-        result = await self.groups_handler.accept_invite(
-            group_id, requester_user_id, content
-        )
-
-        return 200, result
-
-
-class GroupSelfUpdatePublicityServlet(RestServlet):
-    """Update whether we publicise a users membership of a group"""
-
-    PATTERNS = client_patterns("/groups/(?P<group_id>[^/]*)/self/update_publicity$")
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.store = hs.get_datastores().main
-
-    @_validate_group_id
-    async def on_PUT(
-        self, request: SynapseRequest, group_id: str
-    ) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request)
-        requester_user_id = requester.user.to_string()
-
-        content = parse_json_object_from_request(request)
-        publicise = content["publicise"]
-        await self.store.update_group_publicity(group_id, requester_user_id, publicise)
-
-        return 200, {}
-
-
-class PublicisedGroupsForUserServlet(RestServlet):
-    """Get the list of groups a user is advertising"""
-
-    PATTERNS = client_patterns("/publicised_groups/(?P<user_id>[^/]*)$")
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.store = hs.get_datastores().main
-        self.groups_handler = hs.get_groups_local_handler()
-
-    async def on_GET(
-        self, request: SynapseRequest, user_id: str
-    ) -> Tuple[int, JsonDict]:
-        await self.auth.get_user_by_req(request, allow_guest=True)
-
-        result = await self.groups_handler.get_publicised_groups_for_user(user_id)
-
-        return 200, result
-
-
-class PublicisedGroupsForUsersServlet(RestServlet):
-    """Get the list of groups a user is advertising"""
-
-    PATTERNS = client_patterns("/publicised_groups$")
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.store = hs.get_datastores().main
-        self.groups_handler = hs.get_groups_local_handler()
-
-    async def on_POST(self, request: SynapseRequest) -> Tuple[int, JsonDict]:
-        await self.auth.get_user_by_req(request, allow_guest=True)
-
-        content = parse_json_object_from_request(request)
-        user_ids = content["user_ids"]
-
-        result = await self.groups_handler.bulk_get_publicised_groups(user_ids)
-
-        return 200, result
-
-
-class GroupsForUserServlet(RestServlet):
-    """Get all groups the logged in user is joined to"""
-
-    PATTERNS = client_patterns("/joined_groups$")
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        self.auth = hs.get_auth()
-        self.clock = hs.get_clock()
-        self.groups_handler = hs.get_groups_local_handler()
-
-    async def on_GET(self, request: SynapseRequest) -> Tuple[int, JsonDict]:
-        requester = await self.auth.get_user_by_req(request, allow_guest=True)
-        requester_user_id = requester.user.to_string()
-
-        result = await self.groups_handler.get_joined_groups(requester_user_id)
-
-        return 200, result
-
-
-def register_servlets(hs: "HomeServer", http_server: HttpServer) -> None:
-    GroupServlet(hs).register(http_server)
-    GroupSummaryServlet(hs).register(http_server)
-    GroupInvitedUsersServlet(hs).register(http_server)
-    GroupUsersServlet(hs).register(http_server)
-    GroupRoomServlet(hs).register(http_server)
-    GroupSettingJoinPolicyServlet(hs).register(http_server)
-    GroupCreateServlet(hs).register(http_server)
-    GroupAdminRoomsServlet(hs).register(http_server)
-    GroupAdminRoomsConfigServlet(hs).register(http_server)
-    GroupAdminUsersInviteServlet(hs).register(http_server)
-    GroupAdminUsersKickServlet(hs).register(http_server)
-    GroupSelfLeaveServlet(hs).register(http_server)
-    GroupSelfJoinServlet(hs).register(http_server)
-    GroupSelfAcceptInviteServlet(hs).register(http_server)
-    GroupsForUserServlet(hs).register(http_server)
-    GroupCategoryServlet(hs).register(http_server)
-    GroupCategoriesServlet(hs).register(http_server)
-    GroupSummaryRoomsCatServlet(hs).register(http_server)
-    GroupRoleServlet(hs).register(http_server)
-    GroupRolesServlet(hs).register(http_server)
-    GroupSelfUpdatePublicityServlet(hs).register(http_server)
-    GroupSummaryUsersRoleServlet(hs).register(http_server)
-    PublicisedGroupsForUserServlet(hs).register(http_server)
-    PublicisedGroupsForUsersServlet(hs).register(http_server)
@@ -1193,12 +1193,7 @@ class TimestampLookupRestServlet(RestServlet):


 class RoomHierarchyRestServlet(RestServlet):
-    PATTERNS = (
-        re.compile(
-            "^/_matrix/client/(v1|unstable/org.matrix.msc2946)"
-            "/rooms/(?P<room_id>[^/]*)/hierarchy$"
-        ),
-    )
+    PATTERNS = (re.compile("^/_matrix/client/v1/rooms/(?P<room_id>[^/]*)/hierarchy$"),)

    def __init__(self, hs: "HomeServer"):
        super().__init__()
@@ -298,14 +298,6 @@ class SyncRestServlet(RestServlet):
        if archived:
            response["rooms"][Membership.LEAVE] = archived

-        if sync_result.groups is not None:
-            if sync_result.groups.join:
-                response["groups"][Membership.JOIN] = sync_result.groups.join
-            if sync_result.groups.invite:
-                response["groups"][Membership.INVITE] = sync_result.groups.invite
-            if sync_result.groups.leave:
-                response["groups"][Membership.LEAVE] = sync_result.groups.leave
-
        return response

    @staticmethod
@@ -281,7 +281,7 @@ def parse_html_description(tree: "etree.Element") -> Optional[str]:


 def _iterate_over_text(
-    tree: "etree.Element", *tags_to_ignore: Iterable[Union[str, "etree.Comment"]]
+    tree: "etree.Element", *tags_to_ignore: Union[str, "etree.Comment"]
 ) -> Generator[str, None, None]:
    """Iterate over the tree returning text nodes in a depth first fashion,
    skipping text nodes inside certain tags.
@@ -21,17 +21,7 @@
 import abc
 import functools
 import logging
-from typing import (
-    TYPE_CHECKING,
-    Any,
-    Callable,
-    Dict,
-    List,
-    Optional,
-    TypeVar,
-    Union,
-    cast,
-)
+from typing import TYPE_CHECKING, Any, Callable, Dict, List, Optional, TypeVar, cast

 from twisted.internet.interfaces import IOpenSSLContextFactory
 from twisted.internet.tcp import Port
@@ -60,8 +50,6 @@ from synapse.federation.federation_server import (
 from synapse.federation.send_queue import FederationRemoteSendQueue
 from synapse.federation.sender import AbstractFederationSender, FederationSender
 from synapse.federation.transport.client import TransportLayerClient
-from synapse.groups.attestations import GroupAttestationSigning, GroupAttestionRenewer
-from synapse.groups.groups_server import GroupsServerHandler, GroupsServerWorkerHandler
 from synapse.handlers.account import AccountHandler
 from synapse.handlers.account_data import AccountDataHandler
 from synapse.handlers.account_validity import AccountValidityHandler
@@ -79,7 +67,6 @@ from synapse.handlers.event_auth import EventAuthHandler
 from synapse.handlers.events import EventHandler, EventStreamHandler
 from synapse.handlers.federation import FederationHandler
 from synapse.handlers.federation_event import FederationEventHandler
-from synapse.handlers.groups_local import GroupsLocalHandler, GroupsLocalWorkerHandler
 from synapse.handlers.identity import IdentityHandler
 from synapse.handlers.initial_sync import InitialSyncHandler
 from synapse.handlers.message import EventCreationHandler, MessageHandler
@@ -651,30 +638,6 @@ class HomeServer(metaclass=abc.ABCMeta):
    def get_user_directory_handler(self) -> UserDirectoryHandler:
        return UserDirectoryHandler(self)

-    @cache_in_self
-    def get_groups_local_handler(
-        self,
-    ) -> Union[GroupsLocalWorkerHandler, GroupsLocalHandler]:
-        if self.config.worker.worker_app:
-            return GroupsLocalWorkerHandler(self)
-        else:
-            return GroupsLocalHandler(self)
-
-    @cache_in_self
-    def get_groups_server_handler(self):
-        if self.config.worker.worker_app:
-            return GroupsServerWorkerHandler(self)
-        else:
-            return GroupsServerHandler(self)
-
-    @cache_in_self
-    def get_groups_attestation_signing(self) -> GroupAttestationSigning:
-        return GroupAttestationSigning(self)
-
-    @cache_in_self
-    def get_groups_attestation_renewer(self) -> GroupAttestionRenewer:
-        return GroupAttestionRenewer(self)
-
    @cache_in_self
    def get_stats_handler(self) -> StatsHandler:
        return StatsHandler(self)
@@ -127,7 +127,7 @@ class StateHandler:
    def __init__(self, hs: "HomeServer"):
        self.clock = hs.get_clock()
        self.store = hs.get_datastores().main
-        self.state_store = hs.get_storage().state
+        self.state_storage = hs.get_storage().state
        self.hs = hs
        self._state_resolution_handler = hs.get_state_resolution_handler()
        self._storage = hs.get_storage()
@@ -261,7 +261,7 @@ class StateHandler:
    async def compute_event_context(
        self,
        event: EventBase,
-        old_state: Optional[Iterable[EventBase]] = None,
+        state_ids_before_event: Optional[StateMap[str]] = None,
        partial_state: bool = False,
    ) -> EventContext:
        """Build an EventContext structure for a non-outlier event.
@@ -273,12 +273,12 @@ class StateHandler:

        Args:
            event:
-            old_state: The state at the event if it can't be
-                calculated from existing events. This is normally only specified
-                when receiving an event from federation where we don't have the
-                prev events for, e.g. when backfilling.
-            partial_state: True if `old_state` is partial and omits non-critical
-                membership events
+            state_ids_before_event: The event ids of the state before the event if
+                it can't be calculated from existing events. This is normally
+                only specified when receiving an event from federation where we
+                don't have the prev events, e.g. when backfilling.
+            partial_state: True if `state_ids_before_event` is partial and omits
+                non-critical membership events
        Returns:
            The event context.
        """
@@ -286,13 +286,11 @@ class StateHandler:
        assert not event.internal_metadata.is_outlier()

        #
-        # first of all, figure out the state before the event
+        # first of all, figure out the state before the event, unless we
+        # already have it.
        #
-        if old_state:
+        if state_ids_before_event:
            # if we're given the state before the event, then we use that
-            state_ids_before_event: StateMap[str] = {
-                (s.type, s.state_key): s.event_id for s in old_state
-            }
            state_group_before_event = None
            state_group_before_event_prev_group = None
            deltas_to_state_group_before_event = None
@@ -339,7 +337,7 @@ class StateHandler:
        #

        if not state_group_before_event:
-            state_group_before_event = await self.state_store.store_state_group(
+            state_group_before_event = await self.state_storage.store_state_group(
                event.event_id,
                event.room_id,
                prev_group=state_group_before_event_prev_group,
@@ -384,7 +382,7 @@ class StateHandler:
        state_ids_after_event[key] = event.event_id
        delta_ids = {key: event.event_id}

-        state_group_after_event = await self.state_store.store_state_group(
+        state_group_after_event = await self.state_storage.store_state_group(
            event.event_id,
            event.room_id,
            prev_group=state_group_before_event,
@@ -418,7 +416,7 @@ class StateHandler:
        """
        logger.debug("resolve_state_groups event_ids %s", event_ids)

-        state_groups = await self.state_store.get_state_group_for_events(event_ids)
+        state_groups = await self.state_storage.get_state_group_for_events(event_ids)

        state_group_ids = state_groups.values()

@@ -426,8 +424,8 @@ class StateHandler:
        state_group_ids_set = set(state_group_ids)
        if len(state_group_ids_set) == 1:
            (state_group_id,) = state_group_ids_set
-            state = await self.state_store.get_state_for_groups(state_group_ids_set)
-            prev_group, delta_ids = await self.state_store.get_state_group_delta(
+            state = await self.state_storage.get_state_for_groups(state_group_ids_set)
+            prev_group, delta_ids = await self.state_storage.get_state_group_delta(
                state_group_id
            )
            return _StateCacheEntry(
@@ -441,7 +439,7 @@ class StateHandler:

        room_version = await self.store.get_room_version_id(room_id)

-        state_to_resolve = await self.state_store.get_state_for_groups(
+        state_to_resolve = await self.state_storage.get_state_for_groups(
            state_group_ids_set
        )

@@ -1057,7 +1057,7 @@ class EventFederationWorkerStore(SignatureWorkerStore, EventsWorkerStore, SQLBas
            INNER JOIN batch_events AS c
            ON i.next_batch_id = c.batch_id
            /* Get the depth of the batch start event from the events table */
-            INNER JOIN events AS e USING (event_id)
+            INNER JOIN events AS e ON c.event_id = e.event_id
            /* Find an insertion event which matches the given event_id */
            WHERE i.event_id = ?
            LIMIT ?
@@ -938,7 +938,7 @@ class EventPushActionsWorkerStore(SQLBaseStore):
        users can still get a list of recent highlights.

        Args:
-            txn: The transcation
+            txn: The transaction
            room_id: Room ID to delete from
            user_id: user ID to delete for
            stream_ordering: The lowest stream ordering which will
@@ -1828,6 +1828,10 @@ class PersistEventsStore:
            self.store.get_aggregation_groups_for_event.invalidate,
            (relation.parent_id,),
        )
+        txn.call_after(
+            self.store.get_mutual_event_relations_for_rel_type.invalidate,
+            (relation.parent_id,),
+        )

        if relation.rel_type == RelationTypes.REPLACE:
            txn.call_after(
@@ -2004,6 +2008,11 @@ class PersistEventsStore:
            self.store._invalidate_cache_and_stream(
                txn, self.store.get_thread_participated, (redacted_relates_to,)
            )
+            self.store._invalidate_cache_and_stream(
+                txn,
+                self.store.get_mutual_event_relations_for_rel_type,
+                (redacted_relates_to,),
+            )

        self.db_pool.simple_delete_txn(
            txn, table="event_relations", keyvalues={"event_id": redacted_event_id}
@@ -61,6 +61,11 @@ def _is_experimental_rule_enabled(
        and not experimental_config.msc3786_enabled
    ):
        return False
+    if (
+        rule_id == "global/underride/.org.matrix.msc3772.thread_reply"
+        and not experimental_config.msc3772_enabled
+    ):
+        return False
    return True


@@ -169,7 +174,7 @@ class PushRulesWorkerStore(
                "conditions",
                "actions",
            ),
-            desc="get_push_rules_enabled_for_user",
+            desc="get_push_rules_for_user",
        )

        rows.sort(key=lambda row: (-int(row["priority_class"]), -int(row["priority"])))
@@ -183,10 +188,10 @@ class PushRulesWorkerStore(
        results = await self.db_pool.simple_select_list(
            table="push_rules_enable",
            keyvalues={"user_name": user_id},
-            retcols=("user_name", "rule_id", "enabled"),
+            retcols=("rule_id", "enabled"),
            desc="get_push_rules_enabled_for_user",
        )
-        return {r["rule_id"]: False if r["enabled"] == 0 else True for r in results}
+        return {r["rule_id"]: bool(r["enabled"]) for r in results}

    async def have_push_rules_changed_for_user(
        self, user_id: str, last_id: int
@@ -208,11 +213,7 @@ class PushRulesWorkerStore(
                "have_push_rules_changed", have_push_rules_changed_txn
            )

-    @cachedList(
-        cached_method_name="get_push_rules_for_user",
-        list_name="user_ids",
-        num_args=1,
-    )
+    @cachedList(cached_method_name="get_push_rules_for_user", list_name="user_ids")
    async def bulk_get_push_rules(
        self, user_ids: Collection[str]
    ) -> Dict[str, List[JsonDict]]:
@@ -244,9 +245,7 @@ class PushRulesWorkerStore(
        return results

    @cachedList(
-        cached_method_name="get_push_rules_enabled_for_user",
-        list_name="user_ids",
-        num_args=1,
+        cached_method_name="get_push_rules_enabled_for_user", list_name="user_ids"
    )
    async def bulk_get_push_rules_enabled(
        self, user_ids: Collection[str]
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Olivier Wilkinson (reivilibre)	31900e1a3c	Cut out regex splits to get a full idea	2022-05-27 15:13:21 +01:00
Olivier Wilkinson (reivilibre)	e7b109b300	resolve linter arguments	2022-05-27 14:31:51 +01:00
Olivier Wilkinson (reivilibre)	54f3c50826	lints	2022-05-27 14:31:51 +01:00
Olivier Wilkinson (reivilibre)	e643227d3d	Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>	2022-05-27 14:31:51 +01:00
Olivier Wilkinson (reivilibre)	2102b5d4dc	SCARY HACKS	2022-05-27 13:20:43 +01:00
Olivier Wilkinson (reivilibre)	7c79d6c5e2	Add a primitive forking process 'manager' for Synapse workers	2022-05-27 10:45:51 +01:00
Olivier Wilkinson (reivilibre)	45eb81b194	Add mains	2022-05-27 10:26:07 +01:00
Olivier Wilkinson (reivilibre)	a2ae63e89c	Merge branch 'develop' into rei/complement_workers_in_ci	2022-05-26 15:49:57 +01:00
Olivier Wilkinson (reivilibre)	a974ccbdb0	Split workers' tests by regex into 4 different jobs for now	2022-05-26 15:08:19 +01:00
Patrick Cloke	49f06866e4	Remove backing code for groups/communities (#12558 ) Including handlers, configuration code, appservice support, and the GroupID construct.	2022-05-26 09:04:34 -04:00
dependabot[bot]	1cba285a79	Bump pyjwt from 2.3.0 to 2.4.0 (#12865 ) Bumps [pyjwt](https://github.com/jpadilla/pyjwt) from 2.3.0 to 2.4.0. - [Release notes](https://github.com/jpadilla/pyjwt/releases) - [Changelog](https://github.com/jpadilla/pyjwt/blob/master/CHANGELOG.rst) - [Commits](https://github.com/jpadilla/pyjwt/compare/2.3.0...2.4.0) --- updated-dependencies: - dependency-name: pyjwt dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>	2022-05-26 12:42:21 +00:00
reivilibre	e768644368	Fix ambiguous column name that would prevent use of MSC2716 History Import when using Postgres as a database. (#12843 )	2022-05-26 11:19:01 +00:00
Patrick Cloke	1885ee0113	Remove unstable APIs for /hierarchy. (#12851 ) Removes the unstable endpoint as well as a duplicated field which was modified during stabilization.	2022-05-26 07:10:28 -04:00
Patrick Cloke	b5707ceaba	Avoid attempting to delete push actions for remote users. (#12879 ) Remote users will never have push actions, so we can avoid a database round-trip/transaction completely.	2022-05-26 07:09:16 -04:00
Erik Johnston	b83bc5fab5	Pull out less state when handling gaps mk2 (#12852 )	2022-05-26 09:48:12 +00:00
Richard van der Hoff	1b338476af	Allow bigger responses to `/federation/v1/state` (#12877 ) * Refactor HTTP response size limits Rather than passing a separate `max_response_size` down the stack, make it an attribute of the `parser`. * Allow bigger responses on `federation/v1/state` `/state` can return huge responses, so we need to handle that.	2022-05-25 22:24:28 +01:00
Erik Johnston	4660d9fdcf	Fix up `state_store` naming (#12871 )	2022-05-25 12:59:04 +01:00
Patrick Cloke	a8db8c6eba	Remove user-visible groups/communities code (#12553 ) Makes it so that groups/communities no longer exist from a user-POV. E.g. we remove: * All API endpoints (including Client-Server, Server-Server, and admin). * Documented configuration options (and the experimental flag, which is now unused). * Special handling during room upgrades. * The `groups` section of the `/sync` response.	2022-05-25 07:53:40 -04:00
Patrick Cloke	759f9c09e1	Fix caching behavior for relations push rules. (#12859 ) By always returning all requested values from the function wrapped by cachedList. Otherwise implicit None values get added into the cache, which are unexpected.	2022-05-25 07:49:54 -04:00
Patrick Cloke	4cbcd4a999	Misc clean-up of push rules datastore (#12856 )	2022-05-25 07:49:12 -04:00
David Robertson	6aeee9a19d	Correct typo in changelog for #12858 .	2022-05-25 11:19:22 +01:00
Nick Mills-Barrett	1f9013ce60	Add the `batch_send` endpoint to generic workers (#12868 )	2022-05-25 09:51:07 +00:00
Nick Mills-Barrett	33e2916858	Don't create empty AS txns when the AS is down (#12869 )	2022-05-25 09:46:05 +00:00
Nick Mills-Barrett	2e5f88b5e6	Add the `/account/whoami` endpoint to generic workers (#12866 )	2022-05-25 10:41:41 +01:00
Nick Mills-Barrett	b4fab0b14f	Fix incorrect worker-allowed path in documentation (#12867 )	2022-05-25 09:20:34 +00:00
Carl Bordum Hansen	774ac4930d	Make sure `prev_ids` defaults to empty list (#12829 ) Signed-off-by: Carl Bordum Hansen <carl@bordum.dk>	2022-05-25 09:14:45 +00:00
Dirk Klimpel	298911555c	Fix typos in documentation (#12863 )	2022-05-25 10:14:03 +01:00
David Robertson	e7c77a8750	Correct annotation of `_iterate_over_text` (#12860 )	2022-05-24 18:17:21 +00:00
David Robertson	81d9f2a8e9	Fixes to MSC3787 implementation (#12858 )	2022-05-24 16:50:50 +00:00
Šimon Brandner	042e47970b	Remove `dont_notify` from the `.m.rule.room.server_acl` rule (#12849 ) Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>	2022-05-24 17:42:32 +01:00
Will Hunt	6855024e0a	Add authentication to thirdparty bridge APIs (#12746 ) Co-authored-by: Brendan Abolivier <babolivier@matrix.org>	2022-05-24 16:39:54 +02:00
Sean Quah	5d9f886aab	Merge tag 'v1.60.0rc1' into develop Synapse 1.60.0rc1 (2022-05-24) ============================== This release of Synapse adds a unique index to the `state_group_edges` table, in order to prevent accidentally introducing duplicate information (for example, because a database backup was restored multiple times). If your Synapse database already has duplicate rows in this table, this could fail with an error and require manual remediation. Additionally, the signature of the `check_event_for_spam` module callback has changed. The previous signature has been deprecated and remains working for now. Module authors should update their modules to use the new signature where possible. See [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1600) for more details. Features -------- - Measure the time taken in spam-checking callbacks and expose those measurements as metrics. ([\#12513](https://github.com/matrix-org/synapse/issues/12513)) - Add a `default_power_level_content_override` config option to set default room power levels per room preset. ([\#12618](https://github.com/matrix-org/synapse/issues/12618)) - Add support for [MSC3787: Allowing knocks to restricted rooms](https://github.com/matrix-org/matrix-spec-proposals/pull/3787). ([\#12623](https://github.com/matrix-org/synapse/issues/12623)) - Send `USER_IP` commands on a different Redis channel, in order to reduce traffic to workers that do not process these commands. ([\#12672](https://github.com/matrix-org/synapse/issues/12672), [\#12809](https://github.com/matrix-org/synapse/issues/12809)) - Synapse will now reload [cache config](https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html#caching) when it receives a [SIGHUP](https://en.wikipedia.org/wiki/SIGHUP) signal. ([\#12673](https://github.com/matrix-org/synapse/issues/12673)) - Add a config options to allow for auto-tuning of caches. ([\#12701](https://github.com/matrix-org/synapse/issues/12701)) - Update [MSC2716](https://github.com/matrix-org/matrix-spec-proposals/pull/2716) implementation to process marker events from the current state to avoid markers being lost in timeline gaps for federated servers which would cause the imported history to be undiscovered. ([\#12718](https://github.com/matrix-org/synapse/issues/12718)) - Add a `drop_federated_event` callback to `SpamChecker` to disregard inbound federated events before they take up much processing power, in an emergency. ([\#12744](https://github.com/matrix-org/synapse/issues/12744)) - Implement [MSC3818: Copy room type on upgrade](https://github.com/matrix-org/matrix-spec-proposals/pull/3818). ([\#12786](https://github.com/matrix-org/synapse/issues/12786), [\#12792](https://github.com/matrix-org/synapse/issues/12792)) - Update to the `check_event_for_spam` module callback. Deprecate the current callback signature, replace it with a new signature that is both less ambiguous (replacing booleans with explicit allow/block) and more powerful (ability to return explicit error codes). ([\#12808](https://github.com/matrix-org/synapse/issues/12808)) Bugfixes -------- - Fix a bug introduced in Synapse 1.7.0 that would prevent events from being sent to clients if there's a retention policy in the room when the support for retention policies is disabled. ([\#12611](https://github.com/matrix-org/synapse/issues/12611)) - Fix a bug introduced in Synapse 1.57.0 where `/messages` would throw a 500 error when querying for a non-existent room. ([\#12683](https://github.com/matrix-org/synapse/issues/12683)) - Add a unique index to `state_group_edges` to prevent duplicates being accidentally introduced and the consequential impact to performance. ([\#12687](https://github.com/matrix-org/synapse/issues/12687)) - Fix a long-standing bug where an empty room would be created when a user with an insufficient power level tried to upgrade a room. ([\#12696](https://github.com/matrix-org/synapse/issues/12696)) - Fix a bug introduced in Synapse 1.30.0 where empty rooms could be automatically created if a monthly active users limit is set. ([\#12713](https://github.com/matrix-org/synapse/issues/12713)) - Fix push to dismiss notifications when read on another client. Contributed by @SpiritCroc @ Beeper. ([\#12721](https://github.com/matrix-org/synapse/issues/12721)) - Fix poor database performance when reading the cache invalidation stream for large servers with lots of workers. ([\#12747](https://github.com/matrix-org/synapse/issues/12747)) - Delete events from the `federation_inbound_events_staging` table when a room is purged through the admin API. ([\#12770](https://github.com/matrix-org/synapse/issues/12770)) - Give a meaningful error message when a client tries to create a room with an invalid alias localpart. ([\#12779](https://github.com/matrix-org/synapse/issues/12779)) - Fix a bug introduced in 1.43.0 where a file (`providers.json`) was never closed. Contributed by @arkamar. ([\#12794](https://github.com/matrix-org/synapse/issues/12794)) - Fix a long-standing bug where finished log contexts would be re-started when failing to contact remote homeservers. ([\#12803](https://github.com/matrix-org/synapse/issues/12803)) - Fix a bug, introduced in Synapse 1.21.0, that led to media thumbnails being unusable before the index has been added in the background. ([\#12823](https://github.com/matrix-org/synapse/issues/12823)) Updates to the Docker image --------------------------- - Fix the docker file after a dependency update. ([\#12853](https://github.com/matrix-org/synapse/issues/12853)) Improved Documentation ---------------------- - Fix a typo in the Media Admin API documentation. ([\#12715](https://github.com/matrix-org/synapse/issues/12715)) - Update the OpenID Connect example for Keycloak to be compatible with newer versions of Keycloak. Contributed by @nhh. ([\#12727](https://github.com/matrix-org/synapse/issues/12727)) - Fix typo in server listener documentation. ([\#12742](https://github.com/matrix-org/synapse/issues/12742)) - Link to the configuration manual from the welcome page of the documentation. ([\#12748](https://github.com/matrix-org/synapse/issues/12748)) - Fix typo in `run_background_tasks_on` option name in configuration manual documentation. ([\#12749](https://github.com/matrix-org/synapse/issues/12749)) - Add information regarding the `rc_invites` ratelimiting option to the configuration docs. ([\#12759](https://github.com/matrix-org/synapse/issues/12759)) - Add documentation for cancellation of request processing. ([\#12761](https://github.com/matrix-org/synapse/issues/12761)) - Recommend using docker to run tests against postgres. ([\#12765](https://github.com/matrix-org/synapse/issues/12765)) - Add missing user directory endpoint from the generic worker documentation. Contributed by @olmari. ([\#12773](https://github.com/matrix-org/synapse/issues/12773)) - Add additional info to documentation of config option `cache_autotuning`. ([\#12776](https://github.com/matrix-org/synapse/issues/12776)) - Update configuration manual documentation to document size-related suffixes. ([\#12777](https://github.com/matrix-org/synapse/issues/12777)) - Fix invalid YAML syntax in the example documentation for the `url_preview_accept_language` config option. ([\#12785](https://github.com/matrix-org/synapse/issues/12785)) Deprecations and Removals ------------------------- - Require a body in POST requests to `/rooms/{roomId}/receipt/{receiptType}/{eventId}`, as required by the [Matrix specification](https://spec.matrix.org/v1.2/client-server-api/#post_matrixclientv3roomsroomidreceiptreceipttypeeventid). This breaks compatibility with Element Android 1.2.0 and earlier: users of those clients will be unable to send read receipts. ([\#12709](https://github.com/matrix-org/synapse/issues/12709)) Internal Changes ---------------- - Improve event caching mechanism to avoid having multiple copies of an event in memory at a time. ([\#10533](https://github.com/matrix-org/synapse/issues/10533)) - Preparation for faster-room-join work: return subsets of room state which we already have, immediately. ([\#12498](https://github.com/matrix-org/synapse/issues/12498)) - Add `@cancellable` decorator, for use on endpoint methods that can be cancelled when clients disconnect. ([\#12586](https://github.com/matrix-org/synapse/issues/12586), [\#12588](https://github.com/matrix-org/synapse/issues/12588), [\#12630](https://github.com/matrix-org/synapse/issues/12630), [\#12694](https://github.com/matrix-org/synapse/issues/12694), [\#12698](https://github.com/matrix-org/synapse/issues/12698), [\#12699](https://github.com/matrix-org/synapse/issues/12699), [\#12700](https://github.com/matrix-org/synapse/issues/12700), [\#12705](https://github.com/matrix-org/synapse/issues/12705)) - Enable cancellation of `GET /rooms/$room_id/members`, `GET /rooms/$room_id/state` and `GET /rooms/$room_id/state/$event_type/*` requests. ([\#12708](https://github.com/matrix-org/synapse/issues/12708)) - Improve documentation of the `synapse.push` module. ([\#12676](https://github.com/matrix-org/synapse/issues/12676)) - Refactor functions to on `PushRuleEvaluatorForEvent`. ([\#12677](https://github.com/matrix-org/synapse/issues/12677)) - Preparation for database schema simplifications: stop writing to `event_reference_hashes`. ([\#12679](https://github.com/matrix-org/synapse/issues/12679)) - Remove code which updates unused database column `application_services_state.last_txn`. ([\#12680](https://github.com/matrix-org/synapse/issues/12680)) - Refactor `EventContext` class. ([\#12689](https://github.com/matrix-org/synapse/issues/12689)) - Remove an unneeded class in the push code. ([\#12691](https://github.com/matrix-org/synapse/issues/12691)) - Consolidate parsing of relation information from events. ([\#12693](https://github.com/matrix-org/synapse/issues/12693)) - Convert namespace class `Codes` into a string enum. ([\#12703](https://github.com/matrix-org/synapse/issues/12703)) - Optimize private read receipt filtering. ([\#12711](https://github.com/matrix-org/synapse/issues/12711)) - Drop the logging level of status messages for the URL preview cache expiry job from INFO to DEBUG. ([\#12720](https://github.com/matrix-org/synapse/issues/12720)) - Downgrade some OIDC errors to warnings in the logs, to reduce the noise of Sentry reports. ([\#12723](https://github.com/matrix-org/synapse/issues/12723)) - Update configs used by Complement to allow more invites/3PID validations during tests. ([\#12731](https://github.com/matrix-org/synapse/issues/12731)) - Fix a long-standing bug where the user directory background process would fail to make forward progress if a user included a null codepoint in their display name or avatar. ([\#12762](https://github.com/matrix-org/synapse/issues/12762)) - Tweak the mypy plugin so that `@cached` can accept `on_invalidate=None`. ([\#12769](https://github.com/matrix-org/synapse/issues/12769)) - Move methods that call `add_push_rule` to the `PushRuleStore` class. ([\#12772](https://github.com/matrix-org/synapse/issues/12772)) - Make handling of federation Authorization header (more) compliant with RFC7230. ([\#12774](https://github.com/matrix-org/synapse/issues/12774)) - Refactor `resolve_state_groups_for_events` to not pull out full state when no state resolution happens. ([\#12775](https://github.com/matrix-org/synapse/issues/12775)) - Do not keep going if there are 5 back-to-back background update failures. ([\#12781](https://github.com/matrix-org/synapse/issues/12781)) - Fix federation when using the demo scripts. ([\#12783](https://github.com/matrix-org/synapse/issues/12783)) - The `hash_password` script now fails when it is called without specifying a config file. Contributed by @jae1911. ([\#12789](https://github.com/matrix-org/synapse/issues/12789)) - Improve and fix type hints. ([\#12567](https://github.com/matrix-org/synapse/issues/12567), [\#12477](https://github.com/matrix-org/synapse/issues/12477), [\#12717](https://github.com/matrix-org/synapse/issues/12717), [\#12753](https://github.com/matrix-org/synapse/issues/12753), [\#12695](https://github.com/matrix-org/synapse/issues/12695), [\#12734](https://github.com/matrix-org/synapse/issues/12734), [\#12716](https://github.com/matrix-org/synapse/issues/12716), [\#12726](https://github.com/matrix-org/synapse/issues/12726), [\#12790](https://github.com/matrix-org/synapse/issues/12790), [\#12833](https://github.com/matrix-org/synapse/issues/12833)) - Update EventContext `get_current_event_ids` and `get_prev_event_ids` to accept state filters and update calls where possible. ([\#12791](https://github.com/matrix-org/synapse/issues/12791)) - Remove Caddy from the Synapse workers image used in Complement. ([\#12818](https://github.com/matrix-org/synapse/issues/12818)) - Add Complement's shared registration secret to the Complement worker image. This fixes tests that depend on it. ([\#12819](https://github.com/matrix-org/synapse/issues/12819)) - Support registering Application Services when running with workers under Complement. ([\#12826](https://github.com/matrix-org/synapse/issues/12826)) - Disable 'faster room join' Complement tests when testing against Synapse with workers. ([\#12842](https://github.com/matrix-org/synapse/issues/12842))	2022-05-24 15:38:38 +01:00
Patrick Cloke	88ce3080d4	Experimental support for MSC3772 (#12740 ) Implements the following behind an experimental configuration flag: * A new push rule kind for mutually related events. * A new default push rule (`.m.rule.thread_reply`) under an unstable prefix. This is missing part of MSC3772: * The `.m.rule.thread_reply_to_me` push rule, this depends on MSC3664 / #11804.	2022-05-24 13:23:23 +00:00
Olivier Wilkinson (reivilibre)	322d22f04e	Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>	2022-05-20 17:39:49 +01:00
Olivier Wilkinson (reivilibre)	be69bd292a	Run Complement with workers in CI	2022-05-20 17:39:49 +01:00
				`@@ -0,0 +1 @@`
				`Remove support for the non-standard groups/communities feature from Synapse.`
				`@@ -0,0 +1 @@`
				`Experimental support for [MSC3772](https://github.com/matrix-org/matrix-spec-proposals/pull/3772): Push rule for mutually related events.`
				`@@ -0,0 +1 @@`
				Always send an `access_token` in `/thirdparty/` requests to appservices, as required by the [Matrix specification](https://spec.matrix.org/v1.1/application-service-api/#third-party-networks).
				`@@ -0,0 +1 @@`
				`Test Synapse against Complement with workers.`
				`@@ -0,0 +1 @@`
				`Fix a bug where we did not correctly handle invalid device list updates over federation. Contributed by Carl Bordum Hansen.`
				`@@ -0,0 +1 @@`
				Fix bug where servers using a Postgres database would fail to backfill from an insertion event when MSC2716 is enabled (`experimental_features.msc2716_enabled`).
				`@@ -0,0 +1 @@`
				Remove `dont_notify` from the `.m.rule.room.server_acl` rule.
				`@@ -0,0 +1 @@`
				Remove the unstable `/hierarchy` endpoint from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946).
				`@@ -0,0 +1 @@`
				`Pull out less state when handling gaps in room DAG.`