MSC4291 dev

Add recaptcha_{private,public}_key_path config option (#17984 )
Another config option on my quest to a `*_path` variant for every secret. Adds the config options `recaptcha_private_key_path` and `recaptcha_public_key_path`. Tests and docs are included. A public key is of course no secret, but it is closely related to the private key, so it’s still useful to have a `*_path` variant for it.
2025-07-14 15:17:52 -06:00 · 2025-07-14 11:37:36 -05:00 · 2025-07-14 16:55:19 +01:00 · 2025-07-11 13:26:59 +00:00 · 2025-07-11 12:54:46 +00:00 · 2025-07-11 12:34:37 +01:00
258 changed files with 17387 additions and 4711 deletions
@@ -5,7 +5,7 @@ name: Build docker images
 on:
  push:
    tags: ["v*"]
-    branches: [ master, main, develop ]
+    branches: [master, main, develop]
  workflow_dispatch:

 permissions:
@@ -14,23 +14,21 @@ permissions:
  id-token: write # needed for signing the images with GitHub OIDC Token
 jobs:
  build:
-    runs-on: ubuntu-22.04
+    name: Build and push image for ${{ matrix.platform }}
+    runs-on: ${{ matrix.runs_on }}
+    strategy:
+      matrix:
+        include:
+          - platform: linux/amd64
+            runs_on: ubuntu-24.04
+            suffix: linux-amd64
+          - platform: linux/arm64
+            runs_on: ubuntu-24.04-arm
+            suffix: linux-arm64
    steps:
-      - name: Set up QEMU
-        id: qemu
-        uses: docker/setup-qemu-action@29109295f81e9208d7d86ff1c6c12d2833863392 # v3.6.0
-        with:
-          platforms: arm64
-
      - name: Set up Docker Buildx
        id: buildx
-        uses: docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2 # v3.10.0
-
-      - name: Inspect builder
-        run: docker buildx inspect
-
-      - name: Install Cosign
-        uses: sigstore/cosign-installer@3454372f43399081ed03b604cb2d021dabca52bb # v3.8.2
+        uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1

      - name: Checkout repository
        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
@@ -55,13 +53,79 @@ jobs:
          username: ${{ github.repository_owner }}
          password: ${{ secrets.GITHUB_TOKEN }}

-      - name: Calculate docker image tag
-        id: set-tag
-        uses: docker/metadata-action@902fa8ec7d6ecbf8d84d538b9b233a880e428804 # v5.7.0
+      - name: Build and push by digest
+        id: build
+        uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
        with:
-          images: |
+          push: true
+          labels: |
+            gitsha1=${{ github.sha }}
+            org.opencontainers.image.version=${{ env.SYNAPSE_VERSION }}
+          tags: |
            docker.io/matrixdotorg/synapse
            ghcr.io/element-hq/synapse
+          file: "docker/Dockerfile"
+          platforms: ${{ matrix.platform }}
+          outputs: type=image,push-by-digest=true,name-canonical=true,push=true
+
+      - name: Export digest
+        run: |
+          mkdir -p ${{ runner.temp }}/digests
+          digest="${{ steps.build.outputs.digest }}"
+          touch "${{ runner.temp }}/digests/${digest#sha256:}"
+
+      - name: Upload digest
+        uses: actions/upload-artifact@v4
+        with:
+          name: digests-${{ matrix.suffix }}
+          path: ${{ runner.temp }}/digests/*
+          if-no-files-found: error
+          retention-days: 1
+
+  merge:
+    name: Push merged images to ${{ matrix.repository }}
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        repository:
+          - docker.io/matrixdotorg/synapse
+          - ghcr.io/element-hq/synapse
+
+    needs:
+      - build
+    steps:
+      - name: Download digests
+        uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
+        with:
+          path: ${{ runner.temp }}/digests
+          pattern: digests-*
+          merge-multiple: true
+
+      - name: Log in to DockerHub
+        uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
+        if: ${{ startsWith(matrix.repository, 'docker.io') }}
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+
+      - name: Log in to GHCR
+        uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
+        if: ${{ startsWith(matrix.repository, 'ghcr.io') }}
+        with:
+          registry: ghcr.io
+          username: ${{ github.repository_owner }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
+
+      - name: Install Cosign
+        uses: sigstore/cosign-installer@398d4b0eeef1380460a10c8013a76f728fb906ac # v3.9.1
+
+      - name: Calculate docker image tag
+        uses: docker/metadata-action@902fa8ec7d6ecbf8d84d538b9b233a880e428804 # v5.7.0
+        with:
+          images: ${{ matrix.repository }}
          flavor: |
            latest=false
          tags: |
@@ -69,31 +133,23 @@ jobs:
            type=raw,value=latest,enable=${{ github.ref == 'refs/heads/master' }}
            type=raw,value=latest,enable=${{ github.ref == 'refs/heads/main' }}
            type=pep440,pattern={{raw}}
+            type=sha

-      - name: Build and push all platforms
-        id: build-and-push
-        uses: docker/build-push-action@1dc73863535b631f98b2378be8619f83b136f4a0 # v6.17.0
-        with:
-          push: true
-          labels: |
-            gitsha1=${{ github.sha }}
-            org.opencontainers.image.version=${{ env.SYNAPSE_VERSION }}
-          tags: "${{ steps.set-tag.outputs.tags }}"
-          file: "docker/Dockerfile"
-          platforms: linux/amd64,linux/arm64
-
-          # arm64 builds OOM without the git fetch setting. c.f.
-          # https://github.com/rust-lang/cargo/issues/10583
-          build-args: |
-            CARGO_NET_GIT_FETCH_WITH_CLI=true
-
-      - name: Sign the images with GitHub OIDC Token
+      - name: Create manifest list and push
+        working-directory: ${{ runner.temp }}/digests
        env:
-          DIGEST: ${{ steps.build-and-push.outputs.digest }}
-          TAGS: ${{ steps.set-tag.outputs.tags }}
+          REPOSITORY: ${{ matrix.repository }}
        run: |
-          images=""
-          for tag in ${TAGS}; do
-            images+="${tag}@${DIGEST} "
+          docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
+            $(printf "$REPOSITORY@sha256:%s " *)
+
+      - name: Sign each manifest
+        env:
+          REPOSITORY: ${{ matrix.repository }}
+        run: |
+          DIGESTS=""
+          for TAG in $(echo "$DOCKER_METADATA_OUTPUT_JSON" | jq -r '.tags[]'); do
+            DIGEST="$(docker buildx imagetools inspect $TAG --format '{{json .Manifest}}' | jq -r '.digest')"
+            DIGESTS="$DIGESTS $REPOSITORY@$DIGEST"
          done
-          cosign sign --yes ${images}
+          cosign sign --yes $DIGESTS
@@ -14,7 +14,7 @@ jobs:
      # There's a 'download artifact' action, but it hasn't been updated for the workflow_run action
      # (https://github.com/actions/download-artifact/issues/60) so instead we get this mess:
      - name: 📥 Download artifact
-        uses: dawidd6/action-download-artifact@07ab29fd4a977ae4d2b275087cf67563dfdf0295 # v9
+        uses: dawidd6/action-download-artifact@ac66b43f0e6a346234dd65d4d0c8fbb31cb316e5 # v11
        with:
          workflow: docs-pr.yaml
          run_id: ${{ github.event.workflow_run.id }}
@@ -78,6 +78,18 @@ jobs:
          mdbook build
          cp book/welcome_and_overview.html book/index.html

+      - name: Prepare and publish schema files
+        run: |
+          sudo apt-get update && sudo apt-get install -y yq
+          mkdir -p book/schema
+          # Remove developer notice before publishing.
+          rm schema/v*/Do\ not\ edit\ files\ in\ this\ folder
+          # Copy schema files that are independent from current Synapse version.
+          cp -r -t book/schema schema/v*/
+          # Convert config schema from YAML source file to JSON.
+          yq < schema/synapse-config.schema.yaml \
+            > book/schema/synapse-config.schema.json
+
      # Deploy to the target directory.
      - name: Deploy to gh pages
        uses: peaceiris/actions-gh-pages@4f9cc6602d3f66b9c108549d475ec49e8ef4d45e # v4.0.0
@@ -6,6 +6,11 @@ name: Attempt to automatically fix linting errors
 on:
  workflow_dispatch:

+env:
+  # We use nightly so that `fmt` correctly groups together imports, and
+  # clippy correctly fixes up the benchmarks.
+  RUST_VERSION: nightly-2025-06-24
+
 jobs:
  fixup:
    name: Fix up
@@ -16,13 +21,11 @@ jobs:
        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@56f84321dbccf38fb67ce29ab63e4754056677e0 # master (rust 1.85.1)
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
        with:
-          # We use nightly so that `fmt` correctly groups together imports, and
-          # clippy correctly fixes up the benchmarks.
-          toolchain: nightly-2022-12-01
+          toolchain: ${{ env.RUST_VERSION }}
          components: clippy, rustfmt
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

      - name: Setup Poetry
        uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
@@ -44,6 +47,6 @@ jobs:
      - run: cargo fmt
        continue-on-error: true

-      - uses: stefanzweifel/git-auto-commit-action@b863ae1933cb653a53c021fe36dbb774e1fb9403 # v5.2.0
+      - uses: stefanzweifel/git-auto-commit-action@778341af668090896ca464160c2def5d1d1a3eb0 # v6.0.1
        with:
          commit_message: "Attempt to fix linting"
@@ -21,6 +21,9 @@ concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

+env:
+  RUST_VERSION: 1.87.0
+
 jobs:
  check_repo:
    # Prevent this workflow from running on any fork of Synapse other than element-hq/synapse, as it is
@@ -41,8 +44,10 @@ jobs:
    steps:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
      - name: Install Rust
-        uses: dtolnay/rust-toolchain@fcf085fcb4b4b8f63f96906cd713eb52181b5ea4 # stable (rust 1.85.1)
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
+        with:
+          toolchain: ${{ env.RUST_VERSION }}
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

      # The dev dependencies aren't exposed in the wheel metadata (at least with current
      # poetry-core versions), so we install with poetry.
@@ -55,7 +60,7 @@ jobs:
      - run: poetry run pip list > before.txt
      # Upgrade all runtime dependencies only. This is intended to mimic a fresh
      # `pip install matrix-synapse[all]` as closely as possible.
-      - run: poetry update --no-dev
+      - run: poetry update --without dev
      - run: poetry run pip list > after.txt && (diff -u before.txt after.txt || true)
      - name: Remove unhelpful options from mypy config
        run: sed -e '/warn_unused_ignores = True/d' -e '/warn_redundant_casts = True/d' -i mypy.ini
@@ -75,8 +80,10 @@ jobs:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@fcf085fcb4b4b8f63f96906cd713eb52181b5ea4 # stable (rust 1.85.1)
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
+        with:
+          toolchain: ${{ env.RUST_VERSION }}
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

      - run: sudo apt-get -qq install xmlsec1
      - name: Set up PostgreSQL ${{ matrix.postgres-version }}
@@ -148,8 +155,10 @@ jobs:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@fcf085fcb4b4b8f63f96906cd713eb52181b5ea4 # stable (rust 1.85.1)
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
+        with:
+          toolchain: ${{ env.RUST_VERSION }}
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

      - name: Ensure sytest runs `pip install`
        # Delete the lockfile so sytest will `pip install` rather than `poetry install`
@@ -30,7 +30,7 @@ jobs:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
      - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
        with:
-          python-version: '3.x'
+          python-version: "3.x"
      - id: set-distros
        run: |
          # if we're running from a tag, get the full list of distros; otherwise just use debian:sid
@@ -61,7 +61,7 @@ jobs:

      - name: Set up Docker Buildx
        id: buildx
-        uses: docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2 # v3.10.0
+        uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
        with:
          install: true

@@ -76,7 +76,7 @@ jobs:
      - name: Set up python
        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
        with:
-          python-version: '3.x'
+          python-version: "3.x"

      - name: Build the packages
        # see https://github.com/docker/build-push-action/issues/252
@@ -107,12 +107,15 @@ jobs:
          path: debs/*

  build-wheels:
-    name: Build wheels on ${{ matrix.os }} for ${{ matrix.arch }}
+    name: Build wheels on ${{ matrix.os }}
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
-        os: [ubuntu-22.04, macos-13]
-        arch: [x86_64, aarch64]
+        os:
+          - ubuntu-24.04
+          - ubuntu-24.04-arm
+          - macos-13 # This uses x86-64
+          - macos-14 # This uses arm64
        # is_pr is a flag used to exclude certain jobs from the matrix on PRs.
        # It is not read by the rest of the workflow.
        is_pr:
@@ -122,12 +125,11 @@ jobs:
          # Don't build macos wheels on PR CI.
          - is_pr: true
            os: "macos-13"
-          # Don't build aarch64 wheels on mac.
-          - os: "macos-13"
-            arch: aarch64
+          - is_pr: true
+            os: "macos-14"
          # Don't build aarch64 wheels on PR CI.
          - is_pr: true
-            arch: aarch64
+            os: "ubuntu-24.04-arm"

    steps:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
@@ -139,21 +141,11 @@ jobs:
          python-version: "3.x"

      - name: Install cibuildwheel
-        run: python -m pip install cibuildwheel==2.23.0
-
-      - name: Set up QEMU to emulate aarch64
-        if: matrix.arch == 'aarch64'
-        uses: docker/setup-qemu-action@29109295f81e9208d7d86ff1c6c12d2833863392 # v3.6.0
-        with:
-          platforms: arm64
-
-      - name: Build aarch64 wheels
-        if: matrix.arch == 'aarch64'
-        run: echo 'CIBW_ARCHS_LINUX=aarch64' >> $GITHUB_ENV
+        run: python -m pip install cibuildwheel==3.0.0

      - name: Only build a single wheel on PR
        if: startsWith(github.ref, 'refs/pull/')
-        run: echo "CIBW_BUILD="cp39-manylinux_${{ matrix.arch }}"" >> $GITHUB_ENV
+        run: echo "CIBW_BUILD="cp39-manylinux_*"" >> $GITHUB_ENV

      - name: Build wheels
        run: python -m cibuildwheel --output-dir wheelhouse
@@ -161,13 +153,10 @@ jobs:
          # Skip testing for platforms which various libraries don't have wheels
          # for, and so need extra build deps.
          CIBW_TEST_SKIP: pp3*-* *i686* *musl*
-          # Fix Rust OOM errors on emulated aarch64: https://github.com/rust-lang/cargo/issues/10583
-          CARGO_NET_GIT_FETCH_WITH_CLI: true
-          CIBW_ENVIRONMENT_PASS_LINUX: CARGO_NET_GIT_FETCH_WITH_CLI

      - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
        with:
-          name: Wheel-${{ matrix.os }}-${{ matrix.arch }}
+          name: Wheel-${{ matrix.os }}
          path: ./wheelhouse/*.whl

  build-sdist:
@@ -179,7 +168,7 @@ jobs:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
      - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
        with:
-          python-version: '3.10'
+          python-version: "3.10"

      - run: pip install build

@@ -191,7 +180,6 @@ jobs:
          name: Sdist
          path: dist/*.tar.gz

-
  # if it's a tag, create a release and attach the artifacts to it
  attach-assets:
    name: "Attach assets to release"
@@ -0,0 +1,57 @@
+name: Schema
+
+on:
+  pull_request:
+    paths:
+      - schema/**
+      - docs/usage/configuration/config_documentation.md
+  push:
+    branches: ["develop", "release-*"]
+  workflow_dispatch:
+
+jobs:
+  validate-schema:
+    name: Ensure Synapse config schema is valid
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
+        with:
+          python-version: "3.x"
+      - name: Install check-jsonschema
+        run: pip install check-jsonschema==0.33.0
+
+      - name: Validate meta schema
+        run: check-jsonschema --check-metaschema schema/v*/meta.schema.json
+      - name: Validate schema
+        run: |-
+          # Please bump on introduction of a new meta schema.
+          LATEST_META_SCHEMA_VERSION=v1
+          check-jsonschema \
+            --schemafile="schema/$LATEST_META_SCHEMA_VERSION/meta.schema.json" \
+            schema/synapse-config.schema.yaml
+      - name: Validate default config
+      # Populates the empty instance with default values and checks against the schema.
+        run: |-
+          echo "{}" | check-jsonschema \
+            --fill-defaults --schemafile=schema/synapse-config.schema.yaml -
+
+  check-doc-generation:
+    name: Ensure generated documentation is up-to-date
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
+        with:
+          python-version: "3.x"
+      - name: Install PyYAML
+        run: pip install PyYAML==6.0.2
+
+      - name: Regenerate config documentation
+        run: |
+          scripts-dev/gen_config_documentation.py \
+            schema/synapse-config.schema.yaml \
+          > docs/usage/configuration/config_documentation.md
+      - name: Error in case of any differences
+      # Errors if there are now any modified files (untracked files are ignored).
+        run: 'git diff --exit-code'
@@ -11,6 +11,9 @@ concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

+env:
+  RUST_VERSION: 1.87.0
+
 jobs:
  # Job to detect what has changed so we don't run e.g. Rust checks on PRs that
  # don't modify Rust code.
@@ -85,8 +88,10 @@ jobs:
    steps:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
      - name: Install Rust
-        uses: dtolnay/rust-toolchain@e05ebb0e73db581a4877c6ce762e29fe1e0b5073 # 1.66.0
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
+        with:
+          toolchain: ${{ env.RUST_VERSION }}
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
      - uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
        with:
          python-version: "3.x"
@@ -149,8 +154,10 @@ jobs:
        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@e05ebb0e73db581a4877c6ce762e29fe1e0b5073 # 1.66.0
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
+        with:
+          toolchain: ${{ env.RUST_VERSION }}
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

      - name: Setup Poetry
        uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
@@ -210,8 +217,10 @@ jobs:
        with:
          ref: ${{ github.event.pull_request.head.sha }}
      - name: Install Rust
-        uses: dtolnay/rust-toolchain@e05ebb0e73db581a4877c6ce762e29fe1e0b5073 # 1.66.0
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
+        with:
+          toolchain: ${{ env.RUST_VERSION }}
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
      - uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
        with:
          poetry-version: "2.1.1"
@@ -227,10 +236,11 @@ jobs:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@e05ebb0e73db581a4877c6ce762e29fe1e0b5073 # 1.66.0
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
        with:
            components: clippy
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+            toolchain: ${{ env.RUST_VERSION }}
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

      - run: cargo clippy -- -D warnings

@@ -245,11 +255,11 @@ jobs:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@56f84321dbccf38fb67ce29ab63e4754056677e0 # master (rust 1.85.1)
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
        with:
-            toolchain: nightly-2022-12-01
+            toolchain: nightly-2025-04-23
            components: clippy
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

      - run: cargo clippy --all-features -- -D warnings

@@ -262,12 +272,12 @@ jobs:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@56f84321dbccf38fb67ce29ab63e4754056677e0 # master (rust 1.85.1)
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
        with:
          # We use nightly so that it correctly groups together imports
-          toolchain: nightly-2022-12-01
+          toolchain: nightly-2025-04-23
          components: rustfmt
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

      - run: cargo fmt --check

@@ -362,8 +372,10 @@ jobs:
            postgres:${{ matrix.job.postgres-version }}

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@e05ebb0e73db581a4877c6ce762e29fe1e0b5073 # 1.66.0
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
+        with:
+          toolchain: ${{ env.RUST_VERSION }}
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

      - uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
        with:
@@ -404,8 +416,10 @@ jobs:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@e05ebb0e73db581a4877c6ce762e29fe1e0b5073 # 1.66.0
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
+        with:
+          toolchain: ${{ env.RUST_VERSION }}
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

      # There aren't wheels for some of the older deps, so we need to install
      # their build dependencies
@@ -519,8 +533,10 @@ jobs:
        run: cat sytest-blacklist .ci/worker-blacklist > synapse-blacklist-with-workers

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@e05ebb0e73db581a4877c6ce762e29fe1e0b5073 # 1.66.0
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
+        with:
+          toolchain: ${{ env.RUST_VERSION }}
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

      - name: Run SyTest
        run: /bootstrap.sh synapse
@@ -663,8 +679,10 @@ jobs:
          path: synapse

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@e05ebb0e73db581a4877c6ce762e29fe1e0b5073 # 1.66.0
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
+        with:
+          toolchain: ${{ env.RUST_VERSION }}
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

      - name: Prepare Complement's Prerequisites
        run: synapse/.ci/scripts/setup_complement_prerequisites.sh
@@ -695,8 +713,10 @@ jobs:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@e05ebb0e73db581a4877c6ce762e29fe1e0b5073 # 1.66.0
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
+        with:
+          toolchain: ${{ env.RUST_VERSION }}
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

      - run: cargo test

@@ -713,10 +733,10 @@ jobs:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@56f84321dbccf38fb67ce29ab63e4754056677e0 # master (rust 1.85.1)
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
        with:
            toolchain: nightly-2022-12-01
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

      - run: cargo bench --no-run

@@ -20,6 +20,9 @@ concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

+env:
+  RUST_VERSION: 1.87.0
+
 jobs:
  check_repo:
    # Prevent this workflow from running on any fork of Synapse other than element-hq/synapse, as it is
@@ -43,8 +46,10 @@ jobs:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@fcf085fcb4b4b8f63f96906cd713eb52181b5ea4 # stable (rust 1.85.1)
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
+        with:
+          toolchain: ${{ env.RUST_VERSION }}
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

      - uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
        with:
@@ -69,8 +74,10 @@ jobs:
      - run: sudo apt-get -qq install xmlsec1

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@fcf085fcb4b4b8f63f96906cd713eb52181b5ea4 # stable (rust 1.85.1)
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
+        with:
+          toolchain: ${{ env.RUST_VERSION }}
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

      - uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
        with:
@@ -113,8 +120,10 @@ jobs:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@fcf085fcb4b4b8f63f96906cd713eb52181b5ea4 # stable (rust 1.85.1)
-      - uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
+        uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # master
+        with:
+          toolchain: ${{ env.RUST_VERSION }}
+      - uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

      - name: Patch dependencies
        # Note: The poetry commands want to create a virtualenv in /src/.venv/,
@@ -47,6 +47,7 @@ __pycache__/
 /.idea/
 /.ropeproject/
 /.vscode/
+/.zed/

 # build products
 !/.coveragerc
@@ -1,3 +1,185 @@
+# Synapse 1.134.0rc1 (2025-07-09)
+
+### Features
+
+- Support for [MSC4235](https://github.com/matrix-org/matrix-spec-proposals/pull/4235): `via` query param for hierarchy endpoint. Contributed by Krishan (@kfiven). ([\#18070](https://github.com/element-hq/synapse/issues/18070))
+- Add `forget_forced_upon_leave` capability as per [MSC4267](https://github.com/matrix-org/matrix-spec-proposals/pull/4267). ([\#18196](https://github.com/element-hq/synapse/issues/18196))
+- Add `federated_user_may_invite` spam checker callback which receives the entire invite event. Contributed by @tulir @ Beeper. ([\#18241](https://github.com/element-hq/synapse/issues/18241))
+
+### Bugfixes
+
+- Fix `KeyError` on background updates when using split main/state databases. ([\#18509](https://github.com/element-hq/synapse/issues/18509))
+- Improve performance of device deletion by adding missing index. ([\#18582](https://github.com/element-hq/synapse/issues/18582))
+- Fix `avatar_url` and `displayname` being sent on federation profile queries when they are not set. ([\#18593](https://github.com/element-hq/synapse/issues/18593))
+- Respond with 401 & `M_USER_LOCKED` when a locked user calls `POST /login`, as per the spec. ([\#18594](https://github.com/element-hq/synapse/issues/18594))
+- Ensure policy servers are not asked to scan policy server change events, allowing rooms to disable the use of a policy server while the policy server is down. ([\#18605](https://github.com/element-hq/synapse/issues/18605))
+
+### Improved Documentation
+
+- Fix documentation of the Delete Room Admin API's status field. ([\#18519](https://github.com/element-hq/synapse/issues/18519))
+
+### Deprecations and Removals
+
+- Stop adding the "origin" field to newly-created events (PDUs). ([\#18418](https://github.com/element-hq/synapse/issues/18418))
+
+### Internal Changes
+
+- Replace `PyICU` crate with equivalent `icu_segmenter` Rust crate. ([\#18553](https://github.com/element-hq/synapse/issues/18553), [\#18646](https://github.com/element-hq/synapse/issues/18646))
+- Improve docstring on `simple_upsert_many`. ([\#18573](https://github.com/element-hq/synapse/issues/18573))
+- Raise poetry-core version cap to 2.1.3. ([\#18575](https://github.com/element-hq/synapse/issues/18575))
+- Raise setuptools_rust version cap to 1.11.1. ([\#18576](https://github.com/element-hq/synapse/issues/18576))
+- Better handling of ratelimited requests. ([\#18595](https://github.com/element-hq/synapse/issues/18595), [\#18600](https://github.com/element-hq/synapse/issues/18600))
+- Update to Rust 1.87.0 in CI, and bump the pinned commit of the `dtolnay/rust-toolchain` GitHub Action to `b3b07ba8b418998c39fb20f53e8b695cdcc8de1b`. ([\#18596](https://github.com/element-hq/synapse/issues/18596))
+- Speed up bulk device deletion. ([\#18602](https://github.com/element-hq/synapse/issues/18602))
+- Speed up the building of arm-based wheels in CI. ([\#18618](https://github.com/element-hq/synapse/issues/18618))
+- Speed up the building of Docker images in CI. ([\#18620](https://github.com/element-hq/synapse/issues/18620))
+- Add `.zed/` directory to `.gitignore`. ([\#18623](https://github.com/element-hq/synapse/issues/18623))
+- Log the room ID we're purging state for. ([\#18625](https://github.com/element-hq/synapse/issues/18625))
+
+
+
+### Updates to locked dependencies
+
+* Bump Swatinem/rust-cache from 2.7.8 to 2.8.0. ([\#18612](https://github.com/element-hq/synapse/issues/18612))
+* Bump attrs from 24.2.0 to 25.3.0. ([\#18649](https://github.com/element-hq/synapse/issues/18649))
+* Bump authlib from 1.5.2 to 1.6.0. ([\#18642](https://github.com/element-hq/synapse/issues/18642))
+* Bump base64 from 0.21.7 to 0.22.1. ([\#18589](https://github.com/element-hq/synapse/issues/18589))
+* Bump base64 from 0.21.7 to 0.22.1. ([\#18629](https://github.com/element-hq/synapse/issues/18629))
+* Bump docker/build-push-action from 6.17.0 to 6.18.0. ([\#18497](https://github.com/element-hq/synapse/issues/18497))
+* Bump docker/setup-buildx-action from 3.10.0 to 3.11.1. ([\#18587](https://github.com/element-hq/synapse/issues/18587))
+* Bump hiredis from 3.1.0 to 3.2.1. ([\#18638](https://github.com/element-hq/synapse/issues/18638))
+* Bump ijson from 3.3.0 to 3.4.0. ([\#18650](https://github.com/element-hq/synapse/issues/18650))
+* Bump jsonschema from 4.23.0 to 4.24.0. ([\#18630](https://github.com/element-hq/synapse/issues/18630))
+* Bump msgpack from 1.1.0 to 1.1.1. ([\#18651](https://github.com/element-hq/synapse/issues/18651))
+* Bump mypy-zope from 1.0.11 to 1.0.12. ([\#18640](https://github.com/element-hq/synapse/issues/18640))
+* Bump phonenumbers from 9.0.2 to 9.0.8. ([\#18652](https://github.com/element-hq/synapse/issues/18652))
+* Bump pillow from 11.2.1 to 11.3.0. ([\#18624](https://github.com/element-hq/synapse/issues/18624))
+* Bump prometheus-client from 0.21.0 to 0.22.1. ([\#18609](https://github.com/element-hq/synapse/issues/18609))
+* Bump pyasn1-modules from 0.4.1 to 0.4.2. ([\#18495](https://github.com/element-hq/synapse/issues/18495))
+* Bump pydantic from 2.11.4 to 2.11.7. ([\#18639](https://github.com/element-hq/synapse/issues/18639))
+* Bump reqwest from 0.12.15 to 0.12.20. ([\#18590](https://github.com/element-hq/synapse/issues/18590))
+* Bump reqwest from 0.12.20 to 0.12.22. ([\#18627](https://github.com/element-hq/synapse/issues/18627))
+* Bump ruff from 0.11.11 to 0.12.1. ([\#18645](https://github.com/element-hq/synapse/issues/18645))
+* Bump ruff from 0.12.1 to 0.12.2. ([\#18657](https://github.com/element-hq/synapse/issues/18657))
+* Bump sentry-sdk from 2.22.0 to 2.32.0. ([\#18633](https://github.com/element-hq/synapse/issues/18633))
+* Bump setuptools-rust from 1.10.2 to 1.11.1. ([\#18655](https://github.com/element-hq/synapse/issues/18655))
+* Bump sigstore/cosign-installer from 3.8.2 to 3.9.0. ([\#18588](https://github.com/element-hq/synapse/issues/18588))
+* Bump sigstore/cosign-installer from 3.9.0 to 3.9.1. ([\#18608](https://github.com/element-hq/synapse/issues/18608))
+* Bump stefanzweifel/git-auto-commit-action from 5.2.0 to 6.0.1. ([\#18607](https://github.com/element-hq/synapse/issues/18607))
+* Bump tokio from 1.45.1 to 1.46.0. ([\#18628](https://github.com/element-hq/synapse/issues/18628))
+* Bump tokio from 1.46.0 to 1.46.1. ([\#18667](https://github.com/element-hq/synapse/issues/18667))
+* Bump treq from 24.9.1 to 25.5.0. ([\#18610](https://github.com/element-hq/synapse/issues/18610))
+* Bump types-bleach from 6.2.0.20241123 to 6.2.0.20250514. ([\#18634](https://github.com/element-hq/synapse/issues/18634))
+* Bump types-jsonschema from 4.23.0.20250516 to 4.24.0.20250528. ([\#18611](https://github.com/element-hq/synapse/issues/18611))
+* Bump types-opentracing from 2.4.10.6 to 2.4.10.20250622. ([\#18586](https://github.com/element-hq/synapse/issues/18586))
+* Bump types-psycopg2 from 2.9.21.20250318 to 2.9.21.20250516. ([\#18658](https://github.com/element-hq/synapse/issues/18658))
+* Bump types-pyyaml from 6.0.12.20241230 to 6.0.12.20250516. ([\#18643](https://github.com/element-hq/synapse/issues/18643))
+* Bump types-setuptools from 75.2.0.20241019 to 80.9.0.20250529. ([\#18644](https://github.com/element-hq/synapse/issues/18644))
+* Bump typing-extensions from 4.12.2 to 4.14.0. ([\#18654](https://github.com/element-hq/synapse/issues/18654))
+* Bump typing-extensions from 4.14.0 to 4.14.1. ([\#18668](https://github.com/element-hq/synapse/issues/18668))
+* Bump urllib3 from 2.2.2 to 2.5.0. ([\#18572](https://github.com/element-hq/synapse/issues/18572))
+
+# Synapse 1.133.0 (2025-07-01)
+
+Pre-built wheels are now built using the [manylinux_2_28](https://github.com/pypa/manylinux#manylinux_2_28-almalinux-8-based) base, which is expected to be compatible with distros using glibc 2.28 or later, including:
+
+ - Debian 10+
+ - Ubuntu 18.10+
+ - Fedora 29+
+ - CentOS/RHEL 8+
+
+Previously, wheels were built using the [manylinux2014](https://github.com/pypa/manylinux#manylinux2014-centos-7-based-glibc-217) base, which was expected to be compatible with distros using glibc 2.17 or later.
+
+### Bugfixes
+
+- Bump `cibuildwheel` to 3.0.0 to fix the `manylinux` wheel builds. ([\#18615](https://github.com/element-hq/synapse/issues/18615))
+
+
+
+
+# Synapse 1.133.0rc1 (2025-06-24)
+
+### Features
+
+- Add support for the [MSC4260 user report API](https://github.com/matrix-org/matrix-spec-proposals/pull/4260). ([\#18120](https://github.com/element-hq/synapse/issues/18120))
+
+### Bugfixes
+
+- Fix an issue where, during state resolution for v11 rooms, Synapse would incorrectly calculate the power level of the creator when there was no power levels event in the room. ([\#18534](https://github.com/element-hq/synapse/issues/18534), [\#18547](https://github.com/element-hq/synapse/issues/18547))
+- Fix long-standing bug where sliding sync did not honour the `room_id_to_include` config option. ([\#18535](https://github.com/element-hq/synapse/issues/18535))
+- Fix an issue where "Lock timeout is getting excessive" warnings would be logged even when the lock timeout was <10 minutes. ([\#18543](https://github.com/element-hq/synapse/issues/18543))
+- Fix an issue where Synapse could calculate the wrong power level for the creator of the room if there was no power levels event. ([\#18545](https://github.com/element-hq/synapse/issues/18545))
+
+### Improved Documentation
+
+- Generate config documentation from JSON Schema file. ([\#18528](https://github.com/element-hq/synapse/issues/18528))
+- Fix typo in user type documentation. ([\#18568](https://github.com/element-hq/synapse/issues/18568))
+
+### Internal Changes
+
+- Increase performance of introspecting access tokens when using delegated auth. ([\#18357](https://github.com/element-hq/synapse/issues/18357), [\#18561](https://github.com/element-hq/synapse/issues/18561))
+- Log user deactivations. ([\#18541](https://github.com/element-hq/synapse/issues/18541))
+- Enable [`flake8-logging`](https://docs.astral.sh/ruff/rules/#flake8-logging-log) and [`flake8-logging-format`](https://docs.astral.sh/ruff/rules/#flake8-logging-format-g) rules in Ruff and fix related issues throughout the codebase. ([\#18542](https://github.com/element-hq/synapse/issues/18542))
+- Clean up old, unused rows from the `device_federation_inbox` table. ([\#18546](https://github.com/element-hq/synapse/issues/18546))
+- Run config schema CI on develop and release branches. ([\#18551](https://github.com/element-hq/synapse/issues/18551))
+- Add support for Twisted `25.5.0`+ releases. ([\#18577](https://github.com/element-hq/synapse/issues/18577))
+- Update PyO3 to version 0.25. ([\#18578](https://github.com/element-hq/synapse/issues/18578))
+
+
+
+### Updates to locked dependencies
+
+* Bump actions/setup-python from 5.5.0 to 5.6.0. ([\#18555](https://github.com/element-hq/synapse/issues/18555))
+* Bump base64 from 0.21.7 to 0.22.1. ([\#18559](https://github.com/element-hq/synapse/issues/18559))
+* Bump dawidd6/action-download-artifact from 9 to 11. ([\#18556](https://github.com/element-hq/synapse/issues/18556))
+* Bump headers from 0.4.0 to 0.4.1. ([\#18529](https://github.com/element-hq/synapse/issues/18529))
+* Bump requests from 2.32.2 to 2.32.4. ([\#18533](https://github.com/element-hq/synapse/issues/18533))
+* Bump types-requests from 2.32.0.20250328 to 2.32.4.20250611. ([\#18558](https://github.com/element-hq/synapse/issues/18558))
+
+# Synapse 1.132.0 (2025-06-17)
+
+### Improved Documentation
+
+- Improvements to generate config documentation from JSON Schema file. ([\#18522](https://github.com/element-hq/synapse/issues/18522))
+
+
+
+
+# Synapse 1.132.0rc1 (2025-06-10)
+
+### Features
+
+- Add support for [MSC4155](https://github.com/matrix-org/matrix-spec-proposals/pull/4155) Invite Filtering. ([\#18288](https://github.com/element-hq/synapse/issues/18288))
+- Add experimental `user_may_send_state_event` module API callback. ([\#18455](https://github.com/element-hq/synapse/issues/18455))
+- Add experimental `get_media_config_for_user` and `is_user_allowed_to_upload_media_of_size` module API callbacks that allow overriding of media repository maximum upload size. ([\#18457](https://github.com/element-hq/synapse/issues/18457))
+- Add experimental `get_ratelimit_override_for_user` module API callback that allows overriding of per-user ratelimits. ([\#18458](https://github.com/element-hq/synapse/issues/18458))
+- Pass `room_config` argument to `user_may_create_room` spam checker module callback. ([\#18486](https://github.com/element-hq/synapse/issues/18486))
+- Support configuration of default and extra user types. ([\#18456](https://github.com/element-hq/synapse/issues/18456))
+- Successful requests to `/_matrix/app/v1/ping` will now force Synapse to reattempt delivering transactions to appservices. ([\#18521](https://github.com/element-hq/synapse/issues/18521))
+- Support the import of the `RatelimitOverride` type from `synapse.module_api` in modules and rename `messages_per_second` to `per_second`. ([\#18513](https://github.com/element-hq/synapse/issues/18513))
+
+### Bugfixes
+
+- Remove destinations from sending if not whitelisted. ([\#18484](https://github.com/element-hq/synapse/issues/18484))
+- Fixed room summary API incorrectly returning that a room is private in the room summary response when the join rule is omitted by the remote server. Contributed by @nexy7574. ([\#18493](https://github.com/element-hq/synapse/issues/18493))
+- Prevent users from adding themselves to their own user ignore list. ([\#18508](https://github.com/element-hq/synapse/issues/18508))
+
+### Improved Documentation
+
+- Generate config documentation from JSON Schema file. ([\#17892](https://github.com/element-hq/synapse/issues/17892))
+- Mention `CAP_NET_BIND_SERVICE` as an alternative to running Synapse as root in order to bind to a privileged port. ([\#18408](https://github.com/element-hq/synapse/issues/18408))
+- Surface hidden Admin API documentation regarding fetching of scheduled tasks. ([\#18516](https://github.com/element-hq/synapse/issues/18516))
+- Mark the new module APIs in this release as experimental. ([\#18536](https://github.com/element-hq/synapse/issues/18536))
+
+### Internal Changes
+
+- Mark dehydrated devices in the [List All User Devices Admin API](https://element-hq.github.io/synapse/latest/admin_api/user_admin_api.html#list-all-devices). ([\#18252](https://github.com/element-hq/synapse/issues/18252))
+- Reduce disk wastage by cleaning up `received_transactions` older than 1 day, rather than 30 days. ([\#18310](https://github.com/element-hq/synapse/issues/18310))
+- Distinguish all vs local events being persisted in the "Event Send Time Quantiles" graph (Grafana). ([\#18510](https://github.com/element-hq/synapse/issues/18510))
+
+
+
+
 # Synapse 1.131.0 (2025-06-03)

 No significant changes since 1.131.0rc1.
@@ -0,0 +1 @@
+Add `recaptcha_private_key_path` and `recaptcha_public_key_path` config option.
@@ -0,0 +1 @@
+Add plain-text handling for rich-text topics as per [MSC3765](https://github.com/matrix-org/matrix-spec-proposals/pull/3765).
@@ -0,0 +1 @@
+If enabled by the user, server admins will see [soft failed](https://spec.matrix.org/v1.13/server-server-api/#soft-failure) events over the Client-Server API.
@@ -0,0 +1 @@
+Add experimental support for [MSC4277](https://github.com/matrix-org/matrix-spec-proposals/pull/4277).
@@ -0,0 +1 @@
+Add ability to limit amount uploaded by a user in a given time period.
@@ -0,0 +1 @@
+Allow user registrations to be done on workers.
@@ -0,0 +1 @@
+Remove unnecessary HTTP replication calls.
@@ -0,0 +1 @@
+Unbreak "Latest dependencies" workflow by using the `--without dev` poetry option instead of removed `--no-dev`.
@@ -0,0 +1 @@
+Use `markdown-it-py` instead of `commonmark` in the release script.
@@ -0,0 +1 @@
+Add doc comment explaining that config files are shallowly merged.
@@ -0,0 +1 @@
+Minor speed up of insertion into `stream_positions` table.
@@ -0,0 +1 @@
+Add support for MSC4291: Room IDs as hashes of create event.
@@ -220,29 +220,24 @@
      "yBucketBound": "auto"
    },
    {
-      "aliasColors": {},
-      "bars": false,
-      "dashLength": 10,
-      "dashes": false,
      "datasource": {
-        "uid": "${DS_PROMETHEUS}"
+        "uid": "${DS_PROMETHEUS}",
+        "type": "prometheus"
      },
-      "description": "",
+      "aliasColors": {},
+      "dashLength": 10,
      "fieldConfig": {
        "defaults": {
          "links": []
        },
        "overrides": []
      },
-      "fill": 0,
-      "fillGradient": 0,
      "gridPos": {
        "h": 9,
        "w": 12,
        "x": 12,
        "y": 1
      },
-      "hiddenSeries": false,
      "id": 152,
      "legend": {
        "avg": false,
@@ -255,71 +250,81 @@
        "values": false
      },
      "lines": true,
-      "linewidth": 0,
-      "links": [],
      "nullPointMode": "connected",
      "options": {
        "alertThreshold": true
      },
      "paceLength": 10,
-      "percentage": false,
-      "pluginVersion": "9.2.2",
+      "pluginVersion": "10.4.3",
      "pointradius": 5,
-      "points": false,
      "renderer": "flot",
      "seriesOverrides": [
        {
          "alias": "Avg",
          "fill": 0,
-          "linewidth": 3
+          "linewidth": 3,
+          "$$hashKey": "object:48"
        },
        {
          "alias": "99%",
          "color": "#C4162A",
-          "fillBelowTo": "90%"
+          "fillBelowTo": "90%",
+          "$$hashKey": "object:49"
        },
        {
          "alias": "90%",
          "color": "#FF7383",
-          "fillBelowTo": "75%"
+          "fillBelowTo": "75%",
+          "$$hashKey": "object:50"
        },
        {
          "alias": "75%",
          "color": "#FFEE52",
-          "fillBelowTo": "50%"
+          "fillBelowTo": "50%",
+          "$$hashKey": "object:51"
        },
        {
          "alias": "50%",
          "color": "#73BF69",
-          "fillBelowTo": "25%"
+          "fillBelowTo": "25%",
+          "$$hashKey": "object:52"
        },
        {
          "alias": "25%",
          "color": "#1F60C4",
-          "fillBelowTo": "5%"
+          "fillBelowTo": "5%",
+          "$$hashKey": "object:53"
        },
        {
          "alias": "5%",
-          "lines": false
+          "lines": false,
+          "$$hashKey": "object:54"
        },
        {
          "alias": "Average",
          "color": "rgb(255, 255, 255)",
          "lines": true,
-          "linewidth": 3
+          "linewidth": 3,
+          "$$hashKey": "object:55"
        },
        {
-          "alias": "Events",
+          "alias": "Local events being persisted",
+          "color": "#96d98D",
+          "points": true,
+          "yaxis": 2,
+          "zindex": -3,
+          "$$hashKey": "object:56"
+        },
+        {
+          "$$hashKey": "object:329",
          "color": "#B877D9",
-          "hideTooltip": true,
+          "alias": "All events being persisted",
          "points": true,
          "yaxis": 2,
          "zindex": -3
        }
      ],
      "spaceLength": 10,
-      "stack": false,
-      "steppedLine": false,
      "targets": [
        {
          "datasource": {
@@ -384,7 +389,20 @@
          },
          "expr": "sum(rate(synapse_http_server_response_time_seconds_sum{servlet='RoomSendEventRestServlet',index=~\"$index\",instance=\"$instance\",code=~\"2..\"}[$bucket_size])) / sum(rate(synapse_http_server_response_time_seconds_count{servlet='RoomSendEventRestServlet',index=~\"$index\",instance=\"$instance\",code=~\"2..\"}[$bucket_size]))",
          "legendFormat": "Average",
-          "refId": "H"
+          "refId": "H",
+          "editorMode": "code",
+          "range": true
+        },
+        {
+          "datasource": {
+            "uid": "${DS_PROMETHEUS}"
+          },
+          "expr": "sum(rate(synapse_http_server_response_time_seconds_count{servlet='RoomSendEventRestServlet',index=~\"$index\",instance=\"$instance\",code=~\"2..\"}[$bucket_size]))",
+          "hide": false,
+          "instant": false,
+          "legendFormat": "Local events being persisted",
+          "refId": "E",
+          "editorMode": "code"
        },
        {
          "datasource": {
@@ -393,8 +411,9 @@
          "expr": "sum(rate(synapse_storage_events_persisted_events_total{instance=\"$instance\"}[$bucket_size]))",
          "hide": false,
          "instant": false,
-          "legendFormat": "Events",
-          "refId": "E"
+          "legendFormat": "All events being persisted",
+          "refId": "I",
+          "editorMode": "code"
        }
      ],
      "thresholds": [
@@ -428,7 +447,9 @@
      "xaxis": {
        "mode": "time",
        "show": true,
-        "values": []
+        "values": [],
+        "name": null,
+        "buckets": null
      },
      "yaxes": [
        {
@@ -450,7 +471,20 @@
      ],
      "yaxis": {
        "align": false
-      }
+      },
+      "bars": false,
+      "dashes": false,
+      "description": "",
+      "fill": 0,
+      "fillGradient": 0,
+      "hiddenSeries": false,
+      "linewidth": 0,
+      "percentage": false,
+      "points": false,
+      "stack": false,
+      "steppedLine": false,
+      "timeFrom": null,
+      "timeShift": null
    },
    {
      "aliasColors": {},
@@ -45,6 +45,10 @@ def make_graph(pdus: List[dict], filename_prefix: str) -> None:
    colors = {"red", "green", "blue", "yellow", "purple"}

    for pdu in pdus:
+        # TODO: The "origin" field has since been removed from events generated
+        # by Synapse. We should consider removing it here as well but since this
+        # is part of `contrib/`, it is left for the community to revise and ensure things
+        # still work correctly.
        origins.add(pdu.get("origin"))

    color_map = {color: color for color in colors if color in origins}
@@ -1,3 +1,33 @@
+matrix-synapse-py3 (1.134.0~rc1) stable; urgency=medium
+
+  * New Synapse release 1.134.0rc1.
+
+ -- Synapse Packaging team <packages@matrix.org>  Wed, 09 Jul 2025 11:27:13 +0100
+
+matrix-synapse-py3 (1.133.0) stable; urgency=medium
+
+  * New synapse release 1.133.0.
+
+ -- Synapse Packaging team <packages@matrix.org>  Tue, 01 Jul 2025 13:13:24 +0000
+
+matrix-synapse-py3 (1.133.0~rc1) stable; urgency=medium
+
+  * New Synapse release 1.133.0rc1.
+
+ -- Synapse Packaging team <packages@matrix.org>  Tue, 24 Jun 2025 11:57:47 +0100
+
+matrix-synapse-py3 (1.132.0) stable; urgency=medium
+
+  * New Synapse release 1.132.0.
+
+ -- Synapse Packaging team <packages@matrix.org>  Tue, 17 Jun 2025 13:16:20 +0100
+
+matrix-synapse-py3 (1.132.0~rc1) stable; urgency=medium
+
+  * New Synapse release 1.132.0rc1.
+
+ -- Synapse Packaging team <packages@matrix.org>  Tue, 10 Jun 2025 11:15:18 +0100
+
 matrix-synapse-py3 (1.131.0) stable; urgency=medium

  * New Synapse release 1.131.0.
@@ -127,6 +127,8 @@ experimental_features:
  msc3983_appservice_otk_claims: true
  # Proxy key queries to exclusive ASes
  msc3984_appservice_key_query: true
+  # Invite filtering
+  msc4155_enabled: true

 server_notices:
  system_mxid_localpart: _server
@@ -63,6 +63,18 @@ mdbook serve

 The URL at which the docs can be viewed at will be logged.

+## Synapse configuration documentation
+
+The [Configuration
+Manual](https://element-hq.github.io/synapse/latest/usage/configuration/config_documentation.html)
+page is generated from a YAML file,
+[schema/synapse-config.schema.yaml](../schema/synapse-config.schema.yaml). To
+add new options or modify existing ones, first edit that file, then run
+[scripts-dev/gen_config_documentation.py](../scripts-dev/gen_config_documentation.py)
+to generate an updated Configuration Manual markdown file.
+
+Build the book as described above to preview it in a web browser.
+
 ## Configuration and theming

 The look and behaviour of the website is configured by the [book.toml](../book.toml) file
@@ -49,6 +49,8 @@
        - [Background update controller callbacks](modules/background_update_controller_callbacks.md)
        - [Account data callbacks](modules/account_data_callbacks.md)
        - [Add extra fields to client events unsigned section callbacks](modules/add_extra_fields_to_client_events_unsigned.md)
+        - [Media repository callbacks](modules/media_repository_callbacks.md)
+        - [Ratelimit callbacks](modules/ratelimit_callbacks.md)
        - [Porting a legacy module to the new interface](modules/porting_legacy_module.md)
    - [Workers](workers.md)
      - [Using `synctl` with Workers](synctl_workers.md)
@@ -66,11 +68,13 @@
      - [Registration Tokens](usage/administration/admin_api/registration_tokens.md)
      - [Manipulate Room Membership](admin_api/room_membership.md)
      - [Rooms](admin_api/rooms.md)
+      - [Scheduled tasks](admin_api/scheduled_tasks.md)
      - [Server Notices](admin_api/server_notices.md)
      - [Statistics](admin_api/statistics.md)
      - [Users](admin_api/user_admin_api.md)
      - [Server Version](admin_api/version_api.md)
      - [Federation](usage/administration/admin_api/federation.md)
+      - [Client-Server API Extensions](admin_api/client_server_api_extensions.md)
    - [Manhole](manhole.md)
    - [Monitoring](metrics-howto.md)
      - [Reporting Homeserver Usage Statistics](usage/administration/monitoring/reporting_homeserver_usage_statistics.md)
@@ -0,0 +1,25 @@
+# Client-Server API Extensions
+
+Server administrators can set special account data to change how the Client-Server API behaves for
+their clients. Setting the account data, or having it already set, as a non-admin has no effect.
+
+All configuration options can be set through the `io.element.synapse.admin_client_config` global 
+account data on the admin's user account. 
+
+Example:
+```
+PUT /_matrix/client/v3/user/{adminUserId}/account_data/io.element.synapse.admin_client_config
+{
+    "return_soft_failed_events": true
+}
+```
+
+## See soft failed events
+
+Learn more about soft failure from [the spec](https://spec.matrix.org/v1.14/server-server-api/#soft-failure).
+
+To receive soft failed events in APIs like `/sync` and `/messages`, set `return_soft_failed_events`
+to `true` in the admin client config. When `false`, the normal behaviour of these endpoints is to
+exclude soft failed events.
+
+Default: `false`
@@ -117,7 +117,6 @@ It returns a JSON body like the following:
        "hashes": {
            "sha256": "xK1//xnmvHJIOvbgXlkI8eEqdvoMmihVDJ9J4SNlsAw"
        },
-        "origin": "matrix.org",
        "origin_server_ts": 1592291711430,
        "prev_events": [
            "$YK4arsKKcc0LRoe700pS8DSjOvUT4NDv0HfInlMFw2M"
@@ -806,7 +806,7 @@ A response body like the following is returned:
        }, {
            "delete_id": "delete_id2",
            "room_id": "!roomid:example.com",
-            "status": "purging",
+            "status": "active",
            "shutdown_room": {
                "kicked_users": [
                    "@foobar:example.com"
@@ -843,7 +843,7 @@ A response body like the following is returned:

 ```json
 {
-    "status": "purging",
+    "status": "active",
    "delete_id": "bHkCNQpHqOaFhPtK",
    "room_id": "!roomid:example.com",
    "shutdown_room": {
@@ -876,8 +876,8 @@ The following fields are returned in the JSON response body:
  - `delete_id` - The ID for this purge
  - `room_id` - The ID of the room being deleted
  - `status` - The status will be one of:
-    - `shutting_down` - The process is removing users from the room.
-    - `purging` - The process is purging the room and event data from database.
+    - `scheduled` - The deletion is waiting to be started
+    - `active` - The process is purging the room and event data from database.
    - `complete` - The process has completed successfully.
    - `failed` - The process is aborted, an error has occurred.
  - `error` - A string that shows an error message if `status` is `failed`.
@@ -163,7 +163,8 @@ Body parameters:
 - `locked` - **bool**, optional. If unspecified, locked state will be left unchanged.
 - `user_type` - **string** or null, optional. If not provided, the user type will be
  not be changed. If `null` is given, the user type will be cleared.
-  Other allowed options are: `bot` and `support`.
+  Other allowed options are: `bot` and `support` and any extra values defined in the homserver
+  [configuration](../usage/configuration/config_documentation.md#user_types).

 ## List Accounts
 ### List Accounts (V2)
@@ -954,7 +955,8 @@ A response body like the following is returned:
      "last_seen_ip": "1.2.3.4",
      "last_seen_user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:103.0) Gecko/20100101 Firefox/103.0",
      "last_seen_ts": 1474491775024,
-      "user_id": "<user_id>"
+      "user_id": "<user_id>",
+      "dehydrated": false
    },
    {
      "device_id": "AUIECTSRND",
@@ -962,7 +964,8 @@ A response body like the following is returned:
      "last_seen_ip": "1.2.3.5",
      "last_seen_user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:103.0) Gecko/20100101 Firefox/103.0",
      "last_seen_ts": 1474491775025,
-      "user_id": "<user_id>"
+      "user_id": "<user_id>",
+      "dehydrated": false
    }
  ],
  "total": 2
@@ -992,6 +995,7 @@ The following fields are returned in the JSON response body:
  - `last_seen_ts` - The timestamp (in milliseconds since the unix epoch) when this
    devices was last seen. (May be a few minutes out of date, for efficiency reasons).
  - `user_id` - Owner of  device.
+  - `dehydrated` - Whether the device is a dehydrated device.

 - `total` - Total number of user's devices.

@@ -29,8 +29,6 @@ easiest way of installing the latest version is to use [rustup](https://rustup.r

 Synapse can connect to PostgreSQL via the [psycopg2](https://pypi.org/project/psycopg2/) Python library. Building this library from source requires access to PostgreSQL's C header files. On Debian or Ubuntu Linux, these can be installed with `sudo apt install libpq-dev`.

-Synapse has an optional, improved user search with better Unicode support. For that you need the development package of `libicu`. On Debian or Ubuntu Linux, this can be installed with `sudo apt install libicu-dev`.
-
 The source code of Synapse is hosted on GitHub. You will also need [a recent version of git](https://github.com/git-guides/install-git).

 For some tests, you will need [a recent version of Docker](https://docs.docker.com/get-docker/).
@@ -164,10 +164,7 @@ $ poetry cache clear --all .
 # including the wheel artifacts which is not covered by the above command
 # (see https://github.com/python-poetry/poetry/issues/10304)
 #
-# This is necessary in order to rebuild or fetch new wheels. For example, if you update
-# the `icu` library in on your system, you will need to rebuild the PyICU Python package
-# in order to incorporate the correct dynamically linked library locations otherwise you
-# will run into errors like: `ImportError: libicui18n.so.75: cannot open shared object file: No such file or directory`
+# This is necessary in order to rebuild or fetch new wheels.
 $ rm -rf $(poetry config cache-dir)
 ```

@@ -0,0 +1,66 @@
+# Media repository callbacks
+
+Media repository callbacks allow module developers to customise the behaviour of the
+media repository on a per user basis. Media repository callbacks can be registered
+using the module API's `register_media_repository_callbacks` method.
+
+The available media repository callbacks are:
+
+### `get_media_config_for_user`
+
+_First introduced in Synapse v1.132.0_
+
+```python
+async def get_media_config_for_user(user_id: str) -> Optional[JsonDict]
+```
+
+**<span style="color:red">
+Caution: This callback is currently experimental . The method signature or behaviour
+may change without notice.
+</span>**
+
+Called when processing a request from a client for the
+[media config endpoint](https://spec.matrix.org/latest/client-server-api/#get_matrixclientv1mediaconfig).
+
+The arguments passed to this callback are:
+
+* `user_id`: The Matrix user ID of the user (e.g. `@alice:example.com`) making the request.
+
+If the callback returns a dictionary then it will be used as the body of the response to the
+client.
+
+If multiple modules implement this callback, they will be considered in order. If a
+callback returns `None`, Synapse falls through to the next one. The value of the first
+callback that does not return `None` will be used. If this happens, Synapse will not call
+any of the subsequent implementations of this callback.
+
+If no module returns a non-`None` value then the default media config will be returned.
+
+### `is_user_allowed_to_upload_media_of_size`
+
+_First introduced in Synapse v1.132.0_
+
+```python
+async def is_user_allowed_to_upload_media_of_size(user_id: str, size: int) -> bool
+```
+
+**<span style="color:red">
+Caution: This callback is currently experimental . The method signature or behaviour
+may change without notice.
+</span>**
+
+Called before media is accepted for upload from a user, in case the module needs to
+enforce a different limit for the particular user.
+
+The arguments passed to this callback are:
+
+* `user_id`: The Matrix user ID of the user (e.g. `@alice:example.com`) making the request.
+* `size`: The size in bytes of media that is being requested to upload.
+
+If the module returns `False`, the current request will be denied with the error code
+`M_TOO_LARGE` and the HTTP status code 413.
+
+If multiple modules implement this callback, they will be considered in order. If a callback
+returns `True`, Synapse falls through to the next one. The value of the first callback that
+returns `False` will be used. If this happens, Synapse will not call any of the subsequent
+implementations of this callback.
@@ -0,0 +1,43 @@
+# Ratelimit callbacks
+
+Ratelimit callbacks allow module developers to override ratelimit settings dynamically whilst
+Synapse is running. Ratelimit callbacks can be registered using the module API's
+`register_ratelimit_callbacks` method.
+
+The available ratelimit callbacks are:
+
+### `get_ratelimit_override_for_user`
+
+_First introduced in Synapse v1.132.0_
+
+```python
+async def get_ratelimit_override_for_user(user: str, limiter_name: str) -> Optional[synapse.module_api.RatelimitOverride]
+```
+
+**<span style="color:red">
+Caution: This callback is currently experimental . The method signature or behaviour
+may change without notice.
+</span>**
+
+Called when constructing a ratelimiter of a particular type for a user. The module can
+return a `messages_per_second` and `burst_count` to be used, or `None` if
+the default settings are adequate. The user is represented by their Matrix user ID
+(e.g. `@alice:example.com`). The limiter name is usually taken from the `RatelimitSettings` key
+value.
+
+The limiters that are currently supported are:
+
+- `rc_invites.per_room`
+- `rc_invites.per_user`
+- `rc_invites.per_issuer`
+
+The `RatelimitOverride` return type has the following fields:
+
+- `per_second: float`. The number of actions that can be performed in a second. `0.0` means that ratelimiting is disabled.
+- `burst_count: int`. The number of actions that can be performed before being limited.
+
+If multiple modules implement this callback, they will be considered in order. If a
+callback returns `None`, Synapse falls through to the next one. The value of the first
+callback that does not return `None` will be used. If this happens, Synapse will not call
+any of the subsequent implementations of this callback. If no module returns a non-`None` value
+then the default settings will be used.
@@ -80,6 +80,8 @@ Called when processing an invitation, both when one is created locally or when
 receiving an invite over federation. Both inviter and invitee are represented by
 their Matrix user ID (e.g. `@alice:example.com`).

+Note that federated invites will call `federated_user_may_invite` before this callback.
+

 The callback must return one of:
  - `synapse.module_api.NOT_SPAM`, to allow the operation. Other callbacks may still 
@@ -97,6 +99,34 @@ be used. If this happens, Synapse will not call any of the subsequent implementa
 this callback.


+### `federated_user_may_invite`
+
+_First introduced in Synapse v1.133.0_
+
+```python
+async def federated_user_may_invite(event: "synapse.events.EventBase") -> Union["synapse.module_api.NOT_SPAM", "synapse.module_api.errors.Codes", bool]
+```
+
+Called when processing an invitation received over federation. Unlike `user_may_invite`,
+this callback receives the entire event, including any stripped state in the `unsigned`
+section, not just the room and user IDs.
+
+The callback must return one of:
+  - `synapse.module_api.NOT_SPAM`, to allow the operation. Other callbacks may still 
+    decide to reject it.
+  - `synapse.module_api.errors.Codes` to reject the operation with an error code. In case
+    of doubt, `synapse.module_api.errors.Codes.FORBIDDEN` is a good error code.
+
+If multiple modules implement this callback, they will be considered in order. If a
+callback returns `synapse.module_api.NOT_SPAM`, Synapse falls through to the next one.
+The value of the first callback that does not return `synapse.module_api.NOT_SPAM` will
+be used. If this happens, Synapse will not call any of the subsequent implementations of
+this callback.
+
+If all of the callbacks return `synapse.module_api.NOT_SPAM`, Synapse will also fall
+through to the `user_may_invite` callback before approving the invite.
+
+
 ### `user_may_send_3pid_invite`

 _First introduced in Synapse v1.45.0_
@@ -159,12 +189,19 @@ _First introduced in Synapse v1.37.0_

 _Changed in Synapse v1.62.0: `synapse.module_api.NOT_SPAM` and `synapse.module_api.errors.Codes` can be returned by this callback. Returning a boolean is now deprecated._ 

+_Changed in Synapse v1.132.0: Added the `room_config` argument. Callbacks that only expect a single `user_id` argument are still supported._
+
 ```python
-async def user_may_create_room(user_id: str) -> Union["synapse.module_api.NOT_SPAM", "synapse.module_api.errors.Codes", bool]
+async def user_may_create_room(user_id: str, room_config: synapse.module_api.JsonDict) -> Union["synapse.module_api.NOT_SPAM", "synapse.module_api.errors.Codes", bool]
 ```

 Called when processing a room creation request.

+The arguments passed to this callback are:
+
+* `user_id`: The Matrix user ID of the user (e.g. `@alice:example.com`).
+* `room_config`: The contents of the body of a [/createRoom request](https://spec.matrix.org/latest/client-server-api/#post_matrixclientv3createroom) as a dictionary.
+
 The callback must return one of:
  - `synapse.module_api.NOT_SPAM`, to allow the operation. Other callbacks may still 
    decide to reject it.
@@ -239,6 +276,41 @@ be used. If this happens, Synapse will not call any of the subsequent implementa
 this callback.


+### `user_may_send_state_event`
+
+_First introduced in Synapse v1.132.0_
+
+```python
+async def user_may_send_state_event(user_id: str, room_id: str, event_type: str, state_key: str, content: JsonDict) -> Union["synapse.module_api.NOT_SPAM", "synapse.module_api.errors.Codes"]
+```
+
+**<span style="color:red">
+Caution: This callback is currently experimental . The method signature or behaviour
+may change without notice.
+</span>**
+
+Called when processing a request to [send state events](https://spec.matrix.org/latest/client-server-api/#put_matrixclientv3roomsroomidstateeventtypestatekey) to a room.
+
+The arguments passed to this callback are:
+
+* `user_id`: The Matrix user ID of the user (e.g. `@alice:example.com`) sending the state event.
+* `room_id`: The ID of the room that the requested state event is being sent to.
+* `event_type`: The requested type of event.
+* `state_key`: The requested state key.
+* `content`: The requested event contents.
+
+The callback must return one of:
+  - `synapse.module_api.NOT_SPAM`, to allow the operation. Other callbacks may still 
+    decide to reject it.
+  - `synapse.module_api.errors.Codes` to reject the operation with an error code. In case
+    of doubt, `synapse.module_api.errors.Codes.FORBIDDEN` is a good error code.
+
+If multiple modules implement this callback, they will be considered in order. If a
+callback returns `synapse.module_api.NOT_SPAM`, Synapse falls through to the next one.
+The value of the first callback that does not return `synapse.module_api.NOT_SPAM` will
+be used. If this happens, Synapse will not call any of the subsequent implementations of
+this callback.
+

 ### `check_username_for_spam`

@@ -5,10 +5,10 @@ It is recommended to put a reverse proxy such as
 [Apache](https://httpd.apache.org/docs/current/mod/mod_proxy_http.html),
 [Caddy](https://caddyserver.com/docs/quick-starts/reverse-proxy),
 [HAProxy](https://www.haproxy.org/) or
-[relayd](https://man.openbsd.org/relayd.8) in front of Synapse. One advantage
-of doing so is that it means that you can expose the default https port
-(443) to Matrix clients without needing to run Synapse with root
-privileges.
+[relayd](https://man.openbsd.org/relayd.8) in front of Synapse.
+This has the advantage of being able to expose the default HTTPS port (443) to Matrix
+clients without requiring Synapse to bind to a privileged port (port numbers less than
+1024), avoiding the need for `CAP_NET_BIND_SERVICE` or running as root.

 You should configure your reverse proxy to forward requests to `/_matrix` or
 `/_synapse/client` to Synapse, and have it set the `X-Forwarded-For` and
@@ -286,7 +286,7 @@ Installing prerequisites on Ubuntu or Debian:
 ```sh
 sudo apt install build-essential python3-dev libffi-dev \
                     python3-pip python3-setuptools sqlite3 \
-                     libssl-dev virtualenv libjpeg-dev libxslt1-dev libicu-dev
+                     libssl-dev virtualenv libjpeg-dev libxslt1-dev
 ```

 ##### ArchLinux
@@ -295,7 +295,7 @@ Installing prerequisites on ArchLinux:

 ```sh
 sudo pacman -S base-devel python python-pip \
-               python-setuptools python-virtualenv sqlite3 icu
+               python-setuptools python-virtualenv sqlite3
 ```

 ##### CentOS/Fedora
@@ -305,8 +305,7 @@ Installing prerequisites on CentOS or Fedora Linux:
 ```sh
 sudo dnf install libtiff-devel libjpeg-devel libzip-devel freetype-devel \
                 libwebp-devel libxml2-devel libxslt-devel libpq-devel \
-                 python3-virtualenv libffi-devel openssl-devel python3-devel \
-                 libicu-devel
+                 python3-virtualenv libffi-devel openssl-devel python3-devel
 sudo dnf group install "Development Tools"
 ```

@@ -333,7 +332,7 @@ dnf install python3.12 python3.12-devel
 ```
 Finally, install common prerequisites
 ```bash
-dnf install libicu libicu-devel libpq5 libpq5-devel lz4 pkgconf
+dnf install libpq5 libpq5-devel lz4 pkgconf
 dnf group install "Development Tools"
 ```
 ###### Using venv module instead of virtualenv command
@@ -365,20 +364,6 @@ xcode-select --install

 Some extra dependencies may be needed. You can use Homebrew (https://brew.sh) for them.

-You may need to install icu, and make the icu binaries and libraries accessible.
-Please follow [the official instructions of PyICU](https://pypi.org/project/PyICU/) to do so.
-
-If you're struggling to get icu discovered, and see:
-```
-  RuntimeError:
-  Please install pkg-config on your system or set the ICU_VERSION environment
-  variable to the version of ICU you have installed.
-```
-despite it being installed and having your `PATH` updated, you can omit this dependency by
-not specifying `--extras all` to `poetry`. If using postgres, you can install Synapse via
-`poetry install --extras saml2 --extras oidc --extras postgres --extras opentracing --extras redis --extras sentry`.
-ICU is not a hard dependency on getting a working installation.
-
 On ARM-based Macs you may also need to install libjpeg and libpq:
 ```sh
 brew install jpeg libpq
@@ -400,8 +385,7 @@ Installing prerequisites on openSUSE:
 ```sh
 sudo zypper in -t pattern devel_basis
 sudo zypper in python-pip python-setuptools sqlite3 python-virtualenv \
-               python-devel libffi-devel libopenssl-devel libjpeg62-devel \
-               libicu-devel
+               python-devel libffi-devel libopenssl-devel libjpeg62-devel
 ```

 ##### OpenBSD
@@ -63,7 +63,7 @@ class ExampleSpamChecker:
    async def user_may_invite(self, inviter_userid, invitee_userid, room_id):
        return True  # allow all invites

-    async def user_may_create_room(self, userid):
+    async def user_may_create_room(self, userid, room_config):
        return True  # allow all room creations

    async def user_may_create_room_alias(self, userid, room_alias):
@@ -117,6 +117,21 @@ each upgrade are complete before moving on to the next upgrade, to avoid
 stacking them up. You can monitor the currently running background updates with
 [the Admin API](usage/administration/admin_api/background_updates.html#status).

+# Upgrading to v1.135.0
+
+## `on_user_registration` module API callback may now run on any worker
+
+Previously, the `on_user_registration` callback would only run on the main
+process. Modules relying on this callback must assume that they may now be
+called from any worker, not just the main process.
+
+# Upgrading to v1.134.0
+
+## ICU bundled with Synapse
+
+Synapse now uses the Rust `icu` library for improved user search. Installing the
+native ICU library on your system is no longer required.
+
 # Upgrading to v1.130.0

 ## Documented endpoint which can be delegated to a federation worker
@@ -255,7 +255,7 @@ line to `/etc/default/matrix-synapse`:

    LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2

-*Note*: You may need to set `PYTHONMALLOC=malloc` to ensure that `jemalloc` can accurately calculate memory usage. By default, Python uses its internal small-object allocator, which may interfere with jemalloc's ability to track memory consumption correctly. This could prevent the [cache_autotuning](../configuration/config_documentation.md#caches-and-associated-values) feature from functioning as expected, as the Python allocator may not reach the memory threshold set by `max_cache_memory_usage`, thus not triggering the cache eviction process.
+*Note*: You may need to set `PYTHONMALLOC=malloc` to ensure that `jemalloc` can accurately calculate memory usage. By default, Python uses its internal small-object allocator, which may interfere with jemalloc's ability to track memory consumption correctly. This could prevent the [cache_autotuning](../configuration/config_documentation.md#caches) feature from functioning as expected, as the Python allocator may not reach the memory threshold set by `max_cache_memory_usage`, thus not triggering the cache eviction process.

 This made a significant difference on Python 2.7 - it's unclear how
 much of an improvement it provides on Python 3.x.
@@ -77,14 +77,11 @@ The user provided search term is lowercased and normalized using [NFKC](https://
 this treats the string as case-insensitive, canonicalizes different forms of the
 same text, and maps some "roughly equivalent" characters together.

-The search term is then split into words:
-
-* If [ICU](https://en.wikipedia.org/wiki/International_Components_for_Unicode) is
-  available, then the system's [default locale](https://unicode-org.github.io/icu/userguide/locale/#default-locales)
-  will be used to break the search term into words. (See the
-  [installation instructions](setup/installation.md) for how to install ICU.)
-* If unavailable, then runs of ASCII characters, numbers, underscores, and hyphens
-  are considered words.
+The search term is then split into segments using the [`icu_segmenter`
+Rust crate](https://crates.io/crates/icu_segmenter). This crate ships with its
+own dictionary and Long Short Term-Memory (LSTM) machine learning models
+per-language to segment words. Read more [in the crate's
+documentation](https://docs.rs/icu/latest/icu/segmenter/struct.WordSegmenter.html#method.new_auto).

 The queries for PostgreSQL and SQLite are detailed below, but their overall goal
 is to find matching users, preferring users who are "real" (e.g. not bots,
@@ -96,7 +96,6 @@
                  gnumake

                  # Native dependencies for running Synapse.
-                  icu
                  libffi
                  libjpeg
                  libpqxx
@@ -74,6 +74,10 @@ select = [
    "PIE",
    # flake8-executable
    "EXE",
+    # flake8-logging
+    "LOG",
+    # flake8-logging-format
+    "G",
 ]

 [tool.ruff.lint.isort]
@@ -97,7 +101,7 @@ module-name = "synapse.synapse_rust"

 [tool.poetry]
 name = "matrix-synapse"
-version = "1.131.0"
+version = "1.134.0rc1"
 description = "Homeserver for the Matrix decentralised comms protocol"
 authors = ["Matrix.org Team and Contributors <packages@matrix.org>"]
 license = "AGPL-3.0-or-later"
@@ -220,7 +224,7 @@ pydantic = ">=1.7.4, <3"
 # https://github.com/python-poetry/poetry/issues/6154). Both `pip install` and
 # `poetry build` do the right thing without this explicit dependency.
 #
-# This isn't really a dev-dependency, as `poetry install --no-dev` will fail,
+# This isn't really a dev-dependency, as `poetry install --without dev` will fail,
 # but the alternative is to add it to the main list of deps where it isn't
 # needed.
 setuptools_rust = ">=1.3"
@@ -250,7 +254,6 @@ hiredis = { version = "*", optional = true }
 Pympler = { version = "*", optional = true }
 parameterized = { version = ">=0.7.4", optional = true }
 idna = { version = ">=2.5", optional = true }
-pyicu = { version = ">=2.10.2", optional = true }

 [tool.poetry.extras]
 # NB: Packages that should be part of `pip install matrix-synapse[all]` need to be specified
@@ -273,10 +276,6 @@ redis = ["txredisapi", "hiredis"]
 # Required to use experimental `caches.track_memory_usage` config option.
 cache-memory = ["pympler"]
 test = ["parameterized", "idna"]
-# Allows for better search for international characters in the user directory. This
-# requires libicu's development headers installed on the system (e.g. libicu-dev on
-# Debian-based distributions).
-user-search = ["pyicu"]

 # The duplication here is awful. I hate hate hate hate hate it. However, for now I want
 # to ensure you can still `pip install matrix-synapse[all]` like today. Two motivations:
@@ -308,8 +307,6 @@ all = [
    "txredisapi", "hiredis",
    # cache-memory
    "pympler",
-    # improved user search
-    "pyicu",
    # omitted:
    #   - test: it's useful to have this separate from dev deps in the olddeps job
    #   - systemd: this is a system-based requirement
@@ -320,7 +317,7 @@ all = [
 # failing on new releases. Keeping lower bounds loose here means that dependabot
 # can bump versions without having to update the content-hash in the lockfile.
 # This helps prevents merge conflicts when running a batch of dependabot updates.
-ruff = "0.11.11"
+ruff = "0.12.2"
 # Type checking only works with the pydantic.v1 compat module from pydantic v2
 pydantic = "^2"

@@ -329,7 +326,6 @@ lxml-stubs = ">=0.4.0"
 mypy = "*"
 mypy-zope = "*"
 types-bleach = ">=4.1.0"
-types-commonmark = ">=0.9.2"
 types-jsonschema = ">=3.2.0"
 types-netaddr = ">=0.8.0.6"
 types-opentracing = ">=2.4.2"
@@ -352,7 +348,7 @@ idna = ">=2.5"
 click = ">=8.1.3"
 # GitPython was == 3.1.14; bumped to 3.1.20, the first release with type hints.
 GitPython = ">=3.1.20"
-commonmark = ">=0.9.1"
+markdown-it-py = ">=3.0.0"
 pygithub = ">=1.55"
 # The following are executed as commands by the release script.
 twine = "*"
@@ -370,7 +366,7 @@ tomli = ">=1.2.3"
 # runtime errors caused by build system changes.
 # We are happy to raise these upper bounds upon request,
 # provided we check that it's safe to do so (i.e. that CI passes).
-requires = ["poetry-core>=1.1.0,<=1.9.1", "setuptools_rust>=1.3,<=1.10.2"]
+requires = ["poetry-core>=1.1.0,<=2.1.3", "setuptools_rust>=1.3,<=1.11.1"]
 build-backend = "poetry.core.masonry.api"


@@ -378,13 +374,10 @@ build-backend = "poetry.core.masonry.api"
 # Skip unsupported platforms (by us or by Rust).
 # See https://cibuildwheel.readthedocs.io/en/stable/options/#build-skip for the list of build targets.
 # We skip:
-#  - CPython 3.6, 3.7 and 3.8: EOLed
-#  - PyPy 3.7 and 3.8: we only support Python 3.9+
+#  - CPython and PyPy 3.8: EOLed
 #  - musllinux i686: excluded to reduce number of wheels we build.
 #    c.f. https://github.com/matrix-org/synapse/pull/12595#discussion_r963107677
-#  - PyPy on Aarch64 and musllinux on aarch64: too slow to build.
-#    c.f. https://github.com/matrix-org/synapse/pull/14259
-skip = "cp36* cp37* cp38* pp37* pp38* *-musllinux_i686 pp*aarch64 *-musllinux_aarch64"
+skip = "cp38* pp38* *-musllinux_i686"
 # Enable non-default builds.
 # "pypy" used to be included by default up until cibuildwheel 3.
 enable = "pypy"
@@ -7,7 +7,7 @@ name = "synapse"
 version = "0.1.0"

 edition = "2021"
-rust-version = "1.66.0"
+rust-version = "1.81.0"

 [lib]
 name = "synapse"
@@ -30,19 +30,28 @@ http = "1.1.0"
 lazy_static = "1.4.0"
 log = "0.4.17"
 mime = "0.3.17"
-pyo3 = { version = "0.24.2", features = [
+pyo3 = { version = "0.25.1", features = [
    "macros",
    "anyhow",
    "abi3",
    "abi3-py39",
 ] }
-pyo3-log = "0.12.0"
-pythonize = "0.24.0"
+pyo3-log = "0.12.4"
+pythonize = "0.25.0"
 regex = "1.6.0"
 sha2 = "0.10.8"
 serde = { version = "1.0.144", features = ["derive"] }
 serde_json = "1.0.85"
 ulid = "1.1.2"
+icu_segmenter = "2.0.0"
+reqwest = { version = "0.12.15", default-features = false, features = [
+    "http2",
+    "stream",
+    "rustls-tls-native-roots",
+] }
+http-body-util = "0.1.3"
+futures = "0.3.31"
+tokio = { version = "1.44.2", features = ["rt", "rt-multi-thread"] }

 [features]
 extension-module = ["pyo3/extension-module"]
@@ -58,3 +58,15 @@ impl NotFoundError {
        NotFoundError::new_err(())
    }
 }
+
+import_exception!(synapse.api.errors, HttpResponseException);
+
+impl HttpResponseException {
+    pub fn new(status: StatusCode, bytes: Vec<u8>) -> pyo3::PyErr {
+        HttpResponseException::new_err((
+            status.as_u16(),
+            status.canonical_reason().unwrap_or_default(),
+            bytes,
+        ))
+    }
+}
@@ -0,0 +1,218 @@
+/*
+ * This file is licensed under the Affero General Public License (AGPL) version 3.
+ *
+ * Copyright (C) 2025 New Vector, Ltd
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * See the GNU Affero General Public License for more details:
+ * <https://www.gnu.org/licenses/agpl-3.0.html>.
+ */
+
+use std::{collections::HashMap, future::Future, panic::AssertUnwindSafe, sync::LazyLock};
+
+use anyhow::Context;
+use futures::{FutureExt, TryStreamExt};
+use pyo3::{exceptions::PyException, prelude::*, types::PyString};
+use reqwest::RequestBuilder;
+use tokio::runtime::Runtime;
+
+use crate::errors::HttpResponseException;
+
+/// The tokio runtime that we're using to run async Rust libs.
+static RUNTIME: LazyLock<Runtime> = LazyLock::new(|| {
+    tokio::runtime::Builder::new_multi_thread()
+        .worker_threads(4)
+        .enable_all()
+        .build()
+        .unwrap()
+});
+
+/// A reference to the `Deferred` python class.
+static DEFERRED_CLASS: LazyLock<PyObject> = LazyLock::new(|| {
+    Python::with_gil(|py| {
+        py.import("twisted.internet.defer")
+            .expect("module 'twisted.internet.defer' should be importable")
+            .getattr("Deferred")
+            .expect("module 'twisted.internet.defer' should have a 'Deferred' class")
+            .unbind()
+    })
+});
+
+/// A reference to the twisted `reactor`.
+static TWISTED_REACTOR: LazyLock<Py<PyModule>> = LazyLock::new(|| {
+    Python::with_gil(|py| {
+        py.import("twisted.internet.reactor")
+            .expect("module 'twisted.internet.reactor' should be importable")
+            .unbind()
+    })
+});
+
+/// Called when registering modules with python.
+pub fn register_module(py: Python<'_>, m: &Bound<'_, PyModule>) -> PyResult<()> {
+    let child_module: Bound<'_, PyModule> = PyModule::new(py, "http_client")?;
+    child_module.add_class::<HttpClient>()?;
+
+    // Make sure we fail early if we can't build the lazy statics.
+    LazyLock::force(&RUNTIME);
+    LazyLock::force(&DEFERRED_CLASS);
+
+    m.add_submodule(&child_module)?;
+
+    // We need to manually add the module to sys.modules to make `from
+    // synapse.synapse_rust import acl` work.
+    py.import("sys")?
+        .getattr("modules")?
+        .set_item("synapse.synapse_rust.http_client", child_module)?;
+
+    Ok(())
+}
+
+#[pyclass]
+#[derive(Clone)]
+struct HttpClient {
+    client: reqwest::Client,
+}
+
+#[pymethods]
+impl HttpClient {
+    #[new]
+    pub fn py_new(user_agent: &str) -> PyResult<HttpClient> {
+        // The twisted reactor can only be imported after Synapse has been
+        // imported, to allow Synapse to change the twisted reactor. If we try
+        // and import the reactor too early twisted installs a default reactor,
+        // which can't be replaced.
+        LazyLock::force(&TWISTED_REACTOR);
+
+        Ok(HttpClient {
+            client: reqwest::Client::builder()
+                .user_agent(user_agent)
+                .build()
+                .context("building reqwest client")?,
+        })
+    }
+
+    pub fn get<'a>(
+        &self,
+        py: Python<'a>,
+        url: String,
+        response_limit: usize,
+    ) -> PyResult<Bound<'a, PyAny>> {
+        self.send_request(py, self.client.get(url), response_limit)
+    }
+
+    pub fn post<'a>(
+        &self,
+        py: Python<'a>,
+        url: String,
+        response_limit: usize,
+        headers: HashMap<String, String>,
+        request_body: String,
+    ) -> PyResult<Bound<'a, PyAny>> {
+        let mut builder = self.client.post(url);
+        for (name, value) in headers {
+            builder = builder.header(name, value);
+        }
+        builder = builder.body(request_body);
+
+        self.send_request(py, builder, response_limit)
+    }
+}
+
+impl HttpClient {
+    fn send_request<'a>(
+        &self,
+        py: Python<'a>,
+        builder: RequestBuilder,
+        response_limit: usize,
+    ) -> PyResult<Bound<'a, PyAny>> {
+        create_deferred(py, async move {
+            let response = builder.send().await.context("sending request")?;
+
+            let status = response.status();
+
+            let mut stream = response.bytes_stream();
+            let mut buffer = Vec::new();
+            while let Some(chunk) = stream.try_next().await.context("reading body")? {
+                if buffer.len() + chunk.len() > response_limit {
+                    Err(anyhow::anyhow!("Response size too large"))?;
+                }
+
+                buffer.extend_from_slice(&chunk);
+            }
+
+            if !status.is_success() {
+                return Err(HttpResponseException::new(status, buffer));
+            }
+
+            let r = Python::with_gil(|py| buffer.into_pyobject(py).map(|o| o.unbind()))?;
+
+            Ok(r)
+        })
+    }
+}
+
+/// Creates a twisted deferred from the given future, spawning the task on the
+/// tokio runtime.
+///
+/// Does not handle deferred cancellation or contextvars.
+fn create_deferred<F, O>(py: Python, fut: F) -> PyResult<Bound<'_, PyAny>>
+where
+    F: Future<Output = PyResult<O>> + Send + 'static,
+    for<'a> O: IntoPyObject<'a>,
+{
+    let deferred = DEFERRED_CLASS.bind(py).call0()?;
+    let deferred_callback = deferred.getattr("callback")?.unbind();
+    let deferred_errback = deferred.getattr("errback")?.unbind();
+
+    RUNTIME.spawn(async move {
+        // TODO: Is it safe to assert unwind safety here? I think so, as we
+        // don't use anything that could be tainted by the panic afterwards.
+        // Note that `.spawn(..)` asserts unwind safety on the future too.
+        let res = AssertUnwindSafe(fut).catch_unwind().await;
+
+        Python::with_gil(move |py| {
+            // Flatten the panic into standard python error
+            let res = match res {
+                Ok(r) => r,
+                Err(panic_err) => {
+                    let panic_message = get_panic_message(&panic_err);
+                    Err(PyException::new_err(
+                        PyString::new(py, panic_message).unbind(),
+                    ))
+                }
+            };
+
+            // Send the result to the deferred, via `.callback(..)` or `.errback(..)`
+            match res {
+                Ok(obj) => {
+                    TWISTED_REACTOR
+                        .call_method(py, "callFromThread", (deferred_callback, obj), None)
+                        .expect("callFromThread should not fail"); // There's nothing we can really do with errors here
+                }
+                Err(err) => {
+                    TWISTED_REACTOR
+                        .call_method(py, "callFromThread", (deferred_errback, err), None)
+                        .expect("callFromThread should not fail"); // There's nothing we can really do with errors here
+                }
+            }
+        });
+    });
+
+    Ok(deferred)
+}
+
+/// Try and get the panic message out of the panic
+fn get_panic_message<'a>(panic_err: &'a (dyn std::any::Any + Send + 'static)) -> &'a str {
+    // Apparently this is how you extract the panic message from a panic
+    if let Some(str_slice) = panic_err.downcast_ref::<&str>() {
+        str_slice
+    } else if let Some(string) = panic_err.downcast_ref::<String>() {
+        string
+    } else {
+        "unknown error"
+    }
+}
@@ -27,7 +27,7 @@ pub enum IdentifierError {

 impl fmt::Display for IdentifierError {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
-        write!(f, "{:?}", self)
+        write!(f, "{self:?}")
    }
 }

@@ -8,10 +8,12 @@ pub mod acl;
 pub mod errors;
 pub mod events;
 pub mod http;
+pub mod http_client;
 pub mod identifier;
 pub mod matrix_const;
 pub mod push;
 pub mod rendezvous;
+pub mod segmenter;

 lazy_static! {
    static ref LOGGING_HANDLE: ResetHandle = pyo3_log::init();
@@ -50,7 +52,9 @@ fn synapse_rust(py: Python<'_>, m: &Bound<'_, PyModule>) -> PyResult<()> {
    acl::register_module(py, m)?;
    push::register_module(py, m)?;
    events::register_module(py, m)?;
+    http_client::register_module(py, m)?;
    rendezvous::register_module(py, m)?;
+    segmenter::register_module(py, m)?;

    Ok(())
 }
@@ -0,0 +1,33 @@
+use icu_segmenter::options::WordBreakInvariantOptions;
+use icu_segmenter::WordSegmenter;
+use pyo3::prelude::*;
+
+#[pyfunction]
+pub fn parse_words(text: &str) -> PyResult<Vec<String>> {
+    let segmenter = WordSegmenter::new_auto(WordBreakInvariantOptions::default());
+    let mut parts = Vec::new();
+    let mut last = 0usize;
+
+    // `segment_str` gives us word boundaries as a vector of indexes. Use that
+    // to build a vector of words, and return.
+    for boundary in segmenter.segment_str(text) {
+        if boundary > last {
+            parts.push(text[last..boundary].to_string());
+        }
+        last = boundary;
+    }
+    Ok(parts)
+}
+
+pub fn register_module(py: Python<'_>, m: &Bound<'_, PyModule>) -> PyResult<()> {
+    let child_module = PyModule::new(py, "segmenter")?;
+    child_module.add_function(wrap_pyfunction!(parse_words, m)?)?;
+
+    m.add_submodule(&child_module)?;
+
+    py.import("sys")?
+        .getattr("modules")?
+        .set_item("synapse.synapse_rust.segmenter", child_module)?;
+
+    Ok(())
+}
@@ -0,0 +1,2 @@
+If you want to update the meta schema, copy this folder and increase its version
+number instead.
@@ -0,0 +1,29 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://element-hq.github.io/synapse/latest/schema/v1/meta.schema.json",
+  "$vocabulary": {
+    "https://json-schema.org/draft/2020-12/vocab/core": true,
+    "https://json-schema.org/draft/2020-12/vocab/applicator": true,
+    "https://json-schema.org/draft/2020-12/vocab/unevaluated": true,
+    "https://json-schema.org/draft/2020-12/vocab/validation": true,
+    "https://json-schema.org/draft/2020-12/vocab/meta-data": true,
+    "https://json-schema.org/draft/2020-12/vocab/format-annotation": true,
+    "https://json-schema.org/draft/2020-12/vocab/content": true,
+    "https://element-hq.github.io/synapse/latest/schema/v1/vocab/documentation": false
+  },
+  "$ref": "https://json-schema.org/draft/2020-12/schema",
+  "properties": {
+    "io.element.type_name": {
+      "type": "string",
+      "description": "Human-readable type of a schema that is displayed instead of the standard JSON Schema types like `object` or `integer`. In case the JSON Schema type contains `null`, this information should be presented alongside the human-readable type name.",
+      "examples": ["duration", "byte size"]
+    },
+    "io.element.post_description": {
+      "type": "string",
+      "description": "Additional description of a schema, better suited to be placed less prominently in the generated documentation, e.g., at the end of a section after listings of items and properties.",
+      "examples": [
+        "### Advanced uses\n\nThe spent coffee grounds can be added to compost for improving soil and growing plants."
+      ]
+    }
+  }
+}
@@ -0,0 +1,11 @@
+<!DOCTYPE html>
+<html lang="en">
+    <head>
+        <meta http-equiv="refresh" content="0; URL=../meta.schema.json">
+        <meta charset="UTF-8">
+        <title>Redirecting to ../meta.schema.json…</title>
+    </head>
+    <body>
+        <p>Redirecting to <a href="../meta.schema.json">../meta.schema.json</a>…</p>
+    </body>
+</html>
@@ -243,7 +243,7 @@ def do_lint() -> Set[str]:
                importlib.import_module(module_info.name)
            except ModelCheckerException as e:
                logger.warning(
-                    f"Bad annotation found when importing {module_info.name}"
+                    "Bad annotation found when importing %s", module_info.name
                )
                failures.add(format_model_checker_exception(e))

@@ -229,6 +229,7 @@ test_packages=(
    ./tests/msc3902
    ./tests/msc3967
    ./tests/msc4140
+    ./tests/msc4155
 )

 # Enable dirty runs, so tests will reuse the same container where possible.
@@ -0,0 +1,503 @@
+#!/usr/bin/env python3
+"""Generate Synapse documentation from JSON Schema file."""
+
+import json
+import re
+import sys
+from typing import Any, Optional
+
+import yaml
+
+HEADER = """<!-- Document auto-generated by scripts-dev/gen_config_documentation.py -->
+
+# Configuring Synapse
+
+This is intended as a guide to the Synapse configuration. The behavior of a Synapse instance can be modified
+through the many configuration settings documented here — each config option is explained,
+including what the default is, how to change the default and what sort of behaviour the setting governs.
+Also included is an example configuration for each setting. If you don't want to spend a lot of time
+thinking about options, the config as generated sets sensible defaults for all values. Do note however that the
+database defaults to SQLite, which is not recommended for production usage. You can read more on this subject
+[here](../../setup/installation.md#using-postgresql).
+
+## Config Conventions
+
+Configuration options that take a time period can be set using a number
+followed by a letter. Letters have the following meanings:
+
+* `s` = second
+* `m` = minute
+* `h` = hour
+* `d` = day
+* `w` = week
+* `y` = year
+
+For example, setting `redaction_retention_period: 5m` would remove redacted
+messages from the database after 5 minutes, rather than 5 months.
+
+In addition, configuration options referring to size use the following suffixes:
+
+* `K` = KiB, or 1024 bytes
+* `M` = MiB, or 1,048,576 bytes
+* `G` = GiB, or 1,073,741,824 bytes
+* `T` = TiB, or 1,099,511,627,776 bytes
+
+For example, setting `max_avatar_size: 10M` means that Synapse will not accept files larger than 10,485,760 bytes
+for a user avatar.
+
+## Config Validation
+
+The configuration file can be validated with the following command:
+```bash
+python -m synapse.config read <config key to print> -c <path to config>
+```
+
+To validate the entire file, omit `read <config key to print>`:
+```bash
+python -m synapse.config -c <path to config>
+```
+
+To see how to set other options, check the help reference:
+```bash
+python -m synapse.config --help
+```
+
+### YAML
+The configuration file is a [YAML](https://yaml.org/) file, which means that certain syntax rules
+apply if you want your config file to be read properly. A few helpful things to know:
+* `#` before any option in the config will comment out that setting and either a default (if available) will
+   be applied or Synapse will ignore the setting. Thus, in example #1 below, the setting will be read and
+   applied, but in example #2 the setting will not be read and a default will be applied.
+
+   Example #1:
+   ```yaml
+   pid_file: DATADIR/homeserver.pid
+   ```
+   Example #2:
+   ```yaml
+   #pid_file: DATADIR/homeserver.pid
+   ```
+* Indentation matters! The indentation before a setting
+  will determine whether a given setting is read as part of another
+  setting, or considered on its own. Thus, in example #1, the `enabled` setting
+  is read as a sub-option of the `presence` setting, and will be properly applied.
+
+  However, the lack of indentation before the `enabled` setting in example #2 means
+  that when reading the config, Synapse will consider both `presence` and `enabled` as
+  different settings. In this case, `presence` has no value, and thus a default applied, and `enabled`
+  is an option that Synapse doesn't recognize and thus ignores.
+
+  Example #1:
+  ```yaml
+  presence:
+    enabled: false
+  ```
+  Example #2:
+  ```yaml
+  presence:
+  enabled: false
+  ```
+  In this manual, all top-level settings (ones with no indentation) are identified
+  at the beginning of their section (i.e. "### `example_setting`") and
+  the sub-options, if any, are identified and listed in the body of the section.
+  In addition, each setting has an example of its usage, with the proper indentation
+  shown.
+"""
+SECTION_HEADERS = {
+    "modules": {
+        "title": "Modules",
+        "description": (
+            "Server admins can expand Synapse's functionality with external "
+            "modules.\n\n"
+            "See [here](../../modules/index.md) for more documentation on how "
+            "to configure or create custom modules for Synapse."
+        ),
+    },
+    "server_name": {
+        "title": "Server",
+        "description": "Define your homeserver name and other base options.",
+    },
+    "admin_contact": {
+        "title": "Homeserver blocking",
+        "description": "Useful options for Synapse admins.",
+    },
+    "tls_certificate_path": {
+        "title": "TLS",
+        "description": "Options related to TLS.",
+    },
+    "federation_domain_whitelist": {
+        "title": "Federation",
+        "description": "Options related to federation.",
+    },
+    "event_cache_size": {
+        "title": "Caching",
+        "description": "Options related to caching.",
+    },
+    "database": {
+        "title": "Database",
+        "description": "Config options related to database settings.",
+    },
+    "log_config": {
+        "title": "Logging",
+        "description": ("Config options related to logging."),
+    },
+    "rc_message": {
+        "title": "Ratelimiting",
+        "description": (
+            "Options related to ratelimiting in Synapse.\n\n"
+            "Each ratelimiting configuration is made of two parameters:\n"
+            "- `per_second`: number of requests a client can send per second.\n"
+            "- `burst_count`: number of requests a client can send before "
+            "being throttled."
+        ),
+    },
+    "enable_authenticated_media": {
+        "title": "Media Store",
+        "description": "Config options related to Synapse's media store.",
+    },
+    "recaptcha_public_key": {
+        "title": "Captcha",
+        "description": (
+            "See [here](../../CAPTCHA_SETUP.md) for full details on setting up captcha."
+        ),
+    },
+    "turn_uris": {
+        "title": "TURN",
+        "description": ("Options related to adding a TURN server to Synapse."),
+    },
+    "enable_registration": {
+        "title": "Registration",
+        "description": (
+            "Registration can be rate-limited using the parameters in the "
+            "[Ratelimiting](#ratelimiting) section of this manual."
+        ),
+    },
+    "session_lifetime": {
+        "title": "User session management",
+        "description": ("Config options related to user session management."),
+    },
+    "enable_metrics": {
+        "title": "Metrics",
+        "description": ("Config options related to metrics."),
+    },
+    "room_prejoin_state": {
+        "title": "API Configuration",
+        "description": ("Config settings related to the client/server API."),
+    },
+    "signing_key_path": {
+        "title": "Signing Keys",
+        "description": ("Config options relating to signing keys."),
+    },
+    "saml2_config": {
+        "title": "Single sign-on integration",
+        "description": (
+            "The following settings can be used to make Synapse use a single sign-on provider for authentication, instead of its internal password database.\n\n"
+            "You will probably also want to set the following options to `false` to disable the regular login/registration flows:\n"
+            "* [`enable_registration`](#enable_registration)\n"
+            "* [`password_config.enabled`](#password_config)"
+        ),
+    },
+    "push": {
+        "title": "Push",
+        "description": ("Configuration settings related to push notifications."),
+    },
+    "encryption_enabled_by_default_for_room_type": {
+        "title": "Rooms",
+        "description": ("Config options relating to rooms."),
+    },
+    "opentracing": {
+        "title": "Opentracing",
+        "description": ("Configuration options related to Opentracing support."),
+    },
+    "worker_replication_secret": {
+        "title": "Coordinating workers",
+        "description": (
+            "Configuration options related to workers which belong in the main config file (usually called `homeserver.yaml`). A Synapse deployment can scale horizontally by running multiple Synapse processes called _workers_. Incoming requests are distributed between workers to handle higher loads. Some workers are privileged and can accept requests from other workers.\n\n"
+            "As a result, the worker configuration is divided into two parts.\n\n"
+            "1. The first part (in this section of the manual) defines which shardable tasks are delegated to privileged workers. This allows unprivileged workers to make requests to a privileged worker to act on their behalf.\n"
+            "2. [The second part](#individual-worker-configuration) controls the behaviour of individual workers in isolation.\n\n"
+            "For guidance on setting up workers, see the [worker documentation](../../workers.md)."
+        ),
+    },
+    "worker_app": {
+        "title": "Individual worker configuration",
+        "description": (
+            "These options configure an individual worker, in its worker configuration file. They should be not be provided when configuring the main process.\n\n"
+            "Note also the configuration above for [coordinating a cluster of workers](#coordinating-workers).\n\n"
+            "For guidance on setting up workers, see the [worker documentation](../../workers.md)."
+        ),
+    },
+    "background_updates": {
+        "title": "Background Updates",
+        "description": ("Configuration settings related to background updates."),
+    },
+    "auto_accept_invites": {
+        "title": "Auto Accept Invites",
+        "description": (
+            "Configuration settings related to automatically accepting invites."
+        ),
+    },
+}
+INDENT = "  "
+
+
+has_error = False
+
+
+def error(text: str) -> None:
+    global has_error
+    print(f"ERROR: {text}", file=sys.stderr)
+    has_error = True
+
+
+def indent(text: str, first_line: bool = True) -> str:
+    """Indents each non-empty line of the given text."""
+    text = re.sub(r"(\n)([^\n])", r"\1" + INDENT + r"\2", text)
+    if first_line:
+        text = re.sub(r"^([^\n])", INDENT + r"\1", text)
+
+    return text
+
+
+def em(s: Optional[str]) -> str:
+    """Add emphasis to text."""
+    return f"*{s}*" if s else ""
+
+
+def a(s: Optional[str], suffix: str = " ") -> str:
+    """Appends a space if the given string is not empty."""
+    return s + suffix if s else ""
+
+
+def p(s: Optional[str], prefix: str = " ") -> str:
+    """Prepend a space if the given string is not empty."""
+    return prefix + s if s else ""
+
+
+def resolve_local_refs(schema: dict) -> dict:
+    """Returns the given schema with local $ref properties replaced by their keywords.
+
+    Crude approximation that will override keywords.
+    """
+    defs = schema["$defs"]
+
+    def replace_ref(d: Any) -> Any:
+        if isinstance(d, dict):
+            the_def = {}
+            if "$ref" in d:
+                # Found a "$ref" key.
+                def_name = d["$ref"].removeprefix("#/$defs/")
+                del d["$ref"]
+                the_def = defs[def_name]
+
+            new_dict = {k: replace_ref(v) for k, v in d.items()}
+            if common_keys := (new_dict.keys() & the_def.keys()) - {"properties"}:
+                print(
+                    f"WARN: '{def_name}' overrides keys '{common_keys}'",
+                    file=sys.stderr,
+                )
+
+            new_dict_props = new_dict.get("properties", {})
+            the_def_props = the_def.get("properties", {})
+            if common_props := new_dict_props.keys() & the_def_props.keys():
+                print(
+                    f"WARN: '{def_name}' overrides properties '{common_props}'",
+                    file=sys.stderr,
+                )
+            if merged_props := {**new_dict_props, **the_def_props}:
+                return {**new_dict, **the_def, "properties": merged_props}
+            else:
+                return {**new_dict, **the_def}
+
+        elif isinstance(d, list):
+            return [replace_ref(v) for v in d]
+        else:
+            return d
+
+    return replace_ref(schema)
+
+
+def sep(values: dict) -> str:
+    """Separator between parts of the description."""
+    # If description is multiple paragraphs already, add new ones. Otherwise
+    # append to same paragraph.
+    return "\n\n" if "\n\n" in values.get("description", "") else " "
+
+
+def type_str(values: dict) -> str:
+    """Type of the current value."""
+    if t := values.get("io.element.type_name"):
+        # Allow custom overrides for the type name, for documentation clarity
+        return f"({t})"
+    if not (t := values.get("type")):
+        return ""
+    if not isinstance(t, list):
+        t = [t]
+    joined = "|".join(t)
+    return f"({joined})"
+
+
+def items(values: dict) -> str:
+    """A block listing properties of array items."""
+    if not (items := values.get("items")):
+        return ""
+    if not (item_props := items.get("properties")):
+        return ""
+    return "\nOptions for each entry include:\n\n" + "\n".join(
+        sub_section(k, v) for k, v in item_props.items()
+    )
+
+
+def properties(values: dict) -> str:
+    """A block listing object properties."""
+    if not (properties := values.get("properties")):
+        return ""
+    return "\nThis setting has the following sub-options:\n\n" + "\n".join(
+        sub_section(k, v) for k, v in properties.items()
+    )
+
+
+def sub_section(prop: str, values: dict) -> str:
+    """Formats a bullet point about the given sub-property."""
+    sep = lambda: globals()["sep"](values)
+    type_str = lambda: globals()["type_str"](values)
+    items = lambda: globals()["items"](values)
+    properties = lambda: globals()["properties"](values)
+
+    def default() -> str:
+        try:
+            default = values["default"]
+            return f"Defaults to `{json.dumps(default)}`."
+        except KeyError:
+            return ""
+
+    def description() -> str:
+        if not (description := values.get("description")):
+            error(f"missing description for {prop}")
+            return "MISSING DESCRIPTION\n"
+
+        return f"{description}{p(default(), sep())}\n"
+
+    return (
+        f"* `{prop}`{p(type_str())}: "
+        + f"{indent(description(), first_line=False)}"
+        + indent(items())
+        + indent(properties())
+    )
+
+
+def section(prop: str, values: dict) -> str:
+    """Formats a section about the given property."""
+    sep = lambda: globals()["sep"](values)
+    type_str = lambda: globals()["type_str"](values)
+    items = lambda: globals()["items"](values)
+    properties = lambda: globals()["properties"](values)
+
+    def is_simple_default() -> bool:
+        """Whether the given default is simple enough for a one-liner."""
+        if not (d := values.get("default")):
+            return True
+        return not isinstance(d, dict) and not isinstance(d, list)
+
+    def default_str() -> str:
+        try:
+            default = values["default"]
+        except KeyError:
+            t = values.get("type", [])
+            if "object" == t or "object" in t:
+                # Skip objects as they probably have child defaults.
+                return ""
+            return "There is no default for this option."
+
+        if not is_simple_default():
+            # Show complex defaults as a code block instead.
+            return ""
+        return f"Defaults to `{json.dumps(default)}`."
+
+    def header() -> str:
+        try:
+            title = SECTION_HEADERS[prop]["title"]
+            description = SECTION_HEADERS[prop]["description"]
+            return f"## {title}\n\n{description}\n\n---\n"
+        except KeyError:
+            return ""
+
+    def title() -> str:
+        return f"### `{prop}`\n"
+
+    def description() -> str:
+        if not (description := values.get("description")):
+            error(f"missing description for {prop}")
+            return "MISSING DESCRIPTION\n"
+        return f"\n{a(em(type_str()))}{description}{p(default_str(), sep())}\n"
+
+    def example_str(example: Any) -> str:
+        return "```yaml\n" + f"{yaml.dump({prop: example}, sort_keys=False)}" + "```\n"
+
+    def default_example() -> str:
+        if is_simple_default():
+            return ""
+        default_cfg = example_str(values["default"])
+        return f"\nDefault configuration:\n{default_cfg}"
+
+    def examples() -> str:
+        if not (examples := values.get("examples")):
+            return ""
+
+        examples_str = "\n".join(example_str(e) for e in examples)
+
+        if len(examples) >= 2:
+            return f"\nExample configurations:\n{examples_str}"
+        else:
+            return f"\nExample configuration:\n{examples_str}"
+
+    def post_description() -> str:
+        # Sometimes it's helpful to have a description after the list of fields,
+        # e.g. with a subsection that consists only of text.
+        # This helps with that.
+        if not (description := values.get("io.element.post_description")):
+            return ""
+        return f"\n{description}\n\n"
+
+    return (
+        "---\n"
+        + header()
+        + title()
+        + description()
+        + items()
+        + properties()
+        + default_example()
+        + examples()
+        + post_description()
+    )
+
+
+def main() -> None:
+    def usage(err_msg: str) -> int:
+        script_name = (sys.argv[:1] or ["__main__.py"])[0]
+        print(err_msg, file=sys.stderr)
+        print(f"Usage: {script_name} <JSON Schema file>", file=sys.stderr)
+        print(f"\n{__doc__}", file=sys.stderr)
+        exit(1)
+
+    def read_json_file_arg() -> Any:
+        if len(sys.argv) > 2:
+            exit(usage("Too many arguments."))
+        if not (filepath := (sys.argv[1:] or [""])[0]):
+            exit(usage("No schema file provided."))
+        with open(filepath) as f:
+            return yaml.safe_load(f)
+
+    schema = read_json_file_arg()
+    schema = resolve_local_refs(schema)
+
+    sections = (section(k, v) for k, v in schema["properties"].items())
+    print(HEADER + "".join(sections), end="")
+
+    if has_error:
+        print("There were errors.", file=sys.stderr)
+        exit(2)
+
+
+if __name__ == "__main__":
+    main()
@@ -139,3 +139,6 @@ cargo-fmt

 # Ensure type hints are correct.
 mypy
+
+# Generate configuration documentation from the JSON Schema
+./scripts-dev/gen_config_documentation.py schema/synapse-config.schema.yaml > docs/usage/configuration/config_documentation.md
@@ -36,11 +36,11 @@ from typing import Any, List, Match, Optional, Union

 import attr
 import click
-import commonmark
 import git
 from click.exceptions import ClickException
 from git import GitCommandError, Repo
 from github import BadCredentialsException, Github
+from markdown_it import MarkdownIt
 from packaging import version


@@ -254,6 +254,12 @@ def _prepare() -> None:
    # Update the version specified in pyproject.toml.
    subprocess.check_output(["poetry", "version", new_version])

+    # Update config schema $id.
+    schema_file = "schema/synapse-config.schema.yaml"
+    major_minor_version = ".".join(new_version.split(".")[:2])
+    url = f"https://element-hq.github.io/synapse/schema/synapse/v{major_minor_version}/synapse-config.schema.json"
+    subprocess.check_output(["sed", "-i", f"0,/^\\$id: .*/s||$id: {url}|", schema_file])
+
    # Generate changelogs.
    generate_and_write_changelog(synapse_repo, current_version, new_version)

@@ -845,7 +851,7 @@ def get_changes_for_version(wanted_version: version.Version) -> str:

    # First we parse the changelog so that we can split it into sections based
    # on the release headings.
-    ast = commonmark.Parser().parse(changes)
+    tokens = MarkdownIt().parse(changes)

    @attr.s(auto_attribs=True)
    class VersionSection:
@@ -856,19 +862,22 @@ def get_changes_for_version(wanted_version: version.Version) -> str:
        end_line: Optional[int] = None  # Is none if its the last entry

    headings: List[VersionSection] = []
-    for node, _ in ast.walker():
-        # We look for all text nodes that are in a level 1 heading.
-        if node.t != "text":
+    for i, token in enumerate(tokens):
+        # We look for level 1 headings (h1 tags).
+        if token.type != "heading_open" or token.tag != "h1":
            continue

-        if node.parent.t != "heading" or node.parent.level != 1:
-            continue
+        # The next token should be an inline token containing the heading text
+        if i + 1 < len(tokens) and tokens[i + 1].type == "inline":
+            heading_text = tokens[i + 1].content
+            # The map property contains [line_begin, line_end] (0-based)
+            start_line = token.map[0] if token.map else 0

-        # If we have a previous heading then we update its `end_line`.
-        if headings:
-            headings[-1].end_line = node.parent.sourcepos[0][0] - 1
+            # If we have a previous heading then we update its `end_line`.
+            if headings:
+                headings[-1].end_line = start_line

-        headings.append(VersionSection(node.literal, node.parent.sourcepos[0][0] - 1))
+            headings.append(VersionSection(heading_text, start_line))

    changes_by_line = changes.split("\n")

@@ -37,7 +37,9 @@ from synapse.appservice import ApplicationService
 from synapse.http import get_request_user_agent
 from synapse.http.site import SynapseRequest
 from synapse.logging.opentracing import trace
+from synapse.state import CREATE_KEY, POWER_KEY
 from synapse.types import Requester, create_requester
+from synapse.types.state import StateFilter
 from synapse.util.cancellation import cancellable

 if TYPE_CHECKING:
@@ -216,18 +218,20 @@ class BaseAuth:
        # by checking if they would (theoretically) be able to change the
        # m.room.canonical_alias events

-        power_level_event = (
-            await self._storage_controllers.state.get_current_state_event(
-                room_id, EventTypes.PowerLevels, ""
-            )
+        auth_events = await self._storage_controllers.state.get_current_state(
+            room_id,
+            StateFilter.from_types(
+                [
+                    POWER_KEY,
+                    CREATE_KEY,
+                ]
+            ),
        )

-        auth_events = {}
-        if power_level_event:
-            auth_events[(EventTypes.PowerLevels, "")] = power_level_event
-
        send_level = event_auth.get_send_level(
-            EventTypes.CanonicalAlias, "", power_level_event
+            EventTypes.CanonicalAlias,
+            "",
+            auth_events.get(POWER_KEY),
        )
        user_level = event_auth.get_user_power_level(
            requester.user.to_string(), auth_events
@@ -29,6 +29,7 @@ from synapse.api.errors import (
    InvalidClientTokenError,
    MissingClientTokenError,
    UnrecognizedRequestError,
+    UserLockedError,
 )
 from synapse.http.site import SynapseRequest
 from synapse.logging.opentracing import active_span, force_tracing, start_active_span
@@ -162,12 +163,7 @@ class InternalAuth(BaseAuth):
                if not allow_locked and await self.store.get_user_locked_status(
                    requester.user.to_string()
                ):
-                    raise AuthError(
-                        401,
-                        "User account has been locked",
-                        errcode=Codes.USER_LOCKED,
-                        additional_fields={"soft_logout": True},
-                    )
+                    raise UserLockedError()

                # Deny the request if the user account has expired.
                # This check is only done for regular users, not appservice ones.
@@ -30,9 +30,6 @@ from authlib.oauth2.rfc7662 import IntrospectionToken
 from authlib.oidc.discovery import OpenIDProviderMetadata, get_well_known_url
 from prometheus_client import Histogram

-from twisted.web.client import readBody
-from twisted.web.http_headers import Headers
-
 from synapse.api.auth.base import BaseAuth
 from synapse.api.errors import (
    AuthError,
@@ -43,8 +40,14 @@ from synapse.api.errors import (
    UnrecognizedRequestError,
 )
 from synapse.http.site import SynapseRequest
-from synapse.logging.context import make_deferred_yieldable
-from synapse.logging.opentracing import active_span, force_tracing, start_active_span
+from synapse.logging.context import PreserveLoggingContext
+from synapse.logging.opentracing import (
+    active_span,
+    force_tracing,
+    inject_request_headers,
+    start_active_span,
+)
+from synapse.synapse_rust.http_client import HttpClient
 from synapse.types import Requester, UserID, create_requester
 from synapse.util import json_decoder
 from synapse.util.caches.cached_call import RetryOnExceptionCachedCall
@@ -179,6 +182,10 @@ class MSC3861DelegatedAuth(BaseAuth):
        self._admin_token: Callable[[], Optional[str]] = self._config.admin_token
        self._force_tracing_for_users = hs.config.tracing.force_tracing_for_users

+        self._rust_http_client = HttpClient(
+            user_agent=self._http_client.user_agent.decode("utf8")
+        )
+
        # # Token Introspection Cache
        # This remembers what users/devices are represented by which access tokens,
        # in order to reduce overall system load:
@@ -301,7 +308,6 @@ class MSC3861DelegatedAuth(BaseAuth):
        introspection_endpoint = await self._introspection_endpoint()
        raw_headers: Dict[str, str] = {
            "Content-Type": "application/x-www-form-urlencoded",
-            "User-Agent": str(self._http_client.user_agent, "utf-8"),
            "Accept": "application/json",
            # Tell MAS that we support reading the device ID as an explicit
            # value, not encoded in the scope. This is supported by MAS 0.15+
@@ -315,38 +321,34 @@ class MSC3861DelegatedAuth(BaseAuth):
        uri, raw_headers, body = self._client_auth.prepare(
            method="POST", uri=introspection_endpoint, headers=raw_headers, body=body
        )
-        headers = Headers({k: [v] for (k, v) in raw_headers.items()})

        # Do the actual request
-        # We're not using the SimpleHttpClient util methods as we don't want to
-        # check the HTTP status code, and we do the body encoding ourselves.

+        logger.debug("Fetching token from MAS")
        start_time = self._clock.time()
        try:
-            response = await self._http_client.request(
-                method="POST",
-                uri=uri,
-                data=body.encode("utf-8"),
-                headers=headers,
-            )
-
-            resp_body = await make_deferred_yieldable(readBody(response))
+            with start_active_span("mas-introspect-token"):
+                inject_request_headers(raw_headers)
+                with PreserveLoggingContext():
+                    resp_body = await self._rust_http_client.post(
+                        url=uri,
+                        response_limit=1 * 1024 * 1024,
+                        headers=raw_headers,
+                        request_body=body,
+                    )
+        except HttpResponseException as e:
+            end_time = self._clock.time()
+            introspection_response_timer.labels(e.code).observe(end_time - start_time)
+            raise
        except Exception:
            end_time = self._clock.time()
            introspection_response_timer.labels("ERR").observe(end_time - start_time)
            raise

-        end_time = self._clock.time()
-        introspection_response_timer.labels(response.code).observe(
-            end_time - start_time
-        )
+        logger.debug("Fetched token from MAS")

-        if response.code < 200 or response.code >= 300:
-            raise HttpResponseException(
-                response.code,
-                response.phrase.decode("ascii", errors="replace"),
-                resp_body,
-            )
+        end_time = self._clock.time()
+        introspection_response_timer.labels(200).observe(end_time - start_time)

        resp = json_decoder.decode(resp_body.decode("utf-8"))

@@ -475,7 +477,7 @@ class MSC3861DelegatedAuth(BaseAuth):
            # XXX: This is a temporary solution so that the admin API can be called by
            # the OIDC provider. This will be removed once we have OIDC client
            # credentials grant support in matrix-authentication-service.
-            logging.info("Admin toked used")
+            logger.info("Admin toked used")
            # XXX: that user doesn't exist and won't be provisioned.
            # This is mostly fine for admin calls, but we should also think about doing
            # requesters without a user_id.
@@ -185,12 +185,18 @@ ServerNoticeLimitReached: Final = "m.server_notice.usage_limit_reached"

 class UserTypes:
    """Allows for user type specific behaviour. With the benefit of hindsight
-    'admin' and 'guest' users should also be UserTypes. Normal users are type None
+    'admin' and 'guest' users should also be UserTypes. Extra user types can be
+    added in the configuration. Normal users are type None or one of the extra
+    user types (if configured).
    """

    SUPPORT: Final = "support"
    BOT: Final = "bot"
-    ALL_USER_TYPES: Final = (SUPPORT, BOT)
+    ALL_BUILTIN_USER_TYPES: Final = (SUPPORT, BOT)
+    """
+    The user types that are built-in to Synapse. Extra user types can be
+    added in the configuration.
+    """


 class RelationTypes:
@@ -256,6 +262,11 @@ class EventContentFields:

    TOMBSTONE_SUCCESSOR_ROOM: Final = "replacement_room"

+    # Used in m.room.topic events.
+    TOPIC: Final = "topic"
+    M_TOPIC: Final = "m.topic"
+    M_TEXT: Final = "m.text"
+

 class EventUnsignedContentFields:
    """Fields found inside the 'unsigned' data on events"""
@@ -264,6 +275,13 @@ class EventUnsignedContentFields:
    MEMBERSHIP: Final = "membership"


+class MTextFields:
+    """Fields found inside m.text content blocks."""
+
+    BODY: Final = "body"
+    MIMETYPE: Final = "mimetype"
+
+
 class RoomTypes:
    """Understood values of the room_type field of m.room.create events."""

@@ -280,6 +298,13 @@ class AccountDataTypes:
    IGNORED_USER_LIST: Final = "m.ignored_user_list"
    TAG: Final = "m.tag"
    PUSH_RULES: Final = "m.push_rules"
+    # MSC4155: Invite filtering
+    MSC4155_INVITE_PERMISSION_CONFIG: Final = (
+        "org.matrix.msc4155.invite_permission_config"
+    )
+    # Synapse-specific behaviour. See "Client-Server API Extensions" documentation
+    # in Admin API for more information.
+    SYNAPSE_ADMIN_CLIENT_CONFIG: Final = "io.element.synapse.admin_client_config"


 class HistoryVisibility:
@@ -137,6 +137,9 @@ class Codes(str, Enum):
    PROFILE_TOO_LARGE = "M_PROFILE_TOO_LARGE"
    KEY_TOO_LARGE = "M_KEY_TOO_LARGE"

+    # Part of MSC4155
+    INVITE_BLOCKED = "ORG.MATRIX.MSC4155.M_INVITE_BLOCKED"
+

 class CodeMessageException(RuntimeError):
    """An exception with integer code, a message string attributes and optional headers.
@@ -303,6 +306,20 @@ class UserDeactivatedError(SynapseError):
        )


+class UserLockedError(SynapseError):
+    """The error returned to the client when the user attempted to access an
+    authenticated endpoint, but the account has been locked.
+    """
+
+    def __init__(self) -> None:
+        super().__init__(
+            code=HTTPStatus.UNAUTHORIZED,
+            msg="User account has been locked",
+            errcode=Codes.USER_LOCKED,
+            additional_fields={"soft_logout": True},
+        )
+
+
 class FederationDeniedError(SynapseError):
    """An error raised when the server tries to federate with a server which
    is not on its federation whitelist.
@@ -524,7 +541,11 @@ class InvalidCaptchaError(SynapseError):


 class LimitExceededError(SynapseError):
-    """A client has sent too many requests and is being throttled."""
+    """A client has sent too many requests and is being throttled.
+
+    Args:
+        pause: Optional time in seconds to pause before responding to the client.
+    """

    def __init__(
        self,
@@ -532,6 +553,7 @@ class LimitExceededError(SynapseError):
        code: int = 429,
        retry_after_ms: Optional[int] = None,
        errcode: str = Codes.LIMIT_EXCEEDED,
+        pause: Optional[float] = None,
    ):
        # Use HTTP header Retry-After to enable library-assisted retry handling.
        headers = (
@@ -542,6 +564,7 @@ class LimitExceededError(SynapseError):
        super().__init__(code, "Too Many Requests", errcode, headers=headers)
        self.retry_after_ms = retry_after_ms
        self.limiter_name = limiter_name
+        self.pause = pause

    def error_dict(self, config: Optional["HomeServerConfig"]) -> "JsonDict":
        return cs_error(self.msg, self.errcode, retry_after_ms=self.retry_after_ms)
@@ -20,7 +20,7 @@
 #
 #

-from typing import Dict, Hashable, Optional, Tuple
+from typing import TYPE_CHECKING, Dict, Hashable, Optional, Tuple

 from synapse.api.errors import LimitExceededError
 from synapse.config.ratelimiting import RatelimitSettings
@@ -28,6 +28,12 @@ from synapse.storage.databases.main import DataStore
 from synapse.types import Requester
 from synapse.util import Clock

+if TYPE_CHECKING:
+    # To avoid circular imports:
+    from synapse.module_api.callbacks.ratelimit_callbacks import (
+        RatelimitModuleApiCallbacks,
+    )
+

 class Ratelimiter:
    """
@@ -72,12 +78,14 @@ class Ratelimiter:
        store: DataStore,
        clock: Clock,
        cfg: RatelimitSettings,
+        ratelimit_callbacks: Optional["RatelimitModuleApiCallbacks"] = None,
    ):
        self.clock = clock
        self.rate_hz = cfg.per_second
        self.burst_count = cfg.burst_count
        self.store = store
        self._limiter_name = cfg.key
+        self._ratelimit_callbacks = ratelimit_callbacks

        # A dictionary representing the token buckets tracked by this rate
        # limiter. Each entry maps a key of arbitrary type to a tuple representing:
@@ -165,6 +173,20 @@ class Ratelimiter:
            if override and not override.messages_per_second:
                return True, -1.0

+        if requester and self._ratelimit_callbacks:
+            # Check if the user has a custom rate limit for this specific limiter
+            # as returned by the module API.
+            module_override = (
+                await self._ratelimit_callbacks.get_ratelimit_override_for_user(
+                    requester.user.to_string(),
+                    self._limiter_name,
+                )
+            )
+
+            if module_override:
+                rate_hz = module_override.per_second
+                burst_count = module_override.burst_count
+
        # Override default values if set
        time_now_s = _time_now_s if _time_now_s is not None else self.clock.time()
        rate_hz = rate_hz if rate_hz is not None else self.rate_hz
@@ -316,12 +338,10 @@ class Ratelimiter:
        )

        if not allowed:
-            if pause:
-                await self.clock.sleep(pause)
-
            raise LimitExceededError(
                limiter_name=self._limiter_name,
                retry_after_ms=int(1000 * (time_allowed - time_now_s)),
+                pause=pause,
            )


@@ -36,12 +36,14 @@ class EventFormatVersions:
    ROOM_V1_V2 = 1  # $id:server event id format: used for room v1 and v2
    ROOM_V3 = 2  # MSC1659-style $hash event id format: used for room v3
    ROOM_V4_PLUS = 3  # MSC1884-style $hash format: introduced for room v4
+    ROOM_V11_PLUS_MSC4291 = 4  # MSC4291 room IDs as hashes: introduced for room MSC4291v11


 KNOWN_EVENT_FORMAT_VERSIONS = {
    EventFormatVersions.ROOM_V1_V2,
    EventFormatVersions.ROOM_V3,
    EventFormatVersions.ROOM_V4_PLUS,
+    EventFormatVersions.ROOM_V11_PLUS_MSC4291,
 }


@@ -109,6 +111,8 @@ class RoomVersion:
    msc3931_push_features: Tuple[str, ...]  # values from PushRuleRoomFlag
    # MSC3757: Restricting who can overwrite a state event
    msc3757_enabled: bool
+    # MSC4291: Room IDs as hashes of the create event
+    msc4291_room_ids_as_hashes: bool


 class RoomVersions:
@@ -131,6 +135,7 @@ class RoomVersions:
        enforce_int_power_levels=False,
        msc3931_push_features=(),
        msc3757_enabled=False,
+        msc4291_room_ids_as_hashes=False,
    )
    V2 = RoomVersion(
        "2",
@@ -151,6 +156,7 @@ class RoomVersions:
        enforce_int_power_levels=False,
        msc3931_push_features=(),
        msc3757_enabled=False,
+        msc4291_room_ids_as_hashes=False,
    )
    V3 = RoomVersion(
        "3",
@@ -171,6 +177,7 @@ class RoomVersions:
        enforce_int_power_levels=False,
        msc3931_push_features=(),
        msc3757_enabled=False,
+        msc4291_room_ids_as_hashes=False,
    )
    V4 = RoomVersion(
        "4",
@@ -191,6 +198,7 @@ class RoomVersions:
        enforce_int_power_levels=False,
        msc3931_push_features=(),
        msc3757_enabled=False,
+        msc4291_room_ids_as_hashes=False,
    )
    V5 = RoomVersion(
        "5",
@@ -211,6 +219,7 @@ class RoomVersions:
        enforce_int_power_levels=False,
        msc3931_push_features=(),
        msc3757_enabled=False,
+        msc4291_room_ids_as_hashes=False,
    )
    V6 = RoomVersion(
        "6",
@@ -231,6 +240,7 @@ class RoomVersions:
        enforce_int_power_levels=False,
        msc3931_push_features=(),
        msc3757_enabled=False,
+        msc4291_room_ids_as_hashes=False,
    )
    V7 = RoomVersion(
        "7",
@@ -251,6 +261,7 @@ class RoomVersions:
        enforce_int_power_levels=False,
        msc3931_push_features=(),
        msc3757_enabled=False,
+        msc4291_room_ids_as_hashes=False,
    )
    V8 = RoomVersion(
        "8",
@@ -271,6 +282,7 @@ class RoomVersions:
        enforce_int_power_levels=False,
        msc3931_push_features=(),
        msc3757_enabled=False,
+        msc4291_room_ids_as_hashes=False,
    )
    V9 = RoomVersion(
        "9",
@@ -291,6 +303,7 @@ class RoomVersions:
        enforce_int_power_levels=False,
        msc3931_push_features=(),
        msc3757_enabled=False,
+        msc4291_room_ids_as_hashes=False,
    )
    V10 = RoomVersion(
        "10",
@@ -311,6 +324,7 @@ class RoomVersions:
        enforce_int_power_levels=True,
        msc3931_push_features=(),
        msc3757_enabled=False,
+        msc4291_room_ids_as_hashes=False,
    )
    MSC1767v10 = RoomVersion(
        # MSC1767 (Extensible Events) based on room version "10"
@@ -332,6 +346,7 @@ class RoomVersions:
        enforce_int_power_levels=True,
        msc3931_push_features=(PushRuleRoomFlag.EXTENSIBLE_EVENTS,),
        msc3757_enabled=False,
+        msc4291_room_ids_as_hashes=False,
    )
    MSC3757v10 = RoomVersion(
        # MSC3757 (Restricting who can overwrite a state event) based on room version "10"
@@ -353,6 +368,7 @@ class RoomVersions:
        enforce_int_power_levels=True,
        msc3931_push_features=(),
        msc3757_enabled=True,
+        msc4291_room_ids_as_hashes=False,
    )
    V11 = RoomVersion(
        "11",
@@ -373,6 +389,7 @@ class RoomVersions:
        enforce_int_power_levels=True,
        msc3931_push_features=(),
        msc3757_enabled=False,
+        msc4291_room_ids_as_hashes=False,
    )
    MSC3757v11 = RoomVersion(
        # MSC3757 (Restricting who can overwrite a state event) based on room version "11"
@@ -394,6 +411,28 @@ class RoomVersions:
        enforce_int_power_levels=True,
        msc3931_push_features=(),
        msc3757_enabled=True,
+        msc4291_room_ids_as_hashes=False,
+    )
+    MSC4291v11 = RoomVersion(
+        "org.matrix.msc4291.11",
+        RoomDisposition.UNSTABLE,
+        EventFormatVersions.ROOM_V11_PLUS_MSC4291,
+        StateResolutionVersions.V2,
+        enforce_key_validity=True,
+        special_case_aliases_auth=False,
+        strict_canonicaljson=True,
+        limit_notifications_power_levels=True,
+        implicit_room_creator=True,  # Used by MSC3820
+        updated_redaction_rules=True,  # Used by MSC3820
+        restricted_join_rule=True,
+        restricted_join_rule_fix=True,
+        knock_join_rule=True,
+        msc3389_relation_redactions=False,
+        knock_restricted_join_rule=True,
+        enforce_int_power_levels=True,
+        msc3931_push_features=(),
+        msc3757_enabled=False,
+        msc4291_room_ids_as_hashes=True,
    )


@@ -413,6 +452,7 @@ KNOWN_ROOM_VERSIONS: Dict[str, RoomVersion] = {
        RoomVersions.V11,
        RoomVersions.MSC3757v10,
        RoomVersions.MSC3757v11,
+        RoomVersions.MSC4291v11,
    )
 }

@@ -445,8 +445,8 @@ def listen_http(
        # getHost() returns a UNIXAddress which contains an instance variable of 'name'
        # encoded as a byte string. Decode as utf-8 so pretty.
        logger.info(
-            "Synapse now listening on Unix Socket at: "
-            f"{ports[0].getHost().name.decode('utf-8')}"
+            "Synapse now listening on Unix Socket at: %s",
+            ports[0].getHost().name.decode("utf-8"),
        )

    return ports
@@ -118,7 +118,6 @@ class GenericWorkerStore(
    # FIXME(https://github.com/matrix-org/synapse/issues/3714): We need to add
    # UserDirectoryStore as we write directly rather than going via the correct worker.
    UserDirectoryStore,
-    StatsStore,
    UIAuthWorkerStore,
    EndToEndRoomKeyStore,
    PresenceStore,
@@ -154,6 +153,7 @@ class GenericWorkerStore(
    StreamWorkerStore,
    EventsWorkerStore,
    RegistrationWorkerStore,
+    StatsStore,
    SearchStore,
    TransactionWorkerStore,
    LockStore,
@@ -28,15 +28,13 @@ from prometheus_client import Gauge

 from synapse.metrics.background_process_metrics import wrap_as_background_process
 from synapse.types import JsonDict
+from synapse.util.constants import ONE_HOUR_SECONDS, ONE_MINUTE_SECONDS

 if TYPE_CHECKING:
    from synapse.server import HomeServer

 logger = logging.getLogger("synapse.app.homeserver")

-ONE_MINUTE_SECONDS = 60
-ONE_HOUR_SECONDS = 60 * ONE_MINUTE_SECONDS
-
 MILLISECONDS_PER_SECOND = 1000

 INITIAL_DELAY_BEFORE_FIRST_PHONE_HOME_SECONDS = 5 * ONE_MINUTE_SECONDS
@@ -173,7 +171,7 @@ async def phone_stats_home(
    stats["log_level"] = logging.getLevelName(log_level)

    logger.info(
-        "Reporting stats to %s: %s" % (hs.config.metrics.report_stats_endpoint, stats)
+        "Reporting stats to %s: %s", hs.config.metrics.report_stats_endpoint, stats
    )
    try:
        await hs.get_proxied_http_client().put_json(
@@ -555,6 +555,9 @@ class ApplicationServiceApi(SimpleHttpClient):
                        )
                        and service.is_interested_in_user(e.state_key)
                    ),
+                    # Appservices are considered 'trusted' by the admin and should have
+                    # applicable metadata on their events.
+                    include_admin_metadata=True,
                ),
            )
            for e in events
@@ -2,7 +2,7 @@
 # This file is licensed under the Affero General Public License (AGPL) version 3.
 #
 # Copyright 2015, 2016 OpenMarket Ltd
-# Copyright (C) 2023 New Vector, Ltd
+# Copyright (C) 2023, 2025 New Vector, Ltd
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License as
@@ -70,6 +70,8 @@ from typing import (
    Tuple,
 )

+from twisted.internet.interfaces import IDelayedCall
+
 from synapse.appservice import (
    ApplicationService,
    ApplicationServiceState,
@@ -450,6 +452,20 @@ class _TransactionController:
        recoverer.recover()
        logger.info("Now %i active recoverers", len(self.recoverers))

+    def force_retry(self, service: ApplicationService) -> None:
+        """Forces a Recoverer to attempt delivery of transations immediately.
+
+        Args:
+            service:
+        """
+        recoverer = self.recoverers.get(service.id)
+        if not recoverer:
+            # No need to force a retry on a happy AS.
+            logger.info("%s is not in recovery, not forcing retry", service.id)
+            return
+
+        recoverer.force_retry()
+
    async def _is_service_up(self, service: ApplicationService) -> bool:
        state = await self.store.get_appservice_state(service)
        return state == ApplicationServiceState.UP or state is None
@@ -482,11 +498,12 @@ class _Recoverer:
        self.service = service
        self.callback = callback
        self.backoff_counter = 1
+        self.scheduled_recovery: Optional[IDelayedCall] = None

    def recover(self) -> None:
        delay = 2**self.backoff_counter
        logger.info("Scheduling retries on %s in %fs", self.service.id, delay)
-        self.clock.call_later(
+        self.scheduled_recovery = self.clock.call_later(
            delay, run_as_background_process, "as-recoverer", self.retry
        )

@@ -496,6 +513,21 @@ class _Recoverer:
            self.backoff_counter += 1
        self.recover()

+    def force_retry(self) -> None:
+        """Cancels the existing timer and forces an immediate retry in the background.
+
+        Args:
+            service:
+        """
+        # Prevent the existing backoff from occuring
+        if self.scheduled_recovery:
+            self.clock.cancel_call_later(self.scheduled_recovery)
+        # Run a retry, which will resechedule a recovery if it fails.
+        run_as_background_process(
+            "retry",
+            self.retry,
+        )
+
    async def retry(self) -> None:
        logger.info("Starting retries on %s", self.service.id)
        try:
@@ -909,7 +909,10 @@ class RootConfig:


 def read_config_files(config_files: Iterable[str]) -> Dict[str, Any]:
-    """Read the config files into a dict
+    """Read the config files and shallowly merge them into a dict.
+
+    Successive configurations are shallowly merged into ones provided earlier,
+    i.e., entirely replacing top-level sections of the configuration.

    Args:
        config_files: A list of the config files to read
@@ -59,6 +59,7 @@ from synapse.config import (  # noqa: F401
    tls,
    tracer,
    user_directory,
+    user_types,
    voip,
    workers,
 )
@@ -122,6 +123,7 @@ class RootConfig:
    retention: retention.RetentionConfig
    background_updates: background_updates.BackgroundUpdateConfig
    auto_accept_invites: auto_accept_invites.AutoAcceptInvitesConfig
+    user_types: user_types.UserTypesConfig

    config_classes: List[Type["Config"]] = ...
    config_files: List[str]
@@ -23,7 +23,17 @@ from typing import Any

 from synapse.types import JsonDict

-from ._base import Config, ConfigError
+from ._base import Config, ConfigError, read_file
+
+CONFLICTING_RECAPTCHA_PRIVATE_KEY_OPTS_ERROR = """\
+You have configured both `recaptcha_private_key` and
+`recaptcha_private_key_path`. These are mutually incompatible.
+"""
+
+CONFLICTING_RECAPTCHA_PUBLIC_KEY_OPTS_ERROR = """\
+You have configured both `recaptcha_public_key` and `recaptcha_public_key_path`.
+These are mutually incompatible.
+"""


 class CaptchaConfig(Config):
@@ -38,6 +48,13 @@ class CaptchaConfig(Config):
                "Config options that expect an in-line secret as value are disabled",
                ("recaptcha_private_key",),
            )
+        recaptcha_private_key_path = config.get("recaptcha_private_key_path")
+        if recaptcha_private_key_path:
+            if recaptcha_private_key:
+                raise ConfigError(CONFLICTING_RECAPTCHA_PRIVATE_KEY_OPTS_ERROR)
+            recaptcha_private_key = read_file(
+                recaptcha_private_key_path, ("recaptcha_private_key_path",)
+            ).strip()
        if recaptcha_private_key is not None and not isinstance(
            recaptcha_private_key, str
        ):
@@ -50,6 +67,13 @@ class CaptchaConfig(Config):
                "Config options that expect an in-line secret as value are disabled",
                ("recaptcha_public_key",),
            )
+        recaptcha_public_key_path = config.get("recaptcha_public_key_path")
+        if recaptcha_public_key_path:
+            if recaptcha_public_key:
+                raise ConfigError(CONFLICTING_RECAPTCHA_PUBLIC_KEY_OPTS_ERROR)
+            recaptcha_public_key = read_file(
+                recaptcha_public_key_path, ("recaptcha_public_key_path",)
+            ).strip()
        if recaptcha_public_key is not None and not isinstance(
            recaptcha_public_key, str
        ):
@@ -561,8 +561,23 @@ class ExperimentalConfig(Config):
        # MSC4076: Add `disable_badge_count`` to pusher configuration
        self.msc4076_enabled: bool = experimental.get("msc4076_enabled", False)

+        # MSC4277: Harmonizing the reporting endpoints
+        #
+        # If enabled, ignore the score parameter and respond with HTTP 200 on
+        # reporting requests regardless of the subject's existence.
+        self.msc4277_enabled: bool = experimental.get("msc4277_enabled", False)
+
+        # MSC4235: Add `via` param to hierarchy endpoint
+        self.msc4235_enabled: bool = experimental.get("msc4235_enabled", False)
+
        # MSC4263: Preventing MXID enumeration via key queries
        self.msc4263_limit_key_queries_to_users_who_share_rooms = experimental.get(
            "msc4263_limit_key_queries_to_users_who_share_rooms",
            False,
        )
+
+        # MSC4267: Automatically forgetting rooms on leave
+        self.msc4267_enabled: bool = experimental.get("msc4267_enabled", False)
+
+        # MSC4155: Invite filtering
+        self.msc4155_enabled: bool = experimental.get("msc4155_enabled", False)
@@ -94,5 +94,21 @@ class FederationConfig(Config):
            2**62,
        )

+    def is_domain_allowed_according_to_federation_whitelist(self, domain: str) -> bool:
+        """
+        Returns whether a domain is allowed according to the federation whitelist. If a
+        federation whitelist is not set, all domains are allowed.
+
+        Args:
+            domain: The domain to test.
+
+        Returns:
+            True if the domain is allowed or if a whitelist is not set, False otherwise.
+        """
+        if self.federation_domain_whitelist is None:
+            return True
+
+        return domain in self.federation_domain_whitelist
+

 _METRICS_FOR_DOMAINS_SCHEMA = {"type": "array", "items": {"type": "string"}}
@@ -59,6 +59,7 @@ from .third_party_event_rules import ThirdPartyRulesConfig
 from .tls import TlsConfig
 from .tracer import TracerConfig
 from .user_directory import UserDirectoryConfig
+from .user_types import UserTypesConfig
 from .voip import VoipConfig
 from .workers import WorkerConfig

@@ -107,4 +108,5 @@ class HomeServerConfig(RootConfig):
        ExperimentalConfig,
        BackgroundUpdateConfig,
        AutoAcceptInvitesConfig,
+        UserTypesConfig,
    ]
@@ -51,6 +51,8 @@ if TYPE_CHECKING:
    from synapse.config.homeserver import HomeServerConfig
    from synapse.server import HomeServer

+logger = logging.getLogger(__name__)
+
 DEFAULT_LOG_CONFIG = Template(
    """\
 # Log configuration for Synapse.
@@ -291,7 +293,7 @@ def _load_logging_config(log_config_path: str) -> None:
        log_config = yaml.safe_load(f.read())

    if not log_config:
-        logging.warning("Loaded a blank logging config?")
+        logger.warning("Loaded a blank logging config?")

    # If the old structured logging configuration is being used, raise an error.
    if "structured" in log_config and log_config.get("structured"):
@@ -312,7 +314,7 @@ def _reload_logging_config(log_config_path: Optional[str]) -> None:
        return

    _load_logging_config(log_config_path)
-    logging.info("Reloaded log config from %s due to SIGHUP", log_config_path)
+    logger.info("Reloaded log config from %s due to SIGHUP", log_config_path)


 def setup_logging(
@@ -349,17 +351,17 @@ def setup_logging(
    appbase.register_sighup(_reload_logging_config, log_config_path)

    # Log immediately so we can grep backwards.
-    logging.warning("***** STARTING SERVER *****")
-    logging.warning(
+    logger.warning("***** STARTING SERVER *****")
+    logger.warning(
        "Server %s version %s",
        sys.argv[0],
        SYNAPSE_VERSION,
    )
-    logging.warning("Copyright (c) 2023 New Vector, Inc")
-    logging.warning(
+    logger.warning("Copyright (c) 2023 New Vector, Inc")
+    logger.warning(
        "Licensed under the AGPL 3.0 license. Website: https://github.com/element-hq/synapse"
    )
-    logging.info("Server hostname: %s", config.server.server_name)
-    logging.info("Public Base URL: %s", config.server.public_baseurl)
-    logging.info("Instance name: %s", hs.get_instance_name())
-    logging.info("Twisted reactor: %s", type(reactor).__name__)
+    logger.info("Server hostname: %s", config.server.server_name)
+    logger.info("Public Base URL: %s", config.server.public_baseurl)
+    logger.info("Instance name: %s", hs.get_instance_name())
+    logger.info("Twisted reactor: %s", type(reactor).__name__)
@@ -240,3 +240,9 @@ class RatelimitConfig(Config):
            "rc_delayed_event_mgmt",
            defaults={"per_second": 1, "burst_count": 5},
        )
+
+        self.rc_reports = RatelimitSettings.parse(
+            config,
+            "rc_reports",
+            defaults={"per_second": 1, "burst_count": 5},
+        )
@@ -119,6 +119,15 @@ def parse_thumbnail_requirements(
    }


+@attr.s(auto_attribs=True, slots=True, frozen=True)
+class MediaUploadLimit:
+    """A limit on the amount of data a user can upload in a given time
+    period."""
+
+    max_bytes: int
+    time_period_ms: int
+
+
 class ContentRepositoryConfig(Config):
    section = "media"

@@ -274,6 +283,13 @@ class ContentRepositoryConfig(Config):

        self.enable_authenticated_media = config.get("enable_authenticated_media", True)

+        self.media_upload_limits: List[MediaUploadLimit] = []
+        for limit_config in config.get("media_upload_limits", []):
+            time_period_ms = self.parse_duration(limit_config["time_period"])
+            max_bytes = self.parse_size(limit_config["max_size"])
+
+            self.media_upload_limits.append(MediaUploadLimit(max_bytes, time_period_ms))
+
    def generate_config_section(self, data_dir_path: str, **kwargs: Any) -> str:
        assert data_dir_path is not None
        media_store = os.path.join(data_dir_path, "media_store")
@@ -27,7 +27,7 @@ from synapse.types import JsonDict

 from ._base import Config, ConfigError

-logger = logging.Logger(__name__)
+logger = logging.getLogger(__name__)


 class RoomDefaultEncryptionTypes:
@@ -85,4 +85,4 @@ class RoomConfig(Config):

        # When enabled, users will forget rooms when they leave them, either via a
        # leave, kick or ban.
-        self.forget_on_leave = config.get("forget_rooms_on_leave", False)
+        self.forget_on_leave: bool = config.get("forget_rooms_on_leave", False)
@@ -41,7 +41,7 @@ from synapse.util.stringutils import parse_and_validate_server_name
 from ._base import Config, ConfigError
 from ._util import validate_config

-logger = logging.Logger(__name__)
+logger = logging.getLogger(__name__)

 DIRECT_TCP_ERROR = """
 Using direct TCP replication for workers is no longer supported.
@@ -0,0 +1,44 @@
+#
+# This file is licensed under the Affero General Public License (AGPL) version 3.
+#
+# Copyright (C) 2025 New Vector, Ltd
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as
+# published by the Free Software Foundation, either version 3 of the
+# License, or (at your option) any later version.
+#
+# See the GNU Affero General Public License for more details:
+# <https://www.gnu.org/licenses/agpl-3.0.html>.
+#
+
+from typing import Any, List, Optional
+
+from synapse.api.constants import UserTypes
+from synapse.types import JsonDict
+
+from ._base import Config, ConfigError
+
+
+class UserTypesConfig(Config):
+    section = "user_types"
+
+    def read_config(self, config: JsonDict, **kwargs: Any) -> None:
+        user_types: JsonDict = config.get("user_types", {})
+
+        self.default_user_type: Optional[str] = user_types.get(
+            "default_user_type", None
+        )
+        self.extra_user_types: List[str] = user_types.get("extra_user_types", [])
+
+        all_user_types: List[str] = []
+        all_user_types.extend(UserTypes.ALL_BUILTIN_USER_TYPES)
+        all_user_types.extend(self.extra_user_types)
+
+        self.all_user_types = all_user_types
+
+        if self.default_user_type is not None:
+            if self.default_user_type not in all_user_types:
+                raise ConfigError(
+                    f"Default user type {self.default_user_type} is not in the list of all user types: {all_user_types}"
+                )
@@ -27,8 +27,6 @@ from typing import Any, Dict, List, Optional, Union
 import attr

 from synapse._pydantic_compat import (
-    BaseModel,
-    Extra,
    StrictBool,
    StrictInt,
    StrictStr,
@@ -47,6 +45,7 @@ from synapse.config.server import (
    parse_listener_def,
 )
 from synapse.types import JsonDict
+from synapse.util.pydantic_models import ParseModel

 _DEPRECATED_WORKER_DUTY_OPTION_USED = """
 The '%s' configuration option is deprecated and will be removed in a future
@@ -90,30 +89,7 @@ def _instance_to_list_converter(obj: Union[str, List[str]]) -> List[str]:
    return obj


-class ConfigModel(BaseModel):
-    """A custom version of Pydantic's BaseModel which
-
-     - ignores unknown fields and
-     - does not allow fields to be overwritten after construction,
-
-    but otherwise uses Pydantic's default behaviour.
-
-    For now, ignore unknown fields. In the future, we could change this so that unknown
-    config values cause a ValidationError, provided the error messages are meaningful to
-    server operators.
-
-    Subclassing in this way is recommended by
-    https://pydantic-docs.helpmanual.io/usage/model_config/#change-behaviour-globally
-    """
-
-    class Config:
-        # By default, ignore fields that we don't recognise.
-        extra = Extra.ignore
-        # By default, don't allow fields to be reassigned after parsing.
-        allow_mutation = False
-
-
-class InstanceTcpLocationConfig(ConfigModel):
+class InstanceTcpLocationConfig(ParseModel):
    """The host and port to talk to an instance via HTTP replication."""

    host: StrictStr
@@ -129,7 +105,7 @@ class InstanceTcpLocationConfig(ConfigModel):
        return f"{self.host}:{self.port}"


-class InstanceUnixLocationConfig(ConfigModel):
+class InstanceUnixLocationConfig(ParseModel):
    """The socket file to talk to an instance via HTTP replication."""

    path: StrictStr
@@ -101,6 +101,9 @@ def compute_content_hash(
    event_dict.pop("outlier", None)
    event_dict.pop("destinations", None)

+    # N.B. no need to pop the room_id from create events in MSC4291 rooms
+    # as they shouldn't have one.
+
    event_json_bytes = encode_canonical_json(event_dict)

    hashed = hash_algorithm(event_json_bytes)
@@ -64,6 +64,7 @@ from synapse.api.room_versions import (
    RoomVersion,
    RoomVersions,
 )
+from synapse.state import CREATE_KEY
 from synapse.storage.databases.main.events_worker import EventRedactBehaviour
 from synapse.types import (
    MutableStateMap,
@@ -260,7 +261,8 @@ async def check_state_independent_auth_rules(
                f"Event {event.event_id} has unexpected auth_event for {k}: {auth_event_id}",
            )

-        # We also need to check that the auth event itself is not rejected.
+        # 2.3 ... If there are entries which were themselves rejected under the checks performed on receipt
+        # of a PDU, reject.
        if auth_event.rejected_reason:
            raise AuthError(
                403,
@@ -270,7 +272,7 @@ async def check_state_independent_auth_rules(

        auth_dict[k] = auth_event_id

-    # 3. If event does not have a m.room.create in its auth_events, reject.
+    # 2.4. If event does not have a m.room.create in its auth_events, reject.
    creation_event = auth_dict.get((EventTypes.Create, ""), None)
    if not creation_event:
        raise AuthError(403, "No create event in auth events")
@@ -308,8 +310,16 @@ def check_state_dependent_auth_rules(

    auth_dict = {(e.type, e.state_key): e for e in auth_events}

+    # Later code relies on there being a create event e.g _can_federate, _is_membership_change_allowed
+    # so produce a more intelligible error if we don't have one.
+    create_event = auth_dict.get(CREATE_KEY)
+    if create_event is None:
+        raise AuthError(
+            403, f"Event {event.event_id} is missing a create event in auth_events."
+        )
+
    # additional check for m.federate
-    creating_domain = get_domain_from_id(event.room_id)
+    creating_domain = get_domain_from_id(create_event.sender)
    originating_domain = get_domain_from_id(event.sender)
    if creating_domain != originating_domain:
        if not _can_federate(event, auth_dict):
@@ -462,12 +472,20 @@ def _check_create(event: "EventBase") -> None:
    if event.prev_event_ids():
        raise AuthError(403, "Create event has prev events")

-    # 1.2 If the domain of the room_id does not match the domain of the sender,
-    # reject.
-    sender_domain = get_domain_from_id(event.sender)
-    room_id_domain = get_domain_from_id(event.room_id)
-    if room_id_domain != sender_domain:
-        raise AuthError(403, "Creation event's room_id domain does not match sender's")
+    if event.room_version.msc4291_room_ids_as_hashes:
+        # 1.2 If the create event has a room_id, reject
+        if "room_id" in event:
+            raise AuthError(403, "Create event has a room_id")
+    else:
+        # 1.2 If the domain of the room_id does not match the domain of the sender,
+        # reject.
+        if not event.room_version.msc4291_room_ids_as_hashes:
+            sender_domain = get_domain_from_id(event.sender)
+            room_id_domain = get_domain_from_id(event.room_id)
+            if room_id_domain != sender_domain:
+                raise AuthError(
+                    403, "Creation event's room_id domain does not match sender's"
+                )

    # 1.3 If content.room_version is present and is not a recognised version, reject
    room_version_prop = event.content.get("room_version", "1")
@@ -525,7 +543,13 @@ def _is_membership_change_allowed(

    target_user_id = event.state_key

-    creating_domain = get_domain_from_id(event.room_id)
+    # We need the create event in order to check if we can federate or not.
+    # If it's missing, yell loudly. Previously we only did this inside the
+    # _can_federate check.
+    create_event = auth_events.get((EventTypes.Create, ""))
+    if not create_event:
+        raise AuthError(403, "Create event missing from auth_events")
+    creating_domain = get_domain_from_id(create_event.sender)
    target_domain = get_domain_from_id(target_user_id)
    if creating_domain != target_domain:
        if not _can_federate(event, auth_events):
@@ -1010,11 +1034,16 @@ def get_user_power_level(user_id: str, auth_events: StateMap["EventBase"]) -> in
        user_id: user's id to look up in power_levels
        auth_events:
            state in force at this point in the room (or rather, a subset of
-            it including at least the create event and power levels event.
+            it including at least the create event, and possibly a power levels event).

    Returns:
        the user's power level in this room.
    """
+    create_event = auth_events.get(CREATE_KEY)
+    assert create_event is not None, (
+        "A create event in the auth events chain is required to calculate user power level correctly,"
+        " but was not found. This indicates a bug"
+    )
    power_level_event = get_power_level_event(auth_events)
    if power_level_event:
        level = power_level_event.content.get("users", {}).get(user_id)
@@ -1028,18 +1057,12 @@ def get_user_power_level(user_id: str, auth_events: StateMap["EventBase"]) -> in
    else:
        # if there is no power levels event, the creator gets 100 and everyone
        # else gets 0.
-
-        # some things which call this don't pass the create event: hack around
-        # that.
-        key = (EventTypes.Create, "")
-        create_event = auth_events.get(key)
-        if create_event is not None:
-            if create_event.room_version.implicit_room_creator:
-                creator = create_event.sender
-            else:
-                creator = create_event.content[EventContentFields.ROOM_CREATOR]
-            if creator == user_id:
-                return 100
+        if create_event.room_version.implicit_room_creator:
+            creator = create_event.sender
+        else:
+            creator = create_event.content[EventContentFields.ROOM_CREATOR]
+        if creator == user_id:
+            return 100
        return 0


@@ -44,7 +44,10 @@ from unpaddedbase64 import encode_base64
 from synapse.api.constants import EventTypes, RelationTypes
 from synapse.api.room_versions import EventFormatVersions, RoomVersion, RoomVersions
 from synapse.synapse_rust.events import EventInternalMetadata
-from synapse.types import JsonDict, StrCollection
+from synapse.types import (
+    JsonDict,
+    StrCollection,
+)
 from synapse.util.caches import intern_dict
 from synapse.util.frozenutils import freeze

@@ -208,9 +211,7 @@ class EventBase(metaclass=abc.ABCMeta):
    depth: DictProperty[int] = DictProperty("depth")
    content: DictProperty[JsonDict] = DictProperty("content")
    hashes: DictProperty[Dict[str, str]] = DictProperty("hashes")
-    origin: DictProperty[str] = DictProperty("origin")
    origin_server_ts: DictProperty[int] = DictProperty("origin_server_ts")
-    room_id: DictProperty[str] = DictProperty("room_id")
    sender: DictProperty[str] = DictProperty("sender")
    # TODO state_key should be Optional[str]. This is generally asserted in Synapse
    # by calling is_state() first (which ensures it is not None), but it is hard (not possible?)
@@ -225,6 +226,10 @@ class EventBase(metaclass=abc.ABCMeta):
    def event_id(self) -> str:
        raise NotImplementedError()

+    @property
+    def room_id(self) -> str:
+        raise NotImplementedError()
+
    @property
    def membership(self) -> str:
        return self.content["membership"]
@@ -387,6 +392,10 @@ class FrozenEvent(EventBase):
    def event_id(self) -> str:
        return self._event_id

+    @property
+    def room_id(self) -> str:
+        return self._dict["room_id"]
+

 class FrozenEventV2(EventBase):
    format_version = EventFormatVersions.ROOM_V3  # All events of this type are V2
@@ -444,6 +453,10 @@ class FrozenEventV2(EventBase):
        self._event_id = "$" + encode_base64(compute_event_reference_hash(self)[1])
        return self._event_id

+    @property
+    def room_id(self) -> str:
+        return self._dict["room_id"]
+
    def prev_event_ids(self) -> List[str]:
        """Returns the list of prev event IDs. The order matches the order
        specified in the event, though there is no meaning to it.
@@ -482,6 +495,67 @@ class FrozenEventV3(FrozenEventV2):
        return self._event_id


+class FrozenEventV4(FrozenEventV3):
+    """FrozenEventV4 for MSC4291 room IDs are hashes"""
+
+    format_version = EventFormatVersions.ROOM_V11_PLUS_MSC4291
+
+    """Override the room_id for m.room.create events"""
+
+    def __init__(
+        self,
+        event_dict: JsonDict,
+        room_version: RoomVersion,
+        internal_metadata_dict: Optional[JsonDict] = None,
+        rejected_reason: Optional[str] = None,
+    ):
+        super().__init__(
+            event_dict=event_dict,
+            room_version=room_version,
+            internal_metadata_dict=internal_metadata_dict,
+            rejected_reason=rejected_reason,
+        )
+        self._room_id: Optional[str] = None
+
+    @property
+    def room_id(self) -> str:
+        # if we have calculated the room ID already, don't do it again.
+        if self._room_id:
+            return self._room_id
+
+        is_create_event = self.type == EventTypes.Create and self.get_state_key() == ""
+
+        # for non-create events: use the supplied value from the JSON, as per FrozenEventV3
+        if not is_create_event:
+            self._room_id = self._dict["room_id"]
+            assert self._room_id is not None
+            return self._room_id
+
+        # for create events: calculate the room ID
+        from synapse.crypto.event_signing import compute_event_reference_hash
+
+        self._room_id = "!" + encode_base64(
+            compute_event_reference_hash(self)[1], urlsafe=True
+        )
+        return self._room_id
+
+    def auth_event_ids(self) -> StrCollection:
+        """Returns the list of auth event IDs. The order matches the order
+        specified in the event, though there is no meaning to it.
+        Returns:
+            The list of event IDs of this event's auth_events
+            Includes the creation event ID for convenience of all the codepaths
+            which expects the auth chain to include the creator ID, even though
+            it's explicitly not included on the wire. Excludes the create event
+            for the create event itself.
+        """
+        create_event_id = "$" + self.room_id[1:]
+        assert create_event_id not in self._dict["auth_events"]
+        if self.type == EventTypes.Create and self.get_state_key() == "":
+            return self._dict["auth_events"]  # should be []
+        return self._dict["auth_events"] + [create_event_id]
+
+
 def _event_type_from_format_version(
    format_version: int,
 ) -> Type[Union[FrozenEvent, FrozenEventV2, FrozenEventV3]]:
@@ -501,6 +575,8 @@ def _event_type_from_format_version(
        return FrozenEventV2
    elif format_version == EventFormatVersions.ROOM_V4_PLUS:
        return FrozenEventV3
+    elif format_version == EventFormatVersions.ROOM_V11_PLUS_MSC4291:
+        return FrozenEventV4
    else:
        raise Exception("No event format %r" % (format_version,))

@@ -195,15 +195,18 @@ class InviteAutoAccepter:
            except SynapseError as e:
                if e.code == HTTPStatus.FORBIDDEN:
                    logger.debug(
-                        f"Update_room_membership was forbidden. This can sometimes be expected for remote invites. Exception: {e}"
+                        "Update_room_membership was forbidden. This can sometimes be expected for remote invites. Exception: %s",
+                        e,
                    )
                else:
-                    logger.warn(
-                        f"Update_room_membership raised the following unexpected (SynapseError) exception: {e}"
+                    logger.warning(
+                        "Update_room_membership raised the following unexpected (SynapseError) exception: %s",
+                        e,
                    )
            except Exception as e:
-                logger.warn(
-                    f"Update_room_membership raised the following unexpected exception: {e}"
+                logger.warning(
+                    "Update_room_membership raised the following unexpected exception: %s",
+                    e,
                )

            sleep = 2**retries
@@ -82,7 +82,8 @@ class EventBuilder:

    room_version: RoomVersion

-    room_id: str
+    # MSC4291 makes the room ID == the create event ID. This means the create event has no room_id.
+    room_id: Optional[str]
    type: str
    sender: str

@@ -142,7 +143,14 @@ class EventBuilder:
        Returns:
            The signed and hashed event.
        """
+        # Create events always have empty auth_events.
+        if self.type == EventTypes.Create and self.is_state() and self.state_key == "":
+            auth_event_ids = []
+
+        # Calculate auth_events for non-create events
        if auth_event_ids is None:
+            # Every non-create event must have a room ID
+            assert self.room_id is not None
            state_ids = await self._state.compute_state_after_events(
                self.room_id,
                prev_event_ids,
@@ -224,12 +232,31 @@ class EventBuilder:
            "auth_events": auth_events,
            "prev_events": prev_events,
            "type": self.type,
-            "room_id": self.room_id,
            "sender": self.sender,
            "content": self.content,
            "unsigned": self.unsigned,
            "depth": depth,
        }
+        if self.room_id is not None:
+            event_dict["room_id"] = self.room_id
+
+        if self.room_version.msc4291_room_ids_as_hashes:
+            # In MSC4291: the create event has no room ID as the create event ID /is/ the room ID.
+            if (
+                self.type == EventTypes.Create
+                and self.is_state()
+                and self._state_key == ""
+            ):
+                assert self.room_id is None
+            else:
+                # All other events do not reference the create event in auth_events, as the room ID
+                # /is/ the create event. However, the rest of the code (for consistency between room
+                # versions) assume that the create event remains part of the auth events. c.f. event
+                # class which automatically adds the create event when `.auth_event_ids()` is called
+                assert self.room_id is not None
+                create_event_id = "$" + self.room_id[1:]
+                auth_event_ids.remove(create_event_id)
+                event_dict["auth_events"] = auth_event_ids

        if self.is_state():
            event_dict["state_key"] = self._state_key
@@ -285,7 +312,7 @@ class EventBuilderFactory:
            room_version=room_version,
            type=key_values["type"],
            state_key=key_values.get("state_key"),
-            room_id=key_values["room_id"],
+            room_id=key_values.get("room_id"),
            sender=key_values["sender"],
            content=key_values.get("content", {}),
            unsigned=key_values.get("unsigned", {}),
@@ -302,8 +329,8 @@ def create_local_event_from_event_dict(
    event_dict: JsonDict,
    internal_metadata_dict: Optional[JsonDict] = None,
 ) -> EventBase:
-    """Takes a fully formed event dict, ensuring that fields like `origin`
-    and `origin_server_ts` have correct values for a locally produced event,
+    """Takes a fully formed event dict, ensuring that fields like
+    `origin_server_ts` have correct values for a locally produced event,
    then signs and hashes it.
    """

@@ -319,7 +346,6 @@ def create_local_event_from_event_dict(
    if format_version == EventFormatVersions.ROOM_V1_V2:
        event_dict["event_id"] = _create_event_id(clock, hostname)

-    event_dict["origin"] = hostname
    event_dict.setdefault("origin_server_ts", time_now)

    event_dict.setdefault("unsigned", {})
@@ -176,9 +176,12 @@ def prune_event_dict(room_version: RoomVersion, event_dict: JsonDict) -> JsonDic
        if room_version.updated_redaction_rules:
            # MSC2176 rules state that create events cannot have their `content` redacted.
            new_content = event_dict["content"]
-        elif not room_version.implicit_room_creator:
+        if not room_version.implicit_room_creator:
            # Some room versions give meaning to `creator`
            add_fields("creator")
+        if room_version.msc4291_room_ids_as_hashes:
+            # room_id is not allowed on the create event as it's derived from the event ID
+            allowed_keys.remove("room_id")

    elif event_type == EventTypes.JoinRules:
        add_fields("join_rule")
@@ -421,11 +424,21 @@ class SerializeEventConfig:
    # False, that state will be removed from the event before it is returned.
    # Otherwise, it will be kept.
    include_stripped_room_state: bool = False
+    # When True, sets unsigned fields to help clients identify events which
+    # only server admins can see through other configuration. For example,
+    # whether an event was soft failed by the server.
+    include_admin_metadata: bool = False


 _DEFAULT_SERIALIZE_EVENT_CONFIG = SerializeEventConfig()


+def make_config_for_admin(existing: SerializeEventConfig) -> SerializeEventConfig:
+    # Set the options which are only available to server admins,
+    # and copy the rest.
+    return attr.evolve(existing, include_admin_metadata=True)
+
+
 def serialize_event(
    e: Union[JsonDict, EventBase],
    time_now_ms: int,
@@ -528,6 +541,9 @@ def serialize_event(
            d["content"] = dict(d["content"])
            d["content"]["redacts"] = e.redacts

+    if config.include_admin_metadata and e.internal_metadata.is_soft_failed():
+        d["unsigned"]["io.element.synapse.soft_failed"] = True
+
    only_event_fields = config.only_event_fields
    if only_event_fields:
        if not isinstance(only_event_fields, list) or not all(
@@ -548,6 +564,7 @@ class EventClientSerializer:

    def __init__(self, hs: "HomeServer") -> None:
        self._store = hs.get_datastores().main
+        self._auth = hs.get_auth()
        self._add_extra_fields_to_unsigned_client_event_callbacks: List[
            ADD_EXTRA_FIELDS_TO_UNSIGNED_CLIENT_EVENT_CALLBACK
        ] = []
@@ -576,6 +593,15 @@ class EventClientSerializer:
        if not isinstance(event, EventBase):
            return event

+        # Force-enable server admin metadata because the only time an event with
+        # relevant metadata will be when the admin requested it via their admin
+        # client config account data. Also, it's "just" some `unsigned` fields, so
+        # shouldn't cause much in terms of problems to downstream consumers.
+        if config.requester is not None and await self._auth.is_server_admin(
+            config.requester
+        ):
+            config = make_config_for_admin(config)
+
        serialized_event = serialize_event(event, time_now, config=config)

        new_unsigned = {}
@@ -67,7 +67,6 @@ class EventValidator:
            "auth_events",
            "content",
            "hashes",
-            "origin",
            "prev_events",
            "sender",
            "type",
@@ -77,13 +76,6 @@ class EventValidator:
            if k not in event:
                raise SynapseError(400, "Event does not have key %s" % (k,))

-        # Check that the following keys have string values
-        event_strings = ["origin"]
-
-        for s in event_strings:
-            if not isinstance(getattr(event, s), str):
-                raise SynapseError(400, "'%s' not a string type" % (s,))
-
        # Depending on the room version, ensure the data is spec compliant JSON.
        if event.room_version.strict_canonicaljson:
            validate_canonicaljson(event.get_pdu_json())
@@ -191,8 +183,18 @@ class EventValidator:
        fields an event would have
        """

+        create_event_as_room_id = (
+            event.room_version.msc4291_room_ids_as_hashes
+            and event.type == EventTypes.Create
+            and hasattr(event, "state_key")
+            and event.state_key == ""
+        )
+
        strings = ["room_id", "sender", "type"]

+        if create_event_as_room_id:
+            strings.remove("room_id")
+
        if hasattr(event, "state_key"):
            strings.append("state_key")

@@ -200,7 +202,14 @@ class EventValidator:
            if not isinstance(getattr(event, s), str):
                raise SynapseError(400, "Not '%s' a string type" % (s,))

-        RoomID.from_string(event.room_id)
+        if not create_event_as_room_id:
+            assert event.room_id is not None
+            RoomID.from_string(event.room_id)
+            if event.room_version.msc4291_room_ids_as_hashes and not RoomID.is_valid(
+                event.room_id
+            ):
+                raise SynapseError(400, f"Invalid room ID '{event.room_id}'")
+
        UserID.from_string(event.sender)

        if event.type == EventTypes.Message:
@@ -322,8 +322,7 @@ def event_from_pdu_json(pdu_json: JsonDict, room_version: RoomVersion) -> EventB
        SynapseError: if the pdu is missing required fields or is otherwise
            not a valid matrix event
    """
-    # we could probably enforce a bunch of other fields here (room_id, sender,
-    # origin, etc etc)
+    # we could probably enforce a bunch of other fields here (room_id, sender, etc.)
    assert_params_in_dict(pdu_json, ("type", "depth"))

    # Strip any unauthorized values from "unsigned" if they exist
@@ -343,6 +342,21 @@ def event_from_pdu_json(pdu_json: JsonDict, room_version: RoomVersion) -> EventB
    if room_version.strict_canonicaljson:
        validate_canonicaljson(pdu_json)

+    # enforce that MSC4291 auth events don't include the create event.
+    # N.B. if they DO include a spurious create event, it'll fail auth checks elsewhere, so we don't
+    # need to do expensive DB lookups to find which event ID is the create event here.
+    if room_version.msc4291_room_ids_as_hashes:
+        room_id = pdu_json.get("room_id")
+        if room_id:
+            create_event_id = "$" + room_id[1:]
+            auth_events = pdu_json.get("auth_events")
+            if auth_events:
+                if create_event_id in auth_events:
+                    raise SynapseError(
+                        400,
+                        "auth_events must not contain the create event",
+                        Codes.BAD_JSON,
+                    )
    event = make_event_from_dict(pdu_json, room_version)
    return event

@@ -1818,7 +1818,7 @@ class FederationClient(FederationBase):
            )
            return timestamp_to_event_response
        except SynapseError as e:
-            logger.warn(
+            logger.warning(
                "timestamp_to_event(room_id=%s, timestamp=%s, direction=%s): encountered error when trying to fetch from destinations: %s",
                room_id,
                timestamp,
@@ -85,7 +85,6 @@ from synapse.logging.opentracing import (
 from synapse.metrics.background_process_metrics import wrap_as_background_process
 from synapse.replication.http.federation import (
    ReplicationFederationSendEduRestServlet,
-    ReplicationGetQueryRestServlet,
 )
 from synapse.storage.databases.main.lock import Lock
 from synapse.storage.databases.main.roommember import extract_heroes_from_room_summary
@@ -928,7 +927,8 @@ class FederationServer(FederationBase):
            # joins) or the full state (for full joins).
            # Return a 404 as we would if we weren't in the room at all.
            logger.info(
-                f"Rejecting /send_{membership_type} to %s because it's a partial state room",
+                "Rejecting /send_%s to %s because it's a partial state room",
+                membership_type,
                room_id,
            )
            raise SynapseError(
@@ -1379,7 +1379,6 @@ class FederationHandlerRegistry:
        # and use them. However we have guards before we use them to ensure that
        # we don't route to ourselves, and in monolith mode that will always be
        # the case.
-        self._get_query_client = ReplicationGetQueryRestServlet.make_client(hs)
        self._send_edu = ReplicationFederationSendEduRestServlet.make_client(hs)

        self.edu_handlers: Dict[str, Callable[[str, dict], Awaitable[None]]] = {}
@@ -1468,10 +1467,6 @@ class FederationHandlerRegistry:
        if handler:
            return await handler(args)

-        # Check if we can route it somewhere else that isn't us
-        if self._instance_name == "master":
-            return await self._get_query_client(query_type=query_type, args=args)
-
        # Uh oh, no handler! Let's raise an exception so the request returns an
        # error.
        logger.warning("No handler registered for query type %s", query_type)
@@ -342,6 +342,8 @@ class _DestinationWakeupQueue:
                destination, _ = self.queue.popitem(last=False)

                queue = self.sender._get_per_destination_queue(destination)
+                if queue is None:
+                    continue

                if not queue._new_data_to_send:
                    # The per destination queue has already been woken up.
@@ -436,12 +438,23 @@ class FederationSender(AbstractFederationSender):
            self._wake_destinations_needing_catchup,
        )

-    def _get_per_destination_queue(self, destination: str) -> PerDestinationQueue:
+    def _get_per_destination_queue(
+        self, destination: str
+    ) -> Optional[PerDestinationQueue]:
        """Get or create a PerDestinationQueue for the given destination

        Args:
            destination: server_name of remote server
+
+        Returns:
+            None if the destination is not allowed by the federation whitelist.
+            Otherwise a PerDestinationQueue for this destination.
        """
+        if not self.hs.config.federation.is_domain_allowed_according_to_federation_whitelist(
+            destination
+        ):
+            return None
+
        queue = self._per_destination_queues.get(destination)
        if not queue:
            queue = PerDestinationQueue(self.hs, self._transaction_manager, destination)
@@ -718,6 +731,16 @@ class FederationSender(AbstractFederationSender):
        # track the fact that we have a PDU for these destinations,
        # to allow us to perform catch-up later on if the remote is unreachable
        # for a while.
+        # Filter out any destinations not present in the federation_domain_whitelist, if
+        # the whitelist exists. These destinations should not be sent to so let's not
+        # waste time or space keeping track of events destined for them.
+        destinations = [
+            d
+            for d in destinations
+            if self.hs.config.federation.is_domain_allowed_according_to_federation_whitelist(
+                d
+            )
+        ]
        await self.store.store_destination_rooms_entries(
            destinations,
            pdu.room_id,
@@ -732,7 +755,12 @@ class FederationSender(AbstractFederationSender):
        )

        for destination in destinations:
-            self._get_per_destination_queue(destination).send_pdu(pdu)
+            queue = self._get_per_destination_queue(destination)
+            # We expect `queue` to not be None as we already filtered out
+            # non-whitelisted destinations above.
+            assert queue is not None
+
+            queue.send_pdu(pdu)

    async def send_read_receipt(self, receipt: ReadReceipt) -> None:
        """Send a RR to any other servers in the room
@@ -841,12 +869,16 @@ class FederationSender(AbstractFederationSender):
        for domain in immediate_domains:
            # Add to destination queue and wake the destination up
            queue = self._get_per_destination_queue(domain)
+            if queue is None:
+                continue
            queue.queue_read_receipt(receipt)
            queue.attempt_new_transaction()

        for domain in delay_domains:
            # Add to destination queue...
            queue = self._get_per_destination_queue(domain)
+            if queue is None:
+                continue
            queue.queue_read_receipt(receipt)

            # ... and schedule the destination to be woken up.
@@ -882,9 +914,10 @@ class FederationSender(AbstractFederationSender):
            if self.is_mine_server_name(destination):
                continue

-            self._get_per_destination_queue(destination).send_presence(
-                states, start_loop=False
-            )
+            queue = self._get_per_destination_queue(destination)
+            if queue is None:
+                continue
+            queue.send_presence(states, start_loop=False)

            self._destination_wakeup_queue.add_to_queue(destination)

@@ -934,6 +967,8 @@ class FederationSender(AbstractFederationSender):
            return

        queue = self._get_per_destination_queue(edu.destination)
+        if queue is None:
+            return
        if key:
            queue.send_keyed_edu(edu, key)
        else:
@@ -958,9 +993,15 @@ class FederationSender(AbstractFederationSender):

        for destination in destinations:
            if immediate:
-                self._get_per_destination_queue(destination).attempt_new_transaction()
+                queue = self._get_per_destination_queue(destination)
+                if queue is None:
+                    continue
+                queue.attempt_new_transaction()
            else:
-                self._get_per_destination_queue(destination).mark_new_data()
+                queue = self._get_per_destination_queue(destination)
+                if queue is None:
+                    continue
+                queue.mark_new_data()
                self._destination_wakeup_queue.add_to_queue(destination)

    def wake_destination(self, destination: str) -> None:
@@ -979,7 +1020,9 @@ class FederationSender(AbstractFederationSender):
        ):
            return

-        self._get_per_destination_queue(destination).attempt_new_transaction()
+        queue = self._get_per_destination_queue(destination)
+        if queue is not None:
+            queue.attempt_new_transaction()

    @staticmethod
    def get_current_token() -> int:
@@ -1024,6 +1067,9 @@ class FederationSender(AbstractFederationSender):
                d
                for d in destinations_to_wake
                if self._federation_shard_config.should_handle(self._instance_name, d)
+                and self.hs.config.federation.is_domain_allowed_according_to_federation_whitelist(
+                    d
+                )
            ]

            for destination in destinations_to_wake:
--- a/Show More
+++ b/Show More
				`@@ -0,0 +1 @@`
				Add `recaptcha_private_key_path` and `recaptcha_public_key_path` config option.
				`@@ -0,0 +1 @@`
				`Add plain-text handling for rich-text topics as per [MSC3765](https://github.com/matrix-org/matrix-spec-proposals/pull/3765).`
				`@@ -0,0 +1 @@`
				`If enabled by the user, server admins will see [soft failed](https://spec.matrix.org/v1.13/server-server-api/#soft-failure) events over the Client-Server API.`
				`@@ -0,0 +1 @@`
				`Add experimental support for [MSC4277](https://github.com/matrix-org/matrix-spec-proposals/pull/4277).`
				`@@ -0,0 +1 @@`
				`Add ability to limit amount uploaded by a user in a given time period.`
				`@@ -0,0 +1 @@`
				`Allow user registrations to be done on workers.`
				`@@ -0,0 +1 @@`
				Unbreak "Latest dependencies" workflow by using the `--without dev` poetry option instead of removed `--no-dev`.
				`@@ -0,0 +1 @@`
				Use `markdown-it-py` instead of `commonmark` in the release script.
				`@@ -0,0 +1 @@`
				`Add doc comment explaining that config files are shallowly merged.`
				`@@ -0,0 +1 @@`
				Minor speed up of insertion into `stream_positions` table.