changelog

Remove v2 variant from endpoints
Knocking Endpoints added (federated)
2024-05-17 10:45:20 +01:00 · 2024-05-15 11:59:15 +01:00 · 2024-04-07 01:08:57 +02:00
127 changed files with 1480 additions and 4059 deletions
@@ -30,7 +30,7 @@ jobs:
        run: docker buildx inspect

      - name: Install Cosign
-        uses: sigstore/cosign-installer@v3.5.0
+        uses: sigstore/cosign-installer@v3.4.0

      - name: Checkout repository
        uses: actions/checkout@v4
@@ -19,7 +19,7 @@ jobs:
          fetch-depth: 0

      - name: Setup mdbook
-        uses: peaceiris/actions-mdbook@ee69d230fe19748b7abf22df32acaa93833fad08 # v2.0.0
+        uses: peaceiris/actions-mdbook@adeb05db28a0c0004681db83893d56c0388ea9ea # v1.2.0
        with:
          mdbook-version: '0.4.17'

@@ -53,7 +53,7 @@ jobs:
      - uses: actions/checkout@v4

      - name: Setup mdbook
-        uses: peaceiris/actions-mdbook@ee69d230fe19748b7abf22df32acaa93833fad08 # v2.0.0
+        uses: peaceiris/actions-mdbook@adeb05db28a0c0004681db83893d56c0388ea9ea # v1.2.0
        with:
          mdbook-version: '0.4.17'

@@ -56,7 +56,7 @@ jobs:
          fetch-depth: 0

      - name: Setup mdbook
-        uses: peaceiris/actions-mdbook@ee69d230fe19748b7abf22df32acaa93833fad08 # v2.0.0
+        uses: peaceiris/actions-mdbook@adeb05db28a0c0004681db83893d56c0388ea9ea # v1.2.0
        with:
          mdbook-version: '0.4.17'

@@ -80,8 +80,38 @@ jobs:

      # Deploy to the target directory.
      - name: Deploy to gh pages
-        uses: peaceiris/actions-gh-pages@4f9cc6602d3f66b9c108549d475ec49e8ef4d45e # v4.0.0
+        uses: peaceiris/actions-gh-pages@373f7f263a76c20808c831209c920827a82a2847 # v3.9.3
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          publish_dir: ./book
          destination_dir: ./${{ needs.pre.outputs.branch-version }}
+
+################################################################################
+  pages-devdocs:
+    name: GitHub Pages (developer docs)
+    runs-on: ubuntu-latest
+    needs:
+      - pre
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: "Set up Sphinx"
+        uses: matrix-org/setup-python-poetry@v1
+        with:
+          python-version: "3.x"
+          poetry-version: "1.3.2"
+          groups: "dev-docs"
+          extras: ""
+
+      - name: Build the documentation
+        run: |
+          cd dev-docs
+          poetry run make html
+
+      # Deploy to the target directory.
+      - name: Deploy to gh pages
+        uses: peaceiris/actions-gh-pages@373f7f263a76c20808c831209c920827a82a2847 # v3.9.3
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: ./dev-docs/_build/html
+          destination_dir: ./dev-docs/${{ needs.pre.outputs.branch-version }}
@@ -81,7 +81,7 @@ jobs:
    steps:
      - uses: actions/checkout@v4
      - name: Install Rust
-        uses: dtolnay/rust-toolchain@1.66.0
+        uses: dtolnay/rust-toolchain@1.65.0
      - uses: Swatinem/rust-cache@v2
      - uses: matrix-org/setup-python-poetry@v1
        with:
@@ -148,7 +148,7 @@ jobs:
        uses: actions/checkout@v4

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@1.66.0
+        uses: dtolnay/rust-toolchain@1.65.0
      - uses: Swatinem/rust-cache@v2

      - name: Setup Poetry
@@ -208,7 +208,7 @@ jobs:
        with:
          ref: ${{ github.event.pull_request.head.sha }}
      - name: Install Rust
-        uses: dtolnay/rust-toolchain@1.66.0
+        uses: dtolnay/rust-toolchain@1.65.0
      - uses: Swatinem/rust-cache@v2
      - uses: matrix-org/setup-python-poetry@v1
        with:
@@ -225,7 +225,7 @@ jobs:
      - uses: actions/checkout@v4

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@1.66.0
+        uses: dtolnay/rust-toolchain@1.65.0
        with:
            components: clippy
      - uses: Swatinem/rust-cache@v2
@@ -344,7 +344,7 @@ jobs:
            postgres:${{ matrix.job.postgres-version }}

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@1.66.0
+        uses: dtolnay/rust-toolchain@1.65.0
      - uses: Swatinem/rust-cache@v2

      - uses: matrix-org/setup-python-poetry@v1
@@ -386,7 +386,7 @@ jobs:
      - uses: actions/checkout@v4

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@1.66.0
+        uses: dtolnay/rust-toolchain@1.65.0
      - uses: Swatinem/rust-cache@v2

      # There aren't wheels for some of the older deps, so we need to install
@@ -498,7 +498,7 @@ jobs:
        run: cat sytest-blacklist .ci/worker-blacklist > synapse-blacklist-with-workers

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@1.66.0
+        uses: dtolnay/rust-toolchain@1.65.0
      - uses: Swatinem/rust-cache@v2

      - name: Run SyTest
@@ -642,7 +642,7 @@ jobs:
          path: synapse

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@1.66.0
+        uses: dtolnay/rust-toolchain@1.65.0
      - uses: Swatinem/rust-cache@v2

      - name: Prepare Complement's Prerequisites
@@ -674,7 +674,7 @@ jobs:
      - uses: actions/checkout@v4

      - name: Install Rust
-        uses: dtolnay/rust-toolchain@1.66.0
+        uses: dtolnay/rust-toolchain@1.65.0
      - uses: Swatinem/rust-cache@v2

      - run: cargo test
@@ -1,170 +1,3 @@
-Synapse 1.107.0 (2024-05-14)
-============================
-
-No significant changes since 1.107.0rc1.
-
-
-# Synapse 1.107.0rc1 (2024-05-07)
-
-### Features
-
- Add preliminary support for [MSC3823: Account Suspension](https://github.com/matrix-org/matrix-spec-proposals/pull/3823). ([\#17051](https://github.com/element-hq/synapse/issues/17051))
- Declare support for [Matrix v1.10](https://matrix.org/blog/2024/03/22/matrix-v1.10-release/). Contributed by @clokep. ([\#17082](https://github.com/element-hq/synapse/issues/17082))
- Add support for [MSC4115: membership metadata on events](https://github.com/matrix-org/matrix-spec-proposals/pull/4115). ([\#17104](https://github.com/element-hq/synapse/issues/17104), [\#17137](https://github.com/element-hq/synapse/issues/17137))
-
-### Bugfixes
-
- Fixed search feature of Element Android on homesevers using SQLite by returning search terms as search highlights. ([\#17000](https://github.com/element-hq/synapse/issues/17000))
- Fixes a bug introduced in v1.52.0 where the `destination` query parameter for the  [Destination Rooms Admin API](https://element-hq.github.io/synapse/v1.105/usage/administration/admin_api/federation.html#destination-rooms) failed to actually filter returned rooms. ([\#17077](https://github.com/element-hq/synapse/issues/17077))
- For MSC3266 room summaries, support queries at the recommended endpoint of `/_matrix/client/unstable/im.nheko.summary/summary/{roomIdOrAlias}`. The existing endpoint of `/_matrix/client/unstable/im.nheko.summary/rooms/{roomIdOrAlias}/summary` is deprecated. ([\#17078](https://github.com/element-hq/synapse/issues/17078))
- Apply user email & picture during OIDC registration if present & selected. ([\#17120](https://github.com/element-hq/synapse/issues/17120))
- Improve error message for cross signing reset with [MSC3861](https://github.com/matrix-org/matrix-spec-proposals/pull/3861) enabled. ([\#17121](https://github.com/element-hq/synapse/issues/17121))
- Fix a bug which meant that to-device messages received over federation could be dropped when the server was under load or networking problems caused problems between Synapse processes or the database. ([\#17127](https://github.com/element-hq/synapse/issues/17127))
- Fix bug where `StreamChangeCache` would not respect configured cache factors. ([\#17152](https://github.com/element-hq/synapse/issues/17152))
-
-### Updates to the Docker image
-
- Correct licensing metadata on Docker image. ([\#17141](https://github.com/element-hq/synapse/issues/17141))
-
-### Improved Documentation
-
- Update the `event_cache_size` and `global_factor` configuration options' documentation. ([\#17071](https://github.com/element-hq/synapse/issues/17071))
- Remove broken sphinx docs. ([\#17073](https://github.com/element-hq/synapse/issues/17073), [\#17148](https://github.com/element-hq/synapse/issues/17148))
- Add RuntimeDirectory to example matrix-synapse.service systemd unit. ([\#17084](https://github.com/element-hq/synapse/issues/17084))
- Fix various small typos throughout the docs. ([\#17114](https://github.com/element-hq/synapse/issues/17114))
- Update enable_notifs configuration documentation. ([\#17116](https://github.com/element-hq/synapse/issues/17116))
- Update the Upgrade Notes with the latest minimum supported Rust version of 1.66.0. Contributed by @jahway603. ([\#17140](https://github.com/element-hq/synapse/issues/17140))
-
-### Internal Changes
-
- Enable [MSC3266](https://github.com/matrix-org/matrix-spec-proposals/pull/3266) by default in the Synapse Complement image. ([\#17105](https://github.com/element-hq/synapse/issues/17105))
- Add optimisation to `StreamChangeCache.get_entities_changed(..)`. ([\#17130](https://github.com/element-hq/synapse/issues/17130))
-
-
-
-### Updates to locked dependencies
-
-* Bump furo from 2024.1.29 to 2024.4.27. ([\#17133](https://github.com/element-hq/synapse/issues/17133))
-* Bump idna from 3.6 to 3.7. ([\#17136](https://github.com/element-hq/synapse/issues/17136))
-* Bump jsonschema from 4.21.1 to 4.22.0. ([\#17157](https://github.com/element-hq/synapse/issues/17157))
-* Bump lxml from 5.1.0 to 5.2.1. ([\#17158](https://github.com/element-hq/synapse/issues/17158))
-* Bump phonenumbers from 8.13.29 to 8.13.35. ([\#17106](https://github.com/element-hq/synapse/issues/17106))
- Bump pillow from 10.2.0 to 10.3.0. ([\#17146](https://github.com/element-hq/synapse/issues/17146))
-* Bump pydantic from 2.6.4 to 2.7.0. ([\#17107](https://github.com/element-hq/synapse/issues/17107))
-* Bump pydantic from 2.7.0 to 2.7.1. ([\#17160](https://github.com/element-hq/synapse/issues/17160))
-* Bump pyicu from 2.12 to 2.13. ([\#17109](https://github.com/element-hq/synapse/issues/17109))
-* Bump serde from 1.0.197 to 1.0.198. ([\#17111](https://github.com/element-hq/synapse/issues/17111))
-* Bump serde from 1.0.198 to 1.0.199. ([\#17132](https://github.com/element-hq/synapse/issues/17132))
-* Bump serde from 1.0.199 to 1.0.200. ([\#17161](https://github.com/element-hq/synapse/issues/17161))
-* Bump serde_json from 1.0.115 to 1.0.116. ([\#17112](https://github.com/element-hq/synapse/issues/17112))
- Update `tornado` Python dependency from 6.2 to 6.4. ([\#17131](https://github.com/element-hq/synapse/issues/17131))
-* Bump twisted from 23.10.0 to 24.3.0. ([\#17135](https://github.com/element-hq/synapse/issues/17135))
-* Bump types-bleach from 6.1.0.1 to 6.1.0.20240331. ([\#17110](https://github.com/element-hq/synapse/issues/17110))
-* Bump types-pillow from 10.2.0.20240415 to 10.2.0.20240423. ([\#17159](https://github.com/element-hq/synapse/issues/17159))
-* Bump types-setuptools from 69.0.0.20240125 to 69.5.0.20240423. ([\#17134](https://github.com/element-hq/synapse/issues/17134))
-
-# Synapse 1.106.0 (2024-04-30)
-
-No significant changes since 1.106.0rc1.
-
-
-
-
-# Synapse 1.106.0rc1 (2024-04-25)
-
-### Features
-
- Send an email if the address is already bound to an user account. ([\#16819](https://github.com/element-hq/synapse/issues/16819))
- Implement the rendezvous mechanism described by [MSC4108](https://github.com/matrix-org/matrix-spec-proposals/issues/4108). ([\#17056](https://github.com/element-hq/synapse/issues/17056))
- Support delegating the rendezvous mechanism described [MSC4108](https://github.com/matrix-org/matrix-spec-proposals/issues/4108) to an external implementation. ([\#17086](https://github.com/element-hq/synapse/issues/17086))
-
-### Bugfixes
-
- Add validation to ensure that the `limit` parameter on `/publicRooms` is non-negative. ([\#16920](https://github.com/element-hq/synapse/issues/16920))
- Return `400 M_NOT_JSON` upon receiving invalid JSON in query parameters across various client and admin endpoints, rather than an internal server error. ([\#16923](https://github.com/element-hq/synapse/issues/16923))
- Make the CSAPI endpoint `/keys/device_signing/upload` idempotent. ([\#16943](https://github.com/element-hq/synapse/issues/16943))
- Redact membership events if the user requested erasure upon deactivating. ([\#17076](https://github.com/element-hq/synapse/issues/17076))
-
-### Improved Documentation
-
- Add a prompt in the contributing guide to manually configure icu4c. ([\#17069](https://github.com/element-hq/synapse/issues/17069))
- Clarify what part of message retention is still experimental. ([\#17099](https://github.com/element-hq/synapse/issues/17099))
-
-### Internal Changes
-
- Use new receipts column to optimise receipt and push action SQL queries. Contributed by Nick @ Beeper (@fizzadar). ([\#17032](https://github.com/element-hq/synapse/issues/17032), [\#17096](https://github.com/element-hq/synapse/issues/17096))
- Fix mypy with latest Twisted release. ([\#17036](https://github.com/element-hq/synapse/issues/17036))
- Bump minimum supported Rust version to 1.66.0. ([\#17079](https://github.com/element-hq/synapse/issues/17079))
- Add helpers to transform Twisted requests to Rust http Requests/Responses. ([\#17081](https://github.com/element-hq/synapse/issues/17081))
- Fix type annotation for `visited_chains` after `mypy` upgrade. ([\#17125](https://github.com/element-hq/synapse/issues/17125))
-
-
-
-### Updates to locked dependencies
-
-* Bump anyhow from 1.0.81 to 1.0.82. ([\#17095](https://github.com/element-hq/synapse/issues/17095))
-* Bump peaceiris/actions-gh-pages from 3.9.3 to 4.0.0. ([\#17087](https://github.com/element-hq/synapse/issues/17087))
-* Bump peaceiris/actions-mdbook from 1.2.0 to 2.0.0. ([\#17089](https://github.com/element-hq/synapse/issues/17089))
-* Bump pyasn1-modules from 0.3.0 to 0.4.0. ([\#17093](https://github.com/element-hq/synapse/issues/17093))
-* Bump pygithub from 2.2.0 to 2.3.0. ([\#17092](https://github.com/element-hq/synapse/issues/17092))
-* Bump ruff from 0.3.5 to 0.3.7. ([\#17094](https://github.com/element-hq/synapse/issues/17094))
-* Bump sigstore/cosign-installer from 3.4.0 to 3.5.0. ([\#17088](https://github.com/element-hq/synapse/issues/17088))
-* Bump twine from 4.0.2 to 5.0.0. ([\#17091](https://github.com/element-hq/synapse/issues/17091))
-* Bump types-pillow from 10.2.0.20240406 to 10.2.0.20240415. ([\#17090](https://github.com/element-hq/synapse/issues/17090))
-
-# Synapse 1.105.1 (2024-04-23)
-
-## Security advisory
-
-The following issues are fixed in 1.105.1.
-
- [GHSA-3h7q-rfh9-xm4v](https://github.com/element-hq/synapse/security/advisories/GHSA-3h7q-rfh9-xm4v) / [CVE-2024-31208](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-31208) — High Severity
-
-  Weakness in auth chain indexing allows DoS from remote room members through disk fill and high CPU usage.
-
-See the advisories for more details. If you have any questions, email security@element.io.
-
-
-
-# Synapse 1.105.0 (2024-04-16)
-
-No significant changes since 1.105.0rc1.
-
-
-
-
-# Synapse 1.105.0rc1 (2024-04-11)
-
-### Features
-
- Stabilize support for [MSC4010](https://github.com/matrix-org/matrix-spec-proposals/pull/4010) which clarifies the interaction of push rules and account data. Contributed by @clokep. ([\#17022](https://github.com/element-hq/synapse/issues/17022))
- Stabilize support for [MSC3981](https://github.com/matrix-org/matrix-spec-proposals/pull/3981): `/relations` recursion. Contributed by @clokep. ([\#17023](https://github.com/element-hq/synapse/issues/17023))
- Add support for moving `/pushrules` off of main process. ([\#17037](https://github.com/element-hq/synapse/issues/17037), [\#17038](https://github.com/element-hq/synapse/issues/17038))
-
-### Bugfixes
-
- Fix various long-standing bugs which could cause incorrect state to be returned from `/sync` in certain situations. ([\#16930](https://github.com/element-hq/synapse/issues/16930), [\#16932](https://github.com/element-hq/synapse/issues/16932), [\#16942](https://github.com/element-hq/synapse/issues/16942), [\#17064](https://github.com/element-hq/synapse/issues/17064), [\#17065](https://github.com/element-hq/synapse/issues/17065), [\#17066](https://github.com/element-hq/synapse/issues/17066))
- Fix server notice rooms not always being created as unencrypted rooms, even when `encryption_enabled_by_default_for_room_type` is in use (server notices are always unencrypted). ([\#17033](https://github.com/element-hq/synapse/issues/17033))
- Fix the `.m.rule.encrypted_room_one_to_one` and `.m.rule.room_one_to_one` default underride push rules being in the wrong order. Contributed by @Sumpy1. ([\#17043](https://github.com/element-hq/synapse/issues/17043))
-
-### Internal Changes
-
- Refactor auth chain fetching to reduce duplication. ([\#17044](https://github.com/element-hq/synapse/issues/17044))
- Improve database performance by adding a missing index to `access_tokens.refresh_token_id`. ([\#17045](https://github.com/element-hq/synapse/issues/17045), [\#17054](https://github.com/element-hq/synapse/issues/17054))
- Improve database performance by reducing number of receipts fetched when sending push notifications. ([\#17049](https://github.com/element-hq/synapse/issues/17049))
-
-
-
-### Updates to locked dependencies
-
-* Bump packaging from 23.2 to 24.0. ([\#17027](https://github.com/element-hq/synapse/issues/17027))
-* Bump regex from 1.10.3 to 1.10.4. ([\#17028](https://github.com/element-hq/synapse/issues/17028))
-* Bump ruff from 0.3.2 to 0.3.5. ([\#17060](https://github.com/element-hq/synapse/issues/17060))
-* Bump serde_json from 1.0.114 to 1.0.115. ([\#17041](https://github.com/element-hq/synapse/issues/17041))
-* Bump types-pillow from 10.2.0.20240125 to 10.2.0.20240406. ([\#17061](https://github.com/element-hq/synapse/issues/17061))
-* Bump types-requests from 2.31.0.20240125 to 2.31.0.20240406. ([\#17063](https://github.com/element-hq/synapse/issues/17063))
-* Bump typing-extensions from 4.9.0 to 4.11.0. ([\#17062](https://github.com/element-hq/synapse/issues/17062))
-
 # Synapse 1.104.0 (2024-04-02)

 ### Bugfixes
@@ -13,9 +13,9 @@ dependencies = [

 [[package]]
 name = "anyhow"
-version = "1.0.82"
+version = "1.0.81"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "f538837af36e6f6a9be0faa67f9a314f8119e4e4b5867c6ab40ed60360142519"
+checksum = "0952808a6c2afd1aa8947271f3a60f1a6763c7b912d210184c5149b5cf147247"

 [[package]]
 name = "arc-swap"
@@ -29,12 +29,6 @@ version = "1.1.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "d468802bab17cbc0cc575e9b053f41e72aa36bfa6b7f55e3529ffa43161b97fa"

-[[package]]
-name = "base64"
-version = "0.21.7"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "9d297deb1925b89f2ccc13d7635fa0714f12c87adce1c75356b39ca9b7178567"
-
 [[package]]
 name = "bitflags"
 version = "1.3.2"
@@ -59,33 +53,12 @@ dependencies = [
 "generic-array",
 ]

-[[package]]
-name = "bumpalo"
-version = "3.16.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "79296716171880943b8470b5f8d03aa55eb2e645a4874bdbb28adb49162e012c"
-
-[[package]]
-name = "bytes"
-version = "1.6.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "514de17de45fdb8dc022b1a7975556c53c86f9f0aa5f534b98977b171857c2c9"
-
 [[package]]
 name = "cfg-if"
 version = "1.0.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd"

-[[package]]
-name = "cpufeatures"
-version = "0.2.12"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "53fe5e26ff1b7aef8bca9c6080520cfb8d9333c7568e1829cef191a9723e5504"
-dependencies = [
- "libc",
-]
-
 [[package]]
 name = "crypto-common"
 version = "0.1.6"
@@ -98,21 +71,15 @@ dependencies = [

 [[package]]
 name = "digest"
-version = "0.10.7"
+version = "0.10.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292"
+checksum = "adfbc57365a37acbd2ebf2b64d7e69bb766e2fea813521ed536f5d0520dcf86c"
 dependencies = [
 "block-buffer",
 "crypto-common",
 "subtle",
 ]

-[[package]]
-name = "fnv"
-version = "1.0.7"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1"
-
 [[package]]
 name = "generic-array"
 version = "0.14.6"
@@ -123,43 +90,6 @@ dependencies = [
 "version_check",
 ]

-[[package]]
-name = "getrandom"
-version = "0.2.14"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "94b22e06ecb0110981051723910cbf0b5f5e09a2062dd7663334ee79a9d1286c"
-dependencies = [
- "cfg-if",
- "js-sys",
- "libc",
- "wasi",
- "wasm-bindgen",
-]
-
-[[package]]
-name = "headers"
-version = "0.4.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "322106e6bd0cba2d5ead589ddb8150a13d7c4217cf80d7c4f682ca994ccc6aa9"
-dependencies = [
- "base64",
- "bytes",
- "headers-core",
- "http",
- "httpdate",
- "mime",
- "sha1",
-]
-
-[[package]]
-name = "headers-core"
-version = "0.3.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "54b4a22553d4242c49fddb9ba998a99962b5cc6f22cb5a3482bec22522403ce4"
-dependencies = [
- "http",
-]
-
 [[package]]
 name = "heck"
 version = "0.4.1"
@@ -172,23 +102,6 @@ version = "0.4.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70"

-[[package]]
-name = "http"
-version = "1.1.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "21b9ddb458710bc376481b842f5da65cdf31522de232c1ca8146abce2a358258"
-dependencies = [
- "bytes",
- "fnv",
- "itoa",
-]
-
-[[package]]
-name = "httpdate"
-version = "1.0.3"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "df3b46402a9d5adb4c86a0cf463f42e19994e3ee891101b1841f30a545cb49a9"
-
 [[package]]
 name = "indoc"
 version = "2.0.4"
@@ -201,15 +114,6 @@ version = "1.0.4"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "4217ad341ebadf8d8e724e264f13e593e0648f5b3e94b3896a5df283be015ecc"

-[[package]]
-name = "js-sys"
-version = "0.3.69"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "29c15563dc2726973df627357ce0c9ddddbea194836909d655df6a75d2cf296d"
-dependencies = [
- "wasm-bindgen",
-]
-
 [[package]]
 name = "lazy_static"
 version = "1.4.0"
@@ -218,9 +122,9 @@ checksum = "e2abad23fbc42b3700f2f279844dc832adb2b2eb069b2df918f455c4e18cc646"

 [[package]]
 name = "libc"
-version = "0.2.153"
+version = "0.2.135"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "9c198f91728a82281a64e1f4f9eeb25d82cb32a5de251c6bd1b5154d63a8e7bd"
+checksum = "68783febc7782c6c5cb401fbda4de5a9898be1762314da0bb2c10ced61f18b0c"

 [[package]]
 name = "lock_api"
@@ -253,12 +157,6 @@ dependencies = [
 "autocfg",
 ]

-[[package]]
-name = "mime"
-version = "0.3.17"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "6877bb514081ee2a7ff5ef9de3281f14a4dd4bceac4c09388074a6b5df8a139a"
-
 [[package]]
 name = "once_cell"
 version = "1.15.0"
@@ -294,12 +192,6 @@ version = "1.6.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "7170ef9988bc169ba16dd36a7fa041e5c4cbeb6a35b76d4c03daded371eae7c0"

-[[package]]
-name = "ppv-lite86"
-version = "0.2.17"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "5b40af805b3121feab8a3c29f04d8ad262fa8e0561883e7653e024ae4479e6de"
-
 [[package]]
 name = "proc-macro2"
 version = "1.0.76"
@@ -403,36 +295,6 @@ dependencies = [
 "proc-macro2",
 ]

-[[package]]
-name = "rand"
-version = "0.8.5"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "34af8d1a0e25924bc5b7c43c079c942339d8f0a8b57c39049bef581b46327404"
-dependencies = [
- "libc",
- "rand_chacha",
- "rand_core",
-]
-
-[[package]]
-name = "rand_chacha"
-version = "0.3.1"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88"
-dependencies = [
- "ppv-lite86",
- "rand_core",
-]
-
-[[package]]
-name = "rand_core"
-version = "0.6.4"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c"
-dependencies = [
- "getrandom",
-]
-
 [[package]]
 name = "redox_syscall"
 version = "0.2.16"
@@ -444,9 +306,9 @@ dependencies = [

 [[package]]
 name = "regex"
-version = "1.10.4"
+version = "1.10.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "c117dbdfde9c8308975b6a18d71f3f385c89461f7b3fb054288ecf2a2058ba4c"
+checksum = "b62dbe01f0b06f9d8dc7d49e05a0785f153b00b2c227856282f671e0318c9b15"
 dependencies = [
 "aho-corasick",
 "memchr",
@@ -485,18 +347,18 @@ checksum = "d29ab0c6d3fc0ee92fe66e2d99f700eab17a8d57d1c1d3b748380fb20baa78cd"

 [[package]]
 name = "serde"
-version = "1.0.200"
+version = "1.0.197"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "ddc6f9cc94d67c0e21aaf7eda3a010fd3af78ebf6e096aa6e2e13c79749cce4f"
+checksum = "3fb1c873e1b9b056a4dc4c0c198b24c3ffa059243875552b2bd0933b1aee4ce2"
 dependencies = [
 "serde_derive",
 ]

 [[package]]
 name = "serde_derive"
-version = "1.0.200"
+version = "1.0.197"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "856f046b9400cee3c8c94ed572ecdb752444c24528c035cd35882aad6f492bcb"
+checksum = "7eb0b34b42edc17f6b7cac84a52a1c5f0e1bb2227e997ca9011ea3dd34e8610b"
 dependencies = [
 "proc-macro2",
 "quote",
@@ -505,37 +367,15 @@ dependencies = [

 [[package]]
 name = "serde_json"
-version = "1.0.116"
+version = "1.0.114"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "3e17db7126d17feb94eb3fad46bf1a96b034e8aacbc2e775fe81505f8b0b2813"
+checksum = "c5f09b1bd632ef549eaa9f60a1f8de742bdbc698e6cee2095fc84dde5f549ae0"
 dependencies = [
 "itoa",
 "ryu",
 "serde",
 ]

-[[package]]
-name = "sha1"
-version = "0.10.5"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "f04293dc80c3993519f2d7f6f511707ee7094fe0c6d3406feb330cdb3540eba3"
-dependencies = [
- "cfg-if",
- "cpufeatures",
- "digest",
-]
-
-[[package]]
-name = "sha2"
-version = "0.10.8"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "793db75ad2bcafc3ffa7c68b215fee268f537982cd901d132f89c6343f3a3dc8"
-dependencies = [
- "cfg-if",
- "cpufeatures",
- "digest",
-]
-
 [[package]]
 name = "smallvec"
 version = "1.10.0"
@@ -564,23 +404,16 @@ name = "synapse"
 version = "0.1.0"
 dependencies = [
 "anyhow",
- "base64",
 "blake2",
- "bytes",
- "headers",
 "hex",
- "http",
 "lazy_static",
 "log",
- "mime",
 "pyo3",
 "pyo3-log",
 "pythonize",
 "regex",
 "serde",
 "serde_json",
- "sha2",
- "ulid",
 ]

 [[package]]
@@ -595,17 +428,6 @@ version = "1.15.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "dcf81ac59edc17cc8697ff311e8f5ef2d99fcbd9817b34cec66f90b6c3dfd987"

-[[package]]
-name = "ulid"
-version = "1.1.2"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "34778c17965aa2a08913b57e1f34db9b4a63f5de31768b55bf20d2795f921259"
-dependencies = [
- "getrandom",
- "rand",
- "web-time",
-]
-
 [[package]]
 name = "unicode-ident"
 version = "1.0.5"
@@ -624,76 +446,6 @@ version = "0.9.4"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "49874b5167b65d7193b8aba1567f5c7d93d001cafc34600cee003eda787e483f"

-[[package]]
-name = "wasi"
-version = "0.11.0+wasi-snapshot-preview1"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "9c8d87e72b64a3b4db28d11ce29237c246188f4f51057d65a7eab63b7987e423"
-
-[[package]]
-name = "wasm-bindgen"
-version = "0.2.92"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "4be2531df63900aeb2bca0daaaddec08491ee64ceecbee5076636a3b026795a8"
-dependencies = [
- "cfg-if",
- "wasm-bindgen-macro",
-]
-
-[[package]]
-name = "wasm-bindgen-backend"
-version = "0.2.92"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "614d787b966d3989fa7bb98a654e369c762374fd3213d212cfc0251257e747da"
-dependencies = [
- "bumpalo",
- "log",
- "once_cell",
- "proc-macro2",
- "quote",
- "syn",
- "wasm-bindgen-shared",
-]
-
-[[package]]
-name = "wasm-bindgen-macro"
-version = "0.2.92"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "a1f8823de937b71b9460c0c34e25f3da88250760bec0ebac694b49997550d726"
-dependencies = [
- "quote",
- "wasm-bindgen-macro-support",
-]
-
-[[package]]
-name = "wasm-bindgen-macro-support"
-version = "0.2.92"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "e94f17b526d0a461a191c78ea52bbce64071ed5c04c9ffe424dcb38f74171bb7"
-dependencies = [
- "proc-macro2",
- "quote",
- "syn",
- "wasm-bindgen-backend",
- "wasm-bindgen-shared",
-]
-
-[[package]]
-name = "wasm-bindgen-shared"
-version = "0.2.92"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "af190c94f2773fdb3729c55b007a722abb5384da03bc0986df4c289bf5567e96"
-
-[[package]]
-name = "web-time"
-version = "1.1.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "5a6580f308b1fad9207618087a65c04e7a10bc77e02c8e84e9b00dd4b12fa0bb"
-dependencies = [
- "js-sys",
- "wasm-bindgen",
-]
-
 [[package]]
 name = "windows-sys"
 version = "0.36.1"
@@ -0,0 +1 @@
+Fix various long-standing bugs which could cause incorrect state to be returned from `/sync` in certain situations.
@@ -0,0 +1 @@
+Fix various long-standing bugs which could cause incorrect state to be returned from `/sync` in certain situations.
@@ -0,0 +1 @@
+Fix various long-standing bugs which could cause incorrect state to be returned from `/sync` in certain situations.
@@ -0,0 +1 @@
+Add support for moving `/pushrules` off of main process.
@@ -0,0 +1 @@
+Add support for moving `/pushrules` off of main process.
@@ -0,0 +1 @@
+Refactor auth chain fetching to reduce duplication.
@@ -0,0 +1 @@
+Improve database performance by adding a missing index to `access_tokens.refresh_token_id`.
@@ -0,0 +1 @@
+Improve database performance by reducing number of receipts fetched when sending push notifications.
@@ -0,0 +1 @@
+Document [`/v1/make_knock`](https://spec.matrix.org/v1.10/server-server-api/#get_matrixfederationv1make_knockroomiduserid) and [`/v1/send_knock/](https://spec.matrix.org/v1.10/server-server-api/#put_matrixfederationv1send_knockroomideventid) federation endpoints as worker-compatible.
@@ -1,45 +1,3 @@
-matrix-synapse-py3 (1.107.0) stable; urgency=medium
-
-  * New Synapse release 1.107.0.
-
- -- Synapse Packaging team <packages@matrix.org>  Tue, 14 May 2024 14:15:34 +0100
-
-matrix-synapse-py3 (1.107.0~rc1) stable; urgency=medium
-
-  * New Synapse release 1.107.0rc1.
-
- -- Synapse Packaging team <packages@matrix.org>  Tue, 07 May 2024 16:26:26 +0100
-
-matrix-synapse-py3 (1.106.0) stable; urgency=medium
-
-  * New Synapse release 1.106.0.
-
- -- Synapse Packaging team <packages@matrix.org>  Tue, 30 Apr 2024 11:51:43 +0100
-
-matrix-synapse-py3 (1.106.0~rc1) stable; urgency=medium
-
-  * New Synapse release 1.106.0rc1.
-
- -- Synapse Packaging team <packages@matrix.org>  Thu, 25 Apr 2024 15:54:59 +0100
-
-matrix-synapse-py3 (1.105.1) stable; urgency=medium
-
-  * New Synapse release 1.105.1.
-
- -- Synapse Packaging team <packages@matrix.org>  Tue, 23 Apr 2024 15:56:18 +0100
-
-matrix-synapse-py3 (1.105.0) stable; urgency=medium
-
-  * New Synapse release 1.105.0.
-
- -- Synapse Packaging team <packages@matrix.org>  Tue, 16 Apr 2024 15:53:23 +0100
-
-matrix-synapse-py3 (1.105.0~rc1) stable; urgency=medium
-
-  * New Synapse release 1.105.0rc1.
-
- -- Synapse Packaging team <packages@matrix.org>  Thu, 11 Apr 2024 12:15:49 +0100
-
 matrix-synapse-py3 (1.104.0) stable; urgency=medium

  * New Synapse release 1.104.0.
@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
@@ -0,0 +1,50 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# For the full list of built-in configuration values, see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Project information -----------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
+
+project = "Synapse development"
+copyright = "2023, The Matrix.org Foundation C.I.C."
+author = "The Synapse Maintainers and Community"
+
+# -- General configuration ---------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
+
+extensions = [
+    "autodoc2",
+    "myst_parser",
+]
+
+templates_path = ["_templates"]
+exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
+
+
+# -- Options for Autodoc2 ----------------------------------------------------
+
+autodoc2_docstring_parser_regexes = [
+    # this will render all docstrings as 'MyST' Markdown
+    (r".*", "myst"),
+]
+
+autodoc2_packages = [
+    {
+        "path": "../synapse",
+        # Don't render documentation for everything as a matter of course
+        "auto_mode": False,
+    },
+]
+
+
+# -- Options for MyST (Markdown) ---------------------------------------------
+
+# myst_heading_anchors = 2
+
+
+# -- Options for HTML output -------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
+
+html_theme = "furo"
+html_static_path = ["_static"]
@@ -0,0 +1,22 @@
+.. Synapse Developer Documentation documentation master file, created by
+   sphinx-quickstart on Mon Mar 13 08:59:51 2023.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+Welcome to the Synapse Developer Documentation!
+===========================================================
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   modules/federation_sender
+
+
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`
@@ -0,0 +1,5 @@
+Federation Sender
+=================
+
+```{autodoc2-docstring} synapse.federation.sender
+```
@@ -163,7 +163,7 @@ FROM docker.io/library/python:${PYTHON_VERSION}-slim-bookworm
 LABEL org.opencontainers.image.url='https://matrix.org/docs/projects/server/synapse'
 LABEL org.opencontainers.image.documentation='https://github.com/element-hq/synapse/blob/master/docker/README.md'
 LABEL org.opencontainers.image.source='https://github.com/element-hq/synapse.git'
-LABEL org.opencontainers.image.licenses='AGPL-3.0-or-later'
+LABEL org.opencontainers.image.licenses='Apache-2.0'

 RUN \
  --mount=type=cache,target=/var/cache/apt,sharing=locked \
@@ -92,6 +92,8 @@ allow_device_name_lookup_over_federation: true
 ## Experimental Features ##

 experimental_features:
+  # client-side support for partial state in /send_join responses
+  faster_joins: true
  # Enable support for polls
  msc3381_polls_enabled: true
  # Enable deleting device-specific notification settings stored in account data
@@ -100,12 +102,6 @@ experimental_features:
  msc3391_enabled: true
  # Filtering /messages by relation type.
  msc3874_enabled: true
-  # no UIA for x-signing upload for the first time
-  msc3967_enabled: true
-  # Expose a room summary for public rooms
-  msc3266_enabled: true
-
-  msc4115_membership_on_events: true

 server_notices:
  system_mxid_localpart: _server
@@ -1,6 +1,6 @@
 # Edit Room Membership API

-This API allows an administrator to join a user account with a given `user_id`
+This API allows an administrator to join an user account with a given `user_id`
 to a room with a given `room_id_or_alias`. You can only modify the membership of
 local users. The server administrator must be in the room and have permission to
 invite users.
@@ -86,8 +86,6 @@ poetry install --extras all
 This will install the runtime and developer dependencies for the project.  Be sure to check
 that the `poetry install` step completed cleanly.

-For OSX users, be sure to set `PKG_CONFIG_PATH` to support `icu4c`. Run `brew info icu4c` for more details.
-
 ## Running Synapse via poetry

 To start a local instance of Synapse in the locked poetry environment, create a config file:
@@ -7,10 +7,8 @@ follow the semantics described in
 and allow server and room admins to configure how long messages should
 be kept in a homeserver's database before being purged from it.
 **Please note that, as this feature isn't part of the Matrix
-specification yet, the use of `m.room.retention` events for per-room
-retention policies is to be considered as experimental. However, the use
-of a default message retention policy is considered a stable feature
-in Synapse.**
+specification yet, this implementation is to be considered as
+experimental.**

 A message retention policy is mainly defined by its `max_lifetime`
 parameter, which defines how long a message can be kept around after
@@ -51,8 +49,8 @@ clients.

 ## Server configuration

-Support for this feature can be enabled and configured by adding the
-`retention` option in the Synapse configuration file (see
+Support for this feature can be enabled and configured by adding a the
+`retention` in the Synapse configuration file (see
 [configuration manual](usage/configuration/config_documentation.md#retention)).

 To enable support for message retention policies, set the setting
@@ -117,7 +115,7 @@ In this example, we define three jobs:
  policy's `max_lifetime` is greater than a week.

 Note that this example is tailored to show different configurations and
-features slightly more jobs than is probably necessary (in practice, a
+features slightly more jobs than it's probably necessary (in practice, a
 server admin would probably consider it better to replace the two last
 jobs with one that runs once a day and handles rooms which
 policy's `max_lifetime` is greater than 3 days).
@@ -128,7 +128,7 @@ can read more about that [here](https://www.postgresql.org/docs/10/kernel-resour
 ### Overview

 The script `synapse_port_db` allows porting an existing synapse server
-backed by SQLite to using PostgreSQL. This is done as a two phase
+backed by SQLite to using PostgreSQL. This is done in as a two phase
 process:

 1.  Copy the existing SQLite database to a separate location and run
@@ -259,9 +259,9 @@ users, etc.) to the developers via the `--report-stats` argument.

 This command will generate you a config file that you can then customise, but it will
 also generate a set of keys for you. These keys will allow your homeserver to
-identify itself to other homeservers, so don't lose or delete them. It would be
+identify itself to other homeserver, so don't lose or delete them. It would be
 wise to back them up somewhere safe. (If, for whatever reason, you do need to
-change your homeserver's keys, you may find that other homeservers have the
+change your homeserver's keys, you may find that other homeserver have the
 old key cached. If you update the signing key, you should change the name of the
 key in the `<server name>.signing.key` file (the second word) to something
 different. See the [spec](https://matrix.org/docs/spec/server_server/latest.html#retrieving-server-keys) for more information on key management).
@@ -98,7 +98,6 @@ A custom mapping provider must specify the following methods:
        either accept this localpart or pick their own username. Otherwise this
        option has no effect. If omitted, defaults to `False`.
      - `display_name`: An optional string, the display name for the user.
-      - `picture`: An optional string, the avatar url for the user.
      - `emails`: A list of strings, the email address(es) to associate with
        this user. If omitted, defaults to an empty list.
 * `async def get_extra_attributes(self, userinfo, token)`
@@ -9,7 +9,6 @@ ReloadPropagatedFrom=matrix-synapse.target
 Type=notify
 NotifyAccess=main
 User=matrix-synapse
-RuntimeDirectory=synapse
 WorkingDirectory=/var/lib/matrix-synapse
 EnvironmentFile=-/etc/default/matrix-synapse
 ExecStartPre=/opt/venvs/matrix-synapse/bin/python -m synapse.app.homeserver --config-path=/etc/matrix-synapse/homeserver.yaml --config-path=/etc/matrix-synapse/conf.d/ --generate-keys
@@ -117,14 +117,6 @@ each upgrade are complete before moving on to the next upgrade, to avoid
 stacking them up. You can monitor the currently running background updates with
 [the Admin API](usage/administration/admin_api/background_updates.html#status).

-# Upgrading to v1.106.0
-
-## Minimum supported Rust version
-The minimum supported Rust version has been increased from v1.65.0 to v1.66.0.
-Users building from source will need to ensure their `rustc` version is up to
-date.
-
-
 # Upgrading to v1.100.0

 ## Minimum supported Rust version
@@ -44,7 +44,7 @@ For each update:

 ## Enabled

-This API allows pausing background updates.
+This API allow pausing background updates.

 Background updates should *not* be paused for significant periods of time, as
 this can affect the performance of Synapse.
@@ -241,7 +241,7 @@ in memory constrained environments, or increased if performance starts to
 degrade.

 However, degraded performance due to a low cache factor, common on
-machines with slow disks, often leads to explosions in memory use due to
+machines with slow disks, often leads to explosions in memory use due
 backlogged requests. In this case, reducing the cache factor will make
 things worse. Instead, try increasing it drastically. 2.0 is a good
 starting value.
@@ -676,8 +676,8 @@ This setting has the following sub-options:
    trailing 's'.
 * `app_name`: `app_name` defines the default value for '%(app)s' in `notif_from` and email
   subjects. It defaults to 'Matrix'.
-* `enable_notifs`: Set to true to allow users to receive e-mail notifications. If this is not set,
-    users can configure e-mail notifications but will not receive them. Disabled by default.
+* `enable_notifs`: Set to true to enable sending emails for messages that the user
+   has missed. Disabled by default.
 * `notif_for_new_users`: Set to false to disable automatic subscription to email
   notifications for new users. Enabled by default.
 * `notif_delay_before_mail`: The time to wait before emailing about a notification.
@@ -1317,12 +1317,6 @@ Options related to caching.
 The number of events to cache in memory. Defaults to 10K. Like other caches,
 this is affected by `caches.global_factor` (see below).

-For example, the default is 10K and the global_factor default is 0.5.
-
-Since 10K * 0.5 is 5K then the event cache size will be 5K.
-
-The cache affected by this configuration is named as "*getEvent*".
-
 Note that this option is not part of the `caches` section.

 Example configuration:
@@ -1348,8 +1342,6 @@ number of entries that can be stored.

  Defaults to 0.5, which will halve the size of all caches.

-  Note that changing this value also affects the HTTP connection pool.
-
 * `per_cache_factors`: A dictionary of cache name to cache factor for that individual
   cache. Overrides the global cache factor for a given cache.

@@ -86,9 +86,9 @@ The search term is then split into words:
 * If unavailable, then runs of ASCII characters, numbers, underscores, and hyphens
  are considered words.

-The queries for PostgreSQL and SQLite are detailed below, but their overall goal
+The queries for PostgreSQL and SQLite are detailed below, by their overall goal
 is to find matching users, preferring users who are "real" (e.g. not bots,
-not deactivated). It is assumed that real users will have a display name and
+not deactivated). It is assumed that real users will have an display name and
 avatar set.

 ### PostgreSQL
@@ -211,6 +211,8 @@ information.
    ^/_matrix/federation/v1/make_leave/
    ^/_matrix/federation/(v1|v2)/send_join/
    ^/_matrix/federation/(v1|v2)/send_leave/
+    ^/_matrix/federation/v1/make_knock/
+    ^/_matrix/federation/v1/send_knock/
    ^/_matrix/federation/(v1|v2)/invite/
    ^/_matrix/federation/v1/event_auth/
    ^/_matrix/federation/v1/timestamp_to_event/
@@ -232,7 +234,7 @@ information.
    ^/_matrix/client/v1/rooms/.*/hierarchy$
    ^/_matrix/client/(v1|unstable)/rooms/.*/relations/
    ^/_matrix/client/v1/rooms/.*/threads$
-    ^/_matrix/client/unstable/im.nheko.summary/summary/.*$
+    ^/_matrix/client/unstable/im.nheko.summary/rooms/.*/summary$
    ^/_matrix/client/(r0|v3|unstable)/account/3pid$
    ^/_matrix/client/(r0|v3|unstable)/account/whoami$
    ^/_matrix/client/(r0|v3|unstable)/devices$
@@ -634,7 +636,7 @@ worker application type.

 #### Push Notifications

-You can designate generic workers to send push notifications to
+You can designate generic worker to sending push notifications to
 a [push gateway](https://spec.matrix.org/v1.5/push-gateway-api/) such as
 [sygnal](https://github.com/matrix-org/sygnal) and email.

@@ -96,7 +96,7 @@ module-name = "synapse.synapse_rust"

 [tool.poetry]
 name = "matrix-synapse"
-version = "1.107.0"
+version = "1.104.0"
 description = "Homeserver for the Matrix decentralised comms protocol"
 authors = ["Matrix.org Team and Contributors <packages@matrix.org>"]
 license = "AGPL-3.0-or-later"
@@ -321,7 +321,7 @@ all = [
 # This helps prevents merge conflicts when running a batch of dependabot updates.
 isort = ">=5.10.1"
 black = ">=22.7.0"
-ruff = "0.3.7"
+ruff = "0.3.2"
 # Type checking only works with the pydantic.v1 compat module from pydantic v2
 pydantic = "^2"

@@ -364,6 +364,17 @@ towncrier = ">=18.6.0rc1"
 tomli = ">=1.2.3"


+# Dependencies for building the development documentation
+[tool.poetry.group.dev-docs]
+optional = true
+
+[tool.poetry.group.dev-docs.dependencies]
+sphinx = {version = "^6.1", python = "^3.8"}
+sphinx-autodoc2 = {version = ">=0.4.2,<0.6.0", python = "^3.8"}
+myst-parser = {version = "^1.0.0", python = "^3.8"}
+furo = ">=2022.12.7,<2025.0.0"
+
+
 [build-system]
 # The upper bounds here are defensive, intended to prevent situations like
 # https://github.com/matrix-org/synapse/issues/13849 and
@@ -7,7 +7,7 @@ name = "synapse"
 version = "0.1.0"

 edition = "2021"
-rust-version = "1.66.0"
+rust-version = "1.65.0"

 [lib]
 name = "synapse"
@@ -23,13 +23,8 @@ name = "synapse.synapse_rust"

 [dependencies]
 anyhow = "1.0.63"
-base64 = "0.21.7"
-bytes = "1.6.0"
-headers = "0.4.0"
-http = "1.1.0"
 lazy_static = "1.4.0"
 log = "0.4.17"
-mime = "0.3.17"
 pyo3 = { version = "0.20.0", features = [
    "macros",
    "anyhow",
@@ -39,10 +34,8 @@ pyo3 = { version = "0.20.0", features = [
 pyo3-log = "0.9.0"
 pythonize = "0.20.0"
 regex = "1.6.0"
-sha2 = "0.10.8"
 serde = { version = "1.0.144", features = ["derive"] }
 serde_json = "1.0.85"
-ulid = "1.1.2"

 [features]
 extension-module = ["pyo3/extension-module"]
@@ -1,60 +0,0 @@
-/*
- * This file is licensed under the Affero General Public License (AGPL) version 3.
- *
- * Copyright (C) 2024 New Vector, Ltd
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * See the GNU Affero General Public License for more details:
- * <https://www.gnu.org/licenses/agpl-3.0.html>.
- */
-
-#![allow(clippy::new_ret_no_self)]
-
-use std::collections::HashMap;
-
-use http::{HeaderMap, StatusCode};
-use pyo3::{exceptions::PyValueError, import_exception};
-
-import_exception!(synapse.api.errors, SynapseError);
-
-impl SynapseError {
-    pub fn new(
-        code: StatusCode,
-        message: String,
-        errcode: &'static str,
-        additional_fields: Option<HashMap<String, String>>,
-        headers: Option<HeaderMap>,
-    ) -> pyo3::PyErr {
-        // Transform the HeaderMap into a HashMap<String, String>
-        let headers = if let Some(headers) = headers {
-            let mut map = HashMap::with_capacity(headers.len());
-            for (key, value) in headers.iter() {
-                let Ok(value) = value.to_str() else {
-                    // This should never happen, but we don't want to panic in case it does
-                    return PyValueError::new_err(
-                        "Could not construct SynapseError: header value is not valid ASCII",
-                    );
-                };
-
-                map.insert(key.as_str().to_owned(), value.to_owned());
-            }
-            Some(map)
-        } else {
-            None
-        };
-
-        SynapseError::new_err((code.as_u16(), message, errcode, additional_fields, headers))
-    }
-}
-
-import_exception!(synapse.api.errors, NotFoundError);
-
-impl NotFoundError {
-    pub fn new() -> pyo3::PyErr {
-        NotFoundError::new_err(())
-    }
-}
@@ -20,10 +20,8 @@

 //! Implements the internal metadata class attached to events.
 //!
-//! The internal metadata is a bit like a `TypedDict`, in that most of
-//! it is stored as a JSON dict in the DB (the exceptions being `outlier`
-//! and `stream_ordering` which have their own columns in the database).
-//! Most events have zero, or only a few, of these keys
+//! The internal metadata is a bit like a `TypedDict`, in that it is stored as a
+//! JSON dict in the DB. Most events have zero, or only a few, of these keys
 //! set. Therefore, since we care more about memory size than performance here,
 //! we store these fields in a mapping.
 //!
@@ -236,9 +234,6 @@ impl EventInternalMetadata {
        self.clone()
    }

-    /// Get a dict holding the data stored in the `internal_metadata` column in the database.
-    ///
-    /// Note that `outlier` and `stream_ordering` are stored in separate columns so are not returned here.
    fn get_dict(&self, py: Python<'_>) -> PyResult<PyObject> {
        let dict = PyDict::new(py);

@@ -1,165 +0,0 @@
-/*
- * This file is licensed under the Affero General Public License (AGPL) version 3.
- *
- * Copyright (C) 2024 New Vector, Ltd
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * See the GNU Affero General Public License for more details:
- * <https://www.gnu.org/licenses/agpl-3.0.html>.
- */
-
-use bytes::{Buf, BufMut, Bytes, BytesMut};
-use headers::{Header, HeaderMapExt};
-use http::{HeaderName, HeaderValue, Method, Request, Response, StatusCode, Uri};
-use pyo3::{
-    exceptions::PyValueError,
-    types::{PyBytes, PySequence, PyTuple},
-    PyAny, PyResult,
-};
-
-use crate::errors::SynapseError;
-
-/// Read a file-like Python object by chunks
-///
-/// # Errors
-///
-/// Returns an error if calling the `read` on the Python object failed
-fn read_io_body(body: &PyAny, chunk_size: usize) -> PyResult<Bytes> {
-    let mut buf = BytesMut::new();
-    loop {
-        let bytes: &PyBytes = body.call_method1("read", (chunk_size,))?.downcast()?;
-        if bytes.as_bytes().is_empty() {
-            return Ok(buf.into());
-        }
-        buf.put(bytes.as_bytes());
-    }
-}
-
-/// Transform a Twisted `IRequest` to an [`http::Request`]
-///
-/// It uses the following members of `IRequest`:
-///   - `content`, which is expected to be a file-like object with a `read` method
-///   - `uri`, which is expected to be a valid URI as `bytes`
-///   - `method`, which is expected to be a valid HTTP method as `bytes`
-///   - `requestHeaders`, which is expected to have a `getAllRawHeaders` method
-///
-/// # Errors
-///
-/// Returns an error if the Python object doesn't properly implement `IRequest`
-pub fn http_request_from_twisted(request: &PyAny) -> PyResult<Request<Bytes>> {
-    let content = request.getattr("content")?;
-    let body = read_io_body(content, 4096)?;
-
-    let mut req = Request::new(body);
-
-    let uri: &PyBytes = request.getattr("uri")?.downcast()?;
-    *req.uri_mut() =
-        Uri::try_from(uri.as_bytes()).map_err(|_| PyValueError::new_err("invalid uri"))?;
-
-    let method: &PyBytes = request.getattr("method")?.downcast()?;
-    *req.method_mut() = Method::from_bytes(method.as_bytes())
-        .map_err(|_| PyValueError::new_err("invalid method"))?;
-
-    let headers_iter = request
-        .getattr("requestHeaders")?
-        .call_method0("getAllRawHeaders")?
-        .iter()?;
-
-    for header in headers_iter {
-        let header = header?;
-        let header: &PyTuple = header.downcast()?;
-        let name: &PyBytes = header.get_item(0)?.downcast()?;
-        let name = HeaderName::from_bytes(name.as_bytes())
-            .map_err(|_| PyValueError::new_err("invalid header name"))?;
-
-        let values: &PySequence = header.get_item(1)?.downcast()?;
-        for index in 0..values.len()? {
-            let value: &PyBytes = values.get_item(index)?.downcast()?;
-            let value = HeaderValue::from_bytes(value.as_bytes())
-                .map_err(|_| PyValueError::new_err("invalid header value"))?;
-            req.headers_mut().append(name.clone(), value);
-        }
-    }
-
-    Ok(req)
-}
-
-/// Send an [`http::Response`] through a Twisted `IRequest`
-///
-/// It uses the following members of `IRequest`:
-///
-///  - `responseHeaders`, which is expected to have a `addRawHeader(bytes, bytes)` method
-///  - `setResponseCode(int)` method
-///  - `write(bytes)` method
-///  - `finish()` method
-///
-///  # Errors
-///
-/// Returns an error if the Python object doesn't properly implement `IRequest`
-pub fn http_response_to_twisted<B>(request: &PyAny, response: Response<B>) -> PyResult<()>
-where
-    B: Buf,
-{
-    let (parts, mut body) = response.into_parts();
-
-    request.call_method1("setResponseCode", (parts.status.as_u16(),))?;
-
-    let response_headers = request.getattr("responseHeaders")?;
-    for (name, value) in parts.headers.iter() {
-        response_headers.call_method1("addRawHeader", (name.as_str(), value.as_bytes()))?;
-    }
-
-    while body.remaining() != 0 {
-        let chunk = body.chunk();
-        request.call_method1("write", (chunk,))?;
-        body.advance(chunk.len());
-    }
-
-    request.call_method0("finish")?;
-
-    Ok(())
-}
-
-/// An extension trait for [`HeaderMap`] that provides typed access to headers, and throws the
-/// right python exceptions when the header is missing or fails to parse.
-///
-/// [`HeaderMap`]: headers::HeaderMap
-pub trait HeaderMapPyExt: HeaderMapExt {
-    /// Get a header from the map, returning an error if it is missing or invalid.
-    fn typed_get_required<H>(&self) -> PyResult<H>
-    where
-        H: Header,
-    {
-        self.typed_get_optional::<H>()?.ok_or_else(|| {
-            SynapseError::new(
-                StatusCode::BAD_REQUEST,
-                format!("Missing required header: {}", H::name()),
-                "M_MISSING_PARAM",
-                None,
-                None,
-            )
-        })
-    }
-
-    /// Get a header from the map, returning `None` if it is missing and an error if it is invalid.
-    fn typed_get_optional<H>(&self) -> PyResult<Option<H>>
-    where
-        H: Header,
-    {
-        self.typed_try_get::<H>().map_err(|_| {
-            SynapseError::new(
-                StatusCode::BAD_REQUEST,
-                format!("Invalid header: {}", H::name()),
-                "M_INVALID_PARAM",
-                None,
-                None,
-            )
-        })
-    }
-}
-
-impl<T: HeaderMapExt> HeaderMapPyExt for T {}
@@ -3,11 +3,8 @@ use pyo3::prelude::*;
 use pyo3_log::ResetHandle;

 pub mod acl;
-pub mod errors;
 pub mod events;
-pub mod http;
 pub mod push;
-pub mod rendezvous;

 lazy_static! {
    static ref LOGGING_HANDLE: ResetHandle = pyo3_log::init();
@@ -46,7 +43,6 @@ fn synapse_rust(py: Python<'_>, m: &PyModule) -> PyResult<()> {
    acl::register_module(py, m)?;
    push::register_module(py, m)?;
    events::register_module(py, m)?;
-    rendezvous::register_module(py, m)?;

    Ok(())
 }
@@ -304,12 +304,12 @@ pub const BASE_APPEND_UNDERRIDE_RULES: &[PushRule] = &[
        default_enabled: true,
    },
    PushRule {
-        rule_id: Cow::Borrowed("global/underride/.m.rule.encrypted_room_one_to_one"),
+        rule_id: Cow::Borrowed("global/underride/.m.rule.room_one_to_one"),
        priority_class: 1,
        conditions: Cow::Borrowed(&[
            Condition::Known(KnownCondition::EventMatch(EventMatchCondition {
                key: Cow::Borrowed("type"),
-                pattern: Cow::Borrowed("m.room.encrypted"),
+                pattern: Cow::Borrowed("m.room.message"),
            })),
            Condition::Known(KnownCondition::RoomMemberCount {
                is: Some(Cow::Borrowed("2")),
@@ -320,12 +320,12 @@ pub const BASE_APPEND_UNDERRIDE_RULES: &[PushRule] = &[
        default_enabled: true,
    },
    PushRule {
-        rule_id: Cow::Borrowed("global/underride/.m.rule.room_one_to_one"),
+        rule_id: Cow::Borrowed("global/underride/.m.rule.encrypted_room_one_to_one"),
        priority_class: 1,
        conditions: Cow::Borrowed(&[
            Condition::Known(KnownCondition::EventMatch(EventMatchCondition {
                key: Cow::Borrowed("type"),
-                pattern: Cow::Borrowed("m.room.message"),
+                pattern: Cow::Borrowed("m.room.encrypted"),
            })),
            Condition::Known(KnownCondition::RoomMemberCount {
                is: Some(Cow::Borrowed("2")),
@@ -1,315 +0,0 @@
-/*
- * This file is licensed under the Affero General Public License (AGPL) version 3.
- *
- * Copyright (C) 2024 New Vector, Ltd
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * See the GNU Affero General Public License for more details:
- * <https://www.gnu.org/licenses/agpl-3.0.html>.
- *
- */
-
-use std::{
-    collections::{BTreeMap, HashMap},
-    time::{Duration, SystemTime},
-};
-
-use bytes::Bytes;
-use headers::{
-    AccessControlAllowOrigin, AccessControlExposeHeaders, CacheControl, ContentLength, ContentType,
-    HeaderMapExt, IfMatch, IfNoneMatch, Pragma,
-};
-use http::{header::ETAG, HeaderMap, Response, StatusCode, Uri};
-use mime::Mime;
-use pyo3::{
-    exceptions::PyValueError, pyclass, pymethods, types::PyModule, Py, PyAny, PyObject, PyResult,
-    Python, ToPyObject,
-};
-use ulid::Ulid;
-
-use self::session::Session;
-use crate::{
-    errors::{NotFoundError, SynapseError},
-    http::{http_request_from_twisted, http_response_to_twisted, HeaderMapPyExt},
-};
-
-mod session;
-
-// n.b. Because OPTIONS requests are handled by the Python code, we don't need to set Access-Control-Allow-Headers.
-fn prepare_headers(headers: &mut HeaderMap, session: &Session) {
-    headers.typed_insert(AccessControlAllowOrigin::ANY);
-    headers.typed_insert(AccessControlExposeHeaders::from_iter([ETAG]));
-    headers.typed_insert(Pragma::no_cache());
-    headers.typed_insert(CacheControl::new().with_no_store());
-    headers.typed_insert(session.etag());
-    headers.typed_insert(session.expires());
-    headers.typed_insert(session.last_modified());
-}
-
-#[pyclass]
-struct RendezvousHandler {
-    base: Uri,
-    clock: PyObject,
-    sessions: BTreeMap<Ulid, Session>,
-    capacity: usize,
-    max_content_length: u64,
-    ttl: Duration,
-}
-
-impl RendezvousHandler {
-    /// Check the input headers of a request which sets data for a session, and return the content type.
-    fn check_input_headers(&self, headers: &HeaderMap) -> PyResult<Mime> {
-        let ContentLength(content_length) = headers.typed_get_required()?;
-
-        if content_length > self.max_content_length {
-            return Err(SynapseError::new(
-                StatusCode::PAYLOAD_TOO_LARGE,
-                "Payload too large".to_owned(),
-                "M_TOO_LARGE",
-                None,
-                None,
-            ));
-        }
-
-        let content_type: ContentType = headers.typed_get_required()?;
-
-        // Content-Type must be text/plain
-        if content_type != ContentType::text() {
-            return Err(SynapseError::new(
-                StatusCode::BAD_REQUEST,
-                "Content-Type must be text/plain".to_owned(),
-                "M_INVALID_PARAM",
-                None,
-                None,
-            ));
-        }
-
-        Ok(content_type.into())
-    }
-
-    /// Evict expired sessions and remove the oldest sessions until we're under the capacity.
-    fn evict(&mut self, now: SystemTime) {
-        // First remove all the entries which expired
-        self.sessions.retain(|_, session| !session.expired(now));
-
-        // Then we remove the oldest entires until we're under the limit
-        while self.sessions.len() > self.capacity {
-            self.sessions.pop_first();
-        }
-    }
-}
-
-#[pymethods]
-impl RendezvousHandler {
-    #[new]
-    #[pyo3(signature = (homeserver, /, capacity=100, max_content_length=4*1024, eviction_interval=60*1000, ttl=60*1000))]
-    fn new(
-        py: Python<'_>,
-        homeserver: &PyAny,
-        capacity: usize,
-        max_content_length: u64,
-        eviction_interval: u64,
-        ttl: u64,
-    ) -> PyResult<Py<Self>> {
-        let base: String = homeserver
-            .getattr("config")?
-            .getattr("server")?
-            .getattr("public_baseurl")?
-            .extract()?;
-        let base = Uri::try_from(format!("{base}_synapse/client/rendezvous"))
-            .map_err(|_| PyValueError::new_err("Invalid base URI"))?;
-
-        let clock = homeserver.call_method0("get_clock")?.to_object(py);
-
-        // Construct a Python object so that we can get a reference to the
-        // evict method and schedule it to run.
-        let self_ = Py::new(
-            py,
-            Self {
-                base,
-                clock,
-                sessions: BTreeMap::new(),
-                capacity,
-                max_content_length,
-                ttl: Duration::from_millis(ttl),
-            },
-        )?;
-
-        let evict = self_.getattr(py, "_evict")?;
-        homeserver.call_method0("get_clock")?.call_method(
-            "looping_call",
-            (evict, eviction_interval),
-            None,
-        )?;
-
-        Ok(self_)
-    }
-
-    fn _evict(&mut self, py: Python<'_>) -> PyResult<()> {
-        let clock = self.clock.as_ref(py);
-        let now: u64 = clock.call_method0("time_msec")?.extract()?;
-        let now = SystemTime::UNIX_EPOCH + Duration::from_millis(now);
-        self.evict(now);
-
-        Ok(())
-    }
-
-    fn handle_post(&mut self, py: Python<'_>, twisted_request: &PyAny) -> PyResult<()> {
-        let request = http_request_from_twisted(twisted_request)?;
-
-        let content_type = self.check_input_headers(request.headers())?;
-
-        let clock = self.clock.as_ref(py);
-        let now: u64 = clock.call_method0("time_msec")?.extract()?;
-        let now = SystemTime::UNIX_EPOCH + Duration::from_millis(now);
-
-        // We trigger an immediate eviction if we're at 2x the capacity
-        if self.sessions.len() >= self.capacity * 2 {
-            self.evict(now);
-        }
-
-        // Generate a new ULID for the session from the current time.
-        let id = Ulid::from_datetime(now);
-
-        let uri = format!("{base}/{id}", base = self.base);
-
-        let body = request.into_body();
-
-        let session = Session::new(body, content_type, now, self.ttl);
-
-        let response = serde_json::json!({
-            "url": uri,
-        })
-        .to_string();
-
-        let mut response = Response::new(response.as_bytes());
-        *response.status_mut() = StatusCode::CREATED;
-        response.headers_mut().typed_insert(ContentType::json());
-        prepare_headers(response.headers_mut(), &session);
-        http_response_to_twisted(twisted_request, response)?;
-
-        self.sessions.insert(id, session);
-
-        Ok(())
-    }
-
-    fn handle_get(&mut self, py: Python<'_>, twisted_request: &PyAny, id: &str) -> PyResult<()> {
-        let request = http_request_from_twisted(twisted_request)?;
-
-        let if_none_match: Option<IfNoneMatch> = request.headers().typed_get_optional()?;
-
-        let now: u64 = self.clock.call_method0(py, "time_msec")?.extract(py)?;
-        let now = SystemTime::UNIX_EPOCH + Duration::from_millis(now);
-
-        let id: Ulid = id.parse().map_err(|_| NotFoundError::new())?;
-        let session = self
-            .sessions
-            .get(&id)
-            .filter(|s| !s.expired(now))
-            .ok_or_else(NotFoundError::new)?;
-
-        if let Some(if_none_match) = if_none_match {
-            if !if_none_match.precondition_passes(&session.etag()) {
-                let mut response = Response::new(Bytes::new());
-                *response.status_mut() = StatusCode::NOT_MODIFIED;
-                prepare_headers(response.headers_mut(), session);
-                http_response_to_twisted(twisted_request, response)?;
-                return Ok(());
-            }
-        }
-
-        let mut response = Response::new(session.data());
-        *response.status_mut() = StatusCode::OK;
-        let headers = response.headers_mut();
-        prepare_headers(headers, session);
-        headers.typed_insert(session.content_type());
-        headers.typed_insert(session.content_length());
-        http_response_to_twisted(twisted_request, response)?;
-
-        Ok(())
-    }
-
-    fn handle_put(&mut self, py: Python<'_>, twisted_request: &PyAny, id: &str) -> PyResult<()> {
-        let request = http_request_from_twisted(twisted_request)?;
-
-        let content_type = self.check_input_headers(request.headers())?;
-
-        let if_match: IfMatch = request.headers().typed_get_required()?;
-
-        let data = request.into_body();
-
-        let now: u64 = self.clock.call_method0(py, "time_msec")?.extract(py)?;
-        let now = SystemTime::UNIX_EPOCH + Duration::from_millis(now);
-
-        let id: Ulid = id.parse().map_err(|_| NotFoundError::new())?;
-        let session = self
-            .sessions
-            .get_mut(&id)
-            .filter(|s| !s.expired(now))
-            .ok_or_else(NotFoundError::new)?;
-
-        if !if_match.precondition_passes(&session.etag()) {
-            let mut headers = HeaderMap::new();
-            prepare_headers(&mut headers, session);
-
-            let mut additional_fields = HashMap::with_capacity(1);
-            additional_fields.insert(
-                String::from("org.matrix.msc4108.errcode"),
-                String::from("M_CONCURRENT_WRITE"),
-            );
-
-            return Err(SynapseError::new(
-                StatusCode::PRECONDITION_FAILED,
-                "ETag does not match".to_owned(),
-                "M_UNKNOWN", // Would be M_CONCURRENT_WRITE
-                Some(additional_fields),
-                Some(headers),
-            ));
-        }
-
-        session.update(data, content_type, now);
-
-        let mut response = Response::new(Bytes::new());
-        *response.status_mut() = StatusCode::ACCEPTED;
-        prepare_headers(response.headers_mut(), session);
-        http_response_to_twisted(twisted_request, response)?;
-
-        Ok(())
-    }
-
-    fn handle_delete(&mut self, twisted_request: &PyAny, id: &str) -> PyResult<()> {
-        let _request = http_request_from_twisted(twisted_request)?;
-
-        let id: Ulid = id.parse().map_err(|_| NotFoundError::new())?;
-        let _session = self.sessions.remove(&id).ok_or_else(NotFoundError::new)?;
-
-        let mut response = Response::new(Bytes::new());
-        *response.status_mut() = StatusCode::NO_CONTENT;
-        response
-            .headers_mut()
-            .typed_insert(AccessControlAllowOrigin::ANY);
-        http_response_to_twisted(twisted_request, response)?;
-
-        Ok(())
-    }
-}
-
-pub fn register_module(py: Python<'_>, m: &PyModule) -> PyResult<()> {
-    let child_module = PyModule::new(py, "rendezvous")?;
-
-    child_module.add_class::<RendezvousHandler>()?;
-
-    m.add_submodule(child_module)?;
-
-    // We need to manually add the module to sys.modules to make `from
-    // synapse.synapse_rust import rendezvous` work.
-    py.import("sys")?
-        .getattr("modules")?
-        .set_item("synapse.synapse_rust.rendezvous", child_module)?;
-
-    Ok(())
-}
@@ -1,91 +0,0 @@
-/*
- * This file is licensed under the Affero General Public License (AGPL) version 3.
- *
- * Copyright (C) 2024 New Vector, Ltd
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * See the GNU Affero General Public License for more details:
- * <https://www.gnu.org/licenses/agpl-3.0.html>.
- */
-
-use std::time::{Duration, SystemTime};
-
-use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine as _};
-use bytes::Bytes;
-use headers::{ContentLength, ContentType, ETag, Expires, LastModified};
-use mime::Mime;
-use sha2::{Digest, Sha256};
-
-/// A single session, containing data, metadata, and expiry information.
-pub struct Session {
-    hash: [u8; 32],
-    data: Bytes,
-    content_type: Mime,
-    last_modified: SystemTime,
-    expires: SystemTime,
-}
-
-impl Session {
-    /// Create a new session with the given data, content type, and time-to-live.
-    pub fn new(data: Bytes, content_type: Mime, now: SystemTime, ttl: Duration) -> Self {
-        let hash = Sha256::digest(&data).into();
-        Self {
-            hash,
-            data,
-            content_type,
-            expires: now + ttl,
-            last_modified: now,
-        }
-    }
-
-    /// Returns true if the session has expired at the given time.
-    pub fn expired(&self, now: SystemTime) -> bool {
-        self.expires <= now
-    }
-
-    /// Update the session with new data, content type, and last modified time.
-    pub fn update(&mut self, data: Bytes, content_type: Mime, now: SystemTime) {
-        self.hash = Sha256::digest(&data).into();
-        self.data = data;
-        self.content_type = content_type;
-        self.last_modified = now;
-    }
-
-    /// Returns the Content-Type header of the session.
-    pub fn content_type(&self) -> ContentType {
-        self.content_type.clone().into()
-    }
-
-    /// Returns the Content-Length header of the session.
-    pub fn content_length(&self) -> ContentLength {
-        ContentLength(self.data.len() as _)
-    }
-
-    /// Returns the ETag header of the session.
-    pub fn etag(&self) -> ETag {
-        let encoded = URL_SAFE_NO_PAD.encode(self.hash);
-        // SAFETY: Base64 encoding is URL-safe, so ETag-safe
-        format!("\"{encoded}\"")
-            .parse()
-            .expect("base64-encoded hash should be URL-safe")
-    }
-
-    /// Returns the Last-Modified header of the session.
-    pub fn last_modified(&self) -> LastModified {
-        self.last_modified.into()
-    }
-
-    /// Returns the Expires header of the session.
-    pub fn expires(&self) -> Expires {
-        self.expires.into()
-    }
-
-    /// Returns the current data stored in the session.
-    pub fn data(&self) -> Bytes {
-        self.data.clone()
-    }
-}
@@ -214,17 +214,7 @@ fi

 extra_test_args=()

-test_packages=(
-    ./tests/csapi
-    ./tests
-    ./tests/msc3874
-    ./tests/msc3890
-    ./tests/msc3391
-    ./tests/msc3930
-    ./tests/msc3902
-    ./tests/msc3967
-    ./tests/msc4115
-)
+test_packages="./tests/csapi ./tests ./tests/msc3874 ./tests/msc3890 ./tests/msc3391 ./tests/msc3930 ./tests/msc3902"

 # Enable dirty runs, so tests will reuse the same container where possible.
 # This significantly speeds up tests, but increases the possibility of test pollution.
@@ -288,7 +278,7 @@ fi
 export PASS_SYNAPSE_LOG_TESTING=1

 # Run the tests!
-echo "Images built; running complement with ${extra_test_args[@]} $@ ${test_packages[@]}"
+echo "Images built; running complement with ${extra_test_args[@]} $@ $test_packages"
 cd "$COMPLEMENT_DIR"

-go test -v -tags "synapse_blacklist" -count=1 "${extra_test_args[@]}" "$@" "${test_packages[@]}"
+go test -v -tags "synapse_blacklist" -count=1 "${extra_test_args[@]}" "$@" $test_packages
@@ -91,6 +91,7 @@ else
          "synapse" "docker" "tests"
          "scripts-dev"
          "contrib" "synmark" "stubs" ".ci"
+          "dev-docs"
      )
  fi
 fi
@@ -127,7 +127,7 @@ BOOLEAN_COLUMNS = {
    "redactions": ["have_censored"],
    "room_stats_state": ["is_federatable"],
    "rooms": ["is_public", "has_auth_chain_index"],
-    "users": ["shadow_banned", "approved", "locked", "suspended"],
+    "users": ["shadow_banned", "approved", "locked"],
    "un_partial_stated_event_stream": ["rejection_status_changed"],
    "users_who_share_rooms": ["share_private"],
    "per_user_experimental_features": ["enabled"],
@@ -234,13 +234,6 @@ class EventContentFields:
    TO_DEVICE_MSGID: Final = "org.matrix.msgid"


-class EventUnsignedContentFields:
-    """Fields found inside the 'unsigned' data on events"""
-
-    # Requesting user's membership, per MSC4115
-    MSC4115_MEMBERSHIP: Final = "io.element.msc4115.membership"
-
-
 class RoomTypes:
    """Understood values of the room_type field of m.room.create events."""

@@ -52,7 +52,6 @@ DEFAULT_SUBJECTS = {
    "invite_from_person_to_space": "[%(app)s] %(person)s has invited you to join the %(space)s space on %(app)s...",
    "password_reset": "[%(server_name)s] Password reset",
    "email_validation": "[%(server_name)s] Validate your email",
-    "email_already_in_use": "[%(server_name)s] Email already in use",
 }

 LEGACY_TEMPLATE_DIR_WARNING = """
@@ -77,7 +76,6 @@ class EmailSubjectConfig:
    invite_from_person_to_space: str
    password_reset: str
    email_validation: str
-    email_already_in_use: str


 class EmailConfig(Config):
@@ -182,12 +180,6 @@ class EmailConfig(Config):
            registration_template_text = email_config.get(
                "registration_template_text", "registration.txt"
            )
-            already_in_use_template_html = email_config.get(
-                "already_in_use_template_html", "already_in_use.html"
-            )
-            already_in_use_template_text = email_config.get(
-                "already_in_use_template_html", "already_in_use.txt"
-            )
            add_threepid_template_html = email_config.get(
                "add_threepid_template_html", "add_threepid.html"
            )
@@ -223,8 +215,6 @@ class EmailConfig(Config):
                self.email_password_reset_template_text,
                self.email_registration_template_html,
                self.email_registration_template_text,
-                self.email_already_in_use_template_html,
-                self.email_already_in_use_template_text,
                self.email_add_threepid_template_html,
                self.email_add_threepid_template_text,
                self.email_password_reset_template_confirmation_html,
@@ -240,8 +230,6 @@ class EmailConfig(Config):
                    password_reset_template_text,
                    registration_template_html,
                    registration_template_text,
-                    already_in_use_template_html,
-                    already_in_use_template_text,
                    add_threepid_template_html,
                    add_threepid_template_text,
                    "password_reset_confirmation.html",
@@ -393,6 +393,11 @@ class ExperimentalConfig(Config):
        # MSC3967: Do not require UIA when first uploading cross signing keys
        self.msc3967_enabled = experimental.get("msc3967_enabled", False)

+        # MSC3981: Recurse relations
+        self.msc3981_recurse_relations = experimental.get(
+            "msc3981_recurse_relations", False
+        )
+
        # MSC3861: Matrix architecture change to delegate authentication via OIDC
        try:
            self.msc3861 = MSC3861(**experimental.get("msc3861", {}))
@@ -404,6 +409,11 @@ class ExperimentalConfig(Config):
        # Check that none of the other config options conflict with MSC3861 when enabled
        self.msc3861.check_config_conflicts(self.root)

+        # MSC4010: Do not allow setting m.push_rules account data.
+        self.msc4010_push_rules_account_data = experimental.get(
+            "msc4010_push_rules_account_data", False
+        )
+
        self.msc4028_push_encrypted_events = experimental.get(
            "msc4028_push_encrypted_events", False
        )
@@ -411,28 +421,3 @@ class ExperimentalConfig(Config):
        self.msc4069_profile_inhibit_propagation = experimental.get(
            "msc4069_profile_inhibit_propagation", False
        )
-
-        # MSC4108: Mechanism to allow OIDC sign in and E2EE set up via QR code
-        self.msc4108_enabled = experimental.get("msc4108_enabled", False)
-
-        self.msc4108_delegation_endpoint: Optional[str] = experimental.get(
-            "msc4108_delegation_endpoint", None
-        )
-
-        if (
-            self.msc4108_enabled or self.msc4108_delegation_endpoint is not None
-        ) and not self.msc3861.enabled:
-            raise ConfigError(
-                "MSC4108 requires MSC3861 to be enabled",
-                ("experimental", "msc4108_delegation_endpoint"),
-            )
-
-        if self.msc4108_delegation_endpoint is not None and self.msc4108_enabled:
-            raise ConfigError(
-                "You cannot have MSC4108 both enabled and delegated at the same time",
-                ("experimental", "msc4108_delegation_endpoint"),
-            )
-
-        self.msc4115_membership_on_events = experimental.get(
-            "msc4115_membership_on_events", False
-        )
@@ -49,7 +49,7 @@ from synapse.api.errors import Codes, SynapseError
 from synapse.api.room_versions import RoomVersion
 from synapse.types import JsonDict, Requester

-from . import EventBase, make_event_from_dict
+from . import EventBase

 if TYPE_CHECKING:
    from synapse.handlers.relations import BundledAggregations
@@ -82,14 +82,17 @@ def prune_event(event: EventBase) -> EventBase:
    """
    pruned_event_dict = prune_event_dict(event.room_version, event.get_dict())

+    from . import make_event_from_dict
+
    pruned_event = make_event_from_dict(
        pruned_event_dict, event.room_version, event.internal_metadata.get_dict()
    )

-    # Copy the bits of `internal_metadata` that aren't returned by `get_dict`
+    # copy the internal fields
    pruned_event.internal_metadata.stream_ordering = (
        event.internal_metadata.stream_ordering
    )
+
    pruned_event.internal_metadata.outlier = event.internal_metadata.outlier

    # Mark the event as redacted
@@ -98,29 +101,6 @@ def prune_event(event: EventBase) -> EventBase:
    return pruned_event


-def clone_event(event: EventBase) -> EventBase:
-    """Take a copy of the event.
-
-    This is mostly useful because it does a *shallow* copy of the `unsigned` data,
-    which means it can then be updated without corrupting the in-memory cache. Note that
-    other properties of the event, such as `content`, are *not* (currently) copied here.
-    """
-    # XXX: We rely on at least one of `event.get_dict()` and `make_event_from_dict()`
-    #   making a copy of `unsigned`. Currently, both do, though I don't really know why.
-    #   Still, as long as they do, there's not much point doing yet another copy here.
-    new_event = make_event_from_dict(
-        event.get_dict(), event.room_version, event.internal_metadata.get_dict()
-    )
-
-    # Copy the bits of `internal_metadata` that aren't returned by `get_dict`.
-    new_event.internal_metadata.stream_ordering = (
-        event.internal_metadata.stream_ordering
-    )
-    new_event.internal_metadata.outlier = event.internal_metadata.outlier
-
-    return new_event
-
-
 def prune_event_dict(room_version: RoomVersion, event_dict: JsonDict) -> JsonDict:
    """Redacts the event_dict in the same way as `prune_event`, except it
    operates on dicts rather than event objects
@@ -546,25 +546,7 @@ class FederationServer(FederationBase):
                edu_type=edu_dict["edu_type"],
                content=edu_dict["content"],
            )
-            try:
-                await self.registry.on_edu(edu.edu_type, origin, edu.content)
-            except Exception:
-                # If there was an error handling the EDU, we must reject the
-                # transaction.
-                #
-                # Some EDU types (notably, to-device messages) are, despite their name,
-                # expected to be reliable; if we weren't able to do something with it,
-                # we have to tell the sender that, and the only way the protocol gives
-                # us to do so is by sending an HTTP error back on the transaction.
-                #
-                # We log the exception now, and then raise a new SynapseError to cause
-                # the transaction to be failed.
-                logger.exception("Error handling EDU of type %s", edu.edu_type)
-                raise SynapseError(500, f"Error handing EDU of type {edu.edu_type}")
-
-                # TODO: if the first EDU fails, we should probably abort the whole
-                #   thing rather than carrying on with the rest of them. That would
-                #   probably be best done inside `concurrently_execute`.
+            await self.registry.on_edu(edu.edu_type, origin, edu.content)

        await concurrently_execute(
            _process_edu,
@@ -1432,7 +1414,12 @@ class FederationHandlerRegistry:
        handler = self.edu_handlers.get(edu_type)
        if handler:
            with start_active_span_from_edu(content, "handle_edu"):
-                await handler(origin, content)
+                try:
+                    await handler(origin, content)
+                except SynapseError as e:
+                    logger.info("Failed to handle edu %r: %r", edu_type, e)
+                except Exception:
+                    logger.exception("Failed to handle edu %r", edu_type)
            return

        # Check if we can route it somewhere else that isn't us
@@ -1441,12 +1428,17 @@ class FederationHandlerRegistry:
            # Pick an instance randomly so that we don't overload one.
            route_to = random.choice(instances)

-            await self._send_edu(
-                instance_name=route_to,
-                edu_type=edu_type,
-                origin=origin,
-                content=content,
-            )
+            try:
+                await self._send_edu(
+                    instance_name=route_to,
+                    edu_type=edu_type,
+                    origin=origin,
+                    content=content,
+                )
+            except SynapseError as e:
+                logger.info("Failed to handle edu %r: %r", edu_type, e)
+            except Exception:
+                logger.exception("Failed to handle edu %r", edu_type)
            return

        # Oh well, let's just log and move on.
@@ -42,7 +42,6 @@ class AdminHandler:
        self._device_handler = hs.get_device_handler()
        self._storage_controllers = hs.get_storage_controllers()
        self._state_storage_controller = self._storage_controllers.state
-        self._hs_config = hs.config
        self._msc3866_enabled = hs.config.experimental.msc3866.enabled

    async def get_whois(self, user: UserID) -> JsonMapping:
@@ -218,10 +217,7 @@ class AdminHandler:
                )

                events = await filter_events_for_client(
-                    self._storage_controllers,
-                    user_id,
-                    events,
-                    msc4115_membership_on_events=self._hs_config.experimental.msc4115_membership_on_events,
+                    self._storage_controllers, user_id, events
                )

                writer.write_events(room_id, events)
@@ -261,22 +261,11 @@ class DeactivateAccountHandler:
        user = UserID.from_string(user_id)

        rooms_for_user = await self.store.get_rooms_for_user(user_id)
-        requester = create_requester(user, authenticated_entity=self._server_name)
-        should_erase = await self.store.is_user_erased(user_id)
-
        for room_id in rooms_for_user:
            logger.info("User parter parting %r from %r", user_id, room_id)
            try:
-                # Before parting the user, redact all membership events if requested
-                if should_erase:
-                    event_ids = await self.store.get_membership_event_ids_for_user(
-                        user_id, room_id
-                    )
-                    for event_id in event_ids:
-                        await self.store.expire_event(event_id)
-
                await self._room_member_handler.update_membership(
-                    requester,
+                    create_requester(user, authenticated_entity=self._server_name),
                    user,
                    room_id,
                    "leave",
@@ -104,9 +104,6 @@ class DeviceMessageHandler:
        """
        Handle receiving to-device messages from remote homeservers.

-        Note that any errors thrown from this method will cause the federation /send
-        request to receive an error response.
-
        Args:
            origin: The remote homeserver.
            content: The JSON dictionary containing the to-device messages.
@@ -1476,42 +1476,6 @@ class E2eKeysHandler:
        else:
            return exists, self.clock.time_msec() < ts_replacable_without_uia_before

-    async def has_different_keys(self, user_id: str, body: JsonDict) -> bool:
-        """
-        Check if a key provided in `body` differs from the same key stored in the DB. Returns
-        true on the first difference. If a key exists in `body` but does not exist in the DB,
-        returns True. If `body` has no keys, this always returns False.
-        Note by 'key' we mean Matrix key rather than JSON key.
-
-        The purpose of this function is to detect whether or not we need to apply UIA checks.
-        We must apply UIA checks if any key in the database is being overwritten. If a key is
-        being inserted for the first time, or if the key exactly matches what is in the database,
-        then no UIA check needs to be performed.
-
-        Args:
-            user_id: The user who sent the `body`.
-            body: The JSON request body from POST /keys/device_signing/upload
-        Returns:
-            True if any key in `body` has a different value in the database.
-        """
-        # Ensure that each key provided in the request body exactly matches the one we have stored.
-        # The first time we see the DB having a different key to the matching request key, bail.
-        # Note: we do not care if the DB has a key which the request does not specify, as we only
-        # care about *replacements* or *insertions* (i.e UPSERT)
-        req_body_key_to_db_key = {
-            "master_key": "master",
-            "self_signing_key": "self_signing",
-            "user_signing_key": "user_signing",
-        }
-        for req_body_key, db_key in req_body_key_to_db_key.items():
-            if req_body_key in body:
-                existing_key = await self.store.get_e2e_cross_signing_key(
-                    user_id, db_key
-                )
-                if existing_key != body[req_body_key]:
-                    return True
-        return False
-

 def _check_cross_signing_key(
    key: JsonDict, user_id: str, key_type: str, signing_key: Optional[VerifyKey] = None
@@ -148,7 +148,6 @@ class EventHandler:
    def __init__(self, hs: "HomeServer"):
        self.store = hs.get_datastores().main
        self._storage_controllers = hs.get_storage_controllers()
-        self._config = hs.config

    async def get_event(
        self,
@@ -190,11 +189,7 @@ class EventHandler:
        is_peeking = not is_user_in_room

        filtered = await filter_events_for_client(
-            self._storage_controllers,
-            user.to_string(),
-            [event],
-            is_peeking=is_peeking,
-            msc4115_membership_on_events=self._config.experimental.msc4115_membership_on_events,
+            self._storage_controllers, user.to_string(), [event], is_peeking=is_peeking
        )

        if not filtered:
@@ -221,10 +221,7 @@ class InitialSyncHandler:
                ).addErrback(unwrapFirstError)

                messages = await filter_events_for_client(
-                    self._storage_controllers,
-                    user_id,
-                    messages,
-                    msc4115_membership_on_events=self.hs.config.experimental.msc4115_membership_on_events,
+                    self._storage_controllers, user_id, messages
                )

                start_token = now_token.copy_and_replace(StreamKeyType.ROOM, token)
@@ -383,7 +380,6 @@ class InitialSyncHandler:
            requester.user.to_string(),
            messages,
            is_peeking=is_peeking,
-            msc4115_membership_on_events=self.hs.config.experimental.msc4115_membership_on_events,
        )

        start_token = StreamToken.START.copy_and_replace(StreamKeyType.ROOM, token)
@@ -498,7 +494,6 @@ class InitialSyncHandler:
            requester.user.to_string(),
            messages,
            is_peeking=is_peeking,
-            msc4115_membership_on_events=self.hs.config.experimental.msc4115_membership_on_events,
        )

        start_token = now_token.copy_and_replace(StreamKeyType.ROOM, token)
@@ -623,7 +623,6 @@ class PaginationHandler:
                user_id,
                events,
                is_peeking=(member_event_id is None),
-                msc4115_membership_on_events=self.hs.config.experimental.msc4115_membership_on_events,
            )

        # if after the filter applied there are no more events
@@ -95,7 +95,6 @@ class RelationsHandler:
        self._event_handler = hs.get_event_handler()
        self._event_serializer = hs.get_event_client_serializer()
        self._event_creation_handler = hs.get_event_creation_handler()
-        self._config = hs.config

    async def get_relations(
        self,
@@ -164,7 +163,6 @@ class RelationsHandler:
            user_id,
            events,
            is_peeking=(member_event_id is None),
-            msc4115_membership_on_events=self._config.experimental.msc4115_membership_on_events,
        )

        # The relations returned for the requested event do include their
@@ -610,7 +608,6 @@ class RelationsHandler:
            user_id,
            events,
            is_peeking=(member_event_id is None),
-            msc4115_membership_on_events=self._config.experimental.msc4115_membership_on_events,
        )

        aggregations = await self.get_bundled_aggregations(
@@ -956,7 +956,6 @@ class RoomCreationHandler:
            room_alias=room_alias,
            power_level_content_override=power_level_content_override,
            creator_join_profile=creator_join_profile,
-            ignore_forced_encryption=ignore_forced_encryption,
        )

        # we avoid dropping the lock between invites, as otherwise joins can
@@ -1476,7 +1475,6 @@ class RoomContextHandler:
                user.to_string(),
                events,
                is_peeking=is_peeking,
-                msc4115_membership_on_events=self.hs.config.experimental.msc4115_membership_on_events,
            )

        event = await self.store.get_event(
@@ -752,36 +752,6 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
            and requester.user.to_string() == self._server_notices_mxid
        )

-        requester_suspended = await self.store.get_user_suspended_status(
-            requester.user.to_string()
-        )
-        if action == Membership.INVITE and requester_suspended:
-            raise SynapseError(
-                403,
-                "Sending invites while account is suspended is not allowed.",
-                Codes.USER_ACCOUNT_SUSPENDED,
-            )
-
-        if target.to_string() != requester.user.to_string():
-            target_suspended = await self.store.get_user_suspended_status(
-                target.to_string()
-            )
-        else:
-            target_suspended = requester_suspended
-
-        if action == Membership.JOIN and target_suspended:
-            raise SynapseError(
-                403,
-                "Joining rooms while account is suspended is not allowed.",
-                Codes.USER_ACCOUNT_SUSPENDED,
-            )
-        if action == Membership.KNOCK and target_suspended:
-            raise SynapseError(
-                403,
-                "Knocking on rooms while account is suspended is not allowed.",
-                Codes.USER_ACCOUNT_SUSPENDED,
-            )
-
        if (
            not self.allow_per_room_profiles and not is_requester_server_notices_user
        ) or requester.shadow_banned:
@@ -480,10 +480,7 @@ class SearchHandler:
        filtered_events = await search_filter.filter([r["event"] for r in results])

        events = await filter_events_for_client(
-            self._storage_controllers,
-            user.to_string(),
-            filtered_events,
-            msc4115_membership_on_events=self.hs.config.experimental.msc4115_membership_on_events,
+            self._storage_controllers, user.to_string(), filtered_events
        )

        events.sort(key=lambda e: -rank_map[e.event_id])
@@ -582,10 +579,7 @@ class SearchHandler:
            filtered_events = await search_filter.filter([r["event"] for r in results])

            events = await filter_events_for_client(
-                self._storage_controllers,
-                user.to_string(),
-                filtered_events,
-                msc4115_membership_on_events=self.hs.config.experimental.msc4115_membership_on_events,
+                self._storage_controllers, user.to_string(), filtered_events
            )

            room_events.extend(events)
@@ -670,17 +664,11 @@ class SearchHandler:
            )

            events_before = await filter_events_for_client(
-                self._storage_controllers,
-                user.to_string(),
-                res.events_before,
-                msc4115_membership_on_events=self.hs.config.experimental.msc4115_membership_on_events,
+                self._storage_controllers, user.to_string(), res.events_before
            )

            events_after = await filter_events_for_client(
-                self._storage_controllers,
-                user.to_string(),
-                res.events_after,
-                msc4115_membership_on_events=self.hs.config.experimental.msc4115_membership_on_events,
+                self._storage_controllers, user.to_string(), res.events_after
            )

            context: JsonDict = {
@@ -169,7 +169,6 @@ class UsernameMappingSession:
    # attributes returned by the ID mapper
    display_name: Optional[str]
    emails: StrCollection
-    avatar_url: Optional[str]

    # An optional dictionary of extra attributes to be provided to the client in the
    # login response.
@@ -184,7 +183,6 @@ class UsernameMappingSession:
    # choices made by the user
    chosen_localpart: Optional[str] = None
    use_display_name: bool = True
-    use_avatar: bool = True
    emails_to_use: StrCollection = ()
    terms_accepted_version: Optional[str] = None

@@ -662,9 +660,6 @@ class SsoHandler:
            remote_user_id=remote_user_id,
            display_name=attributes.display_name,
            emails=attributes.emails,
-            avatar_url=attributes.picture,
-            # Default to using all mapped emails. Will be overwritten in handle_submit_username_request.
-            emails_to_use=attributes.emails,
            client_redirect_url=client_redirect_url,
            expiry_time_ms=now + self._MAPPING_SESSION_VALIDITY_PERIOD_MS,
            extra_login_attributes=extra_login_attributes,
@@ -971,7 +966,6 @@ class SsoHandler:
        session_id: str,
        localpart: str,
        use_display_name: bool,
-        use_avatar: bool,
        emails_to_use: Iterable[str],
    ) -> None:
        """Handle a request to the username-picker 'submit' endpoint
@@ -994,7 +988,6 @@ class SsoHandler:
        # update the session with the user's choices
        session.chosen_localpart = localpart
        session.use_display_name = use_display_name
-        session.use_avatar = use_avatar

        emails_from_idp = set(session.emails)
        filtered_emails: Set[str] = set()
@@ -1075,9 +1068,6 @@ class SsoHandler:
        if session.use_display_name:
            attributes.display_name = session.display_name

-        if session.use_avatar:
-            attributes.picture = session.avatar_url
-
        # the following will raise a 400 error if the username has been taken in the
        # meantime.
        user_id = await self._register_mapped_user(
@@ -596,7 +596,6 @@ class SyncHandler:
                    sync_config.user.to_string(),
                    recents,
                    always_include_ids=current_state_ids,
-                    msc4115_membership_on_events=self.hs_config.experimental.msc4115_membership_on_events,
                )
                log_kv({"recents_after_visibility_filtering": len(recents)})
            else:
@@ -682,7 +681,6 @@ class SyncHandler:
                    sync_config.user.to_string(),
                    loaded_recents,
                    always_include_ids=current_state_ids,
-                    msc4115_membership_on_events=self.hs_config.experimental.msc4115_membership_on_events,
                )

                loaded_recents = []
@@ -1261,51 +1259,6 @@ class SyncHandler:
            await_full_state = True
            lazy_load_members = False

-        # For a non-gappy sync if the events in the timeline are simply a linear
-        # chain (i.e. no merging/branching of the graph), then we know the state
-        # delta between the end of the previous sync and start of the new one is
-        # empty.
-        #
-        # c.f. #16941 for an example of why we can't do this for all non-gappy
-        # syncs.
-        is_linear_timeline = True
-        if batch.events:
-            # We need to make sure the first event in our batch points to the
-            # last event in the previous batch.
-            last_event_id_prev_batch = (
-                await self.store.get_last_event_in_room_before_stream_ordering(
-                    room_id,
-                    end_token=since_token.room_key,
-                )
-            )
-
-            prev_event_id = last_event_id_prev_batch
-            for e in batch.events:
-                if e.prev_event_ids() != [prev_event_id]:
-                    is_linear_timeline = False
-                    break
-                prev_event_id = e.event_id
-
-        if is_linear_timeline and not batch.limited:
-            state_ids: StateMap[str] = {}
-            if lazy_load_members:
-                if members_to_fetch and batch.events:
-                    # We're lazy-loading, so the client might need some more
-                    # member events to understand the events in this timeline.
-                    # So we fish out all the member events corresponding to the
-                    # timeline here. The caller will then dedupe any redundant
-                    # ones.
-
-                    state_ids = await self._state_storage_controller.get_state_ids_for_event(
-                        batch.events[0].event_id,
-                        # we only want members!
-                        state_filter=StateFilter.from_types(
-                            (EventTypes.Member, member) for member in members_to_fetch
-                        ),
-                        await_full_state=False,
-                    )
-            return state_ids
-
        if batch:
            state_at_timeline_start = (
                await self._state_storage_controller.get_state_ids_for_event(
@@ -262,8 +262,7 @@ class _ProxyResponseBody(protocol.Protocol):
            self._request.finish()
        else:
            # Abort the underlying request since our remote request also failed.
-            if self._request.channel:
-                self._request.channel.forceAbortClient()
+            self._request.transport.abortConnection()


 class ProxySite(Site):
@@ -153,9 +153,9 @@ def return_json_error(
    # Only respond with an error response if we haven't already started writing,
    # otherwise lets just kill the connection
    if request.startedWriting:
-        if request.channel:
+        if request.transport:
            try:
-                request.channel.forceAbortClient()
+                request.transport.abortConnection()
            except Exception:
                # abortConnection throws if the connection is already closed
                pass
@@ -909,19 +909,7 @@ def set_cors_headers(request: "SynapseRequest") -> None:
    request.setHeader(
        b"Access-Control-Allow-Methods", b"GET, HEAD, POST, PUT, DELETE, OPTIONS"
    )
-    if request.path is not None and (
-        request.path == b"/_matrix/client/unstable/org.matrix.msc4108/rendezvous"
-        or request.path.startswith(b"/_synapse/client/rendezvous")
-    ):
-        request.setHeader(
-            b"Access-Control-Allow-Headers",
-            b"Content-Type, If-Match, If-None-Match",
-        )
-        request.setHeader(
-            b"Access-Control-Expose-Headers",
-            b"Synapse-Trace-Id, Server, ETag",
-        )
-    elif request.experimental_cors_msc3886:
+    if request.experimental_cors_msc3886:
        request.setHeader(
            b"Access-Control-Allow-Headers",
            b"X-Requested-With, Content-Type, Authorization, Date, If-Match, If-None-Match",
@@ -19,11 +19,9 @@
 #
 #

-"""This module contains base REST classes for constructing REST servlets."""
-
+""" This module contains base REST classes for constructing REST servlets. """
 import enum
 import logging
-import urllib.parse as urlparse
 from http import HTTPStatus
 from typing import (
    TYPE_CHECKING,
@@ -67,49 +65,17 @@ def parse_integer(request: Request, name: str, default: int) -> int: ...


@overload
-def parse_integer(
-    request: Request, name: str, *, default: int, negative: bool
-) -> int: ...
+def parse_integer(request: Request, name: str, *, required: Literal[True]) -> int: ...


@overload
 def parse_integer(
-    request: Request, name: str, *, default: int, negative: bool = False
-) -> int: ...
-
-
-@overload
-def parse_integer(
-    request: Request, name: str, *, required: Literal[True], negative: bool = False
-) -> int: ...
-
-
-@overload
-def parse_integer(
-    request: Request, name: str, *, default: Literal[None], negative: bool = False
-) -> None: ...
-
-
-@overload
-def parse_integer(request: Request, name: str, *, negative: bool) -> Optional[int]: ...
-
-
-@overload
-def parse_integer(
-    request: Request,
-    name: str,
-    default: Optional[int] = None,
-    required: bool = False,
-    negative: bool = False,
+    request: Request, name: str, default: Optional[int] = None, required: bool = False
 ) -> Optional[int]: ...


 def parse_integer(
-    request: Request,
-    name: str,
-    default: Optional[int] = None,
-    required: bool = False,
-    negative: bool = False,
+    request: Request, name: str, default: Optional[int] = None, required: bool = False
 ) -> Optional[int]:
    """Parse an integer parameter from the request string

@@ -119,17 +85,16 @@ def parse_integer(
        default: value to use if the parameter is absent, defaults to None.
        required: whether to raise a 400 SynapseError if the parameter is absent,
            defaults to False.
-        negative: whether to allow negative integers, defaults to True.
+
    Returns:
        An int value or the default.

    Raises:
-        SynapseError: if the parameter is absent and required, if the
-            parameter is present and not an integer, or if the
-            parameter is illegitimate negative.
+        SynapseError: if the parameter is absent and required, or if the
+            parameter is present and not an integer.
    """
    args: Mapping[bytes, Sequence[bytes]] = request.args  # type: ignore
-    return parse_integer_from_args(args, name, default, required, negative)
+    return parse_integer_from_args(args, name, default, required)


@overload
@@ -155,7 +120,6 @@ def parse_integer_from_args(
    name: str,
    default: Optional[int] = None,
    required: bool = False,
-    negative: bool = False,
 ) -> Optional[int]: ...


@@ -164,7 +128,6 @@ def parse_integer_from_args(
    name: str,
    default: Optional[int] = None,
    required: bool = False,
-    negative: bool = True,
 ) -> Optional[int]:
    """Parse an integer parameter from the request string

@@ -174,37 +137,33 @@ def parse_integer_from_args(
        default: value to use if the parameter is absent, defaults to None.
        required: whether to raise a 400 SynapseError if the parameter is absent,
            defaults to False.
-        negative: whether to allow negative integers, defaults to True.

    Returns:
        An int value or the default.

    Raises:
-        SynapseError: if the parameter is absent and required, if the
-            parameter is present and not an integer, or if the
-            parameter is illegitimate negative.
+        SynapseError: if the parameter is absent and required, or if the
+            parameter is present and not an integer.
    """
    name_bytes = name.encode("ascii")

-    if name_bytes not in args:
-        if not required:
+    if name_bytes in args:
+        try:
+            return int(args[name_bytes][0])
+        except Exception:
+            message = "Query parameter %r must be an integer" % (name,)
+            raise SynapseError(
+                HTTPStatus.BAD_REQUEST, message, errcode=Codes.INVALID_PARAM
+            )
+    else:
+        if required:
+            message = "Missing integer query parameter %r" % (name,)
+            raise SynapseError(
+                HTTPStatus.BAD_REQUEST, message, errcode=Codes.MISSING_PARAM
+            )
+        else:
            return default

-        message = f"Missing required integer query parameter {name}"
-        raise SynapseError(HTTPStatus.BAD_REQUEST, message, errcode=Codes.MISSING_PARAM)
-
-    try:
-        integer = int(args[name_bytes][0])
-    except Exception:
-        message = f"Query parameter {name} must be an integer"
-        raise SynapseError(HTTPStatus.BAD_REQUEST, message, errcode=Codes.INVALID_PARAM)
-
-    if not negative and integer < 0:
-        message = f"Query parameter {name} must be a positive integer."
-        raise SynapseError(HTTPStatus.BAD_REQUEST, message, errcode=Codes.INVALID_PARAM)
-
-    return integer
-

@overload
 def parse_boolean(request: Request, name: str, default: bool) -> bool: ...
@@ -451,87 +410,6 @@ def parse_string(
    )


-def parse_json(
-    request: Request,
-    name: str,
-    default: Optional[dict] = None,
-    required: bool = False,
-    encoding: str = "ascii",
-) -> Optional[JsonDict]:
-    """
-    Parse a JSON parameter from the request query string.
-
-    Args:
-        request: the twisted HTTP request.
-        name: the name of the query parameter.
-        default: value to use if the parameter is absent,
-            defaults to None.
-        required: whether to raise a 400 SynapseError if the
-           parameter is absent, defaults to False.
-        encoding: The encoding to decode the string content with.
-
-    Returns:
-        A JSON value, or `default` if the named query parameter was not found
-        and `required` was False.
-
-    Raises:
-        SynapseError if the parameter is absent and required, or if the
-            parameter is present and not a JSON object.
-    """
-    args: Mapping[bytes, Sequence[bytes]] = request.args  # type: ignore
-    return parse_json_from_args(
-        args,
-        name,
-        default,
-        required=required,
-        encoding=encoding,
-    )
-
-
-def parse_json_from_args(
-    args: Mapping[bytes, Sequence[bytes]],
-    name: str,
-    default: Optional[dict] = None,
-    required: bool = False,
-    encoding: str = "ascii",
-) -> Optional[JsonDict]:
-    """
-    Parse a JSON parameter from the request query string.
-
-    Args:
-        args: a mapping of request args as bytes to a list of bytes (e.g. request.args).
-        name: the name of the query parameter.
-        default: value to use if the parameter is absent,
-            defaults to None.
-        required: whether to raise a 400 SynapseError if the
-            parameter is absent, defaults to False.
-        encoding: the encoding to decode the string content with.
-
-        A JSON value, or `default` if the named query parameter was not found
-        and `required` was False.
-
-    Raises:
-        SynapseError if the parameter is absent and required, or if the
-            parameter is present and not a JSON object.
-    """
-    name_bytes = name.encode("ascii")
-
-    if name_bytes not in args:
-        if not required:
-            return default
-
-        message = f"Missing required integer query parameter {name}"
-        raise SynapseError(HTTPStatus.BAD_REQUEST, message, errcode=Codes.MISSING_PARAM)
-
-    json_str = parse_string_from_args(args, name, required=True, encoding=encoding)
-
-    try:
-        return json_decoder.decode(urlparse.unquote(json_str))
-    except Exception:
-        message = f"Query parameter {name} must be a valid JSON object"
-        raise SynapseError(HTTPStatus.BAD_REQUEST, message, errcode=Codes.NOT_JSON)
-
-
 EnumT = TypeVar("EnumT", bound=enum.Enum)


@@ -150,8 +150,7 @@ class SynapseRequest(Request):
                self.get_method(),
                self.get_redacted_uri(),
            )
-            if self.channel:
-                self.channel.forceAbortClient()
+            self.transport.abortConnection()
            return
        super().handleContentChunk(data)

@@ -721,7 +721,6 @@ class Notifier:
                        user.to_string(),
                        new_events,
                        is_peeking=is_peeking,
-                        msc4115_membership_on_events=self.hs.config.experimental.msc4115_membership_on_events,
                    )
                elif keyname == StreamKeyType.PRESENCE:
                    now = self.clock.time_msec()
@@ -205,22 +205,6 @@ class Mailer:
            template_vars,
        )

-    emails_sent_counter.labels("already_in_use")
-
-    async def send_already_in_use_mail(self, email_address: str) -> None:
-        """Send an email if the address is already bound to an user account
-
-        Args:
-            email_address: Email address we're sending to the "already in use" mail
-        """
-
-        await self.send_email(
-            email_address,
-            self.email_subjects.email_already_in_use
-            % {"server_name": self.hs.config.server.server_name, "app": self.app_name},
-            {},
-        )
-
    emails_sent_counter.labels("add_threepid")

    async def send_add_threepid_mail(
@@ -529,10 +513,7 @@ class Mailer:
        }

        the_events = await filter_events_for_client(
-            self._storage_controllers,
-            user_id,
-            results.events_before,
-            msc4115_membership_on_events=self.hs.config.experimental.msc4115_membership_on_events,
+            self._storage_controllers, user_id, results.events_before
        )
        the_events.append(notif_event)

@@ -1,12 +0,0 @@
-{% extends "_base.html" %}
-{% block title %}Email already in use{% endblock %}
-
-{% block body %}
-<p>You have asked us to register this email with a new Matrix account, but this email is already registered with an existing account.</p>
-
-<p>Please reset your password if needed.</p>
-
-<p>If this was not you, you can safely disregard this email.</p>
-
-<p>Thank you.</p>
-{% endblock %}
@@ -1,10 +0,0 @@
-Hello there,
-
-You have asked us to register this email with a new Matrix account,
-but this email is already registered with an existing account.
-
-Please reset your password if needed.
-
-If this was not you, you can safely disregard this email.
-
-Thank you.
@@ -23,7 +23,7 @@ from http import HTTPStatus
 from typing import TYPE_CHECKING, Tuple

 from synapse.api.constants import Direction
-from synapse.api.errors import NotFoundError, SynapseError
+from synapse.api.errors import Codes, NotFoundError, SynapseError
 from synapse.federation.transport.server import Authenticator
 from synapse.http.servlet import RestServlet, parse_enum, parse_integer, parse_string
 from synapse.http.site import SynapseRequest
@@ -61,8 +61,22 @@ class ListDestinationsRestServlet(RestServlet):
    async def on_GET(self, request: SynapseRequest) -> Tuple[int, JsonDict]:
        await assert_requester_is_admin(self._auth, request)

-        start = parse_integer(request, "from", default=0, negative=False)
-        limit = parse_integer(request, "limit", default=100, negative=False)
+        start = parse_integer(request, "from", default=0)
+        limit = parse_integer(request, "limit", default=100)
+
+        if start < 0:
+            raise SynapseError(
+                HTTPStatus.BAD_REQUEST,
+                "Query parameter from must be a string representing a positive integer.",
+                errcode=Codes.INVALID_PARAM,
+            )
+
+        if limit < 0:
+            raise SynapseError(
+                HTTPStatus.BAD_REQUEST,
+                "Query parameter limit must be a string representing a positive integer.",
+                errcode=Codes.INVALID_PARAM,
+            )

        destination = parse_string(request, "destination")

@@ -181,8 +195,22 @@ class DestinationMembershipRestServlet(RestServlet):
        if not await self._store.is_destination_known(destination):
            raise NotFoundError("Unknown destination")

-        start = parse_integer(request, "from", default=0, negative=False)
-        limit = parse_integer(request, "limit", default=100, negative=False)
+        start = parse_integer(request, "from", default=0)
+        limit = parse_integer(request, "limit", default=100)
+
+        if start < 0:
+            raise SynapseError(
+                HTTPStatus.BAD_REQUEST,
+                "Query parameter from must be a string representing a positive integer.",
+                errcode=Codes.INVALID_PARAM,
+            )
+
+        if limit < 0:
+            raise SynapseError(
+                HTTPStatus.BAD_REQUEST,
+                "Query parameter limit must be a string representing a positive integer.",
+                errcode=Codes.INVALID_PARAM,
+            )

        direction = parse_enum(request, "dir", Direction, default=Direction.FORWARDS)

@@ -311,17 +311,29 @@ class DeleteMediaByDateSize(RestServlet):
    ) -> Tuple[int, JsonDict]:
        await assert_requester_is_admin(self.auth, request)

-        before_ts = parse_integer(request, "before_ts", required=True, negative=False)
-        size_gt = parse_integer(request, "size_gt", default=0, negative=False)
+        before_ts = parse_integer(request, "before_ts", required=True)
+        size_gt = parse_integer(request, "size_gt", default=0)
        keep_profiles = parse_boolean(request, "keep_profiles", default=True)

-        if before_ts < 30000000000:  # Dec 1970 in milliseconds, Aug 2920 in seconds
+        if before_ts < 0:
+            raise SynapseError(
+                HTTPStatus.BAD_REQUEST,
+                "Query parameter before_ts must be a positive integer.",
+                errcode=Codes.INVALID_PARAM,
+            )
+        elif before_ts < 30000000000:  # Dec 1970 in milliseconds, Aug 2920 in seconds
            raise SynapseError(
                HTTPStatus.BAD_REQUEST,
                "Query parameter before_ts you provided is from the year 1970. "
                + "Double check that you are providing a timestamp in milliseconds.",
                errcode=Codes.INVALID_PARAM,
            )
+        if size_gt < 0:
+            raise SynapseError(
+                HTTPStatus.BAD_REQUEST,
+                "Query parameter size_gt must be a string representing a positive integer.",
+                errcode=Codes.INVALID_PARAM,
+            )

        # This check is useless, we keep it for the legacy endpoint only.
        if server_name is not None and self.server_name != server_name:
@@ -377,8 +389,22 @@ class UserMediaRestServlet(RestServlet):
        if user is None:
            raise NotFoundError("Unknown user")

-        start = parse_integer(request, "from", default=0, negative=False)
-        limit = parse_integer(request, "limit", default=100, negative=False)
+        start = parse_integer(request, "from", default=0)
+        limit = parse_integer(request, "limit", default=100)
+
+        if start < 0:
+            raise SynapseError(
+                HTTPStatus.BAD_REQUEST,
+                "Query parameter from must be a string representing a positive integer.",
+                errcode=Codes.INVALID_PARAM,
+            )
+
+        if limit < 0:
+            raise SynapseError(
+                HTTPStatus.BAD_REQUEST,
+                "Query parameter limit must be a string representing a positive integer.",
+                errcode=Codes.INVALID_PARAM,
+            )

        # If neither `order_by` nor `dir` is set, set the default order
        # to newest media is on top for backward compatibility.
@@ -421,8 +447,22 @@ class UserMediaRestServlet(RestServlet):
        if user is None:
            raise NotFoundError("Unknown user")

-        start = parse_integer(request, "from", default=0, negative=False)
-        limit = parse_integer(request, "limit", default=100, negative=False)
+        start = parse_integer(request, "from", default=0)
+        limit = parse_integer(request, "limit", default=100)
+
+        if start < 0:
+            raise SynapseError(
+                HTTPStatus.BAD_REQUEST,
+                "Query parameter from must be a string representing a positive integer.",
+                errcode=Codes.INVALID_PARAM,
+            )
+
+        if limit < 0:
+            raise SynapseError(
+                HTTPStatus.BAD_REQUEST,
+                "Query parameter limit must be a string representing a positive integer.",
+                errcode=Codes.INVALID_PARAM,
+            )

        # If neither `order_by` nor `dir` is set, set the default order
        # to newest media is on top for backward compatibility.
@@ -21,6 +21,7 @@
 import logging
 from http import HTTPStatus
 from typing import TYPE_CHECKING, List, Optional, Tuple, cast
+from urllib import parse as urlparse

 import attr

@@ -37,7 +38,6 @@ from synapse.http.servlet import (
    assert_params_in_dict,
    parse_enum,
    parse_integer,
-    parse_json,
    parse_json_object_from_request,
    parse_string,
 )
@@ -51,6 +51,7 @@ from synapse.storage.databases.main.room import RoomSortOrder
 from synapse.streams.config import PaginationConfig
 from synapse.types import JsonDict, RoomID, ScheduledTask, UserID, create_requester
 from synapse.types.state import StateFilter
+from synapse.util import json_decoder

 if TYPE_CHECKING:
    from synapse.api.auth import Auth
@@ -775,8 +776,14 @@ class RoomEventContextServlet(RestServlet):
        limit = parse_integer(request, "limit", default=10)

        # picking the API shape for symmetry with /messages
-        filter_json = parse_json(request, "filter", encoding="utf-8")
-        event_filter = Filter(self._hs, filter_json) if filter_json else None
+        filter_str = parse_string(request, "filter", encoding="utf-8")
+        if filter_str:
+            filter_json = urlparse.unquote(filter_str)
+            event_filter: Optional[Filter] = Filter(
+                self._hs, json_decoder.decode(filter_json)
+            )
+        else:
+            event_filter = None

        event_context = await self.room_context_handler.get_event_context(
            requester,
@@ -907,16 +914,21 @@ class RoomMessagesRestServlet(RestServlet):
        )
        # Twisted will have processed the args by now.
        assert request.args is not None
-
-        filter_json = parse_json(request, "filter", encoding="utf-8")
-        event_filter = Filter(self._hs, filter_json) if filter_json else None
-
        as_client_event = b"raw" not in request.args
-        if (
-            event_filter
-            and event_filter.filter_json.get("event_format", "client") == "federation"
-        ):
-            as_client_event = False
+        filter_str = parse_string(request, "filter", encoding="utf-8")
+        if filter_str:
+            filter_json = urlparse.unquote(filter_str)
+            event_filter: Optional[Filter] = Filter(
+                self._hs, json_decoder.decode(filter_json)
+            )
+            if (
+                event_filter
+                and event_filter.filter_json.get("event_format", "client")
+                == "federation"
+            ):
+                as_client_event = False
+        else:
+            event_filter = None

        msgs = await self._pagination_handler.get_messages(
            room_id=room_id,
@@ -63,12 +63,38 @@ class UserMediaStatisticsRestServlet(RestServlet):
            ),
        )

-        start = parse_integer(request, "from", default=0, negative=False)
-        limit = parse_integer(request, "limit", default=100, negative=False)
-        from_ts = parse_integer(request, "from_ts", default=0, negative=False)
-        until_ts = parse_integer(request, "until_ts", negative=False)
+        start = parse_integer(request, "from", default=0)
+        if start < 0:
+            raise SynapseError(
+                HTTPStatus.BAD_REQUEST,
+                "Query parameter from must be a string representing a positive integer.",
+                errcode=Codes.INVALID_PARAM,
+            )

+        limit = parse_integer(request, "limit", default=100)
+        if limit < 0:
+            raise SynapseError(
+                HTTPStatus.BAD_REQUEST,
+                "Query parameter limit must be a string representing a positive integer.",
+                errcode=Codes.INVALID_PARAM,
+            )
+
+        from_ts = parse_integer(request, "from_ts", default=0)
+        if from_ts < 0:
+            raise SynapseError(
+                HTTPStatus.BAD_REQUEST,
+                "Query parameter from_ts must be a string representing a positive integer.",
+                errcode=Codes.INVALID_PARAM,
+            )
+
+        until_ts = parse_integer(request, "until_ts")
        if until_ts is not None:
+            if until_ts < 0:
+                raise SynapseError(
+                    HTTPStatus.BAD_REQUEST,
+                    "Query parameter until_ts must be a string representing a positive integer.",
+                    errcode=Codes.INVALID_PARAM,
+                )
            if until_ts <= from_ts:
                raise SynapseError(
                    HTTPStatus.BAD_REQUEST,
@@ -90,8 +90,22 @@ class UsersRestServletV2(RestServlet):
    async def on_GET(self, request: SynapseRequest) -> Tuple[int, JsonDict]:
        await assert_requester_is_admin(self.auth, request)

-        start = parse_integer(request, "from", default=0, negative=False)
-        limit = parse_integer(request, "limit", default=100, negative=False)
+        start = parse_integer(request, "from", default=0)
+        limit = parse_integer(request, "limit", default=100)
+
+        if start < 0:
+            raise SynapseError(
+                HTTPStatus.BAD_REQUEST,
+                "Query parameter from must be a string representing a positive integer.",
+                errcode=Codes.INVALID_PARAM,
+            )
+
+        if limit < 0:
+            raise SynapseError(
+                HTTPStatus.BAD_REQUEST,
+                "Query parameter limit must be a string representing a positive integer.",
+                errcode=Codes.INVALID_PARAM,
+            )

        user_id = parse_string(request, "user_id")
        name = parse_string(request, "name", encoding="utf-8")
@@ -81,7 +81,8 @@ class AccountDataServlet(RestServlet):
            raise AuthError(403, "Cannot add account data for other users.")

        # Raise an error if the account data type cannot be set directly.
-        _check_can_set_account_data_type(account_data_type)
+        if self._hs.config.experimental.msc4010_push_rules_account_data:
+            _check_can_set_account_data_type(account_data_type)

        body = parse_json_object_from_request(request)

@@ -107,7 +108,10 @@ class AccountDataServlet(RestServlet):
            raise AuthError(403, "Cannot get account data for other users.")

        # Push rules are stored in a separate table and must be queried separately.
-        if account_data_type == AccountDataTypes.PUSH_RULES:
+        if (
+            self._hs.config.experimental.msc4010_push_rules_account_data
+            and account_data_type == AccountDataTypes.PUSH_RULES
+        ):
            account_data: Optional[JsonMapping] = (
                await self._push_rules_handler.push_rules_for_user(requester.user)
            )
@@ -158,7 +162,8 @@ class UnstableAccountDataServlet(RestServlet):
            raise AuthError(403, "Cannot delete account data for other users.")

        # Raise an error if the account data type cannot be set directly.
-        _check_can_set_account_data_type(account_data_type)
+        if self._hs.config.experimental.msc4010_push_rules_account_data:
+            _check_can_set_account_data_type(account_data_type)

        await self.handler.remove_account_data_for_user(user_id, account_data_type)

@@ -204,7 +209,15 @@ class RoomAccountDataServlet(RestServlet):
            )

        # Raise an error if the account data type cannot be set directly.
-        _check_can_set_account_data_type(account_data_type)
+        if self._hs.config.experimental.msc4010_push_rules_account_data:
+            _check_can_set_account_data_type(account_data_type)
+        elif account_data_type == ReceiptTypes.FULLY_READ:
+            raise SynapseError(
+                405,
+                "Cannot set m.fully_read through this API."
+                " Use /rooms/!roomId:server.name/read_markers",
+                Codes.BAD_JSON,
+            )

        body = parse_json_object_from_request(request)

@@ -243,7 +256,10 @@ class RoomAccountDataServlet(RestServlet):
            )

        # Room-specific push rules are not currently supported.
-        if account_data_type == AccountDataTypes.PUSH_RULES:
+        if (
+            self._hs.config.experimental.msc4010_push_rules_account_data
+            and account_data_type == AccountDataTypes.PUSH_RULES
+        ):
            account_data: Optional[JsonMapping] = {}
        else:
            account_data = await self.store.get_account_data_for_room_and_type(
@@ -301,7 +317,8 @@ class UnstableRoomAccountDataServlet(RestServlet):
            )

        # Raise an error if the account data type cannot be set directly.
-        _check_can_set_account_data_type(account_data_type)
+        if self._hs.config.experimental.msc4010_push_rules_account_data:
+            _check_can_set_account_data_type(account_data_type)

        await self.handler.remove_account_data_for_room(
            user_id, room_id, account_data_type
@@ -393,37 +393,23 @@ class SigningKeyUploadServlet(RestServlet):
        # time. Because there is no UIA in MSC3861, for now we throw an error if the
        # user tries to reset the device signing key when MSC3861 is enabled, but allow
        # first-time setup.
+        #
+        # XXX: We now have a get-out clause by which MAS can temporarily mark the master
+        # key as replaceable. It should do its own equivalent of user interactive auth
+        # before doing so.
        if self.hs.config.experimental.msc3861.enabled:
            # The auth service has to explicitly mark the master key as replaceable
            # without UIA to reset the device signing key with MSC3861.
            if is_cross_signing_setup and not master_key_updatable_without_uia:
-                config = self.hs.config.experimental.msc3861
-                if config.account_management_url is not None:
-                    url = f"{config.account_management_url}?action=org.matrix.cross_signing_reset"
-                else:
-                    url = config.issuer
-
                raise SynapseError(
                    HTTPStatus.NOT_IMPLEMENTED,
-                    "To reset your end-to-end encryption cross-signing identity, "
-                    f"you first need to approve it at {url} and then try again.",
+                    "Resetting cross signing keys is not yet supported with MSC3861",
                    Codes.UNRECOGNIZED,
                )
            # But first-time setup is fine

        elif self.hs.config.experimental.msc3967_enabled:
-            # MSC3967 allows this endpoint to 200 OK for idempotency. Resending exactly the same
-            # keys should just 200 OK without doing a UIA prompt.
-            keys_are_different = await self.e2e_keys_handler.has_different_keys(
-                user_id, body
-            )
-            if not keys_are_different:
-                # FIXME: we do not fallthrough to upload_signing_keys_for_user because confusingly
-                # if we do, we 500 as it looks like it tries to INSERT the same key twice, causing a
-                # unique key constraint violation. This sounds like a bug?
-                return 200, {}
-            # the keys are different, is x-signing set up? If no, then the keys don't exist which is
-            # why they are different. If yes, then we need to UIA to change them.
+            # If we already have a master key then cross signing is set up and we require UIA to reset
            if is_cross_signing_setup:
                await self.auth_handler.validate_user_via_ui_auth(
                    requester,
@@ -434,6 +420,7 @@ class SigningKeyUploadServlet(RestServlet):
                    can_skip_ui_auth=False,
                )
            # Otherwise we don't require UIA since we are setting up cross signing for first time
+
        else:
            # Previous behaviour is to always require UIA but allow it to be skipped
            await self.auth_handler.validate_user_via_ui_auth(
@@ -86,18 +86,12 @@ class EmailRegisterRequestTokenRestServlet(RestServlet):
        self.config = hs.config

        if self.hs.config.email.can_verify_email:
-            self.registration_mailer = Mailer(
+            self.mailer = Mailer(
                hs=self.hs,
                app_name=self.config.email.email_app_name,
                template_html=self.config.email.email_registration_template_html,
                template_text=self.config.email.email_registration_template_text,
            )
-            self.already_in_use_mailer = Mailer(
-                hs=self.hs,
-                app_name=self.config.email.email_app_name,
-                template_html=self.config.email.email_already_in_use_template_html,
-                template_text=self.config.email.email_already_in_use_template_text,
-            )

    async def on_POST(self, request: SynapseRequest) -> Tuple[int, JsonDict]:
        if not self.hs.config.email.can_verify_email:
@@ -145,10 +139,8 @@ class EmailRegisterRequestTokenRestServlet(RestServlet):
            if self.hs.config.server.request_token_inhibit_3pid_errors:
                # Make the client think the operation succeeded. See the rationale in the
                # comments for request_token_inhibit_3pid_errors.
-                # Still send an email to warn the user that an account already exists.
                # Also wait for some random amount of time between 100ms and 1s to make it
                # look like we did something.
-                await self.already_in_use_mailer.send_already_in_use_mail(email)
                await self.hs.get_clock().sleep(random.randint(1, 10) / 10)
                return 200, {"sid": random_string(16)}

@@ -159,7 +151,7 @@ class EmailRegisterRequestTokenRestServlet(RestServlet):
            email,
            client_secret,
            send_attempt,
-            self.registration_mailer.send_registration_mail,
+            self.mailer.send_registration_mail,
            next_link,
        )

@@ -55,6 +55,7 @@ class RelationPaginationServlet(RestServlet):
        self.auth = hs.get_auth()
        self._store = hs.get_datastores().main
        self._relations_handler = hs.get_relations_handler()
+        self._support_recurse = hs.config.experimental.msc3981_recurse_relations

    async def on_GET(
        self,
@@ -69,9 +70,12 @@ class RelationPaginationServlet(RestServlet):
        pagination_config = await PaginationConfig.from_request(
            self._store, request, default_limit=5, default_dir=Direction.BACKWARDS
        )
-        recurse = parse_boolean(request, "recurse", default=False) or parse_boolean(
-            request, "org.matrix.msc3981.recurse", default=False
-        )
+        if self._support_recurse:
+            recurse = parse_boolean(request, "recurse", default=False) or parse_boolean(
+                request, "org.matrix.msc3981.recurse", default=False
+            )
+        else:
+            recurse = False

        # The unstable version of this API returns an extra field for client
        # compatibility, see https://github.com/matrix-org/synapse/issues/12930.
@@ -2,7 +2,7 @@
 # This file is licensed under the Affero General Public License (AGPL) version 3.
 #
 # Copyright 2022 The Matrix.org Foundation C.I.C.
-# Copyright (C) 2023-2024 New Vector, Ltd
+# Copyright (C) 2023 New Vector, Ltd
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License as
@@ -34,7 +34,7 @@ if TYPE_CHECKING:
 logger = logging.getLogger(__name__)


-class MSC3886RendezvousServlet(RestServlet):
+class RendezvousServlet(RestServlet):
    """
    This is a placeholder implementation of [MSC3886](https://github.com/matrix-org/matrix-spec-proposals/pull/3886)
    simple client rendezvous capability that is used by the "Sign in with QR" functionality.
@@ -76,46 +76,6 @@ class MSC3886RendezvousServlet(RestServlet):
    # PUT, GET and DELETE are not implemented as they should be fulfilled by the redirect target.


-class MSC4108DelegationRendezvousServlet(RestServlet):
-    PATTERNS = client_patterns(
-        "/org.matrix.msc4108/rendezvous$", releases=[], v1=False, unstable=True
-    )
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__()
-        redirection_target: Optional[str] = (
-            hs.config.experimental.msc4108_delegation_endpoint
-        )
-        assert (
-            redirection_target is not None
-        ), "Servlet is only registered if there is a delegation target"
-        self.endpoint = redirection_target.encode("utf-8")
-
-    async def on_POST(self, request: SynapseRequest) -> None:
-        respond_with_redirect(
-            request, self.endpoint, statusCode=TEMPORARY_REDIRECT, cors=True
-        )
-
-
-class MSC4108RendezvousServlet(RestServlet):
-    PATTERNS = client_patterns(
-        "/org.matrix.msc4108/rendezvous$", releases=[], v1=False, unstable=True
-    )
-
-    def __init__(self, hs: "HomeServer") -> None:
-        super().__init__()
-        self._handler = hs.get_rendezvous_handler()
-
-    def on_POST(self, request: SynapseRequest) -> None:
-        self._handler.handle_post(request)
-
-
 def register_servlets(hs: "HomeServer", http_server: HttpServer) -> None:
    if hs.config.experimental.msc3886_endpoint is not None:
-        MSC3886RendezvousServlet(hs).register(http_server)
-
-    if hs.config.experimental.msc4108_enabled:
-        MSC4108RendezvousServlet(hs).register(http_server)
-
-    if hs.config.experimental.msc4108_delegation_endpoint is not None:
-        MSC4108DelegationRendezvousServlet(hs).register(http_server)
+        RendezvousServlet(hs).register(http_server)
@@ -52,7 +52,6 @@ from synapse.http.servlet import (
    parse_boolean,
    parse_enum,
    parse_integer,
-    parse_json,
    parse_json_object_from_request,
    parse_string,
    parse_strings_from_args,
@@ -66,6 +65,7 @@ from synapse.rest.client.transactions import HttpTransactionCache
 from synapse.streams.config import PaginationConfig
 from synapse.types import JsonDict, Requester, StreamToken, ThirdPartyInstanceID, UserID
 from synapse.types.state import StateFilter
+from synapse.util import json_decoder
 from synapse.util.cancellation import cancellable
 from synapse.util.stringutils import parse_and_validate_server_name, random_string

@@ -499,7 +499,7 @@ class PublicRoomListRestServlet(RestServlet):
            if server:
                raise e

-        limit: Optional[int] = parse_integer(request, "limit", 0, negative=False)
+        limit: Optional[int] = parse_integer(request, "limit", 0)
        since_token = parse_string(request, "since")

        if limit == 0:
@@ -703,16 +703,21 @@ class RoomMessageListRestServlet(RestServlet):
        )
        # Twisted will have processed the args by now.
        assert request.args is not None
-
-        filter_json = parse_json(request, "filter", encoding="utf-8")
-        event_filter = Filter(self._hs, filter_json) if filter_json else None
-
        as_client_event = b"raw" not in request.args
-        if (
-            event_filter
-            and event_filter.filter_json.get("event_format", "client") == "federation"
-        ):
-            as_client_event = False
+        filter_str = parse_string(request, "filter", encoding="utf-8")
+        if filter_str:
+            filter_json = urlparse.unquote(filter_str)
+            event_filter: Optional[Filter] = Filter(
+                self._hs, json_decoder.decode(filter_json)
+            )
+            if (
+                event_filter
+                and event_filter.filter_json.get("event_format", "client")
+                == "federation"
+            ):
+                as_client_event = False
+        else:
+            event_filter = None

        msgs = await self.pagination_handler.get_messages(
            room_id=room_id,
@@ -893,8 +898,14 @@ class RoomEventContextServlet(RestServlet):
        limit = parse_integer(request, "limit", default=10)

        # picking the API shape for symmetry with /messages
-        filter_json = parse_json(request, "filter", encoding="utf-8")
-        event_filter = Filter(self._hs, filter_json) if filter_json else None
+        filter_str = parse_string(request, "filter", encoding="utf-8")
+        if filter_str:
+            filter_json = urlparse.unquote(filter_str)
+            event_filter: Optional[Filter] = Filter(
+                self._hs, json_decoder.decode(filter_json)
+            )
+        else:
+            event_filter = None

        event_context = await self.room_context_handler.get_event_context(
            requester, room_id, event_id, limit, event_filter
@@ -1442,16 +1453,10 @@ class RoomHierarchyRestServlet(RestServlet):

 class RoomSummaryRestServlet(ResolveRoomIdMixin, RestServlet):
    PATTERNS = (
-        # deprecated endpoint, to be removed
        re.compile(
            "^/_matrix/client/unstable/im.nheko.summary"
            "/rooms/(?P<room_identifier>[^/]*)/summary$"
        ),
-        # recommended endpoint
-        re.compile(
-            "^/_matrix/client/unstable/im.nheko.summary"
-            "/summary/(?P<room_identifier>[^/]*)$"
-        ),
    )
    CATEGORY = "Client API requests"

@@ -89,7 +89,6 @@ class VersionsRestServlet(RestServlet):
                    "v1.7",
                    "v1.8",
                    "v1.9",
-                    "v1.10",
                ],
                # as per MSC1497:
                "unstable_features": {
@@ -133,22 +132,13 @@ class VersionsRestServlet(RestServlet):
                    # Adds support for relation-based redactions as per MSC3912.
                    "org.matrix.msc3912": self.config.experimental.msc3912_enabled,
                    # Whether recursively provide relations is supported.
-                    # TODO This is no longer needed once unstable MSC3981 does not need to be supported.
-                    "org.matrix.msc3981": True,
+                    "org.matrix.msc3981": self.config.experimental.msc3981_recurse_relations,
                    # Adds support for deleting account data.
                    "org.matrix.msc3391": self.config.experimental.msc3391_enabled,
                    # Allows clients to inhibit profile update propagation.
                    "org.matrix.msc4069": self.config.experimental.msc4069_profile_inhibit_propagation,
                    # Allows clients to handle push for encrypted events.
                    "org.matrix.msc4028": self.config.experimental.msc4028_push_encrypted_events,
-                    # MSC4108: Mechanism to allow OIDC sign in and E2EE set up via QR code
-                    "org.matrix.msc4108": (
-                        self.config.experimental.msc4108_enabled
-                        or (
-                            self.config.experimental.msc4108_delegation_endpoint
-                            is not None
-                        )
-                    ),
                },
            },
        )
@@ -72,6 +72,9 @@ class PreviewUrlResource(RestServlet):
        # XXX: if get_user_by_req fails, what should we do in an async render?
        requester = await self.auth.get_user_by_req(request)
        url = parse_string(request, "url", required=True)
-        ts = parse_integer(request, "ts", default=self.clock.time_msec())
+        ts = parse_integer(request, "ts")
+        if ts is None:
+            ts = self.clock.time_msec()
+
        og = await self.url_previewer.preview(url, requester.user, ts)
        respond_with_json_bytes(request, 200, og, send_cors=True)
@@ -26,7 +26,6 @@ from twisted.web.resource import Resource
 from synapse.rest.synapse.client.new_user_consent import NewUserConsentResource
 from synapse.rest.synapse.client.pick_idp import PickIdpResource
 from synapse.rest.synapse.client.pick_username import pick_username_resource
-from synapse.rest.synapse.client.rendezvous import MSC4108RendezvousSessionResource
 from synapse.rest.synapse.client.sso_register import SsoRegisterResource
 from synapse.rest.synapse.client.unsubscribe import UnsubscribeResource

@@ -77,9 +76,6 @@ def build_synapse_client_resource_tree(hs: "HomeServer") -> Mapping[str, Resourc
        # To be removed in Synapse v1.32.0.
        resources["/_matrix/saml2"] = res

-    if hs.config.experimental.msc4108_enabled:
-        resources["/_synapse/client/rendezvous"] = MSC4108RendezvousSessionResource(hs)
-
    return resources


@@ -113,7 +113,6 @@ class AccountDetailsResource(DirectServeHtmlResource):
                "display_name": session.display_name,
                "emails": session.emails,
                "localpart": localpart,
-                "avatar_url": session.avatar_url,
            },
        }

@@ -135,7 +134,6 @@ class AccountDetailsResource(DirectServeHtmlResource):
        try:
            localpart = parse_string(request, "username", required=True)
            use_display_name = parse_boolean(request, "use_display_name", default=False)
-            use_avatar = parse_boolean(request, "use_avatar", default=False)

            try:
                emails_to_use: List[str] = [
@@ -149,5 +147,5 @@ class AccountDetailsResource(DirectServeHtmlResource):
            return

        await self._sso_handler.handle_submit_username_request(
-            request, session_id, localpart, use_display_name, use_avatar, emails_to_use
+            request, session_id, localpart, use_display_name, emails_to_use
        )
@@ -1,58 +0,0 @@
-#
-# This file is licensed under the Affero General Public License (AGPL) version 3.
-#
-# Copyright (C) 2024 New Vector, Ltd
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as
-# published by the Free Software Foundation, either version 3 of the
-# License, or (at your option) any later version.
-#
-# See the GNU Affero General Public License for more details:
-# <https://www.gnu.org/licenses/agpl-3.0.html>.
-#
-#
-
-import logging
-from typing import TYPE_CHECKING, List
-
-from synapse.api.errors import UnrecognizedRequestError
-from synapse.http.server import DirectServeJsonResource
-from synapse.http.site import SynapseRequest
-
-if TYPE_CHECKING:
-    from synapse.server import HomeServer
-
-logger = logging.getLogger(__name__)
-
-
-class MSC4108RendezvousSessionResource(DirectServeJsonResource):
-    isLeaf = True
-
-    def __init__(self, hs: "HomeServer") -> None:
-        super().__init__()
-        self._handler = hs.get_rendezvous_handler()
-
-    async def _async_render_GET(self, request: SynapseRequest) -> None:
-        postpath: List[bytes] = request.postpath  # type: ignore
-        if len(postpath) != 1:
-            raise UnrecognizedRequestError()
-        session_id = postpath[0].decode("ascii")
-
-        self._handler.handle_get(request, session_id)
-
-    def _async_render_PUT(self, request: SynapseRequest) -> None:
-        postpath: List[bytes] = request.postpath  # type: ignore
-        if len(postpath) != 1:
-            raise UnrecognizedRequestError()
-        session_id = postpath[0].decode("ascii")
-
-        self._handler.handle_put(request, session_id)
-
-    def _async_render_DELETE(self, request: SynapseRequest) -> None:
-        postpath: List[bytes] = request.postpath  # type: ignore
-        if len(postpath) != 1:
-            raise UnrecognizedRequestError()
-        session_id = postpath[0].decode("ascii")
-
-        self._handler.handle_delete(request, session_id)
@@ -143,7 +143,6 @@ from synapse.state import StateHandler, StateResolutionHandler
 from synapse.storage import Databases
 from synapse.storage.controllers import StorageControllers
 from synapse.streams.events import EventSources
-from synapse.synapse_rust.rendezvous import RendezvousHandler
 from synapse.types import DomainSpecificString, ISynapseReactor
 from synapse.util import Clock
 from synapse.util.distributor import Distributor
@@ -860,10 +859,6 @@ class HomeServer(metaclass=abc.ABCMeta):
    def get_room_forgetter_handler(self) -> RoomForgetterHandler:
        return RoomForgetterHandler(self)

-    @cache_in_self
-    def get_rendezvous_handler(self) -> RendezvousHandler:
-        return RendezvousHandler(self)
-
    @cache_in_self
    def get_outbound_redis_connection(self) -> "ConnectionHandler":
        """
@@ -385,6 +385,7 @@ class EventPushActionsWorkerStore(ReceiptsWorkerStore, StreamWorkerStore, SQLBas
            WITH all_receipts AS (
                SELECT room_id, thread_id, MAX(event_stream_ordering) AS max_receipt_stream_ordering
                FROM receipts_linearized
+                LEFT JOIN events USING (room_id, event_id)
                WHERE
                    {receipt_types_clause}
                    AND user_id = ?
@@ -620,12 +621,13 @@ class EventPushActionsWorkerStore(ReceiptsWorkerStore, StreamWorkerStore, SQLBas
                SELECT notif_count, COALESCE(unread_count, 0), thread_id
                FROM event_push_summary
                LEFT JOIN (
-                    SELECT thread_id, MAX(event_stream_ordering) AS threaded_receipt_stream_ordering
+                    SELECT thread_id, MAX(stream_ordering) AS threaded_receipt_stream_ordering
                    FROM receipts_linearized
+                    LEFT JOIN events USING (room_id, event_id)
                    WHERE
                        user_id = ?
                        AND room_id = ?
-                        AND event_stream_ordering > ?
+                        AND stream_ordering > ?
                        AND {receipt_types_clause}
                    GROUP BY thread_id
                ) AS receipts USING (thread_id)
@@ -657,12 +659,13 @@ class EventPushActionsWorkerStore(ReceiptsWorkerStore, StreamWorkerStore, SQLBas
        sql = f"""
            SELECT COUNT(*), thread_id FROM event_push_actions
            LEFT JOIN (
-                SELECT thread_id, MAX(event_stream_ordering) AS threaded_receipt_stream_ordering
+                SELECT thread_id, MAX(stream_ordering) AS threaded_receipt_stream_ordering
                FROM receipts_linearized
+                LEFT JOIN events USING (room_id, event_id)
                WHERE
                    user_id = ?
                    AND room_id = ?
-                    AND event_stream_ordering > ?
+                    AND stream_ordering > ?
                    AND {receipt_types_clause}
                GROUP BY thread_id
            ) AS receipts USING (thread_id)
@@ -735,12 +738,13 @@ class EventPushActionsWorkerStore(ReceiptsWorkerStore, StreamWorkerStore, SQLBas
                thread_id
            FROM event_push_actions
            LEFT JOIN (
-                SELECT thread_id, MAX(event_stream_ordering) AS threaded_receipt_stream_ordering
+                SELECT thread_id, MAX(stream_ordering) AS threaded_receipt_stream_ordering
                FROM receipts_linearized
+                LEFT JOIN events USING (room_id, event_id)
                WHERE
                    user_id = ?
                    AND room_id = ?
-                    AND event_stream_ordering > ?
+                    AND stream_ordering > ?
                    AND {receipt_types_clause}
                GROUP BY thread_id
            ) AS receipts USING (thread_id)
@@ -906,8 +910,9 @@ class EventPushActionsWorkerStore(ReceiptsWorkerStore, StreamWorkerStore, SQLBas
        # given this function generally gets called with only one room and
        # thread ID.
        sql = f"""
-            SELECT room_id, thread_id, MAX(event_stream_ordering)
+            SELECT room_id, thread_id, MAX(stream_ordering)
            FROM receipts_linearized
+            INNER JOIN events USING (room_id, event_id)
            WHERE {receipt_types_clause}
                AND {thread_ids_clause}
                AND {room_ids_clause}
@@ -1437,8 +1442,9 @@ class EventPushActionsWorkerStore(ReceiptsWorkerStore, StreamWorkerStore, SQLBas
        )

        sql = """
-            SELECT r.stream_id, r.room_id, r.user_id, r.thread_id, r.event_stream_ordering
+            SELECT r.stream_id, r.room_id, r.user_id, r.thread_id, e.stream_ordering
            FROM receipts_linearized AS r
+            INNER JOIN events AS e USING (event_id)
            WHERE ? < r.stream_id AND r.stream_id <= ? AND user_id LIKE ?
            ORDER BY r.stream_id ASC
            LIMIT ?
@@ -19,7 +19,6 @@
 # [This file includes modifications made by New Vector Limited]
 #
 #
-import collections
 import itertools
 import logging
 from collections import OrderedDict
@@ -54,7 +53,6 @@ from synapse.storage.database import (
    LoggingDatabaseConnection,
    LoggingTransaction,
 )
-from synapse.storage.databases.main.event_federation import EventFederationStore
 from synapse.storage.databases.main.events_worker import EventCacheEntry
 from synapse.storage.databases.main.search import SearchEntry
 from synapse.storage.engines import PostgresEngine
@@ -770,26 +768,40 @@ class PersistEventsStore:
        #      that have the same chain ID as the event.
        #   2. For each retained auth event we:
        #       a. Add a link from the event's to the auth event's chain
-        #          ID/sequence number
+        #          ID/sequence number; and
+        #       b. Add a link from the event to every chain reachable by the
+        #          auth event.

        # Step 1, fetch all existing links from all the chains we've seen
        # referenced.
        chain_links = _LinkMap()
-
-        for links in EventFederationStore._get_chain_links(
-            txn, {chain_id for chain_id, _ in chain_map.values()}
-        ):
-            for origin_chain_id, inner_links in links.items():
-                for (
-                    origin_sequence_number,
-                    target_chain_id,
-                    target_sequence_number,
-                ) in inner_links:
-                    chain_links.add_link(
-                        (origin_chain_id, origin_sequence_number),
-                        (target_chain_id, target_sequence_number),
-                        new=False,
-                    )
+        auth_chain_rows = cast(
+            List[Tuple[int, int, int, int]],
+            db_pool.simple_select_many_txn(
+                txn,
+                table="event_auth_chain_links",
+                column="origin_chain_id",
+                iterable={chain_id for chain_id, _ in chain_map.values()},
+                keyvalues={},
+                retcols=(
+                    "origin_chain_id",
+                    "origin_sequence_number",
+                    "target_chain_id",
+                    "target_sequence_number",
+                ),
+            ),
+        )
+        for (
+            origin_chain_id,
+            origin_sequence_number,
+            target_chain_id,
+            target_sequence_number,
+        ) in auth_chain_rows:
+            chain_links.add_link(
+                (origin_chain_id, origin_sequence_number),
+                (target_chain_id, target_sequence_number),
+                new=False,
+            )

        # We do this in toplogical order to avoid adding redundant links.
        for event_id in sorted_topologically(
@@ -824,6 +836,18 @@ class PersistEventsStore:
                    (chain_id, sequence_number), (auth_chain_id, auth_sequence_number)
                )

+                # Step 2b, add a link to chains reachable from the auth
+                # event.
+                for target_id, target_seq in chain_links.get_links_from(
+                    (auth_chain_id, auth_sequence_number)
+                ):
+                    if target_id == chain_id:
+                        continue
+
+                    chain_links.add_link(
+                        (chain_id, sequence_number), (target_id, target_seq)
+                    )
+
        db_pool.simple_insert_many_txn(
            txn,
            table="event_auth_chain_links",
@@ -2427,6 +2451,31 @@ class _LinkMap:
        current_links[src_seq] = target_seq
        return True

+    def get_links_from(
+        self, src_tuple: Tuple[int, int]
+    ) -> Generator[Tuple[int, int], None, None]:
+        """Gets the chains reachable from the given chain/sequence number.
+
+        Yields:
+            The chain ID and sequence number the link points to.
+        """
+        src_chain, src_seq = src_tuple
+        for target_id, sequence_numbers in self.maps.get(src_chain, {}).items():
+            for link_src_seq, target_seq in sequence_numbers.items():
+                if link_src_seq <= src_seq:
+                    yield target_id, target_seq
+
+    def get_links_between(
+        self, source_chain: int, target_chain: int
+    ) -> Generator[Tuple[int, int], None, None]:
+        """Gets the links between two chains.
+
+        Yields:
+            The source and target sequence numbers.
+        """
+
+        yield from self.maps.get(source_chain, {}).get(target_chain, {}).items()
+
    def get_additions(self) -> Generator[Tuple[int, int, int, int], None, None]:
        """Gets any newly added links.

@@ -2453,24 +2502,9 @@ class _LinkMap:
        if src_chain == target_chain:
            return target_seq <= src_seq

-        # We have to graph traverse the links to check for indirect paths.
-        visited_chains: Dict[int, int] = collections.Counter()
-        search = [(src_chain, src_seq)]
-        while search:
-            chain, seq = search.pop()
-            visited_chains[chain] = max(seq, visited_chains[chain])
-            for tc, links in self.maps.get(chain, {}).items():
-                for ss, ts in links.items():
-                    # Don't revisit chains we've already seen, unless the target
-                    # sequence number is higher than last time.
-                    if ts <= visited_chains.get(tc, 0):
-                        continue
-
-                    if ss <= seq:
-                        if tc == target_chain:
-                            if target_seq <= ts:
-                                return True
-                        else:
-                            search.append((tc, ts))
+        links = self.get_links_between(src_chain, target_chain)
+        for link_start_seq, link_end_seq in links:
+            if link_start_seq <= src_seq and target_seq <= link_end_seq:
+                return True

        return False
@@ -178,13 +178,14 @@ class ReceiptsWorkerStore(SQLBaseStore):
        )

        sql = f"""
-            SELECT event_id, event_stream_ordering
+            SELECT event_id, stream_ordering
            FROM receipts_linearized
+            INNER JOIN events USING (room_id, event_id)
            WHERE {clause}
            AND user_id = ?
            AND room_id = ?
            AND thread_id IS NULL
-            ORDER BY event_stream_ordering DESC
+            ORDER BY stream_ordering DESC
            LIMIT 1
        """

@@ -734,13 +735,10 @@ class ReceiptsWorkerStore(SQLBaseStore):
                thread_clause = "r.thread_id = ?"
                thread_args = (thread_id,)

-            # If the receipt doesn't have a stream ordering it is because we
-            # don't have the associated event, and so must be a remote receipt.
-            # Hence it's safe to just allow new receipts to clobber it.
            sql = f"""
-            SELECT r.event_stream_ordering, r.event_id FROM receipts_linearized AS r
-            WHERE r.room_id = ? AND r.receipt_type = ? AND r.user_id = ?
-            AND r.event_stream_ordering IS NOT NULL AND {thread_clause}
+            SELECT stream_ordering, event_id FROM events
+            INNER JOIN receipts_linearized AS r USING (event_id, room_id)
+            WHERE r.room_id = ? AND r.receipt_type = ? AND r.user_id = ? AND {thread_clause}
            """
            txn.execute(
                sql,
@@ -236,8 +236,7 @@ class RegistrationWorkerStore(CacheInvalidationWorkerStore):
                    consent_server_notice_sent, appservice_id, creation_ts, user_type,
                    deactivated, COALESCE(shadow_banned, FALSE) AS shadow_banned,
                    COALESCE(approved, TRUE) AS approved,
-                    COALESCE(locked, FALSE) AS locked,
-                    suspended
+                    COALESCE(locked, FALSE) AS locked
                FROM users
                WHERE name = ?
                """,
@@ -262,7 +261,6 @@ class RegistrationWorkerStore(CacheInvalidationWorkerStore):
                shadow_banned,
                approved,
                locked,
-                suspended,
            ) = row

            return UserInfo(
@@ -279,7 +277,6 @@ class RegistrationWorkerStore(CacheInvalidationWorkerStore):
                user_type=user_type,
                approved=bool(approved),
                locked=bool(locked),
-                suspended=bool(suspended),
            )

        return await self.db_pool.runInteraction(
@@ -1183,27 +1180,6 @@ class RegistrationWorkerStore(CacheInvalidationWorkerStore):
        # Convert the potential integer into a boolean.
        return bool(res)

-    @cached()
-    async def get_user_suspended_status(self, user_id: str) -> bool:
-        """
-        Determine whether the user's account is suspended.
-            Args:
-                user_id: The user ID of the user in question
-            Returns:
-                True if the user's account is suspended, false if it is not suspended or
-                if the user ID cannot be found.
-        """
-
-        res = await self.db_pool.simple_select_one_onecol(
-            table="users",
-            keyvalues={"name": user_id},
-            retcol="suspended",
-            allow_none=True,
-            desc="get_user_suspended",
-        )
-
-        return bool(res)
-
    async def get_threepid_validation_session(
        self,
        medium: Optional[str],
@@ -2132,13 +2108,6 @@ class RegistrationBackgroundUpdateStore(RegistrationWorkerStore):
            unique=False,
        )

-        self.db_pool.updates.register_background_index_update(
-            update_name="access_tokens_refresh_token_id_idx",
-            index_name="access_tokens_refresh_token_id_idx",
-            table="access_tokens",
-            columns=("refresh_token_id",),
-        )
-
    async def _background_update_set_deactivated_flag(
        self, progress: JsonDict, batch_size: int
    ) -> int:
@@ -2237,35 +2206,6 @@ class RegistrationBackgroundUpdateStore(RegistrationWorkerStore):
        self._invalidate_cache_and_stream(txn, self.get_user_by_id, (user_id,))
        txn.call_after(self.is_guest.invalidate, (user_id,))

-    async def set_user_suspended_status(self, user_id: str, suspended: bool) -> None:
-        """
-        Set whether the user's account is suspended in the `users` table.
-
-        Args:
-            user_id: The user ID of the user in question
-            suspended: True if the user is suspended, false if not
-        """
-        await self.db_pool.runInteraction(
-            "set_user_suspended_status",
-            self.set_user_suspended_status_txn,
-            user_id,
-            suspended,
-        )
-
-    def set_user_suspended_status_txn(
-        self, txn: LoggingTransaction, user_id: str, suspended: bool
-    ) -> None:
-        self.db_pool.simple_update_one_txn(
-            txn=txn,
-            table="users",
-            keyvalues={"name": user_id},
-            updatevalues={"suspended": suspended},
-        )
-        self._invalidate_cache_and_stream(
-            txn, self.get_user_suspended_status, (user_id,)
-        )
-        self._invalidate_cache_and_stream(txn, self.get_user_by_id, (user_id,))
-
    async def set_user_locked_status(self, user_id: str, locked: bool) -> None:
        """Set the `locked` property for the provided user to the provided value.

@@ -2326,6 +2266,13 @@ class RegistrationStore(StatsStore, RegistrationBackgroundUpdateStore):
    ):
        super().__init__(database, db_conn, hs)

+        self.db_pool.updates.register_background_index_update(
+            update_name="access_tokens_refresh_token_id_idx",
+            index_name="access_tokens_refresh_token_id_idx",
+            table="access_tokens",
+            columns=("refresh_token_id",),
+        )
+
        self._ignore_unknown_session_error = (
            hs.config.server.request_token_inhibit_3pid_errors
        )
@@ -1234,28 +1234,6 @@ class RoomMemberWorkerStore(EventsWorkerStore, CacheInvalidationWorkerStore):

        return set(room_ids)

-    async def get_membership_event_ids_for_user(
-        self, user_id: str, room_id: str
-    ) -> Set[str]:
-        """Get all event_ids for the given user and room.
-
-        Args:
-            user_id: The user ID to get the event IDs for.
-            room_id: The room ID to look up events for.
-
-        Returns:
-            Set of event IDs
-        """
-
-        event_ids = await self.db_pool.simple_select_onecol(
-            table="room_memberships",
-            keyvalues={"user_id": user_id, "room_id": room_id},
-            retcol="event_id",
-            desc="get_membership_event_ids_for_user",
-        )
-
-        return set(event_ids)
-
    @cached(max_entries=5000)
    async def _get_membership_from_event_id(
        self, member_event_id: str
@@ -470,8 +470,6 @@ class SearchStore(SearchBackgroundUpdateStore):
        count_args = args
        count_clauses = clauses

-        sqlite_highlights: List[str] = []
-
        if isinstance(self.database_engine, PostgresEngine):
            search_query = search_term
            sql = """
@@ -488,7 +486,7 @@ class SearchStore(SearchBackgroundUpdateStore):
            """
            count_args = [search_query] + count_args
        elif isinstance(self.database_engine, Sqlite3Engine):
-            search_query, sqlite_highlights = _parse_query_for_sqlite(search_term)
+            search_query = _parse_query_for_sqlite(search_term)

            sql = """
            SELECT rank(matchinfo(event_search)) as rank, room_id, event_id
@@ -533,11 +531,9 @@ class SearchStore(SearchBackgroundUpdateStore):

        event_map = {ev.event_id: ev for ev in events}

-        highlights: Collection[str] = []
+        highlights = None
        if isinstance(self.database_engine, PostgresEngine):
            highlights = await self._find_highlights_in_postgres(search_query, events)
-        else:
-            highlights = sqlite_highlights

        count_sql += " GROUP BY room_id"

@@ -601,8 +597,6 @@ class SearchStore(SearchBackgroundUpdateStore):
        count_args = list(args)
        count_clauses = list(clauses)

-        sqlite_highlights: List[str] = []
-
        if pagination_token:
            try:
                origin_server_ts_str, stream_str = pagination_token.split(",")
@@ -653,7 +647,7 @@ class SearchStore(SearchBackgroundUpdateStore):
            CROSS JOIN events USING (event_id)
            WHERE
            """
-            search_query, sqlite_highlights = _parse_query_for_sqlite(search_term)
+            search_query = _parse_query_for_sqlite(search_term)
            args = [search_query] + args

            count_sql = """
@@ -700,11 +694,9 @@ class SearchStore(SearchBackgroundUpdateStore):

        event_map = {ev.event_id: ev for ev in events}

-        highlights: Collection[str] = []
+        highlights = None
        if isinstance(self.database_engine, PostgresEngine):
            highlights = await self._find_highlights_in_postgres(search_query, events)
-        else:
-            highlights = sqlite_highlights

        count_sql += " GROUP BY room_id"

@@ -900,25 +892,19 @@ def _tokenize_query(query: str) -> TokenList:
    return tokens


-def _tokens_to_sqlite_match_query(tokens: TokenList) -> Tuple[str, List[str]]:
+def _tokens_to_sqlite_match_query(tokens: TokenList) -> str:
    """
    Convert the list of tokens to a string suitable for passing to sqlite's MATCH.
    Assume sqlite was compiled with enhanced query syntax.

-    Returns the sqlite-formatted query string and the tokenized search terms
-    that can be used as highlights.
-
    Ref: https://www.sqlite.org/fts3.html#full_text_index_queries
    """
    match_query = []
-    highlights = []
    for token in tokens:
        if isinstance(token, str):
            match_query.append(token)
-            highlights.append(token)
        elif isinstance(token, Phrase):
            match_query.append('"' + " ".join(token.phrase) + '"')
-            highlights.append(" ".join(token.phrase))
        elif token == SearchToken.Not:
            # TODO: SQLite treats NOT as a *binary* operator. Hopefully a search
            # term has already been added before this.
@@ -930,14 +916,11 @@ def _tokens_to_sqlite_match_query(tokens: TokenList) -> Tuple[str, List[str]]:
        else:
            raise ValueError(f"unknown token {token}")

-    return "".join(match_query), highlights
+    return "".join(match_query)


-def _parse_query_for_sqlite(search_term: str) -> Tuple[str, List[str]]:
+def _parse_query_for_sqlite(search_term: str) -> str:
    """Takes a plain unicode string from the user and converts it into a form
    that can be passed to sqllite's matchinfo().
-
-    Returns the converted query string and the tokenized search terms
-    that can be used as highlights.
    """
    return _tokens_to_sqlite_match_query(_tokenize_query(search_term))
@@ -660,7 +660,6 @@ class TransactionWorkerStore(CacheInvalidationWorkerStore):
                    limit=limit,
                    retcols=("room_id", "stream_ordering"),
                    order_direction=order,
-                    keyvalues={"destination": destination},
                ),
            )
            return rooms, count
@@ -19,7 +19,7 @@
 #
 #

-SCHEMA_VERSION = 85  # remember to update the list below when updating
+SCHEMA_VERSION = 84  # remember to update the list below when updating
 """Represents the expectations made by the codebase about the database schema

 This should be incremented whenever the codebase changes its requirements on the
@@ -132,19 +132,12 @@ Changes in SCHEMA_VERSION = 82

 Changes in SCHEMA_VERSION = 83
    - The event_txn_id is no longer used.
-
-Changes in SCHEMA_VERSION = 84
-    - No longer assumes that `event_auth_chain_links` holds transitive links, and
-      so read operations must do graph traversal.
-
-Changes in SCHEMA_VERSION = 85
-    - Add a column `suspended` to the `users` table
 """


 SCHEMA_COMPAT_VERSION = (
-    # Transitive links are no longer written to `event_auth_chain_links`
-    84
+    # The event_txn_id table and tables from MSC2716 no longer exist.
+    83
 )
 """Limit on how far the synapse codebase can be rolled back without breaking db compat

@@ -1,15 +0,0 @@
--
-- This file is licensed under the Affero General Public License (AGPL) version 3.
--
-- Copyright (C) 2023 New Vector, Ltd
--
-- This program is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as
-- published by the Free Software Foundation, either version 3 of the
-- License, or (at your option) any later version.
--
-- See the GNU Affero General Public License for more details:
-- <https://www.gnu.org/licenses/agpl-3.0.html>.
-
-INSERT INTO background_updates (ordering, update_name, progress_json) VALUES
-    (8404, 'access_tokens_refresh_token_id_idx', '{}');
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Andrew Morgan	6452c704ce	changelog	2024-05-17 10:45:20 +01:00
Andrew Morgan	0abc0cbe52	Remove v2 variant from endpoints	2024-05-15 11:59:15 +01:00
Dominic Schubert	336a85d1fa	Knocking Endpoints added (federated) Knocking Endpoints was missing for federaded Worker	2024-04-07 01:08:57 +02:00
				`@@ -0,0 +1 @@`
				Fix various long-standing bugs which could cause incorrect state to be returned from `/sync` in certain situations.
				`@@ -0,0 +1 @@`
				Add support for moving `/pushrules` off of main process.
				`@@ -0,0 +1 @@`
				`Refactor auth chain fetching to reduce duplication.`
				`@@ -0,0 +1 @@`
				Improve database performance by adding a missing index to `access_tokens.refresh_token_id`.
				`@@ -0,0 +1 @@`
				`Improve database performance by reducing number of receipts fetched when sending push notifications.`
				`@@ -0,0 +1 @@`
				Document [`/v1/make_knock`](https://spec.matrix.org/v1.10/server-server-api/#get_matrixfederationv1make_knockroomiduserid) and [`/v1/send_knock/](https://spec.matrix.org/v1.10/server-server-api/#put_matrixfederationv1send_knockroomideventid) federation endpoints as worker-compatible.