1
0

Compare commits

..

29 Commits

Author SHA1 Message Date
kaslo acf62f71c5 sync with upstream 2026-03-04 16:48:32 -06:00
kaslo 198ec9e1f3 fix: remove timestamp rounding in TerseJsonFormatter
The round(record.created, 2) call limits timestamps generated by
TerseJsonFormatter to 10ms precision. This can cause log ordering issues
in log aggregators like Loki when multiple events occur within the same
10ms window.

The rounding was introduced in the original structured logging PR and,
to my knowledge, has no technical benefit.
2026-03-04 16:46:13 -06:00
Eric Eastwood 46c6e0ae1e Unify Complement developer docs (#19518)
Instead of having info spread across a few places, consolidate and link
to one spot.
2026-03-03 13:18:49 -06:00
dependabot[bot] c2c05879bb Bump docker/build-push-action from 6.18.0 to 6.19.2 in the minor-and-patches group (#19514)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-03 17:30:30 +00:00
Quentin Gliech fd61b8eeb0 Merge branch 'release-v1.149' into develop 2026-03-03 17:24:12 +01:00
Eric Eastwood 51048b8e36 Update docs to clarify outbound_federation_restricted_to can also be used with the SBG (#19517)
[Secure Border Gateway (SBG)](https://element.io/en/server-suite/secure-border-gateways)

Spawning from [internal
discussion](https://matrix.to/#/!mNoPShRlwEeyHAEJOe:element.io/$6eGip85OUKOmyK1VzqrFMc7eF7dON7Vs76O40kVbRRY?via=banzan.uk&via=element.io&via=jki.re)
around integrating [Synapse Pro for small
hosts](https://docs.element.io/latest/element-server-suite-pro/synapse-pro-for-small-hosts/overview)
in the [Element Server Suite (ESS)](https://element.io/en/server-suite)
stack and wanting it be compatible with the SBG.

We know that the SBG works with monolith Synapse because that's what we
have configured with the [Complement tests in the SBG
repo](https://github.com/element-hq/sbg/blob/b76b05b53e40bf6890e51dd1b83cec3460274eb2/complement/configure_synapse_for_sbg.sh#L8-L10).
2026-03-03 10:04:37 -06:00
Quentin Gliech 639922e835 1.149.0rc1 2026-03-03 15:38:17 +01:00
Eric Eastwood 160d9788c0 Simplify Rust HTTP client response streaming and limiting (#19510)
*As suggested by @sandhose in
https://github.com/element-hq/synapse/pull/19498#discussion_r2865607737,*

Simplify Rust HTTP client response streaming and limiting


### Dev notes

Synapse's Rust HTTP client was introduced in
https://github.com/element-hq/synapse/pull/18357



### Pull Request Checklist

<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->

* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
  - Use markdown where necessary, mostly for `code blocks`.
  - End with either a period (.) or an exclamation mark (!).
  - Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
2026-03-03 15:24:25 +01:00
Eric Eastwood c3af44339c Fix /sync missing membership in state_after (re-introduce) (#19460)
*This PR was originally only to enable
[MSC4222](https://github.com/matrix-org/matrix-spec-proposals/pull/4222)
Complement tests (`/sync` `state_after`) but after merging the [fix
PR](https://github.com/element-hq/synapse/pull/19463), we discovered
that while the tests pass locally, [fail in
CI](https://github.com/element-hq/synapse/pull/19460#discussion_r2818080879).
To unblock the RC, we decided to revert the fix PR (see
https://github.com/element-hq/synapse/pull/19474#discussion_r2818061001
for more info). To better ensure tests actually pass in CI, we're
re-introducing the fix here in the same PR that we enable the tests in.*

---

Fix `/sync` missing membership in `state_after`.

This applies to any scenario where the first membership has a different
`sender` compared to the `state_key` and then the second membership has
the same `sender`/`state_key`. Like someone inviting another person and
then them joining. Or someone being kicked and then they leave.

This bug has been present since the MSC4222 implementation was
introduced into the codebase
(https://github.com/element-hq/synapse/pull/17888).

---

Fix https://github.com/element-hq/synapse/issues/19455
Fix https://github.com/element-hq/customer-success/issues/656

I have a feeling, this might also fix these issues (will close and see
how people report back):

Fix https://github.com/element-hq/synapse/issues/18182
Fix https://github.com/element-hq/synapse/issues/19478

 ### Testing strategy

Complement tests: https://github.com/matrix-org/complement/pull/842

We will need https://github.com/element-hq/synapse/pull/19460 to merge
in order to enable the Complement tests in Synapse but this PR should be
merged first so they pass in the first place. I've tested locally that
the Complement tests pass with this fix.




### Dev notes


[MSC4222](https://github.com/matrix-org/matrix-spec-proposals/pull/4222)
has already been merged into the spec and is already part of Matrix
v1.16 but we haven't [stabilized support in Synapse
yet](https://github.com/element-hq/synapse/issues/19414).

---

In the same ballpark:

 - https://github.com/element-hq/synapse/issues/19455
 - https://github.com/element-hq/synapse/issues/17050
 - https://github.com/element-hq/synapse/issues/17430
 - https://github.com/element-hq/synapse/issues/16940
 - https://github.com/element-hq/synapse/issues/18182
 - https://github.com/element-hq/synapse/issues/18793
 - https://github.com/element-hq/synapse/issues/19478

---

Docker builds preferring remote image over the local image we just
built,
https://github.com/element-hq/synapse/pull/19460#discussion_r2818080879

`containerd` image store (storage driver, driver type)

-> https://github.com/element-hq/synapse/pull/19475


### Todo

- [x] Wait for https://github.com/element-hq/synapse/pull/19463 to merge
so the Complement tests all pass
- [x] Wait for https://github.com/element-hq/synapse/pull/19475 to merge

### Pull Request Checklist

<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->

* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
  - Use markdown where necessary, mostly for `code blocks`.
  - End with either a period (.) or an exclamation mark (!).
  - Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))

---------

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>
2026-03-03 15:13:59 +01:00
Quentin Gliech 094a48efb5 Bump all locked dependencies to their latest versions. (#19519)
This is a manual lock bump, as it looks like Dependabot is currently
timing out updating dependencies. This should hopefully unlock it, as it
will have fewer dependencies to update.

Two outstanding exceptions:

- pympler upgrade adds a pywin32 deps, which is missing sdist (so CI is
complaining)
- pysaml2 for some unknown reason pinned the MAX version of pyopenssl,
which duplicates pyopenssl and cryptography, which obviously breaks
stuff
2026-03-03 14:29:59 +01:00
dependabot[bot] 2deeef4118 Bump futures from 0.3.31 to 0.3.32 in the patches group (#19513)
Bumps the patches group with 1 update:
[futures](https://github.com/rust-lang/futures-rs).

Updates `futures` from 0.3.31 to 0.3.32
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/rust-lang/futures-rs/releases">futures's
releases</a>.</em></p>
<blockquote>
<h2>0.3.32</h2>
<ul>
<li>Bump MSRV of utility crates to 1.71. (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2989">#2989</a>)</li>
<li>Soft-deprecate <code>ready!</code> macro in favor of
<code>std::task::ready!</code> added in Rust 1.64 (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2925">#2925</a>)</li>
<li>Soft-deprecate <code>pin_mut!</code> macro in favor of
<code>std::pin::pin!</code> added in Rust 1.68 (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2929">#2929</a>)</li>
<li>Add <code>FuturesOrdered::clear</code> (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2927">#2927</a>)</li>
<li>Add <code>mpsc::*Receiver::recv</code> (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2947">#2947</a>)</li>
<li>Add <code>mpsc::*Receiver::try_recv</code> and deprecate
<code>mpsc::*Receiver::::try_next</code> (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2944">#2944</a>)</li>
<li>Implement <code>FusedStream</code> for <code>sink::With</code> (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2948">#2948</a>)</li>
<li>Add <code>no_std</code> support for <code>shared</code> (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2868">#2868</a>)</li>
<li>Make <code>Mutex::new()</code> const (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2956">#2956</a>)</li>
<li>Add <code>#[clippy::has_significant_drop]</code> to guards (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2967">#2967</a>)</li>
<li>Remove dependency to <code>pin-utils</code> (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2929">#2929</a>)</li>
<li>Remove dependency on <code>num_cpus</code> (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2946">#2946</a>)</li>
<li>Performance improvements (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2983">#2983</a>)</li>
<li>Documentation improvements (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2925">#2925</a>,
<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2926">#2926</a>,
<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2940">#2940</a>,
<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2971">#2971</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md">futures's
changelog</a>.</em></p>
<blockquote>
<h1>0.3.32 - 2026-02-15</h1>
<ul>
<li>Bump MSRV of utility crates to 1.71. (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2989">#2989</a>)</li>
<li>Soft-deprecate <code>ready!</code> macro in favor of
<code>std::task::ready!</code> added in Rust 1.64 (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2925">#2925</a>)</li>
<li>Soft-deprecate <code>pin_mut!</code> macro in favor of
<code>std::pin::pin!</code> added in Rust 1.68 (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2929">#2929</a>)</li>
<li>Add <code>FuturesOrdered::clear</code> (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2927">#2927</a>)</li>
<li>Add <code>mpsc::*Receiver::recv</code> (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2947">#2947</a>)</li>
<li>Add <code>mpsc::*Receiver::try_recv</code> and deprecate
<code>mpsc::*Receiver::::try_next</code> (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2944">#2944</a>)</li>
<li>Implement <code>FusedStream</code> for <code>sink::With</code> (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2948">#2948</a>)</li>
<li>Add <code>no_std</code> support for <code>shared</code> (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2868">#2868</a>)</li>
<li>Make <code>Mutex::new()</code> const (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2956">#2956</a>)</li>
<li>Add <code>#[clippy::has_significant_drop]</code> to guards (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2967">#2967</a>)</li>
<li>Remove dependency to <code>pin-utils</code> (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2929">#2929</a>)</li>
<li>Remove dependency on <code>num_cpus</code> (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2946">#2946</a>)</li>
<li>Performance improvements (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2983">#2983</a>)</li>
<li>Documentation improvements (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2925">#2925</a>,
<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2926">#2926</a>,
<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2940">#2940</a>,
<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2971">#2971</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/rust-lang/futures-rs/commit/d9bba94c239daa1175a5bb2958f37a5c72db3f6a"><code>d9bba94</code></a>
Release 0.3.32</li>
<li><a
href="https://github.com/rust-lang/futures-rs/commit/151e0b90dec62103df5239f0612f42467994f406"><code>151e0b9</code></a>
Add comments on rust-version field in Cargo.toml</li>
<li><a
href="https://github.com/rust-lang/futures-rs/commit/4aaf00c35176d7180557559f54b0c151e2e608aa"><code>4aaf00c</code></a>
Bump MSRV of utility crates to 1.71</li>
<li><a
href="https://github.com/rust-lang/futures-rs/commit/a4cce12c55942c6e1f2a507061fc6ca94c5b8862"><code>a4cce12</code></a>
perf: improve AtomicWaker::wake performance (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2983">#2983</a>)</li>
<li><a
href="https://github.com/rust-lang/futures-rs/commit/ba9d102ca6e4a941a5068a1a8dcf0ff3a6c9085a"><code>ba9d102</code></a>
Add <code>#[clippy::has_significant_drop]</code> to guards (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2967">#2967</a>)</li>
<li><a
href="https://github.com/rust-lang/futures-rs/commit/20396a83eff35414d17320dc35858243e54f0bc8"><code>20396a8</code></a>
Fix rustdoc::broken_intra_doc_links warning</li>
<li><a
href="https://github.com/rust-lang/futures-rs/commit/815f6eb4e40ca9ff81d7d9a25a863d3c1ffdb79e"><code>815f6eb</code></a>
Fix documentation of <code>BiLock::lock</code> (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2971">#2971</a>)</li>
<li><a
href="https://github.com/rust-lang/futures-rs/commit/0f0db0421d4edc9fc56c3643f7e7f3bd23058023"><code>0f0db04</code></a>
futures-util: make <code>Mutex::new()</code> const (<a
href="https://redirect.github.com/rust-lang/futures-rs/issues/2956">#2956</a>)</li>
<li><a
href="https://github.com/rust-lang/futures-rs/commit/5d6fc5e4083f5da376ccc7a50403d842e553e286"><code>5d6fc5e</code></a>
ci: Test big-endian target (s390x Linux)</li>
<li><a
href="https://github.com/rust-lang/futures-rs/commit/9f739fe40b9c3e80b8f40054a739a220428a4675"><code>9f739fe</code></a>
Ignore dead_code lint on Fn1 trait</li>
<li>Additional commits viewable in <a
href="https://github.com/rust-lang/futures-rs/compare/0.3.31...0.3.32">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=futures&package-manager=cargo&previous-version=0.3.31&new-version=0.3.32)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-03 11:24:17 +01:00
Olivier 'reivilibre 825f3087bf Replace deprecated collection import locations with current locations. (#19515)
Use non-deprecated imports for collections

Other than being deprecated, these legacy imports also don't seem to be
compatible with [Ty](https://github.com/astral-sh/ty)

---------

Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
2026-03-02 18:15:33 +00:00
Erik Johnston 0d3e42f21f Yield to reactor in large loops (#19507)
When a worker gets very busy some of these loops can get large and end
up taking hundreds of ms to complete. To help keep the reactor tick
times reasonable we add a periodic yield into these loops.

These were found by doing a `py-spy` and speedscope.net (in time order)
to see where we were spending blocks of time
2026-03-02 09:36:27 +00:00
Eric Eastwood 979566ed8f Pre-allocate the buffer based on the expected Content-Length with the Rust HTTP client (#19498)
Spawning from
[looking](https://matrix.to/#/!cnVVNLKqgUzNTOFQkz:matrix.org/$XOVFm5mjCzzmhUaGc202zGdSq8eWgjr00MJqNSfzHiA?via=element.io&via=matrix.org&via=one.ems.host)
at some traces and seeing the Synapse Rust HTTP client taking way longer
than what the Synapse Pro Event Cache claims it was able to respond in
(added some [better
tracing](https://github.com/element-hq/synapse-pro-modules/pull/38) for
that). I don't think this specific change will have a meaningful impact
but just something I saw (pre-optimization).
2026-02-27 16:25:26 -06:00
Richard van der Hoff b9ea2285b3 Add stable support for MSC4380 invite blocking. (#19431)
MSC4380 has now completed FCP, so we can add stable support for it.

Co-authored-by: Quentin Gliech <quenting@element.io>
2026-02-27 14:47:07 +00:00
dependabot[bot] 9de28df7a2 Bump docker/login-action from 3.6.0 to 3.7.0 in the minor-and-patches group (#19493)
Bumps the minor-and-patches group with 1 update:
[docker/login-action](https://github.com/docker/login-action).

Updates `docker/login-action` from 3.6.0 to 3.7.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/docker/login-action/releases">docker/login-action's
releases</a>.</em></p>
<blockquote>
<h2>v3.7.0</h2>
<ul>
<li>Add <code>scope</code> input to set scopes for the authentication
token by <a
href="https://github.com/crazy-max"><code>@​crazy-max</code></a> in <a
href="https://redirect.github.com/docker/login-action/pull/912">docker/login-action#912</a></li>
<li>Add support for AWS European Sovereign Cloud ECR by <a
href="https://github.com/dphi"><code>@​dphi</code></a> in <a
href="https://redirect.github.com/docker/login-action/pull/914">docker/login-action#914</a></li>
<li>Ensure passwords are redacted with <code>registry-auth</code> input
by <a href="https://github.com/crazy-max"><code>@​crazy-max</code></a>
in <a
href="https://redirect.github.com/docker/login-action/pull/911">docker/login-action#911</a></li>
<li>build(deps): bump lodash from 4.17.21 to 4.17.23 in <a
href="https://redirect.github.com/docker/login-action/pull/915">docker/login-action#915</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/docker/login-action/compare/v3.6.0...v3.7.0">https://github.com/docker/login-action/compare/v3.6.0...v3.7.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/docker/login-action/commit/c94ce9fb468520275223c153574b00df6fe4bcc9"><code>c94ce9f</code></a>
Merge pull request <a
href="https://redirect.github.com/docker/login-action/issues/915">#915</a>
from docker/dependabot/npm_and_yarn/lodash-4.17.23</li>
<li><a
href="https://github.com/docker/login-action/commit/8339c958ce8511f38d0c474c1886a87c802bf1ef"><code>8339c95</code></a>
Merge pull request <a
href="https://redirect.github.com/docker/login-action/issues/912">#912</a>
from docker/scope</li>
<li><a
href="https://github.com/docker/login-action/commit/c83e9320c8beb50b77dd007c46d5c8161f0cac4a"><code>c83e932</code></a>
build(deps): bump lodash from 4.17.21 to 4.17.23</li>
<li><a
href="https://github.com/docker/login-action/commit/b268aa57e39ff0a5386d2fd1eded4e2e1d60d705"><code>b268aa5</code></a>
chore: update generated content</li>
<li><a
href="https://github.com/docker/login-action/commit/a60322927812ddc99316dd6252b4fba6d8f09ac1"><code>a603229</code></a>
documentation for scope input</li>
<li><a
href="https://github.com/docker/login-action/commit/7567f92a74b2639be1bd8bc932a112a0d81283da"><code>7567f92</code></a>
Add scope input to set scopes for the authentication token</li>
<li><a
href="https://github.com/docker/login-action/commit/0567fa5ae8c9a197cb207537dc5cbb43ca3d803f"><code>0567fa5</code></a>
Merge pull request <a
href="https://redirect.github.com/docker/login-action/issues/914">#914</a>
from dphi/add-support-for-amazonaws.eu</li>
<li><a
href="https://github.com/docker/login-action/commit/f6ef57754547a85003a0e18f789be661346d4a6e"><code>f6ef577</code></a>
feat: add support for AWS European Sovereign Cloud ECR registries</li>
<li><a
href="https://github.com/docker/login-action/commit/916386b00027d425839f8da46d302dab33f5875b"><code>916386b</code></a>
Merge pull request <a
href="https://redirect.github.com/docker/login-action/issues/911">#911</a>
from crazy-max/ensure-redact</li>
<li><a
href="https://github.com/docker/login-action/commit/5b3f94a294ea5478af3af437baa6ad0d3dcd04fd"><code>5b3f94a</code></a>
chore: update generated content</li>
<li>Additional commits viewable in <a
href="https://github.com/docker/login-action/compare/v3.6.0...c94ce9fb468520275223c153574b00df6fe4bcc9">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=docker/login-action&package-manager=github_actions&previous-version=3.6.0&new-version=3.7.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-27 15:32:19 +01:00
Erik Johnston 2c73e8daef Allow long lived syncs to be cancelled if client has gone away (#19499) 2026-02-26 21:41:06 +00:00
Hugh Nimmo-Smith f78d011df1 Experimental implementation of unstable MSC4388 for Sign in with QR (#19127)
Co-authored-by: Olivier 'reivilibre' <oliverw@element.io>
2026-02-25 17:41:51 +00:00
Eric Eastwood ac3a115511 Log if we ever gc.freeze() (#19440)
Spawning from
https://github.com/element-hq/synapse-small-hosts/issues/348 where some
test appears to be flaky because some homeserver objects are frozen in
the garbage collector.

We set
[`freeze=False`](https://github.com/element-hq/synapse-small-hosts/blob/a9a6869aa9a67176bdddc3b8ae2d0de0996d8cf4/multi_synapse/app/shard.py#L319-L321)
in the [Synapse Pro for small
hosts](https://docs.element.io/latest/element-server-suite-pro/synapse-pro-for-small-hosts/overview/)
code but I just want to use this log to make extra sure this isn't being
run somehow. The follow-up here would be to see what else would cause
something to be frozen in the garbage collector.
2026-02-25 09:47:13 -06:00
Brad Murray bc15ed3c62 DeviceHandler: Add a log line when we delete a device (#19496)
Deleting devices should be fairly
rare, and if someone gets logged out it's helpful to grep logs for a
user id or device id and see where it died.
2026-02-24 14:18:52 -06:00
Quentin Gliech 3d30735e79 Merge branch 'master' into develop 2026-02-24 15:29:24 +01:00
Olivier 'reivilibre 16245f0550 Fix the 'Login as a user' Admin API not checking if the user exists before issuing an access token. (#18518)
Fixes: #18503

---------

Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
Co-authored-by: Quentin Gliech <quenting@element.io>
2026-02-20 15:52:29 +00:00
Olivier 'reivilibre 4500652459 Rename the test_disconnect test helper so that pytest doesn't see it as a test. (#19486)
This fixes one of the 2 blockers to using pytest instead of Trial (which
is not formally-motivated, but sometimes seems like an interesting idea
because
pytest has seen a lot of developer experience features that Trial
hasn't. It would also removes one more coupling to the Twisted
framework.)

---

The `test_` prefix to this test helper makes it appear as a test to
pytest.

We *can* set a `__test__ = False` attribute on the test, but it felt
cleaner to just rename it (as I also thought it would be a test from
that name!).

This was previously reported as:
https://github.com/element-hq/synapse/issues/18665

---------

Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
2026-02-20 15:37:34 +00:00
dependabot[bot] 9b738d2ec5 Bump the patches group with 2 updates (#19488)
Bumps the patches group with 2 updates:
[anyhow](https://github.com/dtolnay/anyhow) and
[regex](https://github.com/rust-lang/regex).

Updates `anyhow` from 1.0.100 to 1.0.101
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/dtolnay/anyhow/releases">anyhow's
releases</a>.</em></p>
<blockquote>
<h2>1.0.101</h2>
<ul>
<li>Add #[inline] to anyhow::Ok helper (<a
href="https://redirect.github.com/dtolnay/anyhow/issues/437">#437</a>,
thanks <a
href="https://github.com/Ibitier"><code>@​Ibitier</code></a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/dtolnay/anyhow/commit/80bfe291b16071c70f141e90e67e7032d966826b"><code>80bfe29</code></a>
Release 1.0.101</li>
<li><a
href="https://github.com/dtolnay/anyhow/commit/dff8c432f95095cac19aa446da5047880b8ebdf3"><code>dff8c43</code></a>
Merge pull request <a
href="https://redirect.github.com/dtolnay/anyhow/issues/437">#437</a>
from Ibitier/inline-ok-helper</li>
<li><a
href="https://github.com/dtolnay/anyhow/commit/85d9ea9a1c7d7490578865e16ff64787efc7d01d"><code>85d9ea9</code></a>
Add #[inline] to anyhow::Ok helper</li>
<li><a
href="https://github.com/dtolnay/anyhow/commit/54036cc289b754775b884485f486e000bcda2875"><code>54036cc</code></a>
Update ui test suite to nightly-2026-01-21</li>
<li><a
href="https://github.com/dtolnay/anyhow/commit/cce0579d85fd1f6352a5955a9c134fc8655c853d"><code>cce0579</code></a>
Update actions/upload-artifact@v5 -&gt; v6</li>
<li><a
href="https://github.com/dtolnay/anyhow/commit/f2c598ca0e8ffd7ffcbcf93b8a6ad4df57c719fd"><code>f2c598c</code></a>
Update actions/upload-artifact@v4 -&gt; v5</li>
<li><a
href="https://github.com/dtolnay/anyhow/commit/2c0bda4ce944d943e7141f0316b0ea996602238e"><code>2c0bda4</code></a>
Update to 2021 edition</li>
<li><a
href="https://github.com/dtolnay/anyhow/commit/0d822681293d71c72440c9cdd635b4f15da064c4"><code>0d82268</code></a>
Remove rustc version requirement from readme</li>
<li><a
href="https://github.com/dtolnay/anyhow/commit/67df01216d29bc3bede925ab4483353b66c159f2"><code>67df012</code></a>
Merge pull request <a
href="https://redirect.github.com/dtolnay/anyhow/issues/436">#436</a>
from dtolnay/up</li>
<li><a
href="https://github.com/dtolnay/anyhow/commit/c8984880a87ae4fd4b04c956cfdc9af5f69eab55"><code>c898488</code></a>
Raise required compiler to Rust 1.68</li>
<li>Additional commits viewable in <a
href="https://github.com/dtolnay/anyhow/compare/1.0.100...1.0.101">compare
view</a></li>
</ul>
</details>
<br />

Updates `regex` from 1.12.2 to 1.12.3
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/rust-lang/regex/blob/master/CHANGELOG.md">regex's
changelog</a>.</em></p>
<blockquote>
<h1>1.12.3 (2025-02-03)</h1>
<p>This release excludes some unnecessary things from the archive
published to
crates.io. Specifically, fuzzing data and various shell scripts are now
excluded. If you run into problems, please file an issue.</p>
<p>Improvements:</p>
<ul>
<li><a
href="https://redirect.github.com/rust-lang/regex/pull/1319">#1319</a>:
Switch from a Cargo <code>exclude</code> list to an <code>include</code>
list, and exclude some
unnecessary stuff.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/rust-lang/regex/commit/b028e4f40eac8959d05e82abf8404906b1c565c0"><code>b028e4f</code></a>
1.12.3</li>
<li><a
href="https://github.com/rust-lang/regex/commit/5e195de266e203441b2c8001d6ebefab1161a59e"><code>5e195de</code></a>
regex-automata-0.4.14</li>
<li><a
href="https://github.com/rust-lang/regex/commit/a3433f691863d80300dfd6a52e332cb5a568e895"><code>a3433f6</code></a>
regex-syntax-0.8.9</li>
<li><a
href="https://github.com/rust-lang/regex/commit/0c07fae444adf0802d84455e689f1143d2dd7790"><code>0c07fae</code></a>
regex-lite-0.1.9</li>
<li><a
href="https://github.com/rust-lang/regex/commit/6a810068f030c023a12c93ccae49bc5fd907c4f6"><code>6a81006</code></a>
cargo: exclude development scripts and fuzzing data</li>
<li><a
href="https://github.com/rust-lang/regex/commit/4733e28ba4f281f643ce93e4089eccbb9a9d5a5a"><code>4733e28</code></a>
automata: fix <code>onepass::DFA::try_search_slots</code> panic when too
many slots are ...</li>
<li>See full diff in <a
href="https://github.com/rust-lang/regex/compare/1.12.2...1.12.3">compare
view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-20 15:14:46 +00:00
dependabot[bot] 0ac772f082 Bump pillow from 12.0.0 to 12.1.1 (#19454)
Bumps [pillow](https://github.com/python-pillow/Pillow) from 12.0.0 to
12.1.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/python-pillow/Pillow/releases">pillow's
releases</a>.</em></p>
<blockquote>
<h2>12.1.1</h2>
<p><a
href="https://pillow.readthedocs.io/en/stable/releasenotes/12.1.1.html">https://pillow.readthedocs.io/en/stable/releasenotes/12.1.1.html</a></p>
<h2>Dependencies</h2>
<ul>
<li>Patch libavif for svt-av1 4.0 compatibility <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9413">#9413</a>
[<a href="https://github.com/hugovk"><code>@​hugovk</code></a>]</li>
</ul>
<h2>Other changes</h2>
<ul>
<li>Fix OOB Write with invalid tile extents <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9427">#9427</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
</ul>
<h2>12.1.0</h2>
<p><a
href="https://pillow.readthedocs.io/en/stable/releasenotes/12.1.0.html">https://pillow.readthedocs.io/en/stable/releasenotes/12.1.0.html</a></p>
<h2>Deprecations</h2>
<ul>
<li>Deprecate getdata(), in favour of new get_flattened_data() <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9292">#9292</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
</ul>
<h2>Documentation</h2>
<ul>
<li>Specify APNG duration type when opening <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9368">#9368</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
<li>Added release notes for <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9350">#9350</a>
<a
href="https://redirect.github.com/python-pillow/Pillow/issues/9366">#9366</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
<li>Update ImageMorph documentation <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9349">#9349</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
<li>Docs: update major bump cadence <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9334">#9334</a>
[<a href="https://github.com/hugovk"><code>@​hugovk</code></a>]</li>
<li>Add release notes for <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9070">#9070</a>
<a
href="https://redirect.github.com/python-pillow/Pillow/issues/9320">#9320</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
<li>Updated Ubuntu version <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9306">#9306</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
<li>Update macOS tested Pillow versions <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9265">#9265</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
</ul>
<h2>Dependencies</h2>
<ul>
<li>Update harfbuzz to 12.3.0 <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9355">#9355</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
<li>Update xz to 5.8.2 <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9343">#9343</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
<li>Updated libjpeg-turbo to 3.1.3 <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9333">#9333</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
<li>Updated zlib-ng to 2.3.2 <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9324">#9324</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
<li>Updated libpng to 1.6.53 <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9325">#9325</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
<li>Update actions/checkout action to v6 <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9323">#9323</a>
[@<a href="https://github.com/apps/renovate">renovate[bot]</a>]</li>
<li>Update dependency mypy to v1.19.0 <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9322">#9322</a>
[@<a href="https://github.com/apps/renovate">renovate[bot]</a>]</li>
<li>Updated libpng to 1.6.51 <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9305">#9305</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
<li>Updated brotli to 1.2.0 <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9284">#9284</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
<li>Update libimagequant to 4.4.1 <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9301">#9301</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
<li>Update zlib-ng to 2.3.1, except on manylinux2014 aarch64 <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9312">#9312</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
<li>Updated harfbuzz to 12.2.0 <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9289">#9289</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
<li>Update github-actions <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9277">#9277</a>
[@<a href="https://github.com/apps/renovate">renovate[bot]</a>]</li>
</ul>
<h2>Testing</h2>
<ul>
<li>Replace pre-commit with prek <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9360">#9360</a>
[<a href="https://github.com/hugovk"><code>@​hugovk</code></a>]</li>
<li>Test PyQt6 on Python 3.14 on Windows <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9353">#9353</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
<li>Test 32-bit Windows on Windows Server 2022 <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9345">#9345</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
<li>Correct variable type <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9335">#9335</a>
[<a
href="https://github.com/radarhere"><code>@​radarhere</code></a>]</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/python-pillow/Pillow/commit/5158d98c807e719c5938aa3886913ef0ea6814e9"><code>5158d98</code></a>
12.1.1 version bump</li>
<li><a
href="https://github.com/python-pillow/Pillow/commit/9000313cc5d4a31bdcdd6d7f0781101abab553aa"><code>9000313</code></a>
Fix OOB Write with invalid tile extents (<a
href="https://redirect.github.com/python-pillow/Pillow/issues/9427">#9427</a>)</li>
<li><a
href="https://github.com/python-pillow/Pillow/commit/cd0111849fb32c40860e3ee3d57b9b1cee4260cf"><code>cd01118</code></a>
Patch libavif for svt-av1 4.0 compatibility</li>
<li><a
href="https://github.com/python-pillow/Pillow/commit/46f45f674d47b5d8bc54230dda8fe9e214598b87"><code>46f45f6</code></a>
12.1.0 version bump</li>
<li><a
href="https://github.com/python-pillow/Pillow/commit/c9ac097edb5594f63c40acd9afe6802547200379"><code>c9ac097</code></a>
Simplify band splitting (<a
href="https://redirect.github.com/python-pillow/Pillow/issues/9291">#9291</a>)</li>
<li><a
href="https://github.com/python-pillow/Pillow/commit/3baedf264804d199bc19458d11bcff02ce7598eb"><code>3baedf2</code></a>
Deprecate getdata(), in favour of new get_flattened_data() (<a
href="https://redirect.github.com/python-pillow/Pillow/issues/9292">#9292</a>)</li>
<li><a
href="https://github.com/python-pillow/Pillow/commit/b51a0366852c1d519d108dfec8fc2d738cd8080f"><code>b51a036</code></a>
Specify APNG duration type when opening (<a
href="https://redirect.github.com/python-pillow/Pillow/issues/9368">#9368</a>)</li>
<li><a
href="https://github.com/python-pillow/Pillow/commit/8d08e31533065b623399a54bc92b39a756599ad4"><code>8d08e31</code></a>
Add release notes for <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9348">#9348</a>
(<a
href="https://redirect.github.com/python-pillow/Pillow/issues/9369">#9369</a>)</li>
<li><a
href="https://github.com/python-pillow/Pillow/commit/432707ea810ae619e2a9e4a9737c169cacaa8eda"><code>432707e</code></a>
Added release notes for <a
href="https://redirect.github.com/python-pillow/Pillow/issues/9348">#9348</a></li>
<li><a
href="https://github.com/python-pillow/Pillow/commit/2d589107fb3a4aba8389932a65ff771bf9b4deb1"><code>2d58910</code></a>
Specify APNG duration type when opening</li>
<li>Additional commits viewable in <a
href="https://github.com/python-pillow/Pillow/compare/12.0.0...12.1.1">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pillow&package-manager=pip&previous-version=12.0.0&new-version=12.1.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/element-hq/synapse/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Quentin Gliech <quenting@element.io>
2026-02-20 15:42:16 +01:00
Eric Eastwood 04206aebdf Log docker system info in CI (#19480)
Follow-up to
https://github.com/element-hq/synapse/pull/19460#discussion_r2819139638
and https://github.com/element-hq/synapse/pull/19475
2026-02-19 09:57:25 -06:00
Eric Eastwood b2778dae70 Fix Complement CI not running against the code from our PRs (remote images being chosen over local) (#19475)
Fix remote images being chosen over the local ones we just built with
Complement in CI (any Docker environment using the `containerd` image
store). This problem means that Complement jobs in CI don't actually
test against the code from the PR (since 2026-02-10).

This PR approaches the problem the same way that @AndrewFerr proposed in
https://github.com/element-hq/synapse/pull/18210. This is better than
the alternative listed below as we can just make our code compatible
with whatever image store is being used.
### Problem

Spawning from
https://github.com/element-hq/synapse/pull/19460#discussion_r2818760635
where we found that our Complement jobs in CI don't actually test
against the code from the PR at the moment.

This is caused by a change in Docker Engine 29.0.0:

> `containerd` image store is now the default for **fresh installs**.
This doesn't apply to daemons configured with `userns-remap` (see
[moby#47377](https://github.com/moby/moby/issues/47377)).
>
> *-- 29.0.0 (2025-11-10),
https://docs.docker.com/engine/release-notes/29/#2900*

And our `ubuntu-latest` GitHub runner (`Current runner version:
'2.331.0'`)
[points](https://github.com/actions/runner-images/blob/ubuntu24/20260209.23/images/ubuntu/Ubuntu2404-Readme.md)
to using Docker client/server `29.1.5` 🎯

This Docker version bump happened on
https://github.com/actions/runner-images/commit/416418df15a48321bdf51c3978c07a84a1873c75
(2026-02-10) (`28.0.4` -> `29.1.5`). Specific PR:
https://github.com/actions/runner-images/pull/13633

---

I found this because I reviewed and remembered
https://github.com/element-hq/synapse/pull/18210 was a thing that
@AndrewFerr ran into. And then running `dockers system prune` also
revealed the problematic `containerd` in CI. Checking the Docker
changelogs, I found the new default culprit and then could trace down
where the GitHub runners made the dependency update.

---------

Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>
2026-02-18 11:37:09 -06:00
Quentin Gliech 3833eb49cf Merge branch 'release-v1.148' into develop 2026-02-17 23:53:04 +01:00
Eric Eastwood b80774efb2 Better instrument JoinRoomAliasServlet with tracing (#19461)
So we can better see why it decides to do a local vs remote join.

Spawning from [investigating a join issue on `matrix.org`](https://matrix.to/#/!SGNQGPGUwtcPBUotTL:matrix.org/$Odvd47QtkRscxilzkhcFOsDZWNvJUSEhSrD8GpukKWo?via=jki.re&via=element.io&via=matrix.org).
2026-02-17 13:15:57 -06:00
84 changed files with 3450 additions and 3286 deletions
+1
View File
@@ -0,0 +1 @@
.github/workflows/* merge=ours
-221
View File
@@ -1,221 +0,0 @@
# GitHub actions workflow which builds and publishes the docker images.
name: Build docker images
on:
push:
tags: ["v*"]
branches: [master, main, develop]
workflow_dispatch:
permissions:
contents: read
packages: write
id-token: write # needed for signing the images with GitHub OIDC Token
jobs:
build:
name: Build and push image for ${{ matrix.platform }}
runs-on: ${{ matrix.runs_on }}
strategy:
matrix:
include:
- platform: linux/amd64
runs_on: ubuntu-24.04
suffix: linux-amd64
- platform: linux/arm64
runs_on: ubuntu-24.04-arm
suffix: linux-arm64
steps:
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Extract version from pyproject.toml
# Note: explicitly requesting bash will mean bash is invoked with `-eo pipefail`, see
# https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsshell
shell: bash
run: |
echo "SYNAPSE_VERSION=$(grep "^version" pyproject.toml | sed -E 's/version\s*=\s*["]([^"]*)["]/\1/')" >> $GITHUB_ENV
- name: Log in to DockerHub
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Log in to GHCR
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Tailscale
uses: tailscale/github-action@53acf823325fe9ca47f4cdaa951f90b4b0de5bb9 # v4.1.1
with:
oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
audience: ${{ secrets.TS_AUDIENCE }}
tags: tag:github-actions
- name: Compute vault jwt role name
id: vault-jwt-role
run: |
echo "role_name=github_service_management_$( echo "${{ github.repository }}" | sed -r 's|[/-]|_|g')" | tee -a "$GITHUB_OUTPUT"
- name: Get team registry token
id: import-secrets
uses: hashicorp/vault-action@4c06c5ccf5c0761b6029f56cfb1dcf5565918a3b # v3.4.0
with:
url: https://vault.infra.ci.i.element.dev
role: ${{ steps.vault-jwt-role.outputs.role_name }}
path: service-management/github-actions
jwtGithubAudience: https://vault.infra.ci.i.element.dev
method: jwt
secrets: |
services/backend-repositories/secret/data/oci.element.io username | OCI_USERNAME ;
services/backend-repositories/secret/data/oci.element.io password | OCI_PASSWORD ;
- name: Login to Element OCI Registry
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
with:
registry: oci-push.vpn.infra.element.io
username: ${{ steps.import-secrets.outputs.OCI_USERNAME }}
password: ${{ steps.import-secrets.outputs.OCI_PASSWORD }}
- name: Build and push by digest
id: build
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
push: true
labels: |
gitsha1=${{ github.sha }}
org.opencontainers.image.version=${{ env.SYNAPSE_VERSION }}
tags: |
docker.io/matrixdotorg/synapse
ghcr.io/element-hq/synapse
oci-push.vpn.infra.element.io/synapse
file: "docker/Dockerfile"
platforms: ${{ matrix.platform }}
outputs: type=image,push-by-digest=true,name-canonical=true,push=true
- name: Export digest
run: |
mkdir -p ${{ runner.temp }}/digests
digest="${{ steps.build.outputs.digest }}"
touch "${{ runner.temp }}/digests/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with:
name: digests-${{ matrix.suffix }}
path: ${{ runner.temp }}/digests/*
if-no-files-found: error
retention-days: 1
merge:
name: Push merged images to ${{ matrix.repository }}
runs-on: ubuntu-latest
strategy:
matrix:
repository:
- docker.io/matrixdotorg/synapse
- ghcr.io/element-hq/synapse
- oci-push.vpn.infra.element.io/synapse
needs:
- build
steps:
- name: Download digests
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
with:
path: ${{ runner.temp }}/digests
pattern: digests-*
merge-multiple: true
- name: Log in to DockerHub
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
if: ${{ startsWith(matrix.repository, 'docker.io') }}
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Log in to GHCR
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
if: ${{ startsWith(matrix.repository, 'ghcr.io') }}
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Tailscale
uses: tailscale/github-action@53acf823325fe9ca47f4cdaa951f90b4b0de5bb9 # v4.1.1
with:
oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
audience: ${{ secrets.TS_AUDIENCE }}
tags: tag:github-actions
- name: Compute vault jwt role name
id: vault-jwt-role
run: |
echo "role_name=github_service_management_$( echo "${{ github.repository }}" | sed -r 's|[/-]|_|g')" | tee -a "$GITHUB_OUTPUT"
- name: Get team registry token
id: import-secrets
uses: hashicorp/vault-action@4c06c5ccf5c0761b6029f56cfb1dcf5565918a3b # v3.4.0
with:
url: https://vault.infra.ci.i.element.dev
role: ${{ steps.vault-jwt-role.outputs.role_name }}
path: service-management/github-actions
jwtGithubAudience: https://vault.infra.ci.i.element.dev
method: jwt
secrets: |
services/backend-repositories/secret/data/oci.element.io username | OCI_USERNAME ;
services/backend-repositories/secret/data/oci.element.io password | OCI_PASSWORD ;
- name: Login to Element OCI Registry
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
with:
registry: oci-push.vpn.infra.element.io
username: ${{ steps.import-secrets.outputs.OCI_USERNAME }}
password: ${{ steps.import-secrets.outputs.OCI_PASSWORD }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
- name: Install Cosign
uses: sigstore/cosign-installer@faadad0cce49287aee09b3a48701e75088a2c6ad # v4.0.0
- name: Calculate docker image tag
uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5.10.0
with:
images: ${{ matrix.repository }}
flavor: |
latest=false
tags: |
type=raw,value=develop,enable=${{ github.ref == 'refs/heads/develop' }}
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/master' }}
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/main' }}
type=pep440,pattern={{raw}}
type=sha
- name: Create manifest list and push
working-directory: ${{ runner.temp }}/digests
env:
REPOSITORY: ${{ matrix.repository }}
run: |
docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf "$REPOSITORY@sha256:%s " *)
- name: Sign each manifest
env:
REPOSITORY: ${{ matrix.repository }}
run: |
DIGESTS=""
for TAG in $(echo "$DOCKER_METADATA_OUTPUT_JSON" | jq -r '.tags[]'); do
DIGEST="$(docker buildx imagetools inspect $TAG --format '{{json .Manifest}}' | jq -r '.digest')"
DIGESTS="$DIGESTS $REPOSITORY@$DIGEST"
done
cosign sign --yes $DIGESTS
-80
View File
@@ -1,80 +0,0 @@
name: Prepare documentation PR preview
on:
pull_request:
paths:
- docs/**
- book.toml
- .github/workflows/docs-pr.yaml
- scripts-dev/schema_versions.py
jobs:
pages:
name: GitHub Pages
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
# Fetch all history so that the schema_versions script works.
fetch-depth: 0
- name: Setup mdbook
uses: peaceiris/actions-mdbook@ee69d230fe19748b7abf22df32acaa93833fad08 # v2.0.0
with:
mdbook-version: '0.5.2'
- name: Setup python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.x"
- run: "pip install 'packaging>=20.0' 'GitPython>=3.1.20'"
- name: Build the documentation
# mdbook will only create an index.html if we're including docs/README.md in SUMMARY.md.
# However, we're using docs/README.md for other purposes and need to pick a new page
# as the default. Let's opt for the welcome page instead.
run: |
mdbook build
cp book/welcome_and_overview.html book/index.html
- name: Upload Artifact
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with:
name: book
path: book
# We'll only use this in a workflow_run, then we're done with it
retention-days: 1
link-check:
name: Check links in documentation
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup mdbook
uses: peaceiris/actions-mdbook@ee69d230fe19748b7abf22df32acaa93833fad08 # v2.0.0
with:
mdbook-version: '0.5.2'
- name: Setup htmltest
run: |
wget https://github.com/wjdp/htmltest/releases/download/v0.17.0/htmltest_0.17.0_linux_amd64.tar.gz
echo '775c597ee74899d6002cd2d93076f897f4ba68686bceabe2e5d72e84c57bc0fb htmltest_0.17.0_linux_amd64.tar.gz' | sha256sum -c
tar zxf htmltest_0.17.0_linux_amd64.tar.gz
- name: Test links with htmltest
run: |
# Build the book with `./` as the site URL (to make checks on 404.html possible)
MDBOOK_OUTPUT__HTML__SITE_URL="./" mdbook build
# Delete the contents of the print.html file, as it can raise false
# positives during link checking.
#
# We empty out the file, instead of deleting it, as doing so would
# just cause htmltest to complain that links to it were invalid.
# Ideally `htmltest` would have an option to ignore specific files
# instead.
echo '<!DOCTYPE HTML>' > book/print.html
./htmltest book --conf docs/.htmltest.yml
-99
View File
@@ -1,99 +0,0 @@
name: Deploy the documentation
on:
push:
branches:
# For bleeding-edge documentation
- develop
# For documentation specific to a release
- 'release-v*'
# stable docs
- master
workflow_dispatch:
jobs:
pre:
name: Calculate variables for GitHub Pages deployment
runs-on: ubuntu-latest
steps:
# Figure out the target directory.
#
# The target directory depends on the name of the branch
#
- name: Get the target directory name
id: vars
run: |
# first strip the 'refs/heads/' prefix with some shell foo
branch="${GITHUB_REF#refs/heads/}"
case $branch in
release-*)
# strip 'release-' from the name for release branches.
branch="${branch#release-}"
;;
master)
# deploy to "latest" for the master branch.
branch="latest"
;;
esac
# finally, set the 'branch-version' var.
echo "branch-version=$branch" >> "$GITHUB_OUTPUT"
outputs:
branch-version: ${{ steps.vars.outputs.branch-version }}
################################################################################
pages-docs:
name: GitHub Pages
runs-on: ubuntu-latest
needs:
- pre
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
# Fetch all history so that the schema_versions script works.
fetch-depth: 0
- name: Setup mdbook
uses: peaceiris/actions-mdbook@ee69d230fe19748b7abf22df32acaa93833fad08 # v2.0.0
with:
mdbook-version: '0.5.2'
- name: Set version of docs
run: echo 'window.SYNAPSE_VERSION = "${{ needs.pre.outputs.branch-version }}";' > ./docs/website_files/version.js
- name: Setup python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.x"
- run: "pip install 'packaging>=20.0' 'GitPython>=3.1.20'"
- name: Build the documentation
# mdbook will only create an index.html if we're including docs/README.md in SUMMARY.md.
# However, we're using docs/README.md for other purposes and need to pick a new page
# as the default. Let's opt for the welcome page instead.
run: |
mdbook build
cp book/welcome_and_overview.html book/index.html
- name: Prepare and publish schema files
run: |
sudo apt-get update && sudo apt-get install -y yq
mkdir -p book/schema
# Remove developer notice before publishing.
rm schema/v*/Do\ not\ edit\ files\ in\ this\ folder
# Copy schema files that are independent from current Synapse version.
cp -r -t book/schema schema/v*/
# Convert config schema from YAML source file to JSON.
yq < schema/synapse-config.schema.yaml \
> book/schema/synapse-config.schema.json
# Deploy to the target directory.
- name: Deploy to gh pages
uses: peaceiris/actions-gh-pages@4f9cc6602d3f66b9c108549d475ec49e8ef4d45e # v4.0.0
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./book
destination_dir: ./${{ needs.pre.outputs.branch-version }}
-52
View File
@@ -1,52 +0,0 @@
# A helper workflow to automatically fixup any linting errors on a PR. Must be
# triggered manually.
name: Attempt to automatically fix linting errors
on:
workflow_dispatch:
env:
# We use nightly so that `fmt` correctly groups together imports, and
# clippy correctly fixes up the benchmarks.
RUST_VERSION: nightly-2025-06-24
jobs:
fixup:
name: Fix up
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
toolchain: ${{ env.RUST_VERSION }}
components: clippy, rustfmt
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- name: Setup Poetry
uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
with:
install-project: "false"
poetry-version: "2.1.1"
- name: Run ruff check
continue-on-error: true
run: poetry run ruff check --fix .
- name: Run ruff format
continue-on-error: true
run: poetry run ruff format --quiet .
- run: cargo clippy --all-features --fix -- -D warnings
continue-on-error: true
- run: cargo fmt
continue-on-error: true
- uses: stefanzweifel/git-auto-commit-action@04702edda442b2e678b25b537cec683a1493fcb9 # v7.1.0
with:
commit_message: "Attempt to fix linting"
-283
View File
@@ -1,283 +0,0 @@
# People who are freshly `pip install`ing from PyPI will pull in the latest versions of
# dependencies which match the broad requirements. Since most CI runs are against
# the locked poetry environment, run specifically against the latest dependencies to
# know if there's an upcoming breaking change.
#
# As an overview this workflow:
# - checks out develop,
# - installs from source, pulling in the dependencies like a fresh `pip install` would, and
# - runs mypy and test suites in that checkout.
#
# Based on the twisted trunk CI job.
name: Latest dependencies
on:
schedule:
- cron: 0 7 * * *
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
RUST_VERSION: 1.87.0
jobs:
check_repo:
# Prevent this workflow from running on any fork of Synapse other than element-hq/synapse, as it is
# only useful to the Synapse core team.
# All other workflow steps depend on this one, thus if 'should_run_workflow' is not 'true', the rest
# of the workflow will be skipped as well.
runs-on: ubuntu-latest
outputs:
should_run_workflow: ${{ steps.check_condition.outputs.should_run_workflow }}
steps:
- id: check_condition
run: echo "should_run_workflow=${{ github.repository == 'element-hq/synapse' }}" >> "$GITHUB_OUTPUT"
mypy:
needs: check_repo
if: needs.check_repo.outputs.should_run_workflow == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
toolchain: ${{ env.RUST_VERSION }}
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
# The dev dependencies aren't exposed in the wheel metadata (at least with current
# poetry-core versions), so we install with poetry.
- uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
with:
python-version: "3.x"
poetry-version: "2.1.1"
extras: "all"
# Dump installed versions for debugging.
- run: poetry run pip list > before.txt
# Upgrade all runtime dependencies only. This is intended to mimic a fresh
# `pip install matrix-synapse[all]` as closely as possible.
- run: poetry update --without dev
- run: poetry run pip list > after.txt && (diff -u before.txt after.txt || true)
- name: Remove unhelpful options from mypy config
run: sed -e '/warn_unused_ignores = True/d' -e '/warn_redundant_casts = True/d' -i mypy.ini
- run: poetry run mypy
trial:
needs: check_repo
if: needs.check_repo.outputs.should_run_workflow == 'true'
runs-on: ubuntu-latest
strategy:
matrix:
include:
- database: "sqlite"
- database: "postgres"
postgres-version: "14"
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
toolchain: ${{ env.RUST_VERSION }}
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- run: sudo apt-get -qq install xmlsec1
- name: Set up PostgreSQL ${{ matrix.postgres-version }}
if: ${{ matrix.postgres-version }}
run: |
docker run -d -p 5432:5432 \
-e POSTGRES_PASSWORD=postgres \
-e POSTGRES_INITDB_ARGS="--lc-collate C --lc-ctype C --encoding UTF8" \
postgres:${{ matrix.postgres-version }}
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.x"
- run: pip install .[all,test]
- name: Await PostgreSQL
if: ${{ matrix.postgres-version }}
timeout-minutes: 2
run: until pg_isready -h localhost; do sleep 1; done
# We nuke the local copy, as we've installed synapse into the virtualenv
# (rather than use an editable install, which we no longer support). If we
# don't do this then python can't find the native lib.
- run: rm -rf synapse/
- run: python -m twisted.trial --jobs=2 tests
env:
SYNAPSE_POSTGRES: ${{ matrix.database == 'postgres' || '' }}
SYNAPSE_POSTGRES_HOST: localhost
SYNAPSE_POSTGRES_USER: postgres
SYNAPSE_POSTGRES_PASSWORD: postgres
- name: Dump logs
# Logs are most useful when the command fails, always include them.
if: ${{ always() }}
# Note: Dumps to workflow logs instead of using actions/upload-artifact
# This keeps logs colocated with failing jobs
# It also ignores find's exit code; this is a best effort affair
run: >-
find _trial_temp -name '*.log'
-exec echo "::group::{}" \;
-exec cat {} \;
-exec echo "::endgroup::" \;
|| true
sytest:
needs: check_repo
if: needs.check_repo.outputs.should_run_workflow == 'true'
runs-on: ubuntu-latest
container:
image: matrixdotorg/sytest-synapse:testing
volumes:
- ${{ github.workspace }}:/src
strategy:
fail-fast: false
matrix:
include:
- sytest-tag: bookworm
- sytest-tag: bookworm
postgres: postgres
workers: workers
redis: redis
env:
POSTGRES: ${{ matrix.postgres && 1}}
WORKERS: ${{ matrix.workers && 1 }}
REDIS: ${{ matrix.redis && 1 }}
BLACKLIST: ${{ matrix.workers && 'synapse-blacklist-with-workers' }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
toolchain: ${{ env.RUST_VERSION }}
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- name: Ensure sytest runs `pip install`
# Delete the lockfile so sytest will `pip install` rather than `poetry install`
run: rm /src/poetry.lock
working-directory: /src
- name: Prepare test blacklist
run: cat sytest-blacklist .ci/worker-blacklist > synapse-blacklist-with-workers
- name: Run SyTest
run: /bootstrap.sh synapse
working-directory: /src
- name: Summarise results.tap
if: ${{ always() }}
run: /sytest/scripts/tap_to_gha.pl /logs/results.tap
- name: Upload SyTest logs
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
if: ${{ always() }}
with:
name: Sytest Logs - ${{ job.status }} - (${{ join(matrix.*, ', ') }})
path: |
/logs/results.tap
/logs/**/*.log*
complement:
needs: check_repo
if: "!failure() && !cancelled() && needs.check_repo.outputs.should_run_workflow == 'true'"
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- arrangement: monolith
database: SQLite
- arrangement: monolith
database: Postgres
- arrangement: workers
database: Postgres
steps:
- name: Check out synapse codebase
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
path: synapse
- name: Prepare Complement's Prerequisites
run: synapse/.ci/scripts/setup_complement_prerequisites.sh
- uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0
with:
cache-dependency-path: complement/go.sum
go-version-file: complement/go.mod
- name: Run Complement Tests
id: run_complement_tests
# -p=1: We're using `-p 1` to force the test packages to run serially as GHA boxes
# are underpowered and don't like running tons of Synapse instances at once.
# -json: Output JSON format so that gotestfmt can parse it.
#
# tee /tmp/gotest-complement.log: We tee the output to a file so that we can re-process it
# later on for better formatting with gotestfmt. But we still want the command
# to output to the terminal as it runs so we can see what's happening in
# real-time.
run: |
set -o pipefail
COMPLEMENT_DIR=`pwd`/complement synapse/scripts-dev/complement.sh -p 1 -json 2>&1 | tee /tmp/gotest-complement.log
shell: bash
env:
POSTGRES: ${{ (matrix.database == 'Postgres') && 1 || '' }}
WORKERS: ${{ (matrix.arrangement == 'workers') && 1 || '' }}
TEST_ONLY_IGNORE_POETRY_LOCKFILE: 1
- name: Formatted Complement test logs
# Always run this step if we attempted to run the Complement tests.
if: always() && steps.run_complement_tests.outcome != 'skipped'
run: cat /tmp/gotest-complement.log | gotestfmt -hide "successful-downloads,empty-packages"
- name: Run in-repo Complement Tests
id: run_in_repo_complement_tests
# -p=1: We're using `-p 1` to force the test packages to run serially as GHA boxes
# are underpowered and don't like running tons of Synapse instances at once.
# -json: Output JSON format so that gotestfmt can parse it.
#
# tee /tmp/gotest-in-repo-complement.log: We tee the output to a file so that we can re-process it
# later on for better formatting with gotestfmt. But we still want the command
# to output to the terminal as it runs so we can see what's happening in
# real-time.
run: |
set -o pipefail
COMPLEMENT_DIR=`pwd`/complement synapse/scripts-dev/complement.sh --in-repo -p 1 -json 2>&1 | tee /tmp/gotest-in-repo-complement.log
shell: bash
env:
POSTGRES: ${{ (matrix.database == 'Postgres') && 1 || '' }}
WORKERS: ${{ (matrix.arrangement == 'workers') && 1 || '' }}
TEST_ONLY_IGNORE_POETRY_LOCKFILE: 1
- name: Formatted in-repo Complement test logs
# Always run this step if we attempted to run the Complement tests.
if: always() && steps.run_in_repo_complement_tests.outcome != 'skipped'
run: cat /tmp/gotest-in-repo-complement.log | gotestfmt -hide "successful-downloads,empty-packages"
# Open an issue if the build fails, so we know about it.
# Only do this if we're not experimenting with this action in a PR.
open-issue:
if: "failure() && github.event_name != 'push' && github.event_name != 'pull_request' && needs.check_repo.outputs.should_run_workflow == 'true'"
needs:
# TODO: should mypy be included here? It feels more brittle than the others.
- mypy
- trial
- sytest
- complement
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: JasonEtco/create-an-issue@1b14a70e4d8dc185e5cc76d3bec9eab20257b2c5 # v2.9.2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
update_existing: true
filename: .ci/latest_deps_build_failed_issue_template.md
-24
View File
@@ -1,24 +0,0 @@
on:
push:
branches: ["develop", "release-*"]
paths:
- poetry.lock
pull_request:
paths:
- poetry.lock
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
check-sdists:
name: "Check locked dependencies have sdists"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.x'
- run: pip install tomli
- run: ./scripts-dev/check_locked_deps_have_sdists.py
@@ -1,74 +0,0 @@
# This task does not run complement tests, see tests.yaml instead.
# This task does not build docker images for synapse for use on docker hub, see docker.yaml instead
name: Store complement-synapse image in ghcr.io
on:
push:
branches: [ "master" ]
schedule:
- cron: '0 5 * * *'
workflow_dispatch:
inputs:
branch:
required: true
default: 'develop'
type: choice
options:
- develop
- master
# Only run this action once per pull request/branch; restart if a new commit arrives.
# C.f. https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#concurrency
# and https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#github-context
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
name: Build and push complement image
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout specific branch (debug build)
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
if: github.event_name == 'workflow_dispatch'
with:
ref: ${{ inputs.branch }}
- name: Checkout clean copy of develop (scheduled build)
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
if: github.event_name == 'schedule'
with:
ref: develop
- name: Checkout clean copy of master (on-push)
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
if: github.event_name == 'push'
with:
ref: master
- name: Login to registry
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Work out labels for complement image
id: meta
uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5.10.0
with:
images: ghcr.io/${{ github.repository }}/complement-synapse
tags: |
type=schedule,pattern=nightly,enable=${{ github.event_name == 'schedule'}}
type=raw,value=develop,enable=${{ github.event_name == 'schedule' || inputs.branch == 'develop' }}
type=raw,value=latest,enable=${{ github.event_name == 'push' || inputs.branch == 'master' }}
type=sha,format=long
- name: Run scripts-dev/complement.sh to generate complement-synapse:latest image.
run: scripts-dev/complement.sh --build-only
- name: Tag and push generated image
run: |
for TAG in ${{ join(fromJson(steps.meta.outputs.json).tags, ' ') }}; do
echo "tag and push $TAG"
docker tag complement-synapse $TAG
docker push $TAG
done
-206
View File
@@ -1,206 +0,0 @@
# GitHub actions workflow which builds the release artifacts.
name: Build release artifacts
on:
# we build on PRs and develop to (hopefully) get early warning
# of things breaking (but only build one set of debs). PRs skip
# building wheels on ARM.
pull_request:
push:
branches: ["develop", "release-*"]
# we do the full build on tags.
tags: ["v*"]
merge_group:
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: write
jobs:
get-distros:
name: "Calculate list of debian distros"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.x"
- id: set-distros
run: |
# if we're running from a tag, get the full list of distros; otherwise just use debian:sid
# NOTE: inside the actual Dockerfile-dhvirtualenv, the image name is expanded into its full image path
dists='["debian:sid"]'
if [[ $GITHUB_REF == refs/tags/* ]]; then
dists=$(scripts-dev/build_debian_packages.py --show-dists-json)
fi
echo "distros=$dists" >> "$GITHUB_OUTPUT"
# map the step outputs to job outputs
outputs:
distros: ${{ steps.set-distros.outputs.distros }}
# now build the packages with a matrix build.
build-debs:
needs: get-distros
name: "Build .deb packages"
runs-on: ubuntu-latest
strategy:
matrix:
distro: ${{ fromJson(needs.get-distros.outputs.distros) }}
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
path: src
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
- name: Set up docker layer caching
uses: actions/cache@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-
- name: Set up python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.x"
- name: Build the packages
# see https://github.com/docker/build-push-action/issues/252
# for the cache magic here
run: |
./src/scripts-dev/build_debian_packages.py \
--docker-build-arg=--cache-from=type=local,src=/tmp/.buildx-cache \
--docker-build-arg=--cache-to=type=local,mode=max,dest=/tmp/.buildx-cache-new \
--docker-build-arg=--progress=plain \
--docker-build-arg=--load \
"${{ matrix.distro }}"
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
- name: Artifact name
id: artifact-name
# We can't have colons in the upload name of the artifact, so we convert
# e.g. `debian:sid` to `sid`.
env:
DISTRO: ${{ matrix.distro }}
run: |
echo "ARTIFACT_NAME=${DISTRO#*:}" >> "$GITHUB_OUTPUT"
- name: Upload debs as artifacts
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with:
name: debs-${{ steps.artifact-name.outputs.ARTIFACT_NAME }}
path: debs/*
build-wheels:
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os:
- ubuntu-24.04
- ubuntu-24.04-arm
# is_pr is a flag used to exclude certain jobs from the matrix on PRs.
# It is not read by the rest of the workflow.
is_pr:
- ${{ startsWith(github.ref, 'refs/pull/') }}
exclude:
# Don't build aarch64 wheels on PR CI.
- is_pr: true
os: "ubuntu-24.04-arm"
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
# setup-python@v4 doesn't impose a default python version. Need to use 3.x
# here, because `python` on osx points to Python 2.7.
python-version: "3.x"
- name: Install cibuildwheel
run: python -m pip install cibuildwheel==3.2.1
- name: Only build a single wheel on PR
if: startsWith(github.ref, 'refs/pull/')
run: echo "CIBW_BUILD="cp310-manylinux_*"" >> $GITHUB_ENV
- name: Build wheels
run: python -m cibuildwheel --output-dir wheelhouse
env:
# The platforms that we build for are determined by the
# `tool.cibuildwheel.skip` option in `pyproject.toml`.
# We skip testing wheels for the following platforms in CI:
#
# pp3*-* (PyPy wheels) broke in CI (TODO: investigate).
# musl: (TODO: investigate).
CIBW_TEST_SKIP: pp3*-* *musl*
- uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with:
name: Wheel-${{ matrix.os }}
path: ./wheelhouse/*.whl
build-sdist:
name: Build sdist
runs-on: ubuntu-latest
if: ${{ !startsWith(github.ref, 'refs/pull/') }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.10"
- run: pip install build
- name: Build sdist
run: python -m build --sdist
- uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with:
name: Sdist
path: dist/*.tar.gz
# if it's a tag, create a release and attach the artifacts to it
attach-assets:
name: "Attach assets to release"
if: ${{ !failure() && !cancelled() && startsWith(github.ref, 'refs/tags/') }}
needs:
- build-debs
- build-wheels
- build-sdist
runs-on: ubuntu-latest
steps:
- name: Download all workflow run artifacts
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
- name: Build a tarball for the debs
# We need to merge all the debs uploads into one folder, then compress
# that.
run: |
mkdir debs
mv debs*/* debs/
tar -cvJf debs.tar.xz debs
- name: Attach to release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
gh release upload "${{ github.ref_name }}" \
Sdist/* \
Wheel*/* \
debs.tar.xz \
--repo ${{ github.repository }}
-57
View File
@@ -1,57 +0,0 @@
name: Schema
on:
pull_request:
paths:
- schema/**
- docs/usage/configuration/config_documentation.md
push:
branches: ["develop", "release-*"]
workflow_dispatch:
jobs:
validate-schema:
name: Ensure Synapse config schema is valid
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.x"
- name: Install check-jsonschema
run: pip install check-jsonschema==0.33.0
- name: Validate meta schema
run: check-jsonschema --check-metaschema schema/v*/meta.schema.json
- name: Validate schema
run: |-
# Please bump on introduction of a new meta schema.
LATEST_META_SCHEMA_VERSION=v1
check-jsonschema \
--schemafile="schema/$LATEST_META_SCHEMA_VERSION/meta.schema.json" \
schema/synapse-config.schema.yaml
- name: Validate default config
# Populates the empty instance with default values and checks against the schema.
run: |-
echo "{}" | check-jsonschema \
--fill-defaults --schemafile=schema/synapse-config.schema.yaml -
check-doc-generation:
name: Ensure generated documentation is up-to-date
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.x"
- name: Install PyYAML
run: pip install PyYAML==6.0.2
- name: Regenerate config documentation
run: |
scripts-dev/gen_config_documentation.py \
schema/synapse-config.schema.yaml \
> docs/usage/configuration/config_documentation.md
- name: Error in case of any differences
# Errors if there are now any modified files (untracked files are ignored).
run: 'git diff --exit-code'
-824
View File
@@ -1,824 +0,0 @@
name: Tests
on:
push:
branches: ["develop", "release-*"]
pull_request:
merge_group:
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
RUST_VERSION: 1.87.0
jobs:
# Job to detect what has changed so we don't run e.g. Rust checks on PRs that
# don't modify Rust code.
changes:
runs-on: ubuntu-latest
outputs:
rust: ${{ !startsWith(github.ref, 'refs/pull/') || steps.filter.outputs.rust }}
trial: ${{ !startsWith(github.ref, 'refs/pull/') || steps.filter.outputs.trial }}
integration: ${{ !startsWith(github.ref, 'refs/pull/') || steps.filter.outputs.integration }}
linting: ${{ !startsWith(github.ref, 'refs/pull/') || steps.filter.outputs.linting }}
linting_readme: ${{ !startsWith(github.ref, 'refs/pull/') || steps.filter.outputs.linting_readme }}
steps:
- uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
id: filter
# We only check on PRs
if: startsWith(github.ref, 'refs/pull/')
with:
filters: |
rust:
- 'rust/**'
- 'Cargo.toml'
- 'Cargo.lock'
- '.rustfmt.toml'
- '.github/workflows/tests.yml'
trial:
- 'synapse/**'
- 'tests/**'
- 'rust/**'
- '.ci/scripts/calculate_jobs.py'
- 'Cargo.toml'
- 'Cargo.lock'
- 'pyproject.toml'
- 'poetry.lock'
- '.github/workflows/tests.yml'
integration:
- 'synapse/**'
- 'rust/**'
- 'docker/**'
- 'Cargo.toml'
- 'Cargo.lock'
- 'pyproject.toml'
- 'poetry.lock'
- 'docker/**'
- '.ci/**'
- 'scripts-dev/complement.sh'
- '.github/workflows/tests.yml'
linting:
- 'synapse/**'
- 'docker/**'
- 'tests/**'
- 'scripts-dev/**'
- 'contrib/**'
- 'synmark/**'
- 'stubs/**'
- '.ci/**'
- 'mypy.ini'
- 'pyproject.toml'
- 'poetry.lock'
- '.github/workflows/tests.yml'
linting_readme:
- 'README.rst'
check-sampleconfig:
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.linting == 'true' }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
toolchain: ${{ env.RUST_VERSION }}
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
with:
python-version: "3.x"
poetry-version: "2.1.1"
extras: "all"
- run: poetry run scripts-dev/generate_sample_config.sh --check
- run: poetry run scripts-dev/config-lint.sh
check-schema-delta:
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.linting == 'true' }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.x"
- run: "pip install 'click==8.1.1' 'GitPython>=3.1.20' 'sqlglot>=28.0.0'"
- run: scripts-dev/check_schema_delta.py --force-colors
check-lockfile:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.x"
- run: .ci/scripts/check_lockfile.py
lint:
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.linting == 'true' }}
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup Poetry
uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
with:
poetry-version: "2.1.1"
install-project: "false"
- name: Run ruff check
run: poetry run ruff check --output-format=github .
- name: Run ruff format
run: poetry run ruff format --check .
lint-mypy:
runs-on: ubuntu-latest
name: Typechecking
needs: changes
if: ${{ needs.changes.outputs.linting == 'true' }}
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
toolchain: ${{ env.RUST_VERSION }}
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- name: Setup Poetry
uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
with:
# We want to make use of type hints in optional dependencies too.
extras: all
# We have seen odd mypy failures that were resolved when we started
# installing the project again:
# https://github.com/matrix-org/synapse/pull/15376#issuecomment-1498983775
# To make CI green, err towards caution and install the project.
install-project: "true"
poetry-version: "2.1.1"
# Cribbed from
# https://github.com/AustinScola/mypy-cache-github-action/blob/85ea4f2972abed39b33bd02c36e341b28ca59213/src/restore.ts#L10-L17
- name: Restore/persist mypy's cache
uses: actions/cache@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3
with:
path: |
.mypy_cache
key: mypy-cache-${{ github.context.sha }}
restore-keys: mypy-cache-
- name: Run mypy
run: poetry run mypy
lint-crlf:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Check line endings
run: scripts-dev/check_line_terminators.sh
lint-newsfile:
# Only run on pull_request events, targeting develop/release branches, and skip when the PR author is dependabot[bot].
if: ${{ github.event_name == 'pull_request' && (github.base_ref == 'develop' || contains(github.base_ref, 'release-')) && github.event.pull_request.user.login != 'dependabot[bot]' }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.x"
- run: "pip install 'towncrier>=18.6.0rc1'"
- run: scripts-dev/check-newsfragment.sh
env:
PULL_REQUEST_NUMBER: ${{ github.event.number }}
lint-clippy:
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.rust == 'true' }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
components: clippy
toolchain: ${{ env.RUST_VERSION }}
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- run: cargo clippy -- -D warnings
# We also lint against a nightly rustc so that we can lint the benchmark
# suite, which requires a nightly compiler.
lint-clippy-nightly:
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.rust == 'true' }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
toolchain: nightly-2026-02-01
components: clippy
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- run: cargo clippy --all-features -- -D warnings
lint-rust:
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.rust == 'true' }}
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
toolchain: ${{ env.RUST_VERSION }}
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- name: Setup Poetry
uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
with:
# Install like a normal project from source with all optional dependencies
extras: all
install-project: "true"
poetry-version: "2.1.1"
- name: Ensure `Cargo.lock` is up to date (no stray changes after install)
# The `::error::` syntax is using GitHub Actions' error annotations, see
# https://docs.github.com/en/actions/reference/workflow-commands-for-github-actions
run: |
if git diff --quiet Cargo.lock; then
echo "Cargo.lock is up to date"
else
echo "::error::Cargo.lock has uncommitted changes after install. Please run 'poetry install --extras all' and commit the Cargo.lock changes."
git diff --exit-code Cargo.lock
exit 1
fi
# This job is split from `lint-rust` because it requires a nightly Rust toolchain
# for some of the unstable options we use in `.rustfmt.toml`.
lint-rustfmt:
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.rust == 'true' }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
# We use nightly so that we can use some unstable options that we use in
# `.rustfmt.toml`.
toolchain: nightly-2025-04-23
components: rustfmt
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- run: cargo fmt --check
# This is to detect issues with the rst file, which can otherwise cause issues
# when uploading packages to PyPi.
lint-readme:
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.linting_readme == 'true' }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.x"
- run: "pip install rstcheck"
- run: "rstcheck --report-level=WARNING README.rst"
# Dummy step to gate other tests on without repeating the whole list
linting-done:
if: ${{ !cancelled() }} # Run this even if prior jobs were skipped
needs:
- lint
- lint-mypy
- lint-crlf
- lint-newsfile
- check-sampleconfig
- check-schema-delta
- check-lockfile
- lint-clippy
- lint-clippy-nightly
- lint-rust
- lint-rustfmt
- lint-readme
runs-on: ubuntu-latest
steps:
- uses: matrix-org/done-action@3409aa904e8a2aaf2220f09bc954d3d0b0a2ee67 # v3
with:
needs: ${{ toJSON(needs) }}
# Various bits are skipped if there was no applicable changes.
skippable: |
check-sampleconfig
check-schema-delta
lint
lint-mypy
lint-newsfile
lint-clippy
lint-clippy-nightly
lint-rust
lint-rustfmt
lint-readme
calculate-test-jobs:
if: ${{ !cancelled() && !failure() }} # Allow previous steps to be skipped, but not fail
needs: linting-done
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.x"
- id: get-matrix
run: .ci/scripts/calculate_jobs.py
outputs:
trial_test_matrix: ${{ steps.get-matrix.outputs.trial_test_matrix }}
sytest_test_matrix: ${{ steps.get-matrix.outputs.sytest_test_matrix }}
trial:
if: ${{ !cancelled() && !failure() && needs.changes.outputs.trial == 'true' }} # Allow previous steps to be skipped, but not fail
needs:
- calculate-test-jobs
- changes
runs-on: ubuntu-latest
strategy:
matrix:
job: ${{ fromJson(needs.calculate-test-jobs.outputs.trial_test_matrix) }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- run: sudo apt-get -qq install xmlsec1
- name: Set up PostgreSQL ${{ matrix.job.postgres-version }}
if: ${{ matrix.job.postgres-version }}
# 1. Mount postgres data files onto a tmpfs in-memory filesystem to reduce overhead of docker's overlayfs layer.
# 2. Expose the unix socket for postgres. This removes latency of using docker-proxy for connections.
run: |
docker run -d -p 5432:5432 \
--tmpfs /var/lib/postgres:rw,size=6144m \
--mount 'type=bind,src=/var/run/postgresql,dst=/var/run/postgresql' \
-e POSTGRES_PASSWORD=postgres \
-e POSTGRES_INITDB_ARGS="--lc-collate C --lc-ctype C --encoding UTF8" \
postgres:${{ matrix.job.postgres-version }}
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
toolchain: ${{ env.RUST_VERSION }}
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
with:
python-version: ${{ matrix.job.python-version }}
poetry-version: "2.1.1"
extras: ${{ matrix.job.extras }}
- name: Await PostgreSQL
if: ${{ matrix.job.postgres-version }}
timeout-minutes: 2
run: until pg_isready -h localhost; do sleep 1; done
- run: poetry run trial --jobs=6 tests
env:
SYNAPSE_POSTGRES: ${{ matrix.job.database == 'postgres' || '' }}
SYNAPSE_POSTGRES_HOST: /var/run/postgresql
SYNAPSE_POSTGRES_USER: postgres
SYNAPSE_POSTGRES_PASSWORD: postgres
- name: Dump logs
# Logs are most useful when the command fails, always include them.
if: ${{ always() }}
# Note: Dumps to workflow logs instead of using actions/upload-artifact
# This keeps logs colocated with failing jobs
# It also ignores find's exit code; this is a best effort affair
run: >-
find _trial_temp -name '*.log'
-exec echo "::group::{}" \;
-exec cat {} \;
-exec echo "::endgroup::" \;
|| true
trial-olddeps:
# Note: sqlite only; no postgres
if: ${{ !cancelled() && !failure() && needs.changes.outputs.trial == 'true' }} # Allow previous steps to be skipped, but not fail
needs:
- linting-done
- changes
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
toolchain: ${{ env.RUST_VERSION }}
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
# There aren't wheels for some of the older deps, so we need to install
# their build dependencies
- run: |
sudo apt-get -qq update
sudo apt-get -qq install build-essential libffi-dev python3-dev \
libxml2-dev libxslt-dev xmlsec1 zlib1g-dev libjpeg-dev libwebp-dev
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.10"
- name: Prepare old deps
# Note: we install using `uv` here, not poetry or pip to allow us to test with the
# minimum version of all dependencies, both those explicitly specified and those
# implicitly brought in by the explicit dependencies.
run: |
pip install uv
uv pip install --system --resolution=lowest .[all,test]
# We nuke the local copy, as we've installed synapse into the virtualenv
# (rather than use an editable install, which we no longer support). If we
# don't do this then python can't find the native lib.
- run: rm -rf synapse/
# Sanity check we can import/run Synapse
- run: python -m synapse.app.homeserver --help
- run: python -m twisted.trial -j6 tests
- name: Dump logs
# Logs are most useful when the command fails, always include them.
if: ${{ always() }}
# Note: Dumps to workflow logs instead of using actions/upload-artifact
# This keeps logs colocated with failing jobs
# It also ignores find's exit code; this is a best effort affair
run: >-
find _trial_temp -name '*.log'
-exec echo "::group::{}" \;
-exec cat {} \;
-exec echo "::endgroup::" \;
|| true
trial-pypy:
# Very slow; only run if the branch name includes 'pypy'
# Note: sqlite only; no postgres. Completely untested since poetry move.
if: ${{ contains(github.ref, 'pypy') && !failure() && !cancelled() && needs.changes.outputs.trial == 'true' }}
needs:
- linting-done
- changes
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["pypy-3.10"]
extras: ["all"]
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
# Install libs necessary for PyPy to build binary wheels for dependencies
- run: sudo apt-get -qq install xmlsec1 libxml2-dev libxslt-dev
- uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
with:
python-version: ${{ matrix.python-version }}
poetry-version: "2.1.1"
extras: ${{ matrix.extras }}
- run: poetry run trial --jobs=2 tests
- name: Dump logs
# Logs are most useful when the command fails, always include them.
if: ${{ always() }}
# Note: Dumps to workflow logs instead of using actions/upload-artifact
# This keeps logs colocated with failing jobs
# It also ignores find's exit code; this is a best effort affair
run: >-
find _trial_temp -name '*.log'
-exec echo "::group::{}" \;
-exec cat {} \;
-exec echo "::endgroup::" \;
|| true
sytest:
if: ${{ !failure() && !cancelled() && needs.changes.outputs.integration == 'true' }}
needs:
- calculate-test-jobs
- changes
runs-on: ubuntu-latest
container:
image: matrixdotorg/sytest-synapse:${{ matrix.job.sytest-tag }}
volumes:
- ${{ github.workspace }}:/src
env:
# If this is a pull request to a release branch, use that branch as default branch for sytest, else use develop
# This works because the release script always create a branch on the sytest repo with the same name as the release branch
SYTEST_DEFAULT_BRANCH: ${{ startsWith(github.base_ref, 'release-') && github.base_ref || 'develop' }}
SYTEST_BRANCH: ${{ github.head_ref }}
POSTGRES: ${{ matrix.job.postgres && 1}}
MULTI_POSTGRES: ${{ (matrix.job.postgres == 'multi-postgres') || '' }}
ASYNCIO_REACTOR: ${{ (matrix.job.reactor == 'asyncio') || '' }}
WORKERS: ${{ matrix.job.workers && 1 }}
BLACKLIST: ${{ matrix.job.workers && 'synapse-blacklist-with-workers' }}
TOP: ${{ github.workspace }}
strategy:
fail-fast: false
matrix:
job: ${{ fromJson(needs.calculate-test-jobs.outputs.sytest_test_matrix) }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Prepare test blacklist
run: cat sytest-blacklist .ci/worker-blacklist > synapse-blacklist-with-workers
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
toolchain: ${{ env.RUST_VERSION }}
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- name: Run SyTest
run: /bootstrap.sh synapse
working-directory: /src
- name: Summarise results.tap
if: ${{ always() }}
run: /sytest/scripts/tap_to_gha.pl /logs/results.tap
- name: Upload SyTest logs
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
if: ${{ always() }}
with:
name: Sytest Logs - ${{ job.status }} - (${{ join(matrix.job.*, ', ') }})
path: |
/logs/results.tap
/logs/**/*.log*
export-data:
if: ${{ !failure() && !cancelled() && needs.changes.outputs.integration == 'true'}} # Allow previous steps to be skipped, but not fail
needs: [linting-done, portdb, changes]
runs-on: ubuntu-latest
env:
TOP: ${{ github.workspace }}
services:
postgres:
image: postgres
ports:
- 5432:5432
env:
POSTGRES_PASSWORD: "postgres"
POSTGRES_INITDB_ARGS: "--lc-collate C --lc-ctype C --encoding UTF8"
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- run: sudo apt-get -qq install xmlsec1 postgresql-client
- uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
with:
poetry-version: "2.1.1"
extras: "postgres"
- run: .ci/scripts/test_export_data_command.sh
env:
PGHOST: localhost
PGUSER: postgres
PGPASSWORD: postgres
PGDATABASE: postgres
portdb:
if: ${{ !failure() && !cancelled() && needs.changes.outputs.integration == 'true'}} # Allow previous steps to be skipped, but not fail
needs:
- linting-done
- changes
runs-on: ubuntu-latest
strategy:
matrix:
include:
- python-version: "3.10"
postgres-version: "14"
- python-version: "3.14"
postgres-version: "17"
services:
postgres:
image: postgres:${{ matrix.postgres-version }}
ports:
- 5432:5432
env:
POSTGRES_PASSWORD: "postgres"
POSTGRES_INITDB_ARGS: "--lc-collate C --lc-ctype C --encoding UTF8"
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Add PostgreSQL apt repository
# We need a version of pg_dump that can handle the version of
# PostgreSQL being tested against. The Ubuntu package repository lags
# behind new releases, so we have to use the PostreSQL apt repository.
# Steps taken from https://www.postgresql.org/download/linux/ubuntu/
run: |
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo apt-get update
- run: sudo apt-get -qq install xmlsec1 postgresql-client
- uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
with:
python-version: ${{ matrix.python-version }}
poetry-version: "2.1.1"
extras: "postgres"
- run: .ci/scripts/test_synapse_port_db.sh
id: run_tester_script
env:
PGHOST: localhost
PGUSER: postgres
PGPASSWORD: postgres
PGDATABASE: postgres
- name: "Upload schema differences"
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
if: ${{ failure() && !cancelled() && steps.run_tester_script.outcome == 'failure' }}
with:
name: Schema dumps
path: |
unported.sql
ported.sql
schema_diff
complement:
if: "${{ !failure() && !cancelled() && needs.changes.outputs.integration == 'true' }}"
needs:
- linting-done
- changes
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- arrangement: monolith
database: SQLite
- arrangement: monolith
database: Postgres
- arrangement: workers
database: Postgres
steps:
- name: Checkout synapse codebase
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
path: synapse
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
toolchain: ${{ env.RUST_VERSION }}
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- name: Prepare Complement's Prerequisites
run: synapse/.ci/scripts/setup_complement_prerequisites.sh
- uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0
with:
cache-dependency-path: complement/go.sum
go-version-file: complement/go.mod
- name: Run Complement Tests
id: run_complement_tests
# -p=1: We're using `-p 1` to force the test packages to run serially as GHA boxes
# are underpowered and don't like running tons of Synapse instances at once.
# -json: Output JSON format so that gotestfmt can parse it.
#
# tee /tmp/gotest-complement.log: We tee the output to a file so that we can re-process it
# later on for better formatting with gotestfmt. But we still want the command
# to output to the terminal as it runs so we can see what's happening in
# real-time.
run: |
set -o pipefail
COMPLEMENT_DIR=`pwd`/complement synapse/scripts-dev/complement.sh -p 1 -json 2>&1 | tee /tmp/gotest-complement.log
shell: bash
env:
POSTGRES: ${{ (matrix.database == 'Postgres') && 1 || '' }}
WORKERS: ${{ (matrix.arrangement == 'workers') && 1 || '' }}
- name: Formatted Complement test logs
# Always run this step if we attempted to run the Complement tests.
if: always() && steps.run_complement_tests.outcome != 'skipped'
run: cat /tmp/gotest-complement.log | gotestfmt -hide "successful-downloads,empty-packages"
- name: Run in-repo Complement Tests
id: run_in_repo_complement_tests
# -p=1: We're using `-p 1` to force the test packages to run serially as GHA boxes
# are underpowered and don't like running tons of Synapse instances at once.
# -json: Output JSON format so that gotestfmt can parse it.
#
# tee /tmp/gotest-in-repo-complement.log: We tee the output to a file so that we can re-process it
# later on for better formatting with gotestfmt. But we still want the command
# to output to the terminal as it runs so we can see what's happening in
# real-time.
run: |
set -o pipefail
COMPLEMENT_DIR=`pwd`/complement synapse/scripts-dev/complement.sh --in-repo -p 1 -json 2>&1 | tee /tmp/gotest-in-repo-complement.log
shell: bash
env:
POSTGRES: ${{ (matrix.database == 'Postgres') && 1 || '' }}
WORKERS: ${{ (matrix.arrangement == 'workers') && 1 || '' }}
- name: Formatted in-repo Complement test logs
# Always run this step if we attempted to run the Complement tests.
if: always() && steps.run_in_repo_complement_tests.outcome != 'skipped'
run: cat /tmp/gotest-in-repo-complement.log | gotestfmt -hide "successful-downloads,empty-packages"
cargo-test:
if: ${{ needs.changes.outputs.rust == 'true' }}
runs-on: ubuntu-latest
needs:
- linting-done
- changes
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
toolchain: ${{ env.RUST_VERSION }}
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- run: cargo test
# We want to ensure that the cargo benchmarks still compile, which requires a
# nightly compiler.
cargo-bench:
if: ${{ needs.changes.outputs.rust == 'true' }}
runs-on: ubuntu-latest
needs:
- linting-done
- changes
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
toolchain: nightly-2022-12-01
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- run: cargo bench --no-run
# a job which marks all the other jobs as complete, thus allowing PRs to be merged.
tests-done:
if: ${{ always() }}
needs:
- trial
- trial-olddeps
- sytest
- export-data
- portdb
- complement
- cargo-test
- cargo-bench
- linting-done
runs-on: ubuntu-latest
steps:
- uses: matrix-org/done-action@3409aa904e8a2aaf2220f09bc954d3d0b0a2ee67 # v3
with:
needs: ${{ toJSON(needs) }}
# Various bits are skipped if there was no applicable changes.
# The newsfile lint may be skipped on non PR builds.
skippable: |
trial
trial-olddeps
sytest
portdb
export-data
complement
lint-newsfile
cargo-test
cargo-bench
-14
View File
@@ -1,14 +0,0 @@
name: Move new issues into the issue triage board
on:
issues:
types: [ opened ]
jobs:
triage:
uses: matrix-org/backend-meta/.github/workflows/triage-incoming.yml@18beaf3c8e536108bd04d18e6c3dc40ba3931e28 # v2.0.3
with:
project_id: 'PVT_kwDOAIB0Bs4AFDdZ'
content_id: ${{ github.event.issue.node_id }}
secrets:
github_access_token: ${{ secrets.ELEMENT_BOT_TOKEN }}
-31
View File
@@ -1,31 +0,0 @@
name: Move labelled issues to correct projects
on:
issues:
types: [ labeled ]
jobs:
move_needs_info:
runs-on: ubuntu-latest
if: >
contains(github.event.issue.labels.*.name, 'X-Needs-Info')
permissions:
contents: read
env:
# This token must have the following scopes: ["repo:public_repo", "admin:org->read:org", "user->read:user", "project"]
GITHUB_TOKEN: ${{ secrets.ELEMENT_BOT_TOKEN }}
PROJECT_OWNER: matrix-org
# Backend issue triage board.
# https://github.com/orgs/matrix-org/projects/67/views/1
PROJECT_NUMBER: 67
ISSUE_URL: ${{ github.event.issue.html_url }}
# This field is case-sensitive.
TARGET_STATUS: Needs info
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
# Only clone the script file we care about, instead of the whole repo.
sparse-checkout: .ci/scripts/triage_labelled_issue.sh
- name: Ensure issue exists on the board, then set Status
run: .ci/scripts/triage_labelled_issue.sh
-266
View File
@@ -1,266 +0,0 @@
name: Twisted Trunk
on:
schedule:
- cron: 0 8 * * *
workflow_dispatch:
# NB: inputs are only present when this workflow is dispatched manually.
# (The default below is the default field value in the form to trigger
# a manual dispatch). Otherwise the inputs will evaluate to null.
inputs:
twisted_ref:
description: Commit, branch or tag to checkout from upstream Twisted.
required: false
default: "trunk"
type: string
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
RUST_VERSION: 1.87.0
jobs:
check_repo:
# Prevent this workflow from running on any fork of Synapse other than element-hq/synapse, as it is
# only useful to the Synapse core team.
# All other workflow steps depend on this one, thus if 'should_run_workflow' is not 'true', the rest
# of the workflow will be skipped as well.
if: github.repository == 'element-hq/synapse'
runs-on: ubuntu-latest
outputs:
should_run_workflow: ${{ steps.check_condition.outputs.should_run_workflow }}
steps:
- id: check_condition
run: echo "should_run_workflow=${{ github.repository == 'element-hq/synapse' }}" >> "$GITHUB_OUTPUT"
mypy:
needs: check_repo
if: needs.check_repo.outputs.should_run_workflow == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
toolchain: ${{ env.RUST_VERSION }}
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
with:
python-version: "3.x"
extras: "all"
poetry-version: "2.1.1"
- run: |
poetry remove twisted
poetry add --extras tls git+https://github.com/twisted/twisted.git#${{ inputs.twisted_ref || 'trunk' }}
poetry install --no-interaction --extras "all test"
- name: Remove unhelpful options from mypy config
run: sed -e '/warn_unused_ignores = True/d' -e '/warn_redundant_casts = True/d' -i mypy.ini
- run: poetry run mypy
trial:
needs: check_repo
if: needs.check_repo.outputs.should_run_workflow == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- run: sudo apt-get -qq install xmlsec1
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
toolchain: ${{ env.RUST_VERSION }}
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
with:
python-version: "3.x"
extras: "all test"
poetry-version: "2.1.1"
- run: |
poetry remove twisted
poetry add --extras tls git+https://github.com/twisted/twisted.git#trunk
poetry install --no-interaction --extras "all test"
- run: poetry run trial --jobs 2 tests
- name: Dump logs
# Logs are most useful when the command fails, always include them.
if: ${{ always() }}
# Note: Dumps to workflow logs instead of using actions/upload-artifact
# This keeps logs colocated with failing jobs
# It also ignores find's exit code; this is a best effort affair
run: >-
find _trial_temp -name '*.log'
-exec echo "::group::{}" \;
-exec cat {} \;
-exec echo "::endgroup::" \;
|| true
sytest:
needs: check_repo
if: needs.check_repo.outputs.should_run_workflow == 'true'
runs-on: ubuntu-latest
container:
# We're using bookworm because that's what Debian oldstable is at the time of writing.
# This job is a canary to warn us about unreleased twisted changes that would cause problems for us if
# they were to be released immediately. For simplicity's sake (and to save CI runners) we use the oldest
# version, assuming that any incompatibilities on newer versions would also be present on the oldest.
image: matrixdotorg/sytest-synapse:bookworm
volumes:
- ${{ github.workspace }}:/src
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
with:
toolchain: ${{ env.RUST_VERSION }}
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
- name: Patch dependencies
# Note: The poetry commands want to create a virtualenv in /src/.venv/,
# but the sytest-synapse container expects it to be in /venv/.
# We symlink it before running poetry so that poetry actually
# ends up installing to `/venv`.
run: |
ln -s -T /venv /src/.venv
poetry remove twisted
poetry add --extras tls git+https://github.com/twisted/twisted.git#trunk
poetry install --no-interaction --extras "all test"
working-directory: /src
- name: Run SyTest
run: /bootstrap.sh synapse
working-directory: /src
env:
# Use offline mode to avoid reinstalling the pinned version of
# twisted.
OFFLINE: 1
- name: Summarise results.tap
if: ${{ always() }}
run: /sytest/scripts/tap_to_gha.pl /logs/results.tap
- name: Upload SyTest logs
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
if: ${{ always() }}
with:
name: Sytest Logs - ${{ job.status }} - (${{ join(matrix.*, ', ') }})
path: |
/logs/results.tap
/logs/**/*.log*
complement:
needs: check_repo
if: "!failure() && !cancelled() && needs.check_repo.outputs.should_run_workflow == 'true'"
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- arrangement: monolith
database: SQLite
- arrangement: monolith
database: Postgres
- arrangement: workers
database: Postgres
steps:
- name: Run actions/checkout@v4 for synapse
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
path: synapse
- name: Prepare Complement's Prerequisites
run: synapse/.ci/scripts/setup_complement_prerequisites.sh
- uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0
with:
cache-dependency-path: complement/go.sum
go-version-file: complement/go.mod
# This step is specific to the 'Twisted trunk' test run:
- name: Patch dependencies
run: |
set -x
DEBIAN_FRONTEND=noninteractive sudo apt-get install -yqq python3 pipx
pipx install poetry==2.1.1
poetry remove -n twisted
poetry add -n --extras tls git+https://github.com/twisted/twisted.git#trunk
poetry lock
working-directory: synapse
- name: Run Complement Tests
id: run_complement_tests
# -p=1: We're using `-p 1` to force the test packages to run serially as GHA boxes
# are underpowered and don't like running tons of Synapse instances at once.
# -json: Output JSON format so that gotestfmt can parse it.
#
# tee /tmp/gotest-complement.log: We tee the output to a file so that we can re-process it
# later on for better formatting with gotestfmt. But we still want the command
# to output to the terminal as it runs so we can see what's happening in
# real-time.
run: |
set -o pipefail
COMPLEMENT_DIR=`pwd`/complement synapse/scripts-dev/complement.sh -p 1 -json 2>&1 | tee /tmp/gotest-complement.log
shell: bash
env:
POSTGRES: ${{ (matrix.database == 'Postgres') && 1 || '' }}
WORKERS: ${{ (matrix.arrangement == 'workers') && 1 || '' }}
TEST_ONLY_SKIP_DEP_HASH_VERIFICATION: 1
- name: Formatted Complement test logs
# Always run this step if we attempted to run the Complement tests.
if: always() && steps.run_complement_tests.outcome != 'skipped'
run: cat /tmp/gotest-complement.log | gotestfmt -hide "successful-downloads,empty-packages"
- name: Run in-repo Complement Tests
id: run_in_repo_complement_tests
# -p=1: We're using `-p 1` to force the test packages to run serially as GHA boxes
# are underpowered and don't like running tons of Synapse instances at once.
# -json: Output JSON format so that gotestfmt can parse it.
#
# tee /tmp/gotest-in-repo-complement.log: We tee the output to a file so that we can re-process it
# later on for better formatting with gotestfmt. But we still want the command
# to output to the terminal as it runs so we can see what's happening in
# real-time.
run: |
set -o pipefail
COMPLEMENT_DIR=`pwd`/complement synapse/scripts-dev/complement.sh --in-repo -p 1 -json 2>&1 | tee /tmp/gotest-in-repo-complement.log
shell: bash
env:
POSTGRES: ${{ (matrix.database == 'Postgres') && 1 || '' }}
WORKERS: ${{ (matrix.arrangement == 'workers') && 1 || '' }}
TEST_ONLY_SKIP_DEP_HASH_VERIFICATION: 1
- name: Formatted in-repo Complement test logs
# Always run this step if we attempted to run the Complement tests.
if: always() && steps.run_in_repo_complement_tests.outcome != 'skipped'
run: cat /tmp/gotest-in-repo-complement.log | gotestfmt -hide "successful-downloads,empty-packages"
# open an issue if the build fails, so we know about it.
open-issue:
if: failure() && needs.check_repo.outputs.should_run_workflow == 'true'
needs:
- mypy
- trial
- sytest
- complement
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: JasonEtco/create-an-issue@1b14a70e4d8dc185e5cc76d3bec9eab20257b2c5 # v2.9.2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
update_existing: true
filename: .ci/twisted_trunk_build_failed_issue_template.md
+30
View File
@@ -1,3 +1,33 @@
# Synapse 1.149.0rc1 (2026-03-03)
## Features
- Add experimental support for [MSC4388: Secure out-of-band channel for sign in with QR](https://github.com/matrix-org/matrix-spec-proposals/pull/4388). ([\#19127](https://github.com/element-hq/synapse/issues/19127))
- Add stable support for [MSC4380](https://github.com/matrix-org/matrix-spec-proposals/pull/4380) invite blocking. ([\#19431](https://github.com/element-hq/synapse/issues/19431))
## Bugfixes
- Fix the 'Login as a user' Admin API not checking if the user exists before issuing an access token. ([\#18518](https://github.com/element-hq/synapse/issues/18518))
- Fix `/sync` missing membership event in `state_after` (experimental [MSC4222](https://github.com/matrix-org/matrix-spec-proposals/pull/4222) implementation) in some scenarios. ([\#19460](https://github.com/element-hq/synapse/issues/19460))
## Internal Changes
- Add log to explain when and why we freeze objects in the garbage collector. ([\#19440](https://github.com/element-hq/synapse/issues/19440))
- Better instrument `JoinRoomAliasServlet` with tracing. ([\#19461](https://github.com/element-hq/synapse/issues/19461))
- Fix Complement CI not running against the code from our PRs. ([\#19475](https://github.com/element-hq/synapse/issues/19475))
- Log `docker system info` in CI so we have a plain record of how GitHub runners evolve over time. ([\#19480](https://github.com/element-hq/synapse/issues/19480))
- Rename the `test_disconnect` test helper so that pytest doesn't see it as a test. ([\#19486](https://github.com/element-hq/synapse/issues/19486))
- Add a log line when we delete devices. Contributed by @bradtgmurray @ Beeper. ([\#19496](https://github.com/element-hq/synapse/issues/19496))
- Pre-allocate the buffer based on the expected `Content-Length` with the Rust HTTP client. ([\#19498](https://github.com/element-hq/synapse/issues/19498))
- Cancel long-running sync requests if the client has gone away. ([\#19499](https://github.com/element-hq/synapse/issues/19499))
- Try and reduce reactor tick times when under heavy load. ([\#19507](https://github.com/element-hq/synapse/issues/19507))
- Simplify Rust HTTP client response streaming and limiting. ([\#19510](https://github.com/element-hq/synapse/issues/19510))
- Replace deprecated collection import locations with current locations. ([\#19515](https://github.com/element-hq/synapse/issues/19515))
- Bump most locked Python dependencies to their latest versions. ([\#19519](https://github.com/element-hq/synapse/issues/19519))
# Synapse 1.148.0 (2026-02-24)
No significant changes since 1.148.0rc1.
Generated
+23 -29
View File
@@ -13,9 +13,9 @@ dependencies = [
[[package]]
name = "anyhow"
version = "1.0.100"
version = "1.0.101"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a23eb6b1614318a8071c9b2521f36b424b2c83db5eb3a0fead4a6c0809af6e61"
checksum = "5f0e0fee31ef5ed1ba1316088939cea399010ed7731dba877ed44aeb407a75ea"
[[package]]
name = "arc-swap"
@@ -187,9 +187,9 @@ dependencies = [
[[package]]
name = "futures"
version = "0.3.31"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "65bc07b1a8bc7c85c5f2e110c476c7389b4554ba72af57d8445ea63a576b0876"
checksum = "8b147ee9d1f6d097cef9ce628cd2ee62288d963e16fb287bd9286455b241382d"
dependencies = [
"futures-channel",
"futures-core",
@@ -202,9 +202,9 @@ dependencies = [
[[package]]
name = "futures-channel"
version = "0.3.31"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2dff15bf788c671c1934e366d07e30c1814a8ef514e1af724a602e8a2fbe1b10"
checksum = "07bbe89c50d7a535e539b8c17bc0b49bdb77747034daa8087407d655f3f7cc1d"
dependencies = [
"futures-core",
"futures-sink",
@@ -212,15 +212,15 @@ dependencies = [
[[package]]
name = "futures-core"
version = "0.3.31"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05f29059c0c2090612e8d742178b0580d2dc940c837851ad723096f87af6663e"
checksum = "7e3450815272ef58cec6d564423f6e755e25379b217b0bc688e295ba24df6b1d"
[[package]]
name = "futures-executor"
version = "0.3.31"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e28d1d997f585e54aebc3f97d39e72338912123a67330d723fdbb564d646c9f"
checksum = "baf29c38818342a3b26b5b923639e7b1f4a61fc5e76102d4b1981c6dc7a7579d"
dependencies = [
"futures-core",
"futures-task",
@@ -229,15 +229,15 @@ dependencies = [
[[package]]
name = "futures-io"
version = "0.3.31"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9e5c1b78ca4aae1ac06c48a526a655760685149f0d465d21f37abfe57ce075c6"
checksum = "cecba35d7ad927e23624b22ad55235f2239cfa44fd10428eecbeba6d6a717718"
[[package]]
name = "futures-macro"
version = "0.3.31"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "162ee34ebcb7c64a8abebc059ce0fee27c2262618d7b60ed8faf72fef13c3650"
checksum = "e835b70203e41293343137df5c0664546da5745f82ec9b84d40be8336958447b"
dependencies = [
"proc-macro2",
"quote",
@@ -246,21 +246,21 @@ dependencies = [
[[package]]
name = "futures-sink"
version = "0.3.31"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e575fab7d1e0dcb8d0c7bcf9a63ee213816ab51902e6d244a95819acacf1d4f7"
checksum = "c39754e157331b013978ec91992bde1ac089843443c49cbc7f46150b0fad0893"
[[package]]
name = "futures-task"
version = "0.3.31"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f90f7dce0722e95104fcb095585910c0977252f286e354b5e3bd38902cd99988"
checksum = "037711b3d59c33004d3856fbdc83b99d4ff37a24768fa1be9ce3538a1cde4393"
[[package]]
name = "futures-util"
version = "0.3.31"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9fa08315bb612088cc391249efdc3bc77536f16c91f6cf495e6fbe85b20a4a81"
checksum = "389ca41296e6190b48053de0321d02a77f32f8a5d2461dd38762c0593805c6d6"
dependencies = [
"futures-channel",
"futures-core",
@@ -270,7 +270,6 @@ dependencies = [
"futures-task",
"memchr",
"pin-project-lite",
"pin-utils",
"slab",
]
@@ -771,12 +770,6 @@ version = "0.2.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3b3cff922bd51709b605d9ead9aa71031d81447142d828eb4a6eba76fe619f9b"
[[package]]
name = "pin-utils"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184"
[[package]]
name = "portable-atomic"
version = "1.11.1"
@@ -818,6 +811,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ab53c047fcd1a1d2a8820fe84f05d6be69e9526be40cb03b73f86b6b03e6d87d"
dependencies = [
"anyhow",
"bytes",
"indoc",
"libc",
"memoffset",
@@ -995,9 +989,9 @@ dependencies = [
[[package]]
name = "regex"
version = "1.12.2"
version = "1.12.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "843bc0191f75f3e22651ae5f1e72939ab2f72a4bc30fa80a066bd66edefc24d4"
checksum = "e10754a14b9137dd7b1e3e5b0493cc9171fdd105e0ab477f51b72e7f3ac0e276"
dependencies = [
"aho-corasick",
"memchr",
+1
View File
@@ -0,0 +1 @@
Update docs to clarify `outbound_federation_restricted_to` can also be used with the [Secure Border Gateway (SBG)](https://element.io/en/server-suite/secure-border-gateways).
+1
View File
@@ -0,0 +1 @@
Unify Complement developer docs.
+45 -4
View File
@@ -8,8 +8,7 @@ ensure everything works at a holistic level.
## Setup
Nothing beyond a [normal Complement
setup](https://github.com/matrix-org/complement?tab=readme-ov-file#running) (just Go and
Docker).
setup](https://github.com/matrix-org/complement#running) (just Go and Docker).
## Running tests
@@ -28,14 +27,39 @@ scripts-dev/complement.sh ./tests/csapi/... -run TestRoomCreate/Parallel/POST_/c
scripts-dev/complement.sh ./tests/... -run 'TestRoomCreate/Parallel/POST_/createRoom_makes_a_(.*)'
```
Typically, if you're developing the Synapse and Complement tests side-by-side, you will
run something like this:
It's often nice to develop on Synapse and write Complement tests at the same time.
Here is how to run your local Synapse checkout against your local Complement checkout.
```shell
# To run a specific test
COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh ./tests/csapi/... -run TestRoomCreate
```
The above will run a monolithic (single-process) Synapse with SQLite as the database.
For other configurations, try:
- Passing `POSTGRES=1` as an environment variable to use the Postgres database instead.
- Passing `WORKERS=1` as an environment variable to use a workerised setup instead. This
option implies the use of Postgres.
- If setting `WORKERS=1`, optionally set `WORKER_TYPES=` to declare which worker types
you wish to test. A simple comma-delimited string containing the worker types
defined from the `WORKERS_CONFIG` template in
[here](https://github.com/element-hq/synapse/blob/develop/docker/configure_workers_and_start.py#L54).
A safe example would be `WORKER_TYPES="federation_inbound, federation_sender,
synchrotron"`. See the [worker documentation](../workers.md) for additional
information on workers.
- Passing `ASYNCIO_REACTOR=1` as an environment variable to use the asyncio-backed
reactor with Twisted instead of the default one.
- Passing `PODMAN=1` will use the [podman](https://podman.io/) container runtime,
instead of docker.
- Passing `UNIX_SOCKETS=1` will utilise Unix socket functionality for Synapse, Redis,
and Postgres(when applicable).
To increase the log level for the tests, set `SYNAPSE_TEST_LOG_LEVEL`, e.g:
```sh
SYNAPSE_TEST_LOG_LEVEL=DEBUG COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh -run TestRoomCreate
```
### Running in-repo tests
@@ -52,3 +76,20 @@ To run the in-repo Complement tests, use the `--in-repo` command line argument.
# Similarly, you can also use `-run` to specify all or part of a specific test path to run
scripts-dev/complement.sh --in-repo ./tests/... -run TestIntraShardFederation
```
### Access database for homeserver after Complement test runs.
If you're curious what the database looks like after you run some tests, here are some
steps to get you going in Synapse:
1. In your Complement test comment out `defer deployment.Destroy(t)` and replace with
`defer time.Sleep(2 * time.Hour)` to keep the homeserver running after the tests
complete
1. Start the Complement tests
1. Find the name of the container, `docker ps -f name=complement_` (this will filter for
just the Complement related Docker containers)
1. Access the container replacing the name with what you found in the previous step:
`docker exec -it complement_1_hs_with_application_service.hs1_2 /bin/bash`
1. Install sqlite (database driver), `apt-get update && apt-get install -y sqlite3`
1. Then run `sqlite3` and open the database `.open /conf/homeserver.db` (this db path
comes from the Synapse homeserver.yaml)
+6
View File
@@ -1,3 +1,9 @@
matrix-synapse-py3 (1.149.0~rc1) stable; urgency=medium
* New synapse release 1.149.0rc1.
-- Synapse Packaging team <packages@matrix.org> Tue, 03 Mar 2026 14:37:57 +0000
matrix-synapse-py3 (1.148.0) stable; urgency=medium
* New synapse release 1.148.0.
+5 -7
View File
@@ -12,11 +12,9 @@ Note that running Synapse's unit tests from within the docker image is not suppo
`scripts-dev/complement.sh` is a script that will automatically build
and run Synapse against Complement.
Consult the [contributing guide][guideComplementSh] for instructions on how to use it.
Consult our [Complement docs][https://github.com/element-hq/synapse/tree/develop/complement] for instructions on how to use it.
[guideComplementSh]: https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-integration-tests-complement
## Building and running the images manually
Under some circumstances, you may wish to build the images manually.
@@ -31,23 +29,23 @@ release of Synapse, instead of your current checkout, you can skip this step. Fr
root of the repository:
```sh
docker build -t matrixdotorg/synapse -f docker/Dockerfile .
docker build -t localhost/synapse -f docker/Dockerfile .
```
Next, build the workerised Synapse docker image, which is a layer over the base
image.
```sh
docker build -t matrixdotorg/synapse-workers -f docker/Dockerfile-workers .
docker build -t localhost/synapse-workers --build-arg FROM=localhost/synapse -f docker/Dockerfile-workers .
```
Finally, build the multi-purpose image for Complement, which is a layer over the workers image.
```sh
docker build -t complement-synapse -f docker/complement/Dockerfile docker/complement
docker build -t localhost/complement-synapse -f docker/complement/Dockerfile --build-arg FROM=localhost/synapse-workers docker/complement
```
This will build an image with the tag `complement-synapse`, which can be handed to
This will build an image with the tag `localhost/complement-synapse`, which can be handed to
Complement for testing via the `COMPLEMENT_BASE_IMAGE` environment variable. Refer to
[Complement's documentation](https://github.com/matrix-org/complement/#running) for
how to run the tests, as well as the various available command line flags.
@@ -141,6 +141,8 @@ experimental_features:
msc4306_enabled: true
# Sticky Events
msc4354_enabled: true
# `/sync` `state_after`
msc4222_enabled: true
server_notices:
system_mxid_localpart: _server
+2 -51
View File
@@ -334,46 +334,9 @@ For more details about other configurations, see the [Docker-specific documentat
## Run the integration tests ([Complement](https://github.com/matrix-org/complement)).
[Complement](https://github.com/matrix-org/complement) is a suite of black box tests that can be run on any homeserver implementation. It can also be thought of as end-to-end (e2e) tests.
See our [Complement docs](https://github.com/element-hq/synapse/tree/develop/complement)
for how to use the `./scripts-dev/complement.sh` test runner script.
It's often nice to develop on Synapse and write Complement tests at the same time.
Here is how to run your local Synapse checkout against your local Complement checkout.
(checkout [`complement`](https://github.com/matrix-org/complement) alongside your `synapse` checkout)
```sh
COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh
```
To run a specific test file, you can pass the test name at the end of the command. The name passed comes from the naming structure in your Complement tests. If you're unsure of the name, you can do a full run and copy it from the test output:
```sh
COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh -run TestImportHistoricalMessages
```
To run a specific test, you can specify the whole name structure:
```sh
COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh -run TestImportHistoricalMessages/parallel/Historical_events_resolve_in_the_correct_order
```
The above will run a monolithic (single-process) Synapse with SQLite as the database. For other configurations, try:
- Passing `POSTGRES=1` as an environment variable to use the Postgres database instead.
- Passing `WORKERS=1` as an environment variable to use a workerised setup instead. This option implies the use of Postgres.
- If setting `WORKERS=1`, optionally set `WORKER_TYPES=` to declare which worker
types you wish to test. A simple comma-delimited string containing the worker types
defined from the `WORKERS_CONFIG` template in
[here](https://github.com/element-hq/synapse/blob/develop/docker/configure_workers_and_start.py#L54).
A safe example would be `WORKER_TYPES="federation_inbound, federation_sender, synchrotron"`.
See the [worker documentation](../workers.md) for additional information on workers.
- Passing `ASYNCIO_REACTOR=1` as an environment variable to use the Twisted asyncio reactor instead of the default one.
- Passing `PODMAN=1` will use the [podman](https://podman.io/) container runtime, instead of docker.
- Passing `UNIX_SOCKETS=1` will utilise Unix socket functionality for Synapse, Redis, and Postgres(when applicable).
To increase the log level for the tests, set `SYNAPSE_TEST_LOG_LEVEL`, e.g:
```sh
SYNAPSE_TEST_LOG_LEVEL=DEBUG COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh -run TestImportHistoricalMessages
```
### Prettier formatting with `gotestfmt`
@@ -389,18 +352,6 @@ COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh -json | gotestfmt -hide
(Remove `-hide successful-tests` if you don't want to hide successful tests.)
### Access database for homeserver after Complement test runs.
If you're curious what the database looks like after you run some tests, here are some steps to get you going in Synapse:
1. In your Complement test comment out `defer deployment.Destroy(t)` and replace with `defer time.Sleep(2 * time.Hour)` to keep the homeserver running after the tests complete
1. Start the Complement tests
1. Find the name of the container, `docker ps -f name=complement_` (this will filter for just the Compelement related Docker containers)
1. Access the container replacing the name with what you found in the previous step: `docker exec -it complement_1_hs_with_application_service.hs1_2 /bin/bash`
1. Install sqlite (database driver), `apt-get update && apt-get install -y sqlite3`
1. Then run `sqlite3` and open the database `.open /conf/homeserver.db` (this db path comes from the Synapse homeserver.yaml)
# 9. Submit your patch.
Once you're happy with your patch, it's time to prepare a Pull Request.
@@ -4484,7 +4484,7 @@ stream_writers:
---
### `outbound_federation_restricted_to`
*(array)* When using workers, you can restrict outbound federation traffic to only go through a specific subset of workers. Any worker specified here must also be in the [`instance_map`](#instance_map). [`worker_replication_secret`](#worker_replication_secret) must also be configured to authorize inter-worker communication.
*(array)* You can restrict outbound federation traffic to only go through a specific subset of workers including the [Secure Border Gateway (SBG)](https://element.io/en/server-suite/secure-border-gateways). Any worker specified here (including the SBG) must also be in the [`instance_map`](#instance_map). [`worker_replication_secret`](#worker_replication_secret) must also be configured to authorize inter-worker communication.
Also see the [worker documentation](../../workers.md#restrict-outbound-federation-traffic-to-a-specific-set-of-workers) for more info.
Generated
+1004 -788
View File
File diff suppressed because it is too large Load Diff
+7 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "matrix-synapse"
version = "1.148.0"
version = "1.149.0rc1"
description = "Homeserver for the Matrix decentralised comms protocol"
readme = "README.rst"
authors = [
@@ -411,6 +411,12 @@ indent-style = "space"
skip-magic-trailing-comma = false
line-ending = "auto"
[tool.ruff.lint.flake8-bugbear]
extend-immutable-calls = [
# Durations are immutable
"synapse.util.duration.Duration",
]
[tool.maturin]
manifest-path = "rust/Cargo.toml"
module-name = "synapse.synapse_rust"
+3
View File
@@ -35,6 +35,9 @@ pyo3 = { version = "0.27.2", features = [
"anyhow",
"abi3",
"abi3-py310",
# So we can pass `bytes::Bytes` directly back to Python efficiently,
# https://docs.rs/pyo3/latest/pyo3/bytes/index.html
"bytes",
] }
pyo3-log = "0.13.1"
pythonize = "0.27.0"
+1 -1
View File
@@ -62,7 +62,7 @@ impl NotFoundError {
import_exception!(synapse.api.errors, HttpResponseException);
impl HttpResponseException {
pub fn new(status: StatusCode, bytes: Vec<u8>) -> pyo3::PyErr {
pub fn new(status: StatusCode, bytes: bytes::Bytes) -> pyo3::PyErr {
HttpResponseException::new_err((
status.as_u16(),
status.canonical_reason().unwrap_or_default(),
+21 -14
View File
@@ -15,7 +15,7 @@
use std::{collections::HashMap, future::Future, sync::OnceLock};
use anyhow::Context;
use futures::TryStreamExt;
use http_body_util::BodyExt;
use once_cell::sync::OnceCell;
use pyo3::{create_exception, exceptions::PyException, prelude::*};
use reqwest::RequestBuilder;
@@ -235,23 +235,30 @@ impl HttpClient {
let status = response.status();
let mut stream = response.bytes_stream();
let mut buffer = Vec::new();
while let Some(chunk) = stream.try_next().await.context("reading body")? {
if buffer.len() + chunk.len() > response_limit {
Err(anyhow::anyhow!("Response size too large"))?;
}
buffer.extend_from_slice(&chunk);
}
// A light-weight way to read the response up until the `response_limit`. We
// want to avoid allocating a giant response object on the server above our
// expected `response_limit` to avoid out-of-memory DOS problems.
let body = reqwest::Body::from(response);
let limited_body = http_body_util::Limited::new(body, response_limit);
let collected = limited_body
.collect()
.await
.map_err(anyhow::Error::from_boxed)
.with_context(|| {
format!(
"Response body exceeded response limit ({} bytes)",
response_limit
)
})?;
let bytes: bytes::Bytes = collected.to_bytes();
if !status.is_success() {
return Err(HttpResponseException::new(status, buffer));
return Err(HttpResponseException::new(status, bytes));
}
let r = Python::attach(|py| buffer.into_pyobject(py).map(|o| o.unbind()))?;
Ok(r)
// Because of the `pyo3` `bytes` feature, we can pass this back to Python
// land efficiently
Ok(bytes)
})
}
}
+2
View File
@@ -12,6 +12,7 @@ pub mod http;
pub mod http_client;
pub mod identifier;
pub mod matrix_const;
pub mod msc4388_rendezvous;
pub mod push;
pub mod rendezvous;
pub mod segmenter;
@@ -55,6 +56,7 @@ fn synapse_rust(py: Python<'_>, m: &Bound<'_, PyModule>) -> PyResult<()> {
events::register_module(py, m)?;
http_client::register_module(py, m)?;
rendezvous::register_module(py, m)?;
msc4388_rendezvous::register_module(py, m)?;
segmenter::register_module(py, m)?;
Ok(())
+370
View File
@@ -0,0 +1,370 @@
/*
* This file is licensed under the Affero General Public License (AGPL) version 3.
*
* Copyright (C) 2026 Element Creations Ltd
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* See the GNU Affero General Public License for more details:
* <https://www.gnu.org/licenses/agpl-3.0.html>.
*/
use std::{
collections::BTreeMap,
time::{Duration, SystemTime},
};
use http::StatusCode;
use pyo3::{
pyclass, pymethods,
types::{PyAnyMethods, PyModule, PyModuleMethods},
Bound, IntoPyObject, Py, PyAny, PyResult, Python,
};
use serde::Deserialize;
use ulid::Ulid;
use self::session::Session;
use crate::{
duration::SynapseDuration,
errors::{NotFoundError, SynapseError},
http::http_request_from_twisted,
msc4388_rendezvous::session::{GetResponse, PostResponse, PutResponse},
UnwrapInfallible,
};
mod session;
#[pyclass]
struct MSC4388RendezvousHandler {
clock: Py<PyAny>,
sessions: BTreeMap<Ulid, Session>,
soft_limit: usize,
hard_limit: usize,
max_content_length: u64,
ttl: Duration,
}
impl MSC4388RendezvousHandler {
/// Check the length of the data parameter and throw error if invalid.
fn check_data_length(&self, data: &str) -> PyResult<()> {
let data_length = data.len() as u64;
if data_length > self.max_content_length {
return Err(SynapseError::new(
StatusCode::PAYLOAD_TOO_LARGE,
"Payload too large".to_owned(),
"M_TOO_LARGE",
None,
None,
));
}
Ok(())
}
/// Evict expired sessions and remove the oldest sessions until we're under the capacity.
fn evict(&mut self, now: SystemTime) {
// First remove all the entries which expired
self.sessions.retain(|_, session| !session.expired(now));
// Then we remove the oldest entries until we're under the soft limit
while self.sessions.len() > self.soft_limit {
self.sessions.pop_first();
}
}
}
#[derive(Deserialize)]
pub struct PostRequest {
data: String,
}
#[derive(Deserialize)]
pub struct PutRequest {
sequence_token: String,
data: String,
}
#[pymethods]
impl MSC4388RendezvousHandler {
#[new]
#[pyo3(signature = (homeserver, /, soft_limit=100, hard_limit=200,max_content_length=4*1024, eviction_interval=60*1000, ttl=2*60*1000))]
fn new(
py: Python<'_>,
homeserver: &Bound<'_, PyAny>,
soft_limit: usize,
hard_limit: usize,
max_content_length: u64,
eviction_interval: u64,
ttl: u64,
) -> PyResult<Py<Self>> {
let clock = homeserver
.call_method0("get_clock")?
.into_pyobject(py)
.unwrap_infallible()
.unbind();
// Construct a Python object so that we can get a reference to the
// evict method and schedule it to run.
let self_ = Py::new(
py,
Self {
clock,
sessions: BTreeMap::new(),
soft_limit,
hard_limit,
max_content_length,
ttl: Duration::from_millis(ttl),
},
)?;
let eviction_duration = SynapseDuration::from_milliseconds(eviction_interval);
let evict = self_.getattr(py, "_evict")?;
homeserver.call_method0("get_clock")?.call_method(
"looping_call",
(evict, &eviction_duration),
None,
)?;
Ok(self_)
}
fn _evict(&mut self, py: Python<'_>) -> PyResult<()> {
let clock = self.clock.bind(py);
let now: u64 = clock.call_method0("time_msec")?.extract()?;
let now = SystemTime::UNIX_EPOCH + Duration::from_millis(now);
self.evict(now);
Ok(())
}
fn handle_post(
&mut self,
py: Python<'_>,
twisted_request: &Bound<'_, PyAny>,
) -> PyResult<(u8, PostResponse)> {
let clock = self.clock.bind(py);
let now: u64 = clock.call_method0("time_msec")?.extract()?;
let now = SystemTime::UNIX_EPOCH + Duration::from_millis(now);
// We trigger an immediate eviction if we're at the hard limit
if self.sessions.len() >= self.hard_limit {
self.evict(now);
}
// Generate a new ULID for the session from the current time.
let id = Ulid::from_datetime(now);
let request = http_request_from_twisted(twisted_request)?;
// parse JSON body
let post_request: PostRequest =
serde_json::from_slice(&request.into_body()).map_err(|_| {
SynapseError::new(
StatusCode::BAD_REQUEST,
"Invalid JSON in request body".to_owned(),
"M_INVALID_PARAM",
None,
None,
)
})?;
let data: String = post_request.data;
self.check_data_length(&data)?;
let session = Session::new(id, data, now, self.ttl);
let response = session.post_response(now);
self.sessions.insert(id, session);
Ok((200, response))
}
fn handle_get(
&mut self,
py: Python<'_>,
id: &str,
twisted_request: &Bound<'_, PyAny>,
) -> PyResult<(u8, GetResponse)> {
let request = http_request_from_twisted(twisted_request)?;
// As per the MSC, we check the Sec-Fetch-* headers to ensure this request did not come from somewhere that will
// be rendered directly to the user, as the response may contain sensitive data. These headers are added by
// well behaved browsers so are helpful for protecting regular users.
// Sec-Fetch-Dest: https://www.w3.org/TR/fetch-metadata/#sec-fetch-dest-header
//
// If the header is present then this must be "empty". All other values such as document, image etc.
// are considered potentially dangerous as they might be rendered to the user.
//
// Note that because we only ever return JSON, so it is unlikely that it could somehow be rendered as an image,
// video or other media.
let sec_fetch_dest: Option<String> = request
.headers()
.get("sec-fetch-dest")
.and_then(|v| v.to_str().ok())
.map(|s| s.to_owned());
if sec_fetch_dest.is_some() && sec_fetch_dest.as_deref() != Some("empty") {
return Err(SynapseError::new(
StatusCode::FORBIDDEN,
"Rendezvous content is not accessible from the request destination".to_owned(),
"M_FORBIDDEN",
None,
None,
));
}
// Sec-Fetch-Mode: https://www.w3.org/TR/fetch-metadata/#sec-fetch-mode-header
//
// A request mode of "navigate" is not allowed as this indicates the request is being made by the
// browser to navigate to a URL, which could lead to the response being rendered directly to the user.
//
// Note that usually Sec-Fetch-Dest would be "document" in this case and so the request would be rejected earlier,
// but we check the mode just in case the destination is not set correctly.
let sec_fetch_mode: Option<String> = request
.headers()
.get("sec-fetch-mode")
.and_then(|v| v.to_str().ok())
.map(|s| s.to_owned());
if sec_fetch_mode.as_deref() == Some("navigate") {
return Err(SynapseError::new(
StatusCode::FORBIDDEN,
"Rendezvous content is not accessible via top-level navigation".to_owned(),
"M_FORBIDDEN",
None,
None,
));
}
// Sec-Fetch-User: https://www.w3.org/TR/fetch-metadata/#sec-fetch-user-header
//
// If the request has a Sec-Fetch-User header with a value of "?1", this indicates that the
// request was triggered by user activation, such as a click.
//
// Note that usually Sec-Fetch-Mode would be "navigate" or the Sec-Fetch-Dest would be "document" in this case
// and so the request would be rejected earlier, but we check the user activation just in case those headers are
// not set correctly.
let sec_fetch_user: Option<String> = request
.headers()
.get("sec-fetch-user")
.and_then(|v| v.to_str().ok())
.map(|s| s.to_owned());
if sec_fetch_user.as_deref() == Some("?1") {
return Err(SynapseError::new(
StatusCode::FORBIDDEN,
"Rendezvous content is not accessible from requests with user activation"
.to_owned(),
"M_FORBIDDEN",
None,
None,
));
}
// Sec-Fetch-Site: https://www.w3.org/TR/fetch-metadata/#sec-fetch-site-header
//
// "none" indicates the request did not originate from a web page
// (e.g. typed URL, bookmark, or browser extension), so we disallow it.
let sec_fetch_site: Option<String> = request
.headers()
.get("sec-fetch-site")
.and_then(|v| v.to_str().ok())
.map(|s| s.to_owned());
if sec_fetch_site.as_deref() == Some("none") {
return Err(SynapseError::new(
StatusCode::FORBIDDEN,
"Rendezvous content is not accessible from requests from user interaction"
.to_owned(),
"M_FORBIDDEN",
None,
None,
));
}
let clock = self.clock.bind(py);
let now: u64 = clock.call_method0("time_msec")?.extract()?;
let now = SystemTime::UNIX_EPOCH + Duration::from_millis(now);
let id: Ulid = id.parse().map_err(|_| NotFoundError::new())?;
let session = self
.sessions
.get(&id)
.filter(|s| !s.expired(now))
.ok_or_else(NotFoundError::new)?;
Ok((200, session.get_response(now)))
}
fn handle_put(
&mut self,
py: Python<'_>,
id: &str,
twisted_request: &Bound<'_, PyAny>,
) -> PyResult<(u8, PutResponse)> {
let request = http_request_from_twisted(twisted_request)?;
// parse JSON body
let put_request: PutRequest =
serde_json::from_slice(&request.into_body()).map_err(|_| {
SynapseError::new(
StatusCode::BAD_REQUEST,
"Invalid JSON in request body".to_owned(),
"M_INVALID_PARAM",
None,
None,
)
})?;
let sequence_token: String = put_request.sequence_token;
let data: String = put_request.data;
self.check_data_length(&data)?;
let clock = self.clock.bind(py);
let now: u64 = clock.call_method0("time_msec")?.extract()?;
let now = SystemTime::UNIX_EPOCH + Duration::from_millis(now);
let id: Ulid = id.parse().map_err(|_| NotFoundError::new())?;
let session = self
.sessions
.get_mut(&id)
.filter(|s| !s.expired(now))
.ok_or_else(NotFoundError::new)?;
if !session.sequence_token().eq(&sequence_token) {
return Err(SynapseError::new(
StatusCode::CONFLICT,
"sequence_token does not match".to_owned(),
"IO_ELEMENT_MSC4388_CONCURRENT_WRITE",
None,
None,
));
}
session.update(data, now);
Ok((200, session.put_response()))
}
fn handle_delete(&mut self, id: &str) -> PyResult<(u8, ())> {
let id: Ulid = id.parse().map_err(|_| NotFoundError::new())?;
let _session = self.sessions.remove(&id).ok_or_else(NotFoundError::new)?;
Ok((200, ()))
}
}
pub fn register_module(py: Python<'_>, m: &Bound<'_, PyModule>) -> PyResult<()> {
let child_module = PyModule::new(py, "msc4388_rendezvous")?;
child_module.add_class::<MSC4388RendezvousHandler>()?;
m.add_submodule(&child_module)?;
// We need to manually add the module to sys.modules to make `from
// synapse.synapse_rust import rendezvous` work.
py.import("sys")?
.getattr("modules")?
.set_item("synapse.synapse_rust.msc4388_rendezvous", child_module)?;
Ok(())
}
+153
View File
@@ -0,0 +1,153 @@
/*
* This file is licensed under the Affero General Public License (AGPL) version 3.
*
* Copyright (C) 2026 Element Creations Ltd
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* See the GNU Affero General Public License for more details:
* <https://www.gnu.org/licenses/agpl-3.0.html>.
*/
use std::time::{Duration, SystemTime};
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine as _};
use pyo3::{Bound, IntoPyObject, PyAny, Python};
use pythonize::{pythonize, PythonizeError};
use serde::Serialize;
use sha2::{Digest, Sha256};
use ulid::Ulid;
/// A single session, containing data, metadata, and expiry information.
pub struct Session {
id: Ulid,
hash: [u8; 32],
data: String,
last_modified: SystemTime,
expires: SystemTime,
}
#[derive(Serialize)]
pub struct PostResponse {
id: String,
sequence_token: String,
expires_in_ms: u64,
}
impl<'source> IntoPyObject<'source> for PostResponse {
type Target = PyAny;
type Output = Bound<'source, Self::Target>;
type Error = PythonizeError;
fn into_pyobject(self, py: Python<'source>) -> Result<Self::Output, Self::Error> {
pythonize(py, &self)
}
}
#[derive(Serialize)]
pub struct GetResponse {
data: String,
sequence_token: String,
expires_in_ms: u64,
}
impl<'source> IntoPyObject<'source> for GetResponse {
type Target = PyAny;
type Output = Bound<'source, Self::Target>;
type Error = PythonizeError;
fn into_pyobject(self, py: Python<'source>) -> Result<Self::Output, Self::Error> {
pythonize(py, &self)
}
}
#[derive(Serialize)]
pub struct PutResponse {
sequence_token: String,
}
impl<'source> IntoPyObject<'source> for PutResponse {
type Target = PyAny;
type Output = Bound<'source, Self::Target>;
type Error = PythonizeError;
fn into_pyobject(self, py: Python<'source>) -> Result<Self::Output, Self::Error> {
pythonize(py, &self)
}
}
impl Session {
/// Create a new session with the given data and time-to-live.
pub fn new(id: Ulid, data: String, now: SystemTime, ttl: Duration) -> Self {
let hash = Self::compute_hash(&data, now);
Self {
id,
hash,
data,
expires: now + ttl,
last_modified: now,
}
}
/// Returns true if the session has expired at the given time.
pub fn expired(&self, now: SystemTime) -> bool {
self.expires <= now
}
/// Update the session with new data and last modified time.
pub fn update(&mut self, data: String, now: SystemTime) {
self.hash = Self::compute_hash(&data, now);
self.data = data;
self.last_modified = now;
}
/// Compute the hash of the data and timestamp.
fn compute_hash(data: &str, now: SystemTime) -> [u8; 32] {
let mut hasher = Sha256::new();
hasher.update(data);
let now_millis = now
.duration_since(SystemTime::UNIX_EPOCH)
.unwrap_or_default()
.as_millis();
hasher.update(now_millis.to_be_bytes());
hasher.finalize().into()
}
/// The sequence token for the session.
pub fn sequence_token(&self) -> String {
URL_SAFE_NO_PAD.encode(self.hash)
}
pub fn get_response(&self, now: SystemTime) -> GetResponse {
GetResponse {
data: self.data.clone(),
sequence_token: self.sequence_token(),
expires_in_ms: self
.expires
.duration_since(now)
.unwrap_or_default()
.as_millis() as u64,
}
}
pub fn post_response(&self, now: SystemTime) -> PostResponse {
PostResponse {
id: self.id.to_string(),
sequence_token: self.sequence_token(),
expires_in_ms: self
.expires
.duration_since(now)
.unwrap_or_default()
.as_millis() as u64,
}
}
pub fn put_response(&self) -> PutResponse {
PutResponse {
sequence_token: self.sequence_token(),
}
}
}
+8 -6
View File
@@ -1,5 +1,5 @@
$schema: https://element-hq.github.io/synapse/latest/schema/v1/meta.schema.json
$id: https://element-hq.github.io/synapse/schema/synapse/v1.148/synapse-config.schema.json
$id: https://element-hq.github.io/synapse/schema/synapse/v1.149/synapse-config.schema.json
type: object
properties:
modules:
@@ -5529,11 +5529,13 @@ properties:
outbound_federation_restricted_to:
type: array
description: >-
When using workers, you can restrict outbound federation traffic to only
go through a specific subset of workers. Any worker specified here must
also be in the [`instance_map`](#instance_map).
[`worker_replication_secret`](#worker_replication_secret) must also be
configured to authorize inter-worker communication.
You can restrict outbound federation traffic to only go through a specific subset
of workers including the [Secure Border Gateway
(SBG)](https://element.io/en/server-suite/secure-border-gateways). Any worker
specified here (including the SBG) must also be in the
[`instance_map`](#instance_map).
[`worker_replication_secret`](#worker_replication_secret) must also be configured
to authorize inter-worker communication.
Also see the [worker
+49 -23
View File
@@ -35,6 +35,26 @@
# Exit if a line returns a non-zero exit code
set -e
# Tag local builds with a dummy registry namespace so that later builds may reference
# them exactly instead of accidentally pulling from a remote registry.
#
# This is important as some storage drivers/types prefer remote images over local
# (`containerd`) which causes problems as we're testing against some remote image that
# doesn't include all of the changes that we're trying to test (be it locally or in a PR
# in CI). This is spawning from a real-world problem where the GitHub runners were
# updated to use Docker Engine 29.0.0+ which uses `containerd` by default for new
# installations.
LOCAL_IMAGE_NAMESPACE=localhost
# The image tags for how these images will be stored in the registry
SYNAPSE_IMAGE_PATH="$LOCAL_IMAGE_NAMESPACE/synapse"
SYNAPSE_WORKERS_IMAGE_PATH="$LOCAL_IMAGE_NAMESPACE/synapse-workers"
COMPLEMENT_SYNAPSE_IMAGE_PATH="$LOCAL_IMAGE_NAMESPACE/complement-synapse"
SYNAPSE_EDITABLE_IMAGE_PATH="$LOCAL_IMAGE_NAMESPACE/synapse-editable"
SYNAPSE_WORKERS_EDITABLE_IMAGE_PATH="$LOCAL_IMAGE_NAMESPACE/synapse-workers-editable"
COMPLEMENT_SYNAPSE_EDITABLE_IMAGE_PATH="$LOCAL_IMAGE_NAMESPACE/complement-synapse-editable"
# Helper to emit annotations that collapse portions of the log in GitHub Actions
echo_if_github() {
if [[ -n "$GITHUB_WORKFLOW" ]]; then
@@ -53,7 +73,7 @@ Run the complement test suite on Synapse.
-f, --fast
Skip rebuilding the docker images, and just use the most recent
'complement-synapse:latest' image.
'localhost/complement-synapse:latest' image.
Conflicts with --build-only.
--build-only
@@ -154,16 +174,16 @@ main() {
editable_mount="$(realpath .):/editable-src:z"
if [ -n "$rebuild_editable_synapse" ]; then
unset skip_docker_build
elif $CONTAINER_RUNTIME inspect complement-synapse-editable &>/dev/null; then
elif $CONTAINER_RUNTIME inspect "$COMPLEMENT_SYNAPSE_EDITABLE_IMAGE_PATH" &>/dev/null; then
# complement-synapse-editable already exists: see if we can still use it:
# - The Rust module must still be importable; it will fail to import if the Rust source has changed.
# - The Poetry lock file must be the same (otherwise we assume dependencies have changed)
# First set up the module in the right place for an editable installation.
$CONTAINER_RUNTIME run --rm -v $editable_mount --entrypoint 'cp' complement-synapse-editable -- /synapse_rust.abi3.so.bak /editable-src/synapse/synapse_rust.abi3.so
$CONTAINER_RUNTIME run --rm -v $editable_mount --entrypoint 'cp' "$COMPLEMENT_SYNAPSE_EDITABLE_IMAGE_PATH" -- /synapse_rust.abi3.so.bak /editable-src/synapse/synapse_rust.abi3.so
if ($CONTAINER_RUNTIME run --rm -v $editable_mount --entrypoint 'python' complement-synapse-editable -c 'import synapse.synapse_rust' \
&& $CONTAINER_RUNTIME run --rm -v $editable_mount --entrypoint 'diff' complement-synapse-editable --brief /editable-src/poetry.lock /poetry.lock.bak); then
if ($CONTAINER_RUNTIME run --rm -v $editable_mount --entrypoint 'python' "$COMPLEMENT_SYNAPSE_EDITABLE_IMAGE_PATH" -c 'import synapse.synapse_rust' \
&& $CONTAINER_RUNTIME run --rm -v $editable_mount --entrypoint 'diff' "$COMPLEMENT_SYNAPSE_EDITABLE_IMAGE_PATH" --brief /editable-src/poetry.lock /poetry.lock.bak); then
skip_docker_build=1
else
echo "Editable Synapse image is stale. Will rebuild."
@@ -177,42 +197,47 @@ main() {
# Build a special image designed for use in development with editable
# installs.
$CONTAINER_RUNTIME build -t synapse-editable \
$CONTAINER_RUNTIME build \
-t "$SYNAPSE_EDITABLE_IMAGE_PATH" \
-f "docker/editable.Dockerfile" .
$CONTAINER_RUNTIME build -t synapse-workers-editable \
--build-arg FROM=synapse-editable \
$CONTAINER_RUNTIME build \
-t "$SYNAPSE_WORKERS_EDITABLE_IMAGE_PATH" \
--build-arg FROM="$SYNAPSE_EDITABLE_IMAGE_PATH" \
-f "docker/Dockerfile-workers" .
$CONTAINER_RUNTIME build -t complement-synapse-editable \
--build-arg FROM=synapse-workers-editable \
$CONTAINER_RUNTIME build \
-t "$COMPLEMENT_SYNAPSE_EDITABLE_IMAGE_PATH" \
--build-arg FROM="$SYNAPSE_WORKERS_EDITABLE_IMAGE_PATH" \
-f "docker/complement/Dockerfile" "docker/complement"
# Prepare the Rust module
$CONTAINER_RUNTIME run --rm -v $editable_mount --entrypoint 'cp' complement-synapse-editable -- /synapse_rust.abi3.so.bak /editable-src/synapse/synapse_rust.abi3.so
$CONTAINER_RUNTIME run --rm -v $editable_mount --entrypoint 'cp' "$COMPLEMENT_SYNAPSE_EDITABLE_IMAGE_PATH" -- /synapse_rust.abi3.so.bak /editable-src/synapse/synapse_rust.abi3.so
else
# Build the base Synapse image from the local checkout
echo_if_github "::group::Build Docker image: matrixdotorg/synapse"
$CONTAINER_RUNTIME build -t matrixdotorg/synapse \
--build-arg TEST_ONLY_SKIP_DEP_HASH_VERIFICATION \
--build-arg TEST_ONLY_IGNORE_POETRY_LOCKFILE \
-f "docker/Dockerfile" .
$CONTAINER_RUNTIME build \
-t "$SYNAPSE_IMAGE_PATH" \
--build-arg TEST_ONLY_SKIP_DEP_HASH_VERIFICATION \
--build-arg TEST_ONLY_IGNORE_POETRY_LOCKFILE \
-f "docker/Dockerfile" .
echo_if_github "::endgroup::"
# Build the workers docker image (from the base Synapse image we just built).
echo_if_github "::group::Build Docker image: matrixdotorg/synapse-workers"
$CONTAINER_RUNTIME build -t matrixdotorg/synapse-workers -f "docker/Dockerfile-workers" .
$CONTAINER_RUNTIME build \
-t "$SYNAPSE_WORKERS_IMAGE_PATH" \
--build-arg FROM="$SYNAPSE_IMAGE_PATH" \
-f "docker/Dockerfile-workers" .
echo_if_github "::endgroup::"
# Build the unified Complement image (from the worker Synapse image we just built).
echo_if_github "::group::Build Docker image: complement/Dockerfile"
$CONTAINER_RUNTIME build -t complement-synapse \
`# This is the tag we end up pushing to the registry (see` \
`# .github/workflows/push_complement_image.yml) so let's just label it now` \
`# so people can reference it by the same name locally.` \
-t ghcr.io/element-hq/synapse/complement-synapse \
$CONTAINER_RUNTIME build \
-t "$COMPLEMENT_SYNAPSE_IMAGE_PATH" \
--build-arg FROM="$SYNAPSE_WORKERS_IMAGE_PATH" \
-f "docker/complement/Dockerfile" "docker/complement"
echo_if_github "::endgroup::"
@@ -239,6 +264,7 @@ main() {
./tests/msc4140
./tests/msc4155
./tests/msc4306
./tests/msc4222
)
# Export the list of test packages as a space-separated environment variable, so other
@@ -253,9 +279,9 @@ main() {
./tests/...
)
export COMPLEMENT_BASE_IMAGE=complement-synapse
export COMPLEMENT_BASE_IMAGE="$COMPLEMENT_SYNAPSE_IMAGE_PATH"
if [ -n "$use_editable_synapse" ]; then
export COMPLEMENT_BASE_IMAGE=complement-synapse-editable
export COMPLEMENT_BASE_IMAGE="$COMPLEMENT_SYNAPSE_EDITABLE_IMAGE_PATH"
export COMPLEMENT_HOST_MOUNTS="$editable_mount"
fi
+2 -1
View File
@@ -45,6 +45,7 @@ from synapse.synapse_rust.http_client import HttpClient
from synapse.types import JsonDict, Requester, UserID, create_requester
from synapse.util.caches.cached_call import RetryOnExceptionCachedCall
from synapse.util.caches.response_cache import ResponseCache, ResponseCacheContext
from synapse.util.duration import Duration
from synapse.util.json import json_decoder
from . import introspection_response_timer
@@ -139,7 +140,7 @@ class MasDelegatedAuth(BaseAuth):
clock=self._clock,
name="mas_token_introspection",
server_name=self.server_name,
timeout_ms=120_000,
timeout=Duration(minutes=2),
# don't log because the keys are access tokens
enable_logging=False,
)
+2 -1
View File
@@ -49,6 +49,7 @@ from synapse.synapse_rust.http_client import HttpClient
from synapse.types import Requester, UserID, create_requester
from synapse.util.caches.cached_call import RetryOnExceptionCachedCall
from synapse.util.caches.response_cache import ResponseCache, ResponseCacheContext
from synapse.util.duration import Duration
from synapse.util.json import json_decoder
from . import introspection_response_timer
@@ -205,7 +206,7 @@ class MSC3861DelegatedAuth(BaseAuth):
clock=self._clock,
name="token_introspection",
server_name=self.server_name,
timeout_ms=120_000,
timeout=Duration(minutes=2),
# don't log because the keys are access tokens
enable_logging=False,
)
+1 -3
View File
@@ -325,9 +325,7 @@ class AccountDataTypes:
"org.matrix.msc4155.invite_permission_config"
)
# MSC4380: Invite blocking
MSC4380_INVITE_PERMISSION_CONFIG: Final = (
"org.matrix.msc4380.invite_permission_config"
)
INVITE_PERMISSION_CONFIG: Final = "m.invite_permission_config"
# Synapse-specific behaviour. See "Client-Server API Extensions" documentation
# in Admin API for more information.
SYNAPSE_ADMIN_CLIENT_CONFIG: Final = "io.element.synapse.admin_client_config"
+1 -1
View File
@@ -138,7 +138,7 @@ class Codes(str, Enum):
KEY_TOO_LARGE = "M_KEY_TOO_LARGE"
# Part of MSC4155/MSC4380
INVITE_BLOCKED = "ORG.MATRIX.MSC4155.M_INVITE_BLOCKED"
INVITE_BLOCKED = "M_INVITE_BLOCKED"
# Part of MSC4190
APPSERVICE_LOGIN_UNSUPPORTED = "IO.ELEMENT.MSC4190.M_APPSERVICE_LOGIN_UNSUPPORTED"
+5
View File
@@ -776,6 +776,11 @@ async def start(hs: "HomeServer", *, freeze: bool = True) -> None:
#
# PyPy does not (yet?) implement gc.freeze()
if hasattr(gc, "freeze"):
logger.info(
"garbage collector: Freezing all allocated objects in the hopes that (almost) "
"everything currently allocated are things that will be used by the homeserver "
"for the rest of time. Doing so means less work each GC (hopefully)."
)
gc.collect()
gc.freeze()
+2 -1
View File
@@ -46,6 +46,7 @@ from synapse.logging import opentracing
from synapse.metrics import SERVER_NAME_LABEL
from synapse.types import DeviceListUpdates, JsonDict, JsonMapping, ThirdPartyInstanceID
from synapse.util.caches.response_cache import ResponseCache
from synapse.util.duration import Duration
if TYPE_CHECKING:
from synapse.server import HomeServer
@@ -132,7 +133,7 @@ class ApplicationServiceApi(SimpleHttpClient):
clock=hs.get_clock(),
name="as_protocol_meta",
server_name=self.server_name,
timeout_ms=HOUR_IN_MS,
timeout=Duration(hours=1),
)
def _get_headers(self, service: "ApplicationService") -> dict[bytes, list[bytes]]:
+7 -2
View File
@@ -29,6 +29,7 @@ import attr
from synapse.types import JsonDict
from synapse.util.check_dependencies import check_requirements
from synapse.util.duration import Duration
from ._base import Config, ConfigError
@@ -108,7 +109,7 @@ class CacheConfig(Config):
global_factor: float
track_memory_usage: bool
expiry_time_msec: int | None
sync_response_cache_duration: int
sync_response_cache_duration: Duration
@staticmethod
def reset() -> None:
@@ -207,10 +208,14 @@ class CacheConfig(Config):
min_cache_ttl = self.cache_autotuning.get("min_cache_ttl")
self.cache_autotuning["min_cache_ttl"] = self.parse_duration(min_cache_ttl)
self.sync_response_cache_duration = self.parse_duration(
sync_response_cache_duration_ms = self.parse_duration(
cache_config.get("sync_response_cache_duration", "2m")
)
self.sync_response_cache_duration = Duration(
milliseconds=sync_response_cache_duration_ms
)
def resize_all_caches(self) -> None:
"""Ensure all cache sizes are up-to-date.
+22 -4
View File
@@ -509,7 +509,8 @@ class ExperimentalConfig(Config):
"msc4069_profile_inhibit_propagation", False
)
# MSC4108: Mechanism to allow OIDC sign in and E2EE set up via QR code
# MSC4108: Mechanism to allow OIDC sign in and E2EE set up via QR code - 2024 version:
# See: https://github.com/element-hq/synapse/issues/19434
self.msc4108_enabled = experimental.get("msc4108_enabled", False)
self.msc4108_delegation_endpoint: str | None = experimental.get(
@@ -534,6 +535,26 @@ class ExperimentalConfig(Config):
("experimental", "msc4108_delegation_endpoint"),
)
# MSC4388: Secure out-of-band channel for sign in with QR:
# See: https://github.com/element-hq/synapse/issues/19433
msc4388_mode = experimental.get("msc4388_mode", "off")
if msc4388_mode not in ["off", "public", "authenticated"]:
raise ConfigError(
"msc4388_mode must be one of 'off', 'public' or 'authenticated'",
("experimental", "msc4388_mode"),
)
self.msc4388_enabled: bool = msc4388_mode != "off"
self.msc4388_requires_authentication: bool = msc4388_mode == "authenticated"
if self.msc4388_enabled and not (
config.get("matrix_authentication_service") or {}
).get("enabled", False):
raise ConfigError(
"MSC4388 requires matrix_authentication_service to be enabled",
("experimental", "msc4388_enabled"),
)
# MSC4133: Custom profile fields
self.msc4133_enabled: bool = experimental.get("msc4133_enabled", False)
@@ -585,6 +606,3 @@ class ExperimentalConfig(Config):
# Note that sticky events persisted before this feature is enabled will not be
# considered sticky by the local homeserver.
self.msc4354_enabled: bool = experimental.get("msc4354_enabled", False)
# MSC4380: Invite blocking
self.msc4380_enabled: bool = experimental.get("msc4380_enabled", False)
+2 -1
View File
@@ -20,12 +20,13 @@
#
#
import collections
import collections.abc
import logging
import typing
from collections import ChainMap
from typing import (
Any,
ChainMap,
Iterable,
Mapping,
MutableMapping,
+3 -3
View File
@@ -166,7 +166,7 @@ class FederationServer(FederationBase):
clock=hs.get_clock(),
name="fed_txn_handler",
server_name=self.server_name,
timeout_ms=30000,
timeout=Duration(seconds=30),
)
self.transaction_actions = TransactionActions(self.store)
@@ -179,13 +179,13 @@ class FederationServer(FederationBase):
clock=hs.get_clock(),
name="state_resp",
server_name=self.server_name,
timeout_ms=30000,
timeout=Duration(seconds=30),
)
self._state_ids_resp_cache: ResponseCache[tuple[str, str]] = ResponseCache(
clock=hs.get_clock(),
name="state_ids_resp",
server_name=self.server_name,
timeout_ms=30000,
timeout=Duration(seconds=30),
)
self._federation_metrics_domains = (
+2
View File
@@ -290,6 +290,8 @@ class DeviceHandler:
user_id: The user to delete devices from.
device_ids: The list of device IDs to delete
"""
logger.info("Deleting devices %r for %r", list(device_ids), user_id)
to_device_stream_id = self._event_sources.get_current_token().to_device_key
try:
+1 -1
View File
@@ -185,7 +185,7 @@ class RoomCreationHandler:
clock=hs.get_clock(),
name="room_upgrade",
server_name=self.server_name,
timeout_ms=FIVE_MINUTES_IN_MS,
timeout=Duration(minutes=5),
)
self._server_notices_mxid = hs.config.servernotices.server_notices_mxid
+2 -1
View File
@@ -44,6 +44,7 @@ from synapse.storage.databases.main.room import LargestRoomStats
from synapse.types import JsonDict, JsonMapping, ThirdPartyInstanceID
from synapse.util.caches.descriptors import _CacheContext, cached
from synapse.util.caches.response_cache import ResponseCache
from synapse.util.duration import Duration
if TYPE_CHECKING:
from synapse.server import HomeServer
@@ -79,7 +80,7 @@ class RoomListHandler:
clock=hs.get_clock(),
name="remote_room_list",
server_name=self.server_name,
timeout_ms=30 * 1000,
timeout=Duration(seconds=30),
)
async def get_local_public_room_list(
+21 -3
View File
@@ -49,6 +49,7 @@ from synapse.handlers.profile import MAX_AVATAR_URL_LEN, MAX_DISPLAYNAME_LEN
from synapse.handlers.state_deltas import MatchChange, StateDeltasHandler
from synapse.handlers.worker_lock import NEW_EVENT_DURING_PURGE_LOCK_NAME
from synapse.logging import opentracing
from synapse.logging.opentracing import SynapseTags, set_tag, tag_args, trace
from synapse.metrics import SERVER_NAME_LABEL, event_processing_positions
from synapse.replication.http.push import ReplicationCopyPusherRestServlet
from synapse.storage.databases.main.state_deltas import StateDelta
@@ -390,6 +391,7 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
if requester is not None:
await self._invites_per_issuer_limiter.ratelimit(requester)
@trace
async def _local_membership_update(
self,
*,
@@ -1221,6 +1223,8 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
if result is None or result == (None, None):
raise AuthError(403, f"User {user_id} has no membership in room {room_id}")
@trace
@tag_args
async def _should_perform_remote_join(
self,
user_id: str,
@@ -1275,6 +1279,12 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
prev_member_event = await self.store.get_event(prev_member_event_id)
previous_membership = prev_member_event.membership
# Interesting because it's used in the logic below to make decisions
set_tag(
SynapseTags.RESULT_PREFIX + "previous_membership",
str(previous_membership),
)
# If we are not fully joined yet, and the target is not already in the room,
# let's do a remote join so another server with the full state can validate
# that the user has not been banned for example.
@@ -1315,15 +1325,19 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
state_key: event_map[event_id]
for state_key, event_id in state_before_join.items()
}
allowed_servers = get_servers_from_users(
servers_that_can_issue_invite = get_servers_from_users(
get_users_which_can_issue_invite(current_state)
)
set_tag(
SynapseTags.RESULT_PREFIX + "servers_that_can_issue_invite",
str(servers_that_can_issue_invite),
)
# If the local server is not one of allowed servers, then a remote
# join must be done. Return the list of prospective servers based on
# which can issue invites.
if self.hs.hostname not in allowed_servers:
return True, list(allowed_servers)
if self.hs.hostname not in servers_that_can_issue_invite:
return True, list(servers_that_can_issue_invite)
# Ensure the member should be allowed access via membership in a room.
await self.event_auth_handler.check_restricted_join_rules(
@@ -1897,6 +1911,7 @@ class RoomMemberMasterHandler(RoomMemberHandler):
return complexity["v1"] > max_complexity
@trace
async def _remote_join(
self,
requester: Requester,
@@ -1975,6 +1990,7 @@ class RoomMemberMasterHandler(RoomMemberHandler):
return event_id, stream_id
@trace
async def remote_reject_invite(
self,
invite_event_id: str,
@@ -2012,6 +2028,7 @@ class RoomMemberMasterHandler(RoomMemberHandler):
invite_event, txn_id, requester, content
)
@trace
async def remote_rescind_knock(
self,
knock_event_id: str,
@@ -2124,6 +2141,7 @@ class RoomMemberMasterHandler(RoomMemberHandler):
return result_event.event_id, result_event.internal_metadata.stream_ordering
@trace
async def remote_knock(
self,
requester: Requester,
+5
View File
@@ -23,6 +23,7 @@ import logging
from typing import TYPE_CHECKING
from synapse.handlers.room_member import NoKnownServersError, RoomMemberHandler
from synapse.logging.opentracing import trace
from synapse.replication.http.membership import (
ReplicationRemoteJoinRestServlet as ReplRemoteJoin,
ReplicationRemoteKnockRestServlet as ReplRemoteKnock,
@@ -48,6 +49,7 @@ class RoomMemberWorkerHandler(RoomMemberHandler):
self._remote_rescind_client = ReplRescindKnock.make_client(hs)
self._notify_change_client = ReplJoinedLeft.make_client(hs)
@trace
async def _remote_join(
self,
requester: Requester,
@@ -70,6 +72,7 @@ class RoomMemberWorkerHandler(RoomMemberHandler):
return ret["event_id"], ret["stream_id"]
@trace
async def remote_reject_invite(
self,
invite_event_id: str,
@@ -90,6 +93,7 @@ class RoomMemberWorkerHandler(RoomMemberHandler):
)
return ret["event_id"], ret["stream_id"]
@trace
async def remote_rescind_knock(
self,
knock_event_id: str,
@@ -118,6 +122,7 @@ class RoomMemberWorkerHandler(RoomMemberHandler):
)
return ret["event_id"], ret["stream_id"]
@trace
async def remote_knock(
self,
requester: Requester,
+1 -1
View File
@@ -14,10 +14,10 @@
import itertools
import logging
from collections import ChainMap
from typing import (
TYPE_CHECKING,
AbstractSet,
ChainMap,
Mapping,
MutableMapping,
Sequence,
+18 -4
View File
@@ -77,6 +77,7 @@ from synapse.util.async_helpers import concurrently_execute
from synapse.util.caches.expiringcache import ExpiringCache
from synapse.util.caches.lrucache import LruCache
from synapse.util.caches.response_cache import ResponseCache, ResponseCacheContext
from synapse.util.cancellation import cancellable
from synapse.util.metrics import Measure
from synapse.visibility import filter_and_transform_events_for_client
@@ -307,7 +308,7 @@ class SyncHandler:
clock=hs.get_clock(),
name="sync",
server_name=self.server_name,
timeout_ms=hs.config.caches.sync_response_cache_duration,
timeout=hs.config.caches.sync_response_cache_duration,
)
# ExpiringCache((User, Device)) -> LruCache(user_id => event_id)
@@ -367,6 +368,10 @@ class SyncHandler:
logger.debug("Returning sync response for %s", user_id)
return res
# TODO: We mark this as cancellable, and we have tests for it, but we
# haven't gone through and exhaustively checked that all the code paths in
# this method are actually cancellable.
@cancellable
async def _wait_for_sync_for_user(
self,
sync_config: SyncConfig,
@@ -1041,9 +1046,18 @@ class SyncHandler:
if event.sender not in first_event_by_sender_map:
first_event_by_sender_map[event.sender] = event
# We need the event's sender, unless their membership was in a
# previous timeline event.
if (EventTypes.Member, event.sender) not in timeline_state:
# When using `state_after`, there is no special treatment with
# regards to state also being in the `timeline`. Always fetch
# relevant membership regardless of whether the state event is in
# the `timeline`.
if sync_config.use_state_after:
members_to_fetch.add(event.sender)
# For `state`, the client is supposed to do a flawed re-construction
# of state over time by starting with the given `state` and layering
# on state from the `timeline` as you go (flawed because state
# resolution). In this case, we only need their membership in
# `state` when their membership isn't already in the `timeline`.
elif (EventTypes.Member, event.sender) not in timeline_state:
members_to_fetch.add(event.sender)
# FIXME: we also care about invite targets etc.
+2 -2
View File
@@ -102,7 +102,7 @@ class FollowerTypingHandler:
self._room_typing: dict[str, set[str]] = {}
self._member_last_federation_poke: dict[RoomMember, int] = {}
self.wheel_timer: WheelTimer[RoomMember] = WheelTimer(bucket_size=5000)
self.wheel_timer: WheelTimer[RoomMember] = WheelTimer()
self._latest_room_serial = 0
self._rooms_updated: set[str] = set()
@@ -120,7 +120,7 @@ class FollowerTypingHandler:
self._rooms_updated = set()
self._member_last_federation_poke = {}
self.wheel_timer = WheelTimer(bucket_size=5000)
self.wheel_timer = WheelTimer()
@wrap_as_background_process("typing._handle_timeouts")
async def _handle_timeouts(self) -> None:
+1 -1
View File
@@ -130,7 +130,7 @@ class ReplicationEndpoint(metaclass=abc.ABCMeta):
clock=hs.get_clock(),
name="repl." + self.NAME,
server_name=self.server_name,
timeout_ms=30 * 60 * 1000,
timeout=Duration(minutes=30),
)
# We reserve `instance_name` as a parameter to sending requests, so we
+7
View File
@@ -1144,6 +1144,7 @@ class UserTokenRestServlet(RestServlet):
self.store = hs.get_datastores().main
self.auth = hs.get_auth()
self.auth_handler = hs.get_auth_handler()
self.admin_handler = hs.get_admin_handler()
self.is_mine_id = hs.is_mine_id
async def on_POST(
@@ -1158,6 +1159,12 @@ class UserTokenRestServlet(RestServlet):
HTTPStatus.BAD_REQUEST, "Only local users can be logged in as"
)
# Validate user_id
UserID.from_string(user_id)
_user_info_dict = await self.store.get_user_by_id(user_id)
if not _user_info_dict:
raise NotFoundError("User not found")
body = parse_json_object_from_request(request, allow_empty_body=True)
valid_until_ms = body.get("valid_until_ms")
+49 -1
View File
@@ -21,7 +21,7 @@
import logging
from http.client import TEMPORARY_REDIRECT
from typing import TYPE_CHECKING
from typing import TYPE_CHECKING, Any
from synapse.http.server import HttpServer, respond_with_redirect
from synapse.http.servlet import RestServlet
@@ -68,9 +68,57 @@ class MSC4108RendezvousServlet(RestServlet):
self._handler.handle_post(request)
class MSC4388CreateRendezvousServlet(RestServlet):
PATTERNS = client_patterns(
"/io.element.msc4388/rendezvous$", releases=[], v1=False, unstable=True
)
def __init__(self, hs: "HomeServer") -> None:
super().__init__()
self._handler = hs.get_msc4388_rendezvous_handler()
self.auth = hs.get_auth()
self.require_authentication = (
hs.config.experimental.msc4388_requires_authentication
)
async def on_POST(self, request: SynapseRequest) -> tuple[int, Any]:
if self.require_authentication:
# This will raise if the user is not authenticated
await self.auth.get_user_by_req(request)
return self._handler.handle_post(request)
class MSC4388UpdateRendezvousServlet(RestServlet):
PATTERNS = client_patterns(
"/io.element.msc4388/rendezvous/(?P<rendezvous_id>[^/]+)$",
releases=[],
v1=False,
unstable=True,
)
def __init__(self, hs: "HomeServer") -> None:
super().__init__()
self._handler = hs.get_msc4388_rendezvous_handler()
def on_GET(self, request: SynapseRequest, rendezvous_id: str) -> tuple[int, Any]:
return self._handler.handle_get(rendezvous_id, request)
def on_PUT(self, request: SynapseRequest, rendezvous_id: str) -> tuple[int, Any]:
return self._handler.handle_put(rendezvous_id, request)
def on_DELETE(
self, _request: SynapseRequest, rendezvous_id: str
) -> tuple[int, Any]:
return self._handler.handle_delete(rendezvous_id)
def register_servlets(hs: "HomeServer", http_server: HttpServer) -> None:
if hs.config.experimental.msc4108_enabled:
MSC4108RendezvousServlet(hs).register(http_server)
if hs.config.experimental.msc4108_delegation_endpoint is not None:
MSC4108DelegationRendezvousServlet(hs).register(http_server)
if hs.config.experimental.msc4388_enabled:
MSC4388CreateRendezvousServlet(hs).register(http_server)
MSC4388UpdateRendezvousServlet(hs).register(http_server)
+2
View File
@@ -59,6 +59,7 @@ from synapse.rest.admin.experimental_features import ExperimentalFeature
from synapse.types import JsonDict, Requester, SlidingSyncStreamToken, StreamToken
from synapse.types.rest.client import SlidingSyncBody
from synapse.util.caches.lrucache import LruCache
from synapse.util.cancellation import cancellable
from synapse.util.json import json_decoder
from ._base import client_patterns, set_timeline_upper_limit
@@ -138,6 +139,7 @@ class SyncRestServlet(RestServlet):
cfg=hs.config.ratelimiting.rc_presence_per_user,
)
@cancellable
async def on_GET(self, request: SynapseRequest) -> tuple[int, JsonDict]:
# This will always be set by the time Twisted calls us.
assert request.args is not None
+4 -2
View File
@@ -161,7 +161,7 @@ class VersionsRestServlet(RestServlet):
"org.matrix.msc4069": self.config.experimental.msc4069_profile_inhibit_propagation,
# Allows clients to handle push for encrypted events.
"org.matrix.msc4028": self.config.experimental.msc4028_push_encrypted_events,
# MSC4108: Mechanism to allow OIDC sign in and E2EE set up via QR code
# MSC4108: Mechanism to allow OIDC sign in and E2EE set up via QR code - 2024 version
"org.matrix.msc4108": (
self.config.experimental.msc4108_enabled
or (
@@ -169,6 +169,8 @@ class VersionsRestServlet(RestServlet):
is not None
)
),
# MSC4388: Secure out-of-band channel for sign in with QR
"io.element.msc4388": (self.config.experimental.msc4388_enabled),
# MSC4140: Delayed events
"org.matrix.msc4140": bool(self.config.server.max_event_delay_ms),
# Simplified sliding sync
@@ -185,7 +187,7 @@ class VersionsRestServlet(RestServlet):
# MSC4354: Sticky events
"org.matrix.msc4354": self.config.experimental.msc4354_enabled,
# MSC4380: Invite blocking
"org.matrix.msc4380": self.config.experimental.msc4380_enabled,
"org.matrix.msc4380.stable": True,
},
},
)
+5
View File
@@ -174,6 +174,7 @@ from synapse.state import StateHandler, StateResolutionHandler
from synapse.storage import Databases
from synapse.storage.controllers import StorageControllers
from synapse.streams.events import EventSources
from synapse.synapse_rust.msc4388_rendezvous import MSC4388RendezvousHandler
from synapse.synapse_rust.rendezvous import RendezvousHandler
from synapse.types import DomainSpecificString, ISynapseReactor
from synapse.util import SYNAPSE_VERSION
@@ -1184,6 +1185,10 @@ class HomeServer(metaclass=abc.ABCMeta):
def get_rendezvous_handler(self) -> RendezvousHandler:
return RendezvousHandler(self)
@cache_in_self
def get_msc4388_rendezvous_handler(self) -> MSC4388RendezvousHandler:
return MSC4388RendezvousHandler(self)
@cache_in_self
def get_outbound_redis_connection(self) -> "ConnectionHandler":
"""
+2 -1
View File
@@ -20,7 +20,8 @@
#
import logging
from typing import TYPE_CHECKING, Collection, Counter
from collections import Counter
from typing import TYPE_CHECKING, Collection
from synapse.api.errors import SynapseError
from synapse.storage.database import LoggingTransaction
@@ -109,7 +109,6 @@ class AccountDataWorkerStore(PushRulesWorkerStore, CacheInvalidationWorkerStore)
)
self._msc4155_enabled = hs.config.experimental.msc4155_enabled
self._msc4380_enabled = hs.config.experimental.msc4380_enabled
def get_max_account_data_stream_id(self) -> int:
"""Get the current max stream ID for account data stream
@@ -573,14 +572,13 @@ class AccountDataWorkerStore(PushRulesWorkerStore, CacheInvalidationWorkerStore)
Args:
user_id: The user whose invite configuration should be returned.
"""
if self._msc4380_enabled:
data = await self.get_global_account_data_by_type_for_user(
user_id, AccountDataTypes.MSC4380_INVITE_PERMISSION_CONFIG
)
# If the user has an MSC4380-style config setting, prioritise that
# above an MSC4155 one
if data is not None:
return MSC4380InviteRulesConfig.from_account_data(data)
data = await self.get_global_account_data_by_type_for_user(
user_id, AccountDataTypes.INVITE_PERMISSION_CONFIG
)
# If the user has an MSC4380-style config setting, prioritise that
# above an MSC4155 one
if data is not None:
return MSC4380InviteRulesConfig.from_account_data(data)
if self._msc4155_enabled:
data = await self.get_global_account_data_by_type_for_user(
@@ -132,6 +132,11 @@ EVENT_QUEUE_ITERATIONS = 3 # No. times we block waiting for requests for events
EVENT_QUEUE_TIMEOUT_S = 0.1 # Timeout when waiting for requests for events
# Number of iterations in a loop before we yield to the reactor to allow other
# things to be processed, otherwise we can end up tight looping.
ITERATIONS_BEFORE_YIELDING = 500
event_fetch_ongoing_gauge = Gauge(
"synapse_event_fetch_ongoing",
"The number of event fetchers that are running",
@@ -817,7 +822,7 @@ class EventsWorkerStore(SQLBaseStore):
# may be called repeatedly for the same event so at this point we cannot reach
# out to any external cache for performance reasons. The external cache is
# checked later on in the `get_missing_events_from_cache_or_db` function below.
event_entry_map = self._get_events_from_local_cache(
event_entry_map = await self._get_events_from_local_cache(
event_ids,
)
@@ -1004,7 +1009,7 @@ class EventsWorkerStore(SQLBaseStore):
events: list of event_ids to fetch
update_metrics: Whether to update the cache hit ratio metrics
"""
event_map = self._get_events_from_local_cache(
event_map = await self._get_events_from_local_cache(
events, update_metrics=update_metrics
)
@@ -1045,7 +1050,7 @@ class EventsWorkerStore(SQLBaseStore):
return event_map
def _get_events_from_local_cache(
async def _get_events_from_local_cache(
self, events: Iterable[str], update_metrics: bool = True
) -> dict[str, EventCacheEntry]:
"""Fetch events from the local, in memory, caches.
@@ -1058,7 +1063,15 @@ class EventsWorkerStore(SQLBaseStore):
"""
event_map = {}
i = 0
for event_id in events:
i += 1
# Yield to the reactor to allow other things to be processed,
# otherwise we can end up tight looping.
if i % ITERATIONS_BEFORE_YIELDING == 0:
await self.clock.sleep(Duration(seconds=0))
# First check if it's in the event cache
ret = self._get_event_cache.get_local(
(event_id,), None, update_metrics=update_metrics
@@ -1375,7 +1388,15 @@ class EventsWorkerStore(SQLBaseStore):
# build a map from event_id to EventBase
event_map: dict[str, EventBase] = {}
i = 0
for event_id, row in fetched_events.items():
i += 1
# Yield to the reactor to allow other things to be processed,
# otherwise we can end up tight looping.
if i % ITERATIONS_BEFORE_YIELDING == 0:
await self.clock.sleep(Duration(seconds=0))
assert row.event_id == event_id
rejected_reason = row.rejected_reason
+1 -1
View File
@@ -1032,7 +1032,7 @@ class RoomMemberWorkerStore(EventsWorkerStore, CacheInvalidationWorkerStore):
# We don't update the event cache hit ratio as it completely throws off
# the hit ratio counts. After all, we don't populate the cache if we
# miss it here
event_map = self._get_events_from_local_cache(
event_map = await self._get_events_from_local_cache(
member_event_ids, update_metrics=False
)
+1 -1
View File
@@ -20,12 +20,12 @@
#
import logging
from collections import Counter
from enum import Enum
from itertools import chain
from typing import (
TYPE_CHECKING,
Any,
Counter,
Iterable,
cast,
)
@@ -0,0 +1,33 @@
# This file is licensed under the Affero General Public License (AGPL) version 3.
#
# Copyright (C) 2026 Element Creations Ltd
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# See the GNU Affero General Public License for more details:
# <https://www.gnu.org/licenses/agpl-3.0.html>.
from typing import Any
from twisted.web.iweb import IRequest
from synapse.server import HomeServer
class MSC4388RendezvousHandler:
def __init__(
self,
homeserver: HomeServer,
/,
soft_limit: int = 100, # On each background eviction run sessions will be removed until we're under this limit
hard_limit: int = 200, # If this limit is reached an immediate eviction will be triggered
max_content_length: int = 4 * 1024, # MSC4388 specifies maximum of 4KB
eviction_interval: int = 60 * 1000,
ttl: int = 2 * 60 * 1000, # MSC4388 specifies minimum of 120 seconds
) -> None: ...
def handle_post(self, request: IRequest) -> tuple[int, Any]: ...
def handle_get(self, session_id: str, request: IRequest) -> tuple[int, Any]: ...
def handle_put(self, session_id: str, request: IRequest) -> tuple[int, Any]: ...
def handle_delete(self, session_id: str) -> tuple[int, Any]: ...
+2 -3
View File
@@ -13,7 +13,6 @@
#
import logging
import typing
from collections import ChainMap
from enum import Enum
from typing import (
@@ -793,7 +792,7 @@ class MutableRoomStatusMap(RoomStatusMap[T]):
# We use a ChainMap here so that we can easily track what has been updated
# and what hasn't. Note that when we persist the per connection state this
# will get flattened to a normal dict (via calling `.copy()`)
_statuses: typing.ChainMap[str, HaveSentRoom[T]]
_statuses: ChainMap[str, HaveSentRoom[T]]
def __init__(
self,
@@ -973,7 +972,7 @@ class MutablePerConnectionState(PerConnectionState):
receipts: MutableRoomStatusMap[MultiWriterStreamToken]
account_data: MutableRoomStatusMap[int]
room_configs: typing.ChainMap[str, RoomSyncConfig]
room_configs: ChainMap[str, RoomSyncConfig]
# A map from room ID to the lazily-loaded memberships needed for the
# request in that room.
+71 -3
View File
@@ -25,7 +25,7 @@ import collections
import inspect
import itertools
import logging
import typing
from collections import OrderedDict
from contextlib import asynccontextmanager
from typing import (
Any,
@@ -57,6 +57,7 @@ from synapse.logging.context import (
run_coroutine_in_background,
run_in_background,
)
from synapse.util.cancellation import cancellable
from synapse.util.clock import Clock
from synapse.util.duration import Duration
@@ -83,6 +84,13 @@ class AbstractObservableDeferred(Generic[_T], metaclass=abc.ABCMeta):
"""
...
@abc.abstractmethod
def has_observers(self) -> bool:
"""Returns True if there are any observers currently observing this
ObservableDeferred.
"""
...
class ObservableDeferred(Generic[_T], AbstractObservableDeferred[_T]):
"""Wraps a deferred object so that we can add observer deferreds. These
@@ -122,6 +130,11 @@ class ObservableDeferred(Generic[_T], AbstractObservableDeferred[_T]):
for observer in observers:
try:
observer.callback(r)
except defer.CancelledError:
# We do not want to propagate cancellations to the original
# deferred, or to other observers, so we can just ignore
# this.
pass
except Exception as e:
logger.exception(
"%r threw an exception on .callback(%r), ignoring...",
@@ -145,6 +158,11 @@ class ObservableDeferred(Generic[_T], AbstractObservableDeferred[_T]):
f.value.__failure__ = f
try:
observer.errback(f)
except defer.CancelledError:
# We do not want to propagate cancellations to the original
# deferred, or to other observers, so we can just ignore
# this.
pass
except Exception as e:
logger.exception(
"%r threw an exception on .errback(%r), ignoring...",
@@ -160,6 +178,7 @@ class ObservableDeferred(Generic[_T], AbstractObservableDeferred[_T]):
deferred.addCallbacks(callback, errback)
@cancellable
def observe(self) -> "defer.Deferred[_T]":
"""Observe the underlying deferred.
@@ -169,7 +188,7 @@ class ObservableDeferred(Generic[_T], AbstractObservableDeferred[_T]):
"""
if not self._result:
assert isinstance(self._observers, list)
d: "defer.Deferred[_T]" = defer.Deferred()
d: "defer.Deferred[_T]" = defer.Deferred(canceller=self._remove_observer)
self._observers.append(d)
return d
elif self._result[0]:
@@ -180,6 +199,12 @@ class ObservableDeferred(Generic[_T], AbstractObservableDeferred[_T]):
def observers(self) -> "Collection[defer.Deferred[_T]]":
return self._observers
def has_observers(self) -> bool:
"""Returns True if there are any observers currently observing this
ObservableDeferred.
"""
return bool(self._observers)
def has_called(self) -> bool:
return self._result is not None
@@ -204,6 +229,28 @@ class ObservableDeferred(Generic[_T], AbstractObservableDeferred[_T]):
self._deferred,
)
def _remove_observer(self, observer: "defer.Deferred[_T]") -> None:
"""Removes an observer from the list of observers.
Used as a canceller for the observer deferreds, so that if an observer
is cancelled it is removed from the list of observers.
"""
if self._result is not None:
# The underlying deferred has already resolved, so the observer has
# already been resolved. Nothing to do.
return
assert isinstance(self._observers, list)
try:
self._observers.remove(observer)
except ValueError:
# The observer was not in the list. This can happen if the underlying
# deferred resolves at around the same time as we try to remove the
# observer. In this case, it's possible that we tried to remove the
# observer just after it was added to the list, but before it was
# resolved and removed from the list by the callback/errback above.
pass
T = TypeVar("T")
@@ -525,7 +572,7 @@ class _LinearizerEntry:
# The number of things executing.
count: int
# Deferreds for the things blocked from executing.
deferreds: typing.OrderedDict["defer.Deferred[None]", Literal[1]]
deferreds: OrderedDict["defer.Deferred[None]", Literal[1]]
class Linearizer:
@@ -962,6 +1009,27 @@ def delay_cancellation(awaitable: Awaitable[T]) -> Awaitable[T]:
return new_deferred
def observe_deferred(d: "defer.Deferred[T]") -> "defer.Deferred[T]":
"""Returns a new `Deferred` that observes the given `Deferred`.
The returned `Deferred` will resolve with the same result as the given
`Deferred`, but will not "chain" on the deferred so that using the returned
deferred does not affect the given `Deferred` in any way.
"""
new_deferred: "defer.Deferred[T]" = defer.Deferred()
def callback(r: T) -> T:
new_deferred.callback(r)
return r
def errback(f: Failure) -> Failure:
new_deferred.errback(f)
return f
d.addCallbacks(callback, errback)
return new_deferred
class AwakenableSleeper:
"""Allows explicitly waking up deferreds related to an entity that are
currently sleeping.
+13
View File
@@ -37,6 +37,11 @@ logger = logging.getLogger(__name__)
T = TypeVar("T")
# Number of iterations in a loop before we yield to the reactor to allow other
# things to be processed, otherwise we can end up tight looping.
ITERATIONS_BEFORE_YIELDING = 500
class BackgroundQueue(Generic[T]):
"""A single-producer single-consumer async queue processing items in the
background.
@@ -65,6 +70,7 @@ class BackgroundQueue(Generic[T]):
timeout_ms: int = 1000,
) -> None:
self._hs = hs
self._clock = hs.get_clock()
self._name = name
self._callback = callback
self._timeout_ms = Duration(milliseconds=timeout_ms)
@@ -107,7 +113,14 @@ class BackgroundQueue(Generic[T]):
# single threaded nature, but let's be a bit defensive anyway.)
self._wakeup_event.clear()
iterations = 0
while self._queue:
iterations += 1
if iterations % ITERATIONS_BEFORE_YIELDING == 0:
# Yield to the reactor to allow other things to be processed,
# otherwise we can end up tight looping.
await self._clock.sleep(Duration(seconds=0))
item = self._queue.popleft()
try:
await self._callback(item)
+2 -2
View File
@@ -21,7 +21,7 @@
#
import collections
import logging
import typing
from collections import Counter
from enum import Enum, auto
from sys import intern
from typing import Any, Callable, Sized, TypeVar
@@ -134,7 +134,7 @@ class CacheMetric:
hits: int = 0
misses: int = 0
eviction_size_by_reason: typing.Counter[EvictionReason] = attr.ib(
eviction_size_by_reason: Counter[EvictionReason] = attr.ib(
factory=collections.Counter
)
memory_usage: int | None = None
+143 -9
View File
@@ -39,10 +39,15 @@ from synapse.logging.opentracing import (
start_active_span,
start_active_span_follows_from,
)
from synapse.util.async_helpers import AbstractObservableDeferred, ObservableDeferred
from synapse.util.async_helpers import (
ObservableDeferred,
delay_cancellation,
)
from synapse.util.caches import EvictionReason, register_cache
from synapse.util.cancellation import cancellable, is_function_cancellable
from synapse.util.clock import Clock
from synapse.util.duration import Duration
from synapse.util.wheel_timer import WheelTimer
logger = logging.getLogger(__name__)
@@ -79,8 +84,8 @@ class ResponseCacheContext(Generic[KV]):
@attr.s(auto_attribs=True)
class ResponseCacheEntry:
result: AbstractObservableDeferred
class ResponseCacheEntry(Generic[KV]):
result: ObservableDeferred[KV]
"""The (possibly incomplete) result of the operation.
Note that we continue to store an ObservableDeferred even after the operation
@@ -91,6 +96,15 @@ class ResponseCacheEntry:
opentracing_span_context: "opentracing.SpanContext | None"
"""The opentracing span which generated/is generating the result"""
cancellable: bool
"""Whether the deferred is safe to be cancelled."""
last_observer_removed_time_ms: int | None = None
"""The last time that an observer was removed from this entry.
Used to determine when to evict the entry if it has no observers.
"""
class ResponseCache(Generic[KV]):
"""
@@ -98,6 +112,22 @@ class ResponseCache(Generic[KV]):
returned from the cache. This means that if the client retries the request
while the response is still being computed, that original response will be
used rather than trying to compute a new response.
If a timeout is not specified then the cache entry will be kept while the
wrapped function is still running, and will be removed immediately once it
completes.
If a timeout is specified then the cache entry will be kept for the duration
of the timeout after the wrapped function completes. If the wrapped function
is cancellable and during processing nothing waits on the result for longer
than the timeout then the wrapped function will be cancelled and the cache
entry will be removed.
This behaviour is useful for caching responses to requests which are
expensive to compute, but which may be retried by clients if they time out.
For example, /sync requests which may take a long time to compute, and which
clients will retry. However, if the client stops retrying for a while then
we want to stop processing the request and free up the resources.
"""
def __init__(
@@ -106,7 +136,7 @@ class ResponseCache(Generic[KV]):
clock: Clock,
name: str,
server_name: str,
timeout_ms: float = 0,
timeout: Duration | None = None,
enable_logging: bool = True,
):
"""
@@ -121,7 +151,7 @@ class ResponseCache(Generic[KV]):
self._result_cache: dict[KV, ResponseCacheEntry] = {}
self.clock = clock
self.timeout = Duration(milliseconds=timeout_ms)
self.timeout = timeout
self._name = name
self._metrics = register_cache(
@@ -133,6 +163,13 @@ class ResponseCache(Generic[KV]):
)
self._enable_logging = enable_logging
self._prune_timer: WheelTimer[KV] | None = None
if self.timeout:
# Set up the timers for pruning inflight entries. The times here are
# how often we check for entries to prune.
self._prune_timer = WheelTimer(bucket_size=self.timeout / 10)
self.clock.looping_call(self._prune_inflight_entries, self.timeout / 10)
def size(self) -> int:
return len(self._result_cache)
@@ -172,6 +209,7 @@ class ResponseCache(Generic[KV]):
context: ResponseCacheContext[KV],
deferred: "defer.Deferred[RV]",
opentracing_span_context: "opentracing.SpanContext | None",
cancellable: bool,
) -> ResponseCacheEntry:
"""Set the entry for the given key to the given deferred.
@@ -183,13 +221,16 @@ class ResponseCache(Generic[KV]):
context: Information about the cache miss
deferred: The deferred which resolves to the result.
opentracing_span_context: An opentracing span wrapping the calculation
cancellable: Whether the deferred is safe to be cancelled
Returns:
The cache entry object.
"""
result = ObservableDeferred(deferred, consumeErrors=True)
key = context.cache_key
entry = ResponseCacheEntry(result, opentracing_span_context)
entry = ResponseCacheEntry(
result, opentracing_span_context, cancellable=cancellable
)
self._result_cache[key] = entry
def on_complete(r: RV) -> RV:
@@ -233,6 +274,7 @@ class ResponseCache(Generic[KV]):
self._metrics.inc_evictions(EvictionReason.time)
self._result_cache.pop(key, None)
@cancellable
async def wrap(
self,
key: KV,
@@ -301,8 +343,44 @@ class ResponseCache(Generic[KV]):
return await callback(*args, **kwargs)
d = run_in_background(cb)
entry = self._set(context, d, span_context)
return await make_deferred_yieldable(entry.result.observe())
entry = self._set(
context, d, span_context, cancellable=is_function_cancellable(callback)
)
try:
return await make_deferred_yieldable(entry.result.observe())
except defer.CancelledError:
pass
# We've been cancelled.
#
# Since we've kicked off the background operation, we can't just
# give up and return here and need to wait for the background
# operation to stop. We don't want to stop the background process
# immediately to give a chance for retries to come in and wait for
# the result.
#
# Instead, we temporarily swallow the cancellation and mark the
# cache key as one to potentially timeout.
# Update the `last_observer_removed_time_ms` so that the pruning
# mechanism can kick in if needed.
now = self.clock.time_msec()
entry.last_observer_removed_time_ms = now
if self._prune_timer is not None and self.timeout:
self._prune_timer.insert(now, key, now + self.timeout.as_millis())
# Wait on the original deferred, which will continue to run in the
# background until it completes. We don't want to add an observer as
# this would prevent the entry from being pruned.
#
# Note that this deferred has been consumed by the
# ObservableDeferred, so we don't know what it will return. That
# doesn't matter as we just want to throw a CancelledError once it completes anyway.
try:
await make_deferred_yieldable(delay_cancellation(d))
except Exception:
pass
raise defer.CancelledError()
result = entry.result.observe()
if self._enable_logging:
@@ -320,4 +398,60 @@ class ResponseCache(Generic[KV]):
f"ResponseCache[{self._name}].wait",
contexts=(span_context,) if span_context else (),
):
return await make_deferred_yieldable(result)
try:
return await make_deferred_yieldable(result)
except defer.CancelledError:
# If we're cancelled then we update the
# `last_observer_removed_time_ms` so that the pruning mechanism
# can kick in if needed.
now = self.clock.time_msec()
entry.last_observer_removed_time_ms = now
if self._prune_timer is not None and self.timeout:
self._prune_timer.insert(now, key, now + self.timeout.as_millis())
raise
def _prune_inflight_entries(self) -> None:
"""Prune entries which have been in the cache for too long without
observers"""
assert self._prune_timer is not None
assert self.timeout is not None
now = self.clock.time_msec()
keys_to_check = self._prune_timer.fetch(now)
# Loop through the keys and check if they should be evicted. We evict
# entries which have no active observers, and which have been in the
# cache for longer than the timeout since the last observer was removed.
for key in keys_to_check:
entry = self._result_cache.get(key)
if not entry:
continue
if not entry.cancellable:
# this entry is not cancellable, so we should keep it in the cache until it completes.
continue
if entry.result.has_called():
# this entry has already completed, so we should have scheduled it for
# removal at the right time. We can just skip it here and wait for the
# scheduled call to remove it.
continue
if entry.result.has_observers():
# this entry has observers, so we should keep it in the cache for now.
continue
if entry.last_observer_removed_time_ms is None:
# this should never happen, but just in case, we should keep the entry
# in the cache until we have a valid last_observer_removed_time_ms to
# compare against.
continue
if now - entry.last_observer_removed_time_ms > self.timeout.as_millis():
self._metrics.inc_evictions(EvictionReason.time)
self._result_cache.pop(key, None)
try:
entry.result.cancel()
except Exception:
# we ignore exceptions from cancel, as it is best effort anyway.
pass
+15 -1
View File
@@ -62,6 +62,16 @@ this setting won't inherit the log level from the parent logger.
logging.setLoggerClass(original_logger_class)
def _try_wakeup_deferred(d: Deferred) -> None:
"""Try to wake up a deferred, but ignore any exceptions raised by the
callback. This is useful when we want to wake up a deferred that may have
already been cancelled, and we don't care about the result."""
try:
d.callback(None)
except Exception:
pass
class Clock:
"""
A Clock wraps a Twisted reactor and provides utilities on top of it.
@@ -114,7 +124,11 @@ class Clock:
with context.PreserveLoggingContext():
# We can ignore the lint here since this class is the one location callLater should
# be called.
self._reactor.callLater(duration.as_secs(), d.callback, duration.as_secs()) # type: ignore[call-later-not-tracked]
self._reactor.callLater(
duration.as_secs(),
lambda _: _try_wakeup_deferred(d),
duration.as_secs(),
) # type: ignore[call-later-not-tracked]
await d
def time(self) -> float:
+7 -5
View File
@@ -23,6 +23,8 @@ from typing import Generic, Hashable, TypeVar
import attr
from synapse.util.duration import Duration
logger = logging.getLogger(__name__)
T = TypeVar("T", bound=Hashable)
@@ -39,13 +41,13 @@ class WheelTimer(Generic[T]):
expired.
"""
def __init__(self, bucket_size: int = 5000) -> None:
def __init__(self, bucket_size: Duration = Duration(seconds=5)) -> None:
"""
Args:
bucket_size: Size of buckets in ms. Corresponds roughly to the
accuracy of the timer.
"""
self.bucket_size: int = bucket_size
self.bucket_size = bucket_size
self.entries: list[_Entry[T]] = []
def insert(self, now: int, obj: T, then: int) -> None:
@@ -56,8 +58,8 @@ class WheelTimer(Generic[T]):
obj: Object to be inserted
then: When to return the object strictly after.
"""
then_key = int(then / self.bucket_size) + 1
now_key = int(now / self.bucket_size)
then_key = int(then / self.bucket_size.as_millis()) + 1
now_key = int(now / self.bucket_size.as_millis())
if self.entries:
min_key = self.entries[0].end_key
@@ -100,7 +102,7 @@ class WheelTimer(Generic[T]):
Returns:
List of objects that have timed out
"""
now_key = int(now / self.bucket_size)
now_key = int(now / self.bucket_size.as_millis())
ret: list[T] = []
while self.entries and self.entries[0].end_key <= now_key:
@@ -34,7 +34,7 @@ from synapse.util.duration import Duration
from synapse.util.ratelimitutils import FederationRateLimiter
from tests import unittest
from tests.http.server._base import test_disconnect
from tests.http.server._base import disconnect_and_assert
class CancellableFederationServlet(BaseFederationServlet):
@@ -94,7 +94,7 @@ class BaseFederationServletCancellationTests(unittest.FederatingHomeserverTestCa
# request won't be processed.
self.pump()
test_disconnect(
disconnect_and_assert(
self.reactor,
channel,
expect_cancellation=True,
@@ -114,7 +114,7 @@ class BaseFederationServletCancellationTests(unittest.FederatingHomeserverTestCa
# request won't be processed.
self.pump()
test_disconnect(
disconnect_and_assert(
self.reactor,
channel,
expect_cancellation=False,
+8 -35
View File
@@ -503,7 +503,7 @@ class TestMSC4155InviteFiltering(FederatingHomeserverTestCase):
SynapseError,
).value
self.assertEqual(f.code, 403)
self.assertEqual(f.errcode, "ORG.MATRIX.MSC4155.M_INVITE_BLOCKED")
self.assertEqual(f.errcode, "M_INVITE_BLOCKED")
@override_config({"experimental_features": {"msc4155_enabled": False}})
def test_msc4155_disabled_allow_invite_local(self) -> None:
@@ -573,7 +573,7 @@ class TestMSC4155InviteFiltering(FederatingHomeserverTestCase):
SynapseError,
).value
self.assertEqual(f.code, 403)
self.assertEqual(f.errcode, "ORG.MATRIX.MSC4155.M_INVITE_BLOCKED")
self.assertEqual(f.errcode, "M_INVITE_BLOCKED")
@override_config({"experimental_features": {"msc4155_enabled": True}})
def test_msc4155_block_invite_remote_server(self) -> None:
@@ -619,7 +619,7 @@ class TestMSC4155InviteFiltering(FederatingHomeserverTestCase):
SynapseError,
).value
self.assertEqual(f.code, 403)
self.assertEqual(f.errcode, "ORG.MATRIX.MSC4155.M_INVITE_BLOCKED")
self.assertEqual(f.errcode, "M_INVITE_BLOCKED")
class TestMSC4380InviteBlocking(FederatingHomeserverTestCase):
@@ -642,7 +642,6 @@ class TestMSC4380InviteBlocking(FederatingHomeserverTestCase):
self.bob = self.register_user("bob", "pass")
self.bob_token = self.login("bob", "pass")
@override_config({"experimental_features": {"msc4380_enabled": True}})
def test_misc4380_block_invite_local(self) -> None:
"""Test that MSC4380 will block a user from being invited to a room"""
room_id = self.helper.create_room_as(self.alice, tok=self.alice_token)
@@ -650,7 +649,7 @@ class TestMSC4380InviteBlocking(FederatingHomeserverTestCase):
self.get_success(
self.store.add_account_data_for_user(
self.bob,
AccountDataTypes.MSC4380_INVITE_PERMISSION_CONFIG,
AccountDataTypes.INVITE_PERMISSION_CONFIG,
{
"default_action": "block",
},
@@ -667,9 +666,8 @@ class TestMSC4380InviteBlocking(FederatingHomeserverTestCase):
SynapseError,
).value
self.assertEqual(f.code, 403)
self.assertEqual(f.errcode, "ORG.MATRIX.MSC4155.M_INVITE_BLOCKED")
self.assertEqual(f.errcode, "M_INVITE_BLOCKED")
@override_config({"experimental_features": {"msc4380_enabled": True}})
def test_misc4380_non_string_setting(self) -> None:
"""Test that `default_action` being set to something non-stringy is the same as "accept"."""
room_id = self.helper.create_room_as(self.alice, tok=self.alice_token)
@@ -677,7 +675,7 @@ class TestMSC4380InviteBlocking(FederatingHomeserverTestCase):
self.get_success(
self.store.add_account_data_for_user(
self.bob,
AccountDataTypes.MSC4380_INVITE_PERMISSION_CONFIG,
AccountDataTypes.INVITE_PERMISSION_CONFIG,
{
"default_action": 1,
},
@@ -693,31 +691,6 @@ class TestMSC4380InviteBlocking(FederatingHomeserverTestCase):
)
)
@override_config({"experimental_features": {"msc4380_enabled": False}})
def test_msc4380_disabled_allow_invite_local(self) -> None:
"""Test that, when MSC4380 is not enabled, invites are accepted as normal"""
room_id = self.helper.create_room_as(self.alice, tok=self.alice_token)
self.get_success(
self.store.add_account_data_for_user(
self.bob,
AccountDataTypes.MSC4380_INVITE_PERMISSION_CONFIG,
{
"default_action": "block",
},
)
)
self.get_success(
self.handler.update_membership(
requester=create_requester(self.alice),
target=UserID.from_string(self.bob),
room_id=room_id,
action=Membership.INVITE,
),
)
@override_config({"experimental_features": {"msc4380_enabled": True}})
def test_msc4380_block_invite_remote(self) -> None:
"""Test that MSC4380 will block a user from being invited to a room by a remote user."""
# A remote user who sends the invite
@@ -727,7 +700,7 @@ class TestMSC4380InviteBlocking(FederatingHomeserverTestCase):
self.get_success(
self.store.add_account_data_for_user(
self.bob,
AccountDataTypes.MSC4380_INVITE_PERMISSION_CONFIG,
AccountDataTypes.INVITE_PERMISSION_CONFIG,
{"default_action": "block"},
)
)
@@ -761,4 +734,4 @@ class TestMSC4380InviteBlocking(FederatingHomeserverTestCase):
SynapseError,
).value
self.assertEqual(f.code, 403)
self.assertEqual(f.errcode, "ORG.MATRIX.MSC4155.M_INVITE_BLOCKED")
self.assertEqual(f.errcode, "M_INVITE_BLOCKED")
+1 -1
View File
@@ -59,7 +59,7 @@ logger = logging.getLogger(__name__)
T = TypeVar("T")
def test_disconnect(
def disconnect_and_assert(
reactor: MemoryReactorClock,
channel: FakeChannel,
expect_cancellation: bool,
+3 -3
View File
@@ -37,7 +37,7 @@ from synapse.util.cancellation import cancellable
from synapse.util.duration import Duration
from tests import unittest
from tests.http.server._base import test_disconnect
from tests.http.server._base import disconnect_and_assert
def make_request(content: bytes | JsonDict) -> Mock:
@@ -127,7 +127,7 @@ class TestRestServletCancellation(unittest.HomeserverTestCase):
def test_cancellable_disconnect(self) -> None:
"""Test that handlers with the `@cancellable` flag can be cancelled."""
channel = self.make_request("GET", "/sleep", await_result=False)
test_disconnect(
disconnect_and_assert(
self.reactor,
channel,
expect_cancellation=True,
@@ -137,7 +137,7 @@ class TestRestServletCancellation(unittest.HomeserverTestCase):
def test_uncancellable_disconnect(self) -> None:
"""Test that handlers without the `@cancellable` flag cannot be cancelled."""
channel = self.make_request("POST", "/sleep", await_result=False)
test_disconnect(
disconnect_and_assert(
self.reactor,
channel,
expect_cancellation=False,
+3 -3
View File
@@ -33,7 +33,7 @@ from synapse.util.cancellation import cancellable
from synapse.util.duration import Duration
from tests import unittest
from tests.http.server._base import test_disconnect
from tests.http.server._base import disconnect_and_assert
class CancellableReplicationEndpoint(ReplicationEndpoint):
@@ -94,7 +94,7 @@ class ReplicationEndpointCancellationTestCase(unittest.HomeserverTestCase):
"""Test that handlers with the `@cancellable` flag can be cancelled."""
path = f"{REPLICATION_PREFIX}/{CancellableReplicationEndpoint.NAME}/"
channel = self.make_request("POST", path, await_result=False, content={})
test_disconnect(
disconnect_and_assert(
self.reactor,
channel,
expect_cancellation=True,
@@ -105,7 +105,7 @@ class ReplicationEndpointCancellationTestCase(unittest.HomeserverTestCase):
"""Test that handlers without the `@cancellable` flag cannot be cancelled."""
path = f"{REPLICATION_PREFIX}/{UncancellableReplicationEndpoint.NAME}/"
channel = self.make_request("POST", path, await_result=False, content={})
test_disconnect(
disconnect_and_assert(
self.reactor,
channel,
expect_cancellation=False,
+11
View File
@@ -4288,6 +4288,17 @@ class UserTokenRestTestCase(unittest.HomeserverTestCase):
self.assertEqual(403, channel.code, msg=channel.json_body)
def test_no_user(self) -> None:
"""Try to log in as a user that doesn't exist."""
channel = self.make_request(
"POST",
"/_synapse/admin/v1/users/%s/login" % urllib.parse.quote("@ghost:test"),
b"{}",
access_token=self.admin_user_tok,
)
self.assertEqual(404, channel.code, msg=channel.json_body)
self.assertEqual(Codes.NOT_FOUND, channel.json_body["errcode"])
def test_send_event(self) -> None:
"""Test that sending event as a user works."""
# Create a room.
@@ -0,0 +1,743 @@
#
# This file is licensed under the Affero General Public License (AGPL) version 3.
#
# Copyright (C) 2026 Element Creations Ltd
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# See the GNU Affero General Public License for more details:
# <https://www.gnu.org/licenses/agpl-3.0.html>.
import json
import urllib.parse
from typing import Any, Mapping
from unittest.mock import Mock
from parameterized import parameterized
from twisted.internet.testing import MemoryReactor
from synapse.api.auth.mas import MasDelegatedAuth
from synapse.rest import admin
from synapse.rest.client import login, rendezvous
from synapse.server import HomeServer
from synapse.types import UserID
from synapse.util.clock import Clock
from tests import unittest
from tests.unittest import checked_cast, override_config
rz_endpoint = "/_matrix/client/unstable/io.element.msc4388/rendezvous"
class RendezvousServletTestCase(unittest.HomeserverTestCase):
"""
Test the experimental MSC4388 rendezvous endpoint.
"""
servlets = [
admin.register_servlets,
login.register_servlets,
rendezvous.register_servlets,
]
def make_homeserver(self, reactor: MemoryReactor, clock: Clock) -> HomeServer:
self.hs = self.setup_test_homeserver()
return self.hs
def setup_mock_oauth(self) -> None:
"""
This isn't a very elegant way to mock the OAuth API, but it works for our purposes.
"""
self.auth = checked_cast(MasDelegatedAuth, self.hs.get_auth())
self._rust_client = Mock(spec=["post"])
self._rust_client.post = self._mock_oauth_response
self.auth._rust_http_client = self._rust_client
async def _mock_oauth_response(
self,
url: str,
response_limit: int,
headers: Mapping[str, str],
request_body: str,
) -> bytes:
# get the token from the request body which is form encoded
parsed_body = urllib.parse.parse_qs(request_body)
token = parsed_body.get("token", [""])[0]
if not token.startswith("mock_token_"):
return bytes(json.dumps({"active": False}).encode("utf-8"))
token = token.replace("mock_token_", "")
username, device_id = token.split("_", 1)
user_id = UserID(username, self.hs.hostname)
store = self.hs.get_datastores().main
# Check th user exists in the store
user_info = await store.get_user_by_id(user_id=user_id.to_string())
if user_info is None:
return bytes(json.dumps({"active": False}).encode("utf-8"))
# Check the device exists in the store
device = await store.get_device(
user_id=user_id.to_string(), device_id=device_id
)
if device is None:
return bytes(json.dumps({"active": False}).encode("utf-8"))
return bytes(
json.dumps(
{
"active": True,
"scope": "urn:matrix:client:device:"
+ device_id
+ " urn:matrix:client:api:*",
"username": username,
}
).encode("utf-8")
)
def register_oauth_user(self, username: str, device_id: str) -> str:
# Provision the user and the device
store = self.hs.get_datastores().main
user_id = UserID(username, self.hs.hostname)
self.get_success(store.register_user(user_id=user_id.to_string()))
self.get_success(
store.store_device(
user_id=user_id.to_string(),
device_id=device_id,
initial_device_display_name=None,
)
)
# Generate an access token for the device
return "mock_token_" + username + "_" + device_id
def test_disabled(self) -> None:
channel = self.make_request("POST", rz_endpoint, {}, access_token=None)
self.assertEqual(channel.code, 404)
@override_config(
{
"experimental_features": {
"msc4388_mode": "off",
},
}
)
def test_off(self) -> None:
channel = self.make_request("POST", rz_endpoint, {}, access_token=None)
self.assertEqual(channel.code, 404)
@override_config(
{
"disable_registration": True,
"matrix_authentication_service": {
"enabled": True,
"secret": "secret_value",
"endpoint": "https://issuer",
},
"experimental_features": {
"msc4388_mode": "public",
},
}
)
def test_rendezvous_public(self) -> None:
"""
Test the MSC4108 rendezvous endpoint, including:
- Creating a session
- Getting the data back
- Updating the data
- Deleting the data
- Sequence token handling
"""
# We can post arbitrary data to the endpoint
channel = self.make_request(
"POST",
rz_endpoint,
{"data": "foo=bar"},
access_token=None,
)
self.assertEqual(channel.code, 200)
rendezvous_id = channel.json_body["id"]
sequence_token = channel.json_body["sequence_token"]
expires_in_ms = channel.json_body["expires_in_ms"]
self.assertGreater(expires_in_ms, 0)
session_endpoint = rz_endpoint + f"/{rendezvous_id}"
# We can get the data back
# Advances clock by 100ms
channel = self.make_request(
"GET",
session_endpoint,
access_token=None,
)
self.assertEqual(channel.code, 200)
self.assertEqual(channel.json_body["data"], "foo=bar")
self.assertEqual(channel.json_body["sequence_token"], sequence_token)
self.assertEqual(channel.json_body["expires_in_ms"], expires_in_ms - 100)
# We can update the data
# Advances clock by 100ms
channel = self.make_request(
"PUT",
session_endpoint,
{"sequence_token": sequence_token, "data": "foo=baz"},
access_token=None,
)
self.assertEqual(channel.code, 200)
old_sequence_token = sequence_token
new_sequence_token = channel.json_body["sequence_token"]
# If we try to update it again with the old etag, it should fail
# Advances clock by 100ms
channel = self.make_request(
"PUT",
session_endpoint,
{"sequence_token": old_sequence_token, "data": "bar=baz"},
access_token=None,
)
self.assertEqual(channel.code, 409)
self.assertEqual(
channel.json_body["errcode"], "IO_ELEMENT_MSC4388_CONCURRENT_WRITE"
)
# We should get the updated data
# Advances clock by 100ms
channel = self.make_request(
"GET",
session_endpoint,
access_token=None,
)
self.assertEqual(channel.code, 200)
self.assertEqual(channel.json_body["data"], "foo=baz")
self.assertEqual(channel.json_body["sequence_token"], new_sequence_token)
self.assertEqual(channel.json_body["expires_in_ms"], expires_in_ms - 400)
# We can delete the data
channel = self.make_request(
"DELETE",
session_endpoint,
access_token=None,
)
self.assertEqual(channel.code, 200)
# If we try to get the data again, it should fail
channel = self.make_request(
"GET",
session_endpoint,
access_token=None,
)
self.assertEqual(channel.code, 404)
self.assertEqual(channel.json_body["errcode"], "M_NOT_FOUND")
@override_config(
{
"disable_registration": True,
"matrix_authentication_service": {
"enabled": True,
"secret": "secret_value",
"endpoint": "https://issuer",
},
"experimental_features": {
"msc4388_mode": "authenticated",
},
}
)
def test_rendezvous_requires_authentication(self) -> None:
"""
Test the MSC4108 rendezvous endpoint when configured with the mode authenticated, including:
- Creating a session
- Getting the data back
- Updating the data
- Deleting the data
- Sequence token handling
"""
self.setup_mock_oauth()
alice_token = self.register_oauth_user("alice", "device1")
# This should fail without authentication:
channel = self.make_request(
"POST",
rz_endpoint,
{"data": "foo=bar"},
access_token=None,
)
self.assertEqual(channel.code, 401)
# This should work as we are now authenticated
channel = self.make_request(
"POST",
rz_endpoint,
{"data": "foo=bar"},
access_token=alice_token,
)
self.assertEqual(channel.code, 200)
rendezvous_id = channel.json_body["id"]
sequence_token = channel.json_body["sequence_token"]
expires_in_ms = channel.json_body["expires_in_ms"]
self.assertGreater(expires_in_ms, 0)
session_endpoint = rz_endpoint + f"/{rendezvous_id}"
# We can get the data back without authentication
channel = self.make_request(
"GET",
session_endpoint,
access_token=None,
)
self.assertEqual(channel.code, 200)
self.assertEqual(channel.json_body["data"], "foo=bar")
self.assertEqual(channel.json_body["sequence_token"], sequence_token)
self.assertEqual(channel.json_body["expires_in_ms"], expires_in_ms)
# We can update the data without authentication
channel = self.make_request(
"PUT",
session_endpoint,
{"sequence_token": sequence_token, "data": "foo=baz"},
access_token=None,
)
self.assertEqual(channel.code, 200)
new_sequence_token = channel.json_body["sequence_token"]
# We should get the updated data without authentication
channel = self.make_request(
"GET",
session_endpoint,
access_token=None,
)
self.assertEqual(channel.code, 200)
self.assertEqual(channel.json_body["data"], "foo=baz")
self.assertEqual(channel.json_body["sequence_token"], new_sequence_token)
self.assertEqual(channel.json_body["expires_in_ms"], expires_in_ms - 200)
# We can delete the data without authentication
channel = self.make_request(
"DELETE",
session_endpoint,
access_token=None,
)
self.assertEqual(channel.code, 200)
# If we try to get the data again, it should fail
channel = self.make_request(
"GET",
session_endpoint,
access_token=None,
)
self.assertEqual(channel.code, 404)
self.assertEqual(channel.json_body["errcode"], "M_NOT_FOUND")
@override_config(
{
"disable_registration": True,
"matrix_authentication_service": {
"enabled": True,
"secret": "secret_value",
"endpoint": "https://issuer",
},
"experimental_features": {
"msc4388_mode": "public",
},
}
)
def test_expiration(self) -> None:
"""
Test that entries are evicted after a TTL.
"""
# Start a new session
channel = self.make_request(
"POST",
rz_endpoint,
{"data": "foo=bar"},
access_token=None,
)
self.assertEqual(channel.code, 200)
session_endpoint = rz_endpoint + "/" + channel.json_body["id"]
# Sanity check that we can get the data back
channel = self.make_request(
"GET",
session_endpoint,
access_token=None,
)
self.assertEqual(channel.code, 200)
self.assertEqual(channel.json_body["data"], "foo=bar")
# Advance the clock, TTL of entries is 2 minutes
self.reactor.advance(120)
# Get the data back, it should be gone
channel = self.make_request(
"GET",
session_endpoint,
access_token=None,
)
self.assertEqual(channel.code, 404)
@override_config(
{
"disable_registration": True,
"matrix_authentication_service": {
"enabled": True,
"secret": "secret_value",
"endpoint": "https://issuer",
},
"experimental_features": {
"msc4388_mode": "public",
},
}
)
def test_capacity(self) -> None:
"""
Test that the soft capacity limit is enforced on the rendezvous sessions, as old
entries are evicted at an interval when the limit is reached.
"""
# Start a new session
channel = self.make_request(
"POST",
rz_endpoint,
{"data": "foo=bar"},
access_token=None,
)
self.assertEqual(channel.code, 200)
session_endpoint = rz_endpoint + "/" + channel.json_body["id"]
# Sanity check that we can get the data back
channel = self.make_request(
"GET",
session_endpoint,
access_token=None,
)
self.assertEqual(channel.code, 200)
self.assertEqual(channel.json_body["data"], "foo=bar")
# We advance the clock to make sure that this entry is the "lowest" in the session list
self.reactor.advance(1)
# Start a lot of new sessions
for _ in range(100):
channel = self.make_request(
"POST",
rz_endpoint,
{"data": "foo=bar"},
access_token=None,
)
self.assertEqual(channel.code, 200)
# Get the data back, it should still be there, as the eviction hasn't run yet
channel = self.make_request(
"GET",
session_endpoint,
access_token=None,
)
self.assertEqual(channel.code, 200)
# Advance the clock, as it will trigger the eviction
self.reactor.advance(59)
# Get the data back, it should be gone
channel = self.make_request(
"GET",
session_endpoint,
access_token=None,
)
self.assertEqual(channel.code, 404)
@override_config(
{
"disable_registration": True,
"matrix_authentication_service": {
"enabled": True,
"secret": "secret_value",
"endpoint": "https://issuer",
},
"experimental_features": {
"msc4388_mode": "public",
},
}
)
def test_hard_capacity(self) -> None:
"""
Test that the hard capacity limit is enforced on the rendezvous sessions, as old
entries are evicted immediately when the limit is reached.
"""
# Start a new session
channel = self.make_request(
"POST",
rz_endpoint,
{"data": "foo=bar"},
access_token=None,
)
self.assertEqual(channel.code, 200)
session_endpoint = rz_endpoint + "/" + channel.json_body["id"]
# We advance the clock to make sure that this entry is the "lowest" in the session list
self.reactor.advance(1)
# Sanity check that we can get the data back
channel = self.make_request(
"GET",
session_endpoint,
access_token=None,
)
self.assertEqual(channel.code, 200)
self.assertEqual(channel.json_body["data"], "foo=bar")
# Start a lot of new sessions
for _ in range(200):
channel = self.make_request(
"POST",
rz_endpoint,
{"data": "foo=bar"},
access_token=None,
)
self.assertEqual(channel.code, 200)
# Get the data back, it should already be gone as we hit the hard limit
channel = self.make_request(
"GET",
session_endpoint,
access_token=None,
)
self.assertEqual(channel.code, 404)
@override_config(
{
"disable_registration": True,
"matrix_authentication_service": {
"enabled": True,
"secret": "secret_value",
"endpoint": "https://issuer",
},
"experimental_features": {
"msc4388_mode": "public",
},
}
)
def test_data_type(self) -> None:
"""
Test that the data field is restricted to string.
"""
invalid_datas: list[Any] = [123214, ["asd"], {"asd": "asdsad"}, None]
# We cannot post invalid non-string data field values to the endpoint
for invalid_data in invalid_datas:
channel = self.make_request(
"POST",
rz_endpoint,
{"data": invalid_data},
access_token=None,
)
self.assertEqual(channel.code, 400)
self.assertEqual(channel.json_body["errcode"], "M_INVALID_PARAM")
# Make a valid request
channel = self.make_request(
"POST",
rz_endpoint,
{"data": "test"},
access_token=None,
)
self.assertEqual(channel.code, 200)
rendezvous_id = channel.json_body["id"]
sequence_token = channel.json_body["sequence_token"]
session_endpoint = rz_endpoint + f"/{rendezvous_id}"
# We can't update the data with invalid data
for invalid_data in invalid_datas:
channel = self.make_request(
"PUT",
session_endpoint,
{"sequence_token": sequence_token, "data": invalid_data},
access_token=None,
)
self.assertEqual(channel.code, 400)
self.assertEqual(channel.json_body["errcode"], "M_INVALID_PARAM")
@override_config(
{
"disable_registration": True,
"matrix_authentication_service": {
"enabled": True,
"secret": "secret_value",
"endpoint": "https://issuer",
},
"experimental_features": {
"msc4388_mode": "public",
},
}
)
def test_max_length(self) -> None:
"""
Test that the data max length is restricted.
"""
too_long_data = "a" * 5000 # MSC4108 specifies 4KB max length
channel = self.make_request(
"POST",
rz_endpoint,
{"data": too_long_data},
access_token=None,
)
self.assertEqual(channel.code, 413)
self.assertEqual(channel.json_body["errcode"], "M_TOO_LARGE")
# Make a valid request
channel = self.make_request(
"POST",
rz_endpoint,
{"data": "test"},
access_token=None,
)
self.assertEqual(channel.code, 200)
rendezvous_id = channel.json_body["id"]
sequence_token = channel.json_body["sequence_token"]
session_endpoint = rz_endpoint + f"/{rendezvous_id}"
# We can't update the data with invalid data
channel = self.make_request(
"PUT",
session_endpoint,
{"sequence_token": sequence_token, "data": too_long_data},
access_token=None,
)
self.assertEqual(channel.code, 413)
self.assertEqual(channel.json_body["errcode"], "M_TOO_LARGE")
@parameterized.expand(
[
("Sec-Fetch-Dest", "document"),
("Sec-Fetch-Dest", "image"),
("Sec-Fetch-Dest", "iframe"),
("Sec-Fetch-Dest", "embed"),
("Sec-Fetch-Dest", "video"),
("Sec-Fetch-Mode", "navigate"),
("Sec-Fetch-User", "?1"),
("Sec-Fetch-Site", "none"),
]
)
@override_config(
{
"disable_registration": True,
"matrix_authentication_service": {
"enabled": True,
"secret": "secret_value",
"endpoint": "https://issuer",
},
"experimental_features": {
"msc4388_mode": "public",
},
}
)
def test_rendezvous_rejects_unsafe_get_requests(
self, header_name: str, header_value: str
) -> None:
"""
Tests that GET requests have the appropriate Sec-Fetch-* controls applied as per the MSC.
The mode is set to `public` but this doesn't actually matter.
"""
# We can post arbitrary data to the endpoint
channel = self.make_request(
"POST",
rz_endpoint,
{"data": "foo=bar"},
access_token=None,
)
self.assertEqual(channel.code, 200)
rendezvous_id = channel.json_body["id"]
session_endpoint = rz_endpoint + f"/{rendezvous_id}"
# We can get the data back
channel = self.make_request(
"GET",
session_endpoint,
access_token=None,
)
self.assertEqual(channel.code, 200)
channel = self.make_request(
"GET",
session_endpoint,
access_token=None,
custom_headers=[(header_name, header_value)],
)
self.assertEqual(channel.code, 403)
self.assertEqual(channel.json_body["errcode"], "M_FORBIDDEN")
@override_config(
{
"disable_registration": True,
"matrix_authentication_service": {
"enabled": True,
"secret": "secret_value",
"endpoint": "https://issuer",
},
"experimental_features": {
"msc4388_mode": "public",
},
}
)
def test_rendezvous_allows_from_browser_fetch(self) -> None:
"""
We check that the GET policy does allow for an expected browser fetch
The mode is set to `public` but this doesn't actually matter.
"""
# We can post arbitrary data to the endpoint
channel = self.make_request(
"POST",
rz_endpoint,
{"data": "foo=bar"},
access_token=None,
)
self.assertEqual(channel.code, 200)
rendezvous_id = channel.json_body["id"]
session_endpoint = rz_endpoint + f"/{rendezvous_id}"
# We can get the data back
channel = self.make_request(
"GET",
session_endpoint,
access_token=None,
)
self.assertEqual(channel.code, 200)
# Test for a typical browser fetch from a client hosted on a different origin
channel = self.make_request(
"GET",
session_endpoint,
access_token=None,
custom_headers=[
("Sec-Fetch-Dest", "empty"),
("Sec-Fetch-Mode", "cors"),
("Sec-Fetch-Site", "cross-site"),
],
)
self.assertEqual(channel.code, 200)
+63
View File
@@ -41,6 +41,7 @@ from tests import unittest
from tests.federation.transport.test_knocking import (
KnockingStrippedStateEventHelperMixin,
)
from tests.rest.client.test_rooms import make_request_with_cancellation_test
from tests.server import TimedOutException
logger = logging.getLogger(__name__)
@@ -1145,3 +1146,65 @@ class ExcludeRoomTestCase(unittest.HomeserverTestCase):
self.assertNotIn(self.excluded_room_id, channel.json_body["rooms"]["join"])
self.assertIn(self.included_room_id, channel.json_body["rooms"]["join"])
class SyncCancellationTestCase(unittest.HomeserverTestCase):
servlets = [
synapse.rest.admin.register_servlets,
login.register_servlets,
sync.register_servlets,
room.register_servlets,
]
def test_initial_sync(self) -> None:
"""Tests that an initial sync request can be cancelled."""
user_id = self.register_user("user", "password")
tok = self.login("user", "password")
# Populate the account with a few rooms
for _ in range(5):
room_id = self.helper.create_room_as(user_id, tok=tok)
self.helper.send(room_id, tok=tok)
channel = make_request_with_cancellation_test(
"test_initial_sync",
self.reactor,
self.site,
"GET",
"/_matrix/client/v3/sync",
token=tok,
)
self.assertEqual(200, channel.code, msg=channel.result["body"])
def test_incremental_sync(self) -> None:
"""Tests that an incremental sync request can be cancelled."""
user_id = self.register_user("user", "password")
tok = self.login("user", "password")
# Populate the account with a few rooms
room_ids = []
for _ in range(5):
room_id = self.helper.create_room_as(user_id, tok=tok)
self.helper.send(room_id, tok=tok)
room_ids.append(room_id)
# Do an initial sync to get a since token.
channel = self.make_request("GET", "/sync", access_token=tok)
self.assertEqual(200, channel.code, msg=channel.result)
since = channel.json_body["next_batch"]
# Send some more messages to generate activity in the rooms.
for room_id in room_ids:
self.helper.send(room_id, tok=tok)
channel = make_request_with_cancellation_test(
"test_incremental_sync",
self.reactor,
self.site,
"GET",
f"/_matrix/client/v3/sync?since={since}&timeout=10000",
token=tok,
)
self.assertEqual(200, channel.code, msg=channel.result["body"])
+18
View File
@@ -171,6 +171,24 @@ class HttpClientTestCase(HomeserverTestCase):
self.get_success(self.till_deferred_has_result(do_request()))
self.assertEqual(self.server.calls, 1)
def test_request_response_limit_exceeded(self) -> None:
"""
Test to make sure we handle the response limit being exceeded
"""
async def do_request() -> None:
await self._rust_http_client.get(
url=self.server.endpoint,
# Small limit so we hit the limit
response_limit=1,
)
self.assertFailure(
self.till_deferred_has_result(do_request()),
RuntimeError,
)
self.assertEqual(self.server.calls, 1)
async def test_logging_context(self) -> None:
"""
Test to make sure the `LoggingContext` (logcontext) is handled correctly
+5 -5
View File
@@ -41,7 +41,7 @@ from synapse.util.clock import Clock
from synapse.util.duration import Duration
from tests import unittest
from tests.http.server._base import test_disconnect
from tests.http.server._base import disconnect_and_assert
from tests.server import (
FakeChannel,
FakeSite,
@@ -506,7 +506,7 @@ class DirectServeJsonResourceCancellationTests(unittest.TestCase):
channel = make_request(
self.reactor, self.site, "GET", "/sleep", await_result=False
)
test_disconnect(
disconnect_and_assert(
self.reactor,
channel,
expect_cancellation=True,
@@ -518,7 +518,7 @@ class DirectServeJsonResourceCancellationTests(unittest.TestCase):
channel = make_request(
self.reactor, self.site, "POST", "/sleep", await_result=False
)
test_disconnect(
disconnect_and_assert(
self.reactor,
channel,
expect_cancellation=False,
@@ -540,7 +540,7 @@ class DirectServeHtmlResourceCancellationTests(unittest.TestCase):
channel = make_request(
self.reactor, self.site, "GET", "/sleep", await_result=False
)
test_disconnect(
disconnect_and_assert(
self.reactor,
channel,
expect_cancellation=True,
@@ -552,6 +552,6 @@ class DirectServeHtmlResourceCancellationTests(unittest.TestCase):
channel = make_request(
self.reactor, self.site, "POST", "/sleep", await_result=False
)
test_disconnect(
disconnect_and_assert(
self.reactor, channel, expect_cancellation=False, expected_body=b"ok"
)
+342 -3
View File
@@ -19,6 +19,7 @@
#
#
from functools import wraps
from unittest.mock import Mock
from parameterized import parameterized
@@ -26,6 +27,7 @@ from parameterized import parameterized
from twisted.internet import defer
from synapse.util.caches.response_cache import ResponseCache, ResponseCacheContext
from synapse.util.cancellation import cancellable
from synapse.util.duration import Duration
from tests.server import get_clock
@@ -48,15 +50,23 @@ class ResponseCacheTestCase(TestCase):
def with_cache(self, name: str, ms: int = 0) -> ResponseCache:
return ResponseCache(
clock=self.clock, name=name, server_name="test_server", timeout_ms=ms
clock=self.clock,
name=name,
server_name="test_server",
timeout=Duration(milliseconds=ms),
)
@staticmethod
async def instant_return(o: str) -> str:
return o
async def delayed_return(self, o: str) -> str:
await self.clock.sleep(Duration(seconds=1))
@cancellable
async def delayed_return(
self,
o: str,
duration: Duration = Duration(seconds=1), # noqa
) -> str:
await self.clock.sleep(duration)
return o
def test_cache_hit(self) -> None:
@@ -223,3 +233,332 @@ class ResponseCacheTestCase(TestCase):
self.assertCountEqual(
[], cache.keys(), "cache should not have the result now"
)
def test_cache_func_errors(self) -> None:
"""If the callback raises an error, the error should be raised to all
callers and the result should not be cached"""
cache = self.with_cache("error_cache", ms=3000)
expected_error = Exception("oh no")
async def erring(o: str) -> str:
await self.clock.sleep(Duration(seconds=1))
raise expected_error
wrap_d = defer.ensureDeferred(cache.wrap(0, erring, "ignored"))
self.assertNoResult(wrap_d)
# a second call should also return a pending deferred
wrap2_d = defer.ensureDeferred(cache.wrap(0, erring, "ignored"))
self.assertNoResult(wrap2_d)
# let the call complete
self.reactor.advance(1)
# both results should have completed with the error
self.assertFailure(wrap_d, Exception)
self.assertFailure(wrap2_d, Exception)
def test_cache_cancel_first_wait(self) -> None:
"""Test that cancellation of the deferred returned by wrap() on the
first call does not immediately cause a cancellation error to be raised
when its cancelled and the wrapped function continues execution (unless
it times out).
"""
cache = self.with_cache("cancel_cache", ms=3000)
expected_result = "howdy"
wrap_d = defer.ensureDeferred(
cache.wrap(0, self.delayed_return, expected_result)
)
# cancel the deferred before it has a chance to return
wrap_d.cancel()
# The cancel should be ignored for now, and the inner function should
# still be running.
self.assertNoResult(wrap_d)
# Advance the clock until the inner function should have returned, but
# not long enough for the cache entry to have expired.
self.reactor.advance(2)
# The deferred we're waiting on should now return a cancelled error.
self.assertFailure(wrap_d, defer.CancelledError)
# However future callers should get the result.
wrap_d2 = defer.ensureDeferred(
cache.wrap(0, self.delayed_return, expected_result)
)
self.assertEqual(expected_result, self.successResultOf(wrap_d2))
def test_cache_cancel_first_wait_expire(self) -> None:
"""Test that cancellation of the deferred returned by wrap() and the
entry expiring before the wrapped function returns.
The wrapped function should be cancelled.
"""
cache = self.with_cache("cancel_expire_cache", ms=300)
expected_result = "howdy"
# Wrap the function so that we can keep track of when it completes or
# errors.
completed = False
cancelled = False
@wraps(self.delayed_return)
async def wrapped(o: str) -> str:
nonlocal completed, cancelled
try:
return await self.delayed_return(o)
except defer.CancelledError:
cancelled = True
raise
finally:
completed = True
wrap_d = defer.ensureDeferred(cache.wrap(0, wrapped, expected_result))
# cancel the deferred before it has a chance to return
wrap_d.cancel()
# The cancel should be ignored for now, and the inner function should
# still be running.
self.assertNoResult(wrap_d)
self.assertFalse(completed, "wrapped function should not have completed yet")
# Advance the clock until the cache entry should have expired, but not
# long enough for the inner function to have returned.
self.reactor.advance(0.7)
# The deferred we're waiting on should now return a cancelled error.
self.assertFailure(wrap_d, defer.CancelledError)
self.assertTrue(completed, "wrapped function should have completed")
self.assertTrue(cancelled, "wrapped function should have been cancelled")
def test_cache_cancel_first_wait_other_observers(self) -> None:
"""Test that cancellation of the deferred returned by wrap() does not
cause a cancellation error to be raised if there are other observers
still waiting on the result.
"""
cache = self.with_cache("cancel_other_cache", ms=300)
expected_result = "howdy"
# Wrap the function so that we can keep track of when it completes or
# errors.
completed = False
cancelled = False
@wraps(self.delayed_return)
async def wrapped(o: str) -> str:
nonlocal completed, cancelled
try:
return await self.delayed_return(o)
except defer.CancelledError:
cancelled = True
raise
finally:
completed = True
wrap_d1 = defer.ensureDeferred(cache.wrap(0, wrapped, expected_result))
wrap_d2 = defer.ensureDeferred(cache.wrap(0, wrapped, expected_result))
# cancel the first deferred before it has a chance to return
wrap_d1.cancel()
# The cancel should be ignored for now, and the inner function should
# still be running.
self.assertNoResult(wrap_d1)
self.assertNoResult(wrap_d2)
self.assertFalse(completed, "wrapped function should not have completed yet")
# Advance the clock until the cache entry should have expired, but not
# long enough for the inner function to have returned.
self.reactor.advance(0.7)
# Neither deferred should have returned yet, since the inner function
# should still be running.
self.assertNoResult(wrap_d1)
self.assertNoResult(wrap_d2)
self.assertFalse(completed, "wrapped function should not have completed yet")
# Now advance the clock until the inner function should have returned.
self.reactor.advance(2.5)
# The wrapped function should have completed without cancellation.
self.assertTrue(completed, "wrapped function should have completed")
self.assertFalse(cancelled, "wrapped function should not have been cancelled")
# The first deferred we're waiting on should now return a cancelled error.
self.assertFailure(wrap_d1, defer.CancelledError)
# The second deferred should return the result.
self.assertEqual(expected_result, self.successResultOf(wrap_d2))
def test_cache_add_and_cancel(self) -> None:
"""Test that waiting on the cache and cancelling repeatedly keeps the
cache entry alive.
"""
cache = self.with_cache("cancel_add_cache", ms=300)
expected_result = "howdy"
# Wrap the function so that we can keep track of when it completes or
# errors.
completed = False
cancelled = False
@wraps(self.delayed_return)
async def wrapped(o: str) -> str:
nonlocal completed, cancelled
try:
return await self.delayed_return(o)
except defer.CancelledError:
cancelled = True
raise
finally:
completed = True
# Repeatedly await for the result and cancel it, which should keep the
# cache entry alive even though the total time exceeds the cache
# timeout.
deferreds = []
for _ in range(8):
# Await the deferred.
wrap_d = defer.ensureDeferred(cache.wrap(0, wrapped, expected_result))
# cancel the deferred before it has a chance to return
self.reactor.advance(0.05)
wrap_d.cancel()
deferreds.append(wrap_d)
# The cancel should not cause the inner function to be cancelled
# yet.
self.assertFalse(
completed, "wrapped function should not have completed yet"
)
self.assertFalse(
cancelled, "wrapped function should not have been cancelled yet"
)
# Advance the clock until the cache entry should have expired, but not
# long enough for the inner function to have returned.
self.reactor.advance(0.05)
# Now advance the clock until the inner function should have returned.
self.reactor.advance(0.2)
# All the deferreds we're waiting on should now return a cancelled error.
for wrap_d in deferreds:
self.assertFailure(wrap_d, defer.CancelledError)
# The wrapped function should have completed without cancellation.
self.assertTrue(completed, "wrapped function should have completed")
self.assertFalse(cancelled, "wrapped function should not have been cancelled")
# Querying the cache should return the completed result
wrap_d = defer.ensureDeferred(cache.wrap(0, wrapped, expected_result))
self.assertEqual(expected_result, self.successResultOf(wrap_d))
def test_cache_cancel_non_cancellable(self) -> None:
"""Test that cancellation of the deferred returned by wrap() on a
non-cancellable entry does not cause a cancellation error to be raised
when it's cancelled and the wrapped function continues execution.
"""
cache = self.with_cache("cancel_non_cancellable_cache", ms=300)
expected_result = "howdy"
# Wrap the function so that we can keep track of when it completes or
# errors.
completed = False
cancelled = False
async def wrapped(o: str) -> str:
nonlocal completed, cancelled
try:
return await self.delayed_return(o)
except defer.CancelledError:
cancelled = True
raise
finally:
completed = True
wrap_d = defer.ensureDeferred(cache.wrap(0, wrapped, expected_result))
# cancel the deferred before it has a chance to return
wrap_d.cancel()
# The cancel should be ignored for now, and the inner function should
# still be running.
self.assertNoResult(wrap_d)
self.assertFalse(completed, "wrapped function should not have completed yet")
# Advance the clock until the inner function should have returned, but
# not long enough for the cache entry to have expired.
self.reactor.advance(2)
# The deferred we're waiting on should be cancelled, but a new call to
# the cache should return the result.
self.assertFailure(wrap_d, defer.CancelledError)
wrap_d2 = defer.ensureDeferred(cache.wrap(0, wrapped, expected_result))
self.assertEqual(expected_result, self.successResultOf(wrap_d2))
def test_cache_cancel_then_error(self) -> None:
"""Test that cancellation of the deferred returned by wrap() that then
subsequently errors is correctly propagated to a second caller.
"""
cache = self.with_cache("cancel_then_error_cache", ms=3000)
expected_error = Exception("oh no")
# Wrap the function so that we can keep track of when it completes or
# errors.
completed = False
cancelled = False
@wraps(self.delayed_return)
async def wrapped(o: str) -> str:
nonlocal completed, cancelled
try:
await self.delayed_return(o)
raise expected_error
except defer.CancelledError:
cancelled = True
raise
finally:
completed = True
wrap_d1 = defer.ensureDeferred(cache.wrap(0, wrapped, "ignored"))
wrap_d2 = defer.ensureDeferred(cache.wrap(0, wrapped, "ignored"))
# cancel the first deferred before it has a chance to return
wrap_d1.cancel()
# The cancel should be ignored for now, and the inner function should
# still be running.
self.assertNoResult(wrap_d1)
self.assertNoResult(wrap_d2)
self.assertFalse(completed, "wrapped function should not have completed yet")
# Advance the clock until the inner function should have returned.
self.reactor.advance(2)
# The wrapped function should have completed with an error without cancellation.
self.assertTrue(completed, "wrapped function should have completed")
self.assertFalse(cancelled, "wrapped function should not have been cancelled")
# The first deferred we're waiting on should now return a cancelled error.
self.assertFailure(wrap_d1, defer.CancelledError)
# The second deferred should return the error.
self.assertFailure(wrap_d2, Exception)
+21 -1
View File
@@ -120,7 +120,7 @@ class ObservableDeferredTest(TestCase):
assert results[1] is not None
self.assertEqual(str(results[1].value), "gah!", "observer 2 errback result")
def test_cancellation(self) -> None:
def test_cancellation_observer(self) -> None:
"""Test that cancelling an observer does not affect other observers."""
origin_d: "Deferred[int]" = Deferred()
observable = ObservableDeferred(origin_d, consumeErrors=True)
@@ -138,6 +138,10 @@ class ObservableDeferredTest(TestCase):
self.assertFalse(observer1.called)
self.failureResultOf(observer2, CancelledError)
self.assertFalse(observer3.called)
# check that we remove the cancelled observer from the list of observers
# as a clean up.
self.assertEqual(len(observable.observers()), 2)
self.assertNotIn(observer2, observable.observers())
# other observers resolve as normal
origin_d.callback(123)
@@ -148,6 +152,22 @@ class ObservableDeferredTest(TestCase):
observer4 = observable.observe()
self.assertEqual(observer4.result, 123, "observer 4 callback result")
def test_cancellation_observee(self) -> None:
"""Test that cancelling the original deferred cancels all observers."""
origin_d: "Deferred[int]" = Deferred()
observable = ObservableDeferred(origin_d, consumeErrors=True)
observer1 = observable.observe()
observer2 = observable.observe()
self.assertFalse(observer1.called)
self.assertFalse(observer2.called)
# cancel the original deferred
origin_d.cancel()
self.failureResultOf(observer1, CancelledError)
self.failureResultOf(observer2, CancelledError)
class TimeoutDeferredTest(TestCase):
def setUp(self) -> None:
+6 -5
View File
@@ -19,6 +19,7 @@
#
#
from synapse.util.duration import Duration
from synapse.util.wheel_timer import WheelTimer
from .. import unittest
@@ -26,7 +27,7 @@ from .. import unittest
class WheelTimerTestCase(unittest.TestCase):
def test_single_insert_fetch(self) -> None:
wheel: WheelTimer[object] = WheelTimer(bucket_size=5)
wheel: WheelTimer[object] = WheelTimer(bucket_size=Duration(milliseconds=5))
wheel.insert(100, "1", 150)
@@ -39,7 +40,7 @@ class WheelTimerTestCase(unittest.TestCase):
self.assertListEqual(wheel.fetch(170), [])
def test_multi_insert(self) -> None:
wheel: WheelTimer[object] = WheelTimer(bucket_size=5)
wheel: WheelTimer[object] = WheelTimer(bucket_size=Duration(milliseconds=5))
wheel.insert(100, "1", 150)
wheel.insert(105, "2", 130)
@@ -54,13 +55,13 @@ class WheelTimerTestCase(unittest.TestCase):
self.assertListEqual(wheel.fetch(210), [])
def test_insert_past(self) -> None:
wheel: WheelTimer[object] = WheelTimer(bucket_size=5)
wheel: WheelTimer[object] = WheelTimer(bucket_size=Duration(milliseconds=5))
wheel.insert(100, "1", 50)
self.assertListEqual(wheel.fetch(120), ["1"])
def test_insert_past_multi(self) -> None:
wheel: WheelTimer[object] = WheelTimer(bucket_size=5)
wheel: WheelTimer[object] = WheelTimer(bucket_size=Duration(milliseconds=5))
wheel.insert(100, "1", 150)
wheel.insert(100, "2", 140)
@@ -72,7 +73,7 @@ class WheelTimerTestCase(unittest.TestCase):
self.assertListEqual(wheel.fetch(240), [])
def test_multi_insert_then_past(self) -> None:
wheel: WheelTimer[object] = WheelTimer(bucket_size=5)
wheel: WheelTimer[object] = WheelTimer(bucket_size=Duration(milliseconds=5))
wheel.insert(100, "1", 150)
wheel.insert(100, "2", 160)