Merge branch 'develop' into madlittlemods/return-historical-events-in-order-from-backfill
Conflicts: scripts-dev/complement.sh
This commit is contained in:
Eric Eastwood
17
.github/workflows/tests.yml
vendored
17
.github/workflows/tests.yml
vendored
@@ -76,7 +76,7 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
python-version: ["3.6", "3.7", "3.8", "3.9", "3.10"]
|
||||
python-version: ["3.7", "3.8", "3.9", "3.10"]
|
||||
database: ["sqlite"]
|
||||
toxenv: ["py"]
|
||||
include:
|
||||
@@ -85,9 +85,9 @@ jobs:
|
||||
toxenv: "py-noextras"
|
||||
|
||||
# Oldest Python with PostgreSQL
|
||||
- python-version: "3.6"
|
||||
- python-version: "3.7"
|
||||
database: "postgres"
|
||||
postgres-version: "9.6"
|
||||
postgres-version: "10"
|
||||
toxenv: "py"
|
||||
|
||||
# Newest Python with newest PostgreSQL
|
||||
@@ -167,7 +167,7 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
python-version: ["pypy-3.6"]
|
||||
python-version: ["pypy-3.7"]
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
@@ -291,8 +291,8 @@ jobs:
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- python-version: "3.6"
|
||||
postgres-version: "9.6"
|
||||
- python-version: "3.7"
|
||||
postgres-version: "10"
|
||||
|
||||
- python-version: "3.10"
|
||||
postgres-version: "14"
|
||||
@@ -366,6 +366,8 @@ jobs:
|
||||
# Build initial Synapse image
|
||||
- run: docker build -t matrixdotorg/synapse:latest -f docker/Dockerfile .
|
||||
working-directory: synapse
|
||||
env:
|
||||
DOCKER_BUILDKIT: 1
|
||||
|
||||
# Build a ready-to-run Synapse image based on the initial image above.
|
||||
# This new image includes a config file, keys for signing and TLS, and
|
||||
@@ -374,7 +376,8 @@ jobs:
|
||||
working-directory: complement/dockerfiles
|
||||
|
||||
# Run Complement
|
||||
- run: go test -v -tags synapse_blacklist,msc2403 ./tests/...
|
||||
- run: set -o pipefail && go test -v -json -tags synapse_blacklist,msc2403 ./tests/... 2>&1 | gotestfmt
|
||||
shell: bash
|
||||
env:
|
||||
COMPLEMENT_BASE_IMAGE: complement-synapse:latest
|
||||
working-directory: complement
|
||||
|
||||
4
.gitignore
vendored
4
.gitignore
vendored
@@ -50,3 +50,7 @@ __pycache__/
|
||||
|
||||
# docs
|
||||
book/
|
||||
|
||||
# complement
|
||||
/complement-master
|
||||
/master.tar.gz
|
||||
|
||||
115
CHANGES.md
115
CHANGES.md
@@ -1,3 +1,118 @@
|
||||
Synapse 1.50.0rc1 (2022-01-05)
|
||||
==============================
|
||||
|
||||
Please note that we now only support Python 3.7+ and PostgreSQL 10+ (if applicable), because Python 3.6 and PostgreSQL 9.6 have reached end-of-life.
|
||||
|
||||
|
||||
Features
|
||||
--------
|
||||
|
||||
- Allow guests to send state events per [MSC3419](https://github.com/matrix-org/matrix-doc/pull/3419). ([\#11378](https://github.com/matrix-org/synapse/issues/11378))
|
||||
- Add experimental support for part of [MSC3202](https://github.com/matrix-org/matrix-doc/pull/3202): allowing application services to masquerade as specific devices. ([\#11538](https://github.com/matrix-org/synapse/issues/11538))
|
||||
- Add admin API to get users' account data. ([\#11664](https://github.com/matrix-org/synapse/issues/11664))
|
||||
- Include the room topic in the stripped state included with invites and knocking. ([\#11666](https://github.com/matrix-org/synapse/issues/11666))
|
||||
- Send and handle cross-signing messages using the stable prefix. ([\#10520](https://github.com/matrix-org/synapse/issues/10520))
|
||||
- Support unprefixed versions of fallback key property names. ([\#11541](https://github.com/matrix-org/synapse/issues/11541))
|
||||
|
||||
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Fix a long-standing bug where relations from other rooms could be included in the bundled aggregations of an event. ([\#11516](https://github.com/matrix-org/synapse/issues/11516))
|
||||
- Fix a long-standing bug which could cause `AssertionError`s to be written to the log when Synapse was restarted after purging events from the database. ([\#11536](https://github.com/matrix-org/synapse/issues/11536), [\#11642](https://github.com/matrix-org/synapse/issues/11642))
|
||||
- Fix a bug introduced in Synapse 1.17.0 where a pusher created for an email with capital letters would fail to be created. ([\#11547](https://github.com/matrix-org/synapse/issues/11547))
|
||||
- Fix a long-standing bug where responses included bundled aggregations when they should not, per [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675). ([\#11592](https://github.com/matrix-org/synapse/issues/11592), [\#11623](https://github.com/matrix-org/synapse/issues/11623))
|
||||
- Fix a long-standing bug that some unknown endpoints would return HTML error pages instead of JSON `M_UNRECOGNIZED` errors. ([\#11602](https://github.com/matrix-org/synapse/issues/11602))
|
||||
- Fix a bug introduced in Synapse 1.19.3 which could sometimes cause `AssertionError`s when backfilling rooms over federation. ([\#11632](https://github.com/matrix-org/synapse/issues/11632))
|
||||
|
||||
|
||||
Improved Documentation
|
||||
----------------------
|
||||
|
||||
- Update Synapse install command for FreeBSD as the package is now prefixed with `py38`. Contributed by @itchychips. ([\#11267](https://github.com/matrix-org/synapse/issues/11267))
|
||||
- Document the usage of refresh tokens. ([\#11427](https://github.com/matrix-org/synapse/issues/11427))
|
||||
- Add details for how to configure a TURN server when behind a NAT. Contibuted by @AndrewFerr. ([\#11553](https://github.com/matrix-org/synapse/issues/11553))
|
||||
- Add references for using Postgres to the Docker documentation. ([\#11640](https://github.com/matrix-org/synapse/issues/11640))
|
||||
- Fix the documentation link in newly-generated configuration files. ([\#11678](https://github.com/matrix-org/synapse/issues/11678))
|
||||
- Correct the documentation for `nginx` to use a case-sensitive url pattern. Fixes an error introduced in v1.21.0. ([\#11680](https://github.com/matrix-org/synapse/issues/11680))
|
||||
- Clarify SSO mapping provider documentation by writing `def` or `async def` before the names of methods, as appropriate. ([\#11681](https://github.com/matrix-org/synapse/issues/11681))
|
||||
|
||||
|
||||
Deprecations and Removals
|
||||
-------------------------
|
||||
|
||||
- Replace `mock` package by its standard library version. ([\#11588](https://github.com/matrix-org/synapse/issues/11588))
|
||||
- Drop support for Python 3.6 and Ubuntu 18.04. ([\#11633](https://github.com/matrix-org/synapse/issues/11633))
|
||||
|
||||
|
||||
Internal Changes
|
||||
----------------
|
||||
|
||||
- Allow specific, experimental events to be created without `prev_events`. Used by [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716). ([\#11243](https://github.com/matrix-org/synapse/issues/11243))
|
||||
- A test helper (`wait_for_background_updates`) no longer depends on classes defining a `store` property. ([\#11331](https://github.com/matrix-org/synapse/issues/11331))
|
||||
- Add type hints to `synapse.appservice`. ([\#11360](https://github.com/matrix-org/synapse/issues/11360))
|
||||
- Add missing type hints to `synapse.config` module. ([\#11480](https://github.com/matrix-org/synapse/issues/11480))
|
||||
- Add test to ensure we share the same `state_group` across the whole historical batch when using the [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) `/batch_send` endpoint. ([\#11487](https://github.com/matrix-org/synapse/issues/11487))
|
||||
- Refactor `tests.util.setup_test_homeserver` and `tests.server.setup_test_homeserver`. ([\#11503](https://github.com/matrix-org/synapse/issues/11503))
|
||||
- Move `glob_to_regex` and `re_word_boundary` to `matrix-python-common`. ([\#11505](https://github.com/matrix-org/synapse/issues/11505), [\#11687](https://github.com/matrix-org/synapse/issues/11687))
|
||||
- Use `HTTPStatus` constants in place of literals in `tests.rest.client.test_auth`. ([\#11520](https://github.com/matrix-org/synapse/issues/11520))
|
||||
- Add a receipt types constant for `m.read`. ([\#11531](https://github.com/matrix-org/synapse/issues/11531))
|
||||
- Clean up `synapse.rest.admin`. ([\#11535](https://github.com/matrix-org/synapse/issues/11535))
|
||||
- Add missing `errcode` to `parse_string` and `parse_boolean`. ([\#11542](https://github.com/matrix-org/synapse/issues/11542))
|
||||
- Use `HTTPStatus` constants in place of literals in `synapse.http`. ([\#11543](https://github.com/matrix-org/synapse/issues/11543))
|
||||
- Add missing type hints to storage classes. ([\#11546](https://github.com/matrix-org/synapse/issues/11546), [\#11549](https://github.com/matrix-org/synapse/issues/11549), [\#11551](https://github.com/matrix-org/synapse/issues/11551), [\#11555](https://github.com/matrix-org/synapse/issues/11555), [\#11575](https://github.com/matrix-org/synapse/issues/11575), [\#11589](https://github.com/matrix-org/synapse/issues/11589), [\#11594](https://github.com/matrix-org/synapse/issues/11594), [\#11652](https://github.com/matrix-org/synapse/issues/11652), [\#11653](https://github.com/matrix-org/synapse/issues/11653), [\#11654](https://github.com/matrix-org/synapse/issues/11654), [\#11657](https://github.com/matrix-org/synapse/issues/11657))
|
||||
- Fix an inaccurate and misleading comment in the `/sync` code. ([\#11550](https://github.com/matrix-org/synapse/issues/11550))
|
||||
- Add missing type hints to `synapse.logging.context`. ([\#11556](https://github.com/matrix-org/synapse/issues/11556))
|
||||
- Stop populating unused database column `state_events.prev_state`. ([\#11558](https://github.com/matrix-org/synapse/issues/11558))
|
||||
- Minor efficiency improvements in event persistence. ([\#11560](https://github.com/matrix-org/synapse/issues/11560))
|
||||
- Add some safety checks that storage functions are used correctly. ([\#11564](https://github.com/matrix-org/synapse/issues/11564), [\#11580](https://github.com/matrix-org/synapse/issues/11580))
|
||||
- Make `get_device` return `None` if the device doesn't exist rather than raising an exception. ([\#11565](https://github.com/matrix-org/synapse/issues/11565))
|
||||
- Split the HTML parsing code from the URL preview resource code. ([\#11566](https://github.com/matrix-org/synapse/issues/11566))
|
||||
- Remove redundant `COALESCE()`s around `COUNT()`s in database queries. ([\#11570](https://github.com/matrix-org/synapse/issues/11570))
|
||||
- Add missing type hints to `synapse.http`. ([\#11571](https://github.com/matrix-org/synapse/issues/11571))
|
||||
- Add [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) and [MSC3030](https://github.com/matrix-org/matrix-doc/pull/3030) to `/versions` -> `unstable_features` to detect server support. ([\#11582](https://github.com/matrix-org/synapse/issues/11582))
|
||||
- Add type hints to `synapse/tests/rest/admin`. ([\#11590](https://github.com/matrix-org/synapse/issues/11590))
|
||||
- Drop end-of-life Python 3.6 and Postgres 9.6 from CI. ([\#11595](https://github.com/matrix-org/synapse/issues/11595))
|
||||
- Update black version and run it on all the files. ([\#11596](https://github.com/matrix-org/synapse/issues/11596))
|
||||
- Add opentracing type stubs and fix associated mypy errors. ([\#11603](https://github.com/matrix-org/synapse/issues/11603), [\#11622](https://github.com/matrix-org/synapse/issues/11622))
|
||||
- Improve OpenTracing support for requests which use a `ResponseCache`. ([\#11607](https://github.com/matrix-org/synapse/issues/11607))
|
||||
- Improve OpenTracing support for incoming HTTP requests. ([\#11618](https://github.com/matrix-org/synapse/issues/11618))
|
||||
- A number of improvements to opentracing support. ([\#11619](https://github.com/matrix-org/synapse/issues/11619))
|
||||
- Refactor the way that the `outlier` flag is set on events received over federation. ([\#11634](https://github.com/matrix-org/synapse/issues/11634))
|
||||
- Improve the error messages from `get_create_event_for_room`. ([\#11638](https://github.com/matrix-org/synapse/issues/11638))
|
||||
- Remove redundant `get_current_events_token` method. ([\#11643](https://github.com/matrix-org/synapse/issues/11643))
|
||||
- Convert `namedtuples` to `attrs`. ([\#11665](https://github.com/matrix-org/synapse/issues/11665), [\#11574](https://github.com/matrix-org/synapse/issues/11574))
|
||||
- Update the `/capabilities` response to include whether support for [MSC3440](https://github.com/matrix-org/matrix-doc/pull/3440) is available. ([\#11690](https://github.com/matrix-org/synapse/issues/11690))
|
||||
- Send the `Accept` header in HTTP requests made using `SimpleHttpClient.get_json`. ([\#11677](https://github.com/matrix-org/synapse/issues/11677))
|
||||
- Work around Mjolnir compatibility issue by adding an import for `glob_to_regex` in `synapse.util`, where it moved from. ([\#11696](https://github.com/matrix-org/synapse/issues/11696))
|
||||
|
||||
|
||||
Synapse 1.49.2 (2021-12-21)
|
||||
===========================
|
||||
|
||||
This release fixes a regression introduced in Synapse 1.49.0 which could cause `/sync` requests to take significantly longer. This would particularly affect "initial" syncs for users participating in a large number of rooms, and in extreme cases, could make it impossible for such users to log in on a new client.
|
||||
|
||||
**Note:** in line with our [deprecation policy](https://matrix-org.github.io/synapse/latest/deprecation_policy.html) for platform dependencies, this will be the last release to support Python 3.6 and PostgreSQL 9.6, both of which have now reached upstream end-of-life. Synapse will require Python 3.7+ and PostgreSQL 10+.
|
||||
|
||||
**Note:** We will also stop producing packages for Ubuntu 18.04 (Bionic Beaver) after this release, as it uses Python 3.6.
|
||||
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Fix a performance regression in `/sync` handling, introduced in 1.49.0. ([\#11583](https://github.com/matrix-org/synapse/issues/11583))
|
||||
|
||||
Internal Changes
|
||||
----------------
|
||||
|
||||
- Work around a build problem on Debian Buster. ([\#11625](https://github.com/matrix-org/synapse/issues/11625))
|
||||
|
||||
|
||||
Synapse 1.49.1 (2021-12-21)
|
||||
===========================
|
||||
|
||||
Not released due to problems building the debian packages.
|
||||
|
||||
|
||||
Synapse 1.49.0 (2021-12-14)
|
||||
===========================
|
||||
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
Send and handle cross-signing messages using the stable prefix.
|
||||
@@ -1 +0,0 @@
|
||||
Allow specific, experimental events to be created without `prev_events`. Used by [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716).
|
||||
@@ -1 +0,0 @@
|
||||
A test helper (`wait_for_background_updates`) no longer depends on classes defining a `store` property.
|
||||
@@ -1 +0,0 @@
|
||||
Add type hints to `synapse.appservice`.
|
||||
@@ -1 +0,0 @@
|
||||
Allow guests to send state events per [MSC3419](https://github.com/matrix-org/matrix-doc/pull/3419).
|
||||
@@ -1 +0,0 @@
|
||||
Document the usage of refresh tokens.
|
||||
@@ -1 +0,0 @@
|
||||
Add missing type hints to `synapse.config` module.
|
||||
@@ -1 +0,0 @@
|
||||
Add test to ensure we share the same `state_group` across the whole historical batch when using the [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) `/batch_send` endpoint.
|
||||
@@ -1 +0,0 @@
|
||||
Fix a long-standing bug where relations from other rooms could be included in the bundled aggregations of an event.
|
||||
@@ -1 +0,0 @@
|
||||
Use HTTPStatus constants in place of literals in `tests.rest.client.test_auth`.
|
||||
2
changelog.d/11530.bugfix
Normal file
2
changelog.d/11530.bugfix
Normal file
@@ -0,0 +1,2 @@
|
||||
Fix a long-standing issue which could cause Synapse to incorrectly accept data in the unsigned field of events
|
||||
received over federation.
|
||||
@@ -1 +0,0 @@
|
||||
Add a receipt types constant for `m.read`.
|
||||
@@ -1 +0,0 @@
|
||||
Clean up `synapse.rest.admin`.
|
||||
@@ -1 +0,0 @@
|
||||
Improvements to log messages around handling stream ids.
|
||||
@@ -1 +0,0 @@
|
||||
Add experimental support for MSC3202: allowing application services to masquerade as specific devices.
|
||||
@@ -1 +0,0 @@
|
||||
Support unprefixed versions of fallback key property names.
|
||||
@@ -1 +0,0 @@
|
||||
Add missing `errcode` to `parse_string` and `parse_boolean`.
|
||||
@@ -1 +0,0 @@
|
||||
Use HTTPStatus constants in place of literals in `synapse.http`.
|
||||
@@ -1 +0,0 @@
|
||||
Add missing type hints to storage classes.
|
||||
@@ -1 +0,0 @@
|
||||
Fix a bug introduced in Synapse 1.17.0 where a pusher created for an email with capital letters would fail to be created.
|
||||
@@ -1 +0,0 @@
|
||||
Add missing type hints to storage classes.
|
||||
@@ -1 +0,0 @@
|
||||
Fix an inaccurate and misleading comment in the `/sync` code.
|
||||
@@ -1 +0,0 @@
|
||||
Add missing type hints to storage classes.
|
||||
@@ -1 +0,0 @@
|
||||
Add missing type hints to storage classes.
|
||||
@@ -1 +0,0 @@
|
||||
Add missing type hints to `synapse.logging.context`.
|
||||
@@ -1 +0,0 @@
|
||||
Stop populating unused database column `state_events.prev_state`.
|
||||
@@ -1 +0,0 @@
|
||||
Minor efficiency improvements in event persistence.
|
||||
1
changelog.d/11561.feature
Normal file
1
changelog.d/11561.feature
Normal file
@@ -0,0 +1 @@
|
||||
Add `track_puppeted_user_ips` config flag to track puppeted user IP addresses. This also includes them in monthly active user counts.
|
||||
@@ -1 +0,0 @@
|
||||
Add some safety checks that storage functions are used correctly.
|
||||
@@ -1 +0,0 @@
|
||||
Make `get_device` return `None` if the device doesn't exist rather than raising an exception.
|
||||
@@ -1 +0,0 @@
|
||||
Split the HTML parsing code from the URL preview resource code.
|
||||
@@ -1 +0,0 @@
|
||||
Remove redundant `COALESCE()`s around `COUNT()`s in database queries.
|
||||
@@ -1 +0,0 @@
|
||||
Add missing type hints to `synapse.http`.
|
||||
@@ -1 +0,0 @@
|
||||
Convert `EventStreamResult` from a `namedtuple` to `attrs` to improve type hints.
|
||||
@@ -1 +0,0 @@
|
||||
Add missing type hints to storage classes.
|
||||
@@ -1 +0,0 @@
|
||||
Add some safety checks that storage functions are used correctly.
|
||||
1
changelog.d/11587.bugfix
Normal file
1
changelog.d/11587.bugfix
Normal file
@@ -0,0 +1 @@
|
||||
Fix a long-standing bug where Synapse wouldn't cache a response indicating that a remote user has no devices.
|
||||
1
changelog.d/11593.bugfix
Normal file
1
changelog.d/11593.bugfix
Normal file
@@ -0,0 +1 @@
|
||||
Fix an error in to get federation status of a destination server even if no error has occurred. This admin API was new introduced in Synapse 1.49.0.
|
||||
1
changelog.d/11612.misc
Normal file
1
changelog.d/11612.misc
Normal file
@@ -0,0 +1 @@
|
||||
Avoid database access in the JSON serialization process.
|
||||
1
changelog.d/11667.bugfix
Normal file
1
changelog.d/11667.bugfix
Normal file
@@ -0,0 +1 @@
|
||||
Fix `/_matrix/client/v1/room/{roomId}/hierarchy` endpoint returning incorrect fields which have been present since Synapse 1.49.0.
|
||||
1
changelog.d/11672.feature
Normal file
1
changelog.d/11672.feature
Normal file
@@ -0,0 +1 @@
|
||||
Return an `M_FORBIDDEN` error code instead of `M_UNKNOWN` when a spam checker module prevents a user from creating a room.
|
||||
1
changelog.d/11682.removal
Normal file
1
changelog.d/11682.removal
Normal file
@@ -0,0 +1 @@
|
||||
Remove the unstable `/send_relation` endpoint.
|
||||
1
changelog.d/11685.misc
Normal file
1
changelog.d/11685.misc
Normal file
@@ -0,0 +1 @@
|
||||
Run `pyupgrade --py37-plus --keep-percent-format` on Synapse.
|
||||
1
changelog.d/11691.misc
Normal file
1
changelog.d/11691.misc
Normal file
@@ -0,0 +1 @@
|
||||
Use buildkit's cache feature to speed up docker builds.
|
||||
1
changelog.d/11693.misc
Normal file
1
changelog.d/11693.misc
Normal file
@@ -0,0 +1 @@
|
||||
Remove debug logging for #4422, which has been closed since Synapse 0.99.
|
||||
1
changelog.d/11695.bugfix
Normal file
1
changelog.d/11695.bugfix
Normal file
@@ -0,0 +1 @@
|
||||
Fix a bug where the only the first 50 rooms from a space were returned from the `/hierarchy` API. This has existed since the introduction of the API in Synapse v1.41.0.
|
||||
1
changelog.d/11699.misc
Normal file
1
changelog.d/11699.misc
Normal file
@@ -0,0 +1 @@
|
||||
Remove fallback code for Python 2.
|
||||
1
changelog.d/11701.misc
Normal file
1
changelog.d/11701.misc
Normal file
@@ -0,0 +1 @@
|
||||
Add a test for [an edge case](https://github.com/matrix-org/synapse/pull/11532#discussion_r769104461) in the `/sync` logic.
|
||||
1
changelog.d/11702.misc
Normal file
1
changelog.d/11702.misc
Normal file
@@ -0,0 +1 @@
|
||||
Add the option to write sqlite test dbs to disk when running tests.
|
||||
1
changelog.d/11707.misc
Normal file
1
changelog.d/11707.misc
Normal file
@@ -0,0 +1 @@
|
||||
Improve Complement test output for Gitub Actions.
|
||||
1
changelog.d/11714.misc
Normal file
1
changelog.d/11714.misc
Normal file
@@ -0,0 +1 @@
|
||||
Fix a typechecker problem related to our (ab)use of `nacl.signing.SigningKey`s.
|
||||
1
changelog.d/11715.doc
Normal file
1
changelog.d/11715.doc
Normal file
@@ -0,0 +1 @@
|
||||
Document the new `SYNAPSE_TEST_PERSIST_SQLITE_DB` environment variable in the contributing guide.
|
||||
1
changelog.d/11716.misc
Normal file
1
changelog.d/11716.misc
Normal file
@@ -0,0 +1 @@
|
||||
Fix docstring on `add_account_data_for_user`.
|
||||
1
changelog.d/11718.misc
Normal file
1
changelog.d/11718.misc
Normal file
@@ -0,0 +1 @@
|
||||
Complement environment variable name change and update `.gitignore`.
|
||||
1
changelog.d/11725.doc
Normal file
1
changelog.d/11725.doc
Normal file
@@ -0,0 +1 @@
|
||||
Document that now the minimum supported PostgreSQL version is 10.
|
||||
@@ -14,6 +14,7 @@ services:
|
||||
# failure
|
||||
restart: unless-stopped
|
||||
# See the readme for a full documentation of the environment settings
|
||||
# NOTE: You must edit homeserver.yaml to use postgres, it defaults to sqlite
|
||||
environment:
|
||||
- SYNAPSE_CONFIG_PATH=/data/homeserver.yaml
|
||||
volumes:
|
||||
|
||||
18
debian/changelog
vendored
18
debian/changelog
vendored
@@ -1,3 +1,21 @@
|
||||
matrix-synapse-py3 (1.50.0~rc1) stable; urgency=medium
|
||||
|
||||
* New synapse release 1.50.0~rc1.
|
||||
|
||||
-- Synapse Packaging team <packages@matrix.org> Wed, 05 Jan 2022 12:36:17 +0000
|
||||
|
||||
matrix-synapse-py3 (1.49.2) stable; urgency=medium
|
||||
|
||||
* New synapse release 1.49.2.
|
||||
|
||||
-- Synapse Packaging team <packages@matrix.org> Tue, 21 Dec 2021 17:31:03 +0000
|
||||
|
||||
matrix-synapse-py3 (1.49.1) stable; urgency=medium
|
||||
|
||||
* New synapse release 1.49.1.
|
||||
|
||||
-- Synapse Packaging team <packages@matrix.org> Tue, 21 Dec 2021 11:07:30 +0000
|
||||
|
||||
matrix-synapse-py3 (1.49.0) stable; urgency=medium
|
||||
|
||||
* New synapse release 1.49.0.
|
||||
|
||||
@@ -1,14 +1,17 @@
|
||||
# Dockerfile to build the matrixdotorg/synapse docker images.
|
||||
#
|
||||
# Note that it uses features which are only available in BuildKit - see
|
||||
# https://docs.docker.com/go/buildkit/ for more information.
|
||||
#
|
||||
# To build the image, run `docker build` command from the root of the
|
||||
# synapse repository:
|
||||
#
|
||||
# docker build -f docker/Dockerfile .
|
||||
# DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile .
|
||||
#
|
||||
# There is an optional PYTHON_VERSION build argument which sets the
|
||||
# version of python to build against: for example:
|
||||
#
|
||||
# docker build -f docker/Dockerfile --build-arg PYTHON_VERSION=3.6 .
|
||||
# DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile --build-arg PYTHON_VERSION=3.9 .
|
||||
#
|
||||
|
||||
ARG PYTHON_VERSION=3.8
|
||||
@@ -19,7 +22,16 @@ ARG PYTHON_VERSION=3.8
|
||||
FROM docker.io/python:${PYTHON_VERSION}-slim as builder
|
||||
|
||||
# install the OS build deps
|
||||
RUN apt-get update && apt-get install -y \
|
||||
#
|
||||
# RUN --mount is specific to buildkit and is documented at
|
||||
# https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/syntax.md#build-mounts-run---mount.
|
||||
# Here we use it to set up a cache for apt, to improve rebuild speeds on
|
||||
# slow connections.
|
||||
#
|
||||
RUN \
|
||||
--mount=type=cache,target=/var/cache/apt,sharing=locked \
|
||||
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
||||
apt-get update && apt-get install -y \
|
||||
build-essential \
|
||||
libffi-dev \
|
||||
libjpeg-dev \
|
||||
@@ -44,7 +56,8 @@ COPY synapse/python_dependencies.py /synapse/synapse/python_dependencies.py
|
||||
# used while you develop on the source
|
||||
#
|
||||
# This is aiming at installing the `install_requires` and `extras_require` from `setup.py`
|
||||
RUN pip install --prefix="/install" --no-warn-script-location \
|
||||
RUN --mount=type=cache,target=/root/.cache/pip \
|
||||
pip install --prefix="/install" --no-warn-script-location \
|
||||
/synapse[all]
|
||||
|
||||
# Copy over the rest of the project
|
||||
@@ -66,7 +79,10 @@ LABEL org.opencontainers.image.documentation='https://github.com/matrix-org/syna
|
||||
LABEL org.opencontainers.image.source='https://github.com/matrix-org/synapse.git'
|
||||
LABEL org.opencontainers.image.licenses='Apache-2.0'
|
||||
|
||||
RUN apt-get update && apt-get install -y \
|
||||
RUN \
|
||||
--mount=type=cache,target=/var/cache/apt,sharing=locked \
|
||||
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
||||
apt-get update && apt-get install -y \
|
||||
curl \
|
||||
gosu \
|
||||
libjpeg62-turbo \
|
||||
|
||||
@@ -16,7 +16,7 @@ ARG distro=""
|
||||
### Stage 0: build a dh-virtualenv
|
||||
###
|
||||
|
||||
# This is only really needed on bionic and focal, since other distributions we
|
||||
# This is only really needed on focal, since other distributions we
|
||||
# care about have a recent version of dh-virtualenv by default. Unfortunately,
|
||||
# it looks like focal is going to be with us for a while.
|
||||
#
|
||||
@@ -36,9 +36,8 @@ RUN env DEBIAN_FRONTEND=noninteractive apt-get install \
|
||||
wget
|
||||
|
||||
# fetch and unpack the package
|
||||
# TODO: Upgrade to 1.2.2 once bionic is dropped (1.2.2 requires debhelper 12; bionic has only 11)
|
||||
RUN mkdir /dh-virtualenv
|
||||
RUN wget -q -O /dh-virtualenv.tar.gz https://github.com/spotify/dh-virtualenv/archive/ac6e1b1.tar.gz
|
||||
RUN wget -q -O /dh-virtualenv.tar.gz https://github.com/spotify/dh-virtualenv/archive/refs/tags/1.2.2.tar.gz
|
||||
RUN tar -xv --strip-components=1 -C /dh-virtualenv -f /dh-virtualenv.tar.gz
|
||||
|
||||
# install its build deps. We do another apt-cache-update here, because we might
|
||||
@@ -86,12 +85,12 @@ RUN apt-get update -qq -o Acquire::Languages=none \
|
||||
libpq-dev \
|
||||
xmlsec1
|
||||
|
||||
COPY --from=builder /dh-virtualenv_1.2~dev-1_all.deb /
|
||||
COPY --from=builder /dh-virtualenv_1.2.2-1_all.deb /
|
||||
|
||||
# install dhvirtualenv. Update the apt cache again first, in case we got a
|
||||
# cached cache from docker the first time.
|
||||
RUN apt-get update -qq -o Acquire::Languages=none \
|
||||
&& apt-get install -yq /dh-virtualenv_1.2~dev-1_all.deb
|
||||
&& apt-get install -yq /dh-virtualenv_1.2.2-1_all.deb
|
||||
|
||||
WORKDIR /synapse/source
|
||||
ENTRYPOINT ["bash","/synapse/source/docker/build_debian.sh"]
|
||||
|
||||
@@ -68,6 +68,10 @@ The following environment variables are supported in `generate` mode:
|
||||
directories. If unset, and no user is set via `docker run --user`, defaults
|
||||
to `991`, `991`.
|
||||
|
||||
## Postgres
|
||||
|
||||
By default the config will use SQLite. See the [docs on using Postgres](https://github.com/matrix-org/synapse/blob/develop/docs/postgres.md) for more info on how to use Postgres. Until this section is improved [this issue](https://github.com/matrix-org/synapse/issues/8304) may provide useful information.
|
||||
|
||||
## Running synapse
|
||||
|
||||
Once you have a valid configuration file, you can start synapse as follows:
|
||||
|
||||
@@ -480,6 +480,81 @@ The following fields are returned in the JSON response body:
|
||||
- `joined_rooms` - An array of `room_id`.
|
||||
- `total` - Number of rooms.
|
||||
|
||||
## Account Data
|
||||
Gets information about account data for a specific `user_id`.
|
||||
|
||||
The API is:
|
||||
|
||||
```
|
||||
GET /_synapse/admin/v1/users/<user_id>/accountdata
|
||||
```
|
||||
|
||||
A response body like the following is returned:
|
||||
|
||||
```json
|
||||
{
|
||||
"account_data": {
|
||||
"global": {
|
||||
"m.secret_storage.key.LmIGHTg5W": {
|
||||
"algorithm": "m.secret_storage.v1.aes-hmac-sha2",
|
||||
"iv": "fwjNZatxg==",
|
||||
"mac": "eWh9kNnLWZUNOgnc="
|
||||
},
|
||||
"im.vector.hide_profile": {
|
||||
"hide_profile": true
|
||||
},
|
||||
"org.matrix.preview_urls": {
|
||||
"disable": false
|
||||
},
|
||||
"im.vector.riot.breadcrumb_rooms": {
|
||||
"rooms": [
|
||||
"!LxcBDAsDUVAfJDEo:matrix.org",
|
||||
"!MAhRxqasbItjOqxu:matrix.org"
|
||||
]
|
||||
},
|
||||
"m.accepted_terms": {
|
||||
"accepted": [
|
||||
"https://example.org/somewhere/privacy-1.2-en.html",
|
||||
"https://example.org/somewhere/terms-2.0-en.html"
|
||||
]
|
||||
},
|
||||
"im.vector.setting.breadcrumbs": {
|
||||
"recent_rooms": [
|
||||
"!MAhRxqasbItqxuEt:matrix.org",
|
||||
"!ZtSaPCawyWtxiImy:matrix.org"
|
||||
]
|
||||
}
|
||||
},
|
||||
"rooms": {
|
||||
"!GUdfZSHUJibpiVqHYd:matrix.org": {
|
||||
"m.fully_read": {
|
||||
"event_id": "$156334540fYIhZ:matrix.org"
|
||||
}
|
||||
},
|
||||
"!tOZwOOiqwCYQkLhV:matrix.org": {
|
||||
"m.fully_read": {
|
||||
"event_id": "$xjsIyp4_NaVl2yPvIZs_k1Jl8tsC_Sp23wjqXPno"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Parameters**
|
||||
|
||||
The following parameters should be set in the URL:
|
||||
|
||||
- `user_id` - fully qualified: for example, `@user:server.com`.
|
||||
|
||||
**Response**
|
||||
|
||||
The following fields are returned in the JSON response body:
|
||||
|
||||
- `account_data` - A map containing the account data for the user
|
||||
- `global` - A map containing the global account data for the user
|
||||
- `rooms` - A map containing the account data per room for the user
|
||||
|
||||
## User media
|
||||
|
||||
### List media uploaded by a user
|
||||
|
||||
@@ -14,8 +14,8 @@ i.e. when a version reaches End of Life Synapse will withdraw support for that
|
||||
version in future releases.
|
||||
|
||||
Details on the upstream support life cycles for Python and PostgreSQL are
|
||||
documented at https://endoflife.date/python and
|
||||
https://endoflife.date/postgresql.
|
||||
documented at [https://endoflife.date/python](https://endoflife.date/python) and
|
||||
[https://endoflife.date/postgresql](https://endoflife.date/postgresql).
|
||||
|
||||
|
||||
Context
|
||||
|
||||
@@ -169,6 +169,27 @@ To increase the log level for the tests, set `SYNAPSE_TEST_LOG_LEVEL`:
|
||||
SYNAPSE_TEST_LOG_LEVEL=DEBUG trial tests
|
||||
```
|
||||
|
||||
By default, tests will use an in-memory SQLite database for test data. For additional
|
||||
help with debugging, one can use an on-disk SQLite database file instead, in order to
|
||||
review database state during and after running tests. This can be done by setting
|
||||
the `SYNAPSE_TEST_PERSIST_SQLITE_DB` environment variable. Doing so will cause the
|
||||
database state to be stored in a file named `test.db` under the trial process'
|
||||
working directory. Typically, this ends up being `_trial_temp/test.db`. For example:
|
||||
|
||||
```sh
|
||||
SYNAPSE_TEST_PERSIST_SQLITE_DB=1 trial tests
|
||||
```
|
||||
|
||||
The database file can then be inspected with:
|
||||
|
||||
```sh
|
||||
sqlite3 _trial_temp/test.db
|
||||
```
|
||||
|
||||
Note that the database file is cleared at the beginning of each test run. Thus it
|
||||
will always only contain the data generated by the *last run test*. Though generally
|
||||
when debugging, one is only running a single test anyway.
|
||||
|
||||
### Running tests under PostgreSQL
|
||||
|
||||
Invoking `trial` as above will use an in-memory SQLite database. This is great for
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Using Postgres
|
||||
|
||||
Synapse supports PostgreSQL versions 9.6 or later.
|
||||
Synapse supports PostgreSQL versions 10 or later.
|
||||
|
||||
## Install postgres client libraries
|
||||
|
||||
|
||||
@@ -63,7 +63,7 @@ server {
|
||||
|
||||
server_name matrix.example.com;
|
||||
|
||||
location ~* ^(\/_matrix|\/_synapse\/client) {
|
||||
location ~ ^(/_matrix|/_synapse/client) {
|
||||
# note: do not add a path (even a single /) after the port in `proxy_pass`,
|
||||
# otherwise nginx will canonicalise the URI and cause signature verification
|
||||
# errors.
|
||||
|
||||
@@ -37,7 +37,7 @@
|
||||
|
||||
# Server admins can expand Synapse's functionality with external modules.
|
||||
#
|
||||
# See https://matrix-org.github.io/synapse/latest/modules.html for more
|
||||
# See https://matrix-org.github.io/synapse/latest/modules/index.html for more
|
||||
# documentation on how to configure or create custom modules for Synapse.
|
||||
#
|
||||
modules:
|
||||
@@ -1488,6 +1488,7 @@ room_prejoin_state:
|
||||
# - m.room.encryption
|
||||
# - m.room.name
|
||||
# - m.room.create
|
||||
# - m.room.topic
|
||||
#
|
||||
# Uncomment the following to disable these defaults (so that only the event
|
||||
# types listed in 'additional_event_types' are shared). Defaults to 'false'.
|
||||
@@ -1502,6 +1503,12 @@ room_prejoin_state:
|
||||
#additional_event_types:
|
||||
# - org.example.custom.event.type
|
||||
|
||||
# If enabled, puppeted user IP's can also be tracked. By default when
|
||||
# puppeting another user, the user who has created the access token
|
||||
# for puppeting is tracked. If this is enabled, both requests are tracked.
|
||||
# Implicitly enables MAU tracking for puppeted users.
|
||||
#track_puppeted_user_ips: false
|
||||
|
||||
|
||||
# A list of application service config files to use
|
||||
#
|
||||
|
||||
@@ -164,7 +164,7 @@ xbps-install -S synapse
|
||||
Synapse can be installed via FreeBSD Ports or Packages contributed by Brendan Molloy from:
|
||||
|
||||
- Ports: `cd /usr/ports/net-im/py-matrix-synapse && make install clean`
|
||||
- Packages: `pkg install py37-matrix-synapse`
|
||||
- Packages: `pkg install py38-matrix-synapse`
|
||||
|
||||
#### OpenBSD
|
||||
|
||||
|
||||
@@ -49,12 +49,12 @@ comment these options out and use those specified by the module instead.
|
||||
|
||||
A custom mapping provider must specify the following methods:
|
||||
|
||||
* `__init__(self, parsed_config)`
|
||||
* `def __init__(self, parsed_config)`
|
||||
- Arguments:
|
||||
- `parsed_config` - A configuration object that is the return value of the
|
||||
`parse_config` method. You should set any configuration options needed by
|
||||
the module here.
|
||||
* `parse_config(config)`
|
||||
* `def parse_config(config)`
|
||||
- This method should have the `@staticmethod` decoration.
|
||||
- Arguments:
|
||||
- `config` - A `dict` representing the parsed content of the
|
||||
@@ -63,13 +63,13 @@ A custom mapping provider must specify the following methods:
|
||||
any option values they need here.
|
||||
- Whatever is returned will be passed back to the user mapping provider module's
|
||||
`__init__` method during construction.
|
||||
* `get_remote_user_id(self, userinfo)`
|
||||
* `def get_remote_user_id(self, userinfo)`
|
||||
- Arguments:
|
||||
- `userinfo` - A `authlib.oidc.core.claims.UserInfo` object to extract user
|
||||
information from.
|
||||
- This method must return a string, which is the unique, immutable identifier
|
||||
for the user. Commonly the `sub` claim of the response.
|
||||
* `map_user_attributes(self, userinfo, token, failures)`
|
||||
* `async def map_user_attributes(self, userinfo, token, failures)`
|
||||
- This method must be async.
|
||||
- Arguments:
|
||||
- `userinfo` - A `authlib.oidc.core.claims.UserInfo` object to extract user
|
||||
@@ -91,7 +91,7 @@ A custom mapping provider must specify the following methods:
|
||||
during a user's first login. Once a localpart has been associated with a
|
||||
remote user ID (see `get_remote_user_id`) it cannot be updated.
|
||||
- `displayname`: An optional string, the display name for the user.
|
||||
* `get_extra_attributes(self, userinfo, token)`
|
||||
* `async def get_extra_attributes(self, userinfo, token)`
|
||||
- This method must be async.
|
||||
- Arguments:
|
||||
- `userinfo` - A `authlib.oidc.core.claims.UserInfo` object to extract user
|
||||
@@ -125,15 +125,15 @@ comment these options out and use those specified by the module instead.
|
||||
|
||||
A custom mapping provider must specify the following methods:
|
||||
|
||||
* `__init__(self, parsed_config, module_api)`
|
||||
* `def __init__(self, parsed_config, module_api)`
|
||||
- Arguments:
|
||||
- `parsed_config` - A configuration object that is the return value of the
|
||||
`parse_config` method. You should set any configuration options needed by
|
||||
the module here.
|
||||
- `module_api` - a `synapse.module_api.ModuleApi` object which provides the
|
||||
stable API available for extension modules.
|
||||
* `parse_config(config)`
|
||||
- This method should have the `@staticmethod` decoration.
|
||||
* `def parse_config(config)`
|
||||
- **This method should have the `@staticmethod` decoration.**
|
||||
- Arguments:
|
||||
- `config` - A `dict` representing the parsed content of the
|
||||
`saml_config.user_mapping_provider.config` homeserver config option.
|
||||
@@ -141,15 +141,15 @@ A custom mapping provider must specify the following methods:
|
||||
any option values they need here.
|
||||
- Whatever is returned will be passed back to the user mapping provider module's
|
||||
`__init__` method during construction.
|
||||
* `get_saml_attributes(config)`
|
||||
- This method should have the `@staticmethod` decoration.
|
||||
* `def get_saml_attributes(config)`
|
||||
- **This method should have the `@staticmethod` decoration.**
|
||||
- Arguments:
|
||||
- `config` - A object resulting from a call to `parse_config`.
|
||||
- Returns a tuple of two sets. The first set equates to the SAML auth
|
||||
response attributes that are required for the module to function, whereas
|
||||
the second set consists of those attributes which can be used if available,
|
||||
but are not necessary.
|
||||
* `get_remote_user_id(self, saml_response, client_redirect_url)`
|
||||
* `def get_remote_user_id(self, saml_response, client_redirect_url)`
|
||||
- Arguments:
|
||||
- `saml_response` - A `saml2.response.AuthnResponse` object to extract user
|
||||
information from.
|
||||
@@ -157,7 +157,7 @@ A custom mapping provider must specify the following methods:
|
||||
redirected to.
|
||||
- This method must return a string, which is the unique, immutable identifier
|
||||
for the user. Commonly the `uid` claim of the response.
|
||||
* `saml_response_to_user_attributes(self, saml_response, failures, client_redirect_url)`
|
||||
* `def saml_response_to_user_attributes(self, saml_response, failures, client_redirect_url)`
|
||||
- Arguments:
|
||||
- `saml_response` - A `saml2.response.AuthnResponse` object to extract user
|
||||
information from.
|
||||
|
||||
@@ -15,8 +15,8 @@ The following sections describe how to install [coturn](<https://github.com/cotu
|
||||
|
||||
For TURN relaying with `coturn` to work, it must be hosted on a server/endpoint with a public IP.
|
||||
|
||||
Hosting TURN behind a NAT (even with appropriate port forwarding) is known to cause issues
|
||||
and to often not work.
|
||||
Hosting TURN behind NAT requires port forwaring and for the NAT gateway to have a public IP.
|
||||
However, even with appropriate configuration, NAT is known to cause issues and to often not work.
|
||||
|
||||
## `coturn` setup
|
||||
|
||||
@@ -103,7 +103,23 @@ This will install and start a systemd service called `coturn`.
|
||||
denied-peer-ip=192.168.0.0-192.168.255.255
|
||||
denied-peer-ip=172.16.0.0-172.31.255.255
|
||||
|
||||
# recommended additional local peers to block, to mitigate external access to internal services.
|
||||
# https://www.rtcsec.com/article/slack-webrtc-turn-compromise-and-bug-bounty/#how-to-fix-an-open-turn-relay-to-address-this-vulnerability
|
||||
no-multicast-peers
|
||||
denied-peer-ip=0.0.0.0-0.255.255.255
|
||||
denied-peer-ip=100.64.0.0-100.127.255.255
|
||||
denied-peer-ip=127.0.0.0-127.255.255.255
|
||||
denied-peer-ip=169.254.0.0-169.254.255.255
|
||||
denied-peer-ip=192.0.0.0-192.0.0.255
|
||||
denied-peer-ip=192.0.2.0-192.0.2.255
|
||||
denied-peer-ip=192.88.99.0-192.88.99.255
|
||||
denied-peer-ip=198.18.0.0-198.19.255.255
|
||||
denied-peer-ip=198.51.100.0-198.51.100.255
|
||||
denied-peer-ip=203.0.113.0-203.0.113.255
|
||||
denied-peer-ip=240.0.0.0-255.255.255.255
|
||||
|
||||
# special case the turn server itself so that client->TURN->TURN->client flows work
|
||||
# this should be one of the turn server's listening IPs
|
||||
allowed-peer-ip=10.0.0.1
|
||||
|
||||
# consider whether you want to limit the quota of relayed streams per user (or total) to avoid risk of DoS.
|
||||
@@ -123,7 +139,7 @@ This will install and start a systemd service called `coturn`.
|
||||
pkey=/path/to/privkey.pem
|
||||
```
|
||||
|
||||
In this case, replace the `turn:` schemes in the `turn_uri` settings below
|
||||
In this case, replace the `turn:` schemes in the `turn_uris` settings below
|
||||
with `turns:`.
|
||||
|
||||
We recommend that you only try to set up TLS/DTLS once you have set up a
|
||||
@@ -134,21 +150,33 @@ This will install and start a systemd service called `coturn`.
|
||||
traffic (remember to allow both TCP and UDP traffic), and ports 49152-65535
|
||||
for the UDP relay.)
|
||||
|
||||
1. We do not recommend running a TURN server behind NAT, and are not aware of
|
||||
anyone doing so successfully.
|
||||
|
||||
If you want to try it anyway, you will at least need to tell coturn its
|
||||
external IP address:
|
||||
1. If your TURN server is behind NAT, the NAT gateway must have an external,
|
||||
publicly-reachable IP address. You must configure coturn to advertise that
|
||||
address to connecting clients:
|
||||
|
||||
```
|
||||
external-ip=192.88.99.1
|
||||
external-ip=EXTERNAL_NAT_IPv4_ADDRESS
|
||||
```
|
||||
|
||||
... and your NAT gateway must forward all of the relayed ports directly
|
||||
(eg, port 56789 on the external IP must be always be forwarded to port
|
||||
56789 on the internal IP).
|
||||
You may optionally limit the TURN server to listen only on the local
|
||||
address that is mapped by NAT to the external address:
|
||||
|
||||
If you get this working, let us know!
|
||||
```
|
||||
listening-ip=INTERNAL_TURNSERVER_IPv4_ADDRESS
|
||||
```
|
||||
|
||||
If your NAT gateway is reachable over both IPv4 and IPv6, you may
|
||||
configure coturn to advertise each available address:
|
||||
|
||||
```
|
||||
external-ip=EXTERNAL_NAT_IPv4_ADDRESS
|
||||
external-ip=EXTERNAL_NAT_IPv6_ADDRESS
|
||||
```
|
||||
|
||||
When advertising an external IPv6 address, ensure that the firewall and
|
||||
network settings of the system running your TURN server are configured to
|
||||
accept IPv6 traffic, and that the TURN server is listening on the local
|
||||
IPv6 address that is mapped by NAT to the external IPv6 address.
|
||||
|
||||
1. (Re)start the turn server:
|
||||
|
||||
@@ -216,9 +244,6 @@ connecting". Unfortunately, troubleshooting this can be tricky.
|
||||
|
||||
Here are a few things to try:
|
||||
|
||||
* Check that your TURN server is not behind NAT. As above, we're not aware of
|
||||
anyone who has successfully set this up.
|
||||
|
||||
* Check that you have opened your firewall to allow TCP and UDP traffic to the
|
||||
TURN ports (normally 3478 and 5349).
|
||||
|
||||
@@ -234,6 +259,18 @@ Here are a few things to try:
|
||||
Try removing any AAAA records for your TURN server, so that it is only
|
||||
reachable over IPv4.
|
||||
|
||||
* If your TURN server is behind NAT:
|
||||
|
||||
* double-check that your NAT gateway is correctly forwarding all TURN
|
||||
ports (normally 3478 & 5349 for TCP & UDP TURN traffic, and 49152-65535 for the UDP
|
||||
relay) to the NAT-internal address of your TURN server. If advertising
|
||||
both IPv4 and IPv6 external addresses via the `external-ip` option, ensure
|
||||
that the NAT is forwarding both IPv4 and IPv6 traffic to the IPv4 and IPv6
|
||||
internal addresses of your TURN server. When in doubt, remove AAAA records
|
||||
for your TURN server and specify only an IPv4 address as your `external-ip`.
|
||||
|
||||
* ensure that your TURN server uses the NAT gateway as its default route.
|
||||
|
||||
* Enable more verbose logging in coturn via the `verbose` setting:
|
||||
|
||||
```
|
||||
|
||||
@@ -85,6 +85,17 @@ process, for example:
|
||||
dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
|
||||
```
|
||||
|
||||
# Upgrading to v1.50.0
|
||||
|
||||
## Dropping support for old Python and Postgres versions
|
||||
|
||||
In line with our [deprecation policy](deprecation_policy.md),
|
||||
we've dropped support for Python 3.6 and PostgreSQL 9.6, as they are no
|
||||
longer supported upstream.
|
||||
|
||||
This release of Synapse requires Python 3.7+ and PostgreSQL 10+.
|
||||
|
||||
|
||||
# Upgrading to v1.47.0
|
||||
|
||||
## Removal of old Room Admin API
|
||||
|
||||
23
mypy.ini
23
mypy.ini
@@ -28,8 +28,6 @@ exclude = (?x)
|
||||
|synapse/storage/databases/main/cache.py
|
||||
|synapse/storage/databases/main/devices.py
|
||||
|synapse/storage/databases/main/event_federation.py
|
||||
|synapse/storage/databases/main/event_push_actions.py
|
||||
|synapse/storage/databases/main/events_bg_updates.py
|
||||
|synapse/storage/databases/main/group_server.py
|
||||
|synapse/storage/databases/main/metrics.py
|
||||
|synapse/storage/databases/main/monthly_active_users.py
|
||||
@@ -40,8 +38,6 @@ exclude = (?x)
|
||||
|synapse/storage/databases/main/roommember.py
|
||||
|synapse/storage/databases/main/search.py
|
||||
|synapse/storage/databases/main/state.py
|
||||
|synapse/storage/databases/main/stats.py
|
||||
|synapse/storage/databases/main/transactions.py
|
||||
|synapse/storage/databases/main/user_directory.py
|
||||
|synapse/storage/schema/
|
||||
|
||||
@@ -103,7 +99,6 @@ exclude = (?x)
|
||||
|tests/server.py
|
||||
|tests/server_notices/test_resource_limits_server_notices.py
|
||||
|tests/state/test_v2.py
|
||||
|tests/storage/test_account_data.py
|
||||
|tests/storage/test_background_update.py
|
||||
|tests/storage/test_base.py
|
||||
|tests/storage/test_client_ips.py
|
||||
@@ -201,6 +196,12 @@ disallow_untyped_defs = True
|
||||
[mypy-synapse.storage.databases.main.end_to_end_keys]
|
||||
disallow_untyped_defs = True
|
||||
|
||||
[mypy-synapse.storage.databases.main.event_push_actions]
|
||||
disallow_untyped_defs = True
|
||||
|
||||
[mypy-synapse.storage.databases.main.events_bg_updates]
|
||||
disallow_untyped_defs = True
|
||||
|
||||
[mypy-synapse.storage.databases.main.events_worker]
|
||||
disallow_untyped_defs = True
|
||||
|
||||
@@ -213,9 +214,15 @@ disallow_untyped_defs = True
|
||||
[mypy-synapse.storage.databases.main.profile]
|
||||
disallow_untyped_defs = True
|
||||
|
||||
[mypy-synapse.storage.databases.main.stats]
|
||||
disallow_untyped_defs = True
|
||||
|
||||
[mypy-synapse.storage.databases.main.state_deltas]
|
||||
disallow_untyped_defs = True
|
||||
|
||||
[mypy-synapse.storage.databases.main.transactions]
|
||||
disallow_untyped_defs = True
|
||||
|
||||
[mypy-synapse.storage.databases.main.user_erasure_store]
|
||||
disallow_untyped_defs = True
|
||||
|
||||
@@ -240,6 +247,9 @@ disallow_untyped_defs = True
|
||||
[mypy-tests.storage.test_user_directory]
|
||||
disallow_untyped_defs = True
|
||||
|
||||
[mypy-tests.rest.admin.*]
|
||||
disallow_untyped_defs = True
|
||||
|
||||
[mypy-tests.rest.client.test_directory]
|
||||
disallow_untyped_defs = True
|
||||
|
||||
@@ -303,9 +313,6 @@ ignore_missing_imports = True
|
||||
[mypy-netaddr]
|
||||
ignore_missing_imports = True
|
||||
|
||||
[mypy-opentracing]
|
||||
ignore_missing_imports = True
|
||||
|
||||
[mypy-parameterized.*]
|
||||
ignore_missing_imports = True
|
||||
|
||||
|
||||
@@ -35,7 +35,7 @@
|
||||
showcontent = true
|
||||
|
||||
[tool.black]
|
||||
target-version = ['py36']
|
||||
target-version = ['py37', 'py38', 'py39', 'py310']
|
||||
exclude = '''
|
||||
|
||||
(
|
||||
|
||||
@@ -24,7 +24,6 @@ DISTS = (
|
||||
"debian:bullseye",
|
||||
"debian:bookworm",
|
||||
"debian:sid",
|
||||
"ubuntu:bionic", # 18.04 LTS (our EOL forced by Py36 on 2021-12-23)
|
||||
"ubuntu:focal", # 20.04 LTS (our EOL forced by Py38 on 2024-10-14)
|
||||
"ubuntu:hirsute", # 21.04 (EOL 2022-01-05)
|
||||
"ubuntu:impish", # 21.10 (EOL 2022-07)
|
||||
|
||||
@@ -42,8 +42,8 @@ echo "--------------------------"
|
||||
echo
|
||||
|
||||
matched=0
|
||||
for f in $(git diff --name-only FETCH_HEAD... -- changelog.d); do
|
||||
# check that any modified newsfiles on this branch end with a full stop.
|
||||
for f in $(git diff --diff-filter=d --name-only FETCH_HEAD... -- changelog.d); do
|
||||
# check that any added newsfiles on this branch end with a full stop.
|
||||
lastchar=$(tr -d '\n' < "$f" | tail -c 1)
|
||||
if [ "$lastchar" != '.' ] && [ "$lastchar" != '!' ]; then
|
||||
echo -e "\e[31mERROR: newsfragment $f does not end with a '.' or '!'\e[39m" >&2
|
||||
|
||||
@@ -23,6 +23,9 @@
|
||||
# Exit if a line returns a non-zero exit code
|
||||
set -e
|
||||
|
||||
# enable buildkit for the docker builds
|
||||
export DOCKER_BUILDKIT=1
|
||||
|
||||
# Change to the repository root
|
||||
cd "$(dirname $0)/.."
|
||||
|
||||
@@ -47,7 +50,7 @@ if [[ -n "$WORKERS" ]]; then
|
||||
COMPLEMENT_DOCKERFILE=SynapseWorkers.Dockerfile
|
||||
# And provide some more configuration to complement.
|
||||
export COMPLEMENT_CA=true
|
||||
export COMPLEMENT_VERSION_CHECK_ITERATIONS=500
|
||||
export COMPLEMENT_SPAWN_HS_TIMEOUT_SECS=25
|
||||
else
|
||||
export COMPLEMENT_BASE_IMAGE=complement-synapse
|
||||
COMPLEMENT_DOCKERFILE=Synapse.Dockerfile
|
||||
@@ -65,4 +68,5 @@ if [[ -n "$1" ]]; then
|
||||
fi
|
||||
|
||||
# Run the tests!
|
||||
echo "Images built; running complement"
|
||||
go test -v -tags synapse_blacklist,msc2403,msc2716 -count=1 $EXTRA_COMPLEMENT_ARGS ./tests/...
|
||||
|
||||
8
setup.py
8
setup.py
@@ -96,7 +96,7 @@ CONDITIONAL_REQUIREMENTS["all"] = list(ALL_OPTIONAL_REQUIREMENTS)
|
||||
# We pin black so that our tests don't start failing on new releases.
|
||||
CONDITIONAL_REQUIREMENTS["lint"] = [
|
||||
"isort==5.7.0",
|
||||
"black==21.6b0",
|
||||
"black==21.12b0",
|
||||
"flake8-comprehensions",
|
||||
"flake8-bugbear==21.3.2",
|
||||
"flake8",
|
||||
@@ -107,6 +107,7 @@ CONDITIONAL_REQUIREMENTS["mypy"] = [
|
||||
"mypy-zope==0.3.2",
|
||||
"types-bleach>=4.1.0",
|
||||
"types-jsonschema>=3.2.0",
|
||||
"types-opentracing>=2.4.2",
|
||||
"types-Pillow>=8.3.4",
|
||||
"types-pyOpenSSL>=20.0.7",
|
||||
"types-PyYAML>=5.4.10",
|
||||
@@ -119,9 +120,7 @@ CONDITIONAL_REQUIREMENTS["mypy"] = [
|
||||
# Tests assume that all optional dependencies are installed.
|
||||
#
|
||||
# parameterized_class decorator was introduced in parameterized 0.7.0
|
||||
#
|
||||
# We use `mock` library as that backports `AsyncMock` to Python 3.6
|
||||
CONDITIONAL_REQUIREMENTS["test"] = ["parameterized>=0.7.0", "mock>=4.0.0"]
|
||||
CONDITIONAL_REQUIREMENTS["test"] = ["parameterized>=0.7.0"]
|
||||
|
||||
CONDITIONAL_REQUIREMENTS["dev"] = (
|
||||
CONDITIONAL_REQUIREMENTS["lint"]
|
||||
@@ -163,7 +162,6 @@ setup(
|
||||
"Topic :: Communications :: Chat",
|
||||
"License :: OSI Approved :: Apache Software License",
|
||||
"Programming Language :: Python :: 3 :: Only",
|
||||
"Programming Language :: Python :: 3.6",
|
||||
"Programming Language :: Python :: 3.7",
|
||||
"Programming Language :: Python :: 3.8",
|
||||
"Programming Language :: Python :: 3.9",
|
||||
|
||||
@@ -47,7 +47,7 @@ try:
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
__version__ = "1.49.0"
|
||||
__version__ = "1.50.0rc1"
|
||||
|
||||
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
|
||||
# We import here so that we don't have to install a bunch of deps when
|
||||
|
||||
@@ -32,7 +32,7 @@ from synapse.appservice import ApplicationService
|
||||
from synapse.events import EventBase
|
||||
from synapse.http import get_request_user_agent
|
||||
from synapse.http.site import SynapseRequest
|
||||
from synapse.logging import opentracing as opentracing
|
||||
from synapse.logging.opentracing import active_span, force_tracing, start_active_span
|
||||
from synapse.storage.databases.main.registration import TokenLookupResult
|
||||
from synapse.types import Requester, StateMap, UserID, create_requester
|
||||
from synapse.util.caches.lrucache import LruCache
|
||||
@@ -71,6 +71,7 @@ class Auth:
|
||||
self._auth_blocking = AuthBlocking(self.hs)
|
||||
|
||||
self._track_appservice_user_ips = hs.config.appservice.track_appservice_user_ips
|
||||
self._track_puppeted_user_ips = hs.config.api.track_puppeted_user_ips
|
||||
self._macaroon_secret_key = hs.config.key.macaroon_secret_key
|
||||
self._force_tracing_for_users = hs.config.tracing.force_tracing_for_users
|
||||
|
||||
@@ -149,6 +150,42 @@ class Auth:
|
||||
is invalid.
|
||||
AuthError if access is denied for the user in the access token
|
||||
"""
|
||||
parent_span = active_span()
|
||||
with start_active_span("get_user_by_req"):
|
||||
requester = await self._wrapped_get_user_by_req(
|
||||
request, allow_guest, rights, allow_expired
|
||||
)
|
||||
|
||||
if parent_span:
|
||||
if requester.authenticated_entity in self._force_tracing_for_users:
|
||||
# request tracing is enabled for this user, so we need to force it
|
||||
# tracing on for the parent span (which will be the servlet span).
|
||||
#
|
||||
# It's too late for the get_user_by_req span to inherit the setting,
|
||||
# so we also force it on for that.
|
||||
force_tracing()
|
||||
force_tracing(parent_span)
|
||||
parent_span.set_tag(
|
||||
"authenticated_entity", requester.authenticated_entity
|
||||
)
|
||||
parent_span.set_tag("user_id", requester.user.to_string())
|
||||
if requester.device_id is not None:
|
||||
parent_span.set_tag("device_id", requester.device_id)
|
||||
if requester.app_service is not None:
|
||||
parent_span.set_tag("appservice_id", requester.app_service.id)
|
||||
return requester
|
||||
|
||||
async def _wrapped_get_user_by_req(
|
||||
self,
|
||||
request: SynapseRequest,
|
||||
allow_guest: bool,
|
||||
rights: str,
|
||||
allow_expired: bool,
|
||||
) -> Requester:
|
||||
"""Helper for get_user_by_req
|
||||
|
||||
Once get_user_by_req has set up the opentracing span, this does the actual work.
|
||||
"""
|
||||
try:
|
||||
ip_addr = request.getClientIP()
|
||||
user_agent = get_request_user_agent(request)
|
||||
@@ -177,14 +214,6 @@ class Auth:
|
||||
)
|
||||
|
||||
request.requester = user_id
|
||||
if user_id in self._force_tracing_for_users:
|
||||
opentracing.force_tracing()
|
||||
opentracing.set_tag("authenticated_entity", user_id)
|
||||
opentracing.set_tag("user_id", user_id)
|
||||
if device_id is not None:
|
||||
opentracing.set_tag("device_id", device_id)
|
||||
opentracing.set_tag("appservice_id", app_service.id)
|
||||
|
||||
return requester
|
||||
|
||||
user_info = await self.get_user_by_access_token(
|
||||
@@ -218,6 +247,18 @@ class Auth:
|
||||
user_agent=user_agent,
|
||||
device_id=device_id,
|
||||
)
|
||||
# Track also the puppeted user client IP if enabled and the user is puppeting
|
||||
if (
|
||||
user_info.user_id != user_info.token_owner
|
||||
and self._track_puppeted_user_ips
|
||||
):
|
||||
await self.store.insert_client_ip(
|
||||
user_id=user_info.user_id,
|
||||
access_token=access_token,
|
||||
ip=ip_addr,
|
||||
user_agent=user_agent,
|
||||
device_id=device_id,
|
||||
)
|
||||
|
||||
if is_guest and not allow_guest:
|
||||
raise AuthError(
|
||||
@@ -242,13 +283,6 @@ class Auth:
|
||||
)
|
||||
|
||||
request.requester = requester
|
||||
if user_info.token_owner in self._force_tracing_for_users:
|
||||
opentracing.force_tracing()
|
||||
opentracing.set_tag("authenticated_entity", user_info.token_owner)
|
||||
opentracing.set_tag("user_id", user_info.user_id)
|
||||
if device_id:
|
||||
opentracing.set_tag("device_id", device_id)
|
||||
|
||||
return requester
|
||||
except KeyError:
|
||||
raise MissingClientTokenError()
|
||||
|
||||
@@ -351,8 +351,7 @@ class Filter:
|
||||
True if the event matches the filter.
|
||||
"""
|
||||
# We usually get the full "events" as dictionaries coming through,
|
||||
# except for presence which actually gets passed around as its own
|
||||
# namedtuple type.
|
||||
# except for presence which actually gets passed around as its own type.
|
||||
if isinstance(event, UserPresenceState):
|
||||
user_id = event.user_id
|
||||
field_matchers = {
|
||||
|
||||
@@ -27,6 +27,7 @@ import synapse
|
||||
import synapse.config.logger
|
||||
from synapse import events
|
||||
from synapse.api.urls import (
|
||||
CLIENT_API_PREFIX,
|
||||
FEDERATION_PREFIX,
|
||||
LEGACY_MEDIA_PREFIX,
|
||||
MEDIA_R0_PREFIX,
|
||||
@@ -192,13 +193,7 @@ class SynapseHomeServer(HomeServer):
|
||||
|
||||
resources.update(
|
||||
{
|
||||
"/_matrix/client/api/v1": client_resource,
|
||||
"/_matrix/client/r0": client_resource,
|
||||
"/_matrix/client/v1": client_resource,
|
||||
"/_matrix/client/v3": client_resource,
|
||||
"/_matrix/client/unstable": client_resource,
|
||||
"/_matrix/client/v2_alpha": client_resource,
|
||||
"/_matrix/client/versions": client_resource,
|
||||
CLIENT_API_PREFIX: client_resource,
|
||||
"/.well-known": well_known_resource(self),
|
||||
"/_synapse/admin": AdminRestResource(self),
|
||||
**build_synapse_client_resource_tree(self),
|
||||
|
||||
@@ -29,6 +29,7 @@ class ApiConfig(Config):
|
||||
def read_config(self, config: JsonDict, **kwargs):
|
||||
validate_config(_MAIN_SCHEMA, config, ())
|
||||
self.room_prejoin_state = list(self._get_prejoin_state_types(config))
|
||||
self.track_puppeted_user_ips = config.get("track_puppeted_user_ips", False)
|
||||
|
||||
def generate_config_section(cls, **kwargs) -> str:
|
||||
formatted_default_state_types = "\n".join(
|
||||
@@ -59,6 +60,12 @@ class ApiConfig(Config):
|
||||
#
|
||||
#additional_event_types:
|
||||
# - org.example.custom.event.type
|
||||
|
||||
# If enabled, puppeted user IP's can also be tracked. By default when
|
||||
# puppeting another user, the user who has created the access token
|
||||
# for puppeting is tracked. If this is enabled, both requests are tracked.
|
||||
# Implicitly enables MAU tracking for puppeted users.
|
||||
#track_puppeted_user_ips: false
|
||||
""" % {
|
||||
"formatted_default_state_types": formatted_default_state_types
|
||||
}
|
||||
@@ -107,6 +114,8 @@ _DEFAULT_PREJOIN_STATE_TYPES = [
|
||||
EventTypes.Name,
|
||||
# Per MSC1772.
|
||||
EventTypes.Create,
|
||||
# Per MSC3173.
|
||||
EventTypes.Topic,
|
||||
]
|
||||
|
||||
|
||||
@@ -136,5 +145,8 @@ _MAIN_SCHEMA = {
|
||||
"properties": {
|
||||
"room_prejoin_state": _ROOM_PREJOIN_STATE_CONFIG_SCHEMA,
|
||||
"room_invite_state_types": _ROOM_INVITE_STATE_TYPES_SCHEMA,
|
||||
"track_puppeted_user_ips": {
|
||||
"type": "boolean",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
@@ -32,7 +32,7 @@ class ExperimentalConfig(Config):
|
||||
# MSC3026 (busy presence state)
|
||||
self.msc3026_enabled: bool = experimental.get("msc3026_enabled", False)
|
||||
|
||||
# MSC2716 (backfill existing history)
|
||||
# MSC2716 (importing historical messages)
|
||||
self.msc2716_enabled: bool = experimental.get("msc2716_enabled", False)
|
||||
|
||||
# MSC2285 (hidden read receipts)
|
||||
|
||||
@@ -37,7 +37,7 @@ class ModulesConfig(Config):
|
||||
|
||||
# Server admins can expand Synapse's functionality with external modules.
|
||||
#
|
||||
# See https://matrix-org.github.io/synapse/latest/modules.html for more
|
||||
# See https://matrix-org.github.io/synapse/latest/modules/index.html for more
|
||||
# documentation on how to configure or create custom modules for Synapse.
|
||||
#
|
||||
modules:
|
||||
|
||||
@@ -14,10 +14,11 @@
|
||||
|
||||
import logging
|
||||
import os
|
||||
from collections import namedtuple
|
||||
from typing import Dict, List, Tuple
|
||||
from urllib.request import getproxies_environment # type: ignore
|
||||
|
||||
import attr
|
||||
|
||||
from synapse.config.server import DEFAULT_IP_RANGE_BLACKLIST, generate_ip_set
|
||||
from synapse.python_dependencies import DependencyException, check_requirements
|
||||
from synapse.types import JsonDict
|
||||
@@ -44,18 +45,20 @@ THUMBNAIL_SIZE_YAML = """\
|
||||
HTTP_PROXY_SET_WARNING = """\
|
||||
The Synapse config url_preview_ip_range_blacklist will be ignored as an HTTP(s) proxy is configured."""
|
||||
|
||||
ThumbnailRequirement = namedtuple(
|
||||
"ThumbnailRequirement", ["width", "height", "method", "media_type"]
|
||||
)
|
||||
|
||||
MediaStorageProviderConfig = namedtuple(
|
||||
"MediaStorageProviderConfig",
|
||||
(
|
||||
"store_local", # Whether to store newly uploaded local files
|
||||
"store_remote", # Whether to store newly downloaded remote files
|
||||
"store_synchronous", # Whether to wait for successful storage for local uploads
|
||||
),
|
||||
)
|
||||
@attr.s(frozen=True, slots=True, auto_attribs=True)
|
||||
class ThumbnailRequirement:
|
||||
width: int
|
||||
height: int
|
||||
method: str
|
||||
media_type: str
|
||||
|
||||
|
||||
@attr.s(frozen=True, slots=True, auto_attribs=True)
|
||||
class MediaStorageProviderConfig:
|
||||
store_local: bool # Whether to store newly uploaded local files
|
||||
store_remote: bool # Whether to store newly downloaded remote files
|
||||
store_synchronous: bool # Whether to wait for successful storage for local uploads
|
||||
|
||||
|
||||
def parse_thumbnail_requirements(
|
||||
@@ -66,11 +69,10 @@ def parse_thumbnail_requirements(
|
||||
method, and thumbnail media type to precalculate
|
||||
|
||||
Args:
|
||||
thumbnail_sizes(list): List of dicts with "width", "height", and
|
||||
"method" keys
|
||||
thumbnail_sizes: List of dicts with "width", "height", and "method" keys
|
||||
|
||||
Returns:
|
||||
Dictionary mapping from media type string to list of
|
||||
ThumbnailRequirement tuples.
|
||||
Dictionary mapping from media type string to list of ThumbnailRequirement.
|
||||
"""
|
||||
requirements: Dict[str, List[ThumbnailRequirement]] = {}
|
||||
for size in thumbnail_sizes:
|
||||
|
||||
@@ -15,8 +15,9 @@
|
||||
|
||||
from typing import List
|
||||
|
||||
from matrix_common.regex import glob_to_regex
|
||||
|
||||
from synapse.types import JsonDict
|
||||
from synapse.util import glob_to_regex
|
||||
|
||||
from ._base import Config, ConfigError
|
||||
|
||||
|
||||
@@ -16,11 +16,12 @@ import logging
|
||||
import os
|
||||
from typing import List, Optional, Pattern
|
||||
|
||||
from matrix_common.regex import glob_to_regex
|
||||
|
||||
from OpenSSL import SSL, crypto
|
||||
from twisted.internet._sslverify import Certificate, trustRootFromCertificates
|
||||
|
||||
from synapse.config._base import Config, ConfigError
|
||||
from synapse.util import glob_to_regex
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@@ -14,17 +14,7 @@
|
||||
# limitations under the License.
|
||||
import collections.abc
|
||||
import re
|
||||
from typing import (
|
||||
TYPE_CHECKING,
|
||||
Any,
|
||||
Callable,
|
||||
Dict,
|
||||
Iterable,
|
||||
List,
|
||||
Mapping,
|
||||
Optional,
|
||||
Union,
|
||||
)
|
||||
from typing import Any, Callable, Dict, Iterable, List, Mapping, Optional, Union
|
||||
|
||||
from frozendict import frozendict
|
||||
|
||||
@@ -32,14 +22,10 @@ from synapse.api.constants import EventContentFields, EventTypes, RelationTypes
|
||||
from synapse.api.errors import Codes, SynapseError
|
||||
from synapse.api.room_versions import RoomVersion
|
||||
from synapse.types import JsonDict
|
||||
from synapse.util.async_helpers import yieldable_gather_results
|
||||
from synapse.util.frozenutils import unfreeze
|
||||
|
||||
from . import EventBase
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from synapse.server import HomeServer
|
||||
|
||||
# Split strings on "." but not "\." This uses a negative lookbehind assertion for '\'
|
||||
# (?<!stuff) matches if the current position in the string is not preceded
|
||||
# by a match for 'stuff'.
|
||||
@@ -385,17 +371,12 @@ class EventClientSerializer:
|
||||
clients.
|
||||
"""
|
||||
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self.store = hs.get_datastore()
|
||||
self._msc1849_enabled = hs.config.experimental.msc1849_enabled
|
||||
self._msc3440_enabled = hs.config.experimental.msc3440_enabled
|
||||
|
||||
async def serialize_event(
|
||||
def serialize_event(
|
||||
self,
|
||||
event: Union[JsonDict, EventBase],
|
||||
time_now: int,
|
||||
*,
|
||||
bundle_aggregations: bool = True,
|
||||
bundle_aggregations: Optional[Dict[str, JsonDict]] = None,
|
||||
**kwargs: Any,
|
||||
) -> JsonDict:
|
||||
"""Serializes a single event.
|
||||
@@ -418,66 +399,41 @@ class EventClientSerializer:
|
||||
serialized_event = serialize_event(event, time_now, **kwargs)
|
||||
|
||||
# Check if there are any bundled aggregations to include with the event.
|
||||
#
|
||||
# Do not bundle aggregations if any of the following at true:
|
||||
#
|
||||
# * Support is disabled via the configuration or the caller.
|
||||
# * The event is a state event.
|
||||
# * The event has been redacted.
|
||||
if (
|
||||
self._msc1849_enabled
|
||||
and bundle_aggregations
|
||||
and not event.is_state()
|
||||
and not event.internal_metadata.is_redacted()
|
||||
):
|
||||
await self._injected_bundled_aggregations(event, time_now, serialized_event)
|
||||
if bundle_aggregations:
|
||||
event_aggregations = bundle_aggregations.get(event.event_id)
|
||||
if event_aggregations:
|
||||
self._injected_bundled_aggregations(
|
||||
event,
|
||||
time_now,
|
||||
bundle_aggregations[event.event_id],
|
||||
serialized_event,
|
||||
)
|
||||
|
||||
return serialized_event
|
||||
|
||||
async def _injected_bundled_aggregations(
|
||||
self, event: EventBase, time_now: int, serialized_event: JsonDict
|
||||
def _injected_bundled_aggregations(
|
||||
self,
|
||||
event: EventBase,
|
||||
time_now: int,
|
||||
aggregations: JsonDict,
|
||||
serialized_event: JsonDict,
|
||||
) -> None:
|
||||
"""Potentially injects bundled aggregations into the unsigned portion of the serialized event.
|
||||
|
||||
Args:
|
||||
event: The event being serialized.
|
||||
time_now: The current time in milliseconds
|
||||
aggregations: The bundled aggregation to serialize.
|
||||
serialized_event: The serialized event which may be modified.
|
||||
|
||||
"""
|
||||
# Do not bundle aggregations for an event which represents an edit or an
|
||||
# annotation. It does not make sense for them to have related events.
|
||||
relates_to = event.content.get("m.relates_to")
|
||||
if isinstance(relates_to, (dict, frozendict)):
|
||||
relation_type = relates_to.get("rel_type")
|
||||
if relation_type in (RelationTypes.ANNOTATION, RelationTypes.REPLACE):
|
||||
return
|
||||
# Make a copy in-case the object is cached.
|
||||
aggregations = aggregations.copy()
|
||||
|
||||
event_id = event.event_id
|
||||
room_id = event.room_id
|
||||
|
||||
# The bundled aggregations to include.
|
||||
aggregations = {}
|
||||
|
||||
annotations = await self.store.get_aggregation_groups_for_event(
|
||||
event_id, room_id
|
||||
)
|
||||
if annotations.chunk:
|
||||
aggregations[RelationTypes.ANNOTATION] = annotations.to_dict()
|
||||
|
||||
references = await self.store.get_relations_for_event(
|
||||
event_id, room_id, RelationTypes.REFERENCE, direction="f"
|
||||
)
|
||||
if references.chunk:
|
||||
aggregations[RelationTypes.REFERENCE] = references.to_dict()
|
||||
|
||||
edit = None
|
||||
if event.type == EventTypes.Message:
|
||||
edit = await self.store.get_applicable_edit(event_id, room_id)
|
||||
|
||||
if edit:
|
||||
if RelationTypes.REPLACE in aggregations:
|
||||
# If there is an edit replace the content, preserving existing
|
||||
# relations.
|
||||
edit = aggregations[RelationTypes.REPLACE]
|
||||
|
||||
# Ensure we take copies of the edit content, otherwise we risk modifying
|
||||
# the original event.
|
||||
@@ -502,27 +458,19 @@ class EventClientSerializer:
|
||||
}
|
||||
|
||||
# If this event is the start of a thread, include a summary of the replies.
|
||||
if self._msc3440_enabled:
|
||||
(
|
||||
thread_count,
|
||||
latest_thread_event,
|
||||
) = await self.store.get_thread_summary(event_id, room_id)
|
||||
if latest_thread_event:
|
||||
aggregations[RelationTypes.THREAD] = {
|
||||
# Don't bundle aggregations as this could recurse forever.
|
||||
"latest_event": await self.serialize_event(
|
||||
latest_thread_event, time_now, bundle_aggregations=False
|
||||
),
|
||||
"count": thread_count,
|
||||
}
|
||||
if RelationTypes.THREAD in aggregations:
|
||||
# Serialize the latest thread event.
|
||||
latest_thread_event = aggregations[RelationTypes.THREAD]["latest_event"]
|
||||
|
||||
# If any bundled aggregations were found, include them.
|
||||
if aggregations:
|
||||
serialized_event["unsigned"].setdefault("m.relations", {}).update(
|
||||
aggregations
|
||||
# Don't bundle aggregations as this could recurse forever.
|
||||
aggregations[RelationTypes.THREAD]["latest_event"] = self.serialize_event(
|
||||
latest_thread_event, time_now, bundle_aggregations=None
|
||||
)
|
||||
|
||||
async def serialize_events(
|
||||
# Include the bundled aggregations in the event.
|
||||
serialized_event["unsigned"].setdefault("m.relations", {}).update(aggregations)
|
||||
|
||||
def serialize_events(
|
||||
self, events: Iterable[Union[JsonDict, EventBase]], time_now: int, **kwargs: Any
|
||||
) -> List[JsonDict]:
|
||||
"""Serializes multiple events.
|
||||
@@ -535,9 +483,9 @@ class EventClientSerializer:
|
||||
Returns:
|
||||
The list of serialized events
|
||||
"""
|
||||
return await yieldable_gather_results(
|
||||
self.serialize_event, events, time_now=time_now, **kwargs
|
||||
)
|
||||
return [
|
||||
self.serialize_event(event, time_now=time_now, **kwargs) for event in events
|
||||
]
|
||||
|
||||
|
||||
def copy_power_levels_contents(
|
||||
|
||||
@@ -13,7 +13,6 @@
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
import logging
|
||||
from collections import namedtuple
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from synapse.api.constants import MAX_DEPTH, EventContentFields, EventTypes, Membership
|
||||
@@ -104,10 +103,6 @@ class FederationBase:
|
||||
return pdu
|
||||
|
||||
|
||||
class PduToCheckSig(namedtuple("PduToCheckSig", ["pdu", "sender_domain", "deferreds"])):
|
||||
pass
|
||||
|
||||
|
||||
async def _check_sigs_on_pdu(
|
||||
keyring: Keyring, room_version: RoomVersion, pdu: EventBase
|
||||
) -> None:
|
||||
@@ -220,15 +215,12 @@ def _is_invite_via_3pid(event: EventBase) -> bool:
|
||||
)
|
||||
|
||||
|
||||
def event_from_pdu_json(
|
||||
pdu_json: JsonDict, room_version: RoomVersion, outlier: bool = False
|
||||
) -> EventBase:
|
||||
def event_from_pdu_json(pdu_json: JsonDict, room_version: RoomVersion) -> EventBase:
|
||||
"""Construct an EventBase from an event json received over federation
|
||||
|
||||
Args:
|
||||
pdu_json: pdu as received over federation
|
||||
room_version: The version of the room this event belongs to
|
||||
outlier: True to mark this event as an outlier
|
||||
|
||||
Raises:
|
||||
SynapseError: if the pdu is missing required fields or is otherwise
|
||||
@@ -238,6 +230,10 @@ def event_from_pdu_json(
|
||||
# origin, etc etc)
|
||||
assert_params_in_dict(pdu_json, ("type", "depth"))
|
||||
|
||||
# Strip any unauthorized values from "unsigned" if they exist
|
||||
if "unsigned" in pdu_json:
|
||||
_strip_unsigned_values(pdu_json)
|
||||
|
||||
depth = pdu_json["depth"]
|
||||
if not isinstance(depth, int):
|
||||
raise SynapseError(400, "Depth %r not an intger" % (depth,), Codes.BAD_JSON)
|
||||
@@ -252,6 +248,25 @@ def event_from_pdu_json(
|
||||
validate_canonicaljson(pdu_json)
|
||||
|
||||
event = make_event_from_dict(pdu_json, room_version)
|
||||
event.internal_metadata.outlier = outlier
|
||||
|
||||
return event
|
||||
|
||||
|
||||
def _strip_unsigned_values(pdu_dict: JsonDict) -> None:
|
||||
"""
|
||||
Strip any unsigned values unless specifically allowed, as defined by the whitelist.
|
||||
|
||||
pdu: the json dict to strip values from. Note that the dict is mutated by this
|
||||
function
|
||||
"""
|
||||
unsigned = pdu_dict["unsigned"]
|
||||
|
||||
if not isinstance(unsigned, dict):
|
||||
pdu_dict["unsigned"] = {}
|
||||
|
||||
if pdu_dict["type"] == "m.room.member":
|
||||
whitelist = ["knock_room_state", "invite_room_state", "age"]
|
||||
else:
|
||||
whitelist = ["age"]
|
||||
|
||||
filtered_unsigned = {k: v for k, v in unsigned.items() if k in whitelist}
|
||||
pdu_dict["unsigned"] = filtered_unsigned
|
||||
|
||||
@@ -265,14 +265,11 @@ class FederationClient(FederationBase):
|
||||
|
||||
room_version = await self.store.get_room_version(room_id)
|
||||
|
||||
pdus = [
|
||||
event_from_pdu_json(p, room_version, outlier=False)
|
||||
for p in transaction_data_pdus
|
||||
]
|
||||
pdus = [event_from_pdu_json(p, room_version) for p in transaction_data_pdus]
|
||||
|
||||
# Check signatures and hash of pdus, removing any from the list that fail checks
|
||||
pdus[:] = await self._check_sigs_and_hash_and_fetch(
|
||||
dest, pdus, outlier=True, room_version=room_version
|
||||
dest, pdus, room_version=room_version
|
||||
)
|
||||
|
||||
return pdus
|
||||
@@ -282,7 +279,6 @@ class FederationClient(FederationBase):
|
||||
destination: str,
|
||||
event_id: str,
|
||||
room_version: RoomVersion,
|
||||
outlier: bool = False,
|
||||
timeout: Optional[int] = None,
|
||||
) -> Optional[EventBase]:
|
||||
"""Requests the PDU with given origin and ID from the remote home
|
||||
@@ -292,9 +288,6 @@ class FederationClient(FederationBase):
|
||||
destination: Which homeserver to query
|
||||
event_id: event to fetch
|
||||
room_version: version of the room
|
||||
outlier: Indicates whether the PDU is an `outlier`, i.e. if
|
||||
it's from an arbitrary point in the context as opposed to part
|
||||
of the current block of PDUs. Defaults to `False`
|
||||
timeout: How long to try (in ms) each destination for before
|
||||
moving to the next destination. None indicates no timeout.
|
||||
|
||||
@@ -316,8 +309,7 @@ class FederationClient(FederationBase):
|
||||
)
|
||||
|
||||
pdu_list: List[EventBase] = [
|
||||
event_from_pdu_json(p, room_version, outlier=outlier)
|
||||
for p in transaction_data["pdus"]
|
||||
event_from_pdu_json(p, room_version) for p in transaction_data["pdus"]
|
||||
]
|
||||
|
||||
if pdu_list and pdu_list[0]:
|
||||
@@ -334,7 +326,6 @@ class FederationClient(FederationBase):
|
||||
destinations: Iterable[str],
|
||||
event_id: str,
|
||||
room_version: RoomVersion,
|
||||
outlier: bool = False,
|
||||
timeout: Optional[int] = None,
|
||||
) -> Optional[EventBase]:
|
||||
"""Requests the PDU with given origin and ID from the remote home
|
||||
@@ -347,9 +338,6 @@ class FederationClient(FederationBase):
|
||||
destinations: Which homeservers to query
|
||||
event_id: event to fetch
|
||||
room_version: version of the room
|
||||
outlier: Indicates whether the PDU is an `outlier`, i.e. if
|
||||
it's from an arbitrary point in the context as opposed to part
|
||||
of the current block of PDUs. Defaults to `False`
|
||||
timeout: How long to try (in ms) each destination for before
|
||||
moving to the next destination. None indicates no timeout.
|
||||
|
||||
@@ -377,7 +365,6 @@ class FederationClient(FederationBase):
|
||||
destination=destination,
|
||||
event_id=event_id,
|
||||
room_version=room_version,
|
||||
outlier=outlier,
|
||||
timeout=timeout,
|
||||
)
|
||||
|
||||
@@ -435,7 +422,6 @@ class FederationClient(FederationBase):
|
||||
origin: str,
|
||||
pdus: Collection[EventBase],
|
||||
room_version: RoomVersion,
|
||||
outlier: bool = False,
|
||||
) -> List[EventBase]:
|
||||
"""Takes a list of PDUs and checks the signatures and hashes of each
|
||||
one. If a PDU fails its signature check then we check if we have it in
|
||||
@@ -451,7 +437,6 @@ class FederationClient(FederationBase):
|
||||
origin
|
||||
pdu
|
||||
room_version
|
||||
outlier: Whether the events are outliers or not
|
||||
|
||||
Returns:
|
||||
A list of PDUs that have valid signatures and hashes.
|
||||
@@ -466,7 +451,6 @@ class FederationClient(FederationBase):
|
||||
valid_pdu = await self._check_sigs_and_hash_and_fetch_one(
|
||||
pdu=pdu,
|
||||
origin=origin,
|
||||
outlier=outlier,
|
||||
room_version=room_version,
|
||||
)
|
||||
|
||||
@@ -482,7 +466,6 @@ class FederationClient(FederationBase):
|
||||
pdu: EventBase,
|
||||
origin: str,
|
||||
room_version: RoomVersion,
|
||||
outlier: bool = False,
|
||||
) -> Optional[EventBase]:
|
||||
"""Takes a PDU and checks its signatures and hashes. If the PDU fails
|
||||
its signature check then we check if we have it in the database and if
|
||||
@@ -494,9 +477,6 @@ class FederationClient(FederationBase):
|
||||
origin
|
||||
pdu
|
||||
room_version
|
||||
outlier: Whether the events are outliers or not
|
||||
include_none: Whether to include None in the returned list
|
||||
for events that have failed their checks
|
||||
|
||||
Returns:
|
||||
The PDU (possibly redacted) if it has valid signatures and hashes.
|
||||
@@ -521,7 +501,6 @@ class FederationClient(FederationBase):
|
||||
destinations=[pdu_origin],
|
||||
event_id=pdu.event_id,
|
||||
room_version=room_version,
|
||||
outlier=outlier,
|
||||
timeout=10000,
|
||||
)
|
||||
except SynapseError:
|
||||
@@ -541,13 +520,10 @@ class FederationClient(FederationBase):
|
||||
|
||||
room_version = await self.store.get_room_version(room_id)
|
||||
|
||||
auth_chain = [
|
||||
event_from_pdu_json(p, room_version, outlier=True)
|
||||
for p in res["auth_chain"]
|
||||
]
|
||||
auth_chain = [event_from_pdu_json(p, room_version) for p in res["auth_chain"]]
|
||||
|
||||
signed_auth = await self._check_sigs_and_hash_and_fetch(
|
||||
destination, auth_chain, outlier=True, room_version=room_version
|
||||
destination, auth_chain, room_version=room_version
|
||||
)
|
||||
|
||||
return signed_auth
|
||||
@@ -816,7 +792,6 @@ class FederationClient(FederationBase):
|
||||
valid_pdu = await self._check_sigs_and_hash_and_fetch_one(
|
||||
pdu=event,
|
||||
origin=destination,
|
||||
outlier=True,
|
||||
room_version=room_version,
|
||||
)
|
||||
|
||||
@@ -864,7 +839,6 @@ class FederationClient(FederationBase):
|
||||
valid_pdu = await self._check_sigs_and_hash_and_fetch_one(
|
||||
pdu=pdu,
|
||||
origin=destination,
|
||||
outlier=True,
|
||||
room_version=room_version,
|
||||
)
|
||||
|
||||
@@ -1235,7 +1209,7 @@ class FederationClient(FederationBase):
|
||||
]
|
||||
|
||||
signed_events = await self._check_sigs_and_hash_and_fetch(
|
||||
destination, events, outlier=False, room_version=room_version
|
||||
destination, events, room_version=room_version
|
||||
)
|
||||
except HttpResponseException as e:
|
||||
if not e.code == 400:
|
||||
|
||||
@@ -28,6 +28,7 @@ from typing import (
|
||||
Union,
|
||||
)
|
||||
|
||||
from matrix_common.regex import glob_to_regex
|
||||
from prometheus_client import Counter, Gauge, Histogram
|
||||
|
||||
from twisted.internet.abstract import isIPAddress
|
||||
@@ -65,7 +66,7 @@ from synapse.replication.http.federation import (
|
||||
)
|
||||
from synapse.storage.databases.main.lock import Lock
|
||||
from synapse.types import JsonDict, get_domain_from_id
|
||||
from synapse.util import glob_to_regex, json_decoder, unwrapFirstError
|
||||
from synapse.util import json_decoder, unwrapFirstError
|
||||
from synapse.util.async_helpers import Linearizer, concurrently_execute, gather_results
|
||||
from synapse.util.caches.response_cache import ResponseCache
|
||||
from synapse.util.stringutils import parse_server_name
|
||||
|
||||
@@ -30,7 +30,6 @@ Events are replicated via a separate events stream.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from collections import namedtuple
|
||||
from typing import (
|
||||
TYPE_CHECKING,
|
||||
Dict,
|
||||
@@ -43,6 +42,7 @@ from typing import (
|
||||
Type,
|
||||
)
|
||||
|
||||
import attr
|
||||
from sortedcontainers import SortedDict
|
||||
|
||||
from synapse.api.presence import UserPresenceState
|
||||
@@ -382,13 +382,11 @@ class BaseFederationRow:
|
||||
raise NotImplementedError()
|
||||
|
||||
|
||||
class PresenceDestinationsRow(
|
||||
BaseFederationRow,
|
||||
namedtuple(
|
||||
"PresenceDestinationsRow",
|
||||
("state", "destinations"), # UserPresenceState # list[str]
|
||||
),
|
||||
):
|
||||
@attr.s(slots=True, frozen=True, auto_attribs=True)
|
||||
class PresenceDestinationsRow(BaseFederationRow):
|
||||
state: UserPresenceState
|
||||
destinations: List[str]
|
||||
|
||||
TypeId = "pd"
|
||||
|
||||
@staticmethod
|
||||
@@ -404,17 +402,15 @@ class PresenceDestinationsRow(
|
||||
buff.presence_destinations.append((self.state, self.destinations))
|
||||
|
||||
|
||||
class KeyedEduRow(
|
||||
BaseFederationRow,
|
||||
namedtuple(
|
||||
"KeyedEduRow",
|
||||
("key", "edu"), # tuple(str) - the edu key passed to send_edu # Edu
|
||||
),
|
||||
):
|
||||
@attr.s(slots=True, frozen=True, auto_attribs=True)
|
||||
class KeyedEduRow(BaseFederationRow):
|
||||
"""Streams EDUs that have an associated key that is ued to clobber. For example,
|
||||
typing EDUs clobber based on room_id.
|
||||
"""
|
||||
|
||||
key: Tuple[str, ...] # the edu key passed to send_edu
|
||||
edu: Edu
|
||||
|
||||
TypeId = "k"
|
||||
|
||||
@staticmethod
|
||||
@@ -428,9 +424,12 @@ class KeyedEduRow(
|
||||
buff.keyed_edus.setdefault(self.edu.destination, {})[self.key] = self.edu
|
||||
|
||||
|
||||
class EduRow(BaseFederationRow, namedtuple("EduRow", ("edu",))): # Edu
|
||||
@attr.s(slots=True, frozen=True, auto_attribs=True)
|
||||
class EduRow(BaseFederationRow):
|
||||
"""Streams EDUs that don't have keys. See KeyedEduRow"""
|
||||
|
||||
edu: Edu
|
||||
|
||||
TypeId = "e"
|
||||
|
||||
@staticmethod
|
||||
@@ -453,14 +452,14 @@ _rowtypes: Tuple[Type[BaseFederationRow], ...] = (
|
||||
TypeToRow = {Row.TypeId: Row for Row in _rowtypes}
|
||||
|
||||
|
||||
ParsedFederationStreamData = namedtuple(
|
||||
"ParsedFederationStreamData",
|
||||
(
|
||||
"presence_destinations", # list of tuples of UserPresenceState and destinations
|
||||
"keyed_edus", # dict of destination -> { key -> Edu }
|
||||
"edus", # dict of destination -> [Edu]
|
||||
),
|
||||
)
|
||||
@attr.s(slots=True, frozen=True, auto_attribs=True)
|
||||
class ParsedFederationStreamData:
|
||||
# list of tuples of UserPresenceState and destinations
|
||||
presence_destinations: List[Tuple[UserPresenceState, List[str]]]
|
||||
# dict of destination -> { key -> Edu }
|
||||
keyed_edus: Dict[str, Dict[Tuple[str, ...], Edu]]
|
||||
# dict of destination -> [Edu]
|
||||
edus: Dict[str, List[Edu]]
|
||||
|
||||
|
||||
def process_rows_for_federation(
|
||||
|
||||
@@ -22,13 +22,11 @@ from synapse.api.urls import FEDERATION_V1_PREFIX
|
||||
from synapse.http.server import HttpServer, ServletCallback
|
||||
from synapse.http.servlet import parse_json_object_from_request
|
||||
from synapse.http.site import SynapseRequest
|
||||
from synapse.logging import opentracing
|
||||
from synapse.logging.context import run_in_background
|
||||
from synapse.logging.opentracing import (
|
||||
SynapseTags,
|
||||
start_active_span,
|
||||
start_active_span_from_request,
|
||||
tags,
|
||||
set_tag,
|
||||
span_context_from_request,
|
||||
start_active_span_follows_from,
|
||||
whitelisted_homeserver,
|
||||
)
|
||||
from synapse.server import HomeServer
|
||||
@@ -279,30 +277,19 @@ class BaseFederationServlet:
|
||||
logger.warning("authenticate_request failed: %s", e)
|
||||
raise
|
||||
|
||||
request_tags = {
|
||||
SynapseTags.REQUEST_ID: request.get_request_id(),
|
||||
tags.SPAN_KIND: tags.SPAN_KIND_RPC_SERVER,
|
||||
tags.HTTP_METHOD: request.get_method(),
|
||||
tags.HTTP_URL: request.get_redacted_uri(),
|
||||
tags.PEER_HOST_IPV6: request.getClientIP(),
|
||||
"authenticated_entity": origin,
|
||||
"servlet_name": request.request_metrics.name,
|
||||
}
|
||||
# update the active opentracing span with the authenticated entity
|
||||
set_tag("authenticated_entity", origin)
|
||||
|
||||
# Only accept the span context if the origin is authenticated
|
||||
# and whitelisted
|
||||
# if the origin is authenticated and whitelisted, link to its span context
|
||||
context = None
|
||||
if origin and whitelisted_homeserver(origin):
|
||||
scope = start_active_span_from_request(
|
||||
request, "incoming-federation-request", tags=request_tags
|
||||
)
|
||||
else:
|
||||
scope = start_active_span(
|
||||
"incoming-federation-request", tags=request_tags
|
||||
)
|
||||
context = span_context_from_request(request)
|
||||
|
||||
scope = start_active_span_follows_from(
|
||||
"incoming-federation-request", contexts=(context,) if context else ()
|
||||
)
|
||||
|
||||
with scope:
|
||||
opentracing.inject_response_headers(request.responseHeaders)
|
||||
|
||||
if origin and self.RATELIMIT:
|
||||
with ratelimiter.ratelimit(origin) as d:
|
||||
await d
|
||||
|
||||
@@ -77,7 +77,7 @@ class AccountDataHandler:
|
||||
async def add_account_data_for_user(
|
||||
self, user_id: str, account_data_type: str, content: JsonDict
|
||||
) -> int:
|
||||
"""Add some account_data to a room for a user.
|
||||
"""Add some global account_data for a user.
|
||||
|
||||
Args:
|
||||
user_id: The user to add a tag for.
|
||||
|
||||
@@ -462,9 +462,9 @@ class ApplicationServicesHandler:
|
||||
|
||||
Args:
|
||||
room_alias: The room alias to query.
|
||||
|
||||
Returns:
|
||||
namedtuple: with keys "room_id" and "servers" or None if no
|
||||
association can be found.
|
||||
RoomAliasMapping or None if no association can be found.
|
||||
"""
|
||||
room_alias_str = room_alias.to_string()
|
||||
services = self.store.get_app_services()
|
||||
|
||||
@@ -948,8 +948,16 @@ class DeviceListUpdater:
|
||||
devices = []
|
||||
ignore_devices = True
|
||||
else:
|
||||
prev_stream_id = await self.store.get_device_list_last_stream_id_for_remote(
|
||||
user_id
|
||||
)
|
||||
cached_devices = await self.store.get_cached_devices_for_user(user_id)
|
||||
if cached_devices == {d["device_id"]: d for d in devices}:
|
||||
|
||||
# To ensure that a user with no devices is cached, we skip the resync only
|
||||
# if we have a stream_id from previously writing a cache entry.
|
||||
if prev_stream_id is not None and cached_devices == {
|
||||
d["device_id"]: d for d in devices
|
||||
}:
|
||||
logging.info(
|
||||
"Skipping device list resync for %s, as our cache matches already",
|
||||
user_id,
|
||||
|
||||
@@ -278,13 +278,15 @@ class DirectoryHandler:
|
||||
|
||||
users = await self.store.get_users_in_room(room_id)
|
||||
extra_servers = {get_domain_from_id(u) for u in users}
|
||||
servers = set(extra_servers) | set(servers)
|
||||
servers_set = set(extra_servers) | set(servers)
|
||||
|
||||
# If this server is in the list of servers, return it first.
|
||||
if self.server_name in servers:
|
||||
servers = [self.server_name] + [s for s in servers if s != self.server_name]
|
||||
if self.server_name in servers_set:
|
||||
servers = [self.server_name] + [
|
||||
s for s in servers_set if s != self.server_name
|
||||
]
|
||||
else:
|
||||
servers = list(servers)
|
||||
servers = list(servers_set)
|
||||
|
||||
return {"room_id": room_id, "servers": servers}
|
||||
|
||||
|
||||
@@ -119,12 +119,10 @@ class EventStreamHandler:
|
||||
|
||||
events.extend(to_add)
|
||||
|
||||
chunks = await self._event_serializer.serialize_events(
|
||||
chunks = self._event_serializer.serialize_events(
|
||||
events,
|
||||
time_now,
|
||||
as_client_event=as_client_event,
|
||||
# Don't bundle aggregations as this is a deprecated API.
|
||||
bundle_aggregations=False,
|
||||
)
|
||||
|
||||
chunk = {
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user