Merge commit 'd8001' (pre v1.54.0rc1) into matrix-org-hotfixes
This commit is contained in:
Olivier Wilkinson (reivilibre)
@@ -0,0 +1,11 @@
|
||||
# TODO: incorporate this into pyproject.toml if flake8 supports it in the future.
|
||||
# See https://github.com/PyCQA/flake8/issues/234
|
||||
[flake8]
|
||||
# see https://pycodestyle.readthedocs.io/en/latest/intro.html#error-codes
|
||||
# for error codes. The ones we ignore are:
|
||||
# W503: line break before binary operator
|
||||
# W504: line break after binary operator
|
||||
# E203: whitespace before ':' (which is contrary to pep8?)
|
||||
# E731: do not assign a lambda expression, use a def
|
||||
# E501: Line too long (black enforces this for us)
|
||||
ignore=W503,W504,E203,E731,E501
|
||||
@@ -10,12 +10,19 @@ concurrency:
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
check-sampleconfig:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
- uses: actions/setup-python@v2
|
||||
- run: pip install -e .
|
||||
- run: scripts-dev/generate_sample_config --check
|
||||
|
||||
lint:
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
toxenv:
|
||||
- "check-sampleconfig"
|
||||
- "check_codestyle"
|
||||
- "check_isort"
|
||||
- "mypy"
|
||||
@@ -43,7 +50,7 @@ jobs:
|
||||
ref: ${{ github.event.pull_request.head.sha }}
|
||||
fetch-depth: 0
|
||||
- uses: actions/setup-python@v2
|
||||
- run: pip install tox
|
||||
- run: "pip install 'towncrier>=18.6.0rc1'"
|
||||
- run: scripts-dev/check-newsfragment
|
||||
env:
|
||||
PULL_REQUEST_NUMBER: ${{ github.event.number }}
|
||||
@@ -51,7 +58,7 @@ jobs:
|
||||
# Dummy step to gate other tests on without repeating the whole list
|
||||
linting-done:
|
||||
if: ${{ !cancelled() }} # Run this even if prior jobs were skipped
|
||||
needs: [lint, lint-crlf, lint-newsfile]
|
||||
needs: [lint, lint-crlf, lint-newsfile, check-sampleconfig]
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- run: "true"
|
||||
|
||||
+100
@@ -1,3 +1,103 @@
|
||||
Synapse 1.54.0rc1 (2022-03-02)
|
||||
==============================
|
||||
|
||||
Please note that this will be the last release of Synapse that is compatible with Mjolnir 1.3.1 and earlier.
|
||||
Administrators of servers which have the Mjolnir module installed are advised to upgrade Mjolnir to version 1.3.2 or later.
|
||||
|
||||
|
||||
Features
|
||||
--------
|
||||
|
||||
- Add support for [MSC3202](https://github.com/matrix-org/matrix-doc/pull/3202): sending one-time key counts and fallback key usage states to Application Services. ([\#11617](https://github.com/matrix-org/synapse/issues/11617))
|
||||
- Make a `POST` to `/rooms/<room_id>/receipt/m.read/<event_id>` only trigger a push notification if the count of unread messages is different to the one in the last successfully sent push. ([\#11835](https://github.com/matrix-org/synapse/issues/11835))
|
||||
- Fetch images when previewing Twitter URLs. Contributed by @AndrewRyanChama. ([\#11985](https://github.com/matrix-org/synapse/issues/11985))
|
||||
- Track cache invalidations in Prometheus metrics, as already happens for cache eviction based on size or time. ([\#12000](https://github.com/matrix-org/synapse/issues/12000))
|
||||
- Implement experimental support for [MSC3720](https://github.com/matrix-org/matrix-doc/pull/3720) (account status endpoints). ([\#12001](https://github.com/matrix-org/synapse/issues/12001), [\#12067](https://github.com/matrix-org/synapse/issues/12067))
|
||||
- Enable modules to set a custom display name when registering a user. ([\#12009](https://github.com/matrix-org/synapse/issues/12009))
|
||||
- Advertise Matrix 1.1 support on `/_matrix/client/versions`. ([\#12020](https://github.com/matrix-org/synapse/issues/12020))
|
||||
- Support only the stable identifier for [MSC3069](https://github.com/matrix-org/matrix-doc/pull/3069)'s `is_guest` on `/_matrix/client/v3/account/whoami`. ([\#12021](https://github.com/matrix-org/synapse/issues/12021))
|
||||
- Advertise Matrix 1.2 support on `/_matrix/client/versions`. ([\#12022](https://github.com/matrix-org/synapse/issues/12022))
|
||||
- Use room version 9 as the default room version (per [MSC3589](https://github.com/matrix-org/matrix-doc/pull/3589)). ([\#12058](https://github.com/matrix-org/synapse/issues/12058))
|
||||
- Add module callbacks to react to user deactivation status changes (i.e. deactivations and reactivations) and profile updates. ([\#12062](https://github.com/matrix-org/synapse/issues/12062))
|
||||
|
||||
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Fix a bug introduced in Synapse v1.48.0 where an edit of the latest event in a thread would not be properly applied to the thread summary. ([\#11992](https://github.com/matrix-org/synapse/issues/11992))
|
||||
- Fix long standing bug where `get_rooms_for_user` was not correctly invalidated for remote users when the server left a room. ([\#11999](https://github.com/matrix-org/synapse/issues/11999))
|
||||
- Fix a 500 error with Postgres when looking backwards with the [MSC3030](https://github.com/matrix-org/matrix-doc/pull/3030) `/timestamp_to_event?dir=b` endpoint. ([\#12024](https://github.com/matrix-org/synapse/issues/12024))
|
||||
- Properly fix a long-standing bug where wrong data could be inserted into the `event_search` table when using SQLite. This could block running `synapse_port_db` with an "argument of type 'int' is not iterable" error. This bug was partially fixed by a change in Synapse 1.44.0. ([\#12037](https://github.com/matrix-org/synapse/issues/12037))
|
||||
- Fix slow performance of `/logout` in some cases where refresh tokens are in use. The slowness existed since the initial implementation of refresh tokens. ([\#12056](https://github.com/matrix-org/synapse/issues/12056))
|
||||
- Fix a long-standing bug where Synapse would make additional failing requests over federation for missing data. ([\#12077](https://github.com/matrix-org/synapse/issues/12077))
|
||||
- Fix occasional 'Unhandled error in Deferred' error message. ([\#12089](https://github.com/matrix-org/synapse/issues/12089))
|
||||
- Fix a bug introduced in Synapse 1.51.0rc1 where incoming federation transactions containing at least one EDU would be dropped if debug logging was enabled for `synapse.8631_debug`. ([\#12098](https://github.com/matrix-org/synapse/issues/12098))
|
||||
- Fix a long-standing bug which could cause push notifications to malfunction if `use_frozen_dicts` was set in the configuration. ([\#12100](https://github.com/matrix-org/synapse/issues/12100))
|
||||
- Fix an extremely rare, long-standing bug in `ReadWriteLock` that would cause an error when a newly unblocked writer completes instantly. ([\#12105](https://github.com/matrix-org/synapse/issues/12105))
|
||||
|
||||
|
||||
Updates to the Docker image
|
||||
---------------------------
|
||||
|
||||
- The Docker image no longer automatically creates a temporary volume at `/data`. This is not expected to affect normal usage. ([\#11997](https://github.com/matrix-org/synapse/issues/11997))
|
||||
- Use Python 3.9 in Docker images by default. ([\#12112](https://github.com/matrix-org/synapse/issues/12112))
|
||||
|
||||
|
||||
Improved Documentation
|
||||
----------------------
|
||||
|
||||
- Document support for the `to_device`, `account_data`, `receipts`, and `presence` stream writers for workers. ([\#11599](https://github.com/matrix-org/synapse/issues/11599))
|
||||
- Explain the meaning of spam checker callbacks' return values. ([\#12003](https://github.com/matrix-org/synapse/issues/12003))
|
||||
- Clarify information about external Identity Provider IDs. ([\#12004](https://github.com/matrix-org/synapse/issues/12004))
|
||||
|
||||
|
||||
Deprecations and Removals
|
||||
-------------------------
|
||||
|
||||
- Deprecate using `synctl` with the config option `synctl_cache_factor` and print a warning if a user still uses this option. ([\#11865](https://github.com/matrix-org/synapse/issues/11865))
|
||||
- Remove support for the legacy structured logging configuration (please see the the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#legacy-structured-logging-configuration-removal) if you are using `structured: true` in the Synapse configuration). ([\#12008](https://github.com/matrix-org/synapse/issues/12008))
|
||||
- Drop support for [MSC3283](https://github.com/matrix-org/matrix-doc/pull/3283) unstable flags now that the stable flags are supported. ([\#12018](https://github.com/matrix-org/synapse/issues/12018))
|
||||
- Remove the unstable `/spaces` endpoint from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946). ([\#12073](https://github.com/matrix-org/synapse/issues/12073))
|
||||
|
||||
|
||||
Internal Changes
|
||||
----------------
|
||||
|
||||
- Make the `get_room_version` method use `get_room_version_id` to benefit from caching. ([\#11808](https://github.com/matrix-org/synapse/issues/11808))
|
||||
- Remove unnecessary condition on knock -> leave auth rule check. ([\#11900](https://github.com/matrix-org/synapse/issues/11900))
|
||||
- Add tests for device list changes between local users. ([\#11972](https://github.com/matrix-org/synapse/issues/11972))
|
||||
- Optimise calculating `device_list` changes in `/sync`. ([\#11974](https://github.com/matrix-org/synapse/issues/11974))
|
||||
- Add missing type hints to storage classes. ([\#11984](https://github.com/matrix-org/synapse/issues/11984))
|
||||
- Refactor the search code for improved readability. ([\#11991](https://github.com/matrix-org/synapse/issues/11991))
|
||||
- Move common deduplication code down into `_auth_and_persist_outliers`. ([\#11994](https://github.com/matrix-org/synapse/issues/11994))
|
||||
- Limit concurrent joins from applications services. ([\#11996](https://github.com/matrix-org/synapse/issues/11996))
|
||||
- Preparation for faster-room-join work: when parsing the `send_join` response, get the `m.room.create` event from `state`, not `auth_chain`. ([\#12005](https://github.com/matrix-org/synapse/issues/12005), [\#12039](https://github.com/matrix-org/synapse/issues/12039))
|
||||
- Preparation for faster-room-join work: parse MSC3706 fields in send_join response. ([\#12011](https://github.com/matrix-org/synapse/issues/12011))
|
||||
- Preparation for faster-room-join work: persist information on which events and rooms have partial state to the database. ([\#12012](https://github.com/matrix-org/synapse/issues/12012))
|
||||
- Preparation for faster-room-join work: Support for calling `/federation/v1/state` on a remote server. ([\#12013](https://github.com/matrix-org/synapse/issues/12013))
|
||||
- Configure `tox` to use `venv` rather than `virtualenv`. ([\#12015](https://github.com/matrix-org/synapse/issues/12015))
|
||||
- Fix bug in `StateFilter.return_expanded()` and add some tests. ([\#12016](https://github.com/matrix-org/synapse/issues/12016))
|
||||
- Use Matrix v1.1 endpoints (`/_matrix/client/v3/auth/...`) in fallback auth HTML forms. ([\#12019](https://github.com/matrix-org/synapse/issues/12019))
|
||||
- Update the `olddeps` CI job to use an old version of `markupsafe`. ([\#12025](https://github.com/matrix-org/synapse/issues/12025))
|
||||
- Upgrade Mypy to version 0.931. ([\#12030](https://github.com/matrix-org/synapse/issues/12030))
|
||||
- Remove legacy `HomeServer.get_datastore()`. ([\#12031](https://github.com/matrix-org/synapse/issues/12031), [\#12070](https://github.com/matrix-org/synapse/issues/12070))
|
||||
- Minor typing fixes. ([\#12034](https://github.com/matrix-org/synapse/issues/12034), [\#12069](https://github.com/matrix-org/synapse/issues/12069))
|
||||
- After joining a room, create a dedicated logcontext to process the queued events. ([\#12041](https://github.com/matrix-org/synapse/issues/12041))
|
||||
- Tidy up GitHub Actions config which builds distributions for PyPI. ([\#12051](https://github.com/matrix-org/synapse/issues/12051))
|
||||
- Move configuration out of `setup.cfg`. ([\#12052](https://github.com/matrix-org/synapse/issues/12052), [\#12059](https://github.com/matrix-org/synapse/issues/12059))
|
||||
- Fix error message when a worker process fails to talk to another worker process. ([\#12060](https://github.com/matrix-org/synapse/issues/12060))
|
||||
- Fix using the `complement.sh` script without specifying a directory or a branch. Contributed by Nico on behalf of Famedly. ([\#12063](https://github.com/matrix-org/synapse/issues/12063))
|
||||
- Add type hints to `tests/rest/client`. ([\#12066](https://github.com/matrix-org/synapse/issues/12066), [\#12072](https://github.com/matrix-org/synapse/issues/12072), [\#12084](https://github.com/matrix-org/synapse/issues/12084), [\#12094](https://github.com/matrix-org/synapse/issues/12094))
|
||||
- Add some logging to `/sync` to try and track down #11916. ([\#12068](https://github.com/matrix-org/synapse/issues/12068))
|
||||
- Inspect application dependencies using `importlib.metadata` or its backport. ([\#12088](https://github.com/matrix-org/synapse/issues/12088))
|
||||
- Use `assertEqual` instead of the deprecated `assertEquals` in test code. ([\#12092](https://github.com/matrix-org/synapse/issues/12092))
|
||||
- Move experimental support for [MSC3440](https://github.com/matrix-org/matrix-doc/pull/3440) to `/versions`. ([\#12099](https://github.com/matrix-org/synapse/issues/12099))
|
||||
- Add `stop_cancellation` utility function to stop `Deferred`s from being cancelled. ([\#12106](https://github.com/matrix-org/synapse/issues/12106))
|
||||
- Improve exception handling for concurrent execution. ([\#12109](https://github.com/matrix-org/synapse/issues/12109))
|
||||
- Advertise support for Python 3.10 in packaging files. ([\#12111](https://github.com/matrix-org/synapse/issues/12111))
|
||||
- Move CI checks out of tox, to facilitate a move to using poetry. ([\#12119](https://github.com/matrix-org/synapse/issues/12119))
|
||||
|
||||
|
||||
Synapse 1.53.0 (2022-02-22)
|
||||
===========================
|
||||
|
||||
|
||||
@@ -45,6 +45,7 @@ include book.toml
|
||||
include pyproject.toml
|
||||
recursive-include changelog.d *
|
||||
|
||||
include .flake8
|
||||
prune .circleci
|
||||
prune .github
|
||||
prune .ci
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
Make a `POST` to `/rooms/<room_id>/receipt/m.read/<event_id>` only trigger a push notification if the count of unread messages is different to the one in the last successfully sent push.
|
||||
@@ -1 +0,0 @@
|
||||
Add tests for device list changes between local users.
|
||||
@@ -1 +0,0 @@
|
||||
Optimise calculating device_list changes in `/sync`.
|
||||
@@ -1 +0,0 @@
|
||||
Add missing type hints to storage classes.
|
||||
@@ -1 +0,0 @@
|
||||
Fetch images when previewing Twitter URLs. Contributed by @AndrewRyanChama.
|
||||
@@ -1 +0,0 @@
|
||||
Refactor the search code for improved readability.
|
||||
@@ -1 +0,0 @@
|
||||
Fix a bug introduced in Synapse v1.48.0 where an edit of the latest event in a thread would not be properly applied to the thread summary.
|
||||
@@ -1 +0,0 @@
|
||||
Move common deduplication code down into `_auth_and_persist_outliers`.
|
||||
@@ -1 +0,0 @@
|
||||
Limit concurrent joins from applications services.
|
||||
@@ -1 +0,0 @@
|
||||
The docker image no longer automatically creates a temporary volume at `/data`. This is not expected to affect normal usage.
|
||||
@@ -1 +0,0 @@
|
||||
Fix long standing bug where `get_rooms_for_user` was not correctly invalidated for remote users when the server left a room.
|
||||
@@ -1 +0,0 @@
|
||||
Track cache invalidations in Prometheus metrics, as already happens for cache eviction based on size or time.
|
||||
@@ -1 +0,0 @@
|
||||
Explain the meaning of spam checker callbacks' return values.
|
||||
@@ -1 +0,0 @@
|
||||
Clarify information about external Identity Provider IDs.
|
||||
@@ -1 +0,0 @@
|
||||
Preparation for faster-room-join work: when parsing the `send_join` response, get the `m.room.create` event from `state`, not `auth_chain`.
|
||||
@@ -1 +0,0 @@
|
||||
Remove support for the legacy structured logging configuration (please see the the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#legacy-structured-logging-configuration-removal) if you are using `structured: true` in the Synapse configuration).
|
||||
@@ -1 +0,0 @@
|
||||
Enable modules to set a custom display name when registering a user.
|
||||
@@ -1 +0,0 @@
|
||||
Preparation for faster-room-join work: parse msc3706 fields in send_join response.
|
||||
@@ -1 +0,0 @@
|
||||
Preparation for faster-room-join work: Support for calling `/federation/v1/state` on a remote server.
|
||||
@@ -1 +0,0 @@
|
||||
Configure `tox` to use `venv` rather than `virtualenv`.
|
||||
@@ -1 +0,0 @@
|
||||
Fix bug in `StateFilter.return_expanded()` and add some tests.
|
||||
@@ -1 +0,0 @@
|
||||
Drop support for [MSC3283](https://github.com/matrix-org/matrix-doc/pull/3283) unstable flags now that the stable flags are supported.
|
||||
@@ -1 +0,0 @@
|
||||
Use Matrix v1.1 endpoints (`/_matrix/client/v3/auth/...`) in fallback auth HTML forms.
|
||||
@@ -1 +0,0 @@
|
||||
Advertise Matrix 1.1 support on `/_matrix/client/versions`.
|
||||
@@ -1 +0,0 @@
|
||||
Support only the stable identifier for [MSC3069](https://github.com/matrix-org/matrix-doc/pull/3069)'s `is_guest` on `/_matrix/client/v3/account/whoami`.
|
||||
@@ -1 +0,0 @@
|
||||
Advertise Matrix 1.2 support on `/_matrix/client/versions`.
|
||||
@@ -1 +0,0 @@
|
||||
Fix 500 error with Postgres when looking backwards with the [MSC3030](https://github.com/matrix-org/matrix-doc/pull/3030) `/timestamp_to_event?dir=b` endpoint.
|
||||
@@ -1 +0,0 @@
|
||||
Update the `olddeps` CI job to use an old version of `markupsafe`.
|
||||
@@ -1 +0,0 @@
|
||||
Upgrade mypy to version 0.931.
|
||||
@@ -1 +0,0 @@
|
||||
Deduplicate in-flight requests in `_get_state_for_groups`.
|
||||
@@ -1 +0,0 @@
|
||||
Minor typing fixes.
|
||||
@@ -1 +0,0 @@
|
||||
Preparation for faster-room-join work: when parsing the `send_join` response, get the `m.room.create` event from `state`, not `auth_chain`.
|
||||
@@ -1 +0,0 @@
|
||||
After joining a room, create a dedicated logcontext to process the queued events.
|
||||
@@ -1 +0,0 @@
|
||||
Tidy up GitHub Actions config which builds distributions for PyPI.
|
||||
@@ -1 +0,0 @@
|
||||
Move `isort` configuration to `pyproject.toml`.
|
||||
@@ -1 +0,0 @@
|
||||
Fix slow performance of `/logout` in some cases where refresh tokens are in use. The slowness existed since the initial implementation of refresh tokens.
|
||||
@@ -1 +0,0 @@
|
||||
Use room version 9 as the default room version (per [MSC3589](https://github.com/matrix-org/matrix-doc/pull/3589)).
|
||||
Vendored
+6
@@ -1,3 +1,9 @@
|
||||
matrix-synapse-py3 (1.54.0~rc1) stable; urgency=medium
|
||||
|
||||
* New synapse release 1.54.0~rc1.
|
||||
|
||||
-- Synapse Packaging team <packages@matrix.org> Wed, 02 Mar 2022 10:43:22 +0000
|
||||
|
||||
matrix-synapse-py3 (1.53.0) stable; urgency=medium
|
||||
|
||||
* New synapse release 1.53.0.
|
||||
|
||||
+2
-2
@@ -11,10 +11,10 @@
|
||||
# There is an optional PYTHON_VERSION build argument which sets the
|
||||
# version of python to build against: for example:
|
||||
#
|
||||
# DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile --build-arg PYTHON_VERSION=3.9 .
|
||||
# DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile --build-arg PYTHON_VERSION=3.10 .
|
||||
#
|
||||
|
||||
ARG PYTHON_VERSION=3.8
|
||||
ARG PYTHON_VERSION=3.9
|
||||
|
||||
###
|
||||
### Stage 0: builder
|
||||
|
||||
+1
-1
@@ -94,6 +94,6 @@ As a simple example, retrieving an event from the database:
|
||||
|
||||
```pycon
|
||||
>>> from twisted.internet import defer
|
||||
>>> defer.ensureDeferred(hs.get_datastore().get_event('$1416420717069yeQaw:matrix.org'))
|
||||
>>> defer.ensureDeferred(hs.get_datastores().main.get_event('$1416420717069yeQaw:matrix.org'))
|
||||
<Deferred at 0x7ff253fc6998 current result: <FrozenEvent event_id='$1416420717069yeQaw:matrix.org', type='m.room.create', state_key=''>>
|
||||
```
|
||||
|
||||
@@ -148,6 +148,62 @@ deny an incoming event, see [`check_event_for_spam`](spam_checker_callbacks.md#c
|
||||
|
||||
If multiple modules implement this callback, Synapse runs them all in order.
|
||||
|
||||
### `on_profile_update`
|
||||
|
||||
_First introduced in Synapse v1.54.0_
|
||||
|
||||
```python
|
||||
async def on_profile_update(
|
||||
user_id: str,
|
||||
new_profile: "synapse.module_api.ProfileInfo",
|
||||
by_admin: bool,
|
||||
deactivation: bool,
|
||||
) -> None:
|
||||
```
|
||||
|
||||
Called after updating a local user's profile. The update can be triggered either by the
|
||||
user themselves or a server admin. The update can also be triggered by a user being
|
||||
deactivated (in which case their display name is set to an empty string (`""`) and the
|
||||
avatar URL is set to `None`). The module is passed the Matrix ID of the user whose profile
|
||||
has been updated, their new profile, as well as a `by_admin` boolean that is `True` if the
|
||||
update was triggered by a server admin (and `False` otherwise), and a `deactivated`
|
||||
boolean that is `True` if the update is a result of the user being deactivated.
|
||||
|
||||
Note that the `by_admin` boolean is also `True` if the profile change happens as a result
|
||||
of the user logging in through Single Sign-On, or if a server admin updates their own
|
||||
profile.
|
||||
|
||||
Per-room profile changes do not trigger this callback to be called. Synapse administrators
|
||||
wishing this callback to be called on every profile change are encouraged to disable
|
||||
per-room profiles globally using the `allow_per_room_profiles` configuration setting in
|
||||
Synapse's configuration file.
|
||||
This callback is not called when registering a user, even when setting it through the
|
||||
[`get_displayname_for_registration`](https://matrix-org.github.io/synapse/latest/modules/password_auth_provider_callbacks.html#get_displayname_for_registration)
|
||||
module callback.
|
||||
|
||||
If multiple modules implement this callback, Synapse runs them all in order.
|
||||
|
||||
### `on_user_deactivation_status_changed`
|
||||
|
||||
_First introduced in Synapse v1.54.0_
|
||||
|
||||
```python
|
||||
async def on_user_deactivation_status_changed(
|
||||
user_id: str, deactivated: bool, by_admin: bool
|
||||
) -> None:
|
||||
```
|
||||
|
||||
Called after deactivating a local user, or reactivating them through the admin API. The
|
||||
deactivation can be triggered either by the user themselves or a server admin. The module
|
||||
is passed the Matrix ID of the user whose status is changed, as well as a `deactivated`
|
||||
boolean that is `True` if the user is being deactivated and `False` if they're being
|
||||
reactivated, and a `by_admin` boolean that is `True` if the deactivation was triggered by
|
||||
a server admin (and `False` otherwise). This latter `by_admin` boolean is always `True`
|
||||
if the user is being reactivated, as this operation can only be performed through the
|
||||
admin API.
|
||||
|
||||
If multiple modules implement this callback, Synapse runs them all in order.
|
||||
|
||||
## Example
|
||||
|
||||
The example below is a module that implements the third-party rules callback
|
||||
|
||||
+76
-16
@@ -178,8 +178,11 @@ recommend the use of `systemd` where available: for information on setting up
|
||||
|
||||
### `synapse.app.generic_worker`
|
||||
|
||||
This worker can handle API requests matching the following regular
|
||||
expressions:
|
||||
This worker can handle API requests matching the following regular expressions.
|
||||
These endpoints can be routed to any worker. If a worker is set up to handle a
|
||||
stream then, for maximum efficiency, additional endpoints should be routed to that
|
||||
worker: refer to the [stream writers](#stream-writers) section below for further
|
||||
information.
|
||||
|
||||
# Sync requests
|
||||
^/_matrix/client/(v2_alpha|r0|v3)/sync$
|
||||
@@ -209,7 +212,6 @@ expressions:
|
||||
^/_matrix/federation/v1/user/devices/
|
||||
^/_matrix/federation/v1/get_groups_publicised$
|
||||
^/_matrix/key/v2/query
|
||||
^/_matrix/federation/unstable/org.matrix.msc2946/spaces/
|
||||
^/_matrix/federation/(v1|unstable/org.matrix.msc2946)/hierarchy/
|
||||
|
||||
# Inbound federation transaction request
|
||||
@@ -222,22 +224,25 @@ expressions:
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/context/.*$
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/members$
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/state$
|
||||
^/_matrix/client/unstable/org.matrix.msc2946/rooms/.*/spaces$
|
||||
^/_matrix/client/(v1|unstable/org.matrix.msc2946)/rooms/.*/hierarchy$
|
||||
^/_matrix/client/unstable/im.nheko.summary/rooms/.*/summary$
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/account/3pid$
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/devices$
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/keys/query$
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/keys/changes$
|
||||
^/_matrix/client/(r0|v3|unstable)/account/3pid$
|
||||
^/_matrix/client/(r0|v3|unstable)/devices$
|
||||
^/_matrix/client/versions$
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/voip/turnServer$
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/joined_groups$
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/publicised_groups$
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/publicised_groups/
|
||||
^/_matrix/client/(r0|v3|unstable)/joined_groups$
|
||||
^/_matrix/client/(r0|v3|unstable)/publicised_groups$
|
||||
^/_matrix/client/(r0|v3|unstable)/publicised_groups/
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/event/
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/joined_rooms$
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/search$
|
||||
|
||||
# Encryption requests
|
||||
^/_matrix/client/(r0|v3|unstable)/keys/query$
|
||||
^/_matrix/client/(r0|v3|unstable)/keys/changes$
|
||||
^/_matrix/client/(r0|v3|unstable)/keys/claim$
|
||||
^/_matrix/client/(r0|v3|unstable)/room_keys/
|
||||
|
||||
# Registration/login requests
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/login$
|
||||
^/_matrix/client/(r0|v3|unstable)/register$
|
||||
@@ -251,6 +256,20 @@ expressions:
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/join/
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/profile/
|
||||
|
||||
# Device requests
|
||||
^/_matrix/client/(r0|v3|unstable)/sendToDevice/
|
||||
|
||||
# Account data requests
|
||||
^/_matrix/client/(r0|v3|unstable)/.*/tags
|
||||
^/_matrix/client/(r0|v3|unstable)/.*/account_data
|
||||
|
||||
# Receipts requests
|
||||
^/_matrix/client/(r0|v3|unstable)/rooms/.*/receipt
|
||||
^/_matrix/client/(r0|v3|unstable)/rooms/.*/read_markers
|
||||
|
||||
# Presence requests
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/presence/
|
||||
|
||||
|
||||
Additionally, the following REST endpoints can be handled for GET requests:
|
||||
|
||||
@@ -330,12 +349,10 @@ Additionally, there is *experimental* support for moving writing of specific
|
||||
streams (such as events) off of the main process to a particular worker. (This
|
||||
is only supported with Redis-based replication.)
|
||||
|
||||
Currently supported streams are `events` and `typing`.
|
||||
|
||||
To enable this, the worker must have a HTTP replication listener configured,
|
||||
have a `worker_name` and be listed in the `instance_map` config. For example to
|
||||
move event persistence off to a dedicated worker, the shared configuration would
|
||||
include:
|
||||
have a `worker_name` and be listed in the `instance_map` config. The same worker
|
||||
can handle multiple streams. For example, to move event persistence off to a
|
||||
dedicated worker, the shared configuration would include:
|
||||
|
||||
```yaml
|
||||
instance_map:
|
||||
@@ -347,6 +364,12 @@ stream_writers:
|
||||
events: event_persister1
|
||||
```
|
||||
|
||||
Some of the streams have associated endpoints which, for maximum efficiency, should
|
||||
be routed to the workers handling that stream. See below for the currently supported
|
||||
streams and the endpoints associated with them:
|
||||
|
||||
##### The `events` stream
|
||||
|
||||
The `events` stream also experimentally supports having multiple writers, where
|
||||
work is sharded between them by room ID. Note that you *must* restart all worker
|
||||
instances when adding or removing event persisters. An example `stream_writers`
|
||||
@@ -359,6 +382,43 @@ stream_writers:
|
||||
- event_persister2
|
||||
```
|
||||
|
||||
##### The `typing` stream
|
||||
|
||||
The following endpoints should be routed directly to the workers configured as
|
||||
stream writers for the `typing` stream:
|
||||
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/typing
|
||||
|
||||
##### The `to_device` stream
|
||||
|
||||
The following endpoints should be routed directly to the workers configured as
|
||||
stream writers for the `to_device` stream:
|
||||
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/sendToDevice/
|
||||
|
||||
##### The `account_data` stream
|
||||
|
||||
The following endpoints should be routed directly to the workers configured as
|
||||
stream writers for the `account_data` stream:
|
||||
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/.*/tags
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/.*/account_data
|
||||
|
||||
##### The `receipts` stream
|
||||
|
||||
The following endpoints should be routed directly to the workers configured as
|
||||
stream writers for the `receipts` stream:
|
||||
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/receipt
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/read_markers
|
||||
|
||||
##### The `presence` stream
|
||||
|
||||
The following endpoints should be routed directly to the workers configured as
|
||||
stream writers for the `presence` stream:
|
||||
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/presence/
|
||||
|
||||
#### Background tasks
|
||||
|
||||
There is also *experimental* support for moving background tasks to a separate
|
||||
|
||||
@@ -75,16 +75,12 @@ exclude = (?x)
|
||||
|tests/push/test_presentable_names.py
|
||||
|tests/push/test_push_rule_evaluator.py
|
||||
|tests/rest/client/test_account.py
|
||||
|tests/rest/client/test_events.py
|
||||
|tests/rest/client/test_filter.py
|
||||
|tests/rest/client/test_groups.py
|
||||
|tests/rest/client/test_register.py
|
||||
|tests/rest/client/test_report_event.py
|
||||
|tests/rest/client/test_rooms.py
|
||||
|tests/rest/client/test_third_party_rules.py
|
||||
|tests/rest/client/test_transactions.py
|
||||
|tests/rest/client/test_typing.py
|
||||
|tests/rest/client/utils.py
|
||||
|tests/rest/key/v2/test_remote_key_resource.py
|
||||
|tests/rest/media/v1/test_base.py
|
||||
|tests/rest/media/v1/test_media_storage.py
|
||||
@@ -253,7 +249,7 @@ disallow_untyped_defs = True
|
||||
[mypy-tests.rest.admin.*]
|
||||
disallow_untyped_defs = True
|
||||
|
||||
[mypy-tests.rest.client.test_directory]
|
||||
[mypy-tests.rest.client.*]
|
||||
disallow_untyped_defs = True
|
||||
|
||||
[mypy-tests.federation.transport.test_client]
|
||||
|
||||
@@ -35,7 +35,7 @@ CONTRIBUTING_GUIDE_TEXT="!! Please see the contributing guide for help writing y
|
||||
https://github.com/matrix-org/synapse/blob/develop/CONTRIBUTING.md#changelog"
|
||||
|
||||
# If check-newsfragment returns a non-zero exit code, print the contributing guide and exit
|
||||
tox -qe check-newsfragment || (echo -e "$CONTRIBUTING_GUIDE_TEXT" >&2 && exit 1)
|
||||
python -m towncrier.check --compare-with=origin/develop || (echo -e "$CONTRIBUTING_GUIDE_TEXT" >&2 && exit 1)
|
||||
|
||||
echo
|
||||
echo "--------------------------"
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
# It makes a Synapse image which represents the current checkout,
|
||||
# builds a synapse-complement image on top, then runs tests with it.
|
||||
#
|
||||
# By default the script will fetch the latest Complement master branch and
|
||||
# By default the script will fetch the latest Complement main branch and
|
||||
# run tests with that. This can be overridden to use a custom Complement
|
||||
# checkout by setting the COMPLEMENT_DIR environment variable to the
|
||||
# filepath of a local Complement checkout or by setting the COMPLEMENT_REF
|
||||
@@ -32,7 +32,7 @@ cd "$(dirname $0)/.."
|
||||
|
||||
# Check for a user-specified Complement checkout
|
||||
if [[ -z "$COMPLEMENT_DIR" ]]; then
|
||||
COMPLEMENT_REF=${COMPLEMENT_REF:-master}
|
||||
COMPLEMENT_REF=${COMPLEMENT_REF:-main}
|
||||
echo "COMPLEMENT_DIR not set. Fetching Complement checkout from ${COMPLEMENT_REF}..."
|
||||
wget -Nq https://github.com/matrix-org/complement/archive/${COMPLEMENT_REF}.tar.gz
|
||||
tar -xzf ${COMPLEMENT_REF}.tar.gz
|
||||
|
||||
@@ -44,7 +44,7 @@ class MockHomeserver(HomeServer):
|
||||
|
||||
|
||||
def run_background_updates(hs):
|
||||
store = hs.get_datastore()
|
||||
store = hs.get_datastores().main
|
||||
|
||||
async def run_background_updates():
|
||||
await store.db_pool.updates.run_background_updates(sleep=False)
|
||||
|
||||
@@ -1,6 +1,3 @@
|
||||
[trial]
|
||||
test_suite = tests
|
||||
|
||||
[check-manifest]
|
||||
ignore =
|
||||
.git-blame-ignore-revs
|
||||
@@ -10,12 +7,3 @@ ignore =
|
||||
pylint.cfg
|
||||
tox.ini
|
||||
|
||||
[flake8]
|
||||
# see https://pycodestyle.readthedocs.io/en/latest/intro.html#error-codes
|
||||
# for error codes. The ones we ignore are:
|
||||
# W503: line break before binary operator
|
||||
# W504: line break after binary operator
|
||||
# E203: whitespace before ':' (which is contrary to pep8?)
|
||||
# E731: do not assign a lambda expression, use a def
|
||||
# E501: Line too long (black enforces this for us)
|
||||
ignore=W503,W504,E203,E731,E501
|
||||
|
||||
@@ -165,6 +165,7 @@ setup(
|
||||
"Programming Language :: Python :: 3.7",
|
||||
"Programming Language :: Python :: 3.8",
|
||||
"Programming Language :: Python :: 3.9",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
],
|
||||
scripts=["synctl"] + glob.glob("scripts/*"),
|
||||
cmdclass={"test": TestCommand},
|
||||
|
||||
+1
-1
@@ -47,7 +47,7 @@ try:
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
__version__ = "1.53.0"
|
||||
__version__ = "1.54.0rc1"
|
||||
|
||||
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
|
||||
# We import here so that we don't have to install a bunch of deps when
|
||||
|
||||
+1
-1
@@ -60,7 +60,7 @@ class Auth:
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self.hs = hs
|
||||
self.clock = hs.get_clock()
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
self.state = hs.get_state_handler()
|
||||
self._account_validity_handler = hs.get_account_validity_handler()
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
class AuthBlocking:
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
|
||||
self._server_notices_mxid = hs.config.servernotices.server_notices_mxid
|
||||
self._hs_disabled = hs.config.server.hs_disabled
|
||||
|
||||
@@ -22,6 +22,7 @@ from typing import (
|
||||
Dict,
|
||||
Iterable,
|
||||
List,
|
||||
Mapping,
|
||||
Optional,
|
||||
Set,
|
||||
TypeVar,
|
||||
@@ -150,7 +151,7 @@ def matrix_user_id_validator(user_id_str: str) -> UserID:
|
||||
class Filtering:
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self._hs = hs
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
|
||||
self.DEFAULT_FILTER_COLLECTION = FilterCollection(hs, {})
|
||||
|
||||
@@ -294,7 +295,7 @@ class FilterCollection:
|
||||
class Filter:
|
||||
def __init__(self, hs: "HomeServer", filter_json: JsonDict):
|
||||
self._hs = hs
|
||||
self._store = hs.get_datastore()
|
||||
self._store = hs.get_datastores().main
|
||||
self.filter_json = filter_json
|
||||
|
||||
self.limit = filter_json.get("limit", 10)
|
||||
@@ -361,10 +362,10 @@ class Filter:
|
||||
return self._check_fields(field_matchers)
|
||||
else:
|
||||
content = event.get("content")
|
||||
# Content is assumed to be a dict below, so ensure it is. This should
|
||||
# Content is assumed to be a mapping below, so ensure it is. This should
|
||||
# always be true for events, but account_data has been allowed to
|
||||
# have non-dict content.
|
||||
if not isinstance(content, dict):
|
||||
if not isinstance(content, Mapping):
|
||||
content = {}
|
||||
|
||||
sender = event.get("sender", None)
|
||||
|
||||
@@ -15,13 +15,13 @@ import logging
|
||||
import sys
|
||||
from typing import Container
|
||||
|
||||
from synapse import python_dependencies # noqa: E402
|
||||
from synapse.util import check_dependencies
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
try:
|
||||
python_dependencies.check_requirements()
|
||||
except python_dependencies.DependencyException as e:
|
||||
check_dependencies.check_requirements()
|
||||
except check_dependencies.DependencyException as e:
|
||||
sys.stderr.writelines(
|
||||
e.message # noqa: B306, DependencyException.message is a property
|
||||
)
|
||||
|
||||
@@ -448,7 +448,7 @@ async def start(hs: "HomeServer") -> None:
|
||||
|
||||
# It is now safe to start your Synapse.
|
||||
hs.start_listening()
|
||||
hs.get_datastore().db_pool.start_profiling()
|
||||
hs.get_datastores().main.db_pool.start_profiling()
|
||||
hs.get_pusherpool().start()
|
||||
|
||||
# Log when we start the shut down process.
|
||||
|
||||
@@ -142,7 +142,7 @@ class KeyUploadServlet(RestServlet):
|
||||
"""
|
||||
super().__init__()
|
||||
self.auth = hs.get_auth()
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
self.http_client = hs.get_simple_http_client()
|
||||
self.main_uri = hs.config.worker.worker_main_http_uri
|
||||
|
||||
|
||||
@@ -59,7 +59,6 @@ from synapse.http.server import (
|
||||
from synapse.http.site import SynapseSite
|
||||
from synapse.logging.context import LoggingContext
|
||||
from synapse.metrics import METRICS_PREFIX, MetricsResource, RegistryProxy
|
||||
from synapse.python_dependencies import check_requirements
|
||||
from synapse.replication.http import REPLICATION_PREFIX, ReplicationRestResource
|
||||
from synapse.replication.tcp.resource import ReplicationStreamProtocolFactory
|
||||
from synapse.rest import ClientRestResource
|
||||
@@ -70,6 +69,7 @@ from synapse.rest.synapse.client import build_synapse_client_resource_tree
|
||||
from synapse.rest.well_known import well_known_resource
|
||||
from synapse.server import HomeServer
|
||||
from synapse.storage import DataStore
|
||||
from synapse.util.check_dependencies import check_requirements
|
||||
from synapse.util.httpresourcetree import create_resource_tree
|
||||
from synapse.util.module_loader import load_module
|
||||
|
||||
@@ -372,7 +372,7 @@ def setup(config_options: List[str]) -> SynapseHomeServer:
|
||||
|
||||
await _base.start(hs)
|
||||
|
||||
hs.get_datastore().db_pool.updates.start_doing_background_updates()
|
||||
hs.get_datastores().main.db_pool.updates.start_doing_background_updates()
|
||||
|
||||
register_start(start)
|
||||
|
||||
|
||||
@@ -82,7 +82,7 @@ async def phone_stats_home(
|
||||
# General statistics
|
||||
#
|
||||
|
||||
store = hs.get_datastore()
|
||||
store = hs.get_datastores().main
|
||||
|
||||
stats["homeserver"] = hs.config.server.server_name
|
||||
stats["server_context"] = hs.config.server.server_context
|
||||
@@ -170,18 +170,22 @@ def start_phone_stats_home(hs: "HomeServer") -> None:
|
||||
# Rather than update on per session basis, batch up the requests.
|
||||
# If you increase the loop period, the accuracy of user_daily_visits
|
||||
# table will decrease
|
||||
clock.looping_call(hs.get_datastore().generate_user_daily_visits, 5 * 60 * 1000)
|
||||
clock.looping_call(
|
||||
hs.get_datastores().main.generate_user_daily_visits, 5 * 60 * 1000
|
||||
)
|
||||
|
||||
# monthly active user limiting functionality
|
||||
clock.looping_call(hs.get_datastore().reap_monthly_active_users, 1000 * 60 * 60)
|
||||
hs.get_datastore().reap_monthly_active_users()
|
||||
clock.looping_call(
|
||||
hs.get_datastores().main.reap_monthly_active_users, 1000 * 60 * 60
|
||||
)
|
||||
hs.get_datastores().main.reap_monthly_active_users()
|
||||
|
||||
@wrap_as_background_process("generate_monthly_active_users")
|
||||
async def generate_monthly_active_users() -> None:
|
||||
current_mau_count = 0
|
||||
current_mau_count_by_service = {}
|
||||
reserved_users: Sized = ()
|
||||
store = hs.get_datastore()
|
||||
store = hs.get_datastores().main
|
||||
if hs.config.server.limit_usage_by_mau or hs.config.server.mau_stats_only:
|
||||
current_mau_count = await store.get_monthly_active_count()
|
||||
current_mau_count_by_service = (
|
||||
|
||||
@@ -31,6 +31,14 @@ if TYPE_CHECKING:
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Type for the `device_one_time_key_counts` field in an appservice transaction
|
||||
# user ID -> {device ID -> {algorithm -> count}}
|
||||
TransactionOneTimeKeyCounts = Dict[str, Dict[str, Dict[str, int]]]
|
||||
|
||||
# Type for the `device_unused_fallback_keys` field in an appservice transaction
|
||||
# user ID -> {device ID -> [algorithm]}
|
||||
TransactionUnusedFallbackKeys = Dict[str, Dict[str, List[str]]]
|
||||
|
||||
|
||||
class ApplicationServiceState(Enum):
|
||||
DOWN = "down"
|
||||
@@ -72,6 +80,7 @@ class ApplicationService:
|
||||
rate_limited: bool = True,
|
||||
ip_range_whitelist: Optional[IPSet] = None,
|
||||
supports_ephemeral: bool = False,
|
||||
msc3202_transaction_extensions: bool = False,
|
||||
):
|
||||
self.token = token
|
||||
self.url = (
|
||||
@@ -84,6 +93,7 @@ class ApplicationService:
|
||||
self.id = id
|
||||
self.ip_range_whitelist = ip_range_whitelist
|
||||
self.supports_ephemeral = supports_ephemeral
|
||||
self.msc3202_transaction_extensions = msc3202_transaction_extensions
|
||||
|
||||
if "|" in self.id:
|
||||
raise Exception("application service ID cannot contain '|' character")
|
||||
@@ -339,12 +349,16 @@ class AppServiceTransaction:
|
||||
events: List[EventBase],
|
||||
ephemeral: List[JsonDict],
|
||||
to_device_messages: List[JsonDict],
|
||||
one_time_key_counts: TransactionOneTimeKeyCounts,
|
||||
unused_fallback_keys: TransactionUnusedFallbackKeys,
|
||||
):
|
||||
self.service = service
|
||||
self.id = id
|
||||
self.events = events
|
||||
self.ephemeral = ephemeral
|
||||
self.to_device_messages = to_device_messages
|
||||
self.one_time_key_counts = one_time_key_counts
|
||||
self.unused_fallback_keys = unused_fallback_keys
|
||||
|
||||
async def send(self, as_api: "ApplicationServiceApi") -> bool:
|
||||
"""Sends this transaction using the provided AS API interface.
|
||||
@@ -359,6 +373,8 @@ class AppServiceTransaction:
|
||||
events=self.events,
|
||||
ephemeral=self.ephemeral,
|
||||
to_device_messages=self.to_device_messages,
|
||||
one_time_key_counts=self.one_time_key_counts,
|
||||
unused_fallback_keys=self.unused_fallback_keys,
|
||||
txn_id=self.id,
|
||||
)
|
||||
|
||||
|
||||
@@ -19,6 +19,11 @@ from prometheus_client import Counter
|
||||
|
||||
from synapse.api.constants import EventTypes, Membership, ThirdPartyEntityKind
|
||||
from synapse.api.errors import CodeMessageException
|
||||
from synapse.appservice import (
|
||||
ApplicationService,
|
||||
TransactionOneTimeKeyCounts,
|
||||
TransactionUnusedFallbackKeys,
|
||||
)
|
||||
from synapse.events import EventBase
|
||||
from synapse.events.utils import serialize_event
|
||||
from synapse.http.client import SimpleHttpClient
|
||||
@@ -26,7 +31,6 @@ from synapse.types import JsonDict, ThirdPartyInstanceID
|
||||
from synapse.util.caches.response_cache import ResponseCache
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from synapse.appservice import ApplicationService
|
||||
from synapse.server import HomeServer
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -219,6 +223,8 @@ class ApplicationServiceApi(SimpleHttpClient):
|
||||
events: List[EventBase],
|
||||
ephemeral: List[JsonDict],
|
||||
to_device_messages: List[JsonDict],
|
||||
one_time_key_counts: TransactionOneTimeKeyCounts,
|
||||
unused_fallback_keys: TransactionUnusedFallbackKeys,
|
||||
txn_id: Optional[int] = None,
|
||||
) -> bool:
|
||||
"""
|
||||
@@ -252,7 +258,7 @@ class ApplicationServiceApi(SimpleHttpClient):
|
||||
uri = service.url + ("/transactions/%s" % urllib.parse.quote(str(txn_id)))
|
||||
|
||||
# Never send ephemeral events to appservices that do not support it
|
||||
body: Dict[str, List[JsonDict]] = {"events": serialized_events}
|
||||
body: JsonDict = {"events": serialized_events}
|
||||
if service.supports_ephemeral:
|
||||
body.update(
|
||||
{
|
||||
@@ -262,6 +268,16 @@ class ApplicationServiceApi(SimpleHttpClient):
|
||||
}
|
||||
)
|
||||
|
||||
if service.msc3202_transaction_extensions:
|
||||
if one_time_key_counts:
|
||||
body[
|
||||
"org.matrix.msc3202.device_one_time_key_counts"
|
||||
] = one_time_key_counts
|
||||
if unused_fallback_keys:
|
||||
body[
|
||||
"org.matrix.msc3202.device_unused_fallback_keys"
|
||||
] = unused_fallback_keys
|
||||
|
||||
try:
|
||||
await self.put_json(
|
||||
uri=uri,
|
||||
|
||||
@@ -54,12 +54,19 @@ from typing import (
|
||||
Callable,
|
||||
Collection,
|
||||
Dict,
|
||||
Iterable,
|
||||
List,
|
||||
Optional,
|
||||
Set,
|
||||
Tuple,
|
||||
)
|
||||
|
||||
from synapse.appservice import ApplicationService, ApplicationServiceState
|
||||
from synapse.appservice import (
|
||||
ApplicationService,
|
||||
ApplicationServiceState,
|
||||
TransactionOneTimeKeyCounts,
|
||||
TransactionUnusedFallbackKeys,
|
||||
)
|
||||
from synapse.appservice.api import ApplicationServiceApi
|
||||
from synapse.events import EventBase
|
||||
from synapse.logging.context import run_in_background
|
||||
@@ -92,11 +99,11 @@ class ApplicationServiceScheduler:
|
||||
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self.clock = hs.get_clock()
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
self.as_api = hs.get_application_service_api()
|
||||
|
||||
self.txn_ctrl = _TransactionController(self.clock, self.store, self.as_api)
|
||||
self.queuer = _ServiceQueuer(self.txn_ctrl, self.clock)
|
||||
self.queuer = _ServiceQueuer(self.txn_ctrl, self.clock, hs)
|
||||
|
||||
async def start(self) -> None:
|
||||
logger.info("Starting appservice scheduler")
|
||||
@@ -153,7 +160,9 @@ class _ServiceQueuer:
|
||||
appservice at a given time.
|
||||
"""
|
||||
|
||||
def __init__(self, txn_ctrl: "_TransactionController", clock: Clock):
|
||||
def __init__(
|
||||
self, txn_ctrl: "_TransactionController", clock: Clock, hs: "HomeServer"
|
||||
):
|
||||
# dict of {service_id: [events]}
|
||||
self.queued_events: Dict[str, List[EventBase]] = {}
|
||||
# dict of {service_id: [events]}
|
||||
@@ -165,6 +174,10 @@ class _ServiceQueuer:
|
||||
self.requests_in_flight: Set[str] = set()
|
||||
self.txn_ctrl = txn_ctrl
|
||||
self.clock = clock
|
||||
self._msc3202_transaction_extensions_enabled: bool = (
|
||||
hs.config.experimental.msc3202_transaction_extensions
|
||||
)
|
||||
self._store = hs.get_datastores().main
|
||||
|
||||
def start_background_request(self, service: ApplicationService) -> None:
|
||||
# start a sender for this appservice if we don't already have one
|
||||
@@ -202,15 +215,84 @@ class _ServiceQueuer:
|
||||
if not events and not ephemeral and not to_device_messages_to_send:
|
||||
return
|
||||
|
||||
one_time_key_counts: Optional[TransactionOneTimeKeyCounts] = None
|
||||
unused_fallback_keys: Optional[TransactionUnusedFallbackKeys] = None
|
||||
|
||||
if (
|
||||
self._msc3202_transaction_extensions_enabled
|
||||
and service.msc3202_transaction_extensions
|
||||
):
|
||||
# Compute the one-time key counts and fallback key usage states
|
||||
# for the users which are mentioned in this transaction,
|
||||
# as well as the appservice's sender.
|
||||
(
|
||||
one_time_key_counts,
|
||||
unused_fallback_keys,
|
||||
) = await self._compute_msc3202_otk_counts_and_fallback_keys(
|
||||
service, events, ephemeral, to_device_messages_to_send
|
||||
)
|
||||
|
||||
try:
|
||||
await self.txn_ctrl.send(
|
||||
service, events, ephemeral, to_device_messages_to_send
|
||||
service,
|
||||
events,
|
||||
ephemeral,
|
||||
to_device_messages_to_send,
|
||||
one_time_key_counts,
|
||||
unused_fallback_keys,
|
||||
)
|
||||
except Exception:
|
||||
logger.exception("AS request failed")
|
||||
finally:
|
||||
self.requests_in_flight.discard(service.id)
|
||||
|
||||
async def _compute_msc3202_otk_counts_and_fallback_keys(
|
||||
self,
|
||||
service: ApplicationService,
|
||||
events: Iterable[EventBase],
|
||||
ephemerals: Iterable[JsonDict],
|
||||
to_device_messages: Iterable[JsonDict],
|
||||
) -> Tuple[TransactionOneTimeKeyCounts, TransactionUnusedFallbackKeys]:
|
||||
"""
|
||||
Given a list of the events, ephemeral messages and to-device messages,
|
||||
- first computes a list of application services users that may have
|
||||
interesting updates to the one-time key counts or fallback key usage.
|
||||
- then computes one-time key counts and fallback key usages for those users.
|
||||
Given a list of application service users that are interesting,
|
||||
compute one-time key counts and fallback key usages for the users.
|
||||
"""
|
||||
|
||||
# Set of 'interesting' users who may have updates
|
||||
users: Set[str] = set()
|
||||
|
||||
# The sender is always included
|
||||
users.add(service.sender)
|
||||
|
||||
# All AS users that would receive the PDUs or EDUs sent to these rooms
|
||||
# are classed as 'interesting'.
|
||||
rooms_of_interesting_users: Set[str] = set()
|
||||
# PDUs
|
||||
rooms_of_interesting_users.update(event.room_id for event in events)
|
||||
# EDUs
|
||||
rooms_of_interesting_users.update(
|
||||
ephemeral["room_id"] for ephemeral in ephemerals
|
||||
)
|
||||
|
||||
# Look up the AS users in those rooms
|
||||
for room_id in rooms_of_interesting_users:
|
||||
users.update(
|
||||
await self._store.get_app_service_users_in_room(room_id, service)
|
||||
)
|
||||
|
||||
# Add recipients of to-device messages.
|
||||
# device_message["user_id"] is the ID of the recipient.
|
||||
users.update(device_message["user_id"] for device_message in to_device_messages)
|
||||
|
||||
# Compute and return the counts / fallback key usage states
|
||||
otk_counts = await self._store.count_bulk_e2e_one_time_keys_for_as(users)
|
||||
unused_fbks = await self._store.get_e2e_bulk_unused_fallback_key_types(users)
|
||||
return otk_counts, unused_fbks
|
||||
|
||||
|
||||
class _TransactionController:
|
||||
"""Transaction manager.
|
||||
@@ -238,6 +320,8 @@ class _TransactionController:
|
||||
events: List[EventBase],
|
||||
ephemeral: Optional[List[JsonDict]] = None,
|
||||
to_device_messages: Optional[List[JsonDict]] = None,
|
||||
one_time_key_counts: Optional[TransactionOneTimeKeyCounts] = None,
|
||||
unused_fallback_keys: Optional[TransactionUnusedFallbackKeys] = None,
|
||||
) -> None:
|
||||
"""
|
||||
Create a transaction with the given data and send to the provided
|
||||
@@ -248,6 +332,10 @@ class _TransactionController:
|
||||
events: The persistent events to include in the transaction.
|
||||
ephemeral: The ephemeral events to include in the transaction.
|
||||
to_device_messages: The to-device messages to include in the transaction.
|
||||
one_time_key_counts: Counts of remaining one-time keys for relevant
|
||||
appservice devices in the transaction.
|
||||
unused_fallback_keys: Lists of unused fallback keys for relevant
|
||||
appservice devices in the transaction.
|
||||
"""
|
||||
try:
|
||||
txn = await self.store.create_appservice_txn(
|
||||
@@ -255,6 +343,8 @@ class _TransactionController:
|
||||
events=events,
|
||||
ephemeral=ephemeral or [],
|
||||
to_device_messages=to_device_messages or [],
|
||||
one_time_key_counts=one_time_key_counts or {},
|
||||
unused_fallback_keys=unused_fallback_keys or {},
|
||||
)
|
||||
service_is_up = await self._is_service_up(service)
|
||||
if service_is_up:
|
||||
|
||||
@@ -166,6 +166,16 @@ def _load_appservice(
|
||||
|
||||
supports_ephemeral = as_info.get("de.sorunome.msc2409.push_ephemeral", False)
|
||||
|
||||
# Opt-in flag for the MSC3202-specific transactional behaviour.
|
||||
# When enabled, appservice transactions contain the following information:
|
||||
# - device One-Time Key counts
|
||||
# - device unused fallback key usage states
|
||||
msc3202_transaction_extensions = as_info.get("org.matrix.msc3202", False)
|
||||
if not isinstance(msc3202_transaction_extensions, bool):
|
||||
raise ValueError(
|
||||
"The `org.matrix.msc3202` option should be true or false if specified."
|
||||
)
|
||||
|
||||
return ApplicationService(
|
||||
token=as_info["as_token"],
|
||||
hostname=hostname,
|
||||
@@ -174,8 +184,9 @@ def _load_appservice(
|
||||
hs_token=as_info["hs_token"],
|
||||
sender=user_id,
|
||||
id=as_info["id"],
|
||||
supports_ephemeral=supports_ephemeral,
|
||||
protocols=protocols,
|
||||
rate_limited=rate_limited,
|
||||
ip_range_whitelist=ip_range_whitelist,
|
||||
supports_ephemeral=supports_ephemeral,
|
||||
msc3202_transaction_extensions=msc3202_transaction_extensions,
|
||||
)
|
||||
|
||||
@@ -20,7 +20,7 @@ from typing import Callable, Dict, Optional
|
||||
|
||||
import attr
|
||||
|
||||
from synapse.python_dependencies import DependencyException, check_requirements
|
||||
from synapse.util.check_dependencies import DependencyException, check_requirements
|
||||
|
||||
from ._base import Config, ConfigError
|
||||
|
||||
|
||||
@@ -47,11 +47,6 @@ class ExperimentalConfig(Config):
|
||||
# MSC3030 (Jump to date API endpoint)
|
||||
self.msc3030_enabled: bool = experimental.get("msc3030_enabled", False)
|
||||
|
||||
# The portion of MSC3202 which is related to device masquerading.
|
||||
self.msc3202_device_masquerading_enabled: bool = experimental.get(
|
||||
"msc3202_device_masquerading", False
|
||||
)
|
||||
|
||||
# MSC2409 (this setting only relates to optionally sending to-device messages).
|
||||
# Presence, typing and read receipt EDUs are already sent to application services that
|
||||
# have opted in to receive them. If enabled, this adds to-device messages to that list.
|
||||
@@ -59,9 +54,23 @@ class ExperimentalConfig(Config):
|
||||
"msc2409_to_device_messages_enabled", False
|
||||
)
|
||||
|
||||
# The portion of MSC3202 which is related to device masquerading.
|
||||
self.msc3202_device_masquerading_enabled: bool = experimental.get(
|
||||
"msc3202_device_masquerading", False
|
||||
)
|
||||
|
||||
# Portion of MSC3202 related to transaction extensions:
|
||||
# sending one-time key counts and fallback key usage to application services.
|
||||
self.msc3202_transaction_extensions: bool = experimental.get(
|
||||
"msc3202_transaction_extensions", False
|
||||
)
|
||||
|
||||
# MSC3706 (server-side support for partial state in /send_join responses)
|
||||
self.msc3706_enabled: bool = experimental.get("msc3706_enabled", False)
|
||||
|
||||
# experimental support for faster joins over federation (msc2775, msc3706)
|
||||
# requires a target server with msc3706_enabled enabled.
|
||||
self.faster_joins_enabled: bool = experimental.get("faster_joins", False)
|
||||
|
||||
# MSC3720 (Account status endpoint)
|
||||
self.msc3720_enabled: bool = experimental.get("msc3720_enabled", False)
|
||||
|
||||
@@ -15,7 +15,7 @@
|
||||
|
||||
import attr
|
||||
|
||||
from synapse.python_dependencies import DependencyException, check_requirements
|
||||
from synapse.util.check_dependencies import DependencyException, check_requirements
|
||||
|
||||
from ._base import Config, ConfigError
|
||||
|
||||
|
||||
@@ -20,11 +20,11 @@ import attr
|
||||
|
||||
from synapse.config._util import validate_config
|
||||
from synapse.config.sso import SsoAttributeRequirement
|
||||
from synapse.python_dependencies import DependencyException, check_requirements
|
||||
from synapse.types import JsonDict
|
||||
from synapse.util.module_loader import load_module
|
||||
from synapse.util.stringutils import parse_and_validate_mxc_uri
|
||||
|
||||
from ..util.check_dependencies import DependencyException, check_requirements
|
||||
from ._base import Config, ConfigError, read_file
|
||||
|
||||
DEFAULT_USER_MAPPING_PROVIDER = "synapse.handlers.oidc.JinjaOidcMappingProvider"
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
# limitations under the License.
|
||||
|
||||
from synapse.config._base import Config
|
||||
from synapse.python_dependencies import check_requirements
|
||||
from synapse.util.check_dependencies import check_requirements
|
||||
|
||||
|
||||
class RedisConfig(Config):
|
||||
|
||||
@@ -20,8 +20,8 @@ from urllib.request import getproxies_environment # type: ignore
|
||||
import attr
|
||||
|
||||
from synapse.config.server import DEFAULT_IP_RANGE_BLACKLIST, generate_ip_set
|
||||
from synapse.python_dependencies import DependencyException, check_requirements
|
||||
from synapse.types import JsonDict
|
||||
from synapse.util.check_dependencies import DependencyException, check_requirements
|
||||
from synapse.util.module_loader import load_module
|
||||
|
||||
from ._base import Config, ConfigError
|
||||
|
||||
@@ -17,8 +17,8 @@ import logging
|
||||
from typing import Any, List, Set
|
||||
|
||||
from synapse.config.sso import SsoAttributeRequirement
|
||||
from synapse.python_dependencies import DependencyException, check_requirements
|
||||
from synapse.types import JsonDict
|
||||
from synapse.util.check_dependencies import DependencyException, check_requirements
|
||||
from synapse.util.module_loader import load_module, load_python_module
|
||||
|
||||
from ._base import Config, ConfigError
|
||||
|
||||
@@ -14,7 +14,7 @@
|
||||
|
||||
from typing import Set
|
||||
|
||||
from synapse.python_dependencies import DependencyException, check_requirements
|
||||
from synapse.util.check_dependencies import DependencyException, check_requirements
|
||||
|
||||
from ._base import Config, ConfigError
|
||||
|
||||
|
||||
@@ -476,7 +476,7 @@ class StoreKeyFetcher(KeyFetcher):
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
super().__init__(hs)
|
||||
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
|
||||
async def _fetch_keys(
|
||||
self, keys_to_fetch: List[_FetchKeyRequest]
|
||||
@@ -498,7 +498,7 @@ class BaseV2KeyFetcher(KeyFetcher):
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
super().__init__(hs)
|
||||
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
self.config = hs.config
|
||||
|
||||
async def process_v2_response(
|
||||
|
||||
@@ -374,9 +374,9 @@ def _is_membership_change_allowed(
|
||||
return
|
||||
|
||||
# Require the user to be in the room for membership changes other than join/knock.
|
||||
if Membership.JOIN != membership and (
|
||||
RoomVersion.msc2403_knocking and Membership.KNOCK != membership
|
||||
):
|
||||
# Note that the room version check for knocking is done implicitly by `caller_knocked`
|
||||
# and the ability to set a membership of `knock` in the first place.
|
||||
if Membership.JOIN != membership and Membership.KNOCK != membership:
|
||||
# If the user has been invited or has knocked, they are allowed to change their
|
||||
# membership event to leave
|
||||
if (
|
||||
|
||||
@@ -189,7 +189,7 @@ class EventBuilderFactory:
|
||||
self.hostname = hs.hostname
|
||||
self.signing_key = hs.signing_key
|
||||
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
self.state = hs.get_state_handler()
|
||||
self._event_auth_handler = hs.get_event_auth_handler()
|
||||
|
||||
|
||||
@@ -101,6 +101,9 @@ class EventContext:
|
||||
|
||||
As with _current_state_ids, this is a private attribute. It should be
|
||||
accessed via get_prev_state_ids.
|
||||
|
||||
partial_state: if True, we may be storing this event with a temporary,
|
||||
incomplete state.
|
||||
"""
|
||||
|
||||
rejected: Union[bool, str] = False
|
||||
@@ -113,12 +116,15 @@ class EventContext:
|
||||
_current_state_ids: Optional[StateMap[str]] = None
|
||||
_prev_state_ids: Optional[StateMap[str]] = None
|
||||
|
||||
partial_state: bool = False
|
||||
|
||||
@staticmethod
|
||||
def with_state(
|
||||
state_group: Optional[int],
|
||||
state_group_before_event: Optional[int],
|
||||
current_state_ids: Optional[StateMap[str]],
|
||||
prev_state_ids: Optional[StateMap[str]],
|
||||
partial_state: bool,
|
||||
prev_group: Optional[int] = None,
|
||||
delta_ids: Optional[StateMap[str]] = None,
|
||||
) -> "EventContext":
|
||||
@@ -129,6 +135,7 @@ class EventContext:
|
||||
state_group_before_event=state_group_before_event,
|
||||
prev_group=prev_group,
|
||||
delta_ids=delta_ids,
|
||||
partial_state=partial_state,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
@@ -170,6 +177,7 @@ class EventContext:
|
||||
"prev_group": self.prev_group,
|
||||
"delta_ids": _encode_state_dict(self.delta_ids),
|
||||
"app_service_id": self.app_service.id if self.app_service else None,
|
||||
"partial_state": self.partial_state,
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
@@ -196,6 +204,7 @@ class EventContext:
|
||||
prev_group=input["prev_group"],
|
||||
delta_ids=_decode_state_dict(input["delta_ids"]),
|
||||
rejected=input["rejected"],
|
||||
partial_state=input.get("partial_state", False),
|
||||
)
|
||||
|
||||
app_service_id = input["app_service_id"]
|
||||
|
||||
@@ -17,6 +17,7 @@ from typing import TYPE_CHECKING, Any, Awaitable, Callable, List, Optional, Tupl
|
||||
from synapse.api.errors import ModuleFailedException, SynapseError
|
||||
from synapse.events import EventBase
|
||||
from synapse.events.snapshot import EventContext
|
||||
from synapse.storage.roommember import ProfileInfo
|
||||
from synapse.types import Requester, StateMap
|
||||
from synapse.util.async_helpers import maybe_awaitable
|
||||
|
||||
@@ -37,6 +38,8 @@ CHECK_VISIBILITY_CAN_BE_MODIFIED_CALLBACK = Callable[
|
||||
[str, StateMap[EventBase], str], Awaitable[bool]
|
||||
]
|
||||
ON_NEW_EVENT_CALLBACK = Callable[[EventBase, StateMap[EventBase]], Awaitable]
|
||||
ON_PROFILE_UPDATE_CALLBACK = Callable[[str, ProfileInfo, bool, bool], Awaitable]
|
||||
ON_USER_DEACTIVATION_STATUS_CHANGED_CALLBACK = Callable[[str, bool, bool], Awaitable]
|
||||
|
||||
|
||||
def load_legacy_third_party_event_rules(hs: "HomeServer") -> None:
|
||||
@@ -143,7 +146,7 @@ class ThirdPartyEventRules:
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self.third_party_rules = None
|
||||
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
|
||||
self._check_event_allowed_callbacks: List[CHECK_EVENT_ALLOWED_CALLBACK] = []
|
||||
self._on_create_room_callbacks: List[ON_CREATE_ROOM_CALLBACK] = []
|
||||
@@ -154,6 +157,10 @@ class ThirdPartyEventRules:
|
||||
CHECK_VISIBILITY_CAN_BE_MODIFIED_CALLBACK
|
||||
] = []
|
||||
self._on_new_event_callbacks: List[ON_NEW_EVENT_CALLBACK] = []
|
||||
self._on_profile_update_callbacks: List[ON_PROFILE_UPDATE_CALLBACK] = []
|
||||
self._on_user_deactivation_status_changed_callbacks: List[
|
||||
ON_USER_DEACTIVATION_STATUS_CHANGED_CALLBACK
|
||||
] = []
|
||||
|
||||
def register_third_party_rules_callbacks(
|
||||
self,
|
||||
@@ -166,6 +173,8 @@ class ThirdPartyEventRules:
|
||||
CHECK_VISIBILITY_CAN_BE_MODIFIED_CALLBACK
|
||||
] = None,
|
||||
on_new_event: Optional[ON_NEW_EVENT_CALLBACK] = None,
|
||||
on_profile_update: Optional[ON_PROFILE_UPDATE_CALLBACK] = None,
|
||||
on_deactivation: Optional[ON_USER_DEACTIVATION_STATUS_CHANGED_CALLBACK] = None,
|
||||
) -> None:
|
||||
"""Register callbacks from modules for each hook."""
|
||||
if check_event_allowed is not None:
|
||||
@@ -187,6 +196,12 @@ class ThirdPartyEventRules:
|
||||
if on_new_event is not None:
|
||||
self._on_new_event_callbacks.append(on_new_event)
|
||||
|
||||
if on_profile_update is not None:
|
||||
self._on_profile_update_callbacks.append(on_profile_update)
|
||||
|
||||
if on_deactivation is not None:
|
||||
self._on_user_deactivation_status_changed_callbacks.append(on_deactivation)
|
||||
|
||||
async def check_event_allowed(
|
||||
self, event: EventBase, context: EventContext
|
||||
) -> Tuple[bool, Optional[dict]]:
|
||||
@@ -334,9 +349,6 @@ class ThirdPartyEventRules:
|
||||
|
||||
Args:
|
||||
event_id: The ID of the event.
|
||||
|
||||
Raises:
|
||||
ModuleFailureError if a callback raised any exception.
|
||||
"""
|
||||
# Bail out early without hitting the store if we don't have any callbacks
|
||||
if len(self._on_new_event_callbacks) == 0:
|
||||
@@ -370,3 +382,41 @@ class ThirdPartyEventRules:
|
||||
state_events[key] = room_state_events[event_id]
|
||||
|
||||
return state_events
|
||||
|
||||
async def on_profile_update(
|
||||
self, user_id: str, new_profile: ProfileInfo, by_admin: bool, deactivation: bool
|
||||
) -> None:
|
||||
"""Called after the global profile of a user has been updated. Does not include
|
||||
per-room profile changes.
|
||||
|
||||
Args:
|
||||
user_id: The user whose profile was changed.
|
||||
new_profile: The updated profile for the user.
|
||||
by_admin: Whether the profile update was performed by a server admin.
|
||||
deactivation: Whether this change was made while deactivating the user.
|
||||
"""
|
||||
for callback in self._on_profile_update_callbacks:
|
||||
try:
|
||||
await callback(user_id, new_profile, by_admin, deactivation)
|
||||
except Exception as e:
|
||||
logger.exception(
|
||||
"Failed to run module API callback %s: %s", callback, e
|
||||
)
|
||||
|
||||
async def on_user_deactivation_status_changed(
|
||||
self, user_id: str, deactivated: bool, by_admin: bool
|
||||
) -> None:
|
||||
"""Called after a user has been deactivated or reactivated.
|
||||
|
||||
Args:
|
||||
user_id: The deactivated user.
|
||||
deactivated: Whether the user is now deactivated.
|
||||
by_admin: Whether the deactivation was performed by a server admin.
|
||||
"""
|
||||
for callback in self._on_user_deactivation_status_changed_callbacks:
|
||||
try:
|
||||
await callback(user_id, deactivated, by_admin)
|
||||
except Exception as e:
|
||||
logger.exception(
|
||||
"Failed to run module API callback %s: %s", callback, e
|
||||
)
|
||||
|
||||
@@ -39,7 +39,7 @@ class FederationBase:
|
||||
self.server_name = hs.hostname
|
||||
self.keyring = hs.get_keyring()
|
||||
self.spam_checker = hs.get_spam_checker()
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
self._clock = hs.get_clock()
|
||||
|
||||
async def _check_sigs_and_hash(
|
||||
|
||||
@@ -56,7 +56,7 @@ from synapse.api.room_versions import (
|
||||
from synapse.events import EventBase, builder
|
||||
from synapse.federation.federation_base import FederationBase, event_from_pdu_json
|
||||
from synapse.federation.transport.client import SendJoinResponse
|
||||
from synapse.types import JsonDict, get_domain_from_id
|
||||
from synapse.types import JsonDict, UserID, get_domain_from_id
|
||||
from synapse.util.async_helpers import concurrently_execute
|
||||
from synapse.util.caches.expiringcache import ExpiringCache
|
||||
from synapse.util.retryutils import NotRetryingDestination
|
||||
@@ -615,11 +615,15 @@ class FederationClient(FederationBase):
|
||||
synapse_error = e.to_synapse_error()
|
||||
# There is no good way to detect an "unknown" endpoint.
|
||||
#
|
||||
# Dendrite returns a 404 (with no body); synapse returns a 400
|
||||
# Dendrite returns a 404 (with a body of "404 page not found");
|
||||
# Conduit returns a 404 (with no body); and Synapse returns a 400
|
||||
# with M_UNRECOGNISED.
|
||||
return e.code == 404 or (
|
||||
e.code == 400 and synapse_error.errcode == Codes.UNRECOGNIZED
|
||||
)
|
||||
#
|
||||
# This needs to be rather specific as some endpoints truly do return 404
|
||||
# errors.
|
||||
return (
|
||||
e.code == 404 and (not e.response or e.response == b"404 page not found")
|
||||
) or (e.code == 400 and synapse_error.errcode == Codes.UNRECOGNIZED)
|
||||
|
||||
async def _try_destination_list(
|
||||
self,
|
||||
@@ -1002,7 +1006,7 @@ class FederationClient(FederationBase):
|
||||
)
|
||||
except HttpResponseException as e:
|
||||
# If an error is received that is due to an unrecognised endpoint,
|
||||
# fallback to the v1 endpoint. Otherwise consider it a legitmate error
|
||||
# fallback to the v1 endpoint. Otherwise, consider it a legitimate error
|
||||
# and raise.
|
||||
if not self._is_unknown_endpoint(e):
|
||||
raise
|
||||
@@ -1071,7 +1075,7 @@ class FederationClient(FederationBase):
|
||||
except HttpResponseException as e:
|
||||
# If an error is received that is due to an unrecognised endpoint,
|
||||
# fallback to the v1 endpoint if the room uses old-style event IDs.
|
||||
# Otherwise consider it a legitmate error and raise.
|
||||
# Otherwise, consider it a legitimate error and raise.
|
||||
err = e.to_synapse_error()
|
||||
if self._is_unknown_endpoint(e, err):
|
||||
if room_version.event_format != EventFormatVersions.V1:
|
||||
@@ -1132,7 +1136,7 @@ class FederationClient(FederationBase):
|
||||
)
|
||||
except HttpResponseException as e:
|
||||
# If an error is received that is due to an unrecognised endpoint,
|
||||
# fallback to the v1 endpoint. Otherwise consider it a legitmate error
|
||||
# fallback to the v1 endpoint. Otherwise, consider it a legitimate error
|
||||
# and raise.
|
||||
if not self._is_unknown_endpoint(e):
|
||||
raise
|
||||
@@ -1358,61 +1362,6 @@ class FederationClient(FederationBase):
|
||||
# server doesn't give it to us.
|
||||
return None
|
||||
|
||||
async def get_space_summary(
|
||||
self,
|
||||
destinations: Iterable[str],
|
||||
room_id: str,
|
||||
suggested_only: bool,
|
||||
max_rooms_per_space: Optional[int],
|
||||
exclude_rooms: List[str],
|
||||
) -> "FederationSpaceSummaryResult":
|
||||
"""
|
||||
Call other servers to get a summary of the given space
|
||||
|
||||
|
||||
Args:
|
||||
destinations: The remote servers. We will try them in turn, omitting any
|
||||
that have been blacklisted.
|
||||
|
||||
room_id: ID of the space to be queried
|
||||
|
||||
suggested_only: If true, ask the remote server to only return children
|
||||
with the "suggested" flag set
|
||||
|
||||
max_rooms_per_space: A limit on the number of children to return for each
|
||||
space
|
||||
|
||||
exclude_rooms: A list of room IDs to tell the remote server to skip
|
||||
|
||||
Returns:
|
||||
a parsed FederationSpaceSummaryResult
|
||||
|
||||
Raises:
|
||||
SynapseError if we were unable to get a valid summary from any of the
|
||||
remote servers
|
||||
"""
|
||||
|
||||
async def send_request(destination: str) -> FederationSpaceSummaryResult:
|
||||
res = await self.transport_layer.get_space_summary(
|
||||
destination=destination,
|
||||
room_id=room_id,
|
||||
suggested_only=suggested_only,
|
||||
max_rooms_per_space=max_rooms_per_space,
|
||||
exclude_rooms=exclude_rooms,
|
||||
)
|
||||
|
||||
try:
|
||||
return FederationSpaceSummaryResult.from_json_dict(res)
|
||||
except ValueError as e:
|
||||
raise InvalidResponseError(str(e))
|
||||
|
||||
return await self._try_destination_list(
|
||||
"fetch space summary",
|
||||
destinations,
|
||||
send_request,
|
||||
failover_on_unknown_endpoint=True,
|
||||
)
|
||||
|
||||
async def get_room_hierarchy(
|
||||
self,
|
||||
destinations: Iterable[str],
|
||||
@@ -1458,8 +1407,8 @@ class FederationClient(FederationBase):
|
||||
)
|
||||
except HttpResponseException as e:
|
||||
# If an error is received that is due to an unrecognised endpoint,
|
||||
# fallback to the unstable endpoint. Otherwise consider it a
|
||||
# legitmate error and raise.
|
||||
# fallback to the unstable endpoint. Otherwise, consider it a
|
||||
# legitimate error and raise.
|
||||
if not self._is_unknown_endpoint(e):
|
||||
raise
|
||||
|
||||
@@ -1484,10 +1433,8 @@ class FederationClient(FederationBase):
|
||||
if any(not isinstance(e, dict) for e in children_state):
|
||||
raise InvalidResponseError("Invalid event in 'children_state' list")
|
||||
try:
|
||||
[
|
||||
FederationSpaceSummaryEventResult.from_json_dict(e)
|
||||
for e in children_state
|
||||
]
|
||||
for child_state in children_state:
|
||||
_validate_hierarchy_event(child_state)
|
||||
except ValueError as e:
|
||||
raise InvalidResponseError(str(e))
|
||||
|
||||
@@ -1509,62 +1456,12 @@ class FederationClient(FederationBase):
|
||||
|
||||
return room, children_state, children, inaccessible_children
|
||||
|
||||
try:
|
||||
result = await self._try_destination_list(
|
||||
"fetch room hierarchy",
|
||||
destinations,
|
||||
send_request,
|
||||
failover_on_unknown_endpoint=True,
|
||||
)
|
||||
except SynapseError as e:
|
||||
# If an unexpected error occurred, re-raise it.
|
||||
if e.code != 502:
|
||||
raise
|
||||
|
||||
logger.debug(
|
||||
"Couldn't fetch room hierarchy, falling back to the spaces API"
|
||||
)
|
||||
|
||||
# Fallback to the old federation API and translate the results if
|
||||
# no servers implement the new API.
|
||||
#
|
||||
# The algorithm below is a bit inefficient as it only attempts to
|
||||
# parse information for the requested room, but the legacy API may
|
||||
# return additional layers.
|
||||
legacy_result = await self.get_space_summary(
|
||||
destinations,
|
||||
room_id,
|
||||
suggested_only,
|
||||
max_rooms_per_space=None,
|
||||
exclude_rooms=[],
|
||||
)
|
||||
|
||||
# Find the requested room in the response (and remove it).
|
||||
for _i, room in enumerate(legacy_result.rooms):
|
||||
if room.get("room_id") == room_id:
|
||||
break
|
||||
else:
|
||||
# The requested room was not returned, nothing we can do.
|
||||
raise
|
||||
requested_room = legacy_result.rooms.pop(_i)
|
||||
|
||||
# Find any children events of the requested room.
|
||||
children_events = []
|
||||
children_room_ids = set()
|
||||
for event in legacy_result.events:
|
||||
if event.room_id == room_id:
|
||||
children_events.append(event.data)
|
||||
children_room_ids.add(event.state_key)
|
||||
|
||||
# Find the children rooms.
|
||||
children = []
|
||||
for room in legacy_result.rooms:
|
||||
if room.get("room_id") in children_room_ids:
|
||||
children.append(room)
|
||||
|
||||
# It isn't clear from the response whether some of the rooms are
|
||||
# not accessible.
|
||||
result = (requested_room, children_events, children, ())
|
||||
result = await self._try_destination_list(
|
||||
"fetch room hierarchy",
|
||||
destinations,
|
||||
send_request,
|
||||
failover_on_unknown_endpoint=True,
|
||||
)
|
||||
|
||||
# Cache the result to avoid fetching data over federation every time.
|
||||
self._get_room_hierarchy_cache[(room_id, suggested_only)] = result
|
||||
@@ -1610,6 +1507,64 @@ class FederationClient(FederationBase):
|
||||
except ValueError as e:
|
||||
raise InvalidResponseError(str(e))
|
||||
|
||||
async def get_account_status(
|
||||
self, destination: str, user_ids: List[str]
|
||||
) -> Tuple[JsonDict, List[str]]:
|
||||
"""Retrieves account statuses for a given list of users on a given remote
|
||||
homeserver.
|
||||
|
||||
If the request fails for any reason, all user IDs for this destination are marked
|
||||
as failed.
|
||||
|
||||
Args:
|
||||
destination: the destination to contact
|
||||
user_ids: the user ID(s) for which to request account status(es)
|
||||
|
||||
Returns:
|
||||
The account statuses, as well as the list of user IDs for which it was not
|
||||
possible to retrieve a status.
|
||||
"""
|
||||
try:
|
||||
res = await self.transport_layer.get_account_status(destination, user_ids)
|
||||
except Exception:
|
||||
# If the query failed for any reason, mark all the users as failed.
|
||||
return {}, user_ids
|
||||
|
||||
statuses = res.get("account_statuses", {})
|
||||
failures = res.get("failures", [])
|
||||
|
||||
if not isinstance(statuses, dict) or not isinstance(failures, list):
|
||||
# Make sure we're not feeding back malformed data back to the caller.
|
||||
logger.warning(
|
||||
"Destination %s responded with malformed data to account_status query",
|
||||
destination,
|
||||
)
|
||||
return {}, user_ids
|
||||
|
||||
for user_id in user_ids:
|
||||
# Any account whose status is missing is a user we failed to receive the
|
||||
# status of.
|
||||
if user_id not in statuses and user_id not in failures:
|
||||
failures.append(user_id)
|
||||
|
||||
# Filter out any user ID that doesn't belong to the remote server that sent its
|
||||
# status (or failure).
|
||||
def filter_user_id(user_id: str) -> bool:
|
||||
try:
|
||||
return UserID.from_string(user_id).domain == destination
|
||||
except SynapseError:
|
||||
# If the user ID doesn't parse, ignore it.
|
||||
return False
|
||||
|
||||
filtered_statuses = dict(
|
||||
# item is a (key, value) tuple, so item[0] is the user ID.
|
||||
filter(lambda item: filter_user_id(item[0]), statuses.items())
|
||||
)
|
||||
|
||||
filtered_failures = list(filter(filter_user_id, failures))
|
||||
|
||||
return filtered_statuses, filtered_failures
|
||||
|
||||
|
||||
@attr.s(frozen=True, slots=True, auto_attribs=True)
|
||||
class TimestampToEventResponse:
|
||||
@@ -1648,89 +1603,34 @@ class TimestampToEventResponse:
|
||||
return cls(event_id, origin_server_ts, d)
|
||||
|
||||
|
||||
@attr.s(frozen=True, slots=True, auto_attribs=True)
|
||||
class FederationSpaceSummaryEventResult:
|
||||
"""Represents a single event in the result of a successful get_space_summary call.
|
||||
def _validate_hierarchy_event(d: JsonDict) -> None:
|
||||
"""Validate an event within the result of a /hierarchy request
|
||||
|
||||
It's essentially just a serialised event object, but we do a bit of parsing and
|
||||
validation in `from_json_dict` and store some of the validated properties in
|
||||
object attributes.
|
||||
Args:
|
||||
d: json object to be parsed
|
||||
|
||||
Raises:
|
||||
ValueError if d is not a valid event
|
||||
"""
|
||||
|
||||
event_type: str
|
||||
room_id: str
|
||||
state_key: str
|
||||
via: Sequence[str]
|
||||
event_type = d.get("type")
|
||||
if not isinstance(event_type, str):
|
||||
raise ValueError("Invalid event: 'event_type' must be a str")
|
||||
|
||||
# the raw data, including the above keys
|
||||
data: JsonDict
|
||||
room_id = d.get("room_id")
|
||||
if not isinstance(room_id, str):
|
||||
raise ValueError("Invalid event: 'room_id' must be a str")
|
||||
|
||||
@classmethod
|
||||
def from_json_dict(cls, d: JsonDict) -> "FederationSpaceSummaryEventResult":
|
||||
"""Parse an event within the result of a /spaces/ request
|
||||
state_key = d.get("state_key")
|
||||
if not isinstance(state_key, str):
|
||||
raise ValueError("Invalid event: 'state_key' must be a str")
|
||||
|
||||
Args:
|
||||
d: json object to be parsed
|
||||
content = d.get("content")
|
||||
if not isinstance(content, dict):
|
||||
raise ValueError("Invalid event: 'content' must be a dict")
|
||||
|
||||
Raises:
|
||||
ValueError if d is not a valid event
|
||||
"""
|
||||
|
||||
event_type = d.get("type")
|
||||
if not isinstance(event_type, str):
|
||||
raise ValueError("Invalid event: 'event_type' must be a str")
|
||||
|
||||
room_id = d.get("room_id")
|
||||
if not isinstance(room_id, str):
|
||||
raise ValueError("Invalid event: 'room_id' must be a str")
|
||||
|
||||
state_key = d.get("state_key")
|
||||
if not isinstance(state_key, str):
|
||||
raise ValueError("Invalid event: 'state_key' must be a str")
|
||||
|
||||
content = d.get("content")
|
||||
if not isinstance(content, dict):
|
||||
raise ValueError("Invalid event: 'content' must be a dict")
|
||||
|
||||
via = content.get("via")
|
||||
if not isinstance(via, Sequence):
|
||||
raise ValueError("Invalid event: 'via' must be a list")
|
||||
if any(not isinstance(v, str) for v in via):
|
||||
raise ValueError("Invalid event: 'via' must be a list of strings")
|
||||
|
||||
return cls(event_type, room_id, state_key, via, d)
|
||||
|
||||
|
||||
@attr.s(frozen=True, slots=True, auto_attribs=True)
|
||||
class FederationSpaceSummaryResult:
|
||||
"""Represents the data returned by a successful get_space_summary call."""
|
||||
|
||||
rooms: List[JsonDict]
|
||||
events: Sequence[FederationSpaceSummaryEventResult]
|
||||
|
||||
@classmethod
|
||||
def from_json_dict(cls, d: JsonDict) -> "FederationSpaceSummaryResult":
|
||||
"""Parse the result of a /spaces/ request
|
||||
|
||||
Args:
|
||||
d: json object to be parsed
|
||||
|
||||
Raises:
|
||||
ValueError if d is not a valid /spaces/ response
|
||||
"""
|
||||
rooms = d.get("rooms")
|
||||
if not isinstance(rooms, List):
|
||||
raise ValueError("'rooms' must be a list")
|
||||
if any(not isinstance(r, dict) for r in rooms):
|
||||
raise ValueError("Invalid room in 'rooms' list")
|
||||
|
||||
events = d.get("events")
|
||||
if not isinstance(events, Sequence):
|
||||
raise ValueError("'events' must be a list")
|
||||
if any(not isinstance(e, dict) for e in events):
|
||||
raise ValueError("Invalid event in 'events' list")
|
||||
parsed_events = [
|
||||
FederationSpaceSummaryEventResult.from_json_dict(e) for e in events
|
||||
]
|
||||
|
||||
return cls(rooms, parsed_events)
|
||||
via = content.get("via")
|
||||
if not isinstance(via, Sequence):
|
||||
raise ValueError("Invalid event: 'via' must be a list")
|
||||
if any(not isinstance(v, str) for v in via):
|
||||
raise ValueError("Invalid event: 'via' must be a list of strings")
|
||||
|
||||
@@ -228,7 +228,7 @@ class FederationSender(AbstractFederationSender):
|
||||
self.hs = hs
|
||||
self.server_name = hs.hostname
|
||||
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
self.state = hs.get_state_handler()
|
||||
|
||||
self.clock = hs.get_clock()
|
||||
|
||||
@@ -76,7 +76,7 @@ class PerDestinationQueue:
|
||||
):
|
||||
self._server_name = hs.hostname
|
||||
self._clock = hs.get_clock()
|
||||
self._store = hs.get_datastore()
|
||||
self._store = hs.get_datastores().main
|
||||
self._transaction_manager = transaction_manager
|
||||
self._instance_name = hs.get_instance_name()
|
||||
self._federation_shard_config = hs.config.worker.federation_shard_config
|
||||
@@ -381,9 +381,8 @@ class PerDestinationQueue:
|
||||
)
|
||||
)
|
||||
|
||||
last_successful_stream_ordering = self._last_successful_stream_ordering
|
||||
|
||||
if last_successful_stream_ordering is None:
|
||||
_tmp_last_successful_stream_ordering = self._last_successful_stream_ordering
|
||||
if _tmp_last_successful_stream_ordering is None:
|
||||
# if it's still None, then this means we don't have the information
|
||||
# in our database we haven't successfully sent a PDU to this server
|
||||
# (at least since the introduction of the feature tracking
|
||||
@@ -393,6 +392,8 @@ class PerDestinationQueue:
|
||||
self._catching_up = False
|
||||
return
|
||||
|
||||
last_successful_stream_ordering: int = _tmp_last_successful_stream_ordering
|
||||
|
||||
# get at most 50 catchup room/PDUs
|
||||
while True:
|
||||
event_ids = await self._store.get_catch_up_room_event_ids(
|
||||
|
||||
@@ -53,7 +53,7 @@ class TransactionManager:
|
||||
def __init__(self, hs: "synapse.server.HomeServer"):
|
||||
self._server_name = hs.hostname
|
||||
self.clock = hs.get_clock() # nb must be called this for @measure_func
|
||||
self._store = hs.get_datastore()
|
||||
self._store = hs.get_datastores().main
|
||||
self._transaction_actions = TransactionActions(self._store)
|
||||
self._transport_layer = hs.get_federation_transport_client()
|
||||
|
||||
|
||||
@@ -258,8 +258,9 @@ class TransportLayerClient:
|
||||
args: dict,
|
||||
retry_on_dns_fail: bool,
|
||||
ignore_backoff: bool = False,
|
||||
prefix: str = FEDERATION_V1_PREFIX,
|
||||
) -> JsonDict:
|
||||
path = _create_v1_path("/query/%s", query_type)
|
||||
path = _create_path(prefix, "/query/%s", query_type)
|
||||
|
||||
return await self.client.get_json(
|
||||
destination=destination,
|
||||
@@ -1178,39 +1179,6 @@ class TransportLayerClient:
|
||||
|
||||
return await self.client.get_json(destination=destination, path=path)
|
||||
|
||||
async def get_space_summary(
|
||||
self,
|
||||
destination: str,
|
||||
room_id: str,
|
||||
suggested_only: bool,
|
||||
max_rooms_per_space: Optional[int],
|
||||
exclude_rooms: List[str],
|
||||
) -> JsonDict:
|
||||
"""
|
||||
Args:
|
||||
destination: The remote server
|
||||
room_id: The room ID to ask about.
|
||||
suggested_only: if True, only suggested rooms will be returned
|
||||
max_rooms_per_space: an optional limit to the number of children to be
|
||||
returned per space
|
||||
exclude_rooms: a list of any rooms we can skip
|
||||
"""
|
||||
# TODO When switching to the stable endpoint, use GET instead of POST.
|
||||
path = _create_path(
|
||||
FEDERATION_UNSTABLE_PREFIX, "/org.matrix.msc2946/spaces/%s", room_id
|
||||
)
|
||||
|
||||
params = {
|
||||
"suggested_only": suggested_only,
|
||||
"exclude_rooms": exclude_rooms,
|
||||
}
|
||||
if max_rooms_per_space is not None:
|
||||
params["max_rooms_per_space"] = max_rooms_per_space
|
||||
|
||||
return await self.client.post_json(
|
||||
destination=destination, path=path, data=params
|
||||
)
|
||||
|
||||
async def get_room_hierarchy(
|
||||
self, destination: str, room_id: str, suggested_only: bool
|
||||
) -> JsonDict:
|
||||
@@ -1247,6 +1215,22 @@ class TransportLayerClient:
|
||||
args={"suggested_only": "true" if suggested_only else "false"},
|
||||
)
|
||||
|
||||
async def get_account_status(
|
||||
self, destination: str, user_ids: List[str]
|
||||
) -> JsonDict:
|
||||
"""
|
||||
Args:
|
||||
destination: The remote server.
|
||||
user_ids: The user ID(s) for which to request account status(es).
|
||||
"""
|
||||
path = _create_path(
|
||||
FEDERATION_UNSTABLE_PREFIX, "/org.matrix.msc3720/account_status"
|
||||
)
|
||||
|
||||
return await self.client.post_json(
|
||||
destination=destination, path=path, data={"user_ids": user_ids}
|
||||
)
|
||||
|
||||
|
||||
def _create_path(federation_prefix: str, path: str, *args: str) -> str:
|
||||
"""
|
||||
|
||||
@@ -24,6 +24,7 @@ from synapse.federation.transport.server._base import (
|
||||
)
|
||||
from synapse.federation.transport.server.federation import (
|
||||
FEDERATION_SERVLET_CLASSES,
|
||||
FederationAccountStatusServlet,
|
||||
FederationTimestampLookupServlet,
|
||||
)
|
||||
from synapse.federation.transport.server.groups_local import GROUP_LOCAL_SERVLET_CLASSES
|
||||
@@ -336,6 +337,13 @@ def register_servlets(
|
||||
):
|
||||
continue
|
||||
|
||||
# Only allow the `/account_status` servlet if msc3720 is enabled
|
||||
if (
|
||||
servletclass == FederationAccountStatusServlet
|
||||
and not hs.config.experimental.msc3720_enabled
|
||||
):
|
||||
continue
|
||||
|
||||
servletclass(
|
||||
hs=hs,
|
||||
authenticator=authenticator,
|
||||
|
||||
@@ -55,7 +55,7 @@ class Authenticator:
|
||||
self._clock = hs.get_clock()
|
||||
self.keyring = hs.get_keyring()
|
||||
self.server_name = hs.hostname
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
self.federation_domain_whitelist = (
|
||||
hs.config.federation.federation_domain_whitelist
|
||||
)
|
||||
|
||||
@@ -110,7 +110,7 @@ class FederationSendServlet(BaseFederationServerServlet):
|
||||
if issue_8631_logger.isEnabledFor(logging.DEBUG):
|
||||
DEVICE_UPDATE_EDUS = ["m.device_list_update", "m.signing_key_update"]
|
||||
device_list_updates = [
|
||||
edu.content
|
||||
edu.get("content", {})
|
||||
for edu in transaction_data.get("edus", [])
|
||||
if edu.get("edu_type") in DEVICE_UPDATE_EDUS
|
||||
]
|
||||
@@ -624,81 +624,6 @@ class FederationVersionServlet(BaseFederationServlet):
|
||||
)
|
||||
|
||||
|
||||
class FederationSpaceSummaryServlet(BaseFederationServlet):
|
||||
PREFIX = FEDERATION_UNSTABLE_PREFIX + "/org.matrix.msc2946"
|
||||
PATH = "/spaces/(?P<room_id>[^/]*)"
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
hs: "HomeServer",
|
||||
authenticator: Authenticator,
|
||||
ratelimiter: FederationRateLimiter,
|
||||
server_name: str,
|
||||
):
|
||||
super().__init__(hs, authenticator, ratelimiter, server_name)
|
||||
self.handler = hs.get_room_summary_handler()
|
||||
|
||||
async def on_GET(
|
||||
self,
|
||||
origin: str,
|
||||
content: Literal[None],
|
||||
query: Mapping[bytes, Sequence[bytes]],
|
||||
room_id: str,
|
||||
) -> Tuple[int, JsonDict]:
|
||||
suggested_only = parse_boolean_from_args(query, "suggested_only", default=False)
|
||||
|
||||
max_rooms_per_space = parse_integer_from_args(query, "max_rooms_per_space")
|
||||
if max_rooms_per_space is not None and max_rooms_per_space < 0:
|
||||
raise SynapseError(
|
||||
400,
|
||||
"Value for 'max_rooms_per_space' must be a non-negative integer",
|
||||
Codes.BAD_JSON,
|
||||
)
|
||||
|
||||
exclude_rooms = parse_strings_from_args(query, "exclude_rooms", default=[])
|
||||
|
||||
return 200, await self.handler.federation_space_summary(
|
||||
origin, room_id, suggested_only, max_rooms_per_space, exclude_rooms
|
||||
)
|
||||
|
||||
# TODO When switching to the stable endpoint, remove the POST handler.
|
||||
async def on_POST(
|
||||
self,
|
||||
origin: str,
|
||||
content: JsonDict,
|
||||
query: Mapping[bytes, Sequence[bytes]],
|
||||
room_id: str,
|
||||
) -> Tuple[int, JsonDict]:
|
||||
suggested_only = content.get("suggested_only", False)
|
||||
if not isinstance(suggested_only, bool):
|
||||
raise SynapseError(
|
||||
400, "'suggested_only' must be a boolean", Codes.BAD_JSON
|
||||
)
|
||||
|
||||
exclude_rooms = content.get("exclude_rooms", [])
|
||||
if not isinstance(exclude_rooms, list) or any(
|
||||
not isinstance(x, str) for x in exclude_rooms
|
||||
):
|
||||
raise SynapseError(400, "bad value for 'exclude_rooms'", Codes.BAD_JSON)
|
||||
|
||||
max_rooms_per_space = content.get("max_rooms_per_space")
|
||||
if max_rooms_per_space is not None:
|
||||
if not isinstance(max_rooms_per_space, int):
|
||||
raise SynapseError(
|
||||
400, "bad value for 'max_rooms_per_space'", Codes.BAD_JSON
|
||||
)
|
||||
if max_rooms_per_space < 0:
|
||||
raise SynapseError(
|
||||
400,
|
||||
"Value for 'max_rooms_per_space' must be a non-negative integer",
|
||||
Codes.BAD_JSON,
|
||||
)
|
||||
|
||||
return 200, await self.handler.federation_space_summary(
|
||||
origin, room_id, suggested_only, max_rooms_per_space, exclude_rooms
|
||||
)
|
||||
|
||||
|
||||
class FederationRoomHierarchyServlet(BaseFederationServlet):
|
||||
PATH = "/hierarchy/(?P<room_id>[^/]*)"
|
||||
|
||||
@@ -746,7 +671,7 @@ class RoomComplexityServlet(BaseFederationServlet):
|
||||
server_name: str,
|
||||
):
|
||||
super().__init__(hs, authenticator, ratelimiter, server_name)
|
||||
self._store = self.hs.get_datastore()
|
||||
self._store = self.hs.get_datastores().main
|
||||
|
||||
async def on_GET(
|
||||
self,
|
||||
@@ -766,6 +691,40 @@ class RoomComplexityServlet(BaseFederationServlet):
|
||||
return 200, complexity
|
||||
|
||||
|
||||
class FederationAccountStatusServlet(BaseFederationServerServlet):
|
||||
PATH = "/query/account_status"
|
||||
PREFIX = FEDERATION_UNSTABLE_PREFIX + "/org.matrix.msc3720"
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
hs: "HomeServer",
|
||||
authenticator: Authenticator,
|
||||
ratelimiter: FederationRateLimiter,
|
||||
server_name: str,
|
||||
):
|
||||
super().__init__(hs, authenticator, ratelimiter, server_name)
|
||||
self._account_handler = hs.get_account_handler()
|
||||
|
||||
async def on_POST(
|
||||
self,
|
||||
origin: str,
|
||||
content: JsonDict,
|
||||
query: Mapping[bytes, Sequence[bytes]],
|
||||
room_id: str,
|
||||
) -> Tuple[int, JsonDict]:
|
||||
if "user_ids" not in content:
|
||||
raise SynapseError(
|
||||
400, "Required parameter 'user_ids' is missing", Codes.MISSING_PARAM
|
||||
)
|
||||
|
||||
statuses, failures = await self._account_handler.get_account_statuses(
|
||||
content["user_ids"],
|
||||
allow_remote=False,
|
||||
)
|
||||
|
||||
return 200, {"account_statuses": statuses, "failures": failures}
|
||||
|
||||
|
||||
FEDERATION_SERVLET_CLASSES: Tuple[Type[BaseFederationServlet], ...] = (
|
||||
FederationSendServlet,
|
||||
FederationEventServlet,
|
||||
@@ -792,9 +751,9 @@ FEDERATION_SERVLET_CLASSES: Tuple[Type[BaseFederationServlet], ...] = (
|
||||
On3pidBindServlet,
|
||||
FederationVersionServlet,
|
||||
RoomComplexityServlet,
|
||||
FederationSpaceSummaryServlet,
|
||||
FederationRoomHierarchyServlet,
|
||||
FederationRoomHierarchyUnstableServlet,
|
||||
FederationV1SendKnockServlet,
|
||||
FederationMakeKnockServlet,
|
||||
FederationAccountStatusServlet,
|
||||
)
|
||||
|
||||
@@ -140,7 +140,7 @@ class GroupAttestionRenewer:
|
||||
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self.clock = hs.get_clock()
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
self.assestations = hs.get_groups_attestation_signing()
|
||||
self.transport_client = hs.get_federation_transport_client()
|
||||
self.is_mine_id = hs.is_mine_id
|
||||
|
||||
@@ -45,7 +45,7 @@ MAX_LONG_DESC_LEN = 10000
|
||||
class GroupsServerWorkerHandler:
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self.hs = hs
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
self.room_list_handler = hs.get_room_list_handler()
|
||||
self.auth = hs.get_auth()
|
||||
self.clock = hs.get_clock()
|
||||
|
||||
@@ -0,0 +1,144 @@
|
||||
# Copyright 2022 The Matrix.org Foundation C.I.C.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from typing import TYPE_CHECKING, Dict, List, Tuple
|
||||
|
||||
from synapse.api.errors import Codes, SynapseError
|
||||
from synapse.types import JsonDict, UserID
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from synapse.server import HomeServer
|
||||
|
||||
|
||||
class AccountHandler:
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self._main_store = hs.get_datastores().main
|
||||
self._is_mine = hs.is_mine
|
||||
self._federation_client = hs.get_federation_client()
|
||||
|
||||
async def get_account_statuses(
|
||||
self,
|
||||
user_ids: List[str],
|
||||
allow_remote: bool,
|
||||
) -> Tuple[JsonDict, List[str]]:
|
||||
"""Get account statuses for a list of user IDs.
|
||||
|
||||
If one or more account(s) belong to remote homeservers, retrieve their status(es)
|
||||
over federation if allowed.
|
||||
|
||||
Args:
|
||||
user_ids: The list of accounts to retrieve the status of.
|
||||
allow_remote: Whether to try to retrieve the status of remote accounts, if
|
||||
any.
|
||||
|
||||
Returns:
|
||||
The account statuses as well as the list of users whose statuses could not be
|
||||
retrieved.
|
||||
|
||||
Raises:
|
||||
SynapseError if a required parameter is missing or malformed, or if one of
|
||||
the accounts isn't local to this homeserver and allow_remote is False.
|
||||
"""
|
||||
statuses = {}
|
||||
failures = []
|
||||
remote_users: List[UserID] = []
|
||||
|
||||
for raw_user_id in user_ids:
|
||||
try:
|
||||
user_id = UserID.from_string(raw_user_id)
|
||||
except SynapseError:
|
||||
raise SynapseError(
|
||||
400,
|
||||
f"Not a valid Matrix user ID: {raw_user_id}",
|
||||
Codes.INVALID_PARAM,
|
||||
)
|
||||
|
||||
if self._is_mine(user_id):
|
||||
status = await self._get_local_account_status(user_id)
|
||||
statuses[user_id.to_string()] = status
|
||||
else:
|
||||
if not allow_remote:
|
||||
raise SynapseError(
|
||||
400,
|
||||
f"Not a local user: {raw_user_id}",
|
||||
Codes.INVALID_PARAM,
|
||||
)
|
||||
|
||||
remote_users.append(user_id)
|
||||
|
||||
if allow_remote and len(remote_users) > 0:
|
||||
remote_statuses, remote_failures = await self._get_remote_account_statuses(
|
||||
remote_users,
|
||||
)
|
||||
|
||||
statuses.update(remote_statuses)
|
||||
failures += remote_failures
|
||||
|
||||
return statuses, failures
|
||||
|
||||
async def _get_local_account_status(self, user_id: UserID) -> JsonDict:
|
||||
"""Retrieve the status of a local account.
|
||||
|
||||
Args:
|
||||
user_id: The account to retrieve the status of.
|
||||
|
||||
Returns:
|
||||
The account's status.
|
||||
"""
|
||||
status = {"exists": False}
|
||||
|
||||
userinfo = await self._main_store.get_userinfo_by_id(user_id.to_string())
|
||||
|
||||
if userinfo is not None:
|
||||
status = {
|
||||
"exists": True,
|
||||
"deactivated": userinfo.is_deactivated,
|
||||
}
|
||||
|
||||
return status
|
||||
|
||||
async def _get_remote_account_statuses(
|
||||
self, remote_users: List[UserID]
|
||||
) -> Tuple[JsonDict, List[str]]:
|
||||
"""Send out federation requests to retrieve the statuses of remote accounts.
|
||||
|
||||
Args:
|
||||
remote_users: The accounts to retrieve the statuses of.
|
||||
|
||||
Returns:
|
||||
The statuses of the accounts, and a list of accounts for which no status
|
||||
could be retrieved.
|
||||
"""
|
||||
# Group remote users by destination, so we only send one request per remote
|
||||
# homeserver.
|
||||
by_destination: Dict[str, List[str]] = {}
|
||||
for user in remote_users:
|
||||
if user.domain not in by_destination:
|
||||
by_destination[user.domain] = []
|
||||
|
||||
by_destination[user.domain].append(user.to_string())
|
||||
|
||||
# Retrieve the statuses and failures for remote accounts.
|
||||
final_statuses: JsonDict = {}
|
||||
final_failures: List[str] = []
|
||||
for destination, users in by_destination.items():
|
||||
statuses, failures = await self._federation_client.get_account_status(
|
||||
destination,
|
||||
users,
|
||||
)
|
||||
|
||||
final_statuses.update(statuses)
|
||||
final_failures += failures
|
||||
|
||||
return final_statuses, final_failures
|
||||
@@ -30,7 +30,7 @@ if TYPE_CHECKING:
|
||||
|
||||
class AccountDataHandler:
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self._store = hs.get_datastore()
|
||||
self._store = hs.get_datastores().main
|
||||
self._instance_name = hs.get_instance_name()
|
||||
self._notifier = hs.get_notifier()
|
||||
|
||||
@@ -166,7 +166,7 @@ class AccountDataHandler:
|
||||
|
||||
class AccountDataEventSource(EventSource[int, JsonDict]):
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
|
||||
def get_current_key(self, direction: str = "f") -> int:
|
||||
return self.store.get_max_account_data_stream_id()
|
||||
|
||||
@@ -43,7 +43,7 @@ class AccountValidityHandler:
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self.hs = hs
|
||||
self.config = hs.config
|
||||
self.store = self.hs.get_datastore()
|
||||
self.store = self.hs.get_datastores().main
|
||||
self.send_email_handler = self.hs.get_send_email_handler()
|
||||
self.clock = self.hs.get_clock()
|
||||
|
||||
|
||||
@@ -29,7 +29,7 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
class AdminHandler:
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
self.storage = hs.get_storage()
|
||||
self.state_store = self.storage.state
|
||||
|
||||
|
||||
@@ -47,7 +47,7 @@ events_processed_counter = Counter("synapse_handlers_appservice_events_processed
|
||||
|
||||
class ApplicationServicesHandler:
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
self.is_mine_id = hs.is_mine_id
|
||||
self.appservice_api = hs.get_application_service_api()
|
||||
self.scheduler = hs.get_application_service_scheduler()
|
||||
|
||||
@@ -194,7 +194,7 @@ class AuthHandler:
|
||||
SESSION_EXPIRE_MS = 48 * 60 * 60 * 1000
|
||||
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
self.auth = hs.get_auth()
|
||||
self.clock = hs.get_clock()
|
||||
self.checkers: Dict[str, UserInteractiveAuthChecker] = {}
|
||||
@@ -1183,7 +1183,7 @@ class AuthHandler:
|
||||
|
||||
# No password providers were able to handle this 3pid
|
||||
# Check local store
|
||||
user_id = await self.hs.get_datastore().get_user_id_by_threepid(
|
||||
user_id = await self.hs.get_datastores().main.get_user_id_by_threepid(
|
||||
medium, address
|
||||
)
|
||||
if not user_id:
|
||||
|
||||
@@ -61,7 +61,7 @@ class CasHandler:
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self.hs = hs
|
||||
self._hostname = hs.hostname
|
||||
self._store = hs.get_datastore()
|
||||
self._store = hs.get_datastores().main
|
||||
self._auth_handler = hs.get_auth_handler()
|
||||
self._registration_handler = hs.get_registration_handler()
|
||||
|
||||
|
||||
@@ -29,7 +29,7 @@ class DeactivateAccountHandler:
|
||||
"""Handler which deals with deactivating user accounts."""
|
||||
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
self.hs = hs
|
||||
self._auth_handler = hs.get_auth_handler()
|
||||
self._device_handler = hs.get_device_handler()
|
||||
@@ -38,6 +38,7 @@ class DeactivateAccountHandler:
|
||||
self._profile_handler = hs.get_profile_handler()
|
||||
self.user_directory_handler = hs.get_user_directory_handler()
|
||||
self._server_name = hs.hostname
|
||||
self._third_party_rules = hs.get_third_party_event_rules()
|
||||
|
||||
# Flag that indicates whether the process to part users from rooms is running
|
||||
self._user_parter_running = False
|
||||
@@ -135,9 +136,13 @@ class DeactivateAccountHandler:
|
||||
if erase_data:
|
||||
user = UserID.from_string(user_id)
|
||||
# Remove avatar URL from this user
|
||||
await self._profile_handler.set_avatar_url(user, requester, "", by_admin)
|
||||
await self._profile_handler.set_avatar_url(
|
||||
user, requester, "", by_admin, deactivation=True
|
||||
)
|
||||
# Remove displayname from this user
|
||||
await self._profile_handler.set_displayname(user, requester, "", by_admin)
|
||||
await self._profile_handler.set_displayname(
|
||||
user, requester, "", by_admin, deactivation=True
|
||||
)
|
||||
|
||||
logger.info("Marking %s as erased", user_id)
|
||||
await self.store.mark_user_erased(user_id)
|
||||
@@ -160,6 +165,13 @@ class DeactivateAccountHandler:
|
||||
# Remove account data (including ignored users and push rules).
|
||||
await self.store.purge_account_data_for_user(user_id)
|
||||
|
||||
# Let modules know the user has been deactivated.
|
||||
await self._third_party_rules.on_user_deactivation_status_changed(
|
||||
user_id,
|
||||
True,
|
||||
by_admin,
|
||||
)
|
||||
|
||||
return identity_server_supports_unbinding
|
||||
|
||||
async def _reject_pending_invites_for_user(self, user_id: str) -> None:
|
||||
@@ -264,6 +276,10 @@ class DeactivateAccountHandler:
|
||||
# Mark the user as active.
|
||||
await self.store.set_user_deactivated_status(user_id, False)
|
||||
|
||||
await self._third_party_rules.on_user_deactivation_status_changed(
|
||||
user_id, False, True
|
||||
)
|
||||
|
||||
# Add the user to the directory, if necessary. Note that
|
||||
# this must be done after the user is re-activated, because
|
||||
# deactivated users are excluded from the user directory.
|
||||
|
||||
@@ -63,7 +63,7 @@ class DeviceWorkerHandler:
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self.clock = hs.get_clock()
|
||||
self.hs = hs
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
self.notifier = hs.get_notifier()
|
||||
self.state = hs.get_state_handler()
|
||||
self.state_store = hs.get_storage().state
|
||||
@@ -628,7 +628,7 @@ class DeviceListUpdater:
|
||||
"Handles incoming device list updates from federation and updates the DB"
|
||||
|
||||
def __init__(self, hs: "HomeServer", device_handler: DeviceHandler):
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
self.federation = hs.get_federation_client()
|
||||
self.clock = hs.get_clock()
|
||||
self.device_handler = device_handler
|
||||
|
||||
@@ -43,7 +43,7 @@ class DeviceMessageHandler:
|
||||
Args:
|
||||
hs: server
|
||||
"""
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
self.notifier = hs.get_notifier()
|
||||
self.is_mine = hs.is_mine
|
||||
|
||||
|
||||
@@ -44,7 +44,7 @@ class DirectoryHandler:
|
||||
self.state = hs.get_state_handler()
|
||||
self.appservice_handler = hs.get_application_service_handler()
|
||||
self.event_creation_handler = hs.get_event_creation_handler()
|
||||
self.store = hs.get_datastore()
|
||||
self.store = hs.get_datastores().main
|
||||
self.config = hs.config
|
||||
self.enable_room_list_search = hs.config.roomdirectory.enable_room_list_search
|
||||
self.require_membership = hs.config.server.require_membership_for_aliases
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user