kaslo/synapse
1
0
Fork 0
Code Wiki Activity

Merge branch 'develop' into madlittlemods/msc2716-resolve-state-for-all-historical-events

Browse Source 
Conflicts:
	synapse/handlers/message.py
This commit is contained in:
Eric Eastwood
2021-10-01 01:36:51 -05:00
parent b20fd16c56 9e5a429c8b
commit 0362887497
156 changed files with 1219 additions and 905 deletions
Download Patch File Download Diff File Expand all files Collapse all files

88
CHANGES.md
View File

@@ -1,3 +1,91 @@
Synapse 1.44.0rc2 (2021-09-30)
==============================
Bugfixes
--------
- Fix a bug introduced in v1.44.0rc1 which caused the experimental [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) `/batch_send` endpoint to return a 500 error. ([\#10938](https://github.com/matrix-org/synapse/issues/10938))
- Fix a bug introduced in v1.44.0rc1 which prevented sending presence events to application services. ([\#10944](https://github.com/matrix-org/synapse/issues/10944))
Improved Documentation
----------------------
- Minor updates to the installation instructions. ([\#10919](https://github.com/matrix-org/synapse/issues/10919))
Synapse 1.44.0rc1 (2021-09-29)
==============================
Features
--------
- Only allow the [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) `/batch_send?chunk_id=xxx` endpoint to connect to an already existing insertion event. ([\#10776](https://github.com/matrix-org/synapse/issues/10776))
- Improve oEmbed URL previews by processing the author name, photo, and video information. ([\#10814](https://github.com/matrix-org/synapse/issues/10814), [\#10819](https://github.com/matrix-org/synapse/issues/10819))
- Speed up responding with large JSON objects to requests. ([\#10868](https://github.com/matrix-org/synapse/issues/10868), [\#10905](https://github.com/matrix-org/synapse/issues/10905))
- Add a `user_may_create_room_with_invites` spam checker callback to allow modules to allow or deny a room creation request based on the invites and/or 3PID invites it includes. ([\#10898](https://github.com/matrix-org/synapse/issues/10898))
Bugfixes
--------
- Fix a long-standing bug that caused an `AssertionError` when purging history in certain rooms. Contributed by @Kokokokoka. ([\#10690](https://github.com/matrix-org/synapse/issues/10690))
- Fix a long-standing bug which caused deactivated users that were later reactivated to be missing from the user directory. ([\#10782](https://github.com/matrix-org/synapse/issues/10782))
- Fix a long-standing bug that caused unbanning a user by sending a membership event to fail. Contributed by @aaronraimist. ([\#10807](https://github.com/matrix-org/synapse/issues/10807))
- Fix a long-standing bug where logging contexts would go missing when federation requests time out. ([\#10810](https://github.com/matrix-org/synapse/issues/10810))
- Fix a long-standing bug causing an error in the deprecated `/initialSync` endpoint when using the undocumented `from` and `to` parameters. ([\#10827](https://github.com/matrix-org/synapse/issues/10827))
- Fix a bug causing the `remove_stale_pushers` background job to repeatedly fail and log errors. This bug affected Synapse servers that had been upgraded from version 1.28 or older and are using SQLite. ([\#10843](https://github.com/matrix-org/synapse/issues/10843))
- Fix a long-standing bug in Unicode support of the room search admin API breaking search for rooms with non-ASCII characters. ([\#10859](https://github.com/matrix-org/synapse/issues/10859))
- Fix a bug introduced in Synapse 1.37.0 which caused `knock` membership events which we sent to remote servers to be incorrectly stored in the local database. ([\#10873](https://github.com/matrix-org/synapse/issues/10873))
- Fix invalidating one-time key count cache after claiming keys. The bug was introduced in Synapse v1.41.0. Contributed by Tulir at Beeper. ([\#10875](https://github.com/matrix-org/synapse/issues/10875))
- Fix a long-standing bug causing application service users to be subject to MAU blocking if the MAU limit had been reached, even if configured not to be blocked. ([\#10881](https://github.com/matrix-org/synapse/issues/10881))
- Fix a long-standing bug which could cause events pulled over federation to be incorrectly rejected. ([\#10907](https://github.com/matrix-org/synapse/issues/10907))
- Fix a long-standing bug causing URL cache files to be stored in storage providers. Server admins may safely delete the `url_cache/` and `url_cache_thumbnails/` directories from any configured storage providers to reclaim space. ([\#10911](https://github.com/matrix-org/synapse/issues/10911))
- Fix a long-standing bug leading to race conditions when creating media store and config directories. ([\#10913](https://github.com/matrix-org/synapse/issues/10913))
Improved Documentation
----------------------
- Fix some crashes in the Module API example code, by adding JSON encoding/decoding. ([\#10845](https://github.com/matrix-org/synapse/issues/10845))
- Add developer documentation about experimental configuration flags. ([\#10865](https://github.com/matrix-org/synapse/issues/10865))
- Properly remove deleted files from GitHub pages when generating the documentation. ([\#10869](https://github.com/matrix-org/synapse/issues/10869))
Internal Changes
----------------
- Fix GitHub Actions config so we can run sytest on synapse from parallel branches. ([\#10659](https://github.com/matrix-org/synapse/issues/10659))
- Split out [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) meta events to their own fields in the `/batch_send` response. ([\#10777](https://github.com/matrix-org/synapse/issues/10777))
- Add missing type hints to REST servlets. ([\#10785](https://github.com/matrix-org/synapse/issues/10785), [\#10817](https://github.com/matrix-org/synapse/issues/10817))
- Simplify the internal logic which maintains the user directory database tables. ([\#10796](https://github.com/matrix-org/synapse/issues/10796))
- Use direct references to config flags. ([\#10812](https://github.com/matrix-org/synapse/issues/10812), [\#10885](https://github.com/matrix-org/synapse/issues/10885), [\#10893](https://github.com/matrix-org/synapse/issues/10893), [\#10897](https://github.com/matrix-org/synapse/issues/10897))
- Specify the type of token in generic "Invalid token" error messages. ([\#10815](https://github.com/matrix-org/synapse/issues/10815))
- Make `StateFilter` frozen so it is hashable. ([\#10816](https://github.com/matrix-org/synapse/issues/10816))
- Fix a long-standing bug where an `m.room.message` event containing a null byte would cause an internal server error. ([\#10820](https://github.com/matrix-org/synapse/issues/10820))
- Add type hints to the state database. ([\#10823](https://github.com/matrix-org/synapse/issues/10823))
- Opt out of cache expiry for `get_users_who_share_room_with_user`, to hopefully improve `/sync` performance when you
haven't synced recently. ([\#10826](https://github.com/matrix-org/synapse/issues/10826))
- Track cache eviction rates more finely in Prometheus's monitoring. ([\#10829](https://github.com/matrix-org/synapse/issues/10829))
- Add missing type hints to `synapse.handlers`. ([\#10831](https://github.com/matrix-org/synapse/issues/10831), [\#10856](https://github.com/matrix-org/synapse/issues/10856))
- Extend the Module API to let plug-ins check whether an ID is local and to access IP + User Agent data. ([\#10833](https://github.com/matrix-org/synapse/issues/10833))
- Factor out PNG image data to a constant to be used in several tests. ([\#10834](https://github.com/matrix-org/synapse/issues/10834))
- Add a test to ensure state events sent by modules get persisted correctly. ([\#10835](https://github.com/matrix-org/synapse/issues/10835))
- Rename [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) fields and event types from `chunk` to `batch` to match the `/batch_send` endpoint. ([\#10838](https://github.com/matrix-org/synapse/issues/10838))
- Rename [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) `/batch_send` query parameter from `?prev_event` to more obvious usage with `?prev_event_id`. ([\#10839](https://github.com/matrix-org/synapse/issues/10839))
- Add type hints to `synapse.http.site`. ([\#10867](https://github.com/matrix-org/synapse/issues/10867))
- Include outlier status when we log V2 or V3 events. ([\#10879](https://github.com/matrix-org/synapse/issues/10879))
- Break down Grafana's cache expiry time series based on reason for eviction, c.f. [\#10829](https://github.com/matrix-org/synapse/issues/10829). ([\#10880](https://github.com/matrix-org/synapse/issues/10880))
- Clean up some of the federation event authentication code for clarity. ([\#10883](https://github.com/matrix-org/synapse/issues/10883), [\#10884](https://github.com/matrix-org/synapse/issues/10884), [\#10896](https://github.com/matrix-org/synapse/issues/10896), [\#10901](https://github.com/matrix-org/synapse/issues/10901))
- Allow the `.` and `~` characters when creating registration tokens as per the change to [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231). ([\#10887](https://github.com/matrix-org/synapse/issues/10887))
- Clean up some unnecessary parentheses in places around the codebase. ([\#10889](https://github.com/matrix-org/synapse/issues/10889))
- Improve type hinting in the user directory code. ([\#10891](https://github.com/matrix-org/synapse/issues/10891))
- Update development testing script `test_postgresql.sh` to use a supported Python version and make re-runs quicker. ([\#10906](https://github.com/matrix-org/synapse/issues/10906))
- Document and summarize changes in schema version `61` – `64`. ([\#10917](https://github.com/matrix-org/synapse/issues/10917))
- Update release script to sign the newly created git tags. ([\#10925](https://github.com/matrix-org/synapse/issues/10925))
- Fix Debian builds due to `dh-virtualenv` no longer being able to build their docs. ([\#10931](https://github.com/matrix-org/synapse/issues/10931))
Synapse 1.43.0 (2021-09-21)
===========================

2
README.rst
View File

@@ -288,7 +288,7 @@ Quick start
Before setting up a development environment for synapse, make sure you have the
system dependencies (such as the python header files) installed - see
`Installing from source <https://matrix-org.github.io/synapse/latest/setup/installation.html#installing-from-source>`_.
`Platform-specific prerequisites <https://matrix-org.github.io/synapse/latest/setup/installation.html#platform-specific-prerequisites>`_.
To check out a synapse for development, clone the git repo into a working
directory of your choice::

1
changelog.d/10659.misc
View File

@@ -1 +0,0 @@
Fix GitHub Actions config so we can run sytest on synapse from parallel branches.

1
changelog.d/10690.bugfix
View File

@@ -1 +0,0 @@
Fix a long-standing bug that caused an `AssertionError` when purging history in certain rooms. Contributed by @Kokokokoka.

1
changelog.d/10776.feature
View File

@@ -1 +0,0 @@
Only allow the [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) `/batch_send?chunk_id=xxx` endpoint to connect to an already existing insertion event.

1
changelog.d/10777.misc
View File

@@ -1 +0,0 @@
Split out [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) meta events to their own fields in the `/batch_send` response.

1
changelog.d/10782.bugfix
View File

@@ -1 +0,0 @@
Fix a long-standing bug which caused deactivated users that were later reactivated to be missing from the user directory.

1
changelog.d/10785.misc
View File

@@ -1 +0,0 @@
Add missing type hints to REST servlets.

1
changelog.d/10796.misc
View File

@@ -1 +0,0 @@
Simplify the internal logic which maintains the user directory database tables.

1
changelog.d/10807.bugfix
View File

@@ -1 +0,0 @@
Allow sending a membership event to unban a user. Contributed by @aaronraimist.

1
changelog.d/10810.bugfix
View File

@@ -1 +0,0 @@
Fix a case where logging contexts would go missing when federation requests time out.

1
changelog.d/10814.feature
View File

@@ -1 +0,0 @@
Improve oEmbed previews by processing the author name, photo, and video information.

1
changelog.d/10815.misc
View File

@@ -1 +0,0 @@
Specify the type of token in generic "Invalid token" error messages.

1
changelog.d/10816.misc
View File

@@ -1 +0,0 @@
Make `StateFilter` frozen so it is hashable.

1
changelog.d/10817.misc
View File

@@ -1 +0,0 @@
Add missing type hints to REST servlets.

1
changelog.d/10819.feature
View File

@@ -1 +0,0 @@
Improve oEmbed previews by processing the author name, photo, and video information.

1
changelog.d/10820.misc
View File

@@ -1 +0,0 @@
Fix a long-standing bug where an `m.room.message` event containing a null byte would cause an internal server error.

1
changelog.d/10823.misc
View File

@@ -1 +0,0 @@
Add type hints to the state database.

2
changelog.d/10826.misc
View File

@@ -1,2 +0,0 @@
Opt out of cache expiry for `get_users_who_share_room_with_user`, to hopefully improve `/sync` performance when you
haven't synced recently.

1
changelog.d/10827.bugfix
View File

@@ -1 +0,0 @@
Fix error in deprecated `/initialSync` endpoint when using the undocumented `from` and `to` parameters.

1
changelog.d/10829.misc
View File

@@ -1 +0,0 @@
Track cache eviction rates more finely in Prometheus' monitoring.

1
changelog.d/10831.misc
View File

@@ -1 +0,0 @@
Add missing type hints to handlers.

1
changelog.d/10833.misc
View File

@@ -1 +0,0 @@
Extend the ModuleApi to let plug-ins check whether an ID is local and to access IP + User Agent data.

1
changelog.d/10834.misc
View File

@@ -1 +0,0 @@
Factor out PNG image data to a constant to be used in several tests.

1
changelog.d/10835.misc
View File

@@ -1 +0,0 @@
Add a test to ensure state events sent by modules get persisted correctly.

1
changelog.d/10838.misc
View File

@@ -1 +0,0 @@
Rename [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) fields and event types from `chunk` to `batch` to match the `/batch_send` endpoint.

1
changelog.d/10839.misc
View File

@@ -1 +0,0 @@
Rename [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) `/batch_send` query parameter from `?prev_event` to more obvious usage with `?prev_event_id`.

1
changelog.d/10843.bugfix
View File

@@ -1 +0,0 @@
Fix a bug causing the `remove_stale_pushers` background job to repeatedly fail and log errors. This bug affected Synapse servers that had been upgraded from version 1.28 or older and are using SQLite.

1
changelog.d/10845.doc
View File

@@ -1 +0,0 @@
Fix some crashes in the Module API example code, by adding JSON encoding/decoding.

1
changelog.d/10856.misc
View File

@@ -1 +0,0 @@
Add missing type hints to handlers.

1
changelog.d/10859.bugfix
View File

@@ -1 +0,0 @@
Fix a bug in Unicode support of the room search admin API. It is now possible to search for rooms with non-ASCII characters.

1
changelog.d/10865.doc
View File

@@ -1 +0,0 @@
Add developer documentation about experimental configuration flags.

1
changelog.d/10867.misc
View File

@@ -1 +0,0 @@
Add type hints to `synapse.http.site`.

1
changelog.d/10868.feature
View File

@@ -1 +0,0 @@
Speed up responding with large JSON objects to requests.

1
changelog.d/10869.doc
View File

@@ -1 +0,0 @@
Properly remove deleted files from GitHub pages when generating the documentation.

1
changelog.d/10873.bugfix
View File

@@ -1 +0,0 @@
Fix a bug introduced in Synapse 1.37.0 which caused `knock` events which we sent to remote servers to be incorrectly stored in the local database.

1
changelog.d/10875.bugfix
View File

@@ -1 +0,0 @@
Fix invalidating one-time key count cache after claiming keys. Contributed by Tulir at Beeper.

1
changelog.d/10879.misc
View File

@@ -1 +0,0 @@
Include outlier status when we log V2 or V3 events.

1
changelog.d/10880.misc
View File

@@ -1 +0,0 @@
Break down Grafana's cache expiry time series based on reason for eviction---see #10829.

1
changelog.d/10881.bugfix
View File

@@ -1 +0,0 @@
Fix application service users being subject to MAU blocking if MAU had been reached, even if configured not to be blocked.

1
changelog.d/10884.misc
View File

@@ -1 +0,0 @@
Clean up some of the federation event authentication code for clarity.

1
changelog.d/10885.misc
View File

@@ -1 +0,0 @@
Use direct references to config flags.

1
changelog.d/10887.bugfix
View File

@@ -1 +0,0 @@
Allow the `.` and `~` characters when creating registration tokens as per the change to [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231).

1
changelog.d/10889.misc
View File

@@ -1 +0,0 @@
Clean up some unnecessary parentheses in places around the codebase.

1
changelog.d/10891.misc
View File

@@ -1 +0,0 @@
Improve type hinting in the user directory code.

1
changelog.d/10893.misc
View File

@@ -1 +0,0 @@
Use direct references to config flags.

1
changelog.d/10895.misc Normal file
View File

@@ -0,0 +1 @@
Fix type hints to be compatible with an upcoming change to Twisted.

1
changelog.d/10896.misc
View File

@@ -1 +0,0 @@
Clean up some of the federation event authentication code for clarity.

1
changelog.d/10897.misc
View File

@@ -1 +0,0 @@
Use direct references to config flags.

1
changelog.d/10898.feature
View File

@@ -1 +0,0 @@
Add a `user_may_create_room_with_invites` spam checker callback to allow modules to allow or deny a room creation request based on the invites and/or 3PID invites it includes.

1
changelog.d/10901.misc
View File

@@ -1 +0,0 @@
Clean up some of the federation event authentication code for clarity.

1
changelog.d/10905.feature
View File

@@ -1 +0,0 @@
Speed up responding with large JSON objects to requests.

1
changelog.d/10906.misc
View File

@@ -1 +0,0 @@
Update development testing script `test_postgresql.sh` to use a supported Python version and make re-runs quicker.

1
changelog.d/10907.bugfix
View File

@@ -1 +0,0 @@
Fix a long-standing bug which could cause events pulled over federation to be incorrectly rejected.

1
changelog.d/10911.bugfix
View File

@@ -1 +0,0 @@
Avoid storing URL cache files in storage providers. Server admins may safely delete the `url_cache/` and `url_cache_thumbnails/` directories from any configured storage providers to reclaim space.

1
changelog.d/10913.bugfix
View File

@@ -1 +0,0 @@
Fix race conditions when creating media store and config directories.

0
changelog.d/10812.misc → changelog.d/10916.misc
View File

1
changelog.d/10917.misc
View File

@@ -1 +0,0 @@
Document and summarize changes in schema version `61` - `64`.

1
changelog.d/10922.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix a minor bug in the response to `/_matrix/client/r0/voip/turnServer`. Contributed by @lukaslihotzki.

1
changelog.d/10924.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix a bug where empty `yyyy-mm-dd/` directories would be left behind in the media store's `url_cache_thumbnails/` directory.

1
changelog.d/10925.misc
View File

@@ -1 +0,0 @@
Update release script to sign the newly created git tags.

1
changelog.d/10931.bugfix
View File

@@ -1 +0,0 @@
Fix debian builds due to dh-virtualenv no longer being able to build their docs.

1
changelog.d/10933.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix a bug introduced in Synapse v1.40.0 where changing a user's display name or avatar in a restricted room would cause an authentication error.

1
changelog.d/10934.misc Normal file
View File

@@ -0,0 +1 @@
Refactor various parts of the codebase to use `RoomVersion` objects instead of room version identifier strings.

1
changelog.d/10935.misc Normal file
View File

@@ -0,0 +1 @@
Refactor user directory tests in preparation for upcoming changes.

1
changelog.d/10936.misc Normal file
View File

@@ -0,0 +1 @@
Include the event id in the logcontext when handling PDUs received over federation.

1
changelog.d/10939.misc Normal file
View File

@@ -0,0 +1 @@
Fix logged errors in unit tests.

0
changelog.d/10883.misc → changelog.d/10940.misc
View File

1
changelog.d/10945.misc Normal file
View File

@@ -0,0 +1 @@
Fix a broken test to ensure that consent configuration works during registration.

1
changelog.d/9655.feature Normal file
View File

@@ -0,0 +1 @@
Add [MSC3069](https://github.com/matrix-org/matrix-doc/pull/3069) support to `/account/whoami`.

12
debian/changelog vendored
View File

@@ -1,3 +1,15 @@
matrix-synapse-py3 (1.44.0~rc2) stable; urgency=medium
* New synapse release 1.44.0~rc2.
-- Synapse Packaging team <packages@matrix.org> Thu, 30 Sep 2021 12:39:10 +0100
matrix-synapse-py3 (1.44.0~rc1) stable; urgency=medium
* New synapse release 1.44.0~rc1.
-- Synapse Packaging team <packages@matrix.org> Tue, 28 Sep 2021 13:41:28 +0100
matrix-synapse-py3 (1.43.0) stable; urgency=medium
* New synapse release 1.43.0.

328
docs/setup/installation.md
View File

@@ -18,19 +18,179 @@ that your email address is probably `user@example.com` rather than
## Installing Synapse
### Installing from source
### Prebuilt packages
(Prebuilt packages are available for some platforms - see [Prebuilt packages](#prebuilt-packages).)
Prebuilt packages are available for a number of platforms. These are recommended
for most users.
When installing from source please make sure that the [Platform-specific prerequisites](#platform-specific-prerequisites) are already installed.
#### Docker images and Ansible playbooks
There is an official synapse image available at
<https://hub.docker.com/r/matrixdotorg/synapse> which can be used with
the docker-compose file available at
[contrib/docker](https://github.com/matrix-org/synapse/tree/develop/contrib/docker).
Further information on this including configuration options is available in the README
on hub.docker.com.
Alternatively, Andreas Peters (previously Silvio Fricke) has contributed a
Dockerfile to automate a synapse server in a single Docker image, at
<https://hub.docker.com/r/avhost/docker-matrix/tags/>
Slavi Pantaleev has created an Ansible playbook,
which installs the offical Docker image of Matrix Synapse
along with many other Matrix-related services (Postgres database, Element, coturn,
ma1sd, SSL support, etc.).
For more details, see
<https://github.com/spantaleev/matrix-docker-ansible-deploy>
#### Debian/Ubuntu
##### Matrix.org packages
Matrix.org provides Debian/Ubuntu packages of Synapse, for the amd64
architecture via <https://packages.matrix.org/debian/>.
To install the latest release:
```sh
sudo apt install -y lsb-release wget apt-transport-https
sudo wget -O /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/ $(lsb_release -cs) main" |
sudo tee /etc/apt/sources.list.d/matrix-org.list
sudo apt update
sudo apt install matrix-synapse-py3
```
Packages are also published for release candidates. To enable the prerelease
channel, add `prerelease` to the `sources.list` line. For example:
```sh
sudo wget -O /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/ $(lsb_release -cs) main prerelease" |
sudo tee /etc/apt/sources.list.d/matrix-org.list
sudo apt update
sudo apt install matrix-synapse-py3
```
The fingerprint of the repository signing key (as shown by `gpg
/usr/share/keyrings/matrix-org-archive-keyring.gpg`) is
`AAF9AE843A7584B5A3E4CD2BCF45A512DE2DA058`.
##### Downstream Debian packages
We do not recommend using the packages from the default Debian `buster`
repository at this time, as they are old and suffer from known security
vulnerabilities. You can install the latest version of Synapse from
[our repository](#matrixorg-packages) or from `buster-backports`. Please
see the [Debian documentation](https://backports.debian.org/Instructions/)
for information on how to use backports.
If you are using Debian `sid` or testing, Synapse is available in the default
repositories and it should be possible to install it simply with:
```sh
sudo apt install matrix-synapse
```
##### Downstream Ubuntu packages
We do not recommend using the packages in the default Ubuntu repository
at this time, as they are old and suffer from known security vulnerabilities.
The latest version of Synapse can be installed from [our repository](#matrixorg-packages).
#### Fedora
Synapse is in the Fedora repositories as `matrix-synapse`:
```sh
sudo dnf install matrix-synapse
```
Oleg Girko provides Fedora RPMs at
<https://obs.infoserver.lv/project/monitor/matrix-synapse>
#### OpenSUSE
Synapse is in the OpenSUSE repositories as `matrix-synapse`:
```sh
sudo zypper install matrix-synapse
```
#### SUSE Linux Enterprise Server
Unofficial package are built for SLES 15 in the openSUSE:Backports:SLE-15 repository at
<https://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-15/standard/>
#### ArchLinux
The quickest way to get up and running with ArchLinux is probably with the community package
<https://www.archlinux.org/packages/community/any/matrix-synapse/>, which should pull in most of
the necessary dependencies.
pip may be outdated (6.0.7-1 and needs to be upgraded to 6.0.8-1 ):
```sh
sudo pip install --upgrade pip
```
If you encounter an error with lib bcrypt causing an Wrong ELF Class:
ELFCLASS32 (x64 Systems), you may need to reinstall py-bcrypt to correctly
compile it under the right architecture. (This should not be needed if
installing under virtualenv):
```sh
sudo pip uninstall py-bcrypt
sudo pip install py-bcrypt
```
#### Void Linux
Synapse can be found in the void repositories as 'synapse':
```sh
xbps-install -Su
xbps-install -S synapse
```
#### FreeBSD
Synapse can be installed via FreeBSD Ports or Packages contributed by Brendan Molloy from:
- Ports: `cd /usr/ports/net-im/py-matrix-synapse && make install clean`
- Packages: `pkg install py37-matrix-synapse`
#### OpenBSD
As of OpenBSD 6.7 Synapse is available as a pre-compiled binary. The filesystem
underlying the homeserver directory (defaults to `/var/synapse`) has to be
mounted with `wxallowed` (cf. `mount(8)`), so creating a separate filesystem
and mounting it to `/var/synapse` should be taken into consideration.
Installing Synapse:
```sh
doas pkg_add synapse
```
#### NixOS
Robin Lambertz has packaged Synapse for NixOS at:
<https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/misc/matrix-synapse.nix>
### Installing as a Python module from PyPI
It's also possible to install Synapse as a Python module from PyPI.
When following this route please make sure that the [Platform-specific prerequisites](#platform-specific-prerequisites) are already installed.
System requirements:
- POSIX-compliant system (tested on Linux & OS X)
- Python 3.5.2 or later, up to Python 3.9.
- Python 3.6 or later, up to Python 3.9.
- At least 1GB of free RAM if you want to join large public rooms like #matrix:matrix.org
To install the Synapse homeserver run:
```sh
@@ -203,164 +363,6 @@ be found at <https://docs.microsoft.com/en-us/windows/wsl/install-win10> for
Windows 10 and <https://docs.microsoft.com/en-us/windows/wsl/install-on-server>
for Windows Server.
### Prebuilt packages
As an alternative to installing from source, prebuilt packages are available
for a number of platforms.
#### Docker images and Ansible playbooks
There is an official synapse image available at
<https://hub.docker.com/r/matrixdotorg/synapse> which can be used with
the docker-compose file available at
[contrib/docker](https://github.com/matrix-org/synapse/tree/develop/contrib/docker).
Further information on this including configuration options is available in the README
on hub.docker.com.
Alternatively, Andreas Peters (previously Silvio Fricke) has contributed a
Dockerfile to automate a synapse server in a single Docker image, at
<https://hub.docker.com/r/avhost/docker-matrix/tags/>
Slavi Pantaleev has created an Ansible playbook,
which installs the offical Docker image of Matrix Synapse
along with many other Matrix-related services (Postgres database, Element, coturn,
ma1sd, SSL support, etc.).
For more details, see
<https://github.com/spantaleev/matrix-docker-ansible-deploy>
#### Debian/Ubuntu
##### Matrix.org packages
Matrix.org provides Debian/Ubuntu packages of Synapse via
<https://packages.matrix.org/debian/>. To install the latest release:
```sh
sudo apt install -y lsb-release wget apt-transport-https
sudo wget -O /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/ $(lsb_release -cs) main" |
sudo tee /etc/apt/sources.list.d/matrix-org.list
sudo apt update
sudo apt install matrix-synapse-py3
```
Packages are also published for release candidates. To enable the prerelease
channel, add `prerelease` to the `sources.list` line. For example:
```sh
sudo wget -O /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/ $(lsb_release -cs) main prerelease" |
sudo tee /etc/apt/sources.list.d/matrix-org.list
sudo apt update
sudo apt install matrix-synapse-py3
```
The fingerprint of the repository signing key (as shown by `gpg
/usr/share/keyrings/matrix-org-archive-keyring.gpg`) is
`AAF9AE843A7584B5A3E4CD2BCF45A512DE2DA058`.
##### Downstream Debian packages
We do not recommend using the packages from the default Debian `buster`
repository at this time, as they are old and suffer from known security
vulnerabilities. You can install the latest version of Synapse from
[our repository](#matrixorg-packages) or from `buster-backports`. Please
see the [Debian documentation](https://backports.debian.org/Instructions/)
for information on how to use backports.
If you are using Debian `sid` or testing, Synapse is available in the default
repositories and it should be possible to install it simply with:
```sh
sudo apt install matrix-synapse
```
##### Downstream Ubuntu packages
We do not recommend using the packages in the default Ubuntu repository
at this time, as they are old and suffer from known security vulnerabilities.
The latest version of Synapse can be installed from [our repository](#matrixorg-packages).
#### Fedora
Synapse is in the Fedora repositories as `matrix-synapse`:
```sh
sudo dnf install matrix-synapse
```
Oleg Girko provides Fedora RPMs at
<https://obs.infoserver.lv/project/monitor/matrix-synapse>
#### OpenSUSE
Synapse is in the OpenSUSE repositories as `matrix-synapse`:
```sh
sudo zypper install matrix-synapse
```
#### SUSE Linux Enterprise Server
Unofficial package are built for SLES 15 in the openSUSE:Backports:SLE-15 repository at
<https://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-15/standard/>
#### ArchLinux
The quickest way to get up and running with ArchLinux is probably with the community package
<https://www.archlinux.org/packages/community/any/matrix-synapse/>, which should pull in most of
the necessary dependencies.
pip may be outdated (6.0.7-1 and needs to be upgraded to 6.0.8-1 ):
```sh
sudo pip install --upgrade pip
```
If you encounter an error with lib bcrypt causing an Wrong ELF Class:
ELFCLASS32 (x64 Systems), you may need to reinstall py-bcrypt to correctly
compile it under the right architecture. (This should not be needed if
installing under virtualenv):
```sh
sudo pip uninstall py-bcrypt
sudo pip install py-bcrypt
```
#### Void Linux
Synapse can be found in the void repositories as 'synapse':
```sh
xbps-install -Su
xbps-install -S synapse
```
#### FreeBSD
Synapse can be installed via FreeBSD Ports or Packages contributed by Brendan Molloy from:
- Ports: `cd /usr/ports/net-im/py-matrix-synapse && make install clean`
- Packages: `pkg install py37-matrix-synapse`
#### OpenBSD
As of OpenBSD 6.7 Synapse is available as a pre-compiled binary. The filesystem
underlying the homeserver directory (defaults to `/var/synapse`) has to be
mounted with `wxallowed` (cf. `mount(8)`), so creating a separate filesystem
and mounting it to `/var/synapse` should be taken into consideration.
Installing Synapse:
```sh
doas pkg_add synapse
```
#### NixOS
Robin Lambertz has packaged Synapse for NixOS at:
<https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/misc/matrix-synapse.nix>
## Setting up Synapse
Once you have installed synapse as above, you will need to configure it.

6
mypy.ini
View File

@@ -162,6 +162,12 @@ disallow_untyped_defs = True
[mypy-synapse.util.wheel_timer]
disallow_untyped_defs = True
[mypy-tests.handlers.test_user_directory]
disallow_untyped_defs = True
[mypy-tests.storage.test_user_directory]
disallow_untyped_defs = True
[mypy-pymacaroons.*]
ignore_missing_imports = True

2
synapse/__init__.py
View File

@@ -47,7 +47,7 @@ try:
except ImportError:
pass
__version__ = "1.43.0"
__version__ = "1.44.0rc2"
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
# We import here so that we don't have to install a bunch of deps when

3
synapse/api/constants.py
View File

@@ -217,6 +217,9 @@ class EventContentFields:
# For "marker" events
MSC2716_MARKER_INSERTION = "org.matrix.msc2716.marker.insertion"
# The authorising user for joining a restricted room.
AUTHORISING_USER = "join_authorised_via_users_server"
class RoomTypes:
"""Understood values of the room_type field of m.room.create events."""

8
synapse/app/_base.py
View File

@@ -86,11 +86,11 @@ def start_worker_reactor(appname, config, run_command=reactor.run):
start_reactor(
appname,
soft_file_limit=config.soft_file_limit,
gc_thresholds=config.gc_thresholds,
soft_file_limit=config.server.soft_file_limit,
gc_thresholds=config.server.gc_thresholds,
pid_file=config.worker.worker_pid_file,
daemonize=config.worker.worker_daemonize,
print_pidfile=config.print_pidfile,
print_pidfile=config.server.print_pidfile,
logger=logger,
run_command=run_command,
)
@@ -298,7 +298,7 @@ def refresh_certificate(hs):
Refresh the TLS certificates that Synapse is using by re-reading them from
disk and updating the TLS context factories to use them.
"""
if not hs.config.has_tls_listener():
if not hs.config.server.has_tls_listener():
return
hs.config.read_certificate_from_disk()

4
synapse/app/admin_cmd.py
View File

@@ -195,14 +195,14 @@ def start(config_options):
config.logging.no_redirect_stdio = True
# Explicitly disable background processes
config.update_user_directory = False
config.server.update_user_directory = False
config.worker.run_background_tasks = False
config.start_pushers = False
config.pusher_shard_config.instances = []
config.send_federation = False
config.federation_shard_config.instances = []
synapse.events.USE_FROZEN_DICTS = config.use_frozen_dicts
synapse.events.USE_FROZEN_DICTS = config.server.use_frozen_dicts
ss = AdminCmdServer(
config.server.server_name,

2
synapse/app/generic_worker.py
View File

@@ -462,7 +462,7 @@ def start(config_options):
# For other worker types we force this to off.
config.server.update_user_directory = False
synapse.events.USE_FROZEN_DICTS = config.use_frozen_dicts
synapse.events.USE_FROZEN_DICTS = config.server.use_frozen_dicts
synapse.util.caches.TRACK_MEMORY_USAGE = config.caches.track_memory_usage
if config.server.gc_seconds:

14
synapse/app/homeserver.py
View File

@@ -248,7 +248,7 @@ class SynapseHomeServer(HomeServer):
resources[SERVER_KEY_V2_PREFIX] = KeyApiV2Resource(self)
if name == "webclient":
webclient_loc = self.config.web_client_location
webclient_loc = self.config.server.web_client_location
if webclient_loc is None:
logger.warning(
@@ -343,7 +343,7 @@ def setup(config_options):
# generating config files and shouldn't try to continue.
sys.exit(0)
events.USE_FROZEN_DICTS = config.use_frozen_dicts
events.USE_FROZEN_DICTS = config.server.use_frozen_dicts
synapse.util.caches.TRACK_MEMORY_USAGE = config.caches.track_memory_usage
if config.server.gc_seconds:
@@ -439,11 +439,11 @@ def run(hs):
_base.start_reactor(
"synapse-homeserver",
soft_file_limit=hs.config.soft_file_limit,
gc_thresholds=hs.config.gc_thresholds,
pid_file=hs.config.pid_file,
daemonize=hs.config.daemonize,
print_pidfile=hs.config.print_pidfile,
soft_file_limit=hs.config.server.soft_file_limit,
gc_thresholds=hs.config.server.gc_thresholds,
pid_file=hs.config.server.pid_file,
daemonize=hs.config.server.daemonize,
print_pidfile=hs.config.server.print_pidfile,
logger=logger,
)

8
synapse/app/phone_stats_home.py
View File

@@ -74,7 +74,7 @@ async def phone_stats_home(hs, stats, stats_process=_stats_process):
store = hs.get_datastore()
stats["homeserver"] = hs.config.server.server_name
stats["server_context"] = hs.config.server_context
stats["server_context"] = hs.config.server.server_context
stats["timestamp"] = now
stats["uptime_seconds"] = uptime
version = sys.version_info
@@ -171,7 +171,7 @@ def start_phone_stats_home(hs):
current_mau_count_by_service = {}
reserved_users = ()
store = hs.get_datastore()
if hs.config.limit_usage_by_mau or hs.config.mau_stats_only:
if hs.config.server.limit_usage_by_mau or hs.config.server.mau_stats_only:
current_mau_count = await store.get_monthly_active_count()
current_mau_count_by_service = (
await store.get_monthly_active_count_by_service()
@@ -183,9 +183,9 @@ def start_phone_stats_home(hs):
current_mau_by_service_gauge.labels(app_service).set(float(count))
registered_reserved_users_mau_gauge.set(float(len(reserved_users)))
max_mau_gauge.set(float(hs.config.max_mau_value))
max_mau_gauge.set(float(hs.config.server.max_mau_value))
if hs.config.limit_usage_by_mau or hs.config.mau_stats_only:
if hs.config.server.limit_usage_by_mau or hs.config.server.mau_stats_only:
generate_monthly_active_users()
clock.looping_call(generate_monthly_active_users, 5 * 60 * 1000)
# End of monthly active user settings

2
synapse/config/_base.py
View File

@@ -327,7 +327,7 @@ class RootConfig:
"""
Redirect lookups on this object either to config objects, or values on
config objects, so that `config.tls.blah` works, as well as legacy uses
of things like `config.server_name`. It will first look up the config
of things like `config.server.server_name`. It will first look up the config
section name, and then values on those config classes.
"""
if item in self._configs.keys():

4
synapse/config/server.py
View File

@@ -1,6 +1,4 @@
# Copyright 2014-2016 OpenMarket Ltd
# Copyright 2017-2018 New Vector Ltd
# Copyright 2019 The Matrix.org Foundation C.I.C.
# Copyright 2014-2021 The Matrix.org Foundation C.I.C.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

161
synapse/event_auth.py
View File

@@ -41,42 +41,112 @@ from synapse.types import StateMap, UserID, get_domain_from_id
logger = logging.getLogger(__name__)
def check(
room_version_obj: RoomVersion,
event: EventBase,
auth_events: StateMap[EventBase],
do_sig_check: bool = True,
do_size_check: bool = True,
def validate_event_for_room_version(
room_version_obj: RoomVersion, event: EventBase
) -> None:
"""Checks if this event is correctly authed.
"""Ensure that the event complies with the limits, and has the right signatures
NB: does not *validate* the signatures - it assumes that any signatures present
have already been checked.
NB: it does not check that the event satisfies the auth rules (that is done in
check_auth_rules_for_event) - these tests are independent of the rest of the state
in the room.
NB: This is used to check events that have been received over federation. As such,
it can only enforce the checks specified in the relevant room version, to avoid
a split-brain situation where some servers accept such events, and others reject
them.
TODO: consider moving this into EventValidator
Args:
room_version_obj: the version of the room
event: the event being checked.
auth_events: the existing room state.
do_sig_check: True if it should be verified that the sending server
signed the event.
do_size_check: True if the size of the event fields should be verified.
room_version_obj: the version of the room which contains this event
event: the event to be checked
Raises:
AuthError if the checks fail
Returns:
if the auth checks pass.
SynapseError if there is a problem with the event
"""
assert isinstance(auth_events, dict)
if do_size_check:
_check_size_limits(event)
_check_size_limits(event)
if not hasattr(event, "room_id"):
raise AuthError(500, "Event has no room_id: %s" % event)
room_id = event.room_id
# check that the event has the correct signatures
sender_domain = get_domain_from_id(event.sender)
is_invite_via_3pid = (
event.type == EventTypes.Member
and event.membership == Membership.INVITE
and "third_party_invite" in event.content
)
# Check the sender's domain has signed the event
if not event.signatures.get(sender_domain):
# We allow invites via 3pid to have a sender from a different
# HS, as the sender must match the sender of the original
# 3pid invite. This is checked further down with the
# other dedicated membership checks.
if not is_invite_via_3pid:
raise AuthError(403, "Event not signed by sender's server")
if event.format_version in (EventFormatVersions.V1,):
# Only older room versions have event IDs to check.
event_id_domain = get_domain_from_id(event.event_id)
# Check the origin domain has signed the event
if not event.signatures.get(event_id_domain):
raise AuthError(403, "Event not signed by sending server")
is_invite_via_allow_rule = (
room_version_obj.msc3083_join_rules
and event.type == EventTypes.Member
and event.membership == Membership.JOIN
and EventContentFields.AUTHORISING_USER in event.content
)
if is_invite_via_allow_rule:
authoriser_domain = get_domain_from_id(
event.content[EventContentFields.AUTHORISING_USER]
)
if not event.signatures.get(authoriser_domain):
raise AuthError(403, "Event not signed by authorising server")
def check_auth_rules_for_event(
room_version_obj: RoomVersion, event: EventBase, auth_events: StateMap[EventBase]
) -> None:
"""Check that an event complies with the auth rules
Checks whether an event passes the auth rules with a given set of state events
Assumes that we have already checked that the event is the right shape (it has
enough signatures, has a room ID, etc). In other words:
- it's fine for use in state resolution, when we have already decided whether to
accept the event or not, and are now trying to decide whether it should make it
into the room state
- when we're doing the initial event auth, it is only suitable in combination with
a bunch of other tests.
Args:
room_version_obj: the version of the room
event: the event being checked.
auth_events: the room state to check the events against.
Raises:
AuthError if the checks fail
"""
assert isinstance(auth_events, dict)
# We need to ensure that the auth events are actually for the same room, to
# stop people from using powers they've been granted in other rooms for
# example.
#
# Arguably we don't need to do this when we're just doing state res, as presumably
# the state res algorithm isn't silly enough to give us events from different rooms.
# Still, it's easier to do it anyway.
room_id = event.room_id
for auth_event in auth_events.values():
if auth_event.room_id != room_id:
raise AuthError(
@@ -86,45 +156,6 @@ def check(
% (event.event_id, room_id, auth_event.event_id, auth_event.room_id),
)
if do_sig_check:
sender_domain = get_domain_from_id(event.sender)
is_invite_via_3pid = (
event.type == EventTypes.Member
and event.membership == Membership.INVITE
and "third_party_invite" in event.content
)
# Check the sender's domain has signed the event
if not event.signatures.get(sender_domain):
# We allow invites via 3pid to have a sender from a different
# HS, as the sender must match the sender of the original
# 3pid invite. This is checked further down with the
# other dedicated membership checks.
if not is_invite_via_3pid:
raise AuthError(403, "Event not signed by sender's server")
if event.format_version in (EventFormatVersions.V1,):
# Only older room versions have event IDs to check.
event_id_domain = get_domain_from_id(event.event_id)
# Check the origin domain has signed the event
if not event.signatures.get(event_id_domain):
raise AuthError(403, "Event not signed by sending server")
is_invite_via_allow_rule = (
room_version_obj.msc3083_join_rules
and event.type == EventTypes.Member
and event.membership == Membership.JOIN
and "join_authorised_via_users_server" in event.content
)
if is_invite_via_allow_rule:
authoriser_domain = get_domain_from_id(
event.content["join_authorised_via_users_server"]
)
if not event.signatures.get(authoriser_domain):
raise AuthError(403, "Event not signed by authorising server")
# Implementation of https://matrix.org/docs/spec/rooms/v1#authorization-rules
#
# 1. If type is m.room.create:
@@ -382,7 +413,9 @@ def _is_membership_change_allowed(
# Note that if the caller is in the room or invited, then they do
# not need to meet the allow rules.
if not caller_in_room and not caller_invited:
authorising_user = event.content.get("join_authorised_via_users_server")
authorising_user = event.content.get(
EventContentFields.AUTHORISING_USER
)
if authorising_user is None:
raise AuthError(403, "Join event is missing authorising user.")
@@ -837,10 +870,10 @@ def auth_types_for_event(
auth_types.add(key)
if room_version.msc3083_join_rules and membership == Membership.JOIN:
if "join_authorised_via_users_server" in event.content:
if EventContentFields.AUTHORISING_USER in event.content:
key = (
EventTypes.Member,
event.content["join_authorised_via_users_server"],
event.content[EventContentFields.AUTHORISING_USER],
)
auth_types.add(key)

20
synapse/events/builder.py
View File

@@ -18,10 +18,8 @@ import attr
from nacl.signing import SigningKey
from synapse.api.constants import MAX_DEPTH
from synapse.api.errors import UnsupportedRoomVersionError
from synapse.api.room_versions import (
KNOWN_EVENT_FORMAT_VERSIONS,
KNOWN_ROOM_VERSIONS,
EventFormatVersions,
RoomVersion,
)
@@ -197,24 +195,6 @@ class EventBuilderFactory:
self.state = hs.get_state_handler()
self._event_auth_handler = hs.get_event_auth_handler()
def new(self, room_version: str, key_values: dict) -> EventBuilder:
"""Generate an event builder appropriate for the given room version
Deprecated: use for_room_version with a RoomVersion object instead
Args:
room_version: Version of the room that we're creating an event builder for
key_values: Fields used as the basis of the new event
Returns:
EventBuilder
"""
v = KNOWN_ROOM_VERSIONS.get(room_version)
if not v:
# this can happen if support is withdrawn for a room version
raise UnsupportedRoomVersionError()
return self.for_room_version(v, key_values)
def for_room_version(
self, room_version: RoomVersion, key_values: dict
) -> EventBuilder:

6
synapse/events/presence_router.py
View File

@@ -45,11 +45,11 @@ def load_legacy_presence_router(hs: "HomeServer"):
configuration, and registers the hooks they implement.
"""
if hs.config.presence_router_module_class is None:
if hs.config.server.presence_router_module_class is None:
return
module = hs.config.presence_router_module_class
config = hs.config.presence_router_config
module = hs.config.server.presence_router_module_class
config = hs.config.server.presence_router_config
api = hs.get_module_api()
presence_router = module(config=config, module_api=api)

4
synapse/events/utils.py
View File

@@ -105,7 +105,7 @@ def prune_event_dict(room_version: RoomVersion, event_dict: dict) -> dict:
if event_type == EventTypes.Member:
add_fields("membership")
if room_version.msc3375_redaction_rules:
add_fields("join_authorised_via_users_server")
add_fields(EventContentFields.AUTHORISING_USER)
elif event_type == EventTypes.Create:
# MSC2176 rules state that create events cannot be redacted.
if room_version.msc2176_redaction_rules:
@@ -372,7 +372,7 @@ class EventClientSerializer:
def __init__(self, hs):
self.store = hs.get_datastore()
self.experimental_msc1849_support_enabled = (
hs.config.experimental_msc1849_support_enabled
hs.config.server.experimental_msc1849_support_enabled
)
async def serialize_event(

6
synapse/federation/federation_base.py
View File

@@ -15,7 +15,7 @@
import logging
from collections import namedtuple
from synapse.api.constants import MAX_DEPTH, EventTypes, Membership
from synapse.api.constants import MAX_DEPTH, EventContentFields, EventTypes, Membership
from synapse.api.errors import Codes, SynapseError
from synapse.api.room_versions import EventFormatVersions, RoomVersion
from synapse.crypto.event_signing import check_event_content_hash
@@ -184,10 +184,10 @@ async def _check_sigs_on_pdu(
room_version.msc3083_join_rules
and pdu.type == EventTypes.Member
and pdu.membership == Membership.JOIN
and "join_authorised_via_users_server" in pdu.content
and EventContentFields.AUTHORISING_USER in pdu.content
):
authorising_server = get_domain_from_id(
pdu.content["join_authorised_via_users_server"]
pdu.content[EventContentFields.AUTHORISING_USER]
)
try:
await keyring.verify_event_for_server(

6
synapse/federation/federation_client.py
View File

@@ -37,7 +37,7 @@ from typing import (
import attr
from prometheus_client import Counter
from synapse.api.constants import EventTypes, Membership
from synapse.api.constants import EventContentFields, EventTypes, Membership
from synapse.api.errors import (
CodeMessageException,
Codes,
@@ -875,9 +875,9 @@ class FederationClient(FederationBase):
# If the join is being authorised via allow rules, we need to send
# the /send_join back to the same server that was originally used
# with /make_join.
if "join_authorised_via_users_server" in pdu.content:
if EventContentFields.AUTHORISING_USER in pdu.content:
destinations = [
get_domain_from_id(pdu.content["join_authorised_via_users_server"])
get_domain_from_id(pdu.content[EventContentFields.AUTHORISING_USER])
]
return await self._try_destination_list(

11
synapse/federation/federation_server.py
View File

@@ -34,7 +34,7 @@ from twisted.internet import defer
from twisted.internet.abstract import isIPAddress
from twisted.python import failure
from synapse.api.constants import EduTypes, EventTypes, Membership
from synapse.api.constants import EduTypes, EventContentFields, EventTypes, Membership
from synapse.api.errors import (
AuthError,
Codes,
@@ -765,11 +765,11 @@ class FederationServer(FederationBase):
if (
room_version.msc3083_join_rules
and event.membership == Membership.JOIN
and "join_authorised_via_users_server" in event.content
and EventContentFields.AUTHORISING_USER in event.content
):
# We can only authorise our own users.
authorising_server = get_domain_from_id(
event.content["join_authorised_via_users_server"]
event.content[EventContentFields.AUTHORISING_USER]
)
if authorising_server != self.server_name:
raise SynapseError(
@@ -1008,7 +1008,10 @@ class FederationServer(FederationBase):
async with lock:
logger.info("handling received PDU: %s", event)
try:
await self._federation_event_handler.on_receive_pdu(origin, event)
with nested_logging_context(event.event_id):
await self._federation_event_handler.on_receive_pdu(
origin, event
)
except FederationError as e:
# XXX: Ideally we'd inform the remote we failed to process
# the event, but we can't return an error in the transaction

2
synapse/federation/transport/server/__init__.py
View File

@@ -117,7 +117,7 @@ class PublicRoomList(BaseFederationServlet):
):
super().__init__(hs, authenticator, ratelimiter, server_name)
self.handler = hs.get_room_list_handler()
self.allow_access = hs.config.allow_public_rooms_over_federation
self.allow_access = hs.config.server.allow_public_rooms_over_federation
async def on_GET(
self, origin: str, content: Literal[None], query: Dict[bytes, List[bytes]]

2
synapse/handlers/directory.py
View File

@@ -49,7 +49,7 @@ class DirectoryHandler(BaseHandler):
self.store = hs.get_datastore()
self.config = hs.config
self.enable_room_list_search = hs.config.roomdirectory.enable_room_list_search
self.require_membership = hs.config.require_membership_for_aliases
self.require_membership = hs.config.server.require_membership_for_aliases
self.third_party_event_rules = hs.get_third_party_event_rules()
self.federation = hs.get_federation_client()

15
synapse/handlers/event_auth.py
View File

@@ -22,7 +22,8 @@ from synapse.api.constants import (
RestrictedJoinRuleTypes,
)
from synapse.api.errors import AuthError, Codes, SynapseError
from synapse.api.room_versions import KNOWN_ROOM_VERSIONS, RoomVersion
from synapse.api.room_versions import RoomVersion
from synapse.event_auth import check_auth_rules_for_event
from synapse.events import EventBase
from synapse.events.builder import EventBuilder
from synapse.events.snapshot import EventContext
@@ -45,21 +46,17 @@ class EventAuthHandler:
self._store = hs.get_datastore()
self._server_name = hs.hostname
async def check_from_context(
async def check_auth_rules_from_context(
self,
room_version: str,
room_version_obj: RoomVersion,
event: EventBase,
context: EventContext,
do_sig_check: bool = True,
) -> None:
"""Check an event passes the auth rules at its own auth events"""
auth_event_ids = event.auth_event_ids()
auth_events_by_id = await self._store.get_events(auth_event_ids)
auth_events = {(e.type, e.state_key): e for e in auth_events_by_id.values()}
room_version_obj = KNOWN_ROOM_VERSIONS[room_version]
event_auth.check(
room_version_obj, event, auth_events=auth_events, do_sig_check=do_sig_check
)
check_auth_rules_for_event(room_version_obj, event, auth_events)
def compute_auth_events(
self,

79
synapse/handlers/federation.py
View File

@@ -27,7 +27,12 @@ from unpaddedbase64 import decode_base64
from twisted.internet import defer
from synapse import event_auth
from synapse.api.constants import EventTypes, Membership, RejectedReason
from synapse.api.constants import (
EventContentFields,
EventTypes,
Membership,
RejectedReason,
)
from synapse.api.errors import (
AuthError,
CodeMessageException,
@@ -40,6 +45,10 @@ from synapse.api.errors import (
)
from synapse.api.room_versions import KNOWN_ROOM_VERSIONS, RoomVersion, RoomVersions
from synapse.crypto.event_signing import compute_event_signature
from synapse.event_auth import (
check_auth_rules_for_event,
validate_event_for_room_version,
)
from synapse.events import EventBase
from synapse.events.snapshot import EventContext
from synapse.events.validator import EventValidator
@@ -712,14 +721,14 @@ class FederationHandler(BaseHandler):
if include_auth_user_id:
event_content[
"join_authorised_via_users_server"
EventContentFields.AUTHORISING_USER
] = await self._event_auth_handler.get_user_which_could_invite(
room_id,
state_ids,
)
builder = self.event_builder_factory.new(
room_version.identifier,
builder = self.event_builder_factory.for_room_version(
room_version,
{
"type": EventTypes.Member,
"content": event_content,
@@ -742,10 +751,9 @@ class FederationHandler(BaseHandler):
# The remote hasn't signed it yet, obviously. We'll do the full checks
# when we get the event back in `on_send_join_request`
await self._event_auth_handler.check_from_context(
room_version.identifier, event, context, do_sig_check=False
await self._event_auth_handler.check_auth_rules_from_context(
room_version, event, context
)
return event
async def on_invite_request(
@@ -762,7 +770,7 @@ class FederationHandler(BaseHandler):
if is_blocked:
raise SynapseError(403, "This room has been blocked on this server")
if self.hs.config.block_non_admin_invites:
if self.hs.config.server.block_non_admin_invites:
raise SynapseError(403, "This server does not accept room invites")
if not await self.spam_checker.user_may_invite(
@@ -897,9 +905,9 @@ class FederationHandler(BaseHandler):
)
raise SynapseError(403, "User not from origin", Codes.FORBIDDEN)
room_version = await self.store.get_room_version_id(room_id)
builder = self.event_builder_factory.new(
room_version,
room_version_obj = await self.store.get_room_version(room_id)
builder = self.event_builder_factory.for_room_version(
room_version_obj,
{
"type": EventTypes.Member,
"content": {"membership": Membership.LEAVE},
@@ -916,8 +924,8 @@ class FederationHandler(BaseHandler):
try:
# The remote hasn't signed it yet, obviously. We'll do the full checks
# when we get the event back in `on_send_leave_request`
await self._event_auth_handler.check_from_context(
room_version, event, context, do_sig_check=False
await self._event_auth_handler.check_auth_rules_from_context(
room_version_obj, event, context
)
except AuthError as e:
logger.warning("Failed to create new leave %r because %s", event, e)
@@ -949,10 +957,10 @@ class FederationHandler(BaseHandler):
)
raise SynapseError(403, "User not from origin", Codes.FORBIDDEN)
room_version = await self.store.get_room_version_id(room_id)
room_version_obj = await self.store.get_room_version(room_id)
builder = self.event_builder_factory.new(
room_version,
builder = self.event_builder_factory.for_room_version(
room_version_obj,
{
"type": EventTypes.Member,
"content": {"membership": Membership.KNOCK},
@@ -978,8 +986,8 @@ class FederationHandler(BaseHandler):
try:
# The remote hasn't signed it yet, obviously. We'll do the full checks
# when we get the event back in `on_send_knock_request`
await self._event_auth_handler.check_from_context(
room_version, event, context, do_sig_check=False
await self._event_auth_handler.check_auth_rules_from_context(
room_version_obj, event, context
)
except AuthError as e:
logger.warning("Failed to create new knock %r because %s", event, e)
@@ -1168,7 +1176,8 @@ class FederationHandler(BaseHandler):
auth_for_e[(EventTypes.Create, "")] = create_event
try:
event_auth.check(room_version, e, auth_events=auth_for_e)
validate_event_for_room_version(room_version, e)
check_auth_rules_for_event(room_version, e, auth_for_e)
except SynapseError as err:
# we may get SynapseErrors here as well as AuthErrors. For
# instance, there are a couple of (ancient) events in some
@@ -1245,8 +1254,10 @@ class FederationHandler(BaseHandler):
}
if await self._event_auth_handler.check_host_in_room(room_id, self.hs.hostname):
room_version = await self.store.get_room_version_id(room_id)
builder = self.event_builder_factory.new(room_version, event_dict)
room_version_obj = await self.store.get_room_version(room_id)
builder = self.event_builder_factory.for_room_version(
room_version_obj, event_dict
)
EventValidator().validate_builder(builder)
event, context = await self.event_creation_handler.create_new_client_event(
@@ -1254,7 +1265,7 @@ class FederationHandler(BaseHandler):
)
event, context = await self.add_display_name_to_third_party_invite(
room_version, event_dict, event, context
room_version_obj, event_dict, event, context
)
EventValidator().validate_new(event, self.config)
@@ -1264,8 +1275,9 @@ class FederationHandler(BaseHandler):
event.internal_metadata.send_on_behalf_of = self.hs.hostname
try:
await self._event_auth_handler.check_from_context(
room_version, event, context
validate_event_for_room_version(room_version_obj, event)
await self._event_auth_handler.check_auth_rules_from_context(
room_version_obj, event, context
)
except AuthError as e:
logger.warning("Denying new third party invite %r because %s", event, e)
@@ -1299,22 +1311,25 @@ class FederationHandler(BaseHandler):
"""
assert_params_in_dict(event_dict, ["room_id"])
room_version = await self.store.get_room_version_id(event_dict["room_id"])
room_version_obj = await self.store.get_room_version(event_dict["room_id"])
# NB: event_dict has a particular specced format we might need to fudge
# if we change event formats too much.
builder = self.event_builder_factory.new(room_version, event_dict)
builder = self.event_builder_factory.for_room_version(
room_version_obj, event_dict
)
event, context = await self.event_creation_handler.create_new_client_event(
builder=builder
)
event, context = await self.add_display_name_to_third_party_invite(
room_version, event_dict, event, context
room_version_obj, event_dict, event, context
)
try:
await self._event_auth_handler.check_from_context(
room_version, event, context
validate_event_for_room_version(room_version_obj, event)
await self._event_auth_handler.check_auth_rules_from_context(
room_version_obj, event, context
)
except AuthError as e:
logger.warning("Denying third party invite %r because %s", event, e)
@@ -1331,7 +1346,7 @@ class FederationHandler(BaseHandler):
async def add_display_name_to_third_party_invite(
self,
room_version: str,
room_version_obj: RoomVersion,
event_dict: JsonDict,
event: EventBase,
context: EventContext,
@@ -1363,7 +1378,9 @@ class FederationHandler(BaseHandler):
# auth checks. If we need the invite and don't have it then the
# auth check code will explode appropriately.
builder = self.event_builder_factory.new(room_version, event_dict)
builder = self.event_builder_factory.for_room_version(
room_version_obj, event_dict
)
EventValidator().validate_builder(builder)
event, context = await self.event_creation_handler.create_new_client_event(
builder=builder

18
synapse/handlers/federation_event.py
View File

@@ -29,7 +29,6 @@ from typing import (
from prometheus_client import Counter
from synapse import event_auth
from synapse.api.constants import (
EventContentFields,
EventTypes,
@@ -47,7 +46,11 @@ from synapse.api.errors import (
SynapseError,
)
from synapse.api.room_versions import KNOWN_ROOM_VERSIONS
from synapse.event_auth import auth_types_for_event
from synapse.event_auth import (
auth_types_for_event,
check_auth_rules_for_event,
validate_event_for_room_version,
)
from synapse.events import EventBase
from synapse.events.snapshot import EventContext
from synapse.federation.federation_client import InvalidResponseError
@@ -1207,7 +1210,8 @@ class FederationEventHandler:
context = EventContext.for_outlier()
try:
event_auth.check(room_version_obj, event, auth_events=auth)
validate_event_for_room_version(room_version_obj, event)
check_auth_rules_for_event(room_version_obj, event, auth)
except AuthError as e:
logger.warning("Rejecting %r because %s", event, e)
context.rejected = RejectedReason.AUTH_ERROR
@@ -1282,7 +1286,8 @@ class FederationEventHandler:
auth_events_for_auth = calculated_auth_event_map
try:
event_auth.check(room_version_obj, event, auth_events=auth_events_for_auth)
validate_event_for_room_version(room_version_obj, event)
check_auth_rules_for_event(room_version_obj, event, auth_events_for_auth)
except AuthError as e:
logger.warning("Failed auth resolution for %r because %s", event, e)
context.rejected = RejectedReason.AUTH_ERROR
@@ -1394,7 +1399,10 @@ class FederationEventHandler:
}
try:
event_auth.check(room_version_obj, event, auth_events=current_auth_events)
# TODO: skip the call to validate_event_for_room_version? we should already
# have validated the event.
validate_event_for_room_version(room_version_obj, event)
check_auth_rules_for_event(room_version_obj, event, current_auth_events)
except AuthError as e:
logger.warning(
"Soft-failing %r (from %s) because %s",

2
synapse/handlers/identity.py
View File

@@ -57,7 +57,7 @@ class IdentityHandler(BaseHandler):
self.http_client = SimpleHttpClient(hs)
# An HTTP client for contacting identity servers specified by clients.
self.blacklisting_http_client = SimpleHttpClient(
hs, ip_blacklist=hs.config.federation_ip_range_blacklist
hs, ip_blacklist=hs.config.server.federation_ip_range_blacklist
)
self.federation_http_client = hs.get_federation_http_client()
self.hs = hs

45
synapse/handlers/message.py
View File

@@ -40,9 +40,11 @@ from synapse.api.errors import (
NotFoundError,
ShadowBanError,
SynapseError,
UnsupportedRoomVersionError,
)
from synapse.api.room_versions import KNOWN_ROOM_VERSIONS, RoomVersions
from synapse.api.urls import ConsentURIBuilder
from synapse.event_auth import validate_event_for_room_version
from synapse.events import EventBase
from synapse.events.builder import EventBuilder
from synapse.events.snapshot import EventContext
@@ -80,7 +82,7 @@ class MessageHandler:
self.storage = hs.get_storage()
self.state_store = self.storage.state
self._event_serializer = hs.get_event_client_serializer()
self._ephemeral_events_enabled = hs.config.enable_ephemeral_messages
self._ephemeral_events_enabled = hs.config.server.enable_ephemeral_messages
# The scheduled call to self._expire_event. None if no call is currently
# scheduled.
@@ -414,7 +416,9 @@ class EventCreationHandler:
self.server_name = hs.hostname
self.notifier = hs.get_notifier()
self.config = hs.config
self.require_membership_for_aliases = hs.config.require_membership_for_aliases
self.require_membership_for_aliases = (
hs.config.server.require_membership_for_aliases
)
self._events_shard_config = self.config.worker.events_shard_config
self._instance_name = hs.get_instance_name()
@@ -424,7 +428,7 @@ class EventCreationHandler:
Membership.JOIN,
Membership.KNOCK,
}
if self.hs.config.include_profile_data_on_invite:
if self.hs.config.server.include_profile_data_on_invite:
self.membership_types_to_include_profile_data_in.add(Membership.INVITE)
self.send_event = ReplicationSendEventRestServlet.make_client(hs)
@@ -460,11 +464,11 @@ class EventCreationHandler:
#
self._rooms_to_exclude_from_dummy_event_insertion: Dict[str, int] = {}
# The number of forward extremeities before a dummy event is sent.
self._dummy_events_threshold = hs.config.dummy_events_threshold
self._dummy_events_threshold = hs.config.server.dummy_events_threshold
if (
self.config.worker.run_background_tasks
and self.config.cleanup_extremities_with_dummy_events
and self.config.server.cleanup_extremities_with_dummy_events
):
self.clock.looping_call(
lambda: run_as_background_process(
@@ -476,7 +480,7 @@ class EventCreationHandler:
self._message_handler = hs.get_message_handler()
self._ephemeral_events_enabled = hs.config.enable_ephemeral_messages
self._ephemeral_events_enabled = hs.config.server.enable_ephemeral_messages
self._external_cache = hs.get_external_cache()
@@ -550,16 +554,22 @@ class EventCreationHandler:
await self.auth.check_auth_blocking(requester=requester)
if event_dict["type"] == EventTypes.Create and event_dict["state_key"] == "":
room_version = event_dict["content"]["room_version"]
room_version_id = event_dict["content"]["room_version"]
room_version_obj = KNOWN_ROOM_VERSIONS.get(room_version_id)
if not room_version_obj:
# this can happen if support is withdrawn for a room version
raise UnsupportedRoomVersionError(room_version_id)
else:
try:
room_version = await self.store.get_room_version_id(
room_version_obj = await self.store.get_room_version(
event_dict["room_id"]
)
except NotFoundError:
raise AuthError(403, "Unknown room")
builder = self.event_builder_factory.new(room_version, event_dict)
builder = self.event_builder_factory.for_room_version(
room_version_obj, event_dict
)
self.validator.validate_builder(builder)
@@ -1079,9 +1089,17 @@ class EventCreationHandler:
EventTypes.Create,
"",
):
room_version = event.content.get("room_version", RoomVersions.V1.identifier)
room_version_id = event.content.get(
"room_version", RoomVersions.V1.identifier
)
room_version_obj = KNOWN_ROOM_VERSIONS.get(room_version_id)
if not room_version_obj:
raise UnsupportedRoomVersionError(
"Attempt to create a room with unsupported room version %s"
% (room_version_id,)
)
else:
room_version = await self.store.get_room_version_id(event.room_id)
room_version_obj = await self.store.get_room_version(event.room_id)
if event.internal_metadata.is_out_of_band_membership():
# the only sort of out-of-band-membership events we expect to see here are
@@ -1090,8 +1108,9 @@ class EventCreationHandler:
assert event.content["membership"] == Membership.LEAVE
else:
try:
await self._event_auth_handler.check_from_context(
room_version, event, context
validate_event_for_room_version(room_version_obj, event)
await self._event_auth_handler.check_auth_rules_from_context(
room_version_obj, event, context
)
except AuthError as err:
logger.warning("Denying new event %r because %s", event, err)

14
synapse/handlers/pagination.py
View File

@@ -85,12 +85,18 @@ class PaginationHandler:
self._purges_by_id: Dict[str, PurgeStatus] = {}
self._event_serializer = hs.get_event_client_serializer()
self._retention_default_max_lifetime = hs.config.retention_default_max_lifetime
self._retention_default_max_lifetime = (
hs.config.server.retention_default_max_lifetime
)
self._retention_allowed_lifetime_min = hs.config.retention_allowed_lifetime_min
self._retention_allowed_lifetime_max = hs.config.retention_allowed_lifetime_max
self._retention_allowed_lifetime_min = (
hs.config.server.retention_allowed_lifetime_min
)
self._retention_allowed_lifetime_max = (
hs.config.server.retention_allowed_lifetime_max
)
if hs.config.worker.run_background_tasks and hs.config.retention_enabled:
if hs.config.worker.run_background_tasks and hs.config.server.retention_enabled:
# Run the purge jobs described in the configuration file.
for job in hs.config.server.retention_purge_jobs:
logger.info("Setting up purge job with config: %s", job)

4
synapse/handlers/presence.py
View File

@@ -52,6 +52,7 @@ import synapse.metrics
from synapse.api.constants import EventTypes, Membership, PresenceState
from synapse.api.errors import SynapseError
from synapse.api.presence import UserPresenceState
from synapse.appservice import ApplicationService
from synapse.events.presence_router import PresenceRouter
from synapse.logging.context import run_in_background
from synapse.logging.utils import log_function
@@ -1521,10 +1522,11 @@ class PresenceEventSource(EventSource[int, UserPresenceState]):
user: UserID,
from_key: Optional[int],
limit: Optional[int] = None,
room_ids: Optional[List[str]] = None,
room_ids: Optional[Collection[str]] = None,
is_guest: bool = False,
explicit_room_id: Optional[str] = None,
include_offline: bool = True,
service: Optional[ApplicationService] = None,
) -> Tuple[List[UserPresenceState], int]:
# The process for getting presence events are:
# 1. Get the rooms the user is in.

2
synapse/handlers/profile.py
View File

@@ -397,7 +397,7 @@ class ProfileHandler(BaseHandler):
# when building a membership event. In this case, we must allow the
# lookup.
if (
not self.hs.config.limit_profile_requests_to_users_who_share_rooms
not self.hs.config.server.limit_profile_requests_to_users_who_share_rooms
or not requester
):
return

6
synapse/handlers/register.py
View File

@@ -340,6 +340,8 @@ class RegistrationHandler(BaseHandler):
auth_provider=(auth_provider_id or ""),
).inc()
# If the user does not need to consent at registration, auto-join any
# configured rooms.
if not self.hs.config.consent.user_consent_at_registration:
if not self.hs.config.auto_join_rooms_for_guests and make_guest:
logger.info(
@@ -387,7 +389,7 @@ class RegistrationHandler(BaseHandler):
"preset": self.hs.config.registration.autocreate_auto_join_room_preset,
}
# If the configuration providers a user ID to create rooms with, use
# If the configuration provides a user ID to create rooms with, use
# that instead of the first user registered.
requires_join = False
if self.hs.config.registration.auto_join_user_id:
@@ -854,7 +856,7 @@ class RegistrationHandler(BaseHandler):
# Necessary due to auth checks prior to the threepid being
# written to the db
if is_threepid_reserved(
self.hs.config.mau_limits_reserved_threepids, threepid
self.hs.config.server.mau_limits_reserved_threepids, threepid
):
await self.store.upsert_monthly_active_user(user_id)

Some files were not shown because too many files have changed in this diff Show More