1
0

Compare commits

..

48 Commits

Author SHA1 Message Date
Andrew Morgan
0ddaae83c3 Merge branch 'release-v1.12.4' of github.com:matrix-org/synapse into anoa/temp_working_cache_config
* 'release-v1.12.4' of github.com:matrix-org/synapse: (123 commits)
  1.12.4
  formatting for the changelog
  1.12.4rc1
  1.12.4rc1
  Do not treat display names as globs for push rules. (#7271)
  Query missing cross-signing keys on local sig upload (#7289)
  Fix changelog file
  Support GET account_data requests on a worker (#7311)
  Revert "Query missing cross-signing keys on local sig upload"
  Always send the user updates to their own device list (#7160)
  Query missing cross-signing keys on local sig upload
  Only register devices edu handler on the master process (#7255)
  tweak changelog
  1.12.3
  Fix the debian build in a better way. (#7212)
  Fix changelog wording
  1.12.2
  Pin Pillow>=4.3.0,<7.1.0 to fix dep issue
  1.12.1
  Note where bugs were introduced
  ...
2020-04-24 13:36:35 +01:00
Patrick Cloke
ce9b62e13f 1.12.4 2020-04-23 10:59:10 -04:00
Richard van der Hoff
ba0aac5e44 formatting for the changelog 2020-04-22 14:50:51 +01:00
Richard van der Hoff
dc8003f921 1.12.4rc1 2020-04-22 14:29:39 +01:00
Richard van der Hoff
83af1079d6 1.12.4rc1 2020-04-22 14:28:23 +01:00
Patrick Cloke
51f358e2fe Do not treat display names as globs for push rules. (#7271) 2020-04-22 13:07:12 +01:00
Andrew Morgan
f89ad3b6df Query missing cross-signing keys on local sig upload (#7289) 2020-04-22 12:29:36 +01:00
Richard van der Hoff
556566f0b8 Fix changelog file
I updated the PR and forgot to update the changelog.
2020-04-21 13:20:16 +01:00
Richard van der Hoff
974c0d726a Support GET account_data requests on a worker (#7311) 2020-04-21 10:46:30 +01:00
Richard van der Hoff
d41c8f6d4d Revert "Query missing cross-signing keys on local sig upload"
This was incorrectly merged to the release branch before it was ready.

This reverts commit 72fe2affb6.
2020-04-20 17:54:35 +01:00
David Baker
40f79f58bf Always send the user updates to their own device list (#7160) 2020-04-20 17:20:38 +01:00
Andrew Morgan
72fe2affb6 Query missing cross-signing keys on local sig upload
Add changelog

Save retrieved keys to the db

lint

Fix and de-brittle remote result dict processing

Use query_user_devices instead, assume only master, self_signing key types

Make changelog more useful

Remove very specific exception handling

Wrap get_verify_key_from_cross_signing_key in a try/except

Note that _get_e2e_cross_signing_verify_key can raise a SynapseError

lint

Add comment explaining why this is useful

Only fetch master and self_signing key types

Fix log statements, docstrings

Remove extraneous items from remote query try/except

lint

Factor key retrieval out into a separate function

Send device updates, modeled after SigningKeyEduUpdater._handle_signing_key_updates

Update method docstring
2020-04-17 15:47:49 +01:00
Andrew Morgan
ac6a84818f Only register devices edu handler on the master process (#7255) 2020-04-14 11:36:24 +01:00
Amber H. Brown
f004ceed71 Merge remote-tracking branch 'origin/develop' into hawkowl/cache-config-without-synctl 2020-02-27 23:18:16 +11:00
Amber H. Brown
965e259426 fixes 2020-02-27 23:18:12 +11:00
Amber H. Brown
0fc0a7deb8 fix 2020-02-27 23:15:45 +11:00
Amber H. Brown
2619891343 fix 2020-02-17 17:50:35 +11:00
Amber H. Brown
5f508e728a add tests for individual cache sizing, and fix up the individual cache sizing logic that got deleted when resizing-on-the-fly did 2020-02-17 17:47:56 +11:00
Amber H. Brown
4aeb6fbe2c Merge remote-tracking branch 'origin/develop' into hawkowl/cache-config-without-synctl 2020-02-17 17:20:10 +11:00
Amber H. Brown
18c1dbfbac don't need this comment 2020-01-22 02:57:15 +11:00
Amber H. Brown
ac020dee7a fix style 2020-01-22 02:38:50 +11:00
Amber H. Brown
2f4dbfa3e1 document as well as refactor so that CacheMetric is not nested 2020-01-22 00:00:10 +11:00
Amber H. Brown
a96b5d9ca7 Load cache factors from the environment, and add a test. 2020-01-21 23:46:08 +11:00
Amber H. Brown
0b069b70a1 make expiring caches resizeable 2020-01-17 02:14:33 +11:00
Amber H. Brown
125c5a01dd Merge branch 'hawkowl/benchmark-lrucache' into hawkowl/cache-config-without-synctl 2020-01-17 01:50:44 +11:00
Amber H. Brown
a21702fe76 cleanup so it can refer to late-config 2020-01-17 01:48:27 +11:00
Amber H. Brown
0a02b2a1c5 cleanup 2020-01-17 01:05:50 +11:00
Amber H. Brown
c76a412cc3 Merge remote-tracking branch 'origin/develop' into hawkowl/cache-config-without-synctl 2020-01-17 01:03:26 +11:00
Amber H. Brown
9735a08f04 newsfile 2019-12-03 20:24:43 +11:00
Amber H. Brown
946650c65a Merge remote-tracking branch 'origin/develop' into hawkowl/benchmark-lrucache 2019-12-03 20:22:40 +11:00
Amber H. Brown
e174b2d19c fix style 2019-12-02 19:49:13 +11:00
Amber H. Brown
f7ec52670b add some LruCache benchmarks 2019-12-02 19:45:20 +11:00
Amber H. Brown
0e368eec58 more fixes 2019-12-02 19:17:19 +11:00
Amber H. Brown
c11c8ad39f more fixes 2019-12-02 18:30:11 +11:00
Amber H. Brown
d70be1871c Merge remote-tracking branch 'origin/develop' into hawkowl/structured-logging-perf 2019-12-02 17:42:26 +11:00
Amber H. Brown
92d4d1342a fixes 2019-12-02 17:42:24 +11:00
Amber H. Brown
8439ce5e53 Merge remote-tracking branch 'origin/develop' into hawkowl/structured-logging-perf 2019-11-26 03:47:24 +11:00
Amber H. Brown
50fcb4a8c5 Re-sync every so often, in case caches appear 2019-11-22 00:05:12 +11:00
Amber H. Brown
a14831d209 Move cache configuration into a homeserver config option, instead of environment variables. 2019-11-21 06:31:37 +11:00
Amber H. Brown
f888515a3c Merge remote-tracking branch 'origin/develop' into hawkowl/structured-logging-perf 2019-11-05 05:06:20 +11:00
Amber H. Brown
c580eb32ba revert this 2019-11-05 00:51:51 +11:00
Amber H. Brown
e55591dc39 Merge remote-tracking branch 'origin/develop' into hawkowl/structured-logging-perf 2019-11-05 00:49:27 +11:00
Amber Brown
babf4eaece Merge branch 'develop' into hawkowl/structured-logging-perf 2019-11-02 01:39:14 +11:00
Amber H. Brown
73cfdebfec fix 2019-10-31 21:15:07 +11:00
Amber H. Brown
f36f3ab990 Merge remote-tracking branch 'origin/develop' into hawkowl/structured-logging-perf 2019-10-31 21:13:52 +11:00
Amber H. Brown
135fdaae0d update linting 2019-10-31 21:01:37 +11:00
Amber Brown
d684ec8a2b benchmarks 2019-10-24 22:09:43 +03:00
Amber Brown
99ab65af2f fix up logging to use rapidjson 2019-10-24 20:59:06 +03:00
296 changed files with 3679 additions and 6944 deletions

View File

@@ -5,6 +5,8 @@ Message history can be paginated
Can re-join room if re-invited
/upgrade creates a new room
The only membership state included in an initial sync is for all the senders in the timeline
Local device key changes get to remote servers

View File

@@ -1,14 +1,26 @@
Next version
============
Synapse 1.12.4 (2020-04-23)
===========================
* Two new templates (`sso_auth_confirm.html` and `sso_account_deactivated.html`)
were added to Synapse. If your Synapse is configured to use SSO and a custom
`sso_redirect_confirm_template_dir` configuration then these templates will
need to be duplicated into that directory.
No significant changes.
Synapse 1.12.4rc1 (2020-04-22)
==============================
Features
--------
- Always send users their own device updates. ([\#7160](https://github.com/matrix-org/synapse/issues/7160))
- Add support for handling GET requests for `account_data` on a worker. ([\#7311](https://github.com/matrix-org/synapse/issues/7311))
Bugfixes
--------
- Fix a bug that prevented cross-signing with users on worker-mode synapses. ([\#7255](https://github.com/matrix-org/synapse/issues/7255))
- Do not treat display names as globs in push rules. ([\#7271](https://github.com/matrix-org/synapse/issues/7271))
- Fix a bug with cross-signing devices belonging to remote users who did not share a room with any user on the local homeserver. ([\#7289](https://github.com/matrix-org/synapse/issues/7289))
* Plugins using the `complete_sso_login` method of `synapse.module_api.ModuleApi`
should update to using the async/await version `complete_sso_login_async` which
includes additional checks. The non-async version is considered deprecated.
Synapse 1.12.3 (2020-04-03)
===========================
@@ -19,10 +31,13 @@ correctly fix the issue with building the Debian packages. ([\#7212](https://git
Synapse 1.12.2 (2020-04-02)
===========================
This release works around [an issue](https://github.com/matrix-org/synapse/issues/7208) with building the debian packages.
This release works around [an
issue](https://github.com/matrix-org/synapse/issues/7208) with building the
debian packages.
No other significant changes since 1.12.1.
Synapse 1.12.1 (2020-04-02)
===========================
@@ -39,6 +54,7 @@ Bugfixes
- Avoid importing `sqlite3` when using the postgres backend. Contributed by David Vo. ([\#7155](https://github.com/matrix-org/synapse/issues/7155)). Introduced in v1.12.0rc1.
- Fix a bug which could cause outbound federation traffic to stop working if a client uploaded an incorrect e2e device signature. ([\#7177](https://github.com/matrix-org/synapse/issues/7177)). Introduced in v1.11.0.
Synapse 1.12.0 (2020-03-23)
===========================

View File

@@ -2,6 +2,7 @@
- [Installing Synapse](#installing-synapse)
- [Installing from source](#installing-from-source)
- [Platform-Specific Instructions](#platform-specific-instructions)
- [Troubleshooting Installation](#troubleshooting-installation)
- [Prebuilt packages](#prebuilt-packages)
- [Setting up Synapse](#setting-up-synapse)
- [TLS certificates](#tls-certificates)
@@ -9,7 +10,6 @@
- [Registering a user](#registering-a-user)
- [Setting up a TURN server](#setting-up-a-turn-server)
- [URL previews](#url-previews)
- [Troubleshooting Installation](#troubleshooting-installation)
# Choosing your server name
@@ -36,7 +36,7 @@ that your email address is probably `user@example.com` rather than
System requirements:
- POSIX-compliant system (tested on Linux & OS X)
- Python 3.5.2 or later, up to Python 3.8.
- Python 3.5, 3.6, 3.7 or 3.8.
- At least 1GB of free RAM if you want to join large public rooms like #matrix:matrix.org
Synapse is written in Python but some of the libraries it uses are written in
@@ -70,7 +70,7 @@ pip install -U matrix-synapse
```
Before you can start Synapse, you will need to generate a configuration
file. To do this, run (in your virtualenv, as before):
file. To do this, run (in your virtualenv, as before)::
```
cd ~/synapse
@@ -84,24 +84,22 @@ python -m synapse.app.homeserver \
... substituting an appropriate value for `--server-name`.
This command will generate you a config file that you can then customise, but it will
also generate a set of keys for you. These keys will allow your homeserver to
identify itself to other homeserver, so don't lose or delete them. It would be
also generate a set of keys for you. These keys will allow your Home Server to
identify itself to other Home Servers, so don't lose or delete them. It would be
wise to back them up somewhere safe. (If, for whatever reason, you do need to
change your homeserver's keys, you may find that other homeserver have the
change your Home Server's keys, you may find that other Home Servers have the
old key cached. If you update the signing key, you should change the name of the
key in the `<server name>.signing.key` file (the second word) to something
different. See the
[spec](https://matrix.org/docs/spec/server_server/latest.html#retrieving-server-keys)
for more information on key management).
for more information on key management.)
To actually run your new homeserver, pick a working directory for Synapse to
run (e.g. `~/synapse`), and:
run (e.g. `~/synapse`), and::
```
cd ~/synapse
source env/bin/activate
synctl start
```
cd ~/synapse
source env/bin/activate
synctl start
### Platform-Specific Instructions
@@ -112,7 +110,7 @@ Installing prerequisites on Ubuntu or Debian:
```
sudo apt-get install build-essential python3-dev libffi-dev \
python3-pip python3-setuptools sqlite3 \
libssl-dev virtualenv libjpeg-dev libxslt1-dev
libssl-dev python3-virtualenv libjpeg-dev libxslt1-dev
```
#### ArchLinux
@@ -190,7 +188,7 @@ doas pkg_add python libffi py-pip py-setuptools sqlite3 py-virtualenv \
There is currently no port for OpenBSD. Additionally, OpenBSD's security
settings require a slightly more difficult installation process.
(XXX: I suspect this is out of date)
XXX: I suspect this is out of date.
1. Create a new directory in `/usr/local` called `_synapse`. Also, create a
new user called `_synapse` and set that directory as the new user's home.
@@ -198,7 +196,7 @@ settings require a slightly more difficult installation process.
write and execute permissions on the same memory space to be run from
`/usr/local`.
2. `su` to the new `_synapse` user and change to their home directory.
3. Create a new virtualenv: `virtualenv -p python3 ~/.synapse`
3. Create a new virtualenv: `virtualenv -p python2.7 ~/.synapse`
4. Source the virtualenv configuration located at
`/usr/local/_synapse/.synapse/bin/activate`. This is done in `ksh` by
using the `.` command, rather than `bash`'s `source`.
@@ -219,6 +217,45 @@ be found at https://docs.microsoft.com/en-us/windows/wsl/install-win10 for
Windows 10 and https://docs.microsoft.com/en-us/windows/wsl/install-on-server
for Windows Server.
### Troubleshooting Installation
XXX a bunch of this is no longer relevant.
Synapse requires pip 8 or later, so if your OS provides too old a version you
may need to manually upgrade it::
sudo pip install --upgrade pip
Installing may fail with `Could not find any downloads that satisfy the requirement pymacaroons-pynacl (from matrix-synapse==0.12.0)`.
You can fix this by manually upgrading pip and virtualenv::
sudo pip install --upgrade virtualenv
You can next rerun `virtualenv -p python3 synapse` to update the virtual env.
Installing may fail during installing virtualenv with `InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.`
You can fix this by manually installing ndg-httpsclient::
pip install --upgrade ndg-httpsclient
Installing may fail with `mock requires setuptools>=17.1. Aborting installation`.
You can fix this by upgrading setuptools::
pip install --upgrade setuptools
If pip crashes mid-installation for reason (e.g. lost terminal), pip may
refuse to run until you remove the temporary installation directory it
created. To reset the installation::
rm -rf /tmp/pip_install_matrix
pip seems to leak *lots* of memory during installation. For instance, a Linux
host with 512MB of RAM may run out of memory whilst installing Twisted. If this
happens, you will have to individually install the dependencies which are
failing, e.g.::
pip install twisted
## Prebuilt packages
As an alternative to installing from source, prebuilt packages are available
@@ -277,7 +314,7 @@ For `buster` and `sid`, Synapse is available in the Debian repositories and
it should be possible to install it with simply:
```
sudo apt install matrix-synapse
sudo apt install matrix-synapse
```
There is also a version of `matrix-synapse` in `stretch-backports`. Please see
@@ -338,17 +375,15 @@ sudo pip install py-bcrypt
Synapse can be found in the void repositories as 'synapse':
```
xbps-install -Su
xbps-install -S synapse
```
xbps-install -Su
xbps-install -S synapse
### FreeBSD
Synapse can be installed via FreeBSD Ports or Packages contributed by Brendan Molloy from:
- Ports: `cd /usr/ports/net-im/py-matrix-synapse && make install clean`
- Packages: `pkg install py37-matrix-synapse`
- Packages: `pkg install py27-matrix-synapse`
### NixOS
@@ -385,7 +420,6 @@ so, you will need to edit `homeserver.yaml`, as follows:
resources:
- names: [client, federation]
```
* You will also need to uncomment the `tls_certificate_path` and
`tls_private_key_path` lines under the `TLS` section. You can either
point these settings at an existing certificate and key, or you can
@@ -393,15 +427,15 @@ so, you will need to edit `homeserver.yaml`, as follows:
for having Synapse automatically provision and renew federation
certificates through ACME can be found at [ACME.md](docs/ACME.md).
Note that, as pointed out in that document, this feature will not
work with installs set up after November 2019.
work with installs set up after November 2019.
If you are using your own certificate, be sure to use a `.pem` file that
includes the full certificate chain including any intermediate certificates
(for instance, if using certbot, use `fullchain.pem` as your certificate, not
`cert.pem`).
For a more detailed guide to configuring your server for federation, see
[federate.md](docs/federate.md).
[federate.md](docs/federate.md)
## Email
@@ -448,7 +482,7 @@ on your server even if `enable_registration` is `false`.
## Setting up a TURN server
For reliable VoIP calls to be routed via this homeserver, you MUST configure
a TURN server. See [docs/turn-howto.md](docs/turn-howto.md) for details.
a TURN server. See [docs/turn-howto.md](docs/turn-howto.md) for details.
## URL previews
@@ -457,24 +491,10 @@ turn it on you must enable the `url_preview_enabled: True` config parameter
and explicitly specify the IP ranges that Synapse is not allowed to spider for
previewing in the `url_preview_ip_range_blacklist` configuration parameter.
This is critical from a security perspective to stop arbitrary Matrix users
spidering 'internal' URLs on your network. At the very least we recommend that
spidering 'internal' URLs on your network. At the very least we recommend that
your loopback and RFC1918 IP addresses are blacklisted.
This also requires the optional `lxml` and `netaddr` python dependencies to be
installed. This in turn requires the `libxml2` library to be available - on
This also requires the optional lxml and netaddr python dependencies to be
installed. This in turn requires the libxml2 library to be available - on
Debian/Ubuntu this means `apt-get install libxml2-dev`, or equivalent for
your OS.
# Troubleshooting Installation
`pip` seems to leak *lots* of memory during installation. For instance, a Linux
host with 512MB of RAM may run out of memory whilst installing Twisted. If this
happens, you will have to individually install the dependencies which are
failing, e.g.:
```
pip install twisted
```
If you have any other problems, feel free to ask in
[#synapse:matrix.org](https://matrix.to/#/#synapse:matrix.org).

1
changelog.d/6391.feature Normal file
View File

@@ -0,0 +1 @@
Synapse's cache factor can now be configured in `homeserver.yaml` by the `caches.global_factor` setting. Additionally, `caches.per_cache_factors` controls the cache factors for individual caches.

View File

@@ -1 +0,0 @@
Don't attempt to use an invalid sqlite config if no database configuration is provided. Contributed by @nekatak.

View File

@@ -1 +0,0 @@
Fix single-sign on with CAS systems: pass the same service URL when requesting the CAS ticket and when calling the `proxyValidate` URL. Contributed by @Naugrimm.

View File

@@ -1 +0,0 @@
Fix missing field `default` when fetching user-defined push rules.

View File

@@ -1 +0,0 @@
Update Debian installation instructions to recommend installing the `virtualenv` package instead of `python3-virtualenv`.

View File

@@ -1 +0,0 @@
Improve error responses when accessing remote public room lists.

View File

@@ -1 +0,0 @@
Transfer alias mappings on room upgrade.

View File

@@ -1 +0,0 @@
Improve the documentation for database configuration.

View File

@@ -1 +0,0 @@
Extend the `web_client_location` option to accept an absolute URL to use as a redirect. Adds a warning when running the web client on the same hostname as homeserver. Contributed by Martin Milata.

View File

@@ -1 +0,0 @@
Set `Referrer-Policy` header to `no-referrer` on media downloads.

View File

@@ -1 +0,0 @@
Change device list streams to have one row per ID.

View File

@@ -1 +0,0 @@
Remove concept of a non-limited stream.

View File

@@ -1 +0,0 @@
Move catchup of replication streams logic to worker.

View File

@@ -1 +0,0 @@
Admin API `POST /_synapse/admin/v1/join/<roomIdOrAlias>` to join users to a room like `auto_join_rooms` for creation of users.

View File

@@ -1 +0,0 @@
Ensure that a user inteactive authentication session is tied to a single request.

View File

@@ -1 +0,0 @@
Fix a bug in the federation API which could cause occasional "Failed to get PDU" errors.

View File

@@ -1 +0,0 @@
Add options to prevent users from changing their profile or associated 3PIDs.

View File

@@ -1 +0,0 @@
Support SSO in the user interactive authentication workflow.

View File

@@ -1 +0,0 @@
Update pre-built package name for FreeBSD.

View File

@@ -1 +0,0 @@
Return the proper error (M_BAD_ALIAS) when a non-existant canonical alias is provided.

View File

@@ -1 +0,0 @@
Convert some of synapse.rest.media to async/await.

View File

@@ -1 +0,0 @@
De-duplicate / remove unused REST code for login and auth.

View File

@@ -1 +0,0 @@
Convert `*StreamRow` classes to inner classes.

View File

@@ -1 +0,0 @@
Fix a bug which meant that groups updates were not correctly replicated between workers.

View File

@@ -1 +0,0 @@
Allow server admins to define and enforce a password policy (MSC2000).

View File

@@ -1 +0,0 @@
Update postgres docs with login troubleshooting information.

View File

@@ -1 +0,0 @@
Clean up some LoggingContext code.

View File

@@ -1 +0,0 @@
Add explicit `instance_id` for USER_SYNC commands and remove implicit `conn_id` usage.

View File

@@ -1 +0,0 @@
Fix starting workers when federation sending not split out.

View File

@@ -1 +0,0 @@
Refactored the CAS authentication logic to a separate class.

View File

@@ -1 +0,0 @@
Remove nonfunctional `captcha_bypass_secret` option from `homeserver.yaml`.

View File

@@ -1 +0,0 @@
Clean up INSTALL.md a bit.

View File

@@ -1 +0,0 @@
Add documentation for running a local CAS server for testing.

View File

@@ -1 +0,0 @@
Ensure `is_verified` is a boolean in responses to `GET /_matrix/client/r0/room_keys/keys`. Also warn the user if they forgot the `version` query param.

View File

@@ -1 +0,0 @@
Fix error page being shown when a custom SAML handler attempted to redirect when processing an auth response.

View File

@@ -1 +0,0 @@
Improve the support for SSO authentication on the login fallback page.

View File

@@ -1 +0,0 @@
Always whitelist the login fallback in the SSO configuration if `public_baseurl` is set.

View File

@@ -1 +0,0 @@
Avoid importing `sqlite3` when using the postgres backend. Contributed by David Vo.

View File

@@ -1 +0,0 @@
Add tests for outbound device pokes.

View File

@@ -1 +0,0 @@
Fix device list update stream ids going backward.

View File

@@ -1 +0,0 @@
Fix excessive CPU usage by `prune_old_outbound_device_pokes` job.

View File

@@ -1 +0,0 @@
Always send users their own device updates.

View File

@@ -1 +0,0 @@
Improve README.md by being explicit about public IP recommendation for TURN relaying.

View File

@@ -1 +0,0 @@
Fix a small typo in the `metrics_flags` config option.

View File

@@ -1 +0,0 @@
Fix a bug which could cause outbound federation traffic to stop working if a client uploaded an incorrect e2e device signature.

View File

@@ -1 +0,0 @@
Fix a bug which could cause incorrect 'cyclic dependency' error.

View File

@@ -1 +0,0 @@
Clean up some LoggingContext code.

View File

@@ -1 +0,0 @@
Clean up some LoggingContext code.

View File

@@ -1 +0,0 @@
Convert some of synapse.rest.media to async/await.

View File

@@ -1 +0,0 @@
Move client command handling out of TCP protocol.

View File

@@ -1 +0,0 @@
Support SSO in the user interactive authentication workflow.

View File

@@ -1 +0,0 @@
Move server command handling out of TCP protocol.

View File

@@ -1 +0,0 @@
Fix consistency of HTTP status codes reported in log lines.

View File

@@ -1 +0,0 @@
Only run one background database update at a time.

View File

@@ -1 +0,0 @@
Admin users are no longer required to be in a room to create an alias for it.

View File

@@ -1 +0,0 @@
Remove sent outbound device list pokes from the database.

View File

@@ -1 +0,0 @@
Add a background database update job to clear out duplicate `device_lists_outbound_pokes`.

View File

@@ -1 +0,0 @@
Move catchup of replication streams logic to worker.

View File

@@ -1 +0,0 @@
Fix a bug that could cause a user to be invited to a server notices (aka System Alerts) room without any notice being sent.

View File

@@ -1 +0,0 @@
Fix some worker-mode replication handling not being correctly recorded in CPU usage stats.

View File

@@ -1 +0,0 @@
Remove some extraneous debugging log lines.

View File

@@ -1 +0,0 @@
Add typing information to federation server code.

View File

@@ -1 +0,0 @@
Move catchup of replication streams logic to worker.

View File

@@ -1 +0,0 @@
Unblacklist '/upgrade creates a new room' sytest for workers.

View File

@@ -1 +0,0 @@
Require admin privileges to enable room encryption by default. This does not affect existing rooms.

View File

@@ -1 +0,0 @@
Remove redundant checks on `daemonize` from synctl.

View File

@@ -1 +0,0 @@
Update the contributed documentation on managing synapse workers with systemd, and bring it into the core distribution.

View File

@@ -1 +0,0 @@
Improve the support for SSO authentication on the login fallback page.

View File

@@ -1 +0,0 @@
Upgrade jQuery to v3.4.1 on fallback login/registration pages.

View File

@@ -1 +0,0 @@
Change log line that told user to implement onLogin/onRegister fallback js functions to a warning, instead of an info, so it's more visible.

View File

@@ -1 +0,0 @@
Add documentation to the `password_providers` config option. Add known password provider implementations to docs.

View File

@@ -1 +0,0 @@
Do not allow a deactivated user to login via SSO.

View File

@@ -1 +0,0 @@
Convert some of synapse.rest.media to async/await.

View File

@@ -1 +0,0 @@
Correct the parameters of a test fixture. Contributed by Isaiah Singletary.

View File

@@ -1 +0,0 @@
Add documentation to the `password_providers` config option. Add known password provider implementations to docs.

View File

@@ -1 +0,0 @@
Fix --help command-line argument.

View File

@@ -1 +0,0 @@
Modify suggested nginx reverse proxy configuration to match Synapse's default file upload size. Contributed by @ProCycleDev.

View File

@@ -1 +0,0 @@
Fix a bug that prevented cross-signing with users on worker-mode synapses.

View File

@@ -1 +0,0 @@
Do not allow a deactivated user to login via SSO.

View File

@@ -1 +0,0 @@
Fix room publish permissions not being checked on room creation.

View File

@@ -1 +0,0 @@
Convert auth handler to async/await.

View File

@@ -1 +0,0 @@
Add a config option for specifying the value of the Accept-Language HTTP header when generating URL previews.

View File

@@ -1 +0,0 @@
Fix a sql query introduced in Synapse 1.12.0 which could cause large amounts of logging to the postgres slow-query log.

View File

@@ -1 +0,0 @@
Fix a bug with cross-signing devices with remote users when they did not share a room with any user on the local homeserver.

View File

@@ -1 +0,0 @@
Move catchup of replication streams logic to worker.

View File

@@ -1,2 +1,150 @@
The documentation for using systemd to manage synapse workers is now part of
the main synapse distribution. See [docs/systemd-with-workers](../../docs/systemd-with-workers).
# Setup Synapse with Workers and Systemd
This is a setup for managing synapse with systemd including support for
managing workers. It provides a `matrix-synapse`, as well as a
`matrix-synapse-worker@` service for any workers you require. Additionally to
group the required services it sets up a `matrix.target`. You can use this to
automatically start any bot- or bridge-services. More on this in
[Bots and Bridges](#bots-and-bridges).
See the folder [system](system) for any service and target files.
The folder [workers](workers) contains an example configuration for the
`federation_reader` worker. Pay special attention to the name of the
configuration file. In order to work with the `matrix-synapse-worker@.service`
service, it needs to have the exact same name as the worker app.
This setup expects neither the homeserver nor any workers to fork. Forking is
handled by systemd.
## Setup
1. Adjust your matrix configs. Make sure that the worker config files have the
exact same name as the worker app. Compare `matrix-synapse-worker@.service` for
why. You can find an example worker config in the [workers](workers) folder. See
below for relevant settings in the `homeserver.yaml`.
2. Copy the `*.service` and `*.target` files in [system](system) to
`/etc/systemd/system`.
3. `systemctl enable matrix-synapse.service` this adds the homeserver
app to the `matrix.target`
4. *Optional.* `systemctl enable
matrix-synapse-worker@federation_reader.service` this adds the federation_reader
app to the `matrix-synapse.service`
5. *Optional.* Repeat step 4 for any additional workers you require.
6. *Optional.* Add any bots or bridges by enabling them.
7. Start all matrix related services via `systemctl start matrix.target`
8. *Optional.* Enable autostart of all matrix related services on system boot
via `systemctl enable matrix.target`
## Usage
After you have setup you can use the following commands to manage your synapse
installation:
```
# Start matrix-synapse, all workers and any enabled bots or bridges.
systemctl start matrix.target
# Restart matrix-synapse and all workers (not necessarily restarting bots
# or bridges, see "Bots and Bridges")
systemctl restart matrix-synapse.service
# Stop matrix-synapse and all workers (not necessarily restarting bots
# or bridges, see "Bots and Bridges")
systemctl stop matrix-synapse.service
# Restart a specific worker (i. e. federation_reader), the homeserver is
# unaffected by this.
systemctl restart matrix-synapse-worker@federation_reader.service
# Add a new worker (assuming all configs are setup already)
systemctl enable matrix-synapse-worker@federation_writer.service
systemctl restart matrix-synapse.service
```
## The Configs
Make sure the `worker_app` is set in the `homeserver.yaml` and it does not fork.
```
worker_app: synapse.app.homeserver
daemonize: false
```
None of the workers should fork, as forking is handled by systemd. Hence make
sure this is present in all worker config files.
```
worker_daemonize: false
```
The config files of all workers are expected to be located in
`/etc/matrix-synapse/workers`. If you want to use a different location you have
to edit the provided `*.service` files accordingly.
## Bots and Bridges
Most bots and bridges do not care if the homeserver goes down or is restarted.
Depending on the implementation this may crash them though. So look up the docs
or ask the community of the specific bridge or bot you want to run to make sure
you choose the correct setup.
Whichever configuration you choose, after the setup the following will enable
automatically starting (and potentially restarting) your bot/bridge with the
`matrix.target`.
```
systemctl enable <yourBotOrBridgeName>.service
```
**Note** that from an inactive synapse the bots/bridges will only be started with
synapse if you start the `matrix.target`, not if you start the
`matrix-synapse.service`. This is on purpose. Think of `matrix-synapse.service`
as *just* synapse, but `matrix.target` being anything matrix related, including
synapse and any and all enabled bots and bridges.
### Start with synapse but ignore synapse going down
If the bridge can handle shutdowns of the homeserver you'll want to install the
service in the `matrix.target` and optionally add a
`After=matrix-synapse.service` dependency to have the bot/bridge start after
synapse on starting everything.
In this case the service file should look like this.
```
[Unit]
# ...
# Optional, this will only ensure that if you start everything, synapse will
# be started before the bot/bridge will be started.
After=matrix-synapse.service
[Service]
# ...
[Install]
WantedBy=matrix.target
```
### Stop/restart when synapse stops/restarts
If the bridge can't handle shutdowns of the homeserver you'll still want to
install the service in the `matrix.target` but also have to specify the
`After=matrix-synapse.service` *and* `BindsTo=matrix-synapse.service`
dependencies to have the bot/bridge stop/restart with synapse.
In this case the service file should look like this.
```
[Unit]
# ...
# Mandatory
After=matrix-synapse.service
BindsTo=matrix-synapse.service
[Service]
# ...
[Install]
WantedBy=matrix.target
```

View File

@@ -0,0 +1,19 @@
[Unit]
Description=Synapse Matrix Worker
After=matrix-synapse.service
BindsTo=matrix-synapse.service
[Service]
Type=notify
NotifyAccess=main
User=matrix-synapse
WorkingDirectory=/var/lib/matrix-synapse
EnvironmentFile=/etc/default/matrix-synapse
ExecStart=/opt/venvs/matrix-synapse/bin/python -m synapse.app.%i --config-path=/etc/matrix-synapse/homeserver.yaml --config-path=/etc/matrix-synapse/conf.d/ --config-path=/etc/matrix-synapse/workers/%i.yaml
ExecReload=/bin/kill -HUP $MAINPID
Restart=always
RestartSec=3
SyslogIdentifier=matrix-synapse-%i
[Install]
WantedBy=matrix-synapse.service

View File

@@ -1,8 +1,5 @@
[Unit]
Description=Synapse master
# This service should be restarted when the synapse target is restarted.
PartOf=matrix-synapse.target
Description=Synapse Matrix Homeserver
[Service]
Type=notify
@@ -18,4 +15,4 @@ RestartSec=3
SyslogIdentifier=matrix-synapse
[Install]
WantedBy=matrix-synapse.target
WantedBy=matrix.target

View File

@@ -0,0 +1,7 @@
[Unit]
Description=Contains matrix services like synapse, bridges and bots
After=network.target
AllowIsolate=no
[Install]
WantedBy=multi-user.target

View File

@@ -10,4 +10,5 @@ worker_listeners:
resources:
- names: [federation]
worker_daemonize: false
worker_log_config: /etc/matrix-synapse/federation-reader-log.yaml

6
debian/changelog vendored
View File

@@ -1,8 +1,8 @@
matrix-synapse-py3 (1.12.3ubuntu1) UNRELEASED; urgency=medium
matrix-synapse-py3 (1.12.4) stable; urgency=medium
* Add information about .well-known files to Debian installation scripts.
* New synapse release 1.12.4.
-- Patrick Cloke <patrickc@matrix.org> Mon, 06 Apr 2020 10:10:38 -0400
-- Synapse Packaging team <packages@matrix.org> Thu, 23 Apr 2020 10:58:14 -0400
matrix-synapse-py3 (1.12.3) stable; urgency=medium

View File

@@ -1,14 +1,14 @@
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the matrix-synapse-py3 package.
# This file is distributed under the same license as the matrix-synapse package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: matrix-synapse-py3\n"
"Report-Msgid-Bugs-To: matrix-synapse-py3@packages.debian.org\n"
"POT-Creation-Date: 2020-04-06 16:39-0400\n"
"Project-Id-Version: matrix-synapse\n"
"Report-Msgid-Bugs-To: matrix-synapse@packages.debian.org\n"
"POT-Creation-Date: 2017-02-21 07:51+0000\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
@@ -28,10 +28,7 @@ msgstr ""
#: ../templates:1001
msgid ""
"The name that this homeserver will appear as, to clients and other servers "
"via federation. This is normally the public hostname of the server running "
"synapse, but can be different if you set up delegation. Please refer to the "
"delegation documentation in this case: https://github.com/matrix-org/synapse/"
"blob/master/docs/delegate.md."
"via federation. This name should match the SRV record published in DNS."
msgstr ""
#. Type: boolean

6
debian/templates vendored
View File

@@ -2,10 +2,8 @@ Template: matrix-synapse/server-name
Type: string
_Description: Name of the server:
The name that this homeserver will appear as, to clients and other
servers via federation. This is normally the public hostname of the
server running synapse, but can be different if you set up delegation.
Please refer to the delegation documentation in this case:
https://github.com/matrix-org/synapse/blob/master/docs/delegate.md.
servers via federation. This name should match the SRV record
published in DNS.
Template: matrix-synapse/report-stats
Type: boolean

View File

@@ -1,34 +0,0 @@
# Edit Room Membership API
This API allows an administrator to join an user account with a given `user_id`
to a room with a given `room_id_or_alias`. You can only modify the membership of
local users. The server administrator must be in the room and have permission to
invite users.
## Parameters
The following parameters are available:
* `user_id` - Fully qualified user: for example, `@user:server.com`.
* `room_id_or_alias` - The room identifier or alias to join: for example,
`!636q39766251:server.com`.
## Usage
```
POST /_synapse/admin/v1/join/<room_id_or_alias>
{
"user_id": "@user:server.com"
}
```
Including an `access_token` of a server admin.
Response:
```
{
"room_id": "!636q39766251:server.com"
}
```

View File

@@ -1,64 +0,0 @@
# How to test CAS as a developer without a server
The [django-mama-cas](https://github.com/jbittel/django-mama-cas) project is an
easy to run CAS implementation built on top of Django.
## Prerequisites
1. Create a new virtualenv: `python3 -m venv <your virtualenv>`
2. Activate your virtualenv: `source /path/to/your/virtualenv/bin/activate`
3. Install Django and django-mama-cas:
```
python -m pip install "django<3" "django-mama-cas==2.4.0"
```
4. Create a Django project in the current directory:
```
django-admin startproject cas_test .
```
5. Follow the [install directions](https://django-mama-cas.readthedocs.io/en/latest/installation.html#configuring) for django-mama-cas
6. Setup the SQLite database: `python manage.py migrate`
7. Create a user:
```
python manage.py createsuperuser
```
1. Use whatever you want as the username and password.
2. Leave the other fields blank.
8. Use the built-in Django test server to serve the CAS endpoints on port 8000:
```
python manage.py runserver
```
You should now have a Django project configured to serve CAS authentication with
a single user created.
## Configure Synapse (and Riot) to use CAS
1. Modify your `homeserver.yaml` to enable CAS and point it to your locally
running Django test server:
```yaml
cas_config:
enabled: true
server_url: "http://localhost:8000"
service_url: "http://localhost:8081"
#displayname_attribute: name
#required_attributes:
# name: value
```
2. Restart Synapse.
Note that the above configuration assumes the homeserver is running on port 8081
and that the CAS server is on port 8000, both on localhost.
## Testing the configuration
Then in Riot:
1. Visit the login page with a Riot pointing at your homeserver.
2. Click the Single Sign-On button.
3. Login using the credentials created with `createsuperuser`.
4. You should be logged in.
If you want to repeat this process you'll need to manually logout first:
1. http://localhost:8000/admin/
2. Click "logout" in the top right.

View File

@@ -18,13 +18,9 @@ To make Synapse (and therefore Riot) use it:
metadata:
local: ["samling.xml"]
```
5. Ensure that your `homeserver.yaml` has a setting for `public_baseurl`:
```yaml
public_baseurl: http://localhost:8080/
```
6. Run `apt-get install xmlsec1` and `pip install --upgrade --force 'pysaml2>=4.5.0'` to ensure
5. Run `apt-get install xmlsec1` and `pip install --upgrade --force 'pysaml2>=4.5.0'` to ensure
the dependencies are installed and ready to go.
7. Restart Synapse.
6. Restart Synapse.
Then in Riot:

View File

@@ -29,13 +29,14 @@ from synapse.logging import context # omitted from future snippets
def handle_request(request_id):
request_context = context.LoggingContext()
calling_context = context.set_current_context(request_context)
calling_context = context.LoggingContext.current_context()
context.LoggingContext.set_current_context(request_context)
try:
request_context.request = request_id
do_request_handling()
logger.debug("finished")
finally:
context.set_current_context(calling_context)
context.LoggingContext.set_current_context(calling_context)
def do_request_handling():
logger.debug("phew") # this will be logged against request_id

Some files were not shown because too many files have changed in this diff Show More