1
0
Commit Graph

18848 Commits

Author SHA1 Message Date
Patrick Cloke
6d687ebba1 Convert the appservice handler to async/await. (#7775) 2020-07-06 07:40:35 -04:00
reivilibre
57feeab364 Don't ignore set_tweak actions with no explicit value. (#7766)
* Fix spec compliance; tweaks without values are valid

(default to True, which is only concretely specified for
`highlight`, but it seems only reasonable to generalise)

* Changelog for 7766.

* Add documentation to `tweaks_for_actions`

May as well tidy up when I'm here.

* Add a test for `tweaks_for_actions`
2020-07-06 11:43:41 +01:00
Oliver Kurz
4e118742ca Allow to use higher versions of prometheus_client (#7780)
Fixes https://github.com/matrix-org/synapse/issues/7641

The package was pinned to <0.8.0 without an obvious reasoning with
7ad1d7635
in https://github.com/matrix-org/synapse/pull/5636
while the version selection looks to just try to exclude an arbitrary
next minor version number that might introduce API breaking changes.
Selecting the next minor number might be a good conservative selection.

Downstream distributions already reported success patching out the version
requirements.

This also fixes the integration of upgraded packages into openSUSE packages,
e.g. for openSUSE Tumbleweed which already ships prometheus_client >= 0.8 .

Signed-off-by: Oliver Kurz <okurz@suse.de>

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-07-06 10:21:41 +01:00
Will Hunt
62b1ce8539 isort 5 compatibility (#7786)
The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.
2020-07-05 16:32:02 +01:00
Erik Johnston
5cdca53aa0 Merge different Resource implementation classes (#7732) 2020-07-03 19:02:19 +01:00
Dirk Klimpel
21a212f8e5 Fix inconsistent handling of upper and lower cases of email addresses. (#7021)
fixes #7016
2020-07-03 14:03:13 +01:00
Alex Kotov
8097659f6e Allow YAML config file to contain None (#7779)
Useful when config file is fully commented

Signed-off-by: Alex Kotov <kotovalexarian@gmail.com>
2020-07-03 13:19:03 +01:00
Patrick Cloke
f3e0f16240 Merge tag 'v1.16.0rc2' into develop
Synapse 1.16.0rc2 (2020-07-02)
==============================

Synapse 1.16.0rc2 includes the security fixes released with Synapse 1.15.2.
Please see [below](https://github.com/matrix-org/synapse/blob/master/CHANGES.md#synapse-1152-2020-07-02) for more details.

Improved Documentation
----------------------

- Update postgres image in example `docker-compose.yaml` to tag `12-alpine`. ([\#7696](https://github.com/matrix-org/synapse/issues/7696))

Internal Changes
----------------

- Add some metrics for inbound and outbound federation latencies: `synapse_federation_server_pdu_process_time` and `synapse_event_processing_lag_by_event`. ([\#7771](https://github.com/matrix-org/synapse/issues/7771))
2020-07-02 11:25:56 -04:00
Patrick Cloke
6f238a7074 Fix a typo. v1.16.0rc2 2020-07-02 11:14:28 -04:00
Patrick Cloke
1a76cdf8d4 Move 1.15.2 after 1.16.0rc2. 2020-07-02 11:14:00 -04:00
Patrick Cloke
1319e53251 1.16.0rc2 2020-07-02 11:06:35 -04:00
Patrick Cloke
f2bcc6ecbf Merge branch 'master' into release-v1.16.0 2020-07-02 11:02:42 -04:00
Patrick Cloke
4d978d7db4 Merge branch 'master' into develop 2020-07-02 10:55:41 -04:00
Patrick Cloke
fedb632d0a Merge tag 'v1.15.2'
Synapse 1.15.2 (2020-07-02)
===========================

Due to the two security issues highlighted below, server administrators are
encouraged to update Synapse. We are not aware of these vulnerabilities being
exploited in the wild.

Security advisory
-----------------

* A malicious homeserver could force Synapse to reset the state in a room to a
  small subset of the correct state. This affects all Synapse deployments which
  federate with untrusted servers. ([96e9afe6](96e9afe625))
* HTML pages served via Synapse were vulnerable to clickjacking attacks. This
  predominantly affects homeservers with single-sign-on enabled, but all server
  administrators are encouraged to upgrade. ([ea26e9a9](ea26e9a98b))

  This was reported by [Quentin Gliech](https://sandhose.fr/).
2020-07-02 10:54:29 -04:00
Patrick Cloke
244649b7d5 Remove an extraneous space. v1.15.2 2020-07-02 10:53:14 -04:00
Patrick Cloke
5ae0a4cf76 Add links to the fixes. 2020-07-02 10:45:22 -04:00
Patrick Cloke
1d61a24f42 Fix tense in the release notes. 2020-07-02 10:41:11 -04:00
Patrick Cloke
e8c36e527d 1.15.2 2020-07-02 10:35:59 -04:00
Erik Johnston
96e9afe625 Correctly handle outliers as prev events over federation 2020-07-02 10:00:33 -04:00
Patrick Cloke
ea26e9a98b Ensure that HTML pages served from Synapse include headers to avoid embedding. 2020-07-02 09:58:31 -04:00
Andrew Morgan
21821c076a Add option to autobind user's email on registration (#51)
Adds an option, `bind_new_user_emails_to_sydent`, which uses Sydent's [internal bind api](https://github.com/matrix-org/sydent#internal-bind-and-unbind-api) to automatically bind email addresses of users immediately after they register.

This is quite enterprise-specific, but could be generally useful to multiple organizations. This aims to solve the problem of requiring users to verify their email twice when using the functionality of an identity server in a corporate deployment - where both the homeserver and identity server are controlled. It does with while eliminating the need for the `account_threepid_delegates.email` option, which historically has been a very complicated option to reason about.
2020-07-02 11:01:02 +01:00
reivilibre
e5808c4cfb Hack to add push priority to push notifications (#7765)
* Remove obsolete comment about ancient temporary code

Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>

* Implement hack to set push priority

based on whether the tweaks indicate the event might cause
effects.

* Changelog for 7765

Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>

* Antilint

* Add tests for push priority

Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>

* Update synapse/push/httppusher.py

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>

* Antilint

* Remove needless invites from tests.

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2020-07-01 17:02:31 +01:00
Richard van der Hoff
e866512367 Add early returns to _check_for_soft_fail (#7769)
my editor was complaining about unset variables, so let's add some early
returns to fix that and reduce indentation/cognitive load.
2020-07-01 16:41:19 +01:00
Richard van der Hoff
f01e2ca039 Use symbolic names for replication stream names (#7768)
This makes it much easier to find where streams are referenced.
2020-07-01 16:35:40 +01:00
Richard van der Hoff
a6eae69ffe Type checking for FederationHandler (#7770)
fix a few things to make this pass mypy.
2020-07-01 16:21:02 +01:00
Erik Johnston
1e03513f9a Fix new metric where we used ms instead of seconds (#7771)
Introduced in #7755, not yet released.
2020-07-01 15:23:58 +01:00
Richard van der Hoff
244dbb04f7 Fix incorrect error message when database CTYPE was set incorrectly. (#7760) 2020-07-01 13:56:16 +01:00
Andrew Morgan
8718021469 Pin link in CHANGES.md v1.16.0rc1 2020-07-01 11:47:25 +01:00
Andrew Morgan
70e506f0aa Fixes to CHANGES.md 2020-07-01 11:42:01 +01:00
Andrew Morgan
dc80a0762d 1.16.0rc1 2020-07-01 11:26:58 +01:00
Brendan Abolivier
74d3e177f0 Back out MSC2625 implementation (#7761) 2020-07-01 11:08:25 +01:00
Patrick Cloke
71cccf1593 Additional configuration options for auto-join rooms (#7763) 2020-06-30 15:41:36 -04:00
Erik Johnston
a99658074d Add some metrics for inbound and outbound federation processing times (#7755) 2020-06-30 16:58:06 +01:00
Andrew Morgan
2f6afdd8b4 Explain the purpose of the "tests" conditional dependency requirement (#7751) 2020-06-30 10:11:36 +01:00
Erik Johnston
831b31e563 Add another yield point to state res v2 (#7746) 2020-06-26 10:44:52 +01:00
Andrew Morgan
177b2d0c19 Move flake8 to end. Don't exit script on failure (#7738) 2020-06-25 17:58:55 +01:00
Dagfinn Ilmari Mannsåker
b099ef07d6 Make tox actions work on Debian 10 (#7703)
- Remove the requirement for a specific version of Python
- Move dep comment to a separate line, Tox 3.7.0 like trailing ones

Signed-off-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
2020-06-25 17:45:35 +01:00
Erik Johnston
0e0a2817a2 Yield during large v2 state res. (#7735)
State res v2 across large data sets can be very CPU intensive, and if
all the relevant events are in the cache the algorithm will run from
start to finish within a single reactor tick. This can result in
blocking the reactor tick for several seconds, which can have major
repercussions on other requests.

To fix this we simply add the occaisonal `sleep(0)` during iterations to
yield execution until the next reactor tick. The aim is to only do this
for large data sets so that we don't impact otherwise quick resolutions.=
2020-06-24 18:48:18 +01:00
Andrew Morgan
4200f54ca7 Merge pull request #50 from matrix-org/dinsic-release-v1.15.x
Merge mainline release v1.15.1 into dinsic
2020-06-24 15:43:15 +01:00
Andrew Morgan
b21331359b Changelog 2020-06-24 13:13:59 +01:00
Andrew Morgan
a7d49db74f Merge branch 'release-v1.15.0' of github.com:matrix-org/synapse into dinsic-release-v1.15.x
* 'release-v1.15.0' of github.com:matrix-org/synapse: (55 commits)
  1.15.0
  Fix some attributions
  Update CHANGES.md
  1.15.0rc1
  Revert "1.15.0rc1"
  1.15.0rc1
  Fix bug in account data replication stream. (#7656)
  Convert the registration handler to async/await. (#7649)
  Accept device information at the login fallback endpoint. (#7629)
  Convert user directory handler and related classes to async/await. (#7640)
  Add an option to disable autojoin for guest accounts (#6637)
  Clarifications to the admin api documentation (#7647)
  Update to the stable SSO prefix for UI Auth. (#7630)
  Fix type information on `assert_*_is_admin` methods (#7645)
  Remove some unused constants. (#7644)
  Typo fixes.
  Allow new users to be registered via the admin API even if the monthly active user limit has been reached (#7263)
  Add device management to admin API (#7481)
  Attempt to fix PhoneHomeStatsTestCase.test_performance_100 being flaky. (#7634)
  Support CS API v0.6.0 (#6585)
  ...
2020-06-24 12:07:41 +01:00
Sorunome
6920e58136 add org.matrix.login.jwt so that m.login.jwt can be deprecated (#7675) 2020-06-24 10:23:55 +01:00
Christian Svensson
8bbe87f42d Set Content-Length for Metrics requests (#7730)
HTTP requires the response to contain a Content-Length header unless chunked encoding is being used.
Prometheus metrics endpoint did not set this, causing software such as prometheus-proxy to not be able to scrape synapse for metrics.

Signed-off-by: Christian Svensson <blue@cmd.nu>
2020-06-23 18:06:01 +01:00
Patrick Cloke
24110255cd Sync ignored table names in synapse_port_db to current database schema (#7717) 2020-06-23 07:33:25 -04:00
Patrick Cloke
95e41f368b Allow local media to be marked as safe from being quarantined. (#7718) 2020-06-22 08:04:14 -04:00
Andrew Morgan
7da71b701e Prevent M_USER_IN_USE from being raised by registration methods until after email has been verified (#48)
* Just ignore the `username` parameter on registration as it's not used by DINUM
* Have `/register/available` always return `true`
2020-06-22 12:47:09 +01:00
Patrick Cloke
e060bf4462 Convert directory handler to async/await (#7727) 2020-06-22 07:18:00 -04:00
Andrew Morgan
53949a908f Add a bulk user info endpoint and deprecate the old one (#46)
The current `/user/<user_id>/info` API was useful in that it could be used by any user to lookup whether another user was deactivate or expired. However, it was impractical as it only allowed for a single lookup at once. Clients trying to use this API were met with speed issues as they tried to query this information for all users in a room.

This PR adds an equivalent CS and Federation API that takes a list of user IDs, and returning a mapping from user ID to info dictionary.

Note that the federation in this PR was a bit trickier than in the original #12 as we can no longer use a federation query, as those don't allow for JSON bodies - which we require to pass a list of user IDs. Instead we do the whole thing of adding a method to transport/client and transport/server.

This PR also adds unittests. The earlier PR used Sytest, presumably for testing across federation, but as this is Synapse-specific that felt a little gross. Unit tests for the deprecated endpoint have not been added.
2020-06-19 16:17:13 +01:00
Andrew Morgan
6708163271 Performance improvements to marking expired users as inactive (#47)
This is a performance-related improvement to #13, which queried and hid active *and* already inactive users, one-by-one. This PR updates the code to query only **active**, expired users, all at once, and then mark them as inactive, all at once.
2020-06-19 16:14:37 +01:00
Erik Johnston
91e886d615 Speed up state res v2 across large state differences. (#7725) 2020-06-19 13:56:35 +01:00