1
0
Commit Graph

989 Commits

Author SHA1 Message Date
Will Hunt
7d33ba70df Merge remote-tracking branch 'origin/release-v1.36' into hs/hacked-together-event-cache 2021-06-16 11:32:27 +01:00
Dan Callahan
7dc14730d9 Name release branches just after major.minor (#10013)
With the prior format, 1.33.0 / 1.33.1 / 1.33.2 got separate branches:

    release-v1.33.0
    release-v1.33.1
    release-v1.33.2

Under the new model, all three would share a common branch:

    release-v1.33

As before, RCs and actual releases exist as tags on these branches.

This better reflects our support model, e.g., that the "1.33" series had
a formal release followed by two patches / updates.

Signed-off-by: Dan Callahan <danc@element.io>
2021-06-08 11:44:50 +01:00
Rohan Sharma
beb251e3ee Make link in docs use HTTPS (#10130)
Fixes #10121

Signed-off-by: Rohan Sharma <rhnsharma5113@gmail.com>
2021-06-07 16:35:02 +01:00
Andrew Morgan
fd9856e4a9 Compile and render Synapse's docs into a browsable, mobile-friendly and searchable website (#10086) 2021-06-03 17:20:40 +01:00
Andrew Morgan
73636cab69 Convert admin api docs to markdown (#10089)
So that they render nicely in mdbook (see #10086), and so that we no longer have a mix of structured text languages in our documentation (excluding files outside of `docs/`).
2021-06-03 14:06:03 +01:00
Dirk Klimpel
0284d2a297 Add new admin APIs to remove media by media ID from quarantine. (#10044)
Related to: #6681, #5956, #10040

Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-06-02 18:50:35 +01:00
Erik Johnston
5eed6348ce Move some more endpoints off master (#10084) 2021-05-27 22:45:43 +01:00
Callum Brown
8fb9af570f Make reason and score optional for report_event (#10077)
Implements MSC2414: https://github.com/matrix-org/matrix-doc/pull/2414
See #8551 

Signed-off-by: Callum Brown <callum@calcuode.com>
2021-05-27 18:42:23 +01:00
Patrick Cloke
f42e4c4eb9 Remove the experimental spaces enabled flag. (#10063)
In lieu of just always enabling the unstable spaces endpoint and
unstable room version.
2021-05-26 14:35:16 -04:00
Dirk Klimpel
65e6c64d83 Add an admin API for unprotecting local media from quarantine (#10040)
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-05-26 11:19:47 +01:00
Aaron Raimist
3e1beb75e6 Update CAPTCHA documentation to mention turning off verify origin feature (#10046)
* Update CAPTCHA documentation to mention turning off verify origin

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2021-05-26 10:55:30 +01:00
Jerin J Titus
057ce7b754 Remove tls_fingerprints option (#9280)
Signed-off-by: Jerin J Titus <72017981+jerinjtitus@users.noreply.github.com>
2021-05-24 17:43:30 +01:00
Patrick Cloke
316f89e87f Enable experimental spaces by default. (#10011)
The previous spaces_enabled flag now defaults to true and
is exposed in the sample config.
2021-05-24 08:57:14 -04:00
Dirk Klimpel
387c297489 Add missing entry to the table of contents of room admin API (#10043) 2021-05-24 07:37:30 -04:00
Andrew Morgan
d983ced596 Merge branch 'master' of github.com:matrix-org/synapse into develop 2021-05-20 14:39:46 +01:00
Javier Junquera Sánchez
141b073c7b Update user_directory.md (#10016)
Signed-off-by: Javier Junquera Sánchez <javier@junquera.io>
2021-05-20 14:24:19 +01:00
Savyasachee Jha
5bba1b4905 Hardened systemd unit files (#9803)
Signed-off-by: Savyasachee Jha savya.jha@hawkradius.com
2021-05-19 11:44:16 +01:00
Andrew Morgan
4d6e5a5e99 Use a database table to hold the users that should have full presence sent to them, instead of something in-memory (#9823) 2021-05-18 14:13:45 +01:00
Richard van der Hoff
6660912226 Update postgres docs (#9989) 2021-05-14 13:14:48 +01:00
Richard van der Hoff
c14f99be46 Support enabling opentracing by user (#9978)
Add a config option which allows enabling opentracing by user id, eg for
debugging requests made by a test user.
2021-05-14 10:51:08 +01:00
Andrew Morgan
976216959b Update minimum supported version in postgres.md (#9988) 2021-05-14 09:21:00 +01:00
Patrick Cloke
d19bccdbec Update SSO mapping providers documentation about unique IDs. (#9980) 2021-05-13 14:37:20 -04:00
Will Hunt
dacc395dca Merge remote-tracking branch 'origin/develop' into hs/hacked-together-event-cache 2021-05-11 22:28:35 +01:00
Patrick Cloke
28c6841102 Send the m.room.create stripped event with invites (support MSC1772). (#9966)
MSC1772 specifies the m.room.create event should be sent as part
of the invite_state. This was done optionally behind an experimental
flag, but is now done by default due to MSC1772 being approved.
2021-05-11 10:58:58 -04:00
Aaron Raimist
dc6366a9bd Add config option to hide device names over federation (#9945)
Now that cross signing exists there is much less of a need for other people to look at devices and verify them individually. This PR adds a config option to allow you to prevent device display names from being shared with other servers.

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2021-05-11 14:03:23 +01:00
Erik Johnston
1fb9a2d0bf Limit how often GC happens by time. (#9902)
Synapse can be quite memory intensive, and unless care is taken to tune
the GC thresholds it can end up thrashing, causing noticable performance
problems for large servers. We fix this by limiting how often we GC a
given generation, regardless of current counts/thresholds.

This does not help with the reverse problem where the thresholds are set
too high, but that should only happen in situations where they've been
manually configured.

Adds a `gc_min_seconds_between` config option to override the defaults.

Fixes #9890.
2021-05-05 16:53:45 +01:00
Will Hunt
89ad2f60ca Merge remote-tracking branch 'origin/develop' into hs/hacked-together-event-cache 2021-05-05 14:27:02 +01:00
Andrew Morgan
b85821aca2 Add port parameter to the sample config for psycopg2 args (#9911)
Adds the `port` option with the default value to the sample config file.
2021-05-04 13:28:59 +01:00
ThibF
0085dc5abc Delete room endpoint (#9889)
Support the delete of a room through DELETE request and mark
previous request as deprecated through documentation.

Signed-off-by: Thibault Ferrante <thibault.ferrante@pm.me>
2021-04-29 10:31:45 +01:00
Will Hunt
d77072ff6e Merge remote-tracking branch 'origin/develop' into hs/hacked-together-event-cache 2021-04-21 14:54:42 +01:00
Patrick Cloke
683d6f75af Rename handler and config modules which end in handler/config. (#9816) 2021-04-20 14:55:20 -04:00
Will Hunt
82fcd3cce8 Merge remote-tracking branch 'origin/develop' into hs/hacked-together-event-cache 2021-04-20 00:34:28 +01:00
Andrew Morgan
71f0623de9 Port "Allow users to click account renewal links multiple times without hitting an 'Invalid Token' page #74" from synapse-dinsic (#9832)
This attempts to be a direct port of https://github.com/matrix-org/synapse-dinsic/pull/74 to mainline. There was some fiddling required to deal with the changes that have been made to mainline since (mainly dealing with the split of `RegistrationWorkerStore` from `RegistrationStore`, and the changes made to `self.make_request` in test code).
2021-04-19 19:16:34 +01:00
Will Hunt
4be9d0918e Merge remote-tracking branch 'origin/develop' into hs/hacked-together-event-cache 2021-04-13 15:03:34 +01:00
Dirk Klimpel
1fc97ee876 Add an admin API to manage ratelimit for a specific user (#9648) 2021-04-13 10:26:37 +01:00
Will Hunt
5f631512b5 Merge remote-tracking branch 'origin/develop' into hs/hacked-together-event-cache 2021-04-09 14:39:26 +01:00
Andrew Morgan
04819239ba Add a Synapse Module for configuring presence update routing (#9491)
At the moment, if you'd like to share presence between local or remote users, those users must be sharing a room together. This isn't always the most convenient or useful situation though.

This PR adds a module to Synapse that will allow deployments to set up extra logic on where presence updates should be routed. The module must implement two methods, `get_users_for_states` and `get_interested_users`. These methods are given presence updates or user IDs and must return information that Synapse will use to grant passing presence updates around.

A method is additionally added to `ModuleApi` which allows triggering a set of users to receive the current, online presence information for all users they are considered interested in. This is the equivalent of that user receiving presence information during an initial sync. 

The goal of this module is to be fairly generic and useful for a variety of applications, with hard requirements being:

* Sending state for a specific set or all known users to a defined set of local and remote users.
* The ability to trigger an initial sync for specific users, so they receive all current state.
2021-04-06 14:38:30 +01:00
Erik Johnston
5688a74cf3 Merge branch 'master' into develop 2021-04-06 13:29:29 +01:00
Erik Johnston
a888cbdd31 Add deprecation policy doc (#9723)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-04-06 13:07:36 +01:00
Dirk Klimpel
bb0fe02a52 Add order_by to list user admin API (#9691) 2021-04-01 11:28:53 +01:00
Denis Kasak
5ff8eb97c6 Make sample config allowed_local_3pids regex stricter. (#9719)
The regex should be terminated so that subdomain matches of another
domain are not accepted. Just ensuring that someone doesn't shoot
themselves in the foot by copying our example.

Signed-off-by: Denis Kasak <dkasak@termina.org.uk>
2021-03-31 12:27:20 +00:00
Richard van der Hoff
f02663c4dd Replace room_invite_state_types with room_prejoin_state (#9700)
`room_invite_state_types` was inconvenient as a configuration setting, because
anyone that ever set it would not receive any new types that were added to the
defaults. Here, we deprecate the old setting, and replace it with a couple of
new settings under `room_prejoin_state`.
2021-03-30 12:12:44 +01:00
Richard van der Hoff
4bbd535450 Update the OIDC sample config (#9695)
I've reiterated the advice about using `oidc` to migrate, since I've seen a few
people caught by this.

I've also removed a couple of the examples as they are duplicating the OIDC
documentation, and I think they might be leading people astray.
2021-03-29 15:40:11 +01:00
Paul Tötterman
d7d4232a2d Preserve host in example apache config (#9696)
Fixes redirect loop

Signed-off-by: Paul Tötterman <paul.totterman@iki.fi>
2021-03-26 10:38:31 +00:00
Ben Banfield-Zanin
4655d2221e docs: fallback/web endpoint does not appear to be mounted on workers (#9679) 2021-03-24 11:43:04 +00:00
Erik Johnston
e550ab17ad Increase default join burst ratelimiting (#9674)
It's legitimate behaviour to try and join a bunch of rooms at once.
2021-03-23 14:52:20 +00:00
Will Hunt
9009d29b81 Merge remote-tracking branch 'origin/develop' into hs/hacked-together-event-cache 2021-03-22 13:40:27 +00:00
Hubbe
dd5e5dc1d6 Add SSO attribute requirements for OIDC providers (#9609)
Allows limiting who can login using OIDC via the claims
made from the IdP.
2021-03-16 11:46:07 -04:00
Will Hunt
45b1c58898 Merge tag 'v1.30.0rc1' into hs/hacked-together-event-cache
Synapse 1.30.0rc1 (2021-03-16)
==============================

Note that this release deprecates the ability for appservices to
call `POST /_matrix/client/r0/register`  without the body parameter `type`. Appservice
developers should use a `type` value of `m.login.application_service` as
per [the spec](https://matrix.org/docs/spec/application_service/r0.1.2#server-admin-style-permissions).
In future releases, calling this endpoint with an access token - but without a `m.login.application_service`
type - will fail.

Features
--------

- Add prometheus metrics for number of users successfully registering and logging in. ([\#9510](https://github.com/matrix-org/synapse/issues/9510), [\#9511](https://github.com/matrix-org/synapse/issues/9511), [\#9573](https://github.com/matrix-org/synapse/issues/9573))
- Add `synapse_federation_last_sent_pdu_time` and `synapse_federation_last_received_pdu_time` prometheus metrics, which monitor federation delays by reporting the timestamps of messages sent and received to a set of remote servers. ([\#9540](https://github.com/matrix-org/synapse/issues/9540))
- Add support for generating JSON Web Tokens dynamically for use as OIDC client secrets. ([\#9549](https://github.com/matrix-org/synapse/issues/9549))
- Optimise handling of incomplete room history for incoming federation. ([\#9601](https://github.com/matrix-org/synapse/issues/9601))
- Finalise support for allowing clients to pick an SSO Identity Provider ([MSC2858](https://github.com/matrix-org/matrix-doc/pull/2858)). ([\#9617](https://github.com/matrix-org/synapse/issues/9617))
- Tell spam checker modules about the SSO IdP a user registered through if one was used. ([\#9626](https://github.com/matrix-org/synapse/issues/9626))

Bugfixes
--------

- Fix long-standing bug when generating thumbnails for some images with transparency: `TypeError: cannot unpack non-iterable int object`. ([\#9473](https://github.com/matrix-org/synapse/issues/9473))
- Purge chain cover indexes for events that were purged prior to Synapse v1.29.0. ([\#9542](https://github.com/matrix-org/synapse/issues/9542), [\#9583](https://github.com/matrix-org/synapse/issues/9583))
- Fix bug where federation requests were not correctly retried on 5xx responses. ([\#9567](https://github.com/matrix-org/synapse/issues/9567))
- Fix re-activating an account via the admin API when local passwords are disabled. ([\#9587](https://github.com/matrix-org/synapse/issues/9587))
- Fix a bug introduced in Synapse 1.20 which caused incoming federation transactions to stack up, causing slow recovery from outages. ([\#9597](https://github.com/matrix-org/synapse/issues/9597))
- Fix a bug introduced in v1.28.0 where the OpenID Connect callback endpoint could error with a `MacaroonInitException`. ([\#9620](https://github.com/matrix-org/synapse/issues/9620))
- Fix Internal Server Error on `GET /_synapse/client/saml2/authn_response` request. ([\#9623](https://github.com/matrix-org/synapse/issues/9623))

Updates to the Docker image
---------------------------

- Use jemalloc if available in docker. ([\#8553](https://github.com/matrix-org/synapse/issues/8553))

Improved Documentation
----------------------

- Add relayd entry to reverse proxy example configurations. ([\#9508](https://github.com/matrix-org/synapse/issues/9508))
- Improve the SAML2 upgrade notes for 1.27.0. ([\#9550](https://github.com/matrix-org/synapse/issues/9550))
- Link to the "List user's media" admin API from the media admin API docs. ([\#9571](https://github.com/matrix-org/synapse/issues/9571))
- Clarify the spam checker modules documentation example to mention that `parse_config` is a required method. ([\#9580](https://github.com/matrix-org/synapse/issues/9580))
- Clarify the sample configuration for `stats` settings. ([\#9604](https://github.com/matrix-org/synapse/issues/9604))

Deprecations and Removals
-------------------------

- The `synapse_federation_last_sent_pdu_age` and `synapse_federation_last_received_pdu_age` prometheus metrics have been removed. They are replaced by `synapse_federation_last_sent_pdu_time` and `synapse_federation_last_received_pdu_time`. ([\#9540](https://github.com/matrix-org/synapse/issues/9540))
- Registering an Application Service user without using the `m.login.application_service` login type will be unsupported in an upcoming Synapse release. ([\#9559](https://github.com/matrix-org/synapse/issues/9559))

Internal Changes
----------------

- Add tests to ResponseCache. ([\#9458](https://github.com/matrix-org/synapse/issues/9458))
- Add type hints to purge room and server notice admin API. ([\#9520](https://github.com/matrix-org/synapse/issues/9520))
- Add extra logging to ObservableDeferred when callbacks throw exceptions. ([\#9523](https://github.com/matrix-org/synapse/issues/9523))
- Fix incorrect type hints. ([\#9528](https://github.com/matrix-org/synapse/issues/9528), [\#9543](https://github.com/matrix-org/synapse/issues/9543), [\#9591](https://github.com/matrix-org/synapse/issues/9591), [\#9608](https://github.com/matrix-org/synapse/issues/9608), [\#9618](https://github.com/matrix-org/synapse/issues/9618))
- Add an additional test for purging a room. ([\#9541](https://github.com/matrix-org/synapse/issues/9541))
- Add a `.git-blame-ignore-revs` file with the hashes of auto-formatting. ([\#9560](https://github.com/matrix-org/synapse/issues/9560))
- Increase the threshold before which outbound federation to a server goes into "catch up" mode, which is expensive for the remote server to handle. ([\#9561](https://github.com/matrix-org/synapse/issues/9561))
- Fix spurious errors reported by the `config-lint.sh` script. ([\#9562](https://github.com/matrix-org/synapse/issues/9562))
- Fix type hints and tests for BlacklistingAgentWrapper and BlacklistingReactorWrapper. ([\#9563](https://github.com/matrix-org/synapse/issues/9563))
- Do not have mypy ignore type hints from unpaddedbase64. ([\#9568](https://github.com/matrix-org/synapse/issues/9568))
- Improve efficiency of calculating the auth chain in large rooms. ([\#9576](https://github.com/matrix-org/synapse/issues/9576))
- Convert `synapse.types.Requester` to an `attrs` class. ([\#9586](https://github.com/matrix-org/synapse/issues/9586))
- Add logging for redis connection setup. ([\#9590](https://github.com/matrix-org/synapse/issues/9590))
- Improve logging when processing incoming transactions. ([\#9596](https://github.com/matrix-org/synapse/issues/9596))
- Remove unused `stats.retention` setting, and emit a warning if stats are disabled. ([\#9604](https://github.com/matrix-org/synapse/issues/9604))
- Prevent attempting to bundle aggregations for state events in /context APIs. ([\#9619](https://github.com/matrix-org/synapse/issues/9619))
2021-03-16 15:40:21 +00:00
Andrew Morgan
847ecdd8fa Pass SSO IdP information to spam checker's registration function (#9626)
Fixes https://github.com/matrix-org/synapse/issues/9572

When a SSO user logs in for the first time, we create a local Matrix user for them. This goes through the register_user flow, which ends up triggering the spam checker. Spam checker modules don't currently have any way to differentiate between a user trying to sign up initially, versus an SSO user (whom has presumably already been approved elsewhere) trying to log in for the first time.

This PR passes `auth_provider_id` as an argument to the `check_registration_for_spam` function. This argument will contain an ID of an SSO provider (`"saml"`, `"cas"`, etc.) if one was used, else `None`.
2021-03-16 12:41:41 +00:00