1
0

Merge tag 'v1.41.0' into babolivier/dinsic_1.41.0

Synapse 1.41.0 (2021-08-24)
===========================

This release adds support for Debian 12 (Bookworm), but **removes support for Ubuntu 20.10 (Groovy Gorilla)**, which reached End of Life last month.

Note that when using workers the `/_synapse/admin/v1/users/{userId}/media` must now be handled by media workers. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information.

Features
--------

- Enable room capabilities ([MSC3244](https://github.com/matrix-org/matrix-doc/pull/3244)) by default and set room version 8 as the preferred room version when creating restricted rooms. ([\#10571](https://github.com/matrix-org/synapse/issues/10571))

Synapse 1.41.0rc1 (2021-08-18)
==============================

Features
--------

- Add `get_userinfo_by_id` method to ModuleApi. ([\#9581](https://github.com/matrix-org/synapse/issues/9581))
- Initial local support for [MSC3266](https://github.com/matrix-org/synapse/pull/10394), Room Summary over the unstable `/rooms/{roomIdOrAlias}/summary` API. ([\#10394](https://github.com/matrix-org/synapse/issues/10394))
- Experimental support for [MSC3288](https://github.com/matrix-org/matrix-doc/pull/3288), sending `room_type` to the identity server for 3pid invites over the `/store-invite` API. ([\#10435](https://github.com/matrix-org/synapse/issues/10435))
- Add support for sending federation requests through a proxy. Contributed by @Bubu and @dklimpel. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. ([\#10596](https://github.com/matrix-org/synapse/issues/10596)). ([\#10475](https://github.com/matrix-org/synapse/issues/10475))
- Add support for "marker" events which makes historical events discoverable for servers that already have all of the scrollback history (part of [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716)). ([\#10498](https://github.com/matrix-org/synapse/issues/10498))
- Add a configuration setting for the time a `/sync` response is cached for. ([\#10513](https://github.com/matrix-org/synapse/issues/10513))
- The default logging handler for new installations is now `PeriodicallyFlushingMemoryHandler`, a buffered logging handler which periodically flushes itself. ([\#10518](https://github.com/matrix-org/synapse/issues/10518))
- Add support for new redaction rules for historical events specified in [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716). ([\#10538](https://github.com/matrix-org/synapse/issues/10538))
- Add a setting to disable TLS when sending email. ([\#10546](https://github.com/matrix-org/synapse/issues/10546))
- Add pagination to the spaces summary based on updates to [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946). ([\#10549](https://github.com/matrix-org/synapse/issues/10549), [\#10560](https://github.com/matrix-org/synapse/issues/10560), [\#10569](https://github.com/matrix-org/synapse/issues/10569), [\#10574](https://github.com/matrix-org/synapse/issues/10574), [\#10575](https://github.com/matrix-org/synapse/issues/10575), [\#10579](https://github.com/matrix-org/synapse/issues/10579), [\#10583](https://github.com/matrix-org/synapse/issues/10583))
- Admin API to delete several media for a specific user. Contributed by @dklimpel. ([\#10558](https://github.com/matrix-org/synapse/issues/10558), [\#10628](https://github.com/matrix-org/synapse/issues/10628))
- Add support for routing `/createRoom` to workers. ([\#10564](https://github.com/matrix-org/synapse/issues/10564))
- Update the Synapse Grafana dashboard. ([\#10570](https://github.com/matrix-org/synapse/issues/10570))
- Add an admin API (`GET /_synapse/admin/username_available`) to check if a username is available (regardless of registration settings). ([\#10578](https://github.com/matrix-org/synapse/issues/10578))
- Allow editing a user's `external_ids` via the "Edit User" admin API. Contributed by @dklimpel. ([\#10598](https://github.com/matrix-org/synapse/issues/10598))
- The Synapse manhole no longer needs coroutines to be wrapped in `defer.ensureDeferred`. ([\#10602](https://github.com/matrix-org/synapse/issues/10602))
- Add option to allow modules to run periodic tasks on all instances, rather than just the one configured to run background tasks. ([\#10638](https://github.com/matrix-org/synapse/issues/10638))

Bugfixes
--------

- Add some clarification to the sample config file. Contributed by @Kentokamoto. ([\#10129](https://github.com/matrix-org/synapse/issues/10129))
- Fix a long-standing bug where protocols which are not implemented by any appservices were incorrectly returned via `GET /_matrix/client/r0/thirdparty/protocols`. ([\#10532](https://github.com/matrix-org/synapse/issues/10532))
- Fix exceptions in logs when failing to get remote room list. ([\#10541](https://github.com/matrix-org/synapse/issues/10541))
- Fix longstanding bug which caused the user's presence "status message" to be reset when the user went offline. Contributed by @dklimpel. ([\#10550](https://github.com/matrix-org/synapse/issues/10550))
- Allow public rooms to be previewed in the spaces summary APIs from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946). ([\#10580](https://github.com/matrix-org/synapse/issues/10580))
- Fix a bug introduced in v1.37.1 where an error could occur in the asynchronous processing of PDUs when the queue was empty. ([\#10592](https://github.com/matrix-org/synapse/issues/10592))
- Fix errors on /sync when read receipt data is a string. Only affects homeservers with the experimental flag for [MSC2285](https://github.com/matrix-org/matrix-doc/pull/2285) enabled. Contributed by @SimonBrandner. ([\#10606](https://github.com/matrix-org/synapse/issues/10606))
- Additional validation for the spaces summary API to avoid errors like `ValueError: Stop argument for islice() must be None or an integer`. The missing validation has existed since v1.31.0. ([\#10611](https://github.com/matrix-org/synapse/issues/10611))
- Revert behaviour introduced in v1.38.0 that strips `org.matrix.msc2732.device_unused_fallback_key_types` from `/sync` when its value is empty. This field should instead always be present according to [MSC2732](https://github.com/matrix-org/matrix-doc/blob/master/proposals/2732-olm-fallback-keys.md). ([\#10623](https://github.com/matrix-org/synapse/issues/10623))

Improved Documentation
----------------------

- Add documentation for configuring a forward proxy. ([\#10443](https://github.com/matrix-org/synapse/issues/10443))
- Updated the reverse proxy documentation to highlight the homserver configuration that is needed to make Synapse aware that is is intentionally reverse proxied. ([\#10551](https://github.com/matrix-org/synapse/issues/10551))
- Update CONTRIBUTING.md to fix index links and the instructions for SyTest in docker. ([\#10599](https://github.com/matrix-org/synapse/issues/10599))

Deprecations and Removals
-------------------------

- No longer build `.deb` packages for Ubuntu 20.10 Groovy Gorilla, which has now EOLed. ([\#10588](https://github.com/matrix-org/synapse/issues/10588))
- The `template_dir` configuration settings in the `sso`, `account_validity` and `email` sections of the configuration file are now deprecated in favour of the global `templates.custom_template_directory` setting. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. ([\#10596](https://github.com/matrix-org/synapse/issues/10596))

Internal Changes
----------------

- Improve event caching mechanism to avoid having multiple copies of an event in memory at a time. ([\#10119](https://github.com/matrix-org/synapse/issues/10119))
- Reduce errors in PostgreSQL logs due to concurrent serialization errors. ([\#10504](https://github.com/matrix-org/synapse/issues/10504))
- Include room ID in ignored EDU log messages. Contributed by @ilmari. ([\#10507](https://github.com/matrix-org/synapse/issues/10507))
- Add pagination to the spaces summary based on updates to [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946). ([\#10527](https://github.com/matrix-org/synapse/issues/10527), [\#10530](https://github.com/matrix-org/synapse/issues/10530))
- Fix CI to not break when run against branches rather than pull requests. ([\#10529](https://github.com/matrix-org/synapse/issues/10529))
- Mark all events stemming from the [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) `/batch_send` endpoint as historical. ([\#10537](https://github.com/matrix-org/synapse/issues/10537))
- Clean up some of the federation event authentication code for clarity. ([\#10539](https://github.com/matrix-org/synapse/issues/10539), [\#10591](https://github.com/matrix-org/synapse/issues/10591))
- Convert `Transaction` and `Edu` objects to attrs. ([\#10542](https://github.com/matrix-org/synapse/issues/10542))
- Update `/batch_send` endpoint to only return `state_events` created by the `state_events_from_before` passed in. ([\#10552](https://github.com/matrix-org/synapse/issues/10552))
- Update contributing.md to warn against rebasing an open PR. ([\#10563](https://github.com/matrix-org/synapse/issues/10563))
- Remove the unused public rooms replication stream. ([\#10565](https://github.com/matrix-org/synapse/issues/10565))
- Clarify error message when failing to join a restricted room. ([\#10572](https://github.com/matrix-org/synapse/issues/10572))
- Remove references to BuildKite in favour of GitHub Actions. ([\#10573](https://github.com/matrix-org/synapse/issues/10573))
- Move `/batch_send` endpoint defined by [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) to the `/v2_alpha` directory. ([\#10576](https://github.com/matrix-org/synapse/issues/10576))
- Allow multiple custom directories in `read_templates`. ([\#10587](https://github.com/matrix-org/synapse/issues/10587))
- Re-organize the `synapse.federation.transport.server` module to create smaller files. ([\#10590](https://github.com/matrix-org/synapse/issues/10590))
- Flatten the `synapse.rest.client` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\#10600](https://github.com/matrix-org/synapse/issues/10600))
- Build Debian packages for Debian 12 (Bookworm). ([\#10612](https://github.com/matrix-org/synapse/issues/10612))
- Fix up a couple of links to the database schema documentation. ([\#10620](https://github.com/matrix-org/synapse/issues/10620))
- Fix a broken link to the upgrade notes. ([\#10631](https://github.com/matrix-org/synapse/issues/10631))
This commit is contained in:
Brendan Abolivier
2021-09-01 12:18:18 +01:00
239 changed files with 9687 additions and 6034 deletions

View File

@@ -7,6 +7,7 @@
- [Installation](setup/installation.md)
- [Using Postgres](postgres.md)
- [Configuring a Reverse Proxy](reverse_proxy.md)
- [Configuring a Forward/Outbound Proxy](setup/forward_proxy.md)
- [Configuring a Turn Server](turn-howto.md)
- [Delegation](delegate.md)
@@ -20,6 +21,7 @@
- [Homeserver Sample Config File](usage/configuration/homeserver_sample_config.md)
- [Logging Sample Config File](usage/configuration/logging_sample_config.md)
- [Structured Logging](structured_logging.md)
- [Templates](templates.md)
- [User Authentication](usage/configuration/user_authentication/README.md)
- [Single-Sign On]()
- [OpenID Connect](openid.md)

View File

@@ -12,6 +12,7 @@
- [Delete local media](#delete-local-media)
* [Delete a specific local media](#delete-a-specific-local-media)
* [Delete local media by date or size](#delete-local-media-by-date-or-size)
* [Delete media uploaded by a user](#delete-media-uploaded-by-a-user)
- [Purge Remote Media API](#purge-remote-media-api)
# Querying media
@@ -47,7 +48,8 @@ The API returns a JSON body like the following:
## List all media uploaded by a user
Listing all media that has been uploaded by a local user can be achieved through
the use of the [List media of a user](user_admin_api.md#list-media-of-a-user)
the use of the
[List media uploaded by a user](user_admin_api.md#list-media-uploaded-by-a-user)
Admin API.
# Quarantine media
@@ -281,6 +283,11 @@ The following fields are returned in the JSON response body:
* `deleted_media`: an array of strings - List of deleted `media_id`
* `total`: integer - Total number of deleted `media_id`
## Delete media uploaded by a user
You can find details of how to delete multiple media uploaded by a user in
[User Admin API](user_admin_api.md#delete-media-uploaded-by-a-user).
# Purge Remote Media API
The purge remote media API allows server admins to purge old cached remote media.

View File

@@ -81,6 +81,16 @@ with a body of:
"address": "<user_mail_2>"
}
],
"external_ids": [
{
"auth_provider": "<provider1>",
"external_id": "<user_id_provider_1>"
},
{
"auth_provider": "<provider2>",
"external_id": "<user_id_provider_2>"
}
],
"avatar_url": "<avatar_url>",
"admin": false,
"deactivated": false
@@ -90,26 +100,34 @@ with a body of:
To use it, you will need to authenticate by providing an `access_token` for a
server admin: [Admin API](../usage/administration/admin_api)
Returns HTTP status code:
- `201` - When a new user object was created.
- `200` - When a user was modified.
URL parameters:
- `user_id`: fully-qualified user id: for example, `@user:server.com`.
Body parameters:
- `password`, optional. If provided, the user's password is updated and all
- `password` - string, optional. If provided, the user's password is updated and all
devices are logged out.
- `displayname`, optional, defaults to the value of `user_id`.
- `threepids`, optional, allows setting the third-party IDs (email, msisdn)
- `displayname` - string, optional, defaults to the value of `user_id`.
- `threepids` - array, optional, allows setting the third-party IDs (email, msisdn)
- `medium` - string. Kind of third-party ID, either `email` or `msisdn`.
- `address` - string. Value of third-party ID.
belonging to a user.
- `avatar_url`, optional, must be a
- `external_ids` - array, optional. Allow setting the identifier of the external identity
provider for SSO (Single sign-on). Details in
[Sample Configuration File](../usage/configuration/homeserver_sample_config.html)
section `sso` and `oidc_providers`.
- `auth_provider` - string. ID of the external identity provider. Value of `idp_id`
in homeserver configuration.
- `external_id` - string, user ID in the external identity provider.
- `avatar_url` - string, optional, must be a
[MXC URI](https://matrix.org/docs/spec/client_server/r0.6.0#matrix-content-mxc-uris).
- `admin`, optional, defaults to `false`.
- `deactivated`, optional. If unspecified, deactivation state will be left
- `admin` - bool, optional, defaults to `false`.
- `deactivated` - bool, optional. If unspecified, deactivation state will be left
unchanged on existing accounts and set to `false` for new accounts.
A user cannot be erased by deactivating with this API. For details on
deactivating users see [Deactivate Account](#deactivate-account).
@@ -443,8 +461,9 @@ The following fields are returned in the JSON response body:
- `joined_rooms` - An array of `room_id`.
- `total` - Number of rooms.
## User media
## List media of a user
### List media uploaded by a user
Gets a list of all local media that a specific `user_id` has created.
By default, the response is ordered by descending creation date and ascending media ID.
The newest media is on top. You can change the order with parameters
@@ -543,7 +562,6 @@ The following fields are returned in the JSON response body:
- `media` - An array of objects, each containing information about a media.
Media objects contain the following fields:
- `created_ts` - integer - Timestamp when the content was uploaded in ms.
- `last_access_ts` - integer - Timestamp when the content was last accessed in ms.
- `media_id` - string - The id used to refer to the media.
@@ -551,13 +569,58 @@ The following fields are returned in the JSON response body:
- `media_type` - string - The MIME-type of the media.
- `quarantined_by` - string - The user ID that initiated the quarantine request
for this media.
- `safe_from_quarantine` - bool - Status if this media is safe from quarantining.
- `upload_name` - string - The name the media was uploaded with.
- `next_token`: integer - Indication for pagination. See above.
- `total` - integer - Total number of media.
### Delete media uploaded by a user
This API deletes the *local* media from the disk of your own server
that a specific `user_id` has created. This includes any local thumbnails.
This API will not affect media that has been uploaded to external
media repositories (e.g https://github.com/turt2live/matrix-media-repo/).
By default, the API deletes media ordered by descending creation date and ascending media ID.
The newest media is deleted first. You can change the order with parameters
`order_by` and `dir`. If no `limit` is set the API deletes `100` files per request.
The API is:
```
DELETE /_synapse/admin/v1/users/<user_id>/media
```
To use it, you will need to authenticate by providing an `access_token` for a
server admin: [Admin API](../usage/administration/admin_api)
A response body like the following is returned:
```json
{
"deleted_media": [
"abcdefghijklmnopqrstuvwx"
],
"total": 1
}
```
The following fields are returned in the JSON response body:
* `deleted_media`: an array of strings - List of deleted `media_id`
* `total`: integer - Total number of deleted `media_id`
**Note**: There is no `next_token`. This is not useful for deleting media, because
after deleting media the remaining media have a new order.
**Parameters**
This API has the same parameters as
[List media uploaded by a user](#list-media-uploaded-by-a-user).
With the parameters you can for example limit the number of files to delete at once or
delete largest/smallest or newest/oldest files first.
## Login as a user
Get an access token that can be used to authenticate as that user. Useful for
@@ -1013,3 +1076,22 @@ The following parameters should be set in the URL:
- `user_id` - The fully qualified MXID: for example, `@user:server.com`. The user must
be local.
### Check username availability
Checks to see if a username is available, and valid, for the server. See [the client-server
API](https://matrix.org/docs/spec/client_server/r0.6.0#get-matrix-client-r0-register-available)
for more information.
This endpoint will work even if registration is disabled on the server, unlike
`/_matrix/client/r0/register/available`.
The API is:
```
POST /_synapse/admin/v1/username_availabile?username=$localpart
```
The request and response format is the same as the [/_matrix/client/r0/register/available](https://matrix.org/docs/spec/client_server/r0.6.0#get-matrix-client-r0-register-available) API.
To use it, you will need to authenticate by providing an `access_token` for a
server admin: [Admin API](../usage/administration/admin_api)

View File

@@ -67,7 +67,7 @@ This gives a Python REPL in which `hs` gives access to the
`synapse.server.HomeServer` object - which in turn gives access to many other
parts of the process.
Note that any call which returns a coroutine will need to be wrapped in `ensureDeferred`.
Note that, prior to Synapse 1.41, any call which returns a coroutine will need to be wrapped in `ensureDeferred`.
As a simple example, retrieving an event from the database:

View File

@@ -33,6 +33,19 @@ Let's assume that we expect clients to connect to our server at
`https://example.com:8448`. The following sections detail the configuration of
the reverse proxy and the homeserver.
## Homeserver Configuration
The HTTP configuration will need to be updated for Synapse to correctly record
client IP addresses and generate redirect URLs while behind a reverse proxy.
In `homeserver.yaml` set `x_forwarded: true` in the port 8008 section and
consider setting `bind_addresses: ['127.0.0.1']` so that the server only
listens to traffic on localhost. (Do not change `bind_addresses` to `127.0.0.1`
when using a containerized Synapse, as that will prevent it from responding
to proxied traffic.)
## Reverse-proxy configuration examples
**NOTE**: You only need one of these.
@@ -239,16 +252,6 @@ relay "matrix_federation" {
}
```
## Homeserver Configuration
You will also want to set `bind_addresses: ['127.0.0.1']` and
`x_forwarded: true` for port 8008 in `homeserver.yaml` to ensure that
client IP addresses are recorded correctly.
Having done so, you can then use `https://matrix.example.com` (instead
of `https://matrix.example.com:8448`) as the "Custom server" when
connecting to Synapse from a client.
## Health check endpoint

View File

@@ -210,6 +210,8 @@ presence:
#
# This option replaces federation_ip_range_blacklist in Synapse v1.25.0.
#
# Note: The value is ignored when an HTTP proxy is in use
#
#ip_range_blacklist:
# - '127.0.0.0/8'
# - '10.0.0.0/8'
@@ -631,6 +633,19 @@ retention:
#
#next_link_domain_whitelist: ["matrix.org"]
# Templates to use when generating email or HTML page contents.
#
templates:
# Directory in which Synapse will try to find template files to use to generate
# email or HTML page contents.
# If not set, or a file is not found within the template directory, a default
# template from within the Synapse package will be used.
#
# See https://matrix-org.github.io/synapse/latest/templates.html for more
# information about using custom templates.
#
#custom_template_directory: /path/to/custom/templates/
## TLS ##
@@ -779,6 +794,15 @@ caches:
#
#expiry_time: 30m
# Controls how long the results of a /sync request are cached for after
# a successful response is returned. A higher duration can help clients with
# intermittent connections, at the cost of higher memory usage.
#
# By default, this is zero, which means that sync responses are not cached
# at all.
#
#sync_response_cache_duration: 2m
## Database ##
@@ -1061,6 +1085,8 @@ media_store_path: "DATADIR/media_store"
# This must be specified if url_preview_enabled is set. It is recommended that
# you uncomment the following list as a starting point.
#
# Note: The value is ignored when an HTTP proxy is in use
#
#url_preview_ip_range_blacklist:
# - '127.0.0.0/8'
# - '10.0.0.0/8'
@@ -2123,6 +2149,9 @@ cas_config:
# Additional settings to use with single-sign on systems such as OpenID Connect,
# SAML2 and CAS.
#
# Server admins can configure custom templates for pages related to SSO. See
# https://matrix-org.github.io/synapse/latest/templates.html for more information.
#
sso:
# A list of client URLs which are whitelisted so that the user does not
# have to confirm giving access to their account to the URL. Any client
@@ -2155,169 +2184,6 @@ sso:
#
#update_profile_information: true
# Directory in which Synapse will try to find the template files below.
# If not set, or the files named below are not found within the template
# directory, default templates from within the Synapse package will be used.
#
# Synapse will look for the following templates in this directory:
#
# * HTML page to prompt the user to choose an Identity Provider during
# login: 'sso_login_idp_picker.html'.
#
# This is only used if multiple SSO Identity Providers are configured.
#
# When rendering, this template is given the following variables:
# * redirect_url: the URL that the user will be redirected to after
# login.
#
# * server_name: the homeserver's name.
#
# * providers: a list of available Identity Providers. Each element is
# an object with the following attributes:
#
# * idp_id: unique identifier for the IdP
# * idp_name: user-facing name for the IdP
# * idp_icon: if specified in the IdP config, an MXC URI for an icon
# for the IdP
# * idp_brand: if specified in the IdP config, a textual identifier
# for the brand of the IdP
#
# The rendered HTML page should contain a form which submits its results
# back as a GET request, with the following query parameters:
#
# * redirectUrl: the client redirect URI (ie, the `redirect_url` passed
# to the template)
#
# * idp: the 'idp_id' of the chosen IDP.
#
# * HTML page to prompt new users to enter a userid and confirm other
# details: 'sso_auth_account_details.html'. This is only shown if the
# SSO implementation (with any user_mapping_provider) does not return
# a localpart.
#
# When rendering, this template is given the following variables:
#
# * server_name: the homeserver's name.
#
# * idp: details of the SSO Identity Provider that the user logged in
# with: an object with the following attributes:
#
# * idp_id: unique identifier for the IdP
# * idp_name: user-facing name for the IdP
# * idp_icon: if specified in the IdP config, an MXC URI for an icon
# for the IdP
# * idp_brand: if specified in the IdP config, a textual identifier
# for the brand of the IdP
#
# * user_attributes: an object containing details about the user that
# we received from the IdP. May have the following attributes:
#
# * display_name: the user's display_name
# * emails: a list of email addresses
#
# The template should render a form which submits the following fields:
#
# * username: the localpart of the user's chosen user id
#
# * HTML page allowing the user to consent to the server's terms and
# conditions. This is only shown for new users, and only if
# `user_consent.require_at_registration` is set.
#
# When rendering, this template is given the following variables:
#
# * server_name: the homeserver's name.
#
# * user_id: the user's matrix proposed ID.
#
# * user_profile.display_name: the user's proposed display name, if any.
#
# * consent_version: the version of the terms that the user will be
# shown
#
# * terms_url: a link to the page showing the terms.
#
# The template should render a form which submits the following fields:
#
# * accepted_version: the version of the terms accepted by the user
# (ie, 'consent_version' from the input variables).
#
# * HTML page for a confirmation step before redirecting back to the client
# with the login token: 'sso_redirect_confirm.html'.
#
# When rendering, this template is given the following variables:
#
# * redirect_url: the URL the user is about to be redirected to.
#
# * display_url: the same as `redirect_url`, but with the query
# parameters stripped. The intention is to have a
# human-readable URL to show to users, not to use it as
# the final address to redirect to.
#
# * server_name: the homeserver's name.
#
# * new_user: a boolean indicating whether this is the user's first time
# logging in.
#
# * user_id: the user's matrix ID.
#
# * user_profile.avatar_url: an MXC URI for the user's avatar, if any.
# None if the user has not set an avatar.
#
# * user_profile.display_name: the user's display name. None if the user
# has not set a display name.
#
# * HTML page which notifies the user that they are authenticating to confirm
# an operation on their account during the user interactive authentication
# process: 'sso_auth_confirm.html'.
#
# When rendering, this template is given the following variables:
# * redirect_url: the URL the user is about to be redirected to.
#
# * description: the operation which the user is being asked to confirm
#
# * idp: details of the Identity Provider that we will use to confirm
# the user's identity: an object with the following attributes:
#
# * idp_id: unique identifier for the IdP
# * idp_name: user-facing name for the IdP
# * idp_icon: if specified in the IdP config, an MXC URI for an icon
# for the IdP
# * idp_brand: if specified in the IdP config, a textual identifier
# for the brand of the IdP
#
# * HTML page shown after a successful user interactive authentication session:
# 'sso_auth_success.html'.
#
# Note that this page must include the JavaScript which notifies of a successful authentication
# (see https://matrix.org/docs/spec/client_server/r0.6.0#fallback).
#
# This template has no additional variables.
#
# * HTML page shown after a user-interactive authentication session which
# does not map correctly onto the expected user: 'sso_auth_bad_user.html'.
#
# When rendering, this template is given the following variables:
# * server_name: the homeserver's name.
# * user_id_to_verify: the MXID of the user that we are trying to
# validate.
#
# * HTML page shown during single sign-on if a deactivated user (according to Synapse's database)
# attempts to login: 'sso_account_deactivated.html'.
#
# This template has no additional variables.
#
# * HTML page to display to users if something goes wrong during the
# OpenID Connect authentication process: 'sso_error.html'.
#
# When rendering, this template is given two variables:
# * error: the technical name of the error
# * error_description: a human-readable message for the error
#
# You can see the default templates at:
# https://github.com/matrix-org/synapse/tree/master/synapse/res/templates
#
#template_dir: "res/templates"
# JSON web token integration. The following settings can be used to make
# Synapse JSON web tokens for authentication, instead of its internal
@@ -2448,6 +2314,9 @@ ui_auth:
# Configuration for sending emails from Synapse.
#
# Server admins can configure custom templates for email content. See
# https://matrix-org.github.io/synapse/latest/templates.html for more information.
#
email:
# The hostname of the outgoing SMTP server to use. Defaults to 'localhost'.
#
@@ -2470,6 +2339,14 @@ email:
#
#require_transport_security: true
# Uncomment the following to disable TLS for SMTP.
#
# By default, if the server supports TLS, it will be used, and the server
# must present a certificate that is valid for 'smtp_host'. If this option
# is set to false, TLS will not be used.
#
#enable_tls: false
# notif_from defines the "From" address to use when sending emails.
# It must be set if email sending is enabled.
#
@@ -2516,49 +2393,6 @@ email:
#
#invite_client_location: https://app.element.io
# Directory in which Synapse will try to find the template files below.
# If not set, or the files named below are not found within the template
# directory, default templates from within the Synapse package will be used.
#
# Synapse will look for the following templates in this directory:
#
# * The contents of email notifications of missed events: 'notif_mail.html' and
# 'notif_mail.txt'.
#
# * The contents of account expiry notice emails: 'notice_expiry.html' and
# 'notice_expiry.txt'.
#
# * The contents of password reset emails sent by the homeserver:
# 'password_reset.html' and 'password_reset.txt'
#
# * An HTML page that a user will see when they follow the link in the password
# reset email. The user will be asked to confirm the action before their
# password is reset: 'password_reset_confirmation.html'
#
# * HTML pages for success and failure that a user will see when they confirm
# the password reset flow using the page above: 'password_reset_success.html'
# and 'password_reset_failure.html'
#
# * The contents of address verification emails sent during registration:
# 'registration.html' and 'registration.txt'
#
# * HTML pages for success and failure that a user will see when they follow
# the link in an address verification email sent during registration:
# 'registration_success.html' and 'registration_failure.html'
#
# * The contents of address verification emails sent when an address is added
# to a Matrix account: 'add_threepid.html' and 'add_threepid.txt'
#
# * HTML pages for success and failure that a user will see when they follow
# the link in an address verification email sent when an address is added
# to a Matrix account: 'add_threepid_success.html' and
# 'add_threepid_failure.html'
#
# You can see the default templates at:
# https://github.com/matrix-org/synapse/tree/master/synapse/res/templates
#
#template_dir: "res/templates"
# Subjects to use when sending emails from Synapse.
#
# The placeholder '%(app)s' will be replaced with the value of the 'app_name'

View File

@@ -24,18 +24,31 @@ handlers:
backupCount: 3 # Does not include the current log file.
encoding: utf8
# Default to buffering writes to log file for efficiency. This means that
# will be a delay for INFO/DEBUG logs to get written, but WARNING/ERROR
# logs will still be flushed immediately.
# Default to buffering writes to log file for efficiency.
# WARNING/ERROR logs will still be flushed immediately, but there will be a
# delay (of up to `period` seconds, or until the buffer is full with
# `capacity` messages) before INFO/DEBUG logs get written.
buffer:
class: logging.handlers.MemoryHandler
class: synapse.logging.handlers.PeriodicallyFlushingMemoryHandler
target: file
# The capacity is the number of log lines that are buffered before
# being written to disk. Increasing this will lead to better
# The capacity is the maximum number of log lines that are buffered
# before being written to disk. Increasing this will lead to better
# performance, at the expensive of it taking longer for log lines to
# be written to disk.
# This parameter is required.
capacity: 10
flushLevel: 30 # Flush for WARNING logs as well
# Logs with a level at or above the flush level will cause the buffer to
# be flushed immediately.
# Default value: 40 (ERROR)
# Other values: 50 (CRITICAL), 30 (WARNING), 20 (INFO), 10 (DEBUG)
flushLevel: 30 # Flush immediately for WARNING logs and higher
# The period of time, in seconds, between forced flushes.
# Messages will not be delayed for longer than this time.
# Default value: 5 seconds
period: 5
# A handler that writes logs to stderr. Unused by default, but can be used
# instead of "buffer" and "file" in the logger handlers.

View File

@@ -0,0 +1,74 @@
# Using a forward proxy with Synapse
You can use Synapse with a forward or outbound proxy. An example of when
this is necessary is in corporate environments behind a DMZ (demilitarized zone).
Synapse supports routing outbound HTTP(S) requests via a proxy. Only HTTP(S)
proxy is supported, not SOCKS proxy or anything else.
## Configure
The `http_proxy`, `https_proxy`, `no_proxy` environment variables are used to
specify proxy settings. The environment variable is not case sensitive.
- `http_proxy`: Proxy server to use for HTTP requests.
- `https_proxy`: Proxy server to use for HTTPS requests.
- `no_proxy`: Comma-separated list of hosts, IP addresses, or IP ranges in CIDR
format which should not use the proxy. Synapse will directly connect to these hosts.
The `http_proxy` and `https_proxy` environment variables have the form: `[scheme://][<username>:<password>@]<host>[:<port>]`
- Supported schemes are `http://` and `https://`. The default scheme is `http://`
for compatibility reasons; it is recommended to set a scheme. If scheme is set
to `https://` the connection uses TLS between Synapse and the proxy.
**NOTE**: Synapse validates the certificates. If the certificate is not
valid, then the connection is dropped.
- Default port if not given is `1080`.
- Username and password are optional and will be used to authenticate against
the proxy.
**Examples**
- HTTP_PROXY=http://USERNAME:PASSWORD@10.0.1.1:8080/
- HTTPS_PROXY=http://USERNAME:PASSWORD@proxy.example.com:8080/
- NO_PROXY=master.hostname.example.com,10.1.0.0/16,172.30.0.0/16
**NOTE**:
Synapse does not apply the IP blacklist to connections through the proxy (since
the DNS resolution is done by the proxy). It is expected that the proxy or firewall
will apply blacklisting of IP addresses.
## Connection types
The proxy will be **used** for:
- push
- url previews
- phone-home stats
- recaptcha validation
- CAS auth validation
- OpenID Connect
- Outbound federation
- Federation (checking public key revocation)
- Fetching public keys of other servers
- Downloading remote media
It will **not be used** for:
- Application Services
- Identity servers
- In worker configurations
- connections between workers
- connections from workers to Redis
## Troubleshooting
If a proxy server is used with TLS (HTTPS) and no connections are established,
it is most likely due to the proxy's certificates. To test this, the validation
in Synapse can be deactivated.
**NOTE**: This has an impact on security and is for testing purposes only!
To deactivate the certificate validation, the following setting must be made in
[homserver.yaml](../usage/configuration/homeserver_sample_config.md).
```yaml
use_insecure_ssl_client_just_for_testing_do_not_use: true
```

239
docs/templates.md Normal file
View File

@@ -0,0 +1,239 @@
# Templates
Synapse uses parametrised templates to generate the content of emails it sends and
webpages it shows to users.
By default, Synapse will use the templates listed [here](https://github.com/matrix-org/synapse/tree/master/synapse/res/templates).
Server admins can configure an additional directory for Synapse to look for templates
in, allowing them to specify custom templates:
```yaml
templates:
custom_templates_directory: /path/to/custom/templates/
```
If this setting is not set, or the files named below are not found within the directory,
default templates from within the Synapse package will be used.
Templates that are given variables when being rendered are rendered using [Jinja 2](https://jinja.palletsprojects.com/en/2.11.x/).
Templates rendered by Jinja 2 can also access two functions on top of the functions
already available as part of Jinja 2:
```python
format_ts(value: int, format: str) -> str
```
Formats a timestamp in milliseconds.
Example: `reason.last_sent_ts|format_ts("%c")`
```python
mxc_to_http(value: str, width: int, height: int, resize_method: str = "crop") -> str
```
Turns a `mxc://` URL for media content into an HTTP(S) one using the homeserver's
`public_baseurl` configuration setting as the URL's base.
Example: `message.sender_avatar_url|mxc_to_http(32,32)`
## Email templates
Below are the templates Synapse will look for when generating the content of an email:
* `notif_mail.html` and `notif_mail.txt`: The contents of email notifications of missed
events.
When rendering, this template is given the following variables:
* `user_display_name`: the display name for the user receiving the notification
* `unsubscribe_link`: the link users can click to unsubscribe from email notifications
* `summary_text`: a summary of the notification(s). The text used can be customised
by configuring the various settings in the `email.subjects` section of the
configuration file.
* `rooms`: a list of rooms containing events to include in the email. Each element is
an object with the following attributes:
* `title`: a human-readable name for the room
* `hash`: a hash of the ID of the room
* `invite`: a boolean, which is `True` if the room is an invite the user hasn't
accepted yet, `False` otherwise
* `notifs`: a list of events, or an empty list if `invite` is `True`. Each element
is an object with the following attributes:
* `link`: a `matrix.to` link to the event
* `ts`: the time in milliseconds at which the event was received
* `messages`: a list of messages containing one message before the event, the
message in the event, and one message after the event. Each element is an
object with the following attributes:
* `event_type`: the type of the event
* `is_historical`: a boolean, which is `False` if the message is the one
that triggered the notification, `True` otherwise
* `id`: the ID of the event
* `ts`: the time in milliseconds at which the event was sent
* `sender_name`: the display name for the event's sender
* `sender_avatar_url`: the avatar URL (as a `mxc://` URL) for the event's
sender
* `sender_hash`: a hash of the user ID of the sender
* `link`: a `matrix.to` link to the room
* `reason`: information on the event that triggered the email to be sent. It's an
object with the following attributes:
* `room_id`: the ID of the room the event was sent in
* `room_name`: a human-readable name for the room the event was sent in
* `now`: the current time in milliseconds
* `received_at`: the time in milliseconds at which the event was received
* `delay_before_mail_ms`: the amount of time in milliseconds Synapse always waits
before ever emailing about a notification (to give the user a chance to respond
to other push or notice the window)
* `last_sent_ts`: the time in milliseconds at which a notification was last sent
for an event in this room
* `throttle_ms`: the minimum amount of time in milliseconds between two
notifications can be sent for this room
* `password_reset.html` and `password_reset.txt`: The contents of password reset emails
sent by the homeserver.
When rendering, these templates are given a `link` variable which contains the link the
user must click in order to reset their password.
* `registration.html` and `registration.txt`: The contents of address verification emails
sent during registration.
When rendering, these templates are given a `link` variable which contains the link the
user must click in order to validate their email address.
* `add_threepid.html` and `add_threepid.txt`: The contents of address verification emails
sent when an address is added to a Matrix account.
When rendering, these templates are given a `link` variable which contains the link the
user must click in order to validate their email address.
## HTML page templates for registration and password reset
Below are the templates Synapse will look for when generating pages related to
registration and password reset:
* `password_reset_confirmation.html`: An HTML page that a user will see when they follow
the link in the password reset email. The user will be asked to confirm the action
before their password is reset.
When rendering, this template is given the following variables:
* `sid`: the session ID for the password reset
* `token`: the token for the password reset
* `client_secret`: the client secret for the password reset
* `password_reset_success.html` and `password_reset_failure.html`: HTML pages for success
and failure that a user will see when they confirm the password reset flow using the
page above.
When rendering, `password_reset_success.html` is given no variable, and
`password_reset_failure.html` is given a `failure_reason`, which contains the reason
for the password reset failure.
* `registration_success.html` and `registration_failure.html`: HTML pages for success and
failure that a user will see when they follow the link in an address verification email
sent during registration.
When rendering, `registration_success.html` is given no variable, and
`registration_failure.html` is given a `failure_reason`, which contains the reason
for the registration failure.
* `add_threepid_success.html` and `add_threepid_failure.html`: HTML pages for success and
failure that a user will see when they follow the link in an address verification email
sent when an address is added to a Matrix account.
When rendering, `add_threepid_success.html` is given no variable, and
`add_threepid_failure.html` is given a `failure_reason`, which contains the reason
for the registration failure.
## HTML page templates for Single Sign-On (SSO)
Below are the templates Synapse will look for when generating pages related to SSO:
* `sso_login_idp_picker.html`: HTML page to prompt the user to choose an
Identity Provider during login.
This is only used if multiple SSO Identity Providers are configured.
When rendering, this template is given the following variables:
* `redirect_url`: the URL that the user will be redirected to after
login.
* `server_name`: the homeserver's name.
* `providers`: a list of available Identity Providers. Each element is
an object with the following attributes:
* `idp_id`: unique identifier for the IdP
* `idp_name`: user-facing name for the IdP
* `idp_icon`: if specified in the IdP config, an MXC URI for an icon
for the IdP
* `idp_brand`: if specified in the IdP config, a textual identifier
for the brand of the IdP
The rendered HTML page should contain a form which submits its results
back as a GET request, with the following query parameters:
* `redirectUrl`: the client redirect URI (ie, the `redirect_url` passed
to the template)
* `idp`: the 'idp_id' of the chosen IDP.
* `sso_auth_account_details.html`: HTML page to prompt new users to enter a
userid and confirm other details. This is only shown if the
SSO implementation (with any `user_mapping_provider`) does not return
a localpart.
When rendering, this template is given the following variables:
* `server_name`: the homeserver's name.
* `idp`: details of the SSO Identity Provider that the user logged in
with: an object with the following attributes:
* `idp_id`: unique identifier for the IdP
* `idp_name`: user-facing name for the IdP
* `idp_icon`: if specified in the IdP config, an MXC URI for an icon
for the IdP
* `idp_brand`: if specified in the IdP config, a textual identifier
for the brand of the IdP
* `user_attributes`: an object containing details about the user that
we received from the IdP. May have the following attributes:
* display_name: the user's display_name
* emails: a list of email addresses
The template should render a form which submits the following fields:
* `username`: the localpart of the user's chosen user id
* `sso_new_user_consent.html`: HTML page allowing the user to consent to the
server's terms and conditions. This is only shown for new users, and only if
`user_consent.require_at_registration` is set.
When rendering, this template is given the following variables:
* `server_name`: the homeserver's name.
* `user_id`: the user's matrix proposed ID.
* `user_profile.display_name`: the user's proposed display name, if any.
* consent_version: the version of the terms that the user will be
shown
* `terms_url`: a link to the page showing the terms.
The template should render a form which submits the following fields:
* `accepted_version`: the version of the terms accepted by the user
(ie, 'consent_version' from the input variables).
* `sso_redirect_confirm.html`: HTML page for a confirmation step before redirecting back
to the client with the login token.
When rendering, this template is given the following variables:
* `redirect_url`: the URL the user is about to be redirected to.
* `display_url`: the same as `redirect_url`, but with the query
parameters stripped. The intention is to have a
human-readable URL to show to users, not to use it as
the final address to redirect to.
* `server_name`: the homeserver's name.
* `new_user`: a boolean indicating whether this is the user's first time
logging in.
* `user_id`: the user's matrix ID.
* `user_profile.avatar_url`: an MXC URI for the user's avatar, if any.
`None` if the user has not set an avatar.
* `user_profile.display_name`: the user's display name. `None` if the user
has not set a display name.
* `sso_auth_confirm.html`: HTML page which notifies the user that they are authenticating
to confirm an operation on their account during the user interactive authentication
process.
When rendering, this template is given the following variables:
* `redirect_url`: the URL the user is about to be redirected to.
* `description`: the operation which the user is being asked to confirm
* `idp`: details of the Identity Provider that we will use to confirm
the user's identity: an object with the following attributes:
* `idp_id`: unique identifier for the IdP
* `idp_name`: user-facing name for the IdP
* `idp_icon`: if specified in the IdP config, an MXC URI for an icon
for the IdP
* `idp_brand`: if specified in the IdP config, a textual identifier
for the brand of the IdP
* `sso_auth_success.html`: HTML page shown after a successful user interactive
authentication session.
Note that this page must include the JavaScript which notifies of a successful
authentication (see https://matrix.org/docs/spec/client_server/r0.6.0#fallback).
This template has no additional variables.
* `sso_auth_bad_user.html`: HTML page shown after a user-interactive authentication
session which does not map correctly onto the expected user.
When rendering, this template is given the following variables:
* `server_name`: the homeserver's name.
* `user_id_to_verify`: the MXID of the user that we are trying to
validate.
* `sso_account_deactivated.html`: HTML page shown during single sign-on if a deactivated
user (according to Synapse's database) attempts to login.
This template has no additional variables.
* `sso_error.html`: HTML page to display to users if something goes wrong during the
OpenID Connect authentication process.
When rendering, this template is given two variables:
* `error`: the technical name of the error
* `error_description`: a human-readable message for the error

View File

@@ -86,6 +86,50 @@ process, for example:
```
# Upgrading to v1.41.0
## Add support for routing outbound HTTP requests via a proxy for federation
Since Synapse 1.6.0 (2019-11-26) you can set a proxy for outbound HTTP requests via
http_proxy/https_proxy environment variables. This proxy was set for:
- push
- url previews
- phone-home stats
- recaptcha validation
- CAS auth validation
- OpenID Connect
- Federation (checking public key revocation)
In this version we have added support for outbound requests for:
- Outbound federation
- Downloading remote media
- Fetching public keys of other servers
These requests use the same proxy configuration. If you have a proxy configuration we
recommend to verify the configuration. It may be necessary to adjust the `no_proxy`
environment variable.
See [using a forward proxy with Synapse documentation](setup/forward_proxy.md) for
details.
## Deprecation of `template_dir`
The `template_dir` settings in the `sso`, `account_validity` and `email` sections of the
configuration file are now deprecated. Server admins should use the new
`templates.custom_template_directory` setting in the configuration file and use one single
custom template directory for all aforementioned features. Template file names remain
unchanged. See [the related documentation](https://matrix-org.github.io/synapse/latest/templates.html)
for more information and examples.
We plan to remove support for these settings in October 2021.
## `/_synapse/admin/v1/users/{userId}/media` must be handled by media workers
The [media repository worker documentation](https://matrix-org.github.io/synapse/latest/workers.html#synapseappmedia_repository)
has been updated to reflect that calls to `/_synapse/admin/v1/users/{userId}/media`
must now be handled by media repository workers. This is due to the new `DELETE` method
of this endpoint modifying the media store.
# Upgrading to v1.39.0
## Deprecation of the current third-party rules module interface

View File

@@ -214,6 +214,7 @@ expressions:
^/_matrix/federation/v1/send/
# Client API requests
^/_matrix/client/(api/v1|r0|unstable)/createRoom$
^/_matrix/client/(api/v1|r0|unstable)/publicRooms$
^/_matrix/client/(api/v1|r0|unstable)/rooms/.*/joined_members$
^/_matrix/client/(api/v1|r0|unstable)/rooms/.*/context/.*$
@@ -425,10 +426,12 @@ Handles the media repository. It can handle all endpoints starting with:
^/_synapse/admin/v1/user/.*/media.*$
^/_synapse/admin/v1/media/.*$
^/_synapse/admin/v1/quarantine_media/.*$
^/_synapse/admin/v1/users/.*/media$
You should also set `enable_media_repo: False` in the shared configuration
file to stop the main synapse running background jobs related to managing the
media repository.
media repository. Note that doing so will prevent the main process from being
able to handle the above endpoints.
In the `media_repository` worker configuration file, configure the http listener to
expose the `media` resource. For example: