1
0

Compare commits

..

5 Commits

Author SHA1 Message Date
H. Shay
da5c58bb24 run pyupgrade --py37-plus --keep-percent-format on synapse/ 2022-01-04 12:50:21 -08:00
H. Shay
6e94f64edf lint 2022-01-04 11:02:04 -08:00
H. Shay
da1f496487 newsfragment 2022-01-04 10:50:46 -08:00
H. Shay
68882a0251 run pyupgrade on synapse/app and synapse/appservice 2022-01-04 10:24:57 -08:00
H. Shay
56d599e5ac run pyupgrade on synapse/_scripts and synapse/api 2022-01-04 10:23:16 -08:00
114 changed files with 298 additions and 452 deletions

View File

@@ -1,91 +1,3 @@
Synapse 1.50.0rc1 (2022-01-05)
==============================
Please note that we now only support Python 3.7+ and PostgreSQL 10+ (if applicable), because Python 3.6 and PostgreSQL 9.6 have reached end-of-life.
Features
--------
- Allow guests to send state events per [MSC3419](https://github.com/matrix-org/matrix-doc/pull/3419). ([\#11378](https://github.com/matrix-org/synapse/issues/11378))
- Add experimental support for part of [MSC3202](https://github.com/matrix-org/matrix-doc/pull/3202): allowing application services to masquerade as specific devices. ([\#11538](https://github.com/matrix-org/synapse/issues/11538))
- Add admin API to get users' account data. ([\#11664](https://github.com/matrix-org/synapse/issues/11664))
- Include the room topic in the stripped state included with invites and knocking. ([\#11666](https://github.com/matrix-org/synapse/issues/11666))
- Send and handle cross-signing messages using the stable prefix. ([\#10520](https://github.com/matrix-org/synapse/issues/10520))
- Support unprefixed versions of fallback key property names. ([\#11541](https://github.com/matrix-org/synapse/issues/11541))
Bugfixes
--------
- Fix a long-standing bug where relations from other rooms could be included in the bundled aggregations of an event. ([\#11516](https://github.com/matrix-org/synapse/issues/11516))
- Fix a long-standing bug which could cause `AssertionError`s to be written to the log when Synapse was restarted after purging events from the database. ([\#11536](https://github.com/matrix-org/synapse/issues/11536), [\#11642](https://github.com/matrix-org/synapse/issues/11642))
- Fix a bug introduced in Synapse 1.17.0 where a pusher created for an email with capital letters would fail to be created. ([\#11547](https://github.com/matrix-org/synapse/issues/11547))
- Fix a long-standing bug where responses included bundled aggregations when they should not, per [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675). ([\#11592](https://github.com/matrix-org/synapse/issues/11592), [\#11623](https://github.com/matrix-org/synapse/issues/11623))
- Fix a long-standing bug that some unknown endpoints would return HTML error pages instead of JSON `M_UNRECOGNIZED` errors. ([\#11602](https://github.com/matrix-org/synapse/issues/11602))
- Fix a bug introduced in Synapse 1.19.3 which could sometimes cause `AssertionError`s when backfilling rooms over federation. ([\#11632](https://github.com/matrix-org/synapse/issues/11632))
Improved Documentation
----------------------
- Update Synapse install command for FreeBSD as the package is now prefixed with `py38`. Contributed by @itchychips. ([\#11267](https://github.com/matrix-org/synapse/issues/11267))
- Document the usage of refresh tokens. ([\#11427](https://github.com/matrix-org/synapse/issues/11427))
- Add details for how to configure a TURN server when behind a NAT. Contibuted by @AndrewFerr. ([\#11553](https://github.com/matrix-org/synapse/issues/11553))
- Add references for using Postgres to the Docker documentation. ([\#11640](https://github.com/matrix-org/synapse/issues/11640))
- Fix the documentation link in newly-generated configuration files. ([\#11678](https://github.com/matrix-org/synapse/issues/11678))
- Correct the documentation for `nginx` to use a case-sensitive url pattern. Fixes an error introduced in v1.21.0. ([\#11680](https://github.com/matrix-org/synapse/issues/11680))
- Clarify SSO mapping provider documentation by writing `def` or `async def` before the names of methods, as appropriate. ([\#11681](https://github.com/matrix-org/synapse/issues/11681))
Deprecations and Removals
-------------------------
- Replace `mock` package by its standard library version. ([\#11588](https://github.com/matrix-org/synapse/issues/11588))
Internal Changes
----------------
- Allow specific, experimental events to be created without `prev_events`. Used by [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716). ([\#11243](https://github.com/matrix-org/synapse/issues/11243))
- A test helper (`wait_for_background_updates`) no longer depends on classes defining a `store` property. ([\#11331](https://github.com/matrix-org/synapse/issues/11331))
- Add type hints to `synapse.appservice`. ([\#11360](https://github.com/matrix-org/synapse/issues/11360))
- Add missing type hints to `synapse.config` module. ([\#11480](https://github.com/matrix-org/synapse/issues/11480))
- Add test to ensure we share the same `state_group` across the whole historical batch when using the [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) `/batch_send` endpoint. ([\#11487](https://github.com/matrix-org/synapse/issues/11487))
- Refactor `tests.util.setup_test_homeserver` and `tests.server.setup_test_homeserver`. ([\#11503](https://github.com/matrix-org/synapse/issues/11503))
- Move `glob_to_regex` and `re_word_boundary` to `matrix-python-common`. ([\#11505](https://github.com/matrix-org/synapse/issues/11505), [\#11687](https://github.com/matrix-org/synapse/issues/11687))
- Use `HTTPStatus` constants in place of literals in `tests.rest.client.test_auth`. ([\#11520](https://github.com/matrix-org/synapse/issues/11520))
- Add a receipt types constant for `m.read`. ([\#11531](https://github.com/matrix-org/synapse/issues/11531))
- Clean up `synapse.rest.admin`. ([\#11535](https://github.com/matrix-org/synapse/issues/11535))
- Add missing `errcode` to `parse_string` and `parse_boolean`. ([\#11542](https://github.com/matrix-org/synapse/issues/11542))
- Use `HTTPStatus` constants in place of literals in `synapse.http`. ([\#11543](https://github.com/matrix-org/synapse/issues/11543))
- Add missing type hints to storage classes. ([\#11546](https://github.com/matrix-org/synapse/issues/11546), [\#11549](https://github.com/matrix-org/synapse/issues/11549), [\#11551](https://github.com/matrix-org/synapse/issues/11551), [\#11555](https://github.com/matrix-org/synapse/issues/11555), [\#11575](https://github.com/matrix-org/synapse/issues/11575), [\#11589](https://github.com/matrix-org/synapse/issues/11589), [\#11594](https://github.com/matrix-org/synapse/issues/11594), [\#11652](https://github.com/matrix-org/synapse/issues/11652), [\#11653](https://github.com/matrix-org/synapse/issues/11653), [\#11654](https://github.com/matrix-org/synapse/issues/11654), [\#11657](https://github.com/matrix-org/synapse/issues/11657))
- Fix an inaccurate and misleading comment in the `/sync` code. ([\#11550](https://github.com/matrix-org/synapse/issues/11550))
- Add missing type hints to `synapse.logging.context`. ([\#11556](https://github.com/matrix-org/synapse/issues/11556))
- Stop populating unused database column `state_events.prev_state`. ([\#11558](https://github.com/matrix-org/synapse/issues/11558))
- Minor efficiency improvements in event persistence. ([\#11560](https://github.com/matrix-org/synapse/issues/11560))
- Add some safety checks that storage functions are used correctly. ([\#11564](https://github.com/matrix-org/synapse/issues/11564), [\#11580](https://github.com/matrix-org/synapse/issues/11580))
- Make `get_device` return `None` if the device doesn't exist rather than raising an exception. ([\#11565](https://github.com/matrix-org/synapse/issues/11565))
- Split the HTML parsing code from the URL preview resource code. ([\#11566](https://github.com/matrix-org/synapse/issues/11566))
- Remove redundant `COALESCE()`s around `COUNT()`s in database queries. ([\#11570](https://github.com/matrix-org/synapse/issues/11570))
- Add missing type hints to `synapse.http`. ([\#11571](https://github.com/matrix-org/synapse/issues/11571))
- Add [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) and [MSC3030](https://github.com/matrix-org/matrix-doc/pull/3030) to `/versions` -> `unstable_features` to detect server support. ([\#11582](https://github.com/matrix-org/synapse/issues/11582))
- Add type hints to `synapse/tests/rest/admin`. ([\#11590](https://github.com/matrix-org/synapse/issues/11590))
- Drop end-of-life Python 3.6 and Postgres 9.6 from CI. ([\#11595](https://github.com/matrix-org/synapse/issues/11595))
- Update black version and run it on all the files. ([\#11596](https://github.com/matrix-org/synapse/issues/11596))
- Add opentracing type stubs and fix associated mypy errors. ([\#11603](https://github.com/matrix-org/synapse/issues/11603), [\#11622](https://github.com/matrix-org/synapse/issues/11622))
- Improve OpenTracing support for requests which use a `ResponseCache`. ([\#11607](https://github.com/matrix-org/synapse/issues/11607))
- Improve OpenTracing support for incoming HTTP requests. ([\#11618](https://github.com/matrix-org/synapse/issues/11618))
- A number of improvements to opentracing support. ([\#11619](https://github.com/matrix-org/synapse/issues/11619))
- Drop support for Python 3.6 and Ubuntu 18.04. ([\#11633](https://github.com/matrix-org/synapse/issues/11633))
- Refactor the way that the `outlier` flag is set on events received over federation. ([\#11634](https://github.com/matrix-org/synapse/issues/11634))
- Improve the error messages from `get_create_event_for_room`. ([\#11638](https://github.com/matrix-org/synapse/issues/11638))
- Remove redundant `get_current_events_token` method. ([\#11643](https://github.com/matrix-org/synapse/issues/11643))
- Convert `namedtuples` to `attrs`. ([\#11665](https://github.com/matrix-org/synapse/issues/11665), [\#11574](https://github.com/matrix-org/synapse/issues/11574))
- Update the `/capabilities` response to include whether support for [MSC3440](https://github.com/matrix-org/matrix-doc/pull/3440) is available. ([\#11690](https://github.com/matrix-org/synapse/issues/11690))
- Send the `Accept` header in HTTP requests made using `SimpleHttpClient.get_json`. ([\#11677](https://github.com/matrix-org/synapse/issues/11677))
Synapse 1.49.2 (2021-12-21)
===========================

1
changelog.d/10520.misc Normal file
View File

@@ -0,0 +1 @@
Send and handle cross-signing messages using the stable prefix.

1
changelog.d/11243.misc Normal file
View File

@@ -0,0 +1 @@
Allow specific, experimental events to be created without `prev_events`. Used by [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716).

1
changelog.d/11331.misc Normal file
View File

@@ -0,0 +1 @@
A test helper (`wait_for_background_updates`) no longer depends on classes defining a `store` property.

1
changelog.d/11360.misc Normal file
View File

@@ -0,0 +1 @@
Add type hints to `synapse.appservice`.

View File

@@ -0,0 +1 @@
Allow guests to send state events per [MSC3419](https://github.com/matrix-org/matrix-doc/pull/3419).

1
changelog.d/11427.doc Normal file
View File

@@ -0,0 +1 @@
Document the usage of refresh tokens.

1
changelog.d/11480.misc Normal file
View File

@@ -0,0 +1 @@
Add missing type hints to `synapse.config` module.

1
changelog.d/11487.misc Normal file
View File

@@ -0,0 +1 @@
Add test to ensure we share the same `state_group` across the whole historical batch when using the [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) `/batch_send` endpoint.

1
changelog.d/11503.misc Normal file
View File

@@ -0,0 +1 @@
Refactor `tests.util.setup_test_homeserver` and `tests.server.setup_test_homeserver`.

1
changelog.d/11516.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix a long-standing bug where relations from other rooms could be included in the bundled aggregations of an event.

1
changelog.d/11520.misc Normal file
View File

@@ -0,0 +1 @@
Use HTTPStatus constants in place of literals in `tests.rest.client.test_auth`.

1
changelog.d/11531.misc Normal file
View File

@@ -0,0 +1 @@
Add a receipt types constant for `m.read`.

1
changelog.d/11535.misc Normal file
View File

@@ -0,0 +1 @@
Clean up `synapse.rest.admin`.

1
changelog.d/11536.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix a long-standing bug which could cause `AssertionError`s to be written to the log when Synapse was restarted after purging events from the database.

View File

@@ -0,0 +1 @@
Add experimental support for MSC3202: allowing application services to masquerade as specific devices.

1
changelog.d/11541.misc Normal file
View File

@@ -0,0 +1 @@
Support unprefixed versions of fallback key property names.

1
changelog.d/11542.misc Normal file
View File

@@ -0,0 +1 @@
Add missing `errcode` to `parse_string` and `parse_boolean`.

1
changelog.d/11543.misc Normal file
View File

@@ -0,0 +1 @@
Use HTTPStatus constants in place of literals in `synapse.http`.

1
changelog.d/11546.misc Normal file
View File

@@ -0,0 +1 @@
Add missing type hints to storage classes.

1
changelog.d/11547.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix a bug introduced in Synapse 1.17.0 where a pusher created for an email with capital letters would fail to be created.

1
changelog.d/11549.misc Normal file
View File

@@ -0,0 +1 @@
Add missing type hints to storage classes.

1
changelog.d/11550.misc Normal file
View File

@@ -0,0 +1 @@
Fix an inaccurate and misleading comment in the `/sync` code.

1
changelog.d/11551.misc Normal file
View File

@@ -0,0 +1 @@
Add missing type hints to storage classes.

1
changelog.d/11553.doc Normal file
View File

@@ -0,0 +1 @@
Add details for how to configure a TURN server when behind a NAT. Contibuted by @AndrewFerr.

1
changelog.d/11555.misc Normal file
View File

@@ -0,0 +1 @@
Add missing type hints to storage classes.

1
changelog.d/11556.misc Normal file
View File

@@ -0,0 +1 @@
Add missing type hints to `synapse.logging.context`.

1
changelog.d/11558.misc Normal file
View File

@@ -0,0 +1 @@
Stop populating unused database column `state_events.prev_state`.

1
changelog.d/11560.misc Normal file
View File

@@ -0,0 +1 @@
Minor efficiency improvements in event persistence.

1
changelog.d/11564.misc Normal file
View File

@@ -0,0 +1 @@
Add some safety checks that storage functions are used correctly.

1
changelog.d/11565.misc Normal file
View File

@@ -0,0 +1 @@
Make `get_device` return `None` if the device doesn't exist rather than raising an exception.

1
changelog.d/11566.misc Normal file
View File

@@ -0,0 +1 @@
Split the HTML parsing code from the URL preview resource code.

1
changelog.d/11570.misc Normal file
View File

@@ -0,0 +1 @@
Remove redundant `COALESCE()`s around `COUNT()`s in database queries.

1
changelog.d/11571.misc Normal file
View File

@@ -0,0 +1 @@
Add missing type hints to `synapse.http`.

1
changelog.d/11574.misc Normal file
View File

@@ -0,0 +1 @@
Convert `EventStreamResult` from a `namedtuple` to `attrs` to improve type hints.

1
changelog.d/11575.misc Normal file
View File

@@ -0,0 +1 @@
Add missing type hints to storage classes.

1
changelog.d/11580.misc Normal file
View File

@@ -0,0 +1 @@
Add some safety checks that storage functions are used correctly.

1
changelog.d/11582.misc Normal file
View File

@@ -0,0 +1 @@
Add [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) and [MSC3030](https://github.com/matrix-org/matrix-doc/pull/3030) to `/versions` -> `unstable_features` to detect server support.

View File

@@ -1 +0,0 @@
Fix a performance regression in `/sync` handling, introduced in 1.49.0.

View File

@@ -0,0 +1 @@
Replace `mock` package by its standard library version.

1
changelog.d/11589.misc Normal file
View File

@@ -0,0 +1 @@
Add missing type hints to storage classes.

1
changelog.d/11590.misc Normal file
View File

@@ -0,0 +1 @@
Add type hints to `synapse/tests/rest/admin`.

1
changelog.d/11592.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix a long-standing bug where responses included bundled aggregations when they should not, per [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675).

1
changelog.d/11594.misc Normal file
View File

@@ -0,0 +1 @@
Add missing type hints to storage classes.

1
changelog.d/11595.misc Normal file
View File

@@ -0,0 +1 @@
Drop EOL python 3.6 and postgres 9.6 from CI.

1
changelog.d/11596.misc Normal file
View File

@@ -0,0 +1 @@
Update black version and run it on all the files.

1
changelog.d/11602.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix a long-standing bug that some unknown endpoints would return HTML error pages instead of JSON `M_UNRECOGNIZED` errors.

1
changelog.d/11603.misc Normal file
View File

@@ -0,0 +1 @@
Add opentracing type stubs and fix associated mypy errors.

1
changelog.d/11607.misc Normal file
View File

@@ -0,0 +1 @@
Improve opentracing support for requests which use a `ResponseCache`.

1
changelog.d/11618.misc Normal file
View File

@@ -0,0 +1 @@
Improve opentracing support for incoming HTTP requests.

1
changelog.d/11619.misc Normal file
View File

@@ -0,0 +1 @@
A number of improvements to opentracing support.

1
changelog.d/11622.misc Normal file
View File

@@ -0,0 +1 @@
Add opentracing type stubs and fix associated mypy errors.

1
changelog.d/11623.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix a long-standing bug where responses included bundled aggregations when they should not, per [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675).

1
changelog.d/11632.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix a bug introduced in Synapse 1.19.3 which could sometimes cause `AssertionError`s when backfilling rooms over federation.

1
changelog.d/11633.misc Normal file
View File

@@ -0,0 +1 @@
Drop support for Python 3.6 and Ubuntu 18.04.

1
changelog.d/11638.misc Normal file
View File

@@ -0,0 +1 @@
Improve the error messages from `get_create_event_for_room`.

1
changelog.d/11642.bugfix Normal file
View File

@@ -0,0 +1 @@
Fix a long-standing bug which could cause `AssertionError`s to be written to the log when Synapse was restarted after purging events from the database.

1
changelog.d/11643.misc Normal file
View File

@@ -0,0 +1 @@
Remove redundant `get_current_events_token` method.

1
changelog.d/11652.misc Normal file
View File

@@ -0,0 +1 @@
Add missing type hints to storage classes.

1
changelog.d/11653.misc Normal file
View File

@@ -0,0 +1 @@
Add missing type hints to storage classes.

1
changelog.d/11654.misc Normal file
View File

@@ -0,0 +1 @@
Add missing type hints to storage classes.

1
changelog.d/11657.misc Normal file
View File

@@ -0,0 +1 @@
Add missing type hints to storage classes.

1
changelog.d/11665.misc Normal file
View File

@@ -0,0 +1 @@
Convert `namedtuples` to `attrs`.

View File

@@ -0,0 +1 @@
Include the room topic in the stripped state included with invites and knocking.

1
changelog.d/11684.misc Normal file
View File

@@ -0,0 +1 @@
Run `pyupgrade --py36-plus` on `synapse/app`, `synapse/appservice`, `synapse/_scripts`, 'synapse/api'.

View File

@@ -1 +0,0 @@
Work around Mjolnir compatibility issue by adding an import for `glob_to_regex` in `synapse.util`, where it moved from.

View File

@@ -14,7 +14,6 @@ services:
# failure
restart: unless-stopped
# See the readme for a full documentation of the environment settings
# NOTE: You must edit homeserver.yaml to use postgres, it defaults to sqlite
environment:
- SYNAPSE_CONFIG_PATH=/data/homeserver.yaml
volumes:

6
debian/changelog vendored
View File

@@ -1,9 +1,3 @@
matrix-synapse-py3 (1.50.0~rc1) stable; urgency=medium
* New synapse release 1.50.0~rc1.
-- Synapse Packaging team <packages@matrix.org> Wed, 05 Jan 2022 12:36:17 +0000
matrix-synapse-py3 (1.49.2) stable; urgency=medium
* New synapse release 1.49.2.

View File

@@ -68,10 +68,6 @@ The following environment variables are supported in `generate` mode:
directories. If unset, and no user is set via `docker run --user`, defaults
to `991`, `991`.
## Postgres
By default the config will use SQLite. See the [docs on using Postgres](https://github.com/matrix-org/synapse/blob/develop/docs/postgres.md) for more info on how to use Postgres. Until this section is improved [this issue](https://github.com/matrix-org/synapse/issues/8304) may provide useful information.
## Running synapse
Once you have a valid configuration file, you can start synapse as follows:

View File

@@ -480,81 +480,6 @@ The following fields are returned in the JSON response body:
- `joined_rooms` - An array of `room_id`.
- `total` - Number of rooms.
## Account Data
Gets information about account data for a specific `user_id`.
The API is:
```
GET /_synapse/admin/v1/users/<user_id>/accountdata
```
A response body like the following is returned:
```json
{
"account_data": {
"global": {
"m.secret_storage.key.LmIGHTg5W": {
"algorithm": "m.secret_storage.v1.aes-hmac-sha2",
"iv": "fwjNZatxg==",
"mac": "eWh9kNnLWZUNOgnc="
},
"im.vector.hide_profile": {
"hide_profile": true
},
"org.matrix.preview_urls": {
"disable": false
},
"im.vector.riot.breadcrumb_rooms": {
"rooms": [
"!LxcBDAsDUVAfJDEo:matrix.org",
"!MAhRxqasbItjOqxu:matrix.org"
]
},
"m.accepted_terms": {
"accepted": [
"https://example.org/somewhere/privacy-1.2-en.html",
"https://example.org/somewhere/terms-2.0-en.html"
]
},
"im.vector.setting.breadcrumbs": {
"recent_rooms": [
"!MAhRxqasbItqxuEt:matrix.org",
"!ZtSaPCawyWtxiImy:matrix.org"
]
}
},
"rooms": {
"!GUdfZSHUJibpiVqHYd:matrix.org": {
"m.fully_read": {
"event_id": "$156334540fYIhZ:matrix.org"
}
},
"!tOZwOOiqwCYQkLhV:matrix.org": {
"m.fully_read": {
"event_id": "$xjsIyp4_NaVl2yPvIZs_k1Jl8tsC_Sp23wjqXPno"
}
}
}
}
}
```
**Parameters**
The following parameters should be set in the URL:
- `user_id` - fully qualified: for example, `@user:server.com`.
**Response**
The following fields are returned in the JSON response body:
- `account_data` - A map containing the account data for the user
- `global` - A map containing the global account data for the user
- `rooms` - A map containing the account data per room for the user
## User media
### List media uploaded by a user

View File

@@ -63,7 +63,7 @@ server {
server_name matrix.example.com;
location ~ ^(/_matrix|/_synapse/client) {
location ~* ^(\/_matrix|\/_synapse\/client) {
# note: do not add a path (even a single /) after the port in `proxy_pass`,
# otherwise nginx will canonicalise the URI and cause signature verification
# errors.

View File

@@ -37,7 +37,7 @@
# Server admins can expand Synapse's functionality with external modules.
#
# See https://matrix-org.github.io/synapse/latest/modules/index.html for more
# See https://matrix-org.github.io/synapse/latest/modules.html for more
# documentation on how to configure or create custom modules for Synapse.
#
modules:

View File

@@ -164,7 +164,7 @@ xbps-install -S synapse
Synapse can be installed via FreeBSD Ports or Packages contributed by Brendan Molloy from:
- Ports: `cd /usr/ports/net-im/py-matrix-synapse && make install clean`
- Packages: `pkg install py38-matrix-synapse`
- Packages: `pkg install py37-matrix-synapse`
#### OpenBSD

View File

@@ -49,12 +49,12 @@ comment these options out and use those specified by the module instead.
A custom mapping provider must specify the following methods:
* `def __init__(self, parsed_config)`
* `__init__(self, parsed_config)`
- Arguments:
- `parsed_config` - A configuration object that is the return value of the
`parse_config` method. You should set any configuration options needed by
the module here.
* `def parse_config(config)`
* `parse_config(config)`
- This method should have the `@staticmethod` decoration.
- Arguments:
- `config` - A `dict` representing the parsed content of the
@@ -63,13 +63,13 @@ A custom mapping provider must specify the following methods:
any option values they need here.
- Whatever is returned will be passed back to the user mapping provider module's
`__init__` method during construction.
* `def get_remote_user_id(self, userinfo)`
* `get_remote_user_id(self, userinfo)`
- Arguments:
- `userinfo` - A `authlib.oidc.core.claims.UserInfo` object to extract user
information from.
- This method must return a string, which is the unique, immutable identifier
for the user. Commonly the `sub` claim of the response.
* `async def map_user_attributes(self, userinfo, token, failures)`
* `map_user_attributes(self, userinfo, token, failures)`
- This method must be async.
- Arguments:
- `userinfo` - A `authlib.oidc.core.claims.UserInfo` object to extract user
@@ -91,7 +91,7 @@ A custom mapping provider must specify the following methods:
during a user's first login. Once a localpart has been associated with a
remote user ID (see `get_remote_user_id`) it cannot be updated.
- `displayname`: An optional string, the display name for the user.
* `async def get_extra_attributes(self, userinfo, token)`
* `get_extra_attributes(self, userinfo, token)`
- This method must be async.
- Arguments:
- `userinfo` - A `authlib.oidc.core.claims.UserInfo` object to extract user
@@ -125,15 +125,15 @@ comment these options out and use those specified by the module instead.
A custom mapping provider must specify the following methods:
* `def __init__(self, parsed_config, module_api)`
* `__init__(self, parsed_config, module_api)`
- Arguments:
- `parsed_config` - A configuration object that is the return value of the
`parse_config` method. You should set any configuration options needed by
the module here.
- `module_api` - a `synapse.module_api.ModuleApi` object which provides the
stable API available for extension modules.
* `def parse_config(config)`
- **This method should have the `@staticmethod` decoration.**
* `parse_config(config)`
- This method should have the `@staticmethod` decoration.
- Arguments:
- `config` - A `dict` representing the parsed content of the
`saml_config.user_mapping_provider.config` homeserver config option.
@@ -141,15 +141,15 @@ A custom mapping provider must specify the following methods:
any option values they need here.
- Whatever is returned will be passed back to the user mapping provider module's
`__init__` method during construction.
* `def get_saml_attributes(config)`
- **This method should have the `@staticmethod` decoration.**
* `get_saml_attributes(config)`
- This method should have the `@staticmethod` decoration.
- Arguments:
- `config` - A object resulting from a call to `parse_config`.
- Returns a tuple of two sets. The first set equates to the SAML auth
response attributes that are required for the module to function, whereas
the second set consists of those attributes which can be used if available,
but are not necessary.
* `def get_remote_user_id(self, saml_response, client_redirect_url)`
* `get_remote_user_id(self, saml_response, client_redirect_url)`
- Arguments:
- `saml_response` - A `saml2.response.AuthnResponse` object to extract user
information from.
@@ -157,7 +157,7 @@ A custom mapping provider must specify the following methods:
redirected to.
- This method must return a string, which is the unique, immutable identifier
for the user. Commonly the `uid` claim of the response.
* `def saml_response_to_user_attributes(self, saml_response, failures, client_redirect_url)`
* `saml_response_to_user_attributes(self, saml_response, failures, client_redirect_url)`
- Arguments:
- `saml_response` - A `saml2.response.AuthnResponse` object to extract user
information from.

View File

@@ -85,17 +85,6 @@ process, for example:
dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
```
# Upgrading to v1.50.0
## Dropping support for old Python and Postgres versions
In line with our [deprecation policy](deprecation_policy.md),
we've dropped support for Python 3.6 and PostgreSQL 9.6, as they are no
longer supported upstream.
This release of Synapse requires Python 3.7+ and PostgreSQL 10+.
# Upgrading to v1.47.0
## Removal of old Room Admin API

View File

@@ -47,7 +47,7 @@ try:
except ImportError:
pass
__version__ = "1.50.0rc1"
__version__ = "1.49.2"
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
# We import here so that we don't have to install a bunch of deps when

View File

@@ -38,7 +38,7 @@ def request_registration(
exit: Callable[[int], None] = sys.exit,
) -> None:
url = "%s/_synapse/admin/v1/register" % (server_location.rstrip("/"),)
url = "{}/_synapse/admin/v1/register".format(server_location.rstrip("/"))
# Get the nonce
r = requests.get(url, verify=False)
@@ -108,7 +108,7 @@ def register_new_user(
default_user = None
if default_user:
user = input("New user localpart [%s]: " % (default_user,))
user = input(f"New user localpart [{default_user}]: ")
if not user:
user = default_user
else:

View File

@@ -122,7 +122,7 @@ class Auth:
if not forgot:
return member
raise AuthError(403, "User %s not in room %s" % (user_id, room_id))
raise AuthError(403, f"User {user_id} not in room {room_id}")
async def get_user_by_req(
self,

View File

@@ -117,7 +117,7 @@ class RedirectException(CodeMessageException):
location: the URI to redirect to
http_code: the HTTP response code
"""
msg = "Redirect to %s" % (location.decode("utf-8"),)
msg = "Redirect to {}".format(location.decode("utf-8"))
super().__init__(code=http_code, msg=msg)
self.location = location
@@ -237,7 +237,7 @@ class FederationDeniedError(SynapseError):
super().__init__(
code=403,
msg="Federation denied with %s." % (self.destination,),
msg=f"Federation denied with {self.destination}.",
errcode=Codes.FORBIDDEN,
)
@@ -518,14 +518,14 @@ class FederationError(RuntimeError):
source: Optional[str] = None,
):
if level not in ["FATAL", "ERROR", "WARN"]:
raise ValueError("Level is not valid: %s" % (level,))
raise ValueError(f"Level is not valid: {level}")
self.level = level
self.code = code
self.reason = reason
self.affected = affected
self.source = source
msg = "%s %s: %s" % (level, code, reason)
msg = f"{level} {code}: {reason}"
super().__init__(msg)
def get_dict(self) -> "JsonDict":

View File

@@ -218,7 +218,7 @@ class FilterCollection:
self.event_format = filter_json.get("event_format", "client")
def __repr__(self) -> str:
return "<FilterCollection %s>" % (json.dumps(self._filter_json),)
return "<FilterCollection {}>".format(json.dumps(self._filter_json))
def get_filter_json(self) -> JsonDict:
return self._filter_json
@@ -418,7 +418,7 @@ class Filter:
for name, match_func in field_matchers.items():
# If the event matches one of the disallowed values, reject it.
not_name = "not_%s" % (name,)
not_name = f"not_{name}"
disallowed_values = getattr(self, not_name)
if any(map(match_func, disallowed_values)):
return False

View File

@@ -55,7 +55,7 @@ class ConsentURIBuilder:
mac = hmac.new(
key=self._hmac_secret, msg=user_id.encode("ascii"), digestmod=sha256
).hexdigest()
consent_uri = "%s_matrix/consent?%s" % (
consent_uri = "{}_matrix/consent?{}".format(
self._public_baseurl,
urlencode({"u": user_id, "h": mac}),
)

View File

@@ -183,7 +183,7 @@ def quit_with_error(error_string: str) -> NoReturn:
line_length = min(max(len(line) for line in message_lines), 80) + 2
sys.stderr.write("*" * line_length + "\n")
for line in message_lines:
sys.stderr.write(" %s\n" % (line.rstrip(),))
sys.stderr.write(f" {line.rstrip()}\n")
sys.stderr.write("*" * line_length + "\n")
sys.exit(1)

View File

@@ -102,7 +102,7 @@ class FileExfiltrationWriter(ExfiltrationWriter):
self.base_directory = directory
else:
self.base_directory = tempfile.mkdtemp(
prefix="synapse-exfiltrate__%s__" % (user_id,)
prefix=f"synapse-exfiltrate__{user_id}__"
)
os.makedirs(self.base_directory, exist_ok=True)

View File

@@ -381,7 +381,7 @@ class GenericWorkerServer(HomeServer):
bind_addresses,
port,
SynapseSite(
"synapse.access.http.%s" % (site_tag,),
f"synapse.access.http.{site_tag}",
site_tag,
listener_config,
root_resource,

View File

@@ -114,7 +114,7 @@ class SynapseHomeServer(HomeServer):
for path, resmodule in additional_resources.items():
handler_cls, config = load_module(
resmodule,
("listeners", site_tag, "additional_resources", "<%s>" % (path,)),
("listeners", site_tag, "additional_resources", f"<{path}>"),
)
handler = handler_cls(config, module_api)
if isinstance(handler, Resource):
@@ -141,7 +141,7 @@ class SynapseHomeServer(HomeServer):
root_resource = OptionsResource()
site = SynapseSite(
"synapse.access.%s.%s" % ("https" if tls else "http", site_tag),
"synapse.access.{}.{}".format("https" if tls else "http", site_tag),
site_tag,
listener_config,
create_resource_tree(resources, root_resource),
@@ -417,15 +417,15 @@ def format_config_error(e: ConfigError) -> Iterator[str]:
yield "Error in configuration"
if e.path:
yield " at '%s'" % (".".join(e.path),)
yield " at '{}'".format(".".join(e.path))
yield ":\n %s" % (e.msg,)
yield f":\n {e.msg}"
parent_e = e.__cause__
indent = 1
while parent_e:
indent += 1
yield ":\n%s%s" % (" " * indent, str(parent_e))
yield ":\n{}{}".format(" " * indent, str(parent_e))
parent_e = parent_e.__cause__

View File

@@ -143,7 +143,7 @@ async def phone_stats_home(
stats["log_level"] = logging.getLevelName(log_level)
logger.info(
"Reporting stats to %s: %s" % (hs.config.metrics.report_stats_endpoint, stats)
f"Reporting stats to {hs.config.metrics.report_stats_endpoint}: {stats}"
)
try:
await hs.get_proxied_http_client().put_json(

View File

@@ -339,7 +339,7 @@ class ApplicationService:
dict_copy = self.__dict__.copy()
dict_copy["token"] = "<redacted>"
dict_copy["hs_token"] = "<redacted>"
return "ApplicationService: %s" % (dict_copy,)
return f"ApplicationService: {dict_copy}"
class AppServiceTransaction:

View File

@@ -149,7 +149,7 @@ class ApplicationServiceApi(SimpleHttpClient):
if service.url is None:
return []
uri = "%s%s/thirdparty/%s/%s" % (
uri = "{}{}/thirdparty/{}/{}".format(
service.url,
APP_SERVICE_PREFIX,
kind,
@@ -184,7 +184,7 @@ class ApplicationServiceApi(SimpleHttpClient):
return {}
async def _get() -> Optional[JsonDict]:
uri = "%s%s/thirdparty/protocol/%s" % (
uri = "{}{}/thirdparty/protocol/{}".format(
service.url,
APP_SERVICE_PREFIX,
urllib.parse.quote(protocol),

View File

@@ -133,7 +133,7 @@ class _ServiceQueuer:
return
run_as_background_process(
"as-sender-%s" % (service.id,), self._send_request, service
f"as-sender-{service.id}", self._send_request, service
)
def enqueue_event(self, service: ApplicationService, event: EventBase) -> None:
@@ -281,9 +281,7 @@ class _Recoverer:
def recover(self) -> None:
def _retry() -> None:
run_as_background_process(
"as-recoverer-%s" % (self.service.id,), self.retry
)
run_as_background_process(f"as-recoverer-{self.service.id}", self.retry)
delay = 2 ** self.backoff_counter
logger.info("Scheduling retries on %s in %fs", self.service.id, delay)

View File

@@ -37,7 +37,7 @@ class ModulesConfig(Config):
# Server admins can expand Synapse's functionality with external modules.
#
# See https://matrix-org.github.io/synapse/latest/modules/index.html for more
# See https://matrix-org.github.io/synapse/latest/modules.html for more
# documentation on how to configure or create custom modules for Synapse.
#
modules:

View File

@@ -15,9 +15,8 @@
from typing import List
from matrix_common.regex import glob_to_regex
from synapse.types import JsonDict
from synapse.util import glob_to_regex
from ._base import Config, ConfigError

View File

@@ -16,12 +16,11 @@ import logging
import os
from typing import List, Optional, Pattern
from matrix_common.regex import glob_to_regex
from OpenSSL import SSL, crypto
from twisted.internet._sslverify import Certificate, trustRootFromCertificates
from synapse.config._base import Config, ConfigError
from synapse.util import glob_to_regex
logger = logging.getLogger(__name__)

View File

@@ -215,12 +215,15 @@ def _is_invite_via_3pid(event: EventBase) -> bool:
)
def event_from_pdu_json(pdu_json: JsonDict, room_version: RoomVersion) -> EventBase:
def event_from_pdu_json(
pdu_json: JsonDict, room_version: RoomVersion, outlier: bool = False
) -> EventBase:
"""Construct an EventBase from an event json received over federation
Args:
pdu_json: pdu as received over federation
room_version: The version of the room this event belongs to
outlier: True to mark this event as an outlier
Raises:
SynapseError: if the pdu is missing required fields or is otherwise
@@ -244,4 +247,6 @@ def event_from_pdu_json(pdu_json: JsonDict, room_version: RoomVersion) -> EventB
validate_canonicaljson(pdu_json)
event = make_event_from_dict(pdu_json, room_version)
event.internal_metadata.outlier = outlier
return event

View File

@@ -265,7 +265,10 @@ class FederationClient(FederationBase):
room_version = await self.store.get_room_version(room_id)
pdus = [event_from_pdu_json(p, room_version) for p in transaction_data_pdus]
pdus = [
event_from_pdu_json(p, room_version, outlier=False)
for p in transaction_data_pdus
]
# Check signatures and hash of pdus, removing any from the list that fail checks
pdus[:] = await self._check_sigs_and_hash_and_fetch(
@@ -279,6 +282,7 @@ class FederationClient(FederationBase):
destination: str,
event_id: str,
room_version: RoomVersion,
outlier: bool = False,
timeout: Optional[int] = None,
) -> Optional[EventBase]:
"""Requests the PDU with given origin and ID from the remote home
@@ -288,6 +292,9 @@ class FederationClient(FederationBase):
destination: Which homeserver to query
event_id: event to fetch
room_version: version of the room
outlier: Indicates whether the PDU is an `outlier`, i.e. if
it's from an arbitrary point in the context as opposed to part
of the current block of PDUs. Defaults to `False`
timeout: How long to try (in ms) each destination for before
moving to the next destination. None indicates no timeout.
@@ -309,7 +316,8 @@ class FederationClient(FederationBase):
)
pdu_list: List[EventBase] = [
event_from_pdu_json(p, room_version) for p in transaction_data["pdus"]
event_from_pdu_json(p, room_version, outlier=outlier)
for p in transaction_data["pdus"]
]
if pdu_list and pdu_list[0]:
@@ -326,6 +334,7 @@ class FederationClient(FederationBase):
destinations: Iterable[str],
event_id: str,
room_version: RoomVersion,
outlier: bool = False,
timeout: Optional[int] = None,
) -> Optional[EventBase]:
"""Requests the PDU with given origin and ID from the remote home
@@ -338,6 +347,9 @@ class FederationClient(FederationBase):
destinations: Which homeservers to query
event_id: event to fetch
room_version: version of the room
outlier: Indicates whether the PDU is an `outlier`, i.e. if
it's from an arbitrary point in the context as opposed to part
of the current block of PDUs. Defaults to `False`
timeout: How long to try (in ms) each destination for before
moving to the next destination. None indicates no timeout.
@@ -365,6 +377,7 @@ class FederationClient(FederationBase):
destination=destination,
event_id=event_id,
room_version=room_version,
outlier=outlier,
timeout=timeout,
)
@@ -422,6 +435,7 @@ class FederationClient(FederationBase):
origin: str,
pdus: Collection[EventBase],
room_version: RoomVersion,
outlier: bool = False,
) -> List[EventBase]:
"""Takes a list of PDUs and checks the signatures and hashes of each
one. If a PDU fails its signature check then we check if we have it in
@@ -437,6 +451,7 @@ class FederationClient(FederationBase):
origin
pdu
room_version
outlier: Whether the events are outliers or not
Returns:
A list of PDUs that have valid signatures and hashes.
@@ -451,6 +466,7 @@ class FederationClient(FederationBase):
valid_pdu = await self._check_sigs_and_hash_and_fetch_one(
pdu=pdu,
origin=origin,
outlier=outlier,
room_version=room_version,
)
@@ -466,6 +482,7 @@ class FederationClient(FederationBase):
pdu: EventBase,
origin: str,
room_version: RoomVersion,
outlier: bool = False,
) -> Optional[EventBase]:
"""Takes a PDU and checks its signatures and hashes. If the PDU fails
its signature check then we check if we have it in the database and if
@@ -477,6 +494,9 @@ class FederationClient(FederationBase):
origin
pdu
room_version
outlier: Whether the events are outliers or not
include_none: Whether to include None in the returned list
for events that have failed their checks
Returns:
The PDU (possibly redacted) if it has valid signatures and hashes.
@@ -501,6 +521,7 @@ class FederationClient(FederationBase):
destinations=[pdu_origin],
event_id=pdu.event_id,
room_version=room_version,
outlier=outlier,
timeout=10000,
)
except SynapseError:
@@ -520,10 +541,13 @@ class FederationClient(FederationBase):
room_version = await self.store.get_room_version(room_id)
auth_chain = [event_from_pdu_json(p, room_version) for p in res["auth_chain"]]
auth_chain = [
event_from_pdu_json(p, room_version, outlier=True)
for p in res["auth_chain"]
]
signed_auth = await self._check_sigs_and_hash_and_fetch(
destination, auth_chain, room_version=room_version
destination, auth_chain, outlier=True, room_version=room_version
)
return signed_auth
@@ -792,6 +816,7 @@ class FederationClient(FederationBase):
valid_pdu = await self._check_sigs_and_hash_and_fetch_one(
pdu=event,
origin=destination,
outlier=True,
room_version=room_version,
)
@@ -839,6 +864,7 @@ class FederationClient(FederationBase):
valid_pdu = await self._check_sigs_and_hash_and_fetch_one(
pdu=pdu,
origin=destination,
outlier=True,
room_version=room_version,
)
@@ -1209,7 +1235,7 @@ class FederationClient(FederationBase):
]
signed_events = await self._check_sigs_and_hash_and_fetch(
destination, events, room_version=room_version
destination, events, outlier=False, room_version=room_version
)
except HttpResponseException as e:
if not e.code == 400:

View File

@@ -28,7 +28,6 @@ from typing import (
Union,
)
from matrix_common.regex import glob_to_regex
from prometheus_client import Counter, Gauge, Histogram
from twisted.internet.abstract import isIPAddress
@@ -66,7 +65,7 @@ from synapse.replication.http.federation import (
)
from synapse.storage.databases.main.lock import Lock
from synapse.types import JsonDict, get_domain_from_id
from synapse.util import json_decoder, unwrapFirstError
from synapse.util import glob_to_regex, json_decoder, unwrapFirstError
from synapse.util.async_helpers import Linearizer, concurrently_execute, gather_results
from synapse.util.caches.response_cache import ResponseCache
from synapse.util.stringutils import parse_server_name

View File

@@ -421,6 +421,9 @@ class FederationEventHandler:
Raises:
SynapseError if the response is in some way invalid.
"""
for e in itertools.chain(auth_events, state):
e.internal_metadata.outlier = True
event_map = {e.event_id: e for e in itertools.chain(auth_events, state)}
create_event = None
@@ -1191,6 +1194,7 @@ class FederationEventHandler:
[destination],
event_id,
room_version,
outlier=True,
)
if event is None:
logger.warning(
@@ -1219,10 +1223,9 @@ class FederationEventHandler:
"""Persist a batch of outlier events fetched from remote servers.
We first sort the events to make sure that we process each event's auth_events
before the event itself.
before the event itself, and then auth and persist them.
We then mark the events as outliers, persist them to the database, and, where
appropriate (eg, an invite), awake the notifier.
Notifies about the events where appropriate.
Params:
room_id: the room that the events are meant to be in (though this has
@@ -1273,8 +1276,7 @@ class FederationEventHandler:
Persists a batch of events where we have (theoretically) already persisted all
of their auth events.
Marks the events as outliers, auths them, persists them to the database, and,
where appropriate (eg, an invite), awakes the notifier.
Notifies about the events where appropriate.
Params:
origin: where the events came from
@@ -1312,9 +1314,6 @@ class FederationEventHandler:
return None
auth.append(ae)
# we're not bothering about room state, so flag the event as an outlier.
event.internal_metadata.outlier = True
context = EventContext.for_outlier()
try:
validate_event_for_room_version(room_version_obj, event)

View File

@@ -277,7 +277,7 @@ class MessageHandler:
# If this is an AS, double check that they are allowed to see the members.
# This can either be because the AS user is in the room or because there
# is a user in the room that the AS is "interested in"
if False and requester.app_service and user_id not in users_with_profile:
if requester.app_service and user_id not in users_with_profile:
for uid in users_with_profile:
if requester.app_service.is_interested_in_user(uid):
break

View File

@@ -82,7 +82,6 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
self.event_auth_handler = hs.get_event_auth_handler()
self.member_linearizer: Linearizer = Linearizer(name="member")
self.member_limiter = Linearizer(max_count=10, name="member_as_limiter")
self.clock = hs.get_clock()
self.spam_checker = hs.get_spam_checker()
@@ -483,43 +482,24 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
key = (room_id,)
as_id = object()
if requester.app_service:
as_id = requester.app_service.id
then = self.clock.time_msec()
with (await self.member_limiter.queue(as_id)):
diff = self.clock.time_msec() - then
if diff > 80 * 1000:
# haproxy would have timed the request out anyway...
raise SynapseError(504, "took to long to process")
with (await self.member_linearizer.queue(key)):
diff = self.clock.time_msec() - then
if diff > 80 * 1000:
# haproxy would have timed the request out anyway...
raise SynapseError(504, "took to long to process")
result = await self.update_membership_locked(
requester,
target,
room_id,
action,
txn_id=txn_id,
remote_room_hosts=remote_room_hosts,
third_party_signed=third_party_signed,
ratelimit=ratelimit,
content=content,
new_room=new_room,
require_consent=require_consent,
outlier=outlier,
historical=historical,
prev_event_ids=prev_event_ids,
auth_event_ids=auth_event_ids,
)
with (await self.member_linearizer.queue(key)):
result = await self.update_membership_locked(
requester,
target,
room_id,
action,
txn_id=txn_id,
remote_room_hosts=remote_room_hosts,
third_party_signed=third_party_signed,
ratelimit=ratelimit,
content=content,
new_room=new_room,
require_consent=require_consent,
outlier=outlier,
historical=historical,
prev_event_ids=prev_event_ids,
auth_event_ids=auth_event_ids,
)
return result

View File

@@ -588,7 +588,7 @@ class SimpleHttpClient:
if headers:
actual_headers.update(headers) # type: ignore
body = await self.get_raw(uri, args, headers=actual_headers)
body = await self.get_raw(uri, args, headers=headers)
return json_decoder.decode(body.decode("utf-8"))
async def put_json(

View File

@@ -104,11 +104,6 @@ class HttpPusher(Pusher):
"'url' must have a path of '/_matrix/push/v1/notify'"
)
url = url.replace(
"https://matrix.org/_matrix/push/v1/notify",
"http://10.103.0.7/_matrix/push/v1/notify",
)
self.url = url
self.http_client = hs.get_proxied_blacklisted_http_client()
self.data_minus_url = {}

Some files were not shown because too many files have changed in this diff Show More