From b690fe749b5715f4c22b8a0a8640c264d8b94b3c Mon Sep 17 00:00:00 2001 From: Brendan Abolivier Date: Tue, 29 Mar 2022 10:41:04 +0100 Subject: [PATCH 01/55] 1.56.0rc1 --- CHANGES.md | 59 +++++++++++++++++++++++++++++++++++++++ changelog.d/12036.misc | 1 - changelog.d/12038.misc | 1 - changelog.d/12083.misc | 1 - changelog.d/12087.bugfix | 1 - changelog.d/12091.misc | 1 - changelog.d/12195.feature | 1 - changelog.d/12198.misc | 1 - changelog.d/12199.misc | 1 - changelog.d/12216.misc | 1 - changelog.d/12219.misc | 1 - changelog.d/12224.misc | 1 - changelog.d/12225.misc | 1 - changelog.d/12227.bugfix | 1 - changelog.d/12228.bugfix | 1 - changelog.d/12231.doc | 1 - changelog.d/12232.bugfix | 1 - changelog.d/12237.misc | 1 - changelog.d/12240.misc | 1 - changelog.d/12242.misc | 1 - changelog.d/12243.doc | 1 - changelog.d/12244.misc | 1 - changelog.d/12246.doc | 1 - changelog.d/12248.misc | 1 - changelog.d/12250.feature | 1 - changelog.d/12255.misc | 1 - changelog.d/12256.misc | 1 - changelog.d/12258.misc | 1 - changelog.d/12261.bugfix | 1 - changelog.d/12262.misc | 1 - changelog.d/12266.misc | 1 - changelog.d/12269.misc | 1 - changelog.d/12272.misc | 1 - changelog.d/12275.doc | 1 - changelog.d/12279.doc | 1 - changelog.d/12283.misc | 1 - changelog.d/12285.bugfix | 1 - changelog.d/12288.misc | 1 - changelog.d/12291.misc | 1 - changelog.d/12301.misc | 1 - changelog.d/12304.misc | 1 - changelog.d/12311.misc | 1 - changelog.d/12313.misc | 1 - changelog.d/12314.misc | 1 - debian/changelog | 6 ++++ synapse/__init__.py | 2 +- 46 files changed, 66 insertions(+), 44 deletions(-) delete mode 100644 changelog.d/12036.misc delete mode 100644 changelog.d/12038.misc delete mode 100644 changelog.d/12083.misc delete mode 100644 changelog.d/12087.bugfix delete mode 100644 changelog.d/12091.misc delete mode 100644 changelog.d/12195.feature delete mode 100644 changelog.d/12198.misc delete mode 100644 changelog.d/12199.misc delete mode 100644 changelog.d/12216.misc delete mode 100644 changelog.d/12219.misc delete mode 100644 changelog.d/12224.misc delete mode 100644 changelog.d/12225.misc delete mode 100644 changelog.d/12227.bugfix delete mode 100644 changelog.d/12228.bugfix delete mode 100644 changelog.d/12231.doc delete mode 100644 changelog.d/12232.bugfix delete mode 100644 changelog.d/12237.misc delete mode 100644 changelog.d/12240.misc delete mode 100644 changelog.d/12242.misc delete mode 100644 changelog.d/12243.doc delete mode 100644 changelog.d/12244.misc delete mode 100644 changelog.d/12246.doc delete mode 100644 changelog.d/12248.misc delete mode 100644 changelog.d/12250.feature delete mode 100644 changelog.d/12255.misc delete mode 100644 changelog.d/12256.misc delete mode 100644 changelog.d/12258.misc delete mode 100644 changelog.d/12261.bugfix delete mode 100644 changelog.d/12262.misc delete mode 100644 changelog.d/12266.misc delete mode 100644 changelog.d/12269.misc delete mode 100644 changelog.d/12272.misc delete mode 100644 changelog.d/12275.doc delete mode 100644 changelog.d/12279.doc delete mode 100644 changelog.d/12283.misc delete mode 100644 changelog.d/12285.bugfix delete mode 100644 changelog.d/12288.misc delete mode 100644 changelog.d/12291.misc delete mode 100644 changelog.d/12301.misc delete mode 100644 changelog.d/12304.misc delete mode 100644 changelog.d/12311.misc delete mode 100644 changelog.d/12313.misc delete mode 100644 changelog.d/12314.misc diff --git a/CHANGES.md b/CHANGES.md index b0244a16f0..1fce34e725 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,3 +1,62 @@ +Synapse 1.56.0rc1 (2022-03-29) +============================== + +Features +-------- + +- Allow modules to store already existing 3PID associations. ([\#12195](https://github.com/matrix-org/synapse/issues/12195)) +- Allow registering admin users using the module API. Contributed by Famedly. ([\#12250](https://github.com/matrix-org/synapse/issues/12250)) + + +Bugfixes +-------- + +- Fix a long-standing bug which caused the `/_matrix/federation/v1/state` and `.../state_ids` endpoints to return incorrect or invalid data when called for an event which we have stored as an "outlier". ([\#12087](https://github.com/matrix-org/synapse/issues/12087)) +- Fix a long-standing bug where events from ignored users were still considered for relations. ([\#12227](https://github.com/matrix-org/synapse/issues/12227), [\#12232](https://github.com/matrix-org/synapse/issues/12232), [\#12285](https://github.com/matrix-org/synapse/issues/12285)) +- Fix a bug introduced in v1.53.0 where an unnecessary query could be performed when fetching bundled aggregations for threads. ([\#12228](https://github.com/matrix-org/synapse/issues/12228)) +- Fix a bug introduced in Synapse 1.52 where admins could not deactivate and GDPR-erase a user if Synapse was configured with limits on avatars. ([\#12261](https://github.com/matrix-org/synapse/issues/12261)) + + +Improved Documentation +---------------------- + +- Fix the link to the module documentation in the legacy spam checker warning message. ([\#12231](https://github.com/matrix-org/synapse/issues/12231)) +- Remove incorrect prefixes in the worker documentation for some endpoints. ([\#12243](https://github.com/matrix-org/synapse/issues/12243)) +- Correct `check_username_for_spam` annotations and docs. ([\#12246](https://github.com/matrix-org/synapse/issues/12246)) +- Corrected Authentik OpenID typo, added helpful note for troubleshooting. Contributed by @IronTooch. ([\#12275](https://github.com/matrix-org/synapse/issues/12275)) +- HAProxy reverse proxy guide update to stop sending IPv4-mapped address to homeserver. Contributed by @villepeh. ([\#12279](https://github.com/matrix-org/synapse/issues/12279)) + + +Internal Changes +---------------- + +- Rename `shared_rooms` to `mutual_rooms` (MSC2666), as per proposal changes. ([\#12036](https://github.com/matrix-org/synapse/issues/12036)) +- Remove check on `update_user_directory` for shared rooms handler (MSC2666), and update/expand documentation. ([\#12038](https://github.com/matrix-org/synapse/issues/12038)) +- Refactor `create_new_client_event` to use a new parameter, `state_event_ids`, which accurately describes the usage with [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) instead of abusing `auth_event_ids`. ([\#12083](https://github.com/matrix-org/synapse/issues/12083), [\#12304](https://github.com/matrix-org/synapse/issues/12304)) +- Refuse to start if registration is enabled without email, captcha, or token-based verification unless new config flag `enable_registration_without_verification` is set. ([\#12091](https://github.com/matrix-org/synapse/issues/12091)) +- Add tests for database transaction callbacks. ([\#12198](https://github.com/matrix-org/synapse/issues/12198)) +- Handle cancellation in `DatabasePool.runInteraction()`. ([\#12199](https://github.com/matrix-org/synapse/issues/12199)) +- Add missing type hints for cache storage. ([\#12216](https://github.com/matrix-org/synapse/issues/12216)) +- Clean-up logic around rebasing URLs for URL image previews. ([\#12219](https://github.com/matrix-org/synapse/issues/12219)) +- Add type hints to tests files. ([\#12224](https://github.com/matrix-org/synapse/issues/12224), [\#12240](https://github.com/matrix-org/synapse/issues/12240), [\#12256](https://github.com/matrix-org/synapse/issues/12256)) +- Use the `ignored_users` table in additional places instead of re-parsing the account data. ([\#12225](https://github.com/matrix-org/synapse/issues/12225)) +- Refactor the relations endpoints to add a `RelationsHandler`. ([\#12237](https://github.com/matrix-org/synapse/issues/12237)) +- Generate announcement links in the release script. ([\#12242](https://github.com/matrix-org/synapse/issues/12242)) +- Improve error message when dependencies check finds a broken installation. ([\#12244](https://github.com/matrix-org/synapse/issues/12244)) +- Add missing type hints for storage. ([\#12248](https://github.com/matrix-org/synapse/issues/12248), [\#12255](https://github.com/matrix-org/synapse/issues/12255)) +- Compress metrics HTTP resource when enabled. Contributed by Nick @ Beeper. ([\#12258](https://github.com/matrix-org/synapse/issues/12258)) +- Refuse to start if DB has non-`C` locale, unless config flag `allow_unsafe_db_locale` is set to true. ([\#12262](https://github.com/matrix-org/synapse/issues/12262), [\#12288](https://github.com/matrix-org/synapse/issues/12288)) +- Optionally include account validity expiration information to experimental [MSC3720](https://github.com/matrix-org/matrix-doc/pull/3720) account status responses. ([\#12266](https://github.com/matrix-org/synapse/issues/12266)) +- Use type stubs for `psycopg2`. ([\#12269](https://github.com/matrix-org/synapse/issues/12269)) +- Add a new cache `_get_membership_from_event_id` to speed up push rule calculations in large rooms. ([\#12272](https://github.com/matrix-org/synapse/issues/12272)) +- Re-enable Complement concurrency in CI. ([\#12283](https://github.com/matrix-org/synapse/issues/12283)) +- Remove unused test utilities. ([\#12291](https://github.com/matrix-org/synapse/issues/12291)) +- Enhance logging for inbound federation events. ([\#12301](https://github.com/matrix-org/synapse/issues/12301)) +- Improve type annotations for `execute_values`. ([\#12311](https://github.com/matrix-org/synapse/issues/12311)) +- Fix compatibility with the recently-released Jinja 3.1. ([\#12313](https://github.com/matrix-org/synapse/issues/12313)) +- Avoid trying to calculate the state at outlier events. ([\#12314](https://github.com/matrix-org/synapse/issues/12314)) + + Synapse 1.55.2 (2022-03-24) =========================== diff --git a/changelog.d/12036.misc b/changelog.d/12036.misc deleted file mode 100644 index d2996730cc..0000000000 --- a/changelog.d/12036.misc +++ /dev/null @@ -1 +0,0 @@ -Rename `shared_rooms` to `mutual_rooms` (MSC2666), as per proposal changes. \ No newline at end of file diff --git a/changelog.d/12038.misc b/changelog.d/12038.misc deleted file mode 100644 index e2a65726b6..0000000000 --- a/changelog.d/12038.misc +++ /dev/null @@ -1 +0,0 @@ -Remove check on `update_user_directory` for shared rooms handler (MSC2666), and update/expand documentation. \ No newline at end of file diff --git a/changelog.d/12083.misc b/changelog.d/12083.misc deleted file mode 100644 index 88fd6b92ee..0000000000 --- a/changelog.d/12083.misc +++ /dev/null @@ -1 +0,0 @@ -Refactor `create_new_client_event` to use a new parameter, `state_event_ids`, which accurately describes the usage with [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) instead of abusing `auth_event_ids`. diff --git a/changelog.d/12087.bugfix b/changelog.d/12087.bugfix deleted file mode 100644 index 6dacdddd0d..0000000000 --- a/changelog.d/12087.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix a long-standing bug which caused the `/_matrix/federation/v1/state` and `.../state_ids` endpoints to return incorrect or invalid data when called for an event which we have stored as an "outlier". diff --git a/changelog.d/12091.misc b/changelog.d/12091.misc deleted file mode 100644 index def44987b4..0000000000 --- a/changelog.d/12091.misc +++ /dev/null @@ -1 +0,0 @@ -Refuse to start if registration is enabled without email, captcha, or token-based verification unless new config flag `enable_registration_without_verification` is set. diff --git a/changelog.d/12195.feature b/changelog.d/12195.feature deleted file mode 100644 index e8bcb950a1..0000000000 --- a/changelog.d/12195.feature +++ /dev/null @@ -1 +0,0 @@ -Allow modules to store already existing 3PID associations. diff --git a/changelog.d/12198.misc b/changelog.d/12198.misc deleted file mode 100644 index 6b184a9053..0000000000 --- a/changelog.d/12198.misc +++ /dev/null @@ -1 +0,0 @@ -Add tests for database transaction callbacks. diff --git a/changelog.d/12199.misc b/changelog.d/12199.misc deleted file mode 100644 index 16dec1d26d..0000000000 --- a/changelog.d/12199.misc +++ /dev/null @@ -1 +0,0 @@ -Handle cancellation in `DatabasePool.runInteraction()`. diff --git a/changelog.d/12216.misc b/changelog.d/12216.misc deleted file mode 100644 index dc398ac1e0..0000000000 --- a/changelog.d/12216.misc +++ /dev/null @@ -1 +0,0 @@ -Add missing type hints for cache storage. diff --git a/changelog.d/12219.misc b/changelog.d/12219.misc deleted file mode 100644 index 6079414092..0000000000 --- a/changelog.d/12219.misc +++ /dev/null @@ -1 +0,0 @@ -Clean-up logic around rebasing URLs for URL image previews. diff --git a/changelog.d/12224.misc b/changelog.d/12224.misc deleted file mode 100644 index b67a701dbb..0000000000 --- a/changelog.d/12224.misc +++ /dev/null @@ -1 +0,0 @@ -Add type hints to tests files. diff --git a/changelog.d/12225.misc b/changelog.d/12225.misc deleted file mode 100644 index 23105c727c..0000000000 --- a/changelog.d/12225.misc +++ /dev/null @@ -1 +0,0 @@ -Use the `ignored_users` table in additional places instead of re-parsing the account data. diff --git a/changelog.d/12227.bugfix b/changelog.d/12227.bugfix deleted file mode 100644 index 1a7dccf465..0000000000 --- a/changelog.d/12227.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix a long-standing bug where events from ignored users were still considered for relations. diff --git a/changelog.d/12228.bugfix b/changelog.d/12228.bugfix deleted file mode 100644 index 4755777139..0000000000 --- a/changelog.d/12228.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix a bug introduced in v1.53.0 where an unnecessary query could be performed when fetching bundled aggregations for threads. diff --git a/changelog.d/12231.doc b/changelog.d/12231.doc deleted file mode 100644 index 16593d2b92..0000000000 --- a/changelog.d/12231.doc +++ /dev/null @@ -1 +0,0 @@ -Fix the link to the module documentation in the legacy spam checker warning message. diff --git a/changelog.d/12232.bugfix b/changelog.d/12232.bugfix deleted file mode 100644 index 1a7dccf465..0000000000 --- a/changelog.d/12232.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix a long-standing bug where events from ignored users were still considered for relations. diff --git a/changelog.d/12237.misc b/changelog.d/12237.misc deleted file mode 100644 index 41c9dcbd37..0000000000 --- a/changelog.d/12237.misc +++ /dev/null @@ -1 +0,0 @@ -Refactor the relations endpoints to add a `RelationsHandler`. diff --git a/changelog.d/12240.misc b/changelog.d/12240.misc deleted file mode 100644 index c5b6356799..0000000000 --- a/changelog.d/12240.misc +++ /dev/null @@ -1 +0,0 @@ -Add type hints to tests files. \ No newline at end of file diff --git a/changelog.d/12242.misc b/changelog.d/12242.misc deleted file mode 100644 index 38e7e0f7d1..0000000000 --- a/changelog.d/12242.misc +++ /dev/null @@ -1 +0,0 @@ -Generate announcement links in the release script. diff --git a/changelog.d/12243.doc b/changelog.d/12243.doc deleted file mode 100644 index b2031f0a40..0000000000 --- a/changelog.d/12243.doc +++ /dev/null @@ -1 +0,0 @@ -Remove incorrect prefixes in the worker documentation for some endpoints. diff --git a/changelog.d/12244.misc b/changelog.d/12244.misc deleted file mode 100644 index 950d48e4c6..0000000000 --- a/changelog.d/12244.misc +++ /dev/null @@ -1 +0,0 @@ -Improve error message when dependencies check finds a broken installation. \ No newline at end of file diff --git a/changelog.d/12246.doc b/changelog.d/12246.doc deleted file mode 100644 index e7fcc1b99c..0000000000 --- a/changelog.d/12246.doc +++ /dev/null @@ -1 +0,0 @@ -Correct `check_username_for_spam` annotations and docs. \ No newline at end of file diff --git a/changelog.d/12248.misc b/changelog.d/12248.misc deleted file mode 100644 index 2b1290d1e1..0000000000 --- a/changelog.d/12248.misc +++ /dev/null @@ -1 +0,0 @@ -Add missing type hints for storage. \ No newline at end of file diff --git a/changelog.d/12250.feature b/changelog.d/12250.feature deleted file mode 100644 index 29a2724457..0000000000 --- a/changelog.d/12250.feature +++ /dev/null @@ -1 +0,0 @@ -Allow registering admin users using the module API. Contributed by Famedly. diff --git a/changelog.d/12255.misc b/changelog.d/12255.misc deleted file mode 100644 index 2b1290d1e1..0000000000 --- a/changelog.d/12255.misc +++ /dev/null @@ -1 +0,0 @@ -Add missing type hints for storage. \ No newline at end of file diff --git a/changelog.d/12256.misc b/changelog.d/12256.misc deleted file mode 100644 index c5b6356799..0000000000 --- a/changelog.d/12256.misc +++ /dev/null @@ -1 +0,0 @@ -Add type hints to tests files. \ No newline at end of file diff --git a/changelog.d/12258.misc b/changelog.d/12258.misc deleted file mode 100644 index 80024c8e91..0000000000 --- a/changelog.d/12258.misc +++ /dev/null @@ -1 +0,0 @@ -Compress metrics HTTP resource when enabled. Contributed by Nick @ Beeper. diff --git a/changelog.d/12261.bugfix b/changelog.d/12261.bugfix deleted file mode 100644 index 1bfde4c380..0000000000 --- a/changelog.d/12261.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix a bug introduced in Synapse 1.52 where admins could not deactivate and GDPR-erase a user if Synapse was configured with limits on avatars. diff --git a/changelog.d/12262.misc b/changelog.d/12262.misc deleted file mode 100644 index 574ac4752c..0000000000 --- a/changelog.d/12262.misc +++ /dev/null @@ -1 +0,0 @@ -Refuse to start if DB has non-`C` locale, unless config flag `allow_unsafe_db_locale` is set to true. \ No newline at end of file diff --git a/changelog.d/12266.misc b/changelog.d/12266.misc deleted file mode 100644 index 59e2718370..0000000000 --- a/changelog.d/12266.misc +++ /dev/null @@ -1 +0,0 @@ -Optionally include account validity expiration information to experimental [MSC3720](https://github.com/matrix-org/matrix-doc/pull/3720) account status responses. diff --git a/changelog.d/12269.misc b/changelog.d/12269.misc deleted file mode 100644 index ed79cbb528..0000000000 --- a/changelog.d/12269.misc +++ /dev/null @@ -1 +0,0 @@ -Use type stubs for `psycopg2`. diff --git a/changelog.d/12272.misc b/changelog.d/12272.misc deleted file mode 100644 index 95589f3361..0000000000 --- a/changelog.d/12272.misc +++ /dev/null @@ -1 +0,0 @@ -Add a new cache `_get_membership_from_event_id` to speed up push rule calculations in large rooms. diff --git a/changelog.d/12275.doc b/changelog.d/12275.doc deleted file mode 100644 index 2e26ad21eb..0000000000 --- a/changelog.d/12275.doc +++ /dev/null @@ -1 +0,0 @@ -Corrected Authentik OpenID typo, added helpful note for troubleshooting. Contributed by @IronTooch. diff --git a/changelog.d/12279.doc b/changelog.d/12279.doc deleted file mode 100644 index ca07104c90..0000000000 --- a/changelog.d/12279.doc +++ /dev/null @@ -1 +0,0 @@ -HAProxy reverse proxy guide update to stop sending IPv4-mapped address to homeserver. Contributed by @villepeh. diff --git a/changelog.d/12283.misc b/changelog.d/12283.misc deleted file mode 100644 index e9f2208500..0000000000 --- a/changelog.d/12283.misc +++ /dev/null @@ -1 +0,0 @@ -Re-enable Complement concurrency in CI. diff --git a/changelog.d/12285.bugfix b/changelog.d/12285.bugfix deleted file mode 100644 index 1a7dccf465..0000000000 --- a/changelog.d/12285.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix a long-standing bug where events from ignored users were still considered for relations. diff --git a/changelog.d/12288.misc b/changelog.d/12288.misc deleted file mode 100644 index ee8fbfd290..0000000000 --- a/changelog.d/12288.misc +++ /dev/null @@ -1 +0,0 @@ -Refuse to start if DB has non-`C` locale, unless config flag `allow_unsafe_db_locale` is set to true. diff --git a/changelog.d/12291.misc b/changelog.d/12291.misc deleted file mode 100644 index b55dd68f92..0000000000 --- a/changelog.d/12291.misc +++ /dev/null @@ -1 +0,0 @@ -Remove unused test utilities. diff --git a/changelog.d/12301.misc b/changelog.d/12301.misc deleted file mode 100644 index a4cd94ee5e..0000000000 --- a/changelog.d/12301.misc +++ /dev/null @@ -1 +0,0 @@ -Enhance logging for inbound federation events. diff --git a/changelog.d/12304.misc b/changelog.d/12304.misc deleted file mode 100644 index 88fd6b92ee..0000000000 --- a/changelog.d/12304.misc +++ /dev/null @@ -1 +0,0 @@ -Refactor `create_new_client_event` to use a new parameter, `state_event_ids`, which accurately describes the usage with [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) instead of abusing `auth_event_ids`. diff --git a/changelog.d/12311.misc b/changelog.d/12311.misc deleted file mode 100644 index df0e824a7e..0000000000 --- a/changelog.d/12311.misc +++ /dev/null @@ -1 +0,0 @@ -Improve type annotations for `execute_values`. \ No newline at end of file diff --git a/changelog.d/12313.misc b/changelog.d/12313.misc deleted file mode 100644 index f59f6cdc40..0000000000 --- a/changelog.d/12313.misc +++ /dev/null @@ -1 +0,0 @@ -Fix compatibility with the recently-released Jinja 3.1. diff --git a/changelog.d/12314.misc b/changelog.d/12314.misc deleted file mode 100644 index 9f333e718a..0000000000 --- a/changelog.d/12314.misc +++ /dev/null @@ -1 +0,0 @@ -Avoid trying to calculate the state at outlier events. diff --git a/debian/changelog b/debian/changelog index 3c899e6024..d04954457d 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +matrix-synapse-py3 (1.56.0~rc1) stable; urgency=medium + + * New synapse release 1.56.0~rc1. + + -- Synapse Packaging team Tue, 29 Mar 2022 10:40:50 +0100 + matrix-synapse-py3 (1.55.2) stable; urgency=medium * New synapse release 1.55.2. diff --git a/synapse/__init__.py b/synapse/__init__.py index 88aef1889c..0960305d79 100644 --- a/synapse/__init__.py +++ b/synapse/__init__.py @@ -68,7 +68,7 @@ try: except ImportError: pass -__version__ = "1.55.2" +__version__ = "1.56.0rc1" if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)): # We import here so that we don't have to install a bunch of deps when From c20d0ca6c2a27444565f869568b21f297cdcc964 Mon Sep 17 00:00:00 2001 From: Brendan Abolivier Date: Tue, 29 Mar 2022 10:50:35 +0100 Subject: [PATCH 02/55] Fixup changelog --- CHANGES.md | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index 1fce34e725..36ff2499af 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -5,16 +5,16 @@ Features -------- - Allow modules to store already existing 3PID associations. ([\#12195](https://github.com/matrix-org/synapse/issues/12195)) -- Allow registering admin users using the module API. Contributed by Famedly. ([\#12250](https://github.com/matrix-org/synapse/issues/12250)) +- Allow registering server administrators using the module API. Contributed by Famedly. ([\#12250](https://github.com/matrix-org/synapse/issues/12250)) Bugfixes -------- -- Fix a long-standing bug which caused the `/_matrix/federation/v1/state` and `.../state_ids` endpoints to return incorrect or invalid data when called for an event which we have stored as an "outlier". ([\#12087](https://github.com/matrix-org/synapse/issues/12087)) -- Fix a long-standing bug where events from ignored users were still considered for relations. ([\#12227](https://github.com/matrix-org/synapse/issues/12227), [\#12232](https://github.com/matrix-org/synapse/issues/12232), [\#12285](https://github.com/matrix-org/synapse/issues/12285)) -- Fix a bug introduced in v1.53.0 where an unnecessary query could be performed when fetching bundled aggregations for threads. ([\#12228](https://github.com/matrix-org/synapse/issues/12228)) -- Fix a bug introduced in Synapse 1.52 where admins could not deactivate and GDPR-erase a user if Synapse was configured with limits on avatars. ([\#12261](https://github.com/matrix-org/synapse/issues/12261)) +- Fix a long-standing bug which caused the `/_matrix/federation/v1/state` and `/_matrix/federation/v1/state_ids` endpoints to return incorrect or invalid data when called for an event which we have stored as an "outlier". ([\#12087](https://github.com/matrix-org/synapse/issues/12087)) +- Fix a long-standing bug where events from ignored users would still be considered for relations. ([\#12227](https://github.com/matrix-org/synapse/issues/12227), [\#12232](https://github.com/matrix-org/synapse/issues/12232), [\#12285](https://github.com/matrix-org/synapse/issues/12285)) +- Fix a bug introduced in Synapse 1.53.0 where an unnecessary query could be performed when fetching bundled aggregations for threads. ([\#12228](https://github.com/matrix-org/synapse/issues/12228)) +- Fix a bug introduced in Synapse 1.52.0 where admins could not deactivate and GDPR-erase a user if Synapse was configured with limits on avatars. ([\#12261](https://github.com/matrix-org/synapse/issues/12261)) Improved Documentation @@ -23,7 +23,7 @@ Improved Documentation - Fix the link to the module documentation in the legacy spam checker warning message. ([\#12231](https://github.com/matrix-org/synapse/issues/12231)) - Remove incorrect prefixes in the worker documentation for some endpoints. ([\#12243](https://github.com/matrix-org/synapse/issues/12243)) - Correct `check_username_for_spam` annotations and docs. ([\#12246](https://github.com/matrix-org/synapse/issues/12246)) -- Corrected Authentik OpenID typo, added helpful note for troubleshooting. Contributed by @IronTooch. ([\#12275](https://github.com/matrix-org/synapse/issues/12275)) +- Correct Authentik OpenID typo, and add notes on troubleshooting. Contributed by @IronTooch. ([\#12275](https://github.com/matrix-org/synapse/issues/12275)) - HAProxy reverse proxy guide update to stop sending IPv4-mapped address to homeserver. Contributed by @villepeh. ([\#12279](https://github.com/matrix-org/synapse/issues/12279)) @@ -33,26 +33,26 @@ Internal Changes - Rename `shared_rooms` to `mutual_rooms` (MSC2666), as per proposal changes. ([\#12036](https://github.com/matrix-org/synapse/issues/12036)) - Remove check on `update_user_directory` for shared rooms handler (MSC2666), and update/expand documentation. ([\#12038](https://github.com/matrix-org/synapse/issues/12038)) - Refactor `create_new_client_event` to use a new parameter, `state_event_ids`, which accurately describes the usage with [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) instead of abusing `auth_event_ids`. ([\#12083](https://github.com/matrix-org/synapse/issues/12083), [\#12304](https://github.com/matrix-org/synapse/issues/12304)) -- Refuse to start if registration is enabled without email, captcha, or token-based verification unless new config flag `enable_registration_without_verification` is set. ([\#12091](https://github.com/matrix-org/synapse/issues/12091)) +- Refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config flag `enable_registration_without_verification` is set. ([\#12091](https://github.com/matrix-org/synapse/issues/12091)) - Add tests for database transaction callbacks. ([\#12198](https://github.com/matrix-org/synapse/issues/12198)) -- Handle cancellation in `DatabasePool.runInteraction()`. ([\#12199](https://github.com/matrix-org/synapse/issues/12199)) +- Handle cancellation in `DatabasePool.runInteraction`. ([\#12199](https://github.com/matrix-org/synapse/issues/12199)) - Add missing type hints for cache storage. ([\#12216](https://github.com/matrix-org/synapse/issues/12216)) -- Clean-up logic around rebasing URLs for URL image previews. ([\#12219](https://github.com/matrix-org/synapse/issues/12219)) +- Add missing type hints for storage. ([\#12248](https://github.com/matrix-org/synapse/issues/12248), [\#12255](https://github.com/matrix-org/synapse/issues/12255)) - Add type hints to tests files. ([\#12224](https://github.com/matrix-org/synapse/issues/12224), [\#12240](https://github.com/matrix-org/synapse/issues/12240), [\#12256](https://github.com/matrix-org/synapse/issues/12256)) +- Use type stubs for `psycopg2`. ([\#12269](https://github.com/matrix-org/synapse/issues/12269)) +- Improve type annotations for `execute_values`. ([\#12311](https://github.com/matrix-org/synapse/issues/12311)) +- Clean-up logic around rebasing URLs for URL image previews. ([\#12219](https://github.com/matrix-org/synapse/issues/12219)) - Use the `ignored_users` table in additional places instead of re-parsing the account data. ([\#12225](https://github.com/matrix-org/synapse/issues/12225)) - Refactor the relations endpoints to add a `RelationsHandler`. ([\#12237](https://github.com/matrix-org/synapse/issues/12237)) - Generate announcement links in the release script. ([\#12242](https://github.com/matrix-org/synapse/issues/12242)) - Improve error message when dependencies check finds a broken installation. ([\#12244](https://github.com/matrix-org/synapse/issues/12244)) -- Add missing type hints for storage. ([\#12248](https://github.com/matrix-org/synapse/issues/12248), [\#12255](https://github.com/matrix-org/synapse/issues/12255)) - Compress metrics HTTP resource when enabled. Contributed by Nick @ Beeper. ([\#12258](https://github.com/matrix-org/synapse/issues/12258)) -- Refuse to start if DB has non-`C` locale, unless config flag `allow_unsafe_db_locale` is set to true. ([\#12262](https://github.com/matrix-org/synapse/issues/12262), [\#12288](https://github.com/matrix-org/synapse/issues/12288)) +- Refuse to start if the PostgreSQL database has a non-`C` locale, unless the config flag `allow_unsafe_db_locale` is set to true. ([\#12262](https://github.com/matrix-org/synapse/issues/12262), [\#12288](https://github.com/matrix-org/synapse/issues/12288)) - Optionally include account validity expiration information to experimental [MSC3720](https://github.com/matrix-org/matrix-doc/pull/3720) account status responses. ([\#12266](https://github.com/matrix-org/synapse/issues/12266)) -- Use type stubs for `psycopg2`. ([\#12269](https://github.com/matrix-org/synapse/issues/12269)) - Add a new cache `_get_membership_from_event_id` to speed up push rule calculations in large rooms. ([\#12272](https://github.com/matrix-org/synapse/issues/12272)) - Re-enable Complement concurrency in CI. ([\#12283](https://github.com/matrix-org/synapse/issues/12283)) - Remove unused test utilities. ([\#12291](https://github.com/matrix-org/synapse/issues/12291)) - Enhance logging for inbound federation events. ([\#12301](https://github.com/matrix-org/synapse/issues/12301)) -- Improve type annotations for `execute_values`. ([\#12311](https://github.com/matrix-org/synapse/issues/12311)) - Fix compatibility with the recently-released Jinja 3.1. ([\#12313](https://github.com/matrix-org/synapse/issues/12313)) - Avoid trying to calculate the state at outlier events. ([\#12314](https://github.com/matrix-org/synapse/issues/12314)) From d653f6fbec1d8ee4b1c685a2496d9e845dfb3fd5 Mon Sep 17 00:00:00 2001 From: Brendan Abolivier Date: Tue, 29 Mar 2022 11:21:11 +0100 Subject: [PATCH 03/55] Split changelogs --- CHANGES-pre-1.0.md | 3640 +++++++++++++++++++++++++++++++++++++++++++ CHANGES.md | 3642 -------------------------------------------- 2 files changed, 3640 insertions(+), 3642 deletions(-) create mode 100644 CHANGES-pre-1.0.md diff --git a/CHANGES-pre-1.0.md b/CHANGES-pre-1.0.md new file mode 100644 index 0000000000..bcd33d2256 --- /dev/null +++ b/CHANGES-pre-1.0.md @@ -0,0 +1,3640 @@ +Synapse 0.99.5.2 (2019-05-30) +============================= + +Bugfixes +-------- + +- Fix bug where we leaked extremities when we soft failed events, leading to performance degradation. ([\#5274](https://github.com/matrix-org/synapse/issues/5274), [\#5278](https://github.com/matrix-org/synapse/issues/5278), [\#5291](https://github.com/matrix-org/synapse/issues/5291)) + + +Synapse 0.99.5.1 (2019-05-22) +============================= + +0.99.5.1 supersedes 0.99.5 due to malformed debian changelog - no functional changes. + +Synapse 0.99.5 (2019-05-22) +=========================== + +No significant changes. + + +Synapse 0.99.5rc1 (2019-05-21) +============================== + +Features +-------- + +- Add ability to blacklist IP ranges for the federation client. ([\#5043](https://github.com/matrix-org/synapse/issues/5043)) +- Ratelimiting configuration for clients sending messages and the federation server has been altered to match login ratelimiting. The old configuration names will continue working. Check the sample config for details of the new names. ([\#5181](https://github.com/matrix-org/synapse/issues/5181)) +- Drop support for the undocumented /_matrix/client/v2_alpha API prefix. ([\#5190](https://github.com/matrix-org/synapse/issues/5190)) +- Add an option to disable per-room profiles. ([\#5196](https://github.com/matrix-org/synapse/issues/5196)) +- Stick an expiration date to any registered user missing one at startup if account validity is enabled. ([\#5204](https://github.com/matrix-org/synapse/issues/5204)) +- Add experimental support for relations (aka reactions and edits). ([\#5209](https://github.com/matrix-org/synapse/issues/5209), [\#5211](https://github.com/matrix-org/synapse/issues/5211), [\#5203](https://github.com/matrix-org/synapse/issues/5203), [\#5212](https://github.com/matrix-org/synapse/issues/5212)) +- Add a room version 4 which uses a new event ID format, as per [MSC2002](https://github.com/matrix-org/matrix-doc/pull/2002). ([\#5210](https://github.com/matrix-org/synapse/issues/5210), [\#5217](https://github.com/matrix-org/synapse/issues/5217)) + + +Bugfixes +-------- + +- Fix image orientation when generating thumbnails (needs pillow>=4.3.0). Contributed by Pau Rodriguez-Estivill. ([\#5039](https://github.com/matrix-org/synapse/issues/5039)) +- Exclude soft-failed events from forward-extremity candidates: fixes "No forward extremities left!" error. ([\#5146](https://github.com/matrix-org/synapse/issues/5146)) +- Re-order stages in registration flows such that msisdn and email verification are done last. ([\#5174](https://github.com/matrix-org/synapse/issues/5174)) +- Fix 3pid guest invites. ([\#5177](https://github.com/matrix-org/synapse/issues/5177)) +- Fix a bug where the register endpoint would fail with M_THREEPID_IN_USE instead of returning an account previously registered in the same session. ([\#5187](https://github.com/matrix-org/synapse/issues/5187)) +- Prevent registration for user ids that are too long to fit into a state key. Contributed by Reid Anderson. ([\#5198](https://github.com/matrix-org/synapse/issues/5198)) +- Fix incompatibility between ACME support and Python 3.5.2. ([\#5218](https://github.com/matrix-org/synapse/issues/5218)) +- Fix error handling for rooms whose versions are unknown. ([\#5219](https://github.com/matrix-org/synapse/issues/5219)) + + +Internal Changes +---------------- + +- Make /sync attempt to return device updates for both joined and invited users. Note that this doesn't currently work correctly due to other bugs. ([\#3484](https://github.com/matrix-org/synapse/issues/3484)) +- Update tests to consistently be configured via the same code that is used when loading from configuration files. ([\#5171](https://github.com/matrix-org/synapse/issues/5171), [\#5185](https://github.com/matrix-org/synapse/issues/5185)) +- Allow client event serialization to be async. ([\#5183](https://github.com/matrix-org/synapse/issues/5183)) +- Expose DataStore._get_events as get_events_as_list. ([\#5184](https://github.com/matrix-org/synapse/issues/5184)) +- Make generating SQL bounds for pagination generic. ([\#5191](https://github.com/matrix-org/synapse/issues/5191)) +- Stop telling people to install the optional dependencies by default. ([\#5197](https://github.com/matrix-org/synapse/issues/5197)) + + +Synapse 0.99.4 (2019-05-15) +=========================== + +No significant changes. + + +Synapse 0.99.4rc1 (2019-05-13) +============================== + +Features +-------- + +- Add systemd-python to the optional dependencies to enable logging to the systemd journal. Install with `pip install matrix-synapse[systemd]`. ([\#4339](https://github.com/matrix-org/synapse/issues/4339)) +- Add a default .m.rule.tombstone push rule. ([\#4867](https://github.com/matrix-org/synapse/issues/4867)) +- Add ability for password provider modules to bind email addresses to users upon registration. ([\#4947](https://github.com/matrix-org/synapse/issues/4947)) +- Implementation of [MSC1711](https://github.com/matrix-org/matrix-doc/pull/1711) including config options for requiring valid TLS certificates for federation traffic, the ability to disable TLS validation for specific domains, and the ability to specify your own list of CA certificates. ([\#4967](https://github.com/matrix-org/synapse/issues/4967)) +- Remove presence list support as per MSC 1819. ([\#4989](https://github.com/matrix-org/synapse/issues/4989)) +- Reduce CPU usage starting pushers during start up. ([\#4991](https://github.com/matrix-org/synapse/issues/4991)) +- Add a delete group admin API. ([\#5002](https://github.com/matrix-org/synapse/issues/5002)) +- Add config option to block users from looking up 3PIDs. ([\#5010](https://github.com/matrix-org/synapse/issues/5010)) +- Add context to phonehome stats. ([\#5020](https://github.com/matrix-org/synapse/issues/5020)) +- Configure the example systemd units to have a log identifier of `matrix-synapse` + instead of the executable name, `python`. + Contributed by Christoph Müller. ([\#5023](https://github.com/matrix-org/synapse/issues/5023)) +- Add time-based account expiration. ([\#5027](https://github.com/matrix-org/synapse/issues/5027), [\#5047](https://github.com/matrix-org/synapse/issues/5047), [\#5073](https://github.com/matrix-org/synapse/issues/5073), [\#5116](https://github.com/matrix-org/synapse/issues/5116)) +- Add support for handling `/versions`, `/voip` and `/push_rules` client endpoints to client_reader worker. ([\#5063](https://github.com/matrix-org/synapse/issues/5063), [\#5065](https://github.com/matrix-org/synapse/issues/5065), [\#5070](https://github.com/matrix-org/synapse/issues/5070)) +- Add a configuration option to require authentication on /publicRooms and /profile endpoints. ([\#5083](https://github.com/matrix-org/synapse/issues/5083)) +- Move admin APIs to `/_synapse/admin/v1`. (The old paths are retained for backwards-compatibility, for now). ([\#5119](https://github.com/matrix-org/synapse/issues/5119)) +- Implement an admin API for sending server notices. Many thanks to @krombel who provided a foundation for this work. ([\#5121](https://github.com/matrix-org/synapse/issues/5121), [\#5142](https://github.com/matrix-org/synapse/issues/5142)) + + +Bugfixes +-------- + +- Avoid redundant URL encoding of redirect URL for SSO login in the fallback login page. Fixes a regression introduced in [#4220](https://github.com/matrix-org/synapse/pull/4220). Contributed by Marcel Fabian Krüger ("[zaugin](https://github.com/zauguin)"). ([\#4555](https://github.com/matrix-org/synapse/issues/4555)) +- Fix bug where presence updates were sent to all servers in a room when a new server joined, rather than to just the new server. ([\#4942](https://github.com/matrix-org/synapse/issues/4942), [\#5103](https://github.com/matrix-org/synapse/issues/5103)) +- Fix sync bug which made accepting invites unreliable in worker-mode synapses. ([\#4955](https://github.com/matrix-org/synapse/issues/4955), [\#4956](https://github.com/matrix-org/synapse/issues/4956)) +- start.sh: Fix the --no-rate-limit option for messages and make it bypass rate limit on registration and login too. ([\#4981](https://github.com/matrix-org/synapse/issues/4981)) +- Transfer related groups on room upgrade. ([\#4990](https://github.com/matrix-org/synapse/issues/4990)) +- Prevent the ability to kick users from a room they aren't in. ([\#4999](https://github.com/matrix-org/synapse/issues/4999)) +- Fix issue #4596 so synapse_port_db script works with --curses option on Python 3. Contributed by Anders Jensen-Waud . ([\#5003](https://github.com/matrix-org/synapse/issues/5003)) +- Clients timing out/disappearing while downloading from the media repository will now no longer log a spurious "Producer was not unregistered" message. ([\#5009](https://github.com/matrix-org/synapse/issues/5009)) +- Fix "cannot import name execute_batch" error with postgres. ([\#5032](https://github.com/matrix-org/synapse/issues/5032)) +- Fix disappearing exceptions in manhole. ([\#5035](https://github.com/matrix-org/synapse/issues/5035)) +- Workaround bug in twisted where attempting too many concurrent DNS requests could cause it to hang due to running out of file descriptors. ([\#5037](https://github.com/matrix-org/synapse/issues/5037)) +- Make sure we're not registering the same 3pid twice on registration. ([\#5071](https://github.com/matrix-org/synapse/issues/5071)) +- Don't crash on lack of expiry templates. ([\#5077](https://github.com/matrix-org/synapse/issues/5077)) +- Fix the ratelimiting on third party invites. ([\#5104](https://github.com/matrix-org/synapse/issues/5104)) +- Add some missing limitations to room alias creation. ([\#5124](https://github.com/matrix-org/synapse/issues/5124), [\#5128](https://github.com/matrix-org/synapse/issues/5128)) +- Limit the number of EDUs in transactions to 100 as expected by synapse. Thanks to @superboum for this work! ([\#5138](https://github.com/matrix-org/synapse/issues/5138)) + +Internal Changes +---------------- + +- Add test to verify threepid auth check added in #4435. ([\#4474](https://github.com/matrix-org/synapse/issues/4474)) +- Fix/improve some docstrings in the replication code. ([\#4949](https://github.com/matrix-org/synapse/issues/4949)) +- Split synapse.replication.tcp.streams into smaller files. ([\#4953](https://github.com/matrix-org/synapse/issues/4953)) +- Refactor replication row generation/parsing. ([\#4954](https://github.com/matrix-org/synapse/issues/4954)) +- Run `black` to clean up formatting on `synapse/storage/roommember.py` and `synapse/storage/events.py`. ([\#4959](https://github.com/matrix-org/synapse/issues/4959)) +- Remove log line for password via the admin API. ([\#4965](https://github.com/matrix-org/synapse/issues/4965)) +- Fix typo in TLS filenames in docker/README.md. Also add the '-p' commandline option to the 'docker run' example. Contributed by Jurrie Overgoor. ([\#4968](https://github.com/matrix-org/synapse/issues/4968)) +- Refactor room version definitions. ([\#4969](https://github.com/matrix-org/synapse/issues/4969)) +- Reduce log level of .well-known/matrix/client responses. ([\#4972](https://github.com/matrix-org/synapse/issues/4972)) +- Add `config.signing_key_path` that can be read by `synapse.config` utility. ([\#4974](https://github.com/matrix-org/synapse/issues/4974)) +- Track which identity server is used when binding a threepid and use that for unbinding, as per MSC1915. ([\#4982](https://github.com/matrix-org/synapse/issues/4982)) +- Rewrite KeyringTestCase as a HomeserverTestCase. ([\#4985](https://github.com/matrix-org/synapse/issues/4985)) +- README updates: Corrected the default POSTGRES_USER. Added port forwarding hint in TLS section. ([\#4987](https://github.com/matrix-org/synapse/issues/4987)) +- Remove a number of unused tables from the database schema. ([\#4992](https://github.com/matrix-org/synapse/issues/4992), [\#5028](https://github.com/matrix-org/synapse/issues/5028), [\#5033](https://github.com/matrix-org/synapse/issues/5033)) +- Run `black` on the remainder of `synapse/storage/`. ([\#4996](https://github.com/matrix-org/synapse/issues/4996)) +- Fix grammar in get_current_users_in_room and give it a docstring. ([\#4998](https://github.com/matrix-org/synapse/issues/4998)) +- Clean up some code in the server-key Keyring. ([\#5001](https://github.com/matrix-org/synapse/issues/5001)) +- Convert SYNAPSE_NO_TLS Docker variable to boolean for user friendliness. Contributed by Gabriel Eckerson. ([\#5005](https://github.com/matrix-org/synapse/issues/5005)) +- Refactor synapse.storage._base._simple_select_list_paginate. ([\#5007](https://github.com/matrix-org/synapse/issues/5007)) +- Store the notary server name correctly in server_keys_json. ([\#5024](https://github.com/matrix-org/synapse/issues/5024)) +- Rewrite Datastore.get_server_verify_keys to reduce the number of database transactions. ([\#5030](https://github.com/matrix-org/synapse/issues/5030)) +- Remove extraneous period from copyright headers. ([\#5046](https://github.com/matrix-org/synapse/issues/5046)) +- Update documentation for where to get Synapse packages. ([\#5067](https://github.com/matrix-org/synapse/issues/5067)) +- Add workarounds for pep-517 install errors. ([\#5098](https://github.com/matrix-org/synapse/issues/5098)) +- Improve logging when event-signature checks fail. ([\#5100](https://github.com/matrix-org/synapse/issues/5100)) +- Factor out an "assert_requester_is_admin" function. ([\#5120](https://github.com/matrix-org/synapse/issues/5120)) +- Remove the requirement to authenticate for /admin/server_version. ([\#5122](https://github.com/matrix-org/synapse/issues/5122)) +- Prevent an exception from being raised in a IResolutionReceiver and use a more generic error message for blacklisted URL previews. ([\#5155](https://github.com/matrix-org/synapse/issues/5155)) +- Run `black` on the tests directory. ([\#5170](https://github.com/matrix-org/synapse/issues/5170)) +- Fix CI after new release of isort. ([\#5179](https://github.com/matrix-org/synapse/issues/5179)) +- Fix bogus imports in unit tests. ([\#5154](https://github.com/matrix-org/synapse/issues/5154)) + + +Synapse 0.99.3.2 (2019-05-03) +============================= + +Internal Changes +---------------- + +- Ensure that we have `urllib3` <1.25, to resolve incompatibility with `requests`. ([\#5135](https://github.com/matrix-org/synapse/issues/5135)) + + +Synapse 0.99.3.1 (2019-05-03) +============================= + +Security update +--------------- + +This release includes two security fixes: + +- Switch to using a cryptographically-secure random number generator for token strings, ensuring they cannot be predicted by an attacker. Thanks to @opnsec for identifying and responsibly disclosing this issue! ([\#5133](https://github.com/matrix-org/synapse/issues/5133)) +- Blacklist 0.0.0.0 and :: by default for URL previews. Thanks to @opnsec for identifying and responsibly disclosing this issue too! ([\#5134](https://github.com/matrix-org/synapse/issues/5134)) + +Synapse 0.99.3 (2019-04-01) +=========================== + +No significant changes. + + +Synapse 0.99.3rc1 (2019-03-27) +============================== + +Features +-------- + +- The user directory has been rewritten to make it faster, with less chance of falling behind on a large server. ([\#4537](https://github.com/matrix-org/synapse/issues/4537), [\#4846](https://github.com/matrix-org/synapse/issues/4846), [\#4864](https://github.com/matrix-org/synapse/issues/4864), [\#4887](https://github.com/matrix-org/synapse/issues/4887), [\#4900](https://github.com/matrix-org/synapse/issues/4900), [\#4944](https://github.com/matrix-org/synapse/issues/4944)) +- Add configurable rate limiting to the /register endpoint. ([\#4735](https://github.com/matrix-org/synapse/issues/4735), [\#4804](https://github.com/matrix-org/synapse/issues/4804)) +- Move server key queries to federation reader. ([\#4757](https://github.com/matrix-org/synapse/issues/4757)) +- Add support for /account/3pid REST endpoint to client_reader worker. ([\#4759](https://github.com/matrix-org/synapse/issues/4759)) +- Add an endpoint to the admin API for querying the server version. Contributed by Joseph Weston. ([\#4772](https://github.com/matrix-org/synapse/issues/4772)) +- Include a default configuration file in the 'docs' directory. ([\#4791](https://github.com/matrix-org/synapse/issues/4791), [\#4801](https://github.com/matrix-org/synapse/issues/4801)) +- Synapse is now permissive about trailing slashes on some of its federation endpoints, allowing zero or more to be present. ([\#4793](https://github.com/matrix-org/synapse/issues/4793)) +- Add support for /keys/query and /keys/changes REST endpoints to client_reader worker. ([\#4796](https://github.com/matrix-org/synapse/issues/4796)) +- Add checks to incoming events over federation for events evading auth (aka "soft fail"). ([\#4814](https://github.com/matrix-org/synapse/issues/4814)) +- Add configurable rate limiting to the /login endpoint. ([\#4821](https://github.com/matrix-org/synapse/issues/4821), [\#4865](https://github.com/matrix-org/synapse/issues/4865)) +- Remove trailing slashes from certain outbound federation requests. Retry if receiving a 404. Context: #3622. ([\#4840](https://github.com/matrix-org/synapse/issues/4840)) +- Allow passing --daemonize flags to workers in the same way as with master. ([\#4853](https://github.com/matrix-org/synapse/issues/4853)) +- Batch up outgoing read-receipts to reduce federation traffic. ([\#4890](https://github.com/matrix-org/synapse/issues/4890), [\#4927](https://github.com/matrix-org/synapse/issues/4927)) +- Add option to disable searching the user directory. ([\#4895](https://github.com/matrix-org/synapse/issues/4895)) +- Add option to disable searching of local and remote public room lists. ([\#4896](https://github.com/matrix-org/synapse/issues/4896)) +- Add ability for password providers to login/register a user via 3PID (email, phone). ([\#4931](https://github.com/matrix-org/synapse/issues/4931)) + + +Bugfixes +-------- + +- Fix a bug where media with spaces in the name would get a corrupted name. ([\#2090](https://github.com/matrix-org/synapse/issues/2090)) +- Fix attempting to paginate in rooms where server cannot see any events, to avoid unnecessarily pulling in lots of redacted events. ([\#4699](https://github.com/matrix-org/synapse/issues/4699)) +- 'event_id' is now a required parameter in federated state requests, as per the matrix spec. ([\#4740](https://github.com/matrix-org/synapse/issues/4740)) +- Fix tightloop over connecting to replication server. ([\#4749](https://github.com/matrix-org/synapse/issues/4749)) +- Fix parsing of Content-Disposition headers on remote media requests and URL previews. ([\#4763](https://github.com/matrix-org/synapse/issues/4763)) +- Fix incorrect log about not persisting duplicate state event. ([\#4776](https://github.com/matrix-org/synapse/issues/4776)) +- Fix v4v6 option in HAProxy example config. Contributed by Flakebi. ([\#4790](https://github.com/matrix-org/synapse/issues/4790)) +- Handle batch updates in worker replication protocol. ([\#4792](https://github.com/matrix-org/synapse/issues/4792)) +- Fix bug where we didn't correctly throttle sending of USER_IP commands over replication. ([\#4818](https://github.com/matrix-org/synapse/issues/4818)) +- Fix potential race in handling missing updates in device list updates. ([\#4829](https://github.com/matrix-org/synapse/issues/4829)) +- Fix bug where synapse expected an un-specced `prev_state` field on state events. ([\#4837](https://github.com/matrix-org/synapse/issues/4837)) +- Transfer a user's notification settings (push rules) on room upgrade. ([\#4838](https://github.com/matrix-org/synapse/issues/4838)) +- fix test_auto_create_auto_join_where_no_consent. ([\#4886](https://github.com/matrix-org/synapse/issues/4886)) +- Fix a bug where hs_disabled_message was sometimes not correctly enforced. ([\#4888](https://github.com/matrix-org/synapse/issues/4888)) +- Fix bug in shutdown room admin API where it would fail if a user in the room hadn't consented to the privacy policy. ([\#4904](https://github.com/matrix-org/synapse/issues/4904)) +- Fix bug where blocked world-readable rooms were still peekable. ([\#4908](https://github.com/matrix-org/synapse/issues/4908)) + + +Internal Changes +---------------- + +- Add a systemd setup that supports synapse workers. Contributed by Luca Corbatto. ([\#4662](https://github.com/matrix-org/synapse/issues/4662)) +- Change from TravisCI to Buildkite for CI. ([\#4752](https://github.com/matrix-org/synapse/issues/4752)) +- When presence is disabled don't send over replication. ([\#4757](https://github.com/matrix-org/synapse/issues/4757)) +- Minor docstring fixes for MatrixFederationAgent. ([\#4765](https://github.com/matrix-org/synapse/issues/4765)) +- Optimise EDU transmission for the federation_sender worker. ([\#4770](https://github.com/matrix-org/synapse/issues/4770)) +- Update test_typing to use HomeserverTestCase. ([\#4771](https://github.com/matrix-org/synapse/issues/4771)) +- Update URLs for riot.im icons and logos in the default notification templates. ([\#4779](https://github.com/matrix-org/synapse/issues/4779)) +- Removed unnecessary $ from some federation endpoint path regexes. ([\#4794](https://github.com/matrix-org/synapse/issues/4794)) +- Remove link to deleted title in README. ([\#4795](https://github.com/matrix-org/synapse/issues/4795)) +- Clean up read-receipt handling. ([\#4797](https://github.com/matrix-org/synapse/issues/4797)) +- Add some debug about processing read receipts. ([\#4798](https://github.com/matrix-org/synapse/issues/4798)) +- Clean up some replication code. ([\#4799](https://github.com/matrix-org/synapse/issues/4799)) +- Add some docstrings. ([\#4815](https://github.com/matrix-org/synapse/issues/4815)) +- Add debug logger to try and track down #4422. ([\#4816](https://github.com/matrix-org/synapse/issues/4816)) +- Make shutdown API send explanation message to room after users have been forced joined. ([\#4817](https://github.com/matrix-org/synapse/issues/4817)) +- Update example_log_config.yaml. ([\#4820](https://github.com/matrix-org/synapse/issues/4820)) +- Document the `generate` option for the docker image. ([\#4824](https://github.com/matrix-org/synapse/issues/4824)) +- Fix check-newsfragment for debian-only changes. ([\#4825](https://github.com/matrix-org/synapse/issues/4825)) +- Add some debug logging for device list updates to help with #4828. ([\#4828](https://github.com/matrix-org/synapse/issues/4828)) +- Improve federation documentation, specifically .well-known support. Many thanks to @vaab. ([\#4832](https://github.com/matrix-org/synapse/issues/4832)) +- Disable captcha registration by default in unit tests. ([\#4839](https://github.com/matrix-org/synapse/issues/4839)) +- Add stuff back to the .gitignore. ([\#4843](https://github.com/matrix-org/synapse/issues/4843)) +- Clarify what registration_shared_secret allows for. ([\#4844](https://github.com/matrix-org/synapse/issues/4844)) +- Correctly log expected errors when fetching server keys. ([\#4847](https://github.com/matrix-org/synapse/issues/4847)) +- Update install docs to explicitly state a full-chain (not just the top-level) TLS certificate must be provided to Synapse. This caused some people's Synapse ports to appear correct in a browser but still (rightfully so) upset the federation tester. ([\#4849](https://github.com/matrix-org/synapse/issues/4849)) +- Move client read-receipt processing to federation sender worker. ([\#4852](https://github.com/matrix-org/synapse/issues/4852)) +- Refactor federation TransactionQueue. ([\#4855](https://github.com/matrix-org/synapse/issues/4855)) +- Comment out most options in the generated config. ([\#4863](https://github.com/matrix-org/synapse/issues/4863)) +- Fix yaml library warnings by using safe_load. ([\#4869](https://github.com/matrix-org/synapse/issues/4869)) +- Update Apache setup to remove location syntax. Thanks to @cwmke! ([\#4870](https://github.com/matrix-org/synapse/issues/4870)) +- Reinstate test case that runs unit tests against oldest supported dependencies. ([\#4879](https://github.com/matrix-org/synapse/issues/4879)) +- Update link to federation docs. ([\#4881](https://github.com/matrix-org/synapse/issues/4881)) +- fix test_auto_create_auto_join_where_no_consent. ([\#4886](https://github.com/matrix-org/synapse/issues/4886)) +- Use a regular HomeServerConfig object for unit tests rater than a Mock. ([\#4889](https://github.com/matrix-org/synapse/issues/4889)) +- Add some notes about tuning postgres for larger deployments. ([\#4895](https://github.com/matrix-org/synapse/issues/4895)) +- Add a config option for torture-testing worker replication. ([\#4902](https://github.com/matrix-org/synapse/issues/4902)) +- Log requests which are simulated by the unit tests. ([\#4905](https://github.com/matrix-org/synapse/issues/4905)) +- Allow newsfragments to end with exclamation marks. Exciting! ([\#4912](https://github.com/matrix-org/synapse/issues/4912)) +- Refactor some more tests to use HomeserverTestCase. ([\#4913](https://github.com/matrix-org/synapse/issues/4913)) +- Refactor out the state deltas portion of the user directory store and handler. ([\#4917](https://github.com/matrix-org/synapse/issues/4917)) +- Fix nginx example in ACME doc. ([\#4923](https://github.com/matrix-org/synapse/issues/4923)) +- Use an explicit dbname for postgres connections in the tests. ([\#4928](https://github.com/matrix-org/synapse/issues/4928)) +- Fix `ClientReplicationStreamProtocol.__str__()`. ([\#4929](https://github.com/matrix-org/synapse/issues/4929)) + + +Synapse 0.99.2 (2019-03-01) +=========================== + +Features +-------- + +- Added an HAProxy example in the reverse proxy documentation. Contributed by Benoît S. (“Benpro”). ([\#4541](https://github.com/matrix-org/synapse/issues/4541)) +- Add basic optional sentry integration. ([\#4632](https://github.com/matrix-org/synapse/issues/4632), [\#4694](https://github.com/matrix-org/synapse/issues/4694)) +- Transfer bans on room upgrade. ([\#4642](https://github.com/matrix-org/synapse/issues/4642)) +- Add configurable room list publishing rules. ([\#4647](https://github.com/matrix-org/synapse/issues/4647)) +- Support .well-known delegation when issuing certificates through ACME. ([\#4652](https://github.com/matrix-org/synapse/issues/4652)) +- Allow registration and login to be handled by a worker instance. ([\#4666](https://github.com/matrix-org/synapse/issues/4666), [\#4670](https://github.com/matrix-org/synapse/issues/4670), [\#4682](https://github.com/matrix-org/synapse/issues/4682)) +- Reduce the overhead of creating outbound federation connections over TLS by caching the TLS client options. ([\#4674](https://github.com/matrix-org/synapse/issues/4674)) +- Add prometheus metrics for number of outgoing EDUs, by type. ([\#4695](https://github.com/matrix-org/synapse/issues/4695)) +- Return correct error code when inviting a remote user to a room whose homeserver does not support the room version. ([\#4721](https://github.com/matrix-org/synapse/issues/4721)) +- Prevent showing rooms to other servers that were set to not federate. ([\#4746](https://github.com/matrix-org/synapse/issues/4746)) + + +Bugfixes +-------- + +- Fix possible exception when paginating. ([\#4263](https://github.com/matrix-org/synapse/issues/4263)) +- The dependency checker now correctly reports a version mismatch for optional + dependencies, instead of reporting the dependency missing. ([\#4450](https://github.com/matrix-org/synapse/issues/4450)) +- Set CORS headers on .well-known requests. ([\#4651](https://github.com/matrix-org/synapse/issues/4651)) +- Fix kicking guest users on guest access revocation in worker mode. ([\#4667](https://github.com/matrix-org/synapse/issues/4667)) +- Fix an issue in the database migration script where the + `e2e_room_keys.is_verified` column wasn't considered as + a boolean. ([\#4680](https://github.com/matrix-org/synapse/issues/4680)) +- Fix TaskStopped exceptions in logs when outbound requests time out. ([\#4690](https://github.com/matrix-org/synapse/issues/4690)) +- Fix ACME config for python 2. ([\#4717](https://github.com/matrix-org/synapse/issues/4717)) +- Fix paginating over federation persisting incorrect state. ([\#4718](https://github.com/matrix-org/synapse/issues/4718)) + + +Internal Changes +---------------- + +- Run `black` to reformat user directory code. ([\#4635](https://github.com/matrix-org/synapse/issues/4635)) +- Reduce number of exceptions we log. ([\#4643](https://github.com/matrix-org/synapse/issues/4643), [\#4668](https://github.com/matrix-org/synapse/issues/4668)) +- Introduce upsert batching functionality in the database layer. ([\#4644](https://github.com/matrix-org/synapse/issues/4644)) +- Fix various spelling mistakes. ([\#4657](https://github.com/matrix-org/synapse/issues/4657)) +- Cleanup request exception logging. ([\#4669](https://github.com/matrix-org/synapse/issues/4669), [\#4737](https://github.com/matrix-org/synapse/issues/4737), [\#4738](https://github.com/matrix-org/synapse/issues/4738)) +- Improve replication performance by reducing cache invalidation traffic. ([\#4671](https://github.com/matrix-org/synapse/issues/4671), [\#4715](https://github.com/matrix-org/synapse/issues/4715), [\#4748](https://github.com/matrix-org/synapse/issues/4748)) +- Test against Postgres 9.5 as well as 9.4. ([\#4676](https://github.com/matrix-org/synapse/issues/4676)) +- Run unit tests against python 3.7. ([\#4677](https://github.com/matrix-org/synapse/issues/4677)) +- Attempt to clarify installation instructions/config. ([\#4681](https://github.com/matrix-org/synapse/issues/4681)) +- Clean up gitignores. ([\#4688](https://github.com/matrix-org/synapse/issues/4688)) +- Minor tweaks to acme docs. ([\#4689](https://github.com/matrix-org/synapse/issues/4689)) +- Improve the logging in the pusher process. ([\#4691](https://github.com/matrix-org/synapse/issues/4691)) +- Better checks on newsfragments. ([\#4698](https://github.com/matrix-org/synapse/issues/4698), [\#4750](https://github.com/matrix-org/synapse/issues/4750)) +- Avoid some redundant work when processing read receipts. ([\#4706](https://github.com/matrix-org/synapse/issues/4706)) +- Run `push_receipts_to_remotes` as background job. ([\#4707](https://github.com/matrix-org/synapse/issues/4707)) +- Add prometheus metrics for number of badge update pushes. ([\#4709](https://github.com/matrix-org/synapse/issues/4709)) +- Reduce pusher logging on startup ([\#4716](https://github.com/matrix-org/synapse/issues/4716)) +- Don't log exceptions when failing to fetch remote server keys. ([\#4722](https://github.com/matrix-org/synapse/issues/4722)) +- Correctly proxy exception in frontend_proxy worker. ([\#4723](https://github.com/matrix-org/synapse/issues/4723)) +- Add database version to phonehome stats. ([\#4753](https://github.com/matrix-org/synapse/issues/4753)) + + +Synapse 0.99.1.1 (2019-02-14) +============================= + +Bugfixes +-------- + +- Fix "TypeError: '>' not supported" when starting without an existing certificate. + Fix a bug where an existing certificate would be reprovisoned every day. ([\#4648](https://github.com/matrix-org/synapse/issues/4648)) + + +Synapse 0.99.1 (2019-02-14) +=========================== + +Features +-------- + +- Include m.room.encryption on invites by default ([\#3902](https://github.com/matrix-org/synapse/issues/3902)) +- Federation OpenID listener resource can now be activated even if federation is disabled ([\#4420](https://github.com/matrix-org/synapse/issues/4420)) +- Synapse's ACME support will now correctly reprovision a certificate that approaches its expiry while Synapse is running. ([\#4522](https://github.com/matrix-org/synapse/issues/4522)) +- Add ability to update backup versions ([\#4580](https://github.com/matrix-org/synapse/issues/4580)) +- Allow the "unavailable" presence status for /sync. + This change makes Synapse compliant with r0.4.0 of the Client-Server specification. ([\#4592](https://github.com/matrix-org/synapse/issues/4592)) +- There is no longer any need to specify `no_tls`: it is inferred from the absence of TLS listeners ([\#4613](https://github.com/matrix-org/synapse/issues/4613), [\#4615](https://github.com/matrix-org/synapse/issues/4615), [\#4617](https://github.com/matrix-org/synapse/issues/4617), [\#4636](https://github.com/matrix-org/synapse/issues/4636)) +- The default configuration no longer requires TLS certificates. ([\#4614](https://github.com/matrix-org/synapse/issues/4614)) + + +Bugfixes +-------- + +- Copy over room federation ability on room upgrade. ([\#4530](https://github.com/matrix-org/synapse/issues/4530)) +- Fix noisy "twisted.internet.task.TaskStopped" errors in logs ([\#4546](https://github.com/matrix-org/synapse/issues/4546)) +- Synapse is now tolerant of the `tls_fingerprints` option being None or not specified. ([\#4589](https://github.com/matrix-org/synapse/issues/4589)) +- Fix 'no unique or exclusion constraint' error ([\#4591](https://github.com/matrix-org/synapse/issues/4591)) +- Transfer Server ACLs on room upgrade. ([\#4608](https://github.com/matrix-org/synapse/issues/4608)) +- Fix failure to start when not TLS certificate was given even if TLS was disabled. ([\#4618](https://github.com/matrix-org/synapse/issues/4618)) +- Fix self-signed cert notice from generate-config. ([\#4625](https://github.com/matrix-org/synapse/issues/4625)) +- Fix performance of `user_ips` table deduplication background update ([\#4626](https://github.com/matrix-org/synapse/issues/4626), [\#4627](https://github.com/matrix-org/synapse/issues/4627)) + + +Internal Changes +---------------- + +- Change the user directory state query to use a filtered call to the db instead of a generic one. ([\#4462](https://github.com/matrix-org/synapse/issues/4462)) +- Reject federation transactions if they include more than 50 PDUs or 100 EDUs. ([\#4513](https://github.com/matrix-org/synapse/issues/4513)) +- Reduce duplication of ``synapse.app`` code. ([\#4567](https://github.com/matrix-org/synapse/issues/4567)) +- Fix docker upload job to push -py2 images. ([\#4576](https://github.com/matrix-org/synapse/issues/4576)) +- Add port configuration information to ACME instructions. ([\#4578](https://github.com/matrix-org/synapse/issues/4578)) +- Update MSC1711 FAQ to calrify .well-known usage ([\#4584](https://github.com/matrix-org/synapse/issues/4584)) +- Clean up default listener configuration ([\#4586](https://github.com/matrix-org/synapse/issues/4586)) +- Clarifications for reverse proxy docs ([\#4607](https://github.com/matrix-org/synapse/issues/4607)) +- Move ClientTLSOptionsFactory init out of `refresh_certificates` ([\#4611](https://github.com/matrix-org/synapse/issues/4611)) +- Fail cleanly if listener config lacks a 'port' ([\#4616](https://github.com/matrix-org/synapse/issues/4616)) +- Remove redundant entries from docker config ([\#4619](https://github.com/matrix-org/synapse/issues/4619)) +- README updates ([\#4621](https://github.com/matrix-org/synapse/issues/4621)) + + +Synapse 0.99.0 (2019-02-05) +=========================== + +Synapse v0.99.x is a precursor to the upcoming Synapse v1.0 release. It contains foundational changes to room architecture and the federation security model necessary to support the upcoming r0 release of the Server to Server API. + +Features +-------- + +- Synapse's cipher string has been updated to require ECDH key exchange. Configuring and generating dh_params is no longer required, and they will be ignored. ([\#4229](https://github.com/matrix-org/synapse/issues/4229)) +- Synapse can now automatically provision TLS certificates via ACME (the protocol used by CAs like Let's Encrypt). ([\#4384](https://github.com/matrix-org/synapse/issues/4384), [\#4492](https://github.com/matrix-org/synapse/issues/4492), [\#4525](https://github.com/matrix-org/synapse/issues/4525), [\#4572](https://github.com/matrix-org/synapse/issues/4572), [\#4564](https://github.com/matrix-org/synapse/issues/4564), [\#4566](https://github.com/matrix-org/synapse/issues/4566), [\#4547](https://github.com/matrix-org/synapse/issues/4547), [\#4557](https://github.com/matrix-org/synapse/issues/4557)) +- Implement MSC1708 (.well-known routing for server-server federation) ([\#4408](https://github.com/matrix-org/synapse/issues/4408), [\#4409](https://github.com/matrix-org/synapse/issues/4409), [\#4426](https://github.com/matrix-org/synapse/issues/4426), [\#4427](https://github.com/matrix-org/synapse/issues/4427), [\#4428](https://github.com/matrix-org/synapse/issues/4428), [\#4464](https://github.com/matrix-org/synapse/issues/4464), [\#4468](https://github.com/matrix-org/synapse/issues/4468), [\#4487](https://github.com/matrix-org/synapse/issues/4487), [\#4488](https://github.com/matrix-org/synapse/issues/4488), [\#4489](https://github.com/matrix-org/synapse/issues/4489), [\#4497](https://github.com/matrix-org/synapse/issues/4497), [\#4511](https://github.com/matrix-org/synapse/issues/4511), [\#4516](https://github.com/matrix-org/synapse/issues/4516), [\#4520](https://github.com/matrix-org/synapse/issues/4520), [\#4521](https://github.com/matrix-org/synapse/issues/4521), [\#4539](https://github.com/matrix-org/synapse/issues/4539), [\#4542](https://github.com/matrix-org/synapse/issues/4542), [\#4544](https://github.com/matrix-org/synapse/issues/4544)) +- Search now includes results from predecessor rooms after a room upgrade. ([\#4415](https://github.com/matrix-org/synapse/issues/4415)) +- Config option to disable requesting MSISDN on registration. ([\#4423](https://github.com/matrix-org/synapse/issues/4423)) +- Add a metric for tracking event stream position of the user directory. ([\#4445](https://github.com/matrix-org/synapse/issues/4445)) +- Support exposing server capabilities in CS API (MSC1753, MSC1804) ([\#4472](https://github.com/matrix-org/synapse/issues/4472), [81b7e7eed](https://github.com/matrix-org/synapse/commit/81b7e7eed323f55d6550e7a270a9dc2c4c7b0fe0))) +- Add support for room version 3 ([\#4483](https://github.com/matrix-org/synapse/issues/4483), [\#4499](https://github.com/matrix-org/synapse/issues/4499), [\#4515](https://github.com/matrix-org/synapse/issues/4515), [\#4523](https://github.com/matrix-org/synapse/issues/4523), [\#4535](https://github.com/matrix-org/synapse/issues/4535)) +- Synapse will now reload TLS certificates from disk upon SIGHUP. ([\#4495](https://github.com/matrix-org/synapse/issues/4495), [\#4524](https://github.com/matrix-org/synapse/issues/4524)) +- The matrixdotorg/synapse Docker images now use Python 3 by default. ([\#4558](https://github.com/matrix-org/synapse/issues/4558)) + +Bugfixes +-------- + +- Prevent users with access tokens predating the introduction of device IDs from creating spurious entries in the user_ips table. ([\#4369](https://github.com/matrix-org/synapse/issues/4369)) +- Fix typo in ALL_USER_TYPES definition to ensure type is a tuple ([\#4392](https://github.com/matrix-org/synapse/issues/4392)) +- Fix high CPU usage due to remote devicelist updates ([\#4397](https://github.com/matrix-org/synapse/issues/4397)) +- Fix potential bug where creating or joining a room could fail ([\#4404](https://github.com/matrix-org/synapse/issues/4404)) +- Fix bug when rejecting remote invites ([\#4405](https://github.com/matrix-org/synapse/issues/4405), [\#4527](https://github.com/matrix-org/synapse/issues/4527)) +- Fix incorrect logcontexts after a Deferred was cancelled ([\#4407](https://github.com/matrix-org/synapse/issues/4407)) +- Ensure encrypted room state is persisted across room upgrades. ([\#4411](https://github.com/matrix-org/synapse/issues/4411)) +- Copy over whether a room is a direct message and any associated room tags on room upgrade. ([\#4412](https://github.com/matrix-org/synapse/issues/4412)) +- Fix None guard in calling config.server.is_threepid_reserved ([\#4435](https://github.com/matrix-org/synapse/issues/4435)) +- Don't send IP addresses as SNI ([\#4452](https://github.com/matrix-org/synapse/issues/4452)) +- Fix UnboundLocalError in post_urlencoded_get_json ([\#4460](https://github.com/matrix-org/synapse/issues/4460)) +- Add a timeout to filtered room directory queries. ([\#4461](https://github.com/matrix-org/synapse/issues/4461)) +- Workaround for login error when using both LDAP and internal authentication. ([\#4486](https://github.com/matrix-org/synapse/issues/4486)) +- Fix a bug where setting a relative consent directory path would cause a crash. ([\#4512](https://github.com/matrix-org/synapse/issues/4512)) + + +Deprecations and Removals +------------------------- + +- Synapse no longer generates self-signed TLS certificates when generating a configuration file. ([\#4509](https://github.com/matrix-org/synapse/issues/4509)) + + +Improved Documentation +---------------------- + +- Update debian installation instructions ([\#4526](https://github.com/matrix-org/synapse/issues/4526)) + + +Internal Changes +---------------- + +- Synapse will now take advantage of native UPSERT functionality in PostgreSQL 9.5+ and SQLite 3.24+. ([\#4306](https://github.com/matrix-org/synapse/issues/4306), [\#4459](https://github.com/matrix-org/synapse/issues/4459), [\#4466](https://github.com/matrix-org/synapse/issues/4466), [\#4471](https://github.com/matrix-org/synapse/issues/4471), [\#4477](https://github.com/matrix-org/synapse/issues/4477), [\#4505](https://github.com/matrix-org/synapse/issues/4505)) +- Update README to use the new virtualenv everywhere ([\#4342](https://github.com/matrix-org/synapse/issues/4342)) +- Add better logging for unexpected errors while sending transactions ([\#4368](https://github.com/matrix-org/synapse/issues/4368)) +- Apply a unique index to the user_ips table, preventing duplicates. ([\#4370](https://github.com/matrix-org/synapse/issues/4370), [\#4432](https://github.com/matrix-org/synapse/issues/4432), [\#4434](https://github.com/matrix-org/synapse/issues/4434)) +- Silence travis-ci build warnings by removing non-functional python3.6 ([\#4377](https://github.com/matrix-org/synapse/issues/4377)) +- Fix a comment in the generated config file ([\#4387](https://github.com/matrix-org/synapse/issues/4387)) +- Add ground work for implementing future federation API versions ([\#4390](https://github.com/matrix-org/synapse/issues/4390)) +- Update dependencies on msgpack and pymacaroons to use the up-to-date packages. ([\#4399](https://github.com/matrix-org/synapse/issues/4399)) +- Tweak codecov settings to make them less loud. ([\#4400](https://github.com/matrix-org/synapse/issues/4400)) +- Implement server support for MSC1794 - Federation v2 Invite API ([\#4402](https://github.com/matrix-org/synapse/issues/4402)) +- debian package: symlink to explicit python version ([\#4433](https://github.com/matrix-org/synapse/issues/4433)) +- Add infrastructure to support different event formats ([\#4437](https://github.com/matrix-org/synapse/issues/4437), [\#4447](https://github.com/matrix-org/synapse/issues/4447), [\#4448](https://github.com/matrix-org/synapse/issues/4448), [\#4470](https://github.com/matrix-org/synapse/issues/4470), [\#4481](https://github.com/matrix-org/synapse/issues/4481), [\#4482](https://github.com/matrix-org/synapse/issues/4482), [\#4493](https://github.com/matrix-org/synapse/issues/4493), [\#4494](https://github.com/matrix-org/synapse/issues/4494), [\#4496](https://github.com/matrix-org/synapse/issues/4496), [\#4510](https://github.com/matrix-org/synapse/issues/4510), [\#4514](https://github.com/matrix-org/synapse/issues/4514)) +- Generate the debian config during build ([\#4444](https://github.com/matrix-org/synapse/issues/4444)) +- Clarify documentation for the `public_baseurl` config param ([\#4458](https://github.com/matrix-org/synapse/issues/4458), [\#4498](https://github.com/matrix-org/synapse/issues/4498)) +- Fix quoting for allowed_local_3pids example config ([\#4476](https://github.com/matrix-org/synapse/issues/4476)) +- Remove deprecated --process-dependency-links option from UPGRADE.rst ([\#4485](https://github.com/matrix-org/synapse/issues/4485)) +- Make it possible to set the log level for tests via an environment variable ([\#4506](https://github.com/matrix-org/synapse/issues/4506)) +- Reduce the log level of linearizer lock acquirement to DEBUG. ([\#4507](https://github.com/matrix-org/synapse/issues/4507)) +- Fix code to comply with linting in PyFlakes 3.7.1. ([\#4519](https://github.com/matrix-org/synapse/issues/4519)) +- Add some debug for membership syncing issues ([\#4538](https://github.com/matrix-org/synapse/issues/4538)) +- Docker: only copy what we need to the build image ([\#4562](https://github.com/matrix-org/synapse/issues/4562)) + + +Synapse 0.34.1.1 (2019-01-11) +============================= + +This release fixes CVE-2019-5885 and is recommended for all users of Synapse 0.34.1. + +This release is compatible with Python 2.7 and 3.5+. Python 3.7 is fully supported. + +Bugfixes +-------- + +- Fix spontaneous logout on upgrade + ([\#4374](https://github.com/matrix-org/synapse/issues/4374)) + + +Synapse 0.34.1 (2019-01-09) +=========================== + +Internal Changes +---------------- + +- Add better logging for unexpected errors while sending transactions ([\#4361](https://github.com/matrix-org/synapse/issues/4361), [\#4362](https://github.com/matrix-org/synapse/issues/4362)) + + +Synapse 0.34.1rc1 (2019-01-08) +============================== + +Features +-------- + +- Special-case a support user for use in verifying behaviour of a given server. The support user does not appear in user directory or monthly active user counts. ([\#4141](https://github.com/matrix-org/synapse/issues/4141), [\#4344](https://github.com/matrix-org/synapse/issues/4344)) +- Support for serving .well-known files ([\#4262](https://github.com/matrix-org/synapse/issues/4262)) +- Rework SAML2 authentication ([\#4265](https://github.com/matrix-org/synapse/issues/4265), [\#4267](https://github.com/matrix-org/synapse/issues/4267)) +- SAML2 authentication: Initialise user display name from SAML2 data ([\#4272](https://github.com/matrix-org/synapse/issues/4272)) +- Synapse can now have its conditional/extra dependencies installed by pip. This functionality can be used by using `pip install matrix-synapse[feature]`, where feature is a comma separated list with the possible values `email.enable_notifs`, `matrix-synapse-ldap3`, `postgres`, `resources.consent`, `saml2`, `url_preview`, and `test`. If you want to install all optional dependencies, you can use "all" instead. ([\#4298](https://github.com/matrix-org/synapse/issues/4298), [\#4325](https://github.com/matrix-org/synapse/issues/4325), [\#4327](https://github.com/matrix-org/synapse/issues/4327)) +- Add routes for reading account data. ([\#4303](https://github.com/matrix-org/synapse/issues/4303)) +- Add opt-in support for v2 rooms ([\#4307](https://github.com/matrix-org/synapse/issues/4307)) +- Add a script to generate a clean config file ([\#4315](https://github.com/matrix-org/synapse/issues/4315)) +- Return server data in /login response ([\#4319](https://github.com/matrix-org/synapse/issues/4319)) + + +Bugfixes +-------- + +- Fix contains_url check to be consistent with other instances in code-base and check that value is an instance of string. ([\#3405](https://github.com/matrix-org/synapse/issues/3405)) +- Fix CAS login when username is not valid in an MXID ([\#4264](https://github.com/matrix-org/synapse/issues/4264)) +- Send CORS headers for /media/config ([\#4279](https://github.com/matrix-org/synapse/issues/4279)) +- Add 'sandbox' to CSP for media reprository ([\#4284](https://github.com/matrix-org/synapse/issues/4284)) +- Make the new landing page prettier. ([\#4294](https://github.com/matrix-org/synapse/issues/4294)) +- Fix deleting E2E room keys when using old SQLite versions. ([\#4295](https://github.com/matrix-org/synapse/issues/4295)) +- The metric synapse_admin_mau:current previously did not update when config.mau_stats_only was set to True ([\#4305](https://github.com/matrix-org/synapse/issues/4305)) +- Fixed per-room account data filters ([\#4309](https://github.com/matrix-org/synapse/issues/4309)) +- Fix indentation in default config ([\#4313](https://github.com/matrix-org/synapse/issues/4313)) +- Fix synapse:latest docker upload ([\#4316](https://github.com/matrix-org/synapse/issues/4316)) +- Fix test_metric.py compatibility with prometheus_client 0.5. Contributed by Maarten de Vries . ([\#4317](https://github.com/matrix-org/synapse/issues/4317)) +- Avoid packaging _trial_temp directory in -py3 debian packages ([\#4326](https://github.com/matrix-org/synapse/issues/4326)) +- Check jinja version for consent resource ([\#4327](https://github.com/matrix-org/synapse/issues/4327)) +- fix NPE in /messages by checking if all events were filtered out ([\#4330](https://github.com/matrix-org/synapse/issues/4330)) +- Fix `python -m synapse.config` on Python 3. ([\#4356](https://github.com/matrix-org/synapse/issues/4356)) + + +Deprecations and Removals +------------------------- + +- Remove the deprecated v1/register API on Python 2. It was never ported to Python 3. ([\#4334](https://github.com/matrix-org/synapse/issues/4334)) + + +Internal Changes +---------------- + +- Getting URL previews of IP addresses no longer fails on Python 3. ([\#4215](https://github.com/matrix-org/synapse/issues/4215)) +- drop undocumented dependency on dateutil ([\#4266](https://github.com/matrix-org/synapse/issues/4266)) +- Update the example systemd config to use a virtualenv ([\#4273](https://github.com/matrix-org/synapse/issues/4273)) +- Update link to kernel DCO guide ([\#4274](https://github.com/matrix-org/synapse/issues/4274)) +- Make isort tox check print diff when it fails ([\#4283](https://github.com/matrix-org/synapse/issues/4283)) +- Log room_id in Unknown room errors ([\#4297](https://github.com/matrix-org/synapse/issues/4297)) +- Documentation improvements for coturn setup. Contributed by Krithin Sitaram. ([\#4333](https://github.com/matrix-org/synapse/issues/4333)) +- Update pull request template to use absolute links ([\#4341](https://github.com/matrix-org/synapse/issues/4341)) +- Update README to not lie about required restart when updating TLS certificates ([\#4343](https://github.com/matrix-org/synapse/issues/4343)) +- Update debian packaging for compatibility with transitional package ([\#4349](https://github.com/matrix-org/synapse/issues/4349)) +- Fix command hint to generate a config file when trying to start without a config file ([\#4353](https://github.com/matrix-org/synapse/issues/4353)) +- Add better logging for unexpected errors while sending transactions ([\#4358](https://github.com/matrix-org/synapse/issues/4358)) + + +Synapse 0.34.0 (2018-12-20) +=========================== + +Synapse 0.34.0 is the first release to fully support Python 3. Synapse will now +run on Python versions 3.5 or 3.6 (as well as 2.7). Support for Python 3.7 +remains experimental. + +We recommend upgrading to Python 3, but make sure to read the [upgrade +notes](docs/upgrade.md#upgrading-to-v0340) when doing so. + +Features +-------- + +- Add 'sandbox' to CSP for media reprository ([\#4284](https://github.com/matrix-org/synapse/issues/4284)) +- Make the new landing page prettier. ([\#4294](https://github.com/matrix-org/synapse/issues/4294)) +- Fix deleting E2E room keys when using old SQLite versions. ([\#4295](https://github.com/matrix-org/synapse/issues/4295)) +- Add a welcome page for the client API port. Credit to @krombel! ([\#4289](https://github.com/matrix-org/synapse/issues/4289)) +- Remove Matrix console from the default distribution ([\#4290](https://github.com/matrix-org/synapse/issues/4290)) +- Add option to track MAU stats (but not limit people) ([\#3830](https://github.com/matrix-org/synapse/issues/3830)) +- Add an option to enable recording IPs for appservice users ([\#3831](https://github.com/matrix-org/synapse/issues/3831)) +- Rename login type `m.login.cas` to `m.login.sso` ([\#4220](https://github.com/matrix-org/synapse/issues/4220)) +- Add an option to disable search for homeservers that may not be interested in it. ([\#4230](https://github.com/matrix-org/synapse/issues/4230)) + + +Bugfixes +-------- + +- Pushrules can now again be made with non-ASCII rule IDs. ([\#4165](https://github.com/matrix-org/synapse/issues/4165)) +- The media repository now no longer fails to decode UTF-8 filenames when downloading remote media. ([\#4176](https://github.com/matrix-org/synapse/issues/4176)) +- URL previews now correctly decode non-UTF-8 text if the header contains a ` synapse ([\#3897](https://github.com/matrix-org/synapse/issues/3897)) +- Increase the timeout when filling missing events in federation requests ([\#3903](https://github.com/matrix-org/synapse/issues/3903)) +- Improve the logging when handling a federation transaction ([\#3904](https://github.com/matrix-org/synapse/issues/3904), [\#3966](https://github.com/matrix-org/synapse/issues/3966)) +- Improve logging of outbound federation requests ([\#3906](https://github.com/matrix-org/synapse/issues/3906), [\#3909](https://github.com/matrix-org/synapse/issues/3909)) +- Fix the docker image building on python 3 ([\#3911](https://github.com/matrix-org/synapse/issues/3911)) +- Add a regression test for logging failed HTTP requests on Python 3. ([\#3912](https://github.com/matrix-org/synapse/issues/3912)) +- Comments and interface cleanup for on_receive_pdu ([\#3924](https://github.com/matrix-org/synapse/issues/3924)) +- Fix spurious exceptions when remote http client closes conncetion ([\#3925](https://github.com/matrix-org/synapse/issues/3925)) +- Log exceptions thrown by background tasks ([\#3927](https://github.com/matrix-org/synapse/issues/3927)) +- Add a cache to get_destination_retry_timings ([\#3933](https://github.com/matrix-org/synapse/issues/3933), [\#3991](https://github.com/matrix-org/synapse/issues/3991)) +- Automate pushes to docker hub ([\#3946](https://github.com/matrix-org/synapse/issues/3946)) +- Require attrs 16.0.0 or later ([\#3947](https://github.com/matrix-org/synapse/issues/3947)) +- Fix incompatibility with python3 on alpine ([\#3948](https://github.com/matrix-org/synapse/issues/3948)) +- Run the test suite on the oldest supported versions of our dependencies in CI. ([\#3952](https://github.com/matrix-org/synapse/issues/3952)) +- CircleCI now only runs merged jobs on PRs, and commit jobs on develop, master, and release branches. ([\#3957](https://github.com/matrix-org/synapse/issues/3957)) +- Fix docstrings and add tests for state store methods ([\#3958](https://github.com/matrix-org/synapse/issues/3958)) +- fix docstring for FederationClient.get_state_for_room ([\#3963](https://github.com/matrix-org/synapse/issues/3963)) +- Run notify_app_services as a bg process ([\#3965](https://github.com/matrix-org/synapse/issues/3965)) +- Clarifications in FederationHandler ([\#3967](https://github.com/matrix-org/synapse/issues/3967)) +- Further reduce the docker image size ([\#3972](https://github.com/matrix-org/synapse/issues/3972)) +- Build py3 docker images for docker hub too ([\#3976](https://github.com/matrix-org/synapse/issues/3976)) +- Updated the installation instructions to point to the matrix-synapse package on PyPI. ([\#3985](https://github.com/matrix-org/synapse/issues/3985)) +- Disable USE_FROZEN_DICTS for unittests by default. ([\#3987](https://github.com/matrix-org/synapse/issues/3987)) +- Remove unused Jenkins and development related files from the repo. ([\#3988](https://github.com/matrix-org/synapse/issues/3988)) +- Improve stacktraces in certain exceptions in the logs ([\#3989](https://github.com/matrix-org/synapse/issues/3989)) + + +Synapse 0.33.5.1 (2018-09-25) +============================= + +Internal Changes +---------------- + +- Fix incompatibility with older Twisted version in tests. Thanks @OlegGirko! ([\#3940](https://github.com/matrix-org/synapse/issues/3940)) + + +Synapse 0.33.5 (2018-09-24) +=========================== + +No significant changes. + + +Synapse 0.33.5rc1 (2018-09-17) +============================== + +Features +-------- + +- Python 3.5 and 3.6 support is now in beta. ([\#3576](https://github.com/matrix-org/synapse/issues/3576)) +- Implement `event_format` filter param in `/sync` ([\#3790](https://github.com/matrix-org/synapse/issues/3790)) +- Add synapse_admin_mau:registered_reserved_users metric to expose number of real reaserved users ([\#3846](https://github.com/matrix-org/synapse/issues/3846)) + + +Bugfixes +-------- + +- Remove connection ID for replication prometheus metrics, as it creates a large number of new series. ([\#3788](https://github.com/matrix-org/synapse/issues/3788)) +- guest users should not be part of mau total ([\#3800](https://github.com/matrix-org/synapse/issues/3800)) +- Bump dependency on pyopenssl 16.x, to avoid incompatibility with recent Twisted. ([\#3804](https://github.com/matrix-org/synapse/issues/3804)) +- Fix existing room tags not coming down sync when joining a room ([\#3810](https://github.com/matrix-org/synapse/issues/3810)) +- Fix jwt import check ([\#3824](https://github.com/matrix-org/synapse/issues/3824)) +- fix VOIP crashes under Python 3 (#3821) ([\#3835](https://github.com/matrix-org/synapse/issues/3835)) +- Fix manhole so that it works with latest openssh clients ([\#3841](https://github.com/matrix-org/synapse/issues/3841)) +- Fix outbound requests occasionally wedging, which can result in federation breaking between servers. ([\#3845](https://github.com/matrix-org/synapse/issues/3845)) +- Show heroes if room name/canonical alias has been deleted ([\#3851](https://github.com/matrix-org/synapse/issues/3851)) +- Fix handling of redacted events from federation ([\#3859](https://github.com/matrix-org/synapse/issues/3859)) +- ([\#3874](https://github.com/matrix-org/synapse/issues/3874)) +- Mitigate outbound federation randomly becoming wedged ([\#3875](https://github.com/matrix-org/synapse/issues/3875)) + + +Internal Changes +---------------- + +- CircleCI tests now run on the potential merge of a PR. ([\#3704](https://github.com/matrix-org/synapse/issues/3704)) +- http/ is now ported to Python 3. ([\#3771](https://github.com/matrix-org/synapse/issues/3771)) +- Improve human readable error messages for threepid registration/account update ([\#3789](https://github.com/matrix-org/synapse/issues/3789)) +- Make /sync slightly faster by avoiding needless copies ([\#3795](https://github.com/matrix-org/synapse/issues/3795)) +- handlers/ is now ported to Python 3. ([\#3803](https://github.com/matrix-org/synapse/issues/3803)) +- Limit the number of PDUs/EDUs per federation transaction ([\#3805](https://github.com/matrix-org/synapse/issues/3805)) +- Only start postgres instance for postgres tests on Travis CI ([\#3806](https://github.com/matrix-org/synapse/issues/3806)) +- tests/ is now ported to Python 3. ([\#3808](https://github.com/matrix-org/synapse/issues/3808)) +- crypto/ is now ported to Python 3. ([\#3822](https://github.com/matrix-org/synapse/issues/3822)) +- rest/ is now ported to Python 3. ([\#3823](https://github.com/matrix-org/synapse/issues/3823)) +- add some logging for the keyring queue ([\#3826](https://github.com/matrix-org/synapse/issues/3826)) +- speed up lazy loading by 2-3x ([\#3827](https://github.com/matrix-org/synapse/issues/3827)) +- Improved Dockerfile to remove build requirements after building reducing the image size. ([\#3834](https://github.com/matrix-org/synapse/issues/3834)) +- Disable lazy loading for incremental syncs for now ([\#3840](https://github.com/matrix-org/synapse/issues/3840)) +- federation/ is now ported to Python 3. ([\#3847](https://github.com/matrix-org/synapse/issues/3847)) +- Log when we retry outbound requests ([\#3853](https://github.com/matrix-org/synapse/issues/3853)) +- Removed some excess logging messages. ([\#3855](https://github.com/matrix-org/synapse/issues/3855)) +- Speed up purge history for rooms that have been previously purged ([\#3856](https://github.com/matrix-org/synapse/issues/3856)) +- Refactor some HTTP timeout code. ([\#3857](https://github.com/matrix-org/synapse/issues/3857)) +- Fix running merged builds on CircleCI ([\#3858](https://github.com/matrix-org/synapse/issues/3858)) +- Fix typo in replication stream exception. ([\#3860](https://github.com/matrix-org/synapse/issues/3860)) +- Add in flight real time metrics for Measure blocks ([\#3871](https://github.com/matrix-org/synapse/issues/3871)) +- Disable buffering and automatic retrying in treq requests to prevent timeouts. ([\#3872](https://github.com/matrix-org/synapse/issues/3872)) +- mention jemalloc in the README ([\#3877](https://github.com/matrix-org/synapse/issues/3877)) +- Remove unmaintained "nuke-room-from-db.sh" script ([\#3888](https://github.com/matrix-org/synapse/issues/3888)) + + +Synapse 0.33.4 (2018-09-07) +=========================== + +Internal Changes +---------------- + +- Unignore synctl in .dockerignore to fix docker builds ([\#3802](https://github.com/matrix-org/synapse/issues/3802)) + + +Synapse 0.33.4rc2 (2018-09-06) +============================== + +Pull in security fixes from v0.33.3.1 + + +Synapse 0.33.3.1 (2018-09-06) +============================= + +SECURITY FIXES +-------------- + +- Fix an issue where event signatures were not always correctly validated ([\#3796](https://github.com/matrix-org/synapse/issues/3796)) +- Fix an issue where server_acls could be circumvented for incoming events ([\#3796](https://github.com/matrix-org/synapse/issues/3796)) + + +Internal Changes +---------------- + +- Unignore synctl in .dockerignore to fix docker builds ([\#3802](https://github.com/matrix-org/synapse/issues/3802)) + + +Synapse 0.33.4rc1 (2018-09-04) +============================== + +Features +-------- + +- Support profile API endpoints on workers ([\#3659](https://github.com/matrix-org/synapse/issues/3659)) +- Server notices for resource limit blocking ([\#3680](https://github.com/matrix-org/synapse/issues/3680)) +- Allow guests to use /rooms/:roomId/event/:eventId ([\#3724](https://github.com/matrix-org/synapse/issues/3724)) +- Add mau_trial_days config param, so that users only get counted as MAU after N days. ([\#3749](https://github.com/matrix-org/synapse/issues/3749)) +- Require twisted 17.1 or later (fixes [#3741](https://github.com/matrix-org/synapse/issues/3741)). ([\#3751](https://github.com/matrix-org/synapse/issues/3751)) + + +Bugfixes +-------- + +- Fix error collecting prometheus metrics when run on dedicated thread due to threading concurrency issues ([\#3722](https://github.com/matrix-org/synapse/issues/3722)) +- Fix bug where we resent "limit exceeded" server notices repeatedly ([\#3747](https://github.com/matrix-org/synapse/issues/3747)) +- Fix bug where we broke sync when using limit_usage_by_mau but hadn't configured server notices ([\#3753](https://github.com/matrix-org/synapse/issues/3753)) +- Fix 'federation_domain_whitelist' such that an empty list correctly blocks all outbound federation traffic ([\#3754](https://github.com/matrix-org/synapse/issues/3754)) +- Fix tagging of server notice rooms ([\#3755](https://github.com/matrix-org/synapse/issues/3755), [\#3756](https://github.com/matrix-org/synapse/issues/3756)) +- Fix 'admin_uri' config variable and error parameter to be 'admin_contact' to match the spec. ([\#3758](https://github.com/matrix-org/synapse/issues/3758)) +- Don't return non-LL-member state in incremental sync state blocks ([\#3760](https://github.com/matrix-org/synapse/issues/3760)) +- Fix bug in sending presence over federation ([\#3768](https://github.com/matrix-org/synapse/issues/3768)) +- Fix bug where preserved threepid user comes to sign up and server is mau blocked ([\#3777](https://github.com/matrix-org/synapse/issues/3777)) + +Internal Changes +---------------- + +- Removed the link to the unmaintained matrix-synapse-auto-deploy project from the readme. ([\#3378](https://github.com/matrix-org/synapse/issues/3378)) +- Refactor state module to support multiple room versions ([\#3673](https://github.com/matrix-org/synapse/issues/3673)) +- The synapse.storage module has been ported to Python 3. ([\#3725](https://github.com/matrix-org/synapse/issues/3725)) +- Split the state_group_cache into member and non-member state events (and so speed up LL /sync) ([\#3726](https://github.com/matrix-org/synapse/issues/3726)) +- Log failure to authenticate remote servers as warnings (without stack traces) ([\#3727](https://github.com/matrix-org/synapse/issues/3727)) +- The CONTRIBUTING guidelines have been updated to mention our use of Markdown and that .misc files have content. ([\#3730](https://github.com/matrix-org/synapse/issues/3730)) +- Reference the need for an HTTP replication port when using the federation_reader worker ([\#3734](https://github.com/matrix-org/synapse/issues/3734)) +- Fix minor spelling error in federation client documentation. ([\#3735](https://github.com/matrix-org/synapse/issues/3735)) +- Remove redundant state resolution function ([\#3737](https://github.com/matrix-org/synapse/issues/3737)) +- The test suite now passes on PostgreSQL. ([\#3740](https://github.com/matrix-org/synapse/issues/3740)) +- Fix MAU cache invalidation due to missing yield ([\#3746](https://github.com/matrix-org/synapse/issues/3746)) +- Make sure that we close db connections opened during init ([\#3764](https://github.com/matrix-org/synapse/issues/3764)) + + +Synapse 0.33.3 (2018-08-22) +=========================== + +Bugfixes +-------- + +- Fix bug introduced in v0.33.3rc1 which made the ToS give a 500 error ([\#3732](https://github.com/matrix-org/synapse/issues/3732)) + + +Synapse 0.33.3rc2 (2018-08-21) +============================== + +Bugfixes +-------- + +- Fix bug in v0.33.3rc1 which caused infinite loops and OOMs ([\#3723](https://github.com/matrix-org/synapse/issues/3723)) + + +Synapse 0.33.3rc1 (2018-08-21) +============================== + +Features +-------- + +- Add support for the SNI extension to federation TLS connections. Thanks to @vojeroen! ([\#3439](https://github.com/matrix-org/synapse/issues/3439)) +- Add /_media/r0/config ([\#3184](https://github.com/matrix-org/synapse/issues/3184)) +- speed up /members API and add `at` and `membership` params as per MSC1227 ([\#3568](https://github.com/matrix-org/synapse/issues/3568)) +- implement `summary` block in /sync response as per MSC688 ([\#3574](https://github.com/matrix-org/synapse/issues/3574)) +- Add lazy-loading support to /messages as per MSC1227 ([\#3589](https://github.com/matrix-org/synapse/issues/3589)) +- Add ability to limit number of monthly active users on the server ([\#3633](https://github.com/matrix-org/synapse/issues/3633)) +- Support more federation endpoints on workers ([\#3653](https://github.com/matrix-org/synapse/issues/3653)) +- Basic support for room versioning ([\#3654](https://github.com/matrix-org/synapse/issues/3654)) +- Ability to disable client/server Synapse via conf toggle ([\#3655](https://github.com/matrix-org/synapse/issues/3655)) +- Ability to whitelist specific threepids against monthly active user limiting ([\#3662](https://github.com/matrix-org/synapse/issues/3662)) +- Add some metrics for the appservice and federation event sending loops ([\#3664](https://github.com/matrix-org/synapse/issues/3664)) +- Where server is disabled, block ability for locked out users to read new messages ([\#3670](https://github.com/matrix-org/synapse/issues/3670)) +- set admin uri via config, to be used in error messages where the user should contact the administrator ([\#3687](https://github.com/matrix-org/synapse/issues/3687)) +- Synapse's presence functionality can now be disabled with the "use_presence" configuration option. ([\#3694](https://github.com/matrix-org/synapse/issues/3694)) +- For resource limit blocked users, prevent writing into rooms ([\#3708](https://github.com/matrix-org/synapse/issues/3708)) + + +Bugfixes +-------- + +- Fix occasional glitches in the synapse_event_persisted_position metric ([\#3658](https://github.com/matrix-org/synapse/issues/3658)) +- Fix bug on deleting 3pid when using identity servers that don't support unbind API ([\#3661](https://github.com/matrix-org/synapse/issues/3661)) +- Make the tests pass on Twisted < 18.7.0 ([\#3676](https://github.com/matrix-org/synapse/issues/3676)) +- Don’t ship recaptcha_ajax.js, use it directly from Google ([\#3677](https://github.com/matrix-org/synapse/issues/3677)) +- Fixes test_reap_monthly_active_users so it passes under postgres ([\#3681](https://github.com/matrix-org/synapse/issues/3681)) +- Fix mau blocking calulation bug on login ([\#3689](https://github.com/matrix-org/synapse/issues/3689)) +- Fix missing yield in synapse.storage.monthly_active_users.initialise_reserved_users ([\#3692](https://github.com/matrix-org/synapse/issues/3692)) +- Improve HTTP request logging to include all requests ([\#3700](https://github.com/matrix-org/synapse/issues/3700)) +- Avoid timing out requests while we are streaming back the response ([\#3701](https://github.com/matrix-org/synapse/issues/3701)) +- Support more federation endpoints on workers ([\#3705](https://github.com/matrix-org/synapse/issues/3705), [\#3713](https://github.com/matrix-org/synapse/issues/3713)) +- Fix "Starting db txn 'get_all_updated_receipts' from sentinel context" warning ([\#3710](https://github.com/matrix-org/synapse/issues/3710)) +- Fix bug where `state_cache` cache factor ignored environment variables ([\#3719](https://github.com/matrix-org/synapse/issues/3719)) + + +Deprecations and Removals +------------------------- + +- The Shared-Secret registration method of the legacy v1/register REST endpoint has been removed. For a replacement, please see [the admin/register API documentation](https://github.com/matrix-org/synapse/blob/master/docs/admin_api/register_api.rst). ([\#3703](https://github.com/matrix-org/synapse/issues/3703)) + + +Internal Changes +---------------- + +- The test suite now can run under PostgreSQL. ([\#3423](https://github.com/matrix-org/synapse/issues/3423)) +- Refactor HTTP replication endpoints to reduce code duplication ([\#3632](https://github.com/matrix-org/synapse/issues/3632)) +- Tests now correctly execute on Python 3. ([\#3647](https://github.com/matrix-org/synapse/issues/3647)) +- Sytests can now be run inside a Docker container. ([\#3660](https://github.com/matrix-org/synapse/issues/3660)) +- Port over enough to Python 3 to allow the sytests to start. ([\#3668](https://github.com/matrix-org/synapse/issues/3668)) +- Update docker base image from alpine 3.7 to 3.8. ([\#3669](https://github.com/matrix-org/synapse/issues/3669)) +- Rename synapse.util.async to synapse.util.async_helpers to mitigate async becoming a keyword on Python 3.7. ([\#3678](https://github.com/matrix-org/synapse/issues/3678)) +- Synapse's tests are now formatted with the black autoformatter. ([\#3679](https://github.com/matrix-org/synapse/issues/3679)) +- Implemented a new testing base class to reduce test boilerplate. ([\#3684](https://github.com/matrix-org/synapse/issues/3684)) +- Rename MAU prometheus metrics ([\#3690](https://github.com/matrix-org/synapse/issues/3690)) +- add new error type ResourceLimit ([\#3707](https://github.com/matrix-org/synapse/issues/3707)) +- Logcontexts for replication command handlers ([\#3709](https://github.com/matrix-org/synapse/issues/3709)) +- Update admin register API documentation to reference a real user ID. ([\#3712](https://github.com/matrix-org/synapse/issues/3712)) + + +Synapse 0.33.2 (2018-08-09) +=========================== + +No significant changes. + + +Synapse 0.33.2rc1 (2018-08-07) +============================== + +Features +-------- + +- add support for the lazy_loaded_members filter as per MSC1227 ([\#2970](https://github.com/matrix-org/synapse/issues/2970)) +- add support for the include_redundant_members filter param as per MSC1227 ([\#3331](https://github.com/matrix-org/synapse/issues/3331)) +- Add metrics to track resource usage by background processes ([\#3553](https://github.com/matrix-org/synapse/issues/3553), [\#3556](https://github.com/matrix-org/synapse/issues/3556), [\#3604](https://github.com/matrix-org/synapse/issues/3604), [\#3610](https://github.com/matrix-org/synapse/issues/3610)) +- Add `code` label to `synapse_http_server_response_time_seconds` prometheus metric ([\#3554](https://github.com/matrix-org/synapse/issues/3554)) +- Add support for client_reader to handle more APIs ([\#3555](https://github.com/matrix-org/synapse/issues/3555), [\#3597](https://github.com/matrix-org/synapse/issues/3597)) +- make the /context API filter & lazy-load aware as per MSC1227 ([\#3567](https://github.com/matrix-org/synapse/issues/3567)) +- Add ability to limit number of monthly active users on the server ([\#3630](https://github.com/matrix-org/synapse/issues/3630)) +- When we fail to join a room over federation, pass the error code back to the client. ([\#3639](https://github.com/matrix-org/synapse/issues/3639)) +- Add a new /admin/register API for non-interactively creating users. ([\#3415](https://github.com/matrix-org/synapse/issues/3415)) + + +Bugfixes +-------- + +- Make /directory/list API return 404 for room not found instead of 400. Thanks to @fuzzmz! ([\#3620](https://github.com/matrix-org/synapse/issues/3620)) +- Default inviter_display_name to mxid for email invites ([\#3391](https://github.com/matrix-org/synapse/issues/3391)) +- Don't generate TURN credentials if no TURN config options are set ([\#3514](https://github.com/matrix-org/synapse/issues/3514)) +- Correctly announce deleted devices over federation ([\#3520](https://github.com/matrix-org/synapse/issues/3520)) +- Catch failures saving metrics captured by Measure, and instead log the faulty metrics information for further analysis. ([\#3548](https://github.com/matrix-org/synapse/issues/3548)) +- Unicode passwords are now normalised before hashing, preventing the instance where two different devices or browsers might send a different UTF-8 sequence for the password. ([\#3569](https://github.com/matrix-org/synapse/issues/3569)) +- Fix potential stack overflow and deadlock under heavy load ([\#3570](https://github.com/matrix-org/synapse/issues/3570)) +- Respond with M_NOT_FOUND when profiles are not found locally or over federation. Fixes #3585 ([\#3585](https://github.com/matrix-org/synapse/issues/3585)) +- Fix failure to persist events over federation under load ([\#3601](https://github.com/matrix-org/synapse/issues/3601)) +- Fix updating of cached remote profiles ([\#3605](https://github.com/matrix-org/synapse/issues/3605)) +- Fix 'tuple index out of range' error ([\#3607](https://github.com/matrix-org/synapse/issues/3607)) +- Only import secrets when available (fix for py < 3.6) ([\#3626](https://github.com/matrix-org/synapse/issues/3626)) + + +Internal Changes +---------------- + +- Remove redundant checks on who_forgot_in_room ([\#3350](https://github.com/matrix-org/synapse/issues/3350)) +- Remove unnecessary event re-signing hacks ([\#3367](https://github.com/matrix-org/synapse/issues/3367)) +- Rewrite cache list decorator ([\#3384](https://github.com/matrix-org/synapse/issues/3384)) +- Move v1-only REST APIs into their own module. ([\#3460](https://github.com/matrix-org/synapse/issues/3460)) +- Replace more instances of Python 2-only iteritems and itervalues uses. ([\#3562](https://github.com/matrix-org/synapse/issues/3562)) +- Refactor EventContext to accept state during init ([\#3577](https://github.com/matrix-org/synapse/issues/3577)) +- Improve Dockerfile and docker-compose instructions ([\#3543](https://github.com/matrix-org/synapse/issues/3543)) +- Release notes are now in the Markdown format. ([\#3552](https://github.com/matrix-org/synapse/issues/3552)) +- add config for pep8 ([\#3559](https://github.com/matrix-org/synapse/issues/3559)) +- Merge Linearizer and Limiter ([\#3571](https://github.com/matrix-org/synapse/issues/3571), [\#3572](https://github.com/matrix-org/synapse/issues/3572)) +- Lazily load state on master process when using workers to reduce DB consumption ([\#3579](https://github.com/matrix-org/synapse/issues/3579), [\#3581](https://github.com/matrix-org/synapse/issues/3581), [\#3582](https://github.com/matrix-org/synapse/issues/3582), [\#3584](https://github.com/matrix-org/synapse/issues/3584)) +- Fixes and optimisations for resolve_state_groups ([\#3586](https://github.com/matrix-org/synapse/issues/3586)) +- Improve logging for exceptions when handling PDUs ([\#3587](https://github.com/matrix-org/synapse/issues/3587)) +- Add some measure blocks to persist_events ([\#3590](https://github.com/matrix-org/synapse/issues/3590)) +- Fix some random logcontext leaks. ([\#3591](https://github.com/matrix-org/synapse/issues/3591), [\#3606](https://github.com/matrix-org/synapse/issues/3606)) +- Speed up calculating state deltas in persist_event loop ([\#3592](https://github.com/matrix-org/synapse/issues/3592)) +- Attempt to reduce amount of state pulled out of DB during persist_events ([\#3595](https://github.com/matrix-org/synapse/issues/3595)) +- Fix a documentation typo in on_make_leave_request ([\#3609](https://github.com/matrix-org/synapse/issues/3609)) +- Make EventStore inherit from EventFederationStore ([\#3612](https://github.com/matrix-org/synapse/issues/3612)) +- Remove some redundant joins on event_edges.room_id ([\#3613](https://github.com/matrix-org/synapse/issues/3613)) +- Stop populating events.content ([\#3614](https://github.com/matrix-org/synapse/issues/3614)) +- Update the /send_leave path registration to use event_id rather than a transaction ID. ([\#3616](https://github.com/matrix-org/synapse/issues/3616)) +- Refactor FederationHandler to move DB writes into separate functions ([\#3621](https://github.com/matrix-org/synapse/issues/3621)) +- Remove unused field "pdu_failures" from transactions. ([\#3628](https://github.com/matrix-org/synapse/issues/3628)) +- rename replication_layer to federation_client ([\#3634](https://github.com/matrix-org/synapse/issues/3634)) +- Factor out exception handling in federation_client ([\#3638](https://github.com/matrix-org/synapse/issues/3638)) +- Refactor location of docker build script. ([\#3644](https://github.com/matrix-org/synapse/issues/3644)) +- Update CONTRIBUTING to mention newsfragments. ([\#3645](https://github.com/matrix-org/synapse/issues/3645)) + + +Synapse 0.33.1 (2018-08-02) +=========================== + +SECURITY FIXES +-------------- + +- Fix a potential issue where servers could request events for rooms they have not joined. ([\#3641](https://github.com/matrix-org/synapse/issues/3641)) +- Fix a potential issue where users could see events in private rooms before they joined. ([\#3642](https://github.com/matrix-org/synapse/issues/3642)) + +Synapse 0.33.0 (2018-07-19) +=========================== + +Bugfixes +-------- + +- Disable a noisy warning about logcontexts. ([\#3561](https://github.com/matrix-org/synapse/issues/3561)) + +Synapse 0.33.0rc1 (2018-07-18) +============================== + +Features +-------- + +- Enforce the specified API for report\_event. ([\#3316](https://github.com/matrix-org/synapse/issues/3316)) +- Include CPU time from database threads in request/block metrics. ([\#3496](https://github.com/matrix-org/synapse/issues/3496), [\#3501](https://github.com/matrix-org/synapse/issues/3501)) +- Add CPU metrics for \_fetch\_event\_list. ([\#3497](https://github.com/matrix-org/synapse/issues/3497)) +- Optimisation to make handling incoming federation requests more efficient. ([\#3541](https://github.com/matrix-org/synapse/issues/3541)) + +Bugfixes +-------- + +- Fix a significant performance regression in /sync. ([\#3505](https://github.com/matrix-org/synapse/issues/3505), [\#3521](https://github.com/matrix-org/synapse/issues/3521), [\#3530](https://github.com/matrix-org/synapse/issues/3530), [\#3544](https://github.com/matrix-org/synapse/issues/3544)) +- Use more portable syntax in our use of the attrs package, widening the supported versions. ([\#3498](https://github.com/matrix-org/synapse/issues/3498)) +- Fix queued federation requests being processed in the wrong order. ([\#3533](https://github.com/matrix-org/synapse/issues/3533)) +- Ensure that erasure requests are correctly honoured for publicly accessible rooms when accessed over federation. ([\#3546](https://github.com/matrix-org/synapse/issues/3546)) + +Misc +---- + +- Refactoring to improve testability. ([\#3351](https://github.com/matrix-org/synapse/issues/3351), [\#3499](https://github.com/matrix-org/synapse/issues/3499)) +- Use `isort` to sort imports. ([\#3463](https://github.com/matrix-org/synapse/issues/3463), [\#3464](https://github.com/matrix-org/synapse/issues/3464), [\#3540](https://github.com/matrix-org/synapse/issues/3540)) +- Use parse and asserts from http.servlet. ([\#3534](https://github.com/matrix-org/synapse/issues/3534), [\#3535](https://github.com/matrix-org/synapse/issues/3535)). + +Synapse 0.32.2 (2018-07-07) +=========================== + +Bugfixes +-------- + +- Amend the Python dependencies to depend on attrs from PyPI, not attr ([\#3492](https://github.com/matrix-org/synapse/issues/3492)) + +Synapse 0.32.1 (2018-07-06) +=========================== + +Bugfixes +-------- + +- Add explicit dependency on netaddr ([\#3488](https://github.com/matrix-org/synapse/issues/3488)) + +Changes in synapse v0.32.0 (2018-07-06) +======================================= + +No changes since 0.32.0rc1 + +Synapse 0.32.0rc1 (2018-07-05) +============================== + +Features +-------- + +- Add blacklist & whitelist of servers allowed to send events to a room via `m.room.server_acl` event. +- Cache factor override system for specific caches ([\#3334](https://github.com/matrix-org/synapse/issues/3334)) +- Add metrics to track appservice transactions ([\#3344](https://github.com/matrix-org/synapse/issues/3344)) +- Try to log more helpful info when a sig verification fails ([\#3372](https://github.com/matrix-org/synapse/issues/3372)) +- Synapse now uses the best performing JSON encoder/decoder according to your runtime (simplejson on CPython, stdlib json on PyPy). ([\#3462](https://github.com/matrix-org/synapse/issues/3462)) +- Add optional ip\_range\_whitelist param to AS registration files to lock AS IP access ([\#3465](https://github.com/matrix-org/synapse/issues/3465)) +- Reject invalid server names in federation requests ([\#3480](https://github.com/matrix-org/synapse/issues/3480)) +- Reject invalid server names in homeserver.yaml ([\#3483](https://github.com/matrix-org/synapse/issues/3483)) + +Bugfixes +-------- + +- Strip access\_token from outgoing requests ([\#3327](https://github.com/matrix-org/synapse/issues/3327)) +- Redact AS tokens in logs ([\#3349](https://github.com/matrix-org/synapse/issues/3349)) +- Fix federation backfill from SQLite servers ([\#3355](https://github.com/matrix-org/synapse/issues/3355)) +- Fix event-purge-by-ts admin API ([\#3363](https://github.com/matrix-org/synapse/issues/3363)) +- Fix event filtering in get\_missing\_events handler ([\#3371](https://github.com/matrix-org/synapse/issues/3371)) +- Synapse is now stricter regarding accepting events which it cannot retrieve the prev\_events for. ([\#3456](https://github.com/matrix-org/synapse/issues/3456)) +- Fix bug where synapse would explode when receiving unicode in HTTP User-Agent header ([\#3470](https://github.com/matrix-org/synapse/issues/3470)) +- Invalidate cache on correct thread to avoid race ([\#3473](https://github.com/matrix-org/synapse/issues/3473)) + +Improved Documentation +---------------------- + +- `doc/postgres.rst`: fix display of the last command block. Thanks to @ArchangeGabriel! ([\#3340](https://github.com/matrix-org/synapse/issues/3340)) + +Deprecations and Removals +------------------------- + +- Remove was\_forgotten\_at ([\#3324](https://github.com/matrix-org/synapse/issues/3324)) + +Misc +---- + +- [\#3332](https://github.com/matrix-org/synapse/issues/3332), [\#3341](https://github.com/matrix-org/synapse/issues/3341), [\#3347](https://github.com/matrix-org/synapse/issues/3347), [\#3348](https://github.com/matrix-org/synapse/issues/3348), [\#3356](https://github.com/matrix-org/synapse/issues/3356), [\#3385](https://github.com/matrix-org/synapse/issues/3385), [\#3446](https://github.com/matrix-org/synapse/issues/3446), [\#3447](https://github.com/matrix-org/synapse/issues/3447), [\#3467](https://github.com/matrix-org/synapse/issues/3467), [\#3474](https://github.com/matrix-org/synapse/issues/3474) + +Changes in synapse v0.31.2 (2018-06-14) +======================================= + +SECURITY UPDATE: Prevent unauthorised users from setting state events in a room when there is no `m.room.power_levels` event in force in the room. (PR #3397) + +Discussion around the Matrix Spec change proposal for this change can be followed at . + +Changes in synapse v0.31.1 (2018-06-08) +======================================= + +v0.31.1 fixes a security bug in the `get_missing_events` federation API where event visibility rules were not applied correctly. + +We are not aware of it being actively exploited but please upgrade asap. + +Bug Fixes: + +- Fix event filtering in get\_missing\_events handler (PR #3371) + +Changes in synapse v0.31.0 (2018-06-06) +======================================= + +Most notable change from v0.30.0 is to switch to the python prometheus library to improve system stats reporting. WARNING: this changes a number of prometheus metrics in a backwards-incompatible manner. For more details, see [docs/metrics-howto.rst](docs/metrics-howto.rst#removal-of-deprecated-metrics--time-based-counters-becoming-histograms-in-0310). + +Bug Fixes: + +- Fix metric documentation tables (PR #3341) +- Fix LaterGauge error handling (694968f) +- Fix replication metrics (b7e7fd2) + +Changes in synapse v0.31.0-rc1 (2018-06-04) +=========================================== + +Features: + +- Switch to the Python Prometheus library (PR #3256, #3274) +- Let users leave the server notice room after joining (PR #3287) + +Changes: + +- daily user type phone home stats (PR #3264) +- Use iter\* methods for \_filter\_events\_for\_server (PR #3267) +- Docs on consent bits (PR #3268) +- Remove users from user directory on deactivate (PR #3277) +- Avoid sending consent notice to guest users (PR #3288) +- disable CPUMetrics if no /proc/self/stat (PR #3299) +- Consistently use six\'s iteritems and wrap lazy keys/values in list() if they\'re not meant to be lazy (PR #3307) +- Add private IPv6 addresses to example config for url preview blacklist (PR #3317) Thanks to @thegcat! +- Reduce stuck read-receipts: ignore depth when updating (PR #3318) +- Put python\'s logs into Trial when running unit tests (PR #3319) + +Changes, python 3 migration: + +- Replace some more comparisons with six (PR #3243) Thanks to @NotAFile! +- replace some iteritems with six (PR #3244) Thanks to @NotAFile! +- Add batch\_iter to utils (PR #3245) Thanks to @NotAFile! +- use repr, not str (PR #3246) Thanks to @NotAFile! +- Misc Python3 fixes (PR #3247) Thanks to @NotAFile! +- Py3 storage/\_base.py (PR #3278) Thanks to @NotAFile! +- more six iteritems (PR #3279) Thanks to @NotAFile! +- More Misc. py3 fixes (PR #3280) Thanks to @NotAFile! +- remaining isintance fixes (PR #3281) Thanks to @NotAFile! +- py3-ize state.py (PR #3283) Thanks to @NotAFile! +- extend tox testing for py3 to avoid regressions (PR #3302) Thanks to @krombel! +- use memoryview in py3 (PR #3303) Thanks to @NotAFile! + +Bugs: + +- Fix federation backfill bugs (PR #3261) +- federation: fix LaterGauge usage (PR #3328) Thanks to @intelfx! + +Changes in synapse v0.30.0 (2018-05-24) +======================================= + +\'Server Notices\' are a new feature introduced in Synapse 0.30. They provide a channel whereby server administrators can send messages to users on the server. + +They are used as part of communication of the server policies (see `docs/consent_tracking.md`), however the intention is that they may also find a use for features such as \"Message of the day\". + +This feature is specific to Synapse, but uses standard Matrix communication mechanisms, so should work with any Matrix client. For more details see `docs/server_notices.md` + +Further Server Notices/Consent Tracking Support: + +- Allow overriding the server\_notices user\'s avatar (PR #3273) +- Use the localpart in the consent uri (PR #3272) +- Support for putting %(consent\_uri)s in messages (PR #3271) +- Block attempts to send server notices to remote users (PR #3270) +- Docs on consent bits (PR #3268) + +Changes in synapse v0.30.0-rc1 (2018-05-23) +=========================================== + +Server Notices/Consent Tracking Support: + +- ConsentResource to gather policy consent from users (PR #3213) +- Move RoomCreationHandler out of synapse.handlers.Handlers (PR #3225) +- Infrastructure for a server notices room (PR #3232) +- Send users a server notice about consent (PR #3236) +- Reject attempts to send event before privacy consent is given (PR #3257) +- Add a \'has\_consented\' template var to consent forms (PR #3262) +- Fix dependency on jinja2 (PR #3263) + +Features: + +- Cohort analytics (PR #3163, #3241, #3251) +- Add lxml to docker image for web previews (PR #3239) Thanks to @ptman! +- Add in flight request metrics (PR #3252) + +Changes: + +- Remove unused update\_external\_syncs (PR #3233) +- Use stream rather depth ordering for push actions (PR #3212) +- Make purge\_history operate on tokens (PR #3221) +- Don\'t support limitless pagination (PR #3265) + +Bug Fixes: + +- Fix logcontext resource usage tracking (PR #3258) +- Fix error in handling receipts (PR #3235) +- Stop the transaction cache caching failures (PR #3255) + +Changes in synapse v0.29.1 (2018-05-17) +======================================= + +Changes: + +- Update docker documentation (PR #3222) + +Changes in synapse v0.29.0 (2018-05-16) +======================================= + +Not changes since v0.29.0-rc1 + +Changes in synapse v0.29.0-rc1 (2018-05-14) +=========================================== + +Notable changes, a docker file for running Synapse (Thanks to @kaiyou!) and a closed spec bug in the Client Server API. Additionally further prep for Python 3 migration. + +Potentially breaking change: + +- Make Client-Server API return 401 for invalid token (PR #3161). + + This changes the Client-server spec to return a 401 error code instead of 403 when the access token is unrecognised. This is the behaviour required by the specification, but some clients may be relying on the old, incorrect behaviour. + + Thanks to @NotAFile for fixing this. + +Features: + +- Add a Dockerfile for synapse (PR #2846) Thanks to @kaiyou! + +Changes - General: + +- nuke-room-from-db.sh: added postgresql option and help (PR #2337) Thanks to @rubo77! +- Part user from rooms on account deactivate (PR #3201) +- Make \'unexpected logging context\' into warnings (PR #3007) +- Set Server header in SynapseRequest (PR #3208) +- remove duplicates from groups tables (PR #3129) +- Improve exception handling for background processes (PR #3138) +- Add missing consumeErrors to improve exception handling (PR #3139) +- reraise exceptions more carefully (PR #3142) +- Remove redundant call to preserve\_fn (PR #3143) +- Trap exceptions thrown within run\_in\_background (PR #3144) + +Changes - Refactors: + +- Refactor /context to reuse pagination storage functions (PR #3193) +- Refactor recent events func to use pagination func (PR #3195) +- Refactor pagination DB API to return concrete type (PR #3196) +- Refactor get\_recent\_events\_for\_room return type (PR #3198) +- Refactor sync APIs to reuse pagination API (PR #3199) +- Remove unused code path from member change DB func (PR #3200) +- Refactor request handling wrappers (PR #3203) +- transaction\_id, destination defined twice (PR #3209) Thanks to @damir-manapov! +- Refactor event storage to prepare for changes in state calculations (PR #3141) +- Set Server header in SynapseRequest (PR #3208) +- Use deferred.addTimeout instead of time\_bound\_deferred (PR #3127, #3178) +- Use run\_in\_background in preference to preserve\_fn (PR #3140) + +Changes - Python 3 migration: + +- Construct HMAC as bytes on py3 (PR #3156) Thanks to @NotAFile! +- run config tests on py3 (PR #3159) Thanks to @NotAFile! +- Open certificate files as bytes (PR #3084) Thanks to @NotAFile! +- Open config file in non-bytes mode (PR #3085) Thanks to @NotAFile! +- Make event properties raise AttributeError instead (PR #3102) Thanks to @NotAFile! +- Use six.moves.urlparse (PR #3108) Thanks to @NotAFile! +- Add py3 tests to tox with folders that work (PR #3145) Thanks to @NotAFile! +- Don\'t yield in list comprehensions (PR #3150) Thanks to @NotAFile! +- Move more xrange to six (PR #3151) Thanks to @NotAFile! +- make imports local (PR #3152) Thanks to @NotAFile! +- move httplib import to six (PR #3153) Thanks to @NotAFile! +- Replace stringIO imports with six (PR #3154, #3168) Thanks to @NotAFile! +- more bytes strings (PR #3155) Thanks to @NotAFile! + +Bug Fixes: + +- synapse fails to start under Twisted \>= 18.4 (PR #3157) +- Fix a class of logcontext leaks (PR #3170) +- Fix a couple of logcontext leaks in unit tests (PR #3172) +- Fix logcontext leak in media repo (PR #3174) +- Escape label values in prometheus metrics (PR #3175, #3186) +- Fix \'Unhandled Error\' logs with Twisted 18.4 (PR #3182) Thanks to @Half-Shot! +- Fix logcontext leaks in rate limiter (PR #3183) +- notifications: Convert next\_token to string according to the spec (PR #3190) Thanks to @mujx! +- nuke-room-from-db.sh: fix deletion from search table (PR #3194) Thanks to @rubo77! +- add guard for None on purge\_history api (PR #3160) Thanks to @krombel! + +Changes in synapse v0.28.1 (2018-05-01) +======================================= + +SECURITY UPDATE + +- Clamp the allowed values of event depth received over federation to be \[0, 2\^63 - 1\]. This mitigates an attack where malicious events injected with depth = 2\^63 - 1 render rooms unusable. Depth is used to determine the cosmetic ordering of events within a room, and so the ordering of events in such a room will default to using stream\_ordering rather than depth (topological\_ordering). + + This is a temporary solution to mitigate abuse in the wild, whilst a long term solution is being implemented to improve how the depth parameter is used. + + Full details at + +- Pin Twisted to \<18.4 until we stop using the private \_OpenSSLECCurve API. + +Changes in synapse v0.28.0 (2018-04-26) +======================================= + +Bug Fixes: + +- Fix quarantine media admin API and search reindex (PR #3130) +- Fix media admin APIs (PR #3134) + +Changes in synapse v0.28.0-rc1 (2018-04-24) +=========================================== + +Minor performance improvement to federation sending and bug fixes. + +(Note: This release does not include the delta state resolution implementation discussed in matrix live) + +Features: + +- Add metrics for event processing lag (PR #3090) +- Add metrics for ResponseCache (PR #3092) + +Changes: + +- Synapse on PyPy (PR #2760) Thanks to @Valodim! +- move handling of auto\_join\_rooms to RegisterHandler (PR #2996) Thanks to @krombel! +- Improve handling of SRV records for federation connections (PR #3016) Thanks to @silkeh! +- Document the behaviour of ResponseCache (PR #3059) +- Preparation for py3 (PR #3061, #3073, #3074, #3075, #3103, #3104, #3106, #3107, #3109, #3110) Thanks to @NotAFile! +- update prometheus dashboard to use new metric names (PR #3069) Thanks to @krombel! +- use python3-compatible prints (PR #3074) Thanks to @NotAFile! +- Send federation events concurrently (PR #3078) +- Limit concurrent event sends for a room (PR #3079) +- Improve R30 stat definition (PR #3086) +- Send events to ASes concurrently (PR #3088) +- Refactor ResponseCache usage (PR #3093) +- Clarify that SRV may not point to a CNAME (PR #3100) Thanks to @silkeh! +- Use str(e) instead of e.message (PR #3103) Thanks to @NotAFile! +- Use six.itervalues in some places (PR #3106) Thanks to @NotAFile! +- Refactor store.have\_events (PR #3117) + +Bug Fixes: + +- Return 401 for invalid access\_token on logout (PR #2938) Thanks to @dklug! +- Return a 404 rather than a 500 on rejoining empty rooms (PR #3080) +- fix federation\_domain\_whitelist (PR #3099) +- Avoid creating events with huge numbers of prev\_events (PR #3113) +- Reject events which have lots of prev\_events (PR #3118) + +Changes in synapse v0.27.4 (2018-04-13) +======================================= + +Changes: + +- Update canonicaljson dependency (\#3095) + +Changes in synapse v0.27.3 (2018-04-11) +====================================== + +Bug fixes: + +- URL quote path segments over federation (\#3082) + +Changes in synapse v0.27.3-rc2 (2018-04-09) +=========================================== + +v0.27.3-rc1 used a stale version of the develop branch so the changelog overstates the functionality. v0.27.3-rc2 is up to date, rc1 should be ignored. + +Changes in synapse v0.27.3-rc1 (2018-04-09) +=========================================== + +Notable changes include API support for joinability of groups. Also new metrics and phone home stats. Phone home stats include better visibility of system usage so we can tweak synpase to work better for all users rather than our own experience with matrix.org. Also, recording \'r30\' stat which is the measure we use to track overal growth of the Matrix ecosystem. It is defined as:- + +Counts the number of native 30 day retained users, defined as:- \* Users who have created their accounts more than 30 days + +: - Where last seen at most 30 days ago + - Where account creation and last\_seen are \> 30 days\" + +Features: + +- Add joinability for groups (PR #3045) +- Implement group join API (PR #3046) +- Add counter metrics for calculating state delta (PR #3033) +- R30 stats (PR #3041) +- Measure time it takes to calculate state group ID (PR #3043) +- Add basic performance statistics to phone home (PR #3044) +- Add response size metrics (PR #3071) +- phone home cache size configurations (PR #3063) + +Changes: + +- Add a blurb explaining the main synapse worker (PR #2886) Thanks to @turt2live! +- Replace old style error catching with \'as\' keyword (PR #3000) Thanks to @NotAFile! +- Use .iter\* to avoid copies in StateHandler (PR #3006) +- Linearize calls to \_generate\_user\_id (PR #3029) +- Remove last usage of ujson (PR #3030) +- Use simplejson throughout (PR #3048) +- Use static JSONEncoders (PR #3049) +- Remove uses of events.content (PR #3060) +- Improve database cache performance (PR #3068) + +Bug fixes: + +- Add room\_id to the response of rooms/{roomId}/join (PR #2986) Thanks to @jplatte! +- Fix replication after switch to simplejson (PR #3015) +- 404 correctly on missing paths via NoResource (PR #3022) +- Fix error when claiming e2e keys from offline servers (PR #3034) +- fix tests/storage/test\_user\_directory.py (PR #3042) +- use PUT instead of POST for federating groups/m.join\_policy (PR #3070) Thanks to @krombel! +- postgres port script: fix state\_groups\_pkey error (PR #3072) + +Changes in synapse v0.27.2 (2018-03-26) +======================================= + +Bug fixes: + +- Fix bug which broke TCP replication between workers (PR #3015) + +Changes in synapse v0.27.1 (2018-03-26) +======================================= + +Meta release as v0.27.0 temporarily pointed to the wrong commit + +Changes in synapse v0.27.0 (2018-03-26) +======================================= + +No changes since v0.27.0-rc2 + +Changes in synapse v0.27.0-rc2 (2018-03-19) +=========================================== + +Pulls in v0.26.1 + +Bug fixes: + +- Fix bug introduced in v0.27.0-rc1 that causes much increased memory usage in state cache (PR #3005) + +Changes in synapse v0.26.1 (2018-03-15) +======================================= + +Bug fixes: + +- Fix bug where an invalid event caused server to stop functioning correctly, due to parsing and serializing bugs in ujson library (PR #3008) + +Changes in synapse v0.27.0-rc1 (2018-03-14) +=========================================== + +The common case for running Synapse is not to run separate workers, but for those that do, be aware that synctl no longer starts the main synapse when using `-a` option with workers. A new worker file should be added with `worker_app: synapse.app.homeserver`. + +This release also begins the process of renaming a number of the metrics reported to prometheus. See [docs/metrics-howto.rst](docs/metrics-howto.rst#block-and-response-metrics-renamed-for-0-27-0). Note that the v0.28.0 release will remove the deprecated metric names. + +Features: + +- Add ability for ASes to override message send time (PR #2754) +- Add support for custom storage providers for media repository (PR #2867, #2777, #2783, #2789, #2791, #2804, #2812, #2814, #2857, #2868, #2767) +- Add purge API features, see [docs/admin\_api/purge\_history\_api.rst](docs/admin_api/purge_history_api.rst) for full details (PR #2858, #2867, #2882, #2946, #2962, #2943) +- Add support for whitelisting 3PIDs that users can register. (PR #2813) +- Add `/room/{id}/event/{id}` API (PR #2766) +- Add an admin API to get all the media in a room (PR #2818) Thanks to @turt2live! +- Add `federation_domain_whitelist` option (PR #2820, #2821) + +Changes: + +- Continue to factor out processing from main process and into worker processes. See updated [docs/workers.rst](docs/workers.rst) (PR #2892 - \#2904, #2913, #2920 - \#2926, #2947, #2847, #2854, #2872, #2873, #2874, #2928, #2929, #2934, #2856, #2976 - \#2984, #2987 - \#2989, #2991 - \#2993, #2995, #2784) +- Ensure state cache is used when persisting events (PR #2864, #2871, #2802, #2835, #2836, #2841, #2842, #2849) +- Change the default config to bind on both IPv4 and IPv6 on all platforms (PR #2435) Thanks to @silkeh! +- No longer require a specific version of saml2 (PR #2695) Thanks to @okurz! +- Remove `verbosity`/`log_file` from generated config (PR #2755) +- Add and improve metrics and logging (PR #2770, #2778, #2785, #2786, #2787, #2793, #2794, #2795, #2809, #2810, #2833, #2834, #2844, #2965, #2927, #2975, #2790, #2796, #2838) +- When using synctl with workers, don\'t start the main synapse automatically (PR #2774) +- Minor performance improvements (PR #2773, #2792) +- Use a connection pool for non-federation outbound connections (PR #2817) +- Make it possible to run unit tests against postgres (PR #2829) +- Update pynacl dependency to 1.2.1 or higher (PR #2888) Thanks to @bachp! +- Remove ability for AS users to call /events and /sync (PR #2948) +- Use bcrypt.checkpw (PR #2949) Thanks to @krombel! + +Bug fixes: + +- Fix broken `ldap_config` config option (PR #2683) Thanks to @seckrv! +- Fix error message when user is not allowed to unban (PR #2761) Thanks to @turt2live! +- Fix publicised groups GET API (singular) over federation (PR #2772) +- Fix user directory when using `user_directory_search_all_users` config option (PR #2803, #2831) +- Fix error on `/publicRooms` when no rooms exist (PR #2827) +- Fix bug in quarantine\_media (PR #2837) +- Fix url\_previews when no Content-Type is returned from URL (PR #2845) +- Fix rare race in sync API when joining room (PR #2944) +- Fix slow event search, switch back from GIST to GIN indexes (PR #2769, #2848) + +Changes in synapse v0.26.0 (2018-01-05) +======================================= + +No changes since v0.26.0-rc1 + +Changes in synapse v0.26.0-rc1 (2017-12-13) +=========================================== + +Features: + +- Add ability for ASes to publicise groups for their users (PR #2686) +- Add all local users to the user\_directory and optionally search them (PR #2723) +- Add support for custom login types for validating users (PR #2729) + +Changes: + +- Update example Prometheus config to new format (PR #2648) Thanks to @krombel! +- Rename redact\_content option to include\_content in Push API (PR #2650) +- Declare support for r0.3.0 (PR #2677) +- Improve upserts (PR #2684, #2688, #2689, #2713) +- Improve documentation of workers (PR #2700) +- Improve tracebacks on exceptions (PR #2705) +- Allow guest access to group APIs for reading (PR #2715) +- Support for posting content in federation\_client script (PR #2716) +- Delete devices and pushers on logouts etc (PR #2722) + +Bug fixes: + +- Fix database port script (PR #2673) +- Fix internal server error on login with ldap\_auth\_provider (PR #2678) Thanks to @jkolo! +- Fix error on sqlite 3.7 (PR #2697) +- Fix OPTIONS on preview\_url (PR #2707) +- Fix error handling on dns lookup (PR #2711) +- Fix wrong avatars when inviting multiple users when creating room (PR #2717) +- Fix 500 when joining matrix-dev (PR #2719) + +Changes in synapse v0.25.1 (2017-11-17) +======================================= + +Bug fixes: + +- Fix login with LDAP and other password provider modules (PR #2678). Thanks to @jkolo! + +Changes in synapse v0.25.0 (2017-11-15) +======================================= + +Bug fixes: + +- Fix port script (PR #2673) + +Changes in synapse v0.25.0-rc1 (2017-11-14) +=========================================== + +Features: + +- Add is\_public to groups table to allow for private groups (PR #2582) +- Add a route for determining who you are (PR #2668) Thanks to @turt2live! +- Add more features to the password providers (PR #2608, #2610, #2620, #2622, #2623, #2624, #2626, #2628, #2629) +- Add a hook for custom rest endpoints (PR #2627) +- Add API to update group room visibility (PR #2651) + +Changes: + +- Ignore \ tags when generating URL preview descriptions (PR #2576) Thanks to @maximevaillancourt! +- Register some /unstable endpoints in /r0 as well (PR #2579) Thanks to @krombel! +- Support /keys/upload on /r0 as well as /unstable (PR #2585) +- Front-end proxy: pass through auth header (PR #2586) +- Allow ASes to deactivate their own users (PR #2589) +- Remove refresh tokens (PR #2613) +- Automatically set default displayname on register (PR #2617) +- Log login requests (PR #2618) +- Always return is\_public in the /groups/:group\_id/rooms API (PR #2630) +- Avoid no-op media deletes (PR #2637) Thanks to @spantaleev! +- Fix various embarrassing typos around user\_directory and add some doc. (PR #2643) +- Return whether a user is an admin within a group (PR #2647) +- Namespace visibility options for groups (PR #2657) +- Downcase UserIDs on registration (PR #2662) +- Cache failures when fetching URL previews (PR #2669) + +Bug fixes: + +- Fix port script (PR #2577) +- Fix error when running synapse with no logfile (PR #2581) +- Fix UI auth when deleting devices (PR #2591) +- Fix typo when checking if user is invited to group (PR #2599) +- Fix the port script to drop NUL values in all tables (PR #2611) +- Fix appservices being backlogged and not receiving new events due to a bug in notify\_interested\_services (PR #2631) Thanks to @xyzz! +- Fix updating rooms avatar/display name when modified by admin (PR #2636) Thanks to @farialima! +- Fix bug in state group storage (PR #2649) +- Fix 500 on invalid utf-8 in request (PR #2663) + +Changes in synapse v0.24.1 (2017-10-24) +======================================= + +Bug fixes: + +- Fix updating group profiles over federation (PR #2567) + +Changes in synapse v0.24.0 (2017-10-23) +======================================= + +No changes since v0.24.0-rc1 + +Changes in synapse v0.24.0-rc1 (2017-10-19) +=========================================== + +Features: + +- Add Group Server (PR #2352, #2363, #2374, #2377, #2378, #2382, #2410, #2426, #2430, #2454, #2471, #2472, #2544) +- Add support for channel notifications (PR #2501) +- Add basic implementation of backup media store (PR #2538) +- Add config option to auto-join new users to rooms (PR #2545) + +Changes: + +- Make the spam checker a module (PR #2474) +- Delete expired url cache data (PR #2478) +- Ignore incoming events for rooms that we have left (PR #2490) +- Allow spam checker to reject invites too (PR #2492) +- Add room creation checks to spam checker (PR #2495) +- Spam checking: add the invitee to user\_may\_invite (PR #2502) +- Process events from federation for different rooms in parallel (PR #2520) +- Allow error strings from spam checker (PR #2531) +- Improve error handling for missing files in config (PR #2551) + +Bug fixes: + +- Fix handling SERVFAILs when doing AAAA lookups for federation (PR #2477) +- Fix incompatibility with newer versions of ujson (PR #2483) Thanks to @jeremycline! +- Fix notification keywords that start/end with non-word chars (PR #2500) +- Fix stack overflow and logcontexts from linearizer (PR #2532) +- Fix 500 error when fields missing from power\_levels event (PR #2552) +- Fix 500 error when we get an error handling a PDU (PR #2553) + +Changes in synapse v0.23.1 (2017-10-02) +======================================= + +Changes: + +- Make \'affinity\' package optional, as it is not supported on some platforms + +Changes in synapse v0.23.0 (2017-10-02) +======================================= + +No changes since v0.23.0-rc2 + +Changes in synapse v0.23.0-rc2 (2017-09-26) +=========================================== + +Bug fixes: + +- Fix regression in performance of syncs (PR #2470) + +Changes in synapse v0.23.0-rc1 (2017-09-25) +=========================================== + +Features: + +- Add a frontend proxy worker (PR #2344) +- Add support for event\_id\_only push format (PR #2450) +- Add a PoC for filtering spammy events (PR #2456) +- Add a config option to block all room invites (PR #2457) + +Changes: + +- Use bcrypt module instead of py-bcrypt (PR #2288) Thanks to @kyrias! +- Improve performance of generating push notifications (PR #2343, #2357, #2365, #2366, #2371) +- Improve DB performance for device list handling in sync (PR #2362) +- Include a sample prometheus config (PR #2416) +- Document known to work postgres version (PR #2433) Thanks to @ptman! + +Bug fixes: + +- Fix caching error in the push evaluator (PR #2332) +- Fix bug where pusherpool didn\'t start and broke some rooms (PR #2342) +- Fix port script for user directory tables (PR #2375) +- Fix device lists notifications when user rejoins a room (PR #2443, #2449) +- Fix sync to always send down current state events in timeline (PR #2451) +- Fix bug where guest users were incorrectly kicked (PR #2453) +- Fix bug talking to IPv6 only servers using SRV records (PR #2462) + +Changes in synapse v0.22.1 (2017-07-06) +======================================= + +Bug fixes: + +- Fix bug where pusher pool didn\'t start and caused issues when interacting with some rooms (PR #2342) + +Changes in synapse v0.22.0 (2017-07-06) +======================================= + +No changes since v0.22.0-rc2 + +Changes in synapse v0.22.0-rc2 (2017-07-04) +=========================================== + +Changes: + +- Improve performance of storing user IPs (PR #2307, #2308) +- Slightly improve performance of verifying access tokens (PR #2320) +- Slightly improve performance of event persistence (PR #2321) +- Increase default cache factor size from 0.1 to 0.5 (PR #2330) + +Bug fixes: + +- Fix bug with storing registration sessions that caused frequent CPU churn (PR #2319) + +Changes in synapse v0.22.0-rc1 (2017-06-26) +=========================================== + +Features: + +- Add a user directory API (PR #2252, and many more) +- Add shutdown room API to remove room from local server (PR #2291) +- Add API to quarantine media (PR #2292) +- Add new config option to not send event contents to push servers (PR #2301) Thanks to @cjdelisle! + +Changes: + +- Various performance fixes (PR #2177, #2233, #2230, #2238, #2248, #2256, #2274) +- Deduplicate sync filters (PR #2219) Thanks to @krombel! +- Correct a typo in UPGRADE.rst (PR #2231) Thanks to @aaronraimist! +- Add count of one time keys to sync stream (PR #2237) +- Only store event\_auth for state events (PR #2247) +- Store URL cache preview downloads separately (PR #2299) + +Bug fixes: + +- Fix users not getting notifications when AS listened to that user\_id (PR #2216) Thanks to @slipeer! +- Fix users without push set up not getting notifications after joining rooms (PR #2236) +- Fix preview url API to trim long descriptions (PR #2243) +- Fix bug where we used cached but unpersisted state group as prev group, resulting in broken state of restart (PR #2263) +- Fix removing of pushers when using workers (PR #2267) +- Fix CORS headers to allow Authorization header (PR #2285) Thanks to @krombel! + +Changes in synapse v0.21.1 (2017-06-15) +======================================= + +Bug fixes: + +- Fix bug in anonymous usage statistic reporting (PR #2281) + +Changes in synapse v0.21.0 (2017-05-18) +======================================= + +No changes since v0.21.0-rc3 + +Changes in synapse v0.21.0-rc3 (2017-05-17) +=========================================== + +Features: + +- Add per user rate-limiting overrides (PR #2208) +- Add config option to limit maximum number of events requested by `/sync` and `/messages` (PR #2221) Thanks to @psaavedra! + +Changes: + +- Various small performance fixes (PR #2201, #2202, #2224, #2226, #2227, #2228, #2229) +- Update username availability checker API (PR #2209, #2213) +- When purging, don\'t de-delta state groups we\'re about to delete (PR #2214) +- Documentation to check synapse version (PR #2215) Thanks to @hamber-dick! +- Add an index to event\_search to speed up purge history API (PR #2218) + +Bug fixes: + +- Fix API to allow clients to upload one-time-keys with new sigs (PR #2206) + +Changes in synapse v0.21.0-rc2 (2017-05-08) +=========================================== + +Changes: + +- Always mark remotes as up if we receive a signed request from them (PR #2190) + +Bug fixes: + +- Fix bug where users got pushed for rooms they had muted (PR #2200) + +Changes in synapse v0.21.0-rc1 (2017-05-08) +=========================================== + +Features: + +- Add username availability checker API (PR #2183) +- Add read marker API (PR #2120) + +Changes: + +- Enable guest access for the 3pl/3pid APIs (PR #1986) +- Add setting to support TURN for guests (PR #2011) +- Various performance improvements (PR #2075, #2076, #2080, #2083, #2108, #2158, #2176, #2185) +- Make synctl a bit more user friendly (PR #2078, #2127) Thanks @APwhitehat! +- Replace HTTP replication with TCP replication (PR #2082, #2097, #2098, #2099, #2103, #2014, #2016, #2115, #2116, #2117) +- Support authenticated SMTP (PR #2102) Thanks @DanielDent! +- Add a counter metric for successfully-sent transactions (PR #2121) +- Propagate errors sensibly from proxied IS requests (PR #2147) +- Add more granular event send metrics (PR #2178) + +Bug fixes: + +- Fix nuke-room script to work with current schema (PR #1927) Thanks @zuckschwerdt! +- Fix db port script to not assume postgres tables are in the public schema (PR #2024) Thanks @jerrykan! +- Fix getting latest device IP for user with no devices (PR #2118) +- Fix rejection of invites to unreachable servers (PR #2145) +- Fix code for reporting old verify keys in synapse (PR #2156) +- Fix invite state to always include all events (PR #2163) +- Fix bug where synapse would always fetch state for any missing event (PR #2170) +- Fix a leak with timed out HTTP connections (PR #2180) +- Fix bug where we didn\'t time out HTTP requests to ASes (PR #2192) + +Docs: + +- Clarify doc for SQLite to PostgreSQL port (PR #1961) Thanks @benhylau! +- Fix typo in synctl help (PR #2107) Thanks @HarHarLinks! +- `web_client_location` documentation fix (PR #2131) Thanks @matthewjwolff! +- Update README.rst with FreeBSD changes (PR #2132) Thanks @feld! +- Clarify setting up metrics (PR #2149) Thanks @encks! + +Changes in synapse v0.20.0 (2017-04-11) +======================================= + +Bug fixes: + +- Fix joining rooms over federation where not all servers in the room saw the new server had joined (PR #2094) + +Changes in synapse v0.20.0-rc1 (2017-03-30) +=========================================== + +Features: + +- Add delete\_devices API (PR #1993) +- Add phone number registration/login support (PR #1994, #2055) + +Changes: + +- Use JSONSchema for validation of filters. Thanks @pik! (PR #1783) +- Reread log config on SIGHUP (PR #1982) +- Speed up public room list (PR #1989) +- Add helpful texts to logger config options (PR #1990) +- Minor `/sync` performance improvements. (PR #2002, #2013, #2022) +- Add some debug to help diagnose weird federation issue (PR #2035) +- Correctly limit retries for all federation requests (PR #2050, #2061) +- Don\'t lock table when persisting new one time keys (PR #2053) +- Reduce some CPU work on DB threads (PR #2054) +- Cache hosts in room (PR #2060) +- Batch sending of device list pokes (PR #2063) +- Speed up persist event path in certain edge cases (PR #2070) + +Bug fixes: + +- Fix bug where current\_state\_events renamed to current\_state\_ids (PR #1849) +- Fix routing loop when fetching remote media (PR #1992) +- Fix current\_state\_events table to not lie (PR #1996) +- Fix CAS login to handle PartialDownloadError (PR #1997) +- Fix assertion to stop transaction queue getting wedged (PR #2010) +- Fix presence to fallback to last\_active\_ts if it beats the last sync time. Thanks @Half-Shot! (PR #2014) +- Fix bug when federation received a PDU while a room join is in progress (PR #2016) +- Fix resetting state on rejected events (PR #2025) +- Fix installation issues in readme. Thanks @ricco386 (PR #2037) +- Fix caching of remote servers\' signature keys (PR #2042) +- Fix some leaking log context (PR #2048, #2049, #2057, #2058) +- Fix rejection of invites not reaching sync (PR #2056) + +Changes in synapse v0.19.3 (2017-03-20) +======================================= + +No changes since v0.19.3-rc2 + +Changes in synapse v0.19.3-rc2 (2017-03-13) +=========================================== + +Bug fixes: + +- Fix bug in handling of incoming device list updates over federation. + +Changes in synapse v0.19.3-rc1 (2017-03-08) +=========================================== + +Features: + +- Add some administration functionalities. Thanks to morteza-araby! (PR #1784) + +Changes: + +- Reduce database table sizes (PR #1873, #1916, #1923, #1963) +- Update contrib/ to not use syutil. Thanks to andrewshadura! (PR #1907) +- Don\'t fetch current state when sending an event in common case (PR #1955) + +Bug fixes: + +- Fix synapse\_port\_db failure. Thanks to Pneumaticat! (PR #1904) +- Fix caching to not cache error responses (PR #1913) +- Fix APIs to make kick & ban reasons work (PR #1917) +- Fix bugs in the /keys/changes api (PR #1921) +- Fix bug where users couldn\'t forget rooms they were banned from (PR #1922) +- Fix issue with long language values in pushers API (PR #1925) +- Fix a race in transaction queue (PR #1930) +- Fix dynamic thumbnailing to preserve aspect ratio. Thanks to jkolo! (PR #1945) +- Fix device list update to not constantly resync (PR #1964) +- Fix potential for huge memory usage when getting device that have changed (PR #1969) + +Changes in synapse v0.19.2 (2017-02-20) +======================================= + +- Fix bug with event visibility check in /context/ API. Thanks to Tokodomo for pointing it out! (PR #1929) + +Changes in synapse v0.19.1 (2017-02-09) +======================================= + +- Fix bug where state was incorrectly reset in a room when synapse received an event over federation that did not pass auth checks (PR #1892) + +Changes in synapse v0.19.0 (2017-02-04) +======================================= + +No changes since RC 4. + +Changes in synapse v0.19.0-rc4 (2017-02-02) +=========================================== + +- Bump cache sizes for common membership queries (PR #1879) + +Changes in synapse v0.19.0-rc3 (2017-02-02) +=========================================== + +- Fix email push in pusher worker (PR #1875) +- Make presence.get\_new\_events a bit faster (PR #1876) +- Make /keys/changes a bit more performant (PR #1877) + +Changes in synapse v0.19.0-rc2 (2017-02-02) +=========================================== + +- Include newly joined users in /keys/changes API (PR #1872) + +Changes in synapse v0.19.0-rc1 (2017-02-02) +=========================================== + +Features: + +- Add support for specifying multiple bind addresses (PR #1709, #1712, #1795, #1835). Thanks to @kyrias! +- Add /account/3pid/delete endpoint (PR #1714) +- Add config option to configure the Riot URL used in notification emails (PR #1811). Thanks to @aperezdc! +- Add username and password config options for turn server (PR #1832). Thanks to @xsteadfastx! +- Implement device lists updates over federation (PR #1857, #1861, #1864) +- Implement /keys/changes (PR #1869, #1872) + +Changes: + +- Improve IPv6 support (PR #1696). Thanks to @kyrias and @glyph! +- Log which files we saved attachments to in the media\_repository (PR #1791) +- Linearize updates to membership via PUT /state/ to better handle multiple joins (PR #1787) +- Limit number of entries to prefill from cache on startup (PR #1792) +- Remove full\_twisted\_stacktraces option (PR #1802) +- Measure size of some caches by sum of the size of cached values (PR #1815) +- Measure metrics of string\_cache (PR #1821) +- Reduce logging verbosity (PR #1822, #1823, #1824) +- Don\'t clobber a displayname or avatar\_url if provided by an m.room.member event (PR #1852) +- Better handle 401/404 response for federation /send/ (PR #1866, #1871) + +Fixes: + +- Fix ability to change password to a non-ascii one (PR #1711) +- Fix push getting stuck due to looking at the wrong view of state (PR #1820) +- Fix email address comparison to be case insensitive (PR #1827) +- Fix occasional inconsistencies of room membership (PR #1836, #1840) + +Performance: + +- Don\'t block messages sending on bumping presence (PR #1789) +- Change device\_inbox stream index to include user (PR #1793) +- Optimise state resolution (PR #1818) +- Use DB cache of joined users for presence (PR #1862) +- Add an index to make membership queries faster (PR #1867) + +Changes in synapse v0.18.7 (2017-01-09) +======================================= + +No changes from v0.18.7-rc2 + +Changes in synapse v0.18.7-rc2 (2017-01-07) +=========================================== + +Bug fixes: + +- Fix error in rc1\'s discarding invalid inbound traffic logic that was incorrectly discarding missing events + +Changes in synapse v0.18.7-rc1 (2017-01-06) +=========================================== + +Bug fixes: + +- Fix error in \#PR 1764 to actually fix the nightmare \#1753 bug. +- Improve deadlock logging further +- Discard inbound federation traffic from invalid domains, to immunise against \#1753 + +Changes in synapse v0.18.6 (2017-01-06) +======================================= + +Bug fixes: + +- Fix bug when checking if a guest user is allowed to join a room (PR #1772) Thanks to Patrik Oldsberg for diagnosing and the fix! + +Changes in synapse v0.18.6-rc3 (2017-01-05) +=========================================== + +Bug fixes: + +- Fix bug where we failed to send ban events to the banned server (PR #1758) +- Fix bug where we sent event that didn\'t originate on this server to other servers (PR #1764) +- Fix bug where processing an event from a remote server took a long time because we were making long HTTP requests (PR #1765, PR #1744) + +Changes: + +- Improve logging for debugging deadlocks (PR #1766, PR #1767) + +Changes in synapse v0.18.6-rc2 (2016-12-30) +=========================================== + +Bug fixes: + +- Fix memory leak in twisted by initialising logging correctly (PR #1731) +- Fix bug where fetching missing events took an unacceptable amount of time in large rooms (PR #1734) + +Changes in synapse v0.18.6-rc1 (2016-12-29) +=========================================== + +Bug fixes: + +- Make sure that outbound connections are closed (PR #1725) + +Changes in synapse v0.18.5 (2016-12-16) +======================================= + +Bug fixes: + +- Fix federation /backfill returning events it shouldn\'t (PR #1700) +- Fix crash in url preview (PR #1701) + +Changes in synapse v0.18.5-rc3 (2016-12-13) +=========================================== + +Features: + +- Add support for E2E for guests (PR #1653) +- Add new API appservice specific public room list (PR #1676) +- Add new room membership APIs (PR #1680) + +Changes: + +- Enable guest access for private rooms by default (PR #653) +- Limit the number of events that can be created on a given room concurrently (PR #1620) +- Log the args that we have on UI auth completion (PR #1649) +- Stop generating refresh\_tokens (PR #1654) +- Stop putting a time caveat on access tokens (PR #1656) +- Remove unspecced GET endpoints for e2e keys (PR #1694) + +Bug fixes: + +- Fix handling of 500 and 429\'s over federation (PR #1650) +- Fix Content-Type header parsing (PR #1660) +- Fix error when previewing sites that include unicode, thanks to kyrias (PR #1664) +- Fix some cases where we drop read receipts (PR #1678) +- Fix bug where calls to `/sync` didn\'t correctly timeout (PR #1683) +- Fix bug where E2E key query would fail if a single remote host failed (PR #1686) + +Changes in synapse v0.18.5-rc2 (2016-11-24) +=========================================== + +Bug fixes: + +- Don\'t send old events over federation, fixes bug in -rc1. + +Changes in synapse v0.18.5-rc1 (2016-11-24) +=========================================== + +Features: + +- Implement \"event\_fields\" in filters (PR #1638) + +Changes: + +- Use external ldap auth pacakge (PR #1628) +- Split out federation transaction sending to a worker (PR #1635) +- Fail with a coherent error message if /sync?filter= is invalid (PR #1636) +- More efficient notif count queries (PR #1644) + +Changes in synapse v0.18.4 (2016-11-22) +======================================= + +Bug fixes: + +- Add workaround for buggy clients that the fail to register (PR #1632) + +Changes in synapse v0.18.4-rc1 (2016-11-14) +=========================================== + +Changes: + +- Various database efficiency improvements (PR #1188, #1192) +- Update default config to blacklist more internal IPs, thanks to Euan Kemp (PR #1198) +- Allow specifying duration in minutes in config, thanks to Daniel Dent (PR #1625) + +Bug fixes: + +- Fix media repo to set CORs headers on responses (PR #1190) +- Fix registration to not error on non-ascii passwords (PR #1191) +- Fix create event code to limit the number of prev\_events (PR #1615) +- Fix bug in transaction ID deduplication (PR #1624) + +Changes in synapse v0.18.3 (2016-11-08) +======================================= + +SECURITY UPDATE + +Explicitly require authentication when using LDAP3. This is the default on versions of `ldap3` above 1.0, but some distributions will package an older version. + +If you are using LDAP3 login and have a version of `ldap3` older than 1.0 it is **CRITICAL to updgrade**. + +Changes in synapse v0.18.2 (2016-11-01) +======================================= + +No changes since v0.18.2-rc5 + +Changes in synapse v0.18.2-rc5 (2016-10-28) +=========================================== + +Bug fixes: + +- Fix prometheus process metrics in worker processes (PR #1184) + +Changes in synapse v0.18.2-rc4 (2016-10-27) +=========================================== + +Bug fixes: + +- Fix `user_threepids` schema delta, which in some instances prevented startup after upgrade (PR #1183) + +Changes in synapse v0.18.2-rc3 (2016-10-27) +=========================================== + +Changes: + +- Allow clients to supply access tokens as headers (PR #1098) +- Clarify error codes for GET /filter/, thanks to Alexander Maznev (PR #1164) +- Make password reset email field case insensitive (PR #1170) +- Reduce redundant database work in email pusher (PR #1174) +- Allow configurable rate limiting per AS (PR #1175) +- Check whether to ratelimit sooner to avoid work (PR #1176) +- Standardise prometheus metrics (PR #1177) + +Bug fixes: + +- Fix incredibly slow back pagination query (PR #1178) +- Fix infinite typing bug (PR #1179) + +Changes in synapse v0.18.2-rc2 (2016-10-25) +=========================================== + +(This release did not include the changes advertised and was identical to RC1) + +Changes in synapse v0.18.2-rc1 (2016-10-17) +=========================================== + +Changes: + +- Remove redundant event\_auth index (PR #1113) +- Reduce DB hits for replication (PR #1141) +- Implement pluggable password auth (PR #1155) +- Remove rate limiting from app service senders and fix get\_or\_create\_user requester, thanks to Patrik Oldsberg (PR #1157) +- window.postmessage for Interactive Auth fallback (PR #1159) +- Use sys.executable instead of hardcoded python, thanks to Pedro Larroy (PR #1162) +- Add config option for adding additional TLS fingerprints (PR #1167) +- User-interactive auth on delete device (PR #1168) + +Bug fixes: + +- Fix not being allowed to set your own state\_key, thanks to Patrik Oldsberg (PR #1150) +- Fix interactive auth to return 401 from for incorrect password (PR #1160, #1166) +- Fix email push notifs being dropped (PR #1169) + +Changes in synapse v0.18.1 (2016-10-05) +======================================= + +No changes since v0.18.1-rc1 + +Changes in synapse v0.18.1-rc1 (2016-09-30) +=========================================== + +Features: + +- Add total\_room\_count\_estimate to `/publicRooms` (PR #1133) + +Changes: + +- Time out typing over federation (PR #1140) +- Restructure LDAP authentication (PR #1153) + +Bug fixes: + +- Fix 3pid invites when server is already in the room (PR #1136) +- Fix upgrading with SQLite taking lots of CPU for a few days after upgrade (PR #1144) +- Fix upgrading from very old database versions (PR #1145) +- Fix port script to work with recently added tables (PR #1146) + +Changes in synapse v0.18.0 (2016-09-19) +======================================= + +The release includes major changes to the state storage database schemas, which significantly reduce database size. Synapse will attempt to upgrade the current data in the background. Servers with large SQLite database may experience degradation of performance while this upgrade is in progress, therefore you may want to consider migrating to using Postgres before upgrading very large SQLite databases + +Changes: + +- Make public room search case insensitive (PR #1127) + +Bug fixes: + +- Fix and clean up publicRooms pagination (PR #1129) + +Changes in synapse v0.18.0-rc1 (2016-09-16) +=========================================== + +Features: + +- Add `only=highlight` on `/notifications` (PR #1081) +- Add server param to /publicRooms (PR #1082) +- Allow clients to ask for the whole of a single state event (PR #1094) +- Add is\_direct param to /createRoom (PR #1108) +- Add pagination support to publicRooms (PR #1121) +- Add very basic filter API to /publicRooms (PR #1126) +- Add basic direct to device messaging support for E2E (PR #1074, #1084, #1104, #1111) + +Changes: + +- Move to storing state\_groups\_state as deltas, greatly reducing DB size (PR #1065) +- Reduce amount of state pulled out of the DB during common requests (PR #1069) +- Allow PDF to be rendered from media repo (PR #1071) +- Reindex state\_groups\_state after pruning (PR #1085) +- Clobber EDUs in send queue (PR #1095) +- Conform better to the CAS protocol specification (PR #1100) +- Limit how often we ask for keys from dead servers (PR #1114) + +Bug fixes: + +- Fix /notifications API when used with `from` param (PR #1080) +- Fix backfill when cannot find an event. (PR #1107) + +Changes in synapse v0.17.3 (2016-09-09) +======================================= + +This release fixes a major bug that stopped servers from handling rooms with over 1000 members. + +Changes in synapse v0.17.2 (2016-09-08) +======================================= + +This release contains security bug fixes. Please upgrade. + +No changes since v0.17.2-rc1 + +Changes in synapse v0.17.2-rc1 (2016-09-05) +=========================================== + +Features: + +- Start adding store-and-forward direct-to-device messaging (PR #1046, #1050, #1062, #1066) + +Changes: + +- Avoid pulling the full state of a room out so often (PR #1047, #1049, #1063, #1068) +- Don\'t notify for online to online presence transitions. (PR #1054) +- Occasionally persist unpersisted presence updates (PR #1055) +- Allow application services to have an optional \'url\' (PR #1056) +- Clean up old sent transactions from DB (PR #1059) + +Bug fixes: + +- Fix None check in backfill (PR #1043) +- Fix membership changes to be idempotent (PR #1067) +- Fix bug in get\_pdu where it would sometimes return events with incorrect signature + +Changes in synapse v0.17.1 (2016-08-24) +======================================= + +Changes: + +- Delete old received\_transactions rows (PR #1038) +- Pass through user-supplied content in /join/\$room\_id (PR #1039) + +Bug fixes: + +- Fix bug with backfill (PR #1040) + +Changes in synapse v0.17.1-rc1 (2016-08-22) +=========================================== + +Features: + +- Add notification API (PR #1028) + +Changes: + +- Don\'t print stack traces when failing to get remote keys (PR #996) +- Various federation /event/ perf improvements (PR #998) +- Only process one local membership event per room at a time (PR #1005) +- Move default display name push rule (PR #1011, #1023) +- Fix up preview URL API. Add tests. (PR #1015) +- Set `Content-Security-Policy` on media repo (PR #1021) +- Make notify\_interested\_services faster (PR #1022) +- Add usage stats to prometheus monitoring (PR #1037) + +Bug fixes: + +- Fix token login (PR #993) +- Fix CAS login (PR #994, #995) +- Fix /sync to not clobber status\_msg (PR #997) +- Fix redacted state events to include prev\_content (PR #1003) +- Fix some bugs in the auth/ldap handler (PR #1007) +- Fix backfill request to limit URI length, so that remotes don\'t reject the requests due to path length limits (PR #1012) +- Fix AS push code to not send duplicate events (PR #1025) + +Changes in synapse v0.17.0 (2016-08-08) +======================================= + +This release contains significant security bug fixes regarding authenticating events received over federation. PLEASE UPGRADE. + +This release changes the LDAP configuration format in a backwards incompatible way, see PR #843 for details. + +Changes: + +- Add federation /version API (PR #990) +- Make psutil dependency optional (PR #992) + +Bug fixes: + +- Fix URL preview API to exclude HTML comments in description (PR #988) +- Fix error handling of remote joins (PR #991) + +Changes in synapse v0.17.0-rc4 (2016-08-05) +=========================================== + +Changes: + +- Change the way we summarize URLs when previewing (PR #973) +- Add new `/state_ids/` federation API (PR #979) +- Speed up processing of `/state/` response (PR #986) + +Bug fixes: + +- Fix event persistence when event has already been partially persisted (PR #975, #983, #985) +- Fix port script to also copy across backfilled events (PR #982) + +Changes in synapse v0.17.0-rc3 (2016-08-02) +=========================================== + +Changes: + +- Forbid non-ASes from registering users whose names begin with \'\_\' (PR #958) +- Add some basic admin API docs (PR #963) + +Bug fixes: + +- Send the correct host header when fetching keys (PR #941) +- Fix joining a room that has missing auth events (PR #964) +- Fix various push bugs (PR #966, #970) +- Fix adding emails on registration (PR #968) + +Changes in synapse v0.17.0-rc2 (2016-08-02) +=========================================== + +(This release did not include the changes advertised and was identical to RC1) + +Changes in synapse v0.17.0-rc1 (2016-07-28) +=========================================== + +This release changes the LDAP configuration format in a backwards incompatible way, see PR #843 for details. + +Features: + +- Add purge\_media\_cache admin API (PR #902) +- Add deactivate account admin API (PR #903) +- Add optional pepper to password hashing (PR #907, #910 by KentShikama) +- Add an admin option to shared secret registration (breaks backwards compat) (PR #909) +- Add purge local room history API (PR #911, #923, #924) +- Add requestToken endpoints (PR #915) +- Add an /account/deactivate endpoint (PR #921) +- Add filter param to /messages. Add \'contains\_url\' to filter. (PR #922) +- Add device\_id support to /login (PR #929) +- Add device\_id support to /v2/register flow. (PR #937, #942) +- Add GET /devices endpoint (PR #939, #944) +- Add GET /device/{deviceId} (PR #943) +- Add update and delete APIs for devices (PR #949) + +Changes: + +- Rewrite LDAP Authentication against ldap3 (PR #843 by mweinelt) +- Linearize some federation endpoints based on (origin, room\_id) (PR #879) +- Remove the legacy v0 content upload API. (PR #888) +- Use similar naming we use in email notifs for push (PR #894) +- Optionally include password hash in createUser endpoint (PR #905 by KentShikama) +- Use a query that postgresql optimises better for get\_events\_around (PR #906) +- Fall back to \'username\' if \'user\' is not given for appservice registration. (PR #927 by Half-Shot) +- Add metrics for psutil derived memory usage (PR #936) +- Record device\_id in client\_ips (PR #938) +- Send the correct host header when fetching keys (PR #941) +- Log the hostname the reCAPTCHA was completed on (PR #946) +- Make the device id on e2e key upload optional (PR #956) +- Add r0.2.0 to the \"supported versions\" list (PR #960) +- Don\'t include name of room for invites in push (PR #961) + +Bug fixes: + +- Fix substitution failure in mail template (PR #887) +- Put most recent 20 messages in email notif (PR #892) +- Ensure that the guest user is in the database when upgrading accounts (PR #914) +- Fix various edge cases in auth handling (PR #919) +- Fix 500 ISE when sending alias event without a state\_key (PR #925) +- Fix bug where we stored rejections in the state\_group, persist all rejections (PR #948) +- Fix lack of check of if the user is banned when handling 3pid invites (PR #952) +- Fix a couple of bugs in the transaction and keyring code (PR #954, #955) + +Changes in synapse v0.16.1-r1 (2016-07-08) +========================================== + +THIS IS A CRITICAL SECURITY UPDATE. + +This fixes a bug which allowed users\' accounts to be accessed by unauthorised users. + +Changes in synapse v0.16.1 (2016-06-20) +======================================= + +Bug fixes: + +- Fix assorted bugs in `/preview_url` (PR #872) +- Fix TypeError when setting unicode passwords (PR #873) + +Performance improvements: + +- Turn `use_frozen_events` off by default (PR #877) +- Disable responding with canonical json for federation (PR #878) + +Changes in synapse v0.16.1-rc1 (2016-06-15) +=========================================== + +Features: None + +Changes: + +- Log requester for `/publicRoom` endpoints when possible (PR #856) +- 502 on `/thumbnail` when can\'t connect to remote server (PR #862) +- Linearize fetching of gaps on incoming events (PR #871) + +Bugs fixes: + +- Fix bug where rooms where marked as published by default (PR #857) +- Fix bug where joining room with an event with invalid sender (PR #868) +- Fix bug where backfilled events were sent down sync streams (PR #869) +- Fix bug where outgoing connections could wedge indefinitely, causing push notifications to be unreliable (PR #870) + +Performance improvements: + +- Improve `/publicRooms` performance(PR #859) + +Changes in synapse v0.16.0 (2016-06-09) +======================================= + +NB: As of v0.14 all AS config files must have an ID field. + +Bug fixes: + +- Don\'t make rooms published by default (PR #857) + +Changes in synapse v0.16.0-rc2 (2016-06-08) +=========================================== + +Features: + +- Add configuration option for tuning GC via `gc.set_threshold` (PR #849) + +Changes: + +- Record metrics about GC (PR #771, #847, #852) +- Add metric counter for number of persisted events (PR #841) + +Bug fixes: + +- Fix \'From\' header in email notifications (PR #843) +- Fix presence where timeouts were not being fired for the first 8h after restarts (PR #842) +- Fix bug where synapse sent malformed transactions to AS\'s when retrying transactions (Commits 310197b, 8437906) + +Performance improvements: + +- Remove event fetching from DB threads (PR #835) +- Change the way we cache events (PR #836) +- Add events to cache when we persist them (PR #840) + +Changes in synapse v0.16.0-rc1 (2016-06-03) +=========================================== + +Version 0.15 was not released. See v0.15.0-rc1 below for additional changes. + +Features: + +- Add email notifications for missed messages (PR #759, #786, #799, #810, #815, #821) +- Add a `url_preview_ip_range_whitelist` config param (PR #760) +- Add /report endpoint (PR #762) +- Add basic ignore user API (PR #763) +- Add an openidish mechanism for proving that you own a given user\_id (PR #765) +- Allow clients to specify a server\_name to avoid \'No known servers\' (PR #794) +- Add secondary\_directory\_servers option to fetch room list from other servers (PR #808, #813) + +Changes: + +- Report per request metrics for all of the things using request\_handler (PR #756) +- Correctly handle `NULL` password hashes from the database (PR #775) +- Allow receipts for events we haven\'t seen in the db (PR #784) +- Make synctl read a cache factor from config file (PR #785) +- Increment badge count per missed convo, not per msg (PR #793) +- Special case m.room.third\_party\_invite event auth to match invites (PR #814) + +Bug fixes: + +- Fix typo in event\_auth servlet path (PR #757) +- Fix password reset (PR #758) + +Performance improvements: + +- Reduce database inserts when sending transactions (PR #767) +- Queue events by room for persistence (PR #768) +- Add cache to `get_user_by_id` (PR #772) +- Add and use `get_domain_from_id` (PR #773) +- Use tree cache for `get_linearized_receipts_for_room` (PR #779) +- Remove unused indices (PR #782) +- Add caches to `bulk_get_push_rules*` (PR #804) +- Cache `get_event_reference_hashes` (PR #806) +- Add `get_users_with_read_receipts_in_room` cache (PR #809) +- Use state to calculate `get_users_in_room` (PR #811) +- Load push rules in storage layer so that they get cached (PR #825) +- Make `get_joined_hosts_for_room` use get\_users\_in\_room (PR #828) +- Poke notifier on next reactor tick (PR #829) +- Change CacheMetrics to be quicker (PR #830) + +Changes in synapse v0.15.0-rc1 (2016-04-26) +=========================================== + +Features: + +- Add login support for Javascript Web Tokens, thanks to Niklas Riekenbrauck (PR #671,\#687) +- Add URL previewing support (PR #688) +- Add login support for LDAP, thanks to Christoph Witzany (PR #701) +- Add GET endpoint for pushers (PR #716) + +Changes: + +- Never notify for member events (PR #667) +- Deduplicate identical `/sync` requests (PR #668) +- Require user to have left room to forget room (PR #673) +- Use DNS cache if within TTL (PR #677) +- Let users see their own leave events (PR #699) +- Deduplicate membership changes (PR #700) +- Increase performance of pusher code (PR #705) +- Respond with error status 504 if failed to talk to remote server (PR #731) +- Increase search performance on postgres (PR #745) + +Bug fixes: + +- Fix bug where disabling all notifications still resulted in push (PR #678) +- Fix bug where users couldn\'t reject remote invites if remote refused (PR #691) +- Fix bug where synapse attempted to backfill from itself (PR #693) +- Fix bug where profile information was not correctly added when joining remote rooms (PR #703) +- Fix bug where register API required incorrect key name for AS registration (PR #727) + +Changes in synapse v0.14.0 (2016-03-30) +======================================= + +No changes from v0.14.0-rc2 + +Changes in synapse v0.14.0-rc2 (2016-03-23) +=========================================== + +Features: + +- Add published room list API (PR #657) + +Changes: + +- Change various caches to consume less memory (PR #656, #658, #660, #662, #663, #665) +- Allow rooms to be published without requiring an alias (PR #664) +- Intern common strings in caches to reduce memory footprint (\#666) + +Bug fixes: + +- Fix reject invites over federation (PR #646) +- Fix bug where registration was not idempotent (PR #649) +- Update aliases event after deleting aliases (PR #652) +- Fix unread notification count, which was sometimes wrong (PR #661) + +Changes in synapse v0.14.0-rc1 (2016-03-14) +=========================================== + +Features: + +- Add event\_id to response to state event PUT (PR #581) +- Allow guest users access to messages in rooms they have joined (PR #587) +- Add config for what state is included in a room invite (PR #598) +- Send the inviter\'s member event in room invite state (PR #607) +- Add error codes for malformed/bad JSON in /login (PR #608) +- Add support for changing the actions for default rules (PR #609) +- Add environment variable SYNAPSE\_CACHE\_FACTOR, default it to 0.1 (PR #612) +- Add ability for alias creators to delete aliases (PR #614) +- Add profile information to invites (PR #624) + +Changes: + +- Enforce user\_id exclusivity for AS registrations (PR #572) +- Make adding push rules idempotent (PR #587) +- Improve presence performance (PR #582, #586) +- Change presence semantics for `last_active_ago` (PR #582, #586) +- Don\'t allow `m.room.create` to be changed (PR #596) +- Add 800x600 to default list of valid thumbnail sizes (PR #616) +- Always include kicks and bans in full /sync (PR #625) +- Send history visibility on boundary changes (PR #626) +- Register endpoint now returns a refresh\_token (PR #637) + +Bug fixes: + +- Fix bug where we returned incorrect state in /sync (PR #573) +- Always return a JSON object from push rule API (PR #606) +- Fix bug where registering without a user id sometimes failed (PR #610) +- Report size of ExpiringCache in cache size metrics (PR #611) +- Fix rejection of invites to empty rooms (PR #615) +- Fix usage of `bcrypt` to not use `checkpw` (PR #619) +- Pin `pysaml2` dependency (PR #634) +- Fix bug in `/sync` where timeline order was incorrect for backfilled events (PR #635) + +Changes in synapse v0.13.3 (2016-02-11) +======================================= + +- Fix bug where `/sync` would occasionally return events in the wrong room. + +Changes in synapse v0.13.2 (2016-02-11) +======================================= + +- Fix bug where `/events` would fail to skip some events if there had been more events than the limit specified since the last request (PR #570) + +Changes in synapse v0.13.1 (2016-02-10) +======================================= + +- Bump matrix-angular-sdk (matrix web console) dependency to 0.6.8 to pull in the fix for SYWEB-361 so that the default client can display HTML messages again(!) + +Changes in synapse v0.13.0 (2016-02-10) +======================================= + +This version includes an upgrade of the schema, specifically adding an index to the `events` table. This may cause synapse to pause for several minutes the first time it is started after the upgrade. + +Changes: + +- Improve general performance (PR #540, #543. \#544, #54, #549, #567) +- Change guest user ids to be incrementing integers (PR #550) +- Improve performance of public room list API (PR #552) +- Change profile API to omit keys rather than return null (PR #557) +- Add `/media/r0` endpoint prefix, which is equivalent to `/media/v1/` (PR #595) + +Bug fixes: + +- Fix bug with upgrading guest accounts where it would fail if you opened the registration email on a different device (PR #547) +- Fix bug where unread count could be wrong (PR #568) + +Changes in synapse v0.12.1-rc1 (2016-01-29) +=========================================== + +Features: + +- Add unread notification counts in `/sync` (PR #456) +- Add support for inviting 3pids in `/createRoom` (PR #460) +- Add ability for guest accounts to upgrade (PR #462) +- Add `/versions` API (PR #468) +- Add `event` to `/context` API (PR #492) +- Add specific error code for invalid user names in `/register` (PR #499) +- Add support for push badge counts (PR #507) +- Add support for non-guest users to peek in rooms using `/events` (PR #510) + +Changes: + +- Change `/sync` so that guest users only get rooms they\'ve joined (PR #469) +- Change to require unbanning before other membership changes (PR #501) +- Change default push rules to notify for all messages (PR #486) +- Change default push rules to not notify on membership changes (PR #514) +- Change default push rules in one to one rooms to only notify for events that are messages (PR #529) +- Change `/sync` to reject requests with a `from` query param (PR #512) +- Change server manhole to use SSH rather than telnet (PR #473) +- Change server to require AS users to be registered before use (PR #487) +- Change server not to start when ASes are invalidly configured (PR #494) +- Change server to require ID and `as_token` to be unique for AS\'s (PR #496) +- Change maximum pagination limit to 1000 (PR #497) + +Bug fixes: + +- Fix bug where `/sync` didn\'t return when something under the leave key changed (PR #461) +- Fix bug where we returned smaller rather than larger than requested thumbnails when `method=crop` (PR #464) +- Fix thumbnails API to only return cropped thumbnails when asking for a cropped thumbnail (PR #475) +- Fix bug where we occasionally still logged access tokens (PR #477) +- Fix bug where `/events` would always return immediately for guest users (PR #480) +- Fix bug where `/sync` unexpectedly returned old left rooms (PR #481) +- Fix enabling and disabling push rules (PR #498) +- Fix bug where `/register` returned 500 when given unicode username (PR #513) + +Changes in synapse v0.12.0 (2016-01-04) +======================================= + +- Expose `/login` under `r0` (PR #459) + +Changes in synapse v0.12.0-rc3 (2015-12-23) +=========================================== + +- Allow guest accounts access to `/sync` (PR #455) +- Allow filters to include/exclude rooms at the room level rather than just from the components of the sync for each room. (PR #454) +- Include urls for room avatars in the response to `/publicRooms` (PR #453) +- Don\'t set a identicon as the avatar for a user when they register (PR #450) +- Add a `display_name` to third-party invites (PR #449) +- Send more information to the identity server for third-party invites so that it can send richer messages to the invitee (PR #446) +- Cache the responses to `/initialSync` for 5 minutes. If a client retries a request to `/initialSync` before the a response was computed to the first request then the same response is used for both requests (PR #457) +- Fix a bug where synapse would always request the signing keys of remote servers even when the key was cached locally (PR #452) +- Fix 500 when pagination search results (PR #447) +- Fix a bug where synapse was leaking raw email address in third-party invites (PR #448) + +Changes in synapse v0.12.0-rc2 (2015-12-14) +=========================================== + +- Add caches for whether rooms have been forgotten by a user (PR #434) +- Remove instructions to use `--process-dependency-link` since all of the dependencies of synapse are on PyPI (PR #436) +- Parallelise the processing of `/sync` requests (PR #437) +- Fix race updating presence in `/events` (PR #444) +- Fix bug back-populating search results (PR #441) +- Fix bug calculating state in `/sync` requests (PR #442) + +Changes in synapse v0.12.0-rc1 (2015-12-10) +=========================================== + +- Host the client APIs released as r0 by on paths prefixed by `/_matrix/client/r0`. (PR #430, PR #415, PR #400) +- Updates the client APIs to match r0 of the matrix specification. + - All APIs return events in the new event format, old APIs also include the fields needed to parse the event using the old format for compatibility. (PR #402) + - Search results are now given as a JSON array rather than a JSON object (PR #405) + - Miscellaneous changes to search (PR #403, PR #406, PR #412) + - Filter JSON objects may now be passed as query parameters to `/sync` (PR #431) + - Fix implementation of `/admin/whois` (PR #418) + - Only include the rooms that user has left in `/sync` if the client requests them in the filter (PR #423) + - Don\'t push for `m.room.message` by default (PR #411) + - Add API for setting per account user data (PR #392) + - Allow users to forget rooms (PR #385) +- Performance improvements and monitoring: + - Add per-request counters for CPU time spent on the main python thread. (PR #421, PR #420) + - Add per-request counters for time spent in the database (PR #429) + - Make state updates in the C+S API idempotent (PR #416) + - Only fire `user_joined_room` if the user has actually joined. (PR #410) + - Reuse a single http client, rather than creating new ones (PR #413) +- Fixed a bug upgrading from older versions of synapse on postgresql (PR #417) + +Changes in synapse v0.11.1 (2015-11-20) +======================================= + +- Add extra options to search API (PR #394) +- Fix bug where we did not correctly cap federation retry timers. This meant it could take several hours for servers to start talking to ressurected servers, even when they were receiving traffic from them (PR #393) +- Don\'t advertise login token flow unless CAS is enabled. This caused issues where some clients would always use the fallback API if they did not recognize all login flows (PR #391) +- Change /v2 sync API to rename `private_user_data` to `account_data` (PR #386) +- Change /v2 sync API to remove the `event_map` and rename keys in `rooms` object (PR #389) + +Changes in synapse v0.11.0-r2 (2015-11-19) +========================================== + +- Fix bug in database port script (PR #387) + +Changes in synapse v0.11.0-r1 (2015-11-18) +========================================== + +- Retry and fail federation requests more aggressively for requests that block client side requests (PR #384) + +Changes in synapse v0.11.0 (2015-11-17) +======================================= + +- Change CAS login API (PR #349) + +Changes in synapse v0.11.0-rc2 (2015-11-13) +=========================================== + +- Various changes to /sync API response format (PR #373) +- Fix regression when setting display name in newly joined room over federation (PR #368) +- Fix problem where /search was slow when using SQLite (PR #366) + +Changes in synapse v0.11.0-rc1 (2015-11-11) +=========================================== + +- Add Search API (PR #307, #324, #327, #336, #350, #359) +- Add \'archived\' state to v2 /sync API (PR #316) +- Add ability to reject invites (PR #317) +- Add config option to disable password login (PR #322) +- Add the login fallback API (PR #330) +- Add room context API (PR #334) +- Add room tagging support (PR #335) +- Update v2 /sync API to match spec (PR #305, #316, #321, #332, #337, #341) +- Change retry schedule for application services (PR #320) +- Change retry schedule for remote servers (PR #340) +- Fix bug where we hosted static content in the incorrect place (PR #329) +- Fix bug where we didn\'t increment retry interval for remote servers (PR #343) + +Changes in synapse v0.10.1-rc1 (2015-10-15) +=========================================== + +- Add support for CAS, thanks to Steven Hammerton (PR #295, #296) +- Add support for using macaroons for `access_token` (PR #256, #229) +- Add support for `m.room.canonical_alias` (PR #287) +- Add support for viewing the history of rooms that they have left. (PR #276, #294) +- Add support for refresh tokens (PR #240) +- Add flag on creation which disables federation of the room (PR #279) +- Add some room state to invites. (PR #275) +- Atomically persist events when joining a room over federation (PR #283) +- Change default history visibility for private rooms (PR #271) +- Allow users to redact their own sent events (PR #262) +- Use tox for tests (PR #247) +- Split up syutil into separate libraries (PR #243) + +Changes in synapse v0.10.0-r2 (2015-09-16) +========================================== + +- Fix bug where we always fetched remote server signing keys instead of using ones in our cache. +- Fix adding threepids to an existing account. +- Fix bug with invinting over federation where remote server was already in the room. (PR #281, SYN-392) + +Changes in synapse v0.10.0-r1 (2015-09-08) +========================================== + +- Fix bug with python packaging + +Changes in synapse v0.10.0 (2015-09-03) +======================================= + +No change from release candidate. + +Changes in synapse v0.10.0-rc6 (2015-09-02) +=========================================== + +- Remove some of the old database upgrade scripts. +- Fix database port script to work with newly created sqlite databases. + +Changes in synapse v0.10.0-rc5 (2015-08-27) +=========================================== + +- Fix bug that broke downloading files with ascii filenames across federation. + +Changes in synapse v0.10.0-rc4 (2015-08-27) +=========================================== + +- Allow UTF-8 filenames for upload. (PR #259) + +Changes in synapse v0.10.0-rc3 (2015-08-25) +=========================================== + +- Add `--keys-directory` config option to specify where files such as certs and signing keys should be stored in, when using `--generate-config` or `--generate-keys`. (PR #250) +- Allow `--config-path` to specify a directory, causing synapse to use all \*.yaml files in the directory as config files. (PR #249) +- Add `web_client_location` config option to specify static files to be hosted by synapse under `/_matrix/client`. (PR #245) +- Add helper utility to synapse to read and parse the config files and extract the value of a given key. For example: + + $ python -m synapse.config read server_name -c homeserver.yaml + localhost + + (PR #246) + +Changes in synapse v0.10.0-rc2 (2015-08-24) +=========================================== + +- Fix bug where we incorrectly populated the `event_forward_extremities` table, resulting in problems joining large remote rooms (e.g. `#matrix:matrix.org`) +- Reduce the number of times we wake up pushers by not listening for presence or typing events, reducing the CPU cost of each pusher. + +Changes in synapse v0.10.0-rc1 (2015-08-21) +=========================================== + +Also see v0.9.4-rc1 changelog, which has been amalgamated into this release. + +General: + +- Upgrade to Twisted 15 (PR #173) +- Add support for serving and fetching encryption keys over federation. (PR #208) +- Add support for logging in with email address (PR #234) +- Add support for new `m.room.canonical_alias` event. (PR #233) +- Change synapse to treat user IDs case insensitively during registration and login. (If two users already exist with case insensitive matching user ids, synapse will continue to require them to specify their user ids exactly.) +- Error if a user tries to register with an email already in use. (PR #211) +- Add extra and improve existing caches (PR #212, #219, #226, #228) +- Batch various storage request (PR #226, #228) +- Fix bug where we didn\'t correctly log the entity that triggered the request if the request came in via an application service (PR #230) +- Fix bug where we needlessly regenerated the full list of rooms an AS is interested in. (PR #232) +- Add support for AS\'s to use v2\_alpha registration API (PR #210) + +Configuration: + +- Add `--generate-keys` that will generate any missing cert and key files in the configuration files. This is equivalent to running `--generate-config` on an existing configuration file. (PR #220) +- `--generate-config` now no longer requires a `--server-name` parameter when used on existing configuration files. (PR #220) +- Add `--print-pidfile` flag that controls the printing of the pid to stdout of the demonised process. (PR #213) + +Media Repository: + +- Fix bug where we picked a lower resolution image than requested. (PR #205) +- Add support for specifying if a the media repository should dynamically thumbnail images or not. (PR #206) + +Metrics: + +- Add statistics from the reactor to the metrics API. (PR #224, #225) + +Demo Homeservers: + +- Fix starting the demo homeservers without rate-limiting enabled. (PR #182) +- Fix enabling registration on demo homeservers (PR #223) + +Changes in synapse v0.9.4-rc1 (2015-07-21) +========================================== + +General: + +- Add basic implementation of receipts. (SPEC-99) +- Add support for configuration presets in room creation API. (PR #203) +- Add auth event that limits the visibility of history for new users. (SPEC-134) +- Add SAML2 login/registration support. (PR #201. Thanks Muthu Subramanian!) +- Add client side key management APIs for end to end encryption. (PR #198) +- Change power level semantics so that you cannot kick, ban or change power levels of users that have equal or greater power level than you. (SYN-192) +- Improve performance by bulk inserting events where possible. (PR #193) +- Improve performance by bulk verifying signatures where possible. (PR #194) + +Configuration: + +- Add support for including TLS certificate chains. + +Media Repository: + +- Add Content-Disposition headers to content repository responses. (SYN-150) + +Changes in synapse v0.9.3 (2015-07-01) +====================================== + +No changes from v0.9.3 Release Candidate 1. + +Changes in synapse v0.9.3-rc1 (2015-06-23) +========================================== + +General: + +- Fix a memory leak in the notifier. (SYN-412) +- Improve performance of room initial sync. (SYN-418) +- General improvements to logging. +- Remove `access_token` query params from `INFO` level logging. + +Configuration: + +- Add support for specifying and configuring multiple listeners. (SYN-389) + +Application services: + +- Fix bug where synapse failed to send user queries to application services. + +Changes in synapse v0.9.2-r2 (2015-06-15) +========================================= + +Fix packaging so that schema delta python files get included in the package. + +Changes in synapse v0.9.2 (2015-06-12) +====================================== + +General: + +- Use ultrajson for json (de)serialisation when a canonical encoding is not required. Ultrajson is significantly faster than simplejson in certain circumstances. +- Use connection pools for outgoing HTTP connections. +- Process thumbnails on separate threads. + +Configuration: + +- Add option, `gzip_responses`, to disable HTTP response compression. + +Federation: + +- Improve resilience of backfill by ensuring we fetch any missing auth events. +- Improve performance of backfill and joining remote rooms by removing unnecessary computations. This included handling events we\'d previously handled as well as attempting to compute the current state for outliers. + +Changes in synapse v0.9.1 (2015-05-26) +====================================== + +General: + +- Add support for backfilling when a client paginates. This allows servers to request history for a room from remote servers when a client tries to paginate history the server does not have - SYN-36 +- Fix bug where you couldn\'t disable non-default pushrules - SYN-378 +- Fix `register_new_user` script - SYN-359 +- Improve performance of fetching events from the database, this improves both initialSync and sending of events. +- Improve performance of event streams, allowing synapse to handle more simultaneous connected clients. + +Federation: + +- Fix bug with existing backfill implementation where it returned the wrong selection of events in some circumstances. +- Improve performance of joining remote rooms. + +Configuration: + +- Add support for changing the bind host of the metrics listener via the `metrics_bind_host` option. + +Changes in synapse v0.9.0-r5 (2015-05-21) +========================================= + +- Add more database caches to reduce amount of work done for each pusher. This radically reduces CPU usage when multiple pushers are set up in the same room. + +Changes in synapse v0.9.0 (2015-05-07) +====================================== + +General: + +- Add support for using a PostgreSQL database instead of SQLite. See [docs/postgres.rst](docs/postgres.rst) for details. +- Add password change and reset APIs. See [Registration](https://github.com/matrix-org/matrix-doc/blob/master/specification/10_client_server_api.rst#registration) in the spec. +- Fix memory leak due to not releasing stale notifiers - SYN-339. +- Fix race in caches that occasionally caused some presence updates to be dropped - SYN-369. +- Check server name has not changed on restart. +- Add a sample systemd unit file and a logger configuration in contrib/systemd. Contributed Ivan Shapovalov. + +Federation: + +- Add key distribution mechanisms for fetching public keys of unavailable remote homeservers. See [Retrieving Server Keys](https://github.com/matrix-org/matrix-doc/blob/6f2698/specification/30_server_server_api.rst#retrieving-server-keys) in the spec. + +Configuration: + +- Add support for multiple config files. +- Add support for dictionaries in config files. +- Remove support for specifying config options on the command line, except for: + - `--daemonize` - Daemonize the homeserver. + - `--manhole` - Turn on the twisted telnet manhole service on the given port. + - `--database-path` - The path to a sqlite database to use. + - `--verbose` - The verbosity level. + - `--log-file` - File to log to. + - `--log-config` - Python logging config file. + - `--enable-registration` - Enable registration for new users. + +Application services: + +- Reliably retry sending of events from Synapse to application services, as per [Application Services](https://github.com/matrix-org/matrix-doc/blob/0c6bd9/specification/25_application_service_api.rst#home-server---application-service-api) spec. +- Application services can no longer register via the `/register` API, instead their configuration should be saved to a file and listed in the synapse `app_service_config_files` config option. The AS configuration file has the same format as the old `/register` request. See [docs/application\_services.rst](docs/application_services.rst) for more information. + +Changes in synapse v0.8.1 (2015-03-18) +====================================== + +- Disable registration by default. New users can be added using the command `register_new_matrix_user` or by enabling registration in the config. +- Add metrics to synapse. To enable metrics use config options `enable_metrics` and `metrics_port`. +- Fix bug where banning only kicked the user. + +Changes in synapse v0.8.0 (2015-03-06) +====================================== + +General: + +- Add support for registration fallback. This is a page hosted on the server which allows a user to register for an account, regardless of what client they are using (e.g. mobile devices). +- Added new default push rules and made them configurable by clients: + - Suppress all notice messages. + - Notify when invited to a new room. + - Notify for messages that don\'t match any rule. + - Notify on incoming call. + +Federation: + +- Added per host server side rate-limiting of incoming federation requests. +- Added a `/get_missing_events/` API to federation to reduce number of `/events/` requests. + +Configuration: + +- Added configuration option to disable registration: `disable_registration`. +- Added configuration option to change soft limit of number of open file descriptors: `soft_file_limit`. +- Make `tls_private_key_path` optional when running with `no_tls`. + +Application services: + +- Application services can now poll on the CS API `/events` for their events, by providing their application service `access_token`. +- Added exclusive namespace support to application services API. + +Changes in synapse v0.7.1 (2015-02-19) +====================================== + +- Initial alpha implementation of parts of the Application Services API. Including: + - AS Registration / Unregistration + - User Query API + - Room Alias Query API + - Push transport for receiving events. + - User/Alias namespace admin control +- Add cache when fetching events from remote servers to stop repeatedly fetching events with bad signatures. +- Respect the per remote server retry scheme when fetching both events and server keys to reduce the number of times we send requests to dead servers. +- Inform remote servers when the local server fails to handle a received event. +- Turn off python bytecode generation due to problems experienced when upgrading from previous versions. + +Changes in synapse v0.7.0 (2015-02-12) +====================================== + +- Add initial implementation of the query auth federation API, allowing servers to agree on whether an event should be allowed or rejected. +- Persist events we have rejected from federation, fixing the bug where servers would keep requesting the same events. +- Various federation performance improvements, including: + - Add in memory caches on queries such as: + + > - Computing the state of a room at a point in time, used for authorization on federation requests. + > - Fetching events from the database. + > - User\'s room membership, used for authorizing presence updates. + + - Upgraded JSON library to improve parsing and serialisation speeds. + +- Add default avatars to new user accounts using pydenticon library. +- Correctly time out federation requests. +- Retry federation requests against different servers. +- Add support for push and push rules. +- Add alpha versions of proposed new CSv2 APIs, including `/sync` API. + +Changes in synapse 0.6.1 (2015-01-07) +===================================== + +- Major optimizations to improve performance of initial sync and event sending in large rooms (by up to 10x) +- Media repository now includes a Content-Length header on media downloads. +- Improve quality of thumbnails by changing resizing algorithm. + +Changes in synapse 0.6.0 (2014-12-16) +===================================== + +- Add new API for media upload and download that supports thumbnailing. +- Replicate media uploads over multiple homeservers so media is always served to clients from their local homeserver. This obsoletes the \--content-addr parameter and confusion over accessing content directly from remote homeservers. +- Implement exponential backoff when retrying federation requests when sending to remote homeservers which are offline. +- Implement typing notifications. +- Fix bugs where we sent events with invalid signatures due to bugs where we incorrectly persisted events. +- Improve performance of database queries involving retrieving events. + +Changes in synapse 0.5.4a (2014-12-13) +====================================== + +- Fix bug while generating the error message when a file path specified in the config doesn\'t exist. + +Changes in synapse 0.5.4 (2014-12-03) +===================================== + +- Fix presence bug where some rooms did not display presence updates for remote users. +- Do not log SQL timing log lines when started with \"-v\" +- Fix potential memory leak. + +Changes in synapse 0.5.3c (2014-12-02) +====================================== + +- Change the default value for the content\_addr option to use the HTTP listener, as by default the HTTPS listener will be using a self-signed certificate. + +Changes in synapse 0.5.3 (2014-11-27) +===================================== + +- Fix bug that caused joining a remote room to fail if a single event was not signed correctly. +- Fix bug which caused servers to continuously try and fetch events from other servers. + +Changes in synapse 0.5.2 (2014-11-26) +===================================== + +Fix major bug that caused rooms to disappear from peoples initial sync. + +Changes in synapse 0.5.1 (2014-11-26) +===================================== + +See UPGRADES.rst for specific instructions on how to upgrade. + +- Fix bug where we served up an Event that did not match its signatures. +- Fix regression where we no longer correctly handled the case where a homeserver receives an event for a room it doesn\'t recognise (but is in.) + +Changes in synapse 0.5.0 (2014-11-19) +===================================== + +This release includes changes to the federation protocol and client-server API that is not backwards compatible. + +This release also changes the internal database schemas and so requires servers to drop their current history. See UPGRADES.rst for details. + +Homeserver: + +- Add authentication and authorization to the federation protocol. Events are now signed by their originating homeservers. +- Implement the new authorization model for rooms. +- Split out web client into a seperate repository: matrix-angular-sdk. +- Change the structure of PDUs. +- Fix bug where user could not join rooms via an alias containing 4-byte UTF-8 characters. +- Merge concept of PDUs and Events internally. +- Improve logging by adding request ids to log lines. +- Implement a very basic room initial sync API. +- Implement the new invite/join federation APIs. + +Webclient: + +- The webclient has been moved to a seperate repository. + +Changes in synapse 0.4.2 (2014-10-31) +===================================== + +Homeserver: + +- Fix bugs where we did not notify users of correct presence updates. +- Fix bug where we did not handle sub second event stream timeouts. + +Webclient: + +- Add ability to click on messages to see JSON. +- Add ability to redact messages. +- Add ability to view and edit all room state JSON. +- Handle incoming redactions. +- Improve feedback on errors. +- Fix bugs in mobile CSS. +- Fix bugs with desktop notifications. + +Changes in synapse 0.4.1 (2014-10-17) +===================================== + +Webclient: + +- Fix bug with display of timestamps. + +Changes in synpase 0.4.0 (2014-10-17) +===================================== + +This release includes changes to the federation protocol and client-server API that is not backwards compatible. + +The Matrix specification has been moved to a separate git repository: + +You will also need an updated syutil and config. See UPGRADES.rst. + +Homeserver: + +- Sign federation transactions to assert strong identity over federation. +- Rename timestamp keys in PDUs and events from \'ts\' and \'hsob\_ts\' to \'origin\_server\_ts\'. + +Changes in synapse 0.3.4 (2014-09-25) +===================================== + +This version adds support for using a TURN server. See docs/turn-howto.rst on how to set one up. + +Homeserver: + +- Add support for redaction of messages. +- Fix bug where inviting a user on a remote homeserver could take up to 20-30s. +- Implement a get current room state API. +- Add support specifying and retrieving turn server configuration. + +Webclient: + +- Add button to send messages to users from the home page. +- Add support for using TURN for VoIP calls. +- Show display name change messages. +- Fix bug where the client didn\'t get the state of a newly joined room until after it has been refreshed. +- Fix bugs with tab complete. +- Fix bug where holding down the down arrow caused chrome to chew 100% CPU. +- Fix bug where desktop notifications occasionally used \"Undefined\" as the display name. +- Fix more places where we sometimes saw room IDs incorrectly. +- Fix bug which caused lag when entering text in the text box. + +Changes in synapse 0.3.3 (2014-09-22) +===================================== + +Homeserver: + +- Fix bug where you continued to get events for rooms you had left. + +Webclient: + +- Add support for video calls with basic UI. +- Fix bug where one to one chats were named after your display name rather than the other person\'s. +- Fix bug which caused lag when typing in the textarea. +- Refuse to run on browsers we know won\'t work. +- Trigger pagination when joining new rooms. +- Fix bug where we sometimes didn\'t display invitations in recents. +- Automatically join room when accepting a VoIP call. +- Disable outgoing and reject incoming calls on browsers we don\'t support VoIP in. +- Don\'t display desktop notifications for messages in the room you are non-idle and speaking in. + +Changes in synapse 0.3.2 (2014-09-18) +===================================== + +Webclient: + +- Fix bug where an empty \"bing words\" list in old accounts didn\'t send notifications when it should have done. + +Changes in synapse 0.3.1 (2014-09-18) +===================================== + +This is a release to hotfix v0.3.0 to fix two regressions. + +Webclient: + +- Fix a regression where we sometimes displayed duplicate events. +- Fix a regression where we didn\'t immediately remove rooms you were banned in from the recents list. + +Changes in synapse 0.3.0 (2014-09-18) +===================================== + +See UPGRADE for information about changes to the client server API, including breaking backwards compatibility with VoIP calls and registration API. + +Homeserver: + +- When a user changes their displayname or avatar the server will now update all their join states to reflect this. +- The server now adds \"age\" key to events to indicate how old they are. This is clock independent, so at no point does any server or webclient have to assume their clock is in sync with everyone else. +- Fix bug where we didn\'t correctly pull in missing PDUs. +- Fix bug where prev\_content key wasn\'t always returned. +- Add support for password resets. + +Webclient: + +- Improve page content loading. +- Join/parts now trigger desktop notifications. +- Always show room aliases in the UI if one is present. +- No longer show user-count in the recents side panel. +- Add up & down arrow support to the text box for message sending to step through your sent history. +- Don\'t display notifications for our own messages. +- Emotes are now formatted correctly in desktop notifications. +- The recents list now differentiates between public & private rooms. +- Fix bug where when switching between rooms the pagination flickered before the view jumped to the bottom of the screen. +- Add bing word support. + +Registration API: + +- The registration API has been overhauled to function like the login API. In practice, this means registration requests must now include the following: \'type\':\'m.login.password\'. See UPGRADE for more information on this. +- The \'user\_id\' key has been renamed to \'user\' to better match the login API. +- There is an additional login type: \'m.login.email.identity\'. +- The command client and web client have been updated to reflect these changes. + +Changes in synapse 0.2.3 (2014-09-12) +===================================== + +Homeserver: + +- Fix bug where we stopped sending events to remote homeservers if a user from that homeserver left, even if there were some still in the room. +- Fix bugs in the state conflict resolution where it was incorrectly rejecting events. + +Webclient: + +- Display room names and topics. +- Allow setting/editing of room names and topics. +- Display information about rooms on the main page. +- Handle ban and kick events in real time. +- VoIP UI and reliability improvements. +- Add glare support for VoIP. +- Improvements to initial startup speed. +- Don\'t display duplicate join events. +- Local echo of messages. +- Differentiate sending and sent of local echo. +- Various minor bug fixes. + +Changes in synapse 0.2.2 (2014-09-06) +===================================== + +Homeserver: + +- When the server returns state events it now also includes the previous content. +- Add support for inviting people when creating a new room. +- Make the homeserver inform the room via m.room.aliases when a new alias is added for a room. +- Validate m.room.power\_level events. + +Webclient: + +- Add support for captchas on registration. +- Handle m.room.aliases events. +- Asynchronously send messages and show a local echo. +- Inform the UI when a message failed to send. +- Only autoscroll on receiving a new message if the user was already at the bottom of the screen. +- Add support for ban/kick reasons. + +Changes in synapse 0.2.1 (2014-09-03) +===================================== + +Homeserver: + +- Added support for signing up with a third party id. +- Add synctl scripts. +- Added rate limiting. +- Add option to change the external address the content repo uses. +- Presence bug fixes. + +Webclient: + +- Added support for signing up with a third party id. +- Added support for banning and kicking users. +- Added support for displaying and setting ops. +- Added support for room names. +- Fix bugs with room membership event display. + +Changes in synapse 0.2.0 (2014-09-02) +===================================== + +This update changes many configuration options, updates the database schema and mandates SSL for server-server connections. + +Homeserver: + +- Require SSL for server-server connections. +- Add SSL listener for client-server connections. +- Add ability to use config files. +- Add support for kicking/banning and power levels. +- Allow setting of room names and topics on creation. +- Change presence to include last seen time of the user. +- Change url path prefix to /\_matrix/\... +- Bug fixes to presence. + +Webclient: + +- Reskin the CSS for registration and login. +- Various improvements to rooms CSS. +- Support changes in client-server API. +- Bug fixes to VOIP UI. +- Various bug fixes to handling of changes to room member list. + +Changes in synapse 0.1.2 (2014-08-29) +===================================== + +Webclient: + +- Add basic call state UI for VoIP calls. + +Changes in synapse 0.1.1 (2014-08-29) +===================================== + +Homeserver: + +- Fix bug that caused the event stream to not notify some clients about changes. + +Changes in synapse 0.1.0 (2014-08-29) +===================================== + +Presence has been reenabled in this release. + +Homeserver: + +- Update client to server API, including: + - Use a more consistent url scheme. + - Provide more useful information in the initial sync api. +- Change the presence handling to be much more efficient. +- Change the presence server to server API to not require explicit polling of all users who share a room with a user. +- Fix races in the event streaming logic. + +Webclient: + +- Update to use new client to server API. +- Add basic VOIP support. +- Add idle timers that change your status to away. +- Add recent rooms column when viewing a room. +- Various network efficiency improvements. +- Add basic mobile browser support. +- Add a settings page. + +Changes in synapse 0.0.1 (2014-08-22) +===================================== + +Presence has been disabled in this release due to a bug that caused the homeserver to spam other remote homeservers. + +Homeserver: + +- Completely change the database schema to support generic event types. +- Improve presence reliability. +- Improve reliability of joining remote rooms. +- Fix bug where room join events were duplicated. +- Improve initial sync API to return more information to the client. +- Stop generating fake messages for room membership events. + +Webclient: + +- Add tab completion of names. +- Add ability to upload and send images. +- Add profile pages. +- Improve CSS layout of room. +- Disambiguate identical display names. +- Don\'t get remote users display names and avatars individually. +- Use the new initial sync API to reduce number of round trips to the homeserver. +- Change url scheme to use room aliases instead of room ids where known. +- Increase longpoll timeout. + +Changes in synapse 0.0.0 (2014-08-13) +===================================== + +- Initial alpha release diff --git a/CHANGES.md b/CHANGES.md index 36ff2499af..d2c49498e9 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -6440,3645 +6440,3 @@ Internal Changes - Various improvements to debug logging. ([\#5353](https://github.com/matrix-org/synapse/issues/5353)) - Don't run CI build checks until sample config check has passed. ([\#5370](https://github.com/matrix-org/synapse/issues/5370)) - Automatically retry buildkite builds (max twice) when an agent is lost. ([\#5380](https://github.com/matrix-org/synapse/issues/5380)) - - -Synapse 0.99.5.2 (2019-05-30) -============================= - -Bugfixes --------- - -- Fix bug where we leaked extremities when we soft failed events, leading to performance degradation. ([\#5274](https://github.com/matrix-org/synapse/issues/5274), [\#5278](https://github.com/matrix-org/synapse/issues/5278), [\#5291](https://github.com/matrix-org/synapse/issues/5291)) - - -Synapse 0.99.5.1 (2019-05-22) -============================= - -0.99.5.1 supersedes 0.99.5 due to malformed debian changelog - no functional changes. - -Synapse 0.99.5 (2019-05-22) -=========================== - -No significant changes. - - -Synapse 0.99.5rc1 (2019-05-21) -============================== - -Features --------- - -- Add ability to blacklist IP ranges for the federation client. ([\#5043](https://github.com/matrix-org/synapse/issues/5043)) -- Ratelimiting configuration for clients sending messages and the federation server has been altered to match login ratelimiting. The old configuration names will continue working. Check the sample config for details of the new names. ([\#5181](https://github.com/matrix-org/synapse/issues/5181)) -- Drop support for the undocumented /_matrix/client/v2_alpha API prefix. ([\#5190](https://github.com/matrix-org/synapse/issues/5190)) -- Add an option to disable per-room profiles. ([\#5196](https://github.com/matrix-org/synapse/issues/5196)) -- Stick an expiration date to any registered user missing one at startup if account validity is enabled. ([\#5204](https://github.com/matrix-org/synapse/issues/5204)) -- Add experimental support for relations (aka reactions and edits). ([\#5209](https://github.com/matrix-org/synapse/issues/5209), [\#5211](https://github.com/matrix-org/synapse/issues/5211), [\#5203](https://github.com/matrix-org/synapse/issues/5203), [\#5212](https://github.com/matrix-org/synapse/issues/5212)) -- Add a room version 4 which uses a new event ID format, as per [MSC2002](https://github.com/matrix-org/matrix-doc/pull/2002). ([\#5210](https://github.com/matrix-org/synapse/issues/5210), [\#5217](https://github.com/matrix-org/synapse/issues/5217)) - - -Bugfixes --------- - -- Fix image orientation when generating thumbnails (needs pillow>=4.3.0). Contributed by Pau Rodriguez-Estivill. ([\#5039](https://github.com/matrix-org/synapse/issues/5039)) -- Exclude soft-failed events from forward-extremity candidates: fixes "No forward extremities left!" error. ([\#5146](https://github.com/matrix-org/synapse/issues/5146)) -- Re-order stages in registration flows such that msisdn and email verification are done last. ([\#5174](https://github.com/matrix-org/synapse/issues/5174)) -- Fix 3pid guest invites. ([\#5177](https://github.com/matrix-org/synapse/issues/5177)) -- Fix a bug where the register endpoint would fail with M_THREEPID_IN_USE instead of returning an account previously registered in the same session. ([\#5187](https://github.com/matrix-org/synapse/issues/5187)) -- Prevent registration for user ids that are too long to fit into a state key. Contributed by Reid Anderson. ([\#5198](https://github.com/matrix-org/synapse/issues/5198)) -- Fix incompatibility between ACME support and Python 3.5.2. ([\#5218](https://github.com/matrix-org/synapse/issues/5218)) -- Fix error handling for rooms whose versions are unknown. ([\#5219](https://github.com/matrix-org/synapse/issues/5219)) - - -Internal Changes ----------------- - -- Make /sync attempt to return device updates for both joined and invited users. Note that this doesn't currently work correctly due to other bugs. ([\#3484](https://github.com/matrix-org/synapse/issues/3484)) -- Update tests to consistently be configured via the same code that is used when loading from configuration files. ([\#5171](https://github.com/matrix-org/synapse/issues/5171), [\#5185](https://github.com/matrix-org/synapse/issues/5185)) -- Allow client event serialization to be async. ([\#5183](https://github.com/matrix-org/synapse/issues/5183)) -- Expose DataStore._get_events as get_events_as_list. ([\#5184](https://github.com/matrix-org/synapse/issues/5184)) -- Make generating SQL bounds for pagination generic. ([\#5191](https://github.com/matrix-org/synapse/issues/5191)) -- Stop telling people to install the optional dependencies by default. ([\#5197](https://github.com/matrix-org/synapse/issues/5197)) - - -Synapse 0.99.4 (2019-05-15) -=========================== - -No significant changes. - - -Synapse 0.99.4rc1 (2019-05-13) -============================== - -Features --------- - -- Add systemd-python to the optional dependencies to enable logging to the systemd journal. Install with `pip install matrix-synapse[systemd]`. ([\#4339](https://github.com/matrix-org/synapse/issues/4339)) -- Add a default .m.rule.tombstone push rule. ([\#4867](https://github.com/matrix-org/synapse/issues/4867)) -- Add ability for password provider modules to bind email addresses to users upon registration. ([\#4947](https://github.com/matrix-org/synapse/issues/4947)) -- Implementation of [MSC1711](https://github.com/matrix-org/matrix-doc/pull/1711) including config options for requiring valid TLS certificates for federation traffic, the ability to disable TLS validation for specific domains, and the ability to specify your own list of CA certificates. ([\#4967](https://github.com/matrix-org/synapse/issues/4967)) -- Remove presence list support as per MSC 1819. ([\#4989](https://github.com/matrix-org/synapse/issues/4989)) -- Reduce CPU usage starting pushers during start up. ([\#4991](https://github.com/matrix-org/synapse/issues/4991)) -- Add a delete group admin API. ([\#5002](https://github.com/matrix-org/synapse/issues/5002)) -- Add config option to block users from looking up 3PIDs. ([\#5010](https://github.com/matrix-org/synapse/issues/5010)) -- Add context to phonehome stats. ([\#5020](https://github.com/matrix-org/synapse/issues/5020)) -- Configure the example systemd units to have a log identifier of `matrix-synapse` - instead of the executable name, `python`. - Contributed by Christoph Müller. ([\#5023](https://github.com/matrix-org/synapse/issues/5023)) -- Add time-based account expiration. ([\#5027](https://github.com/matrix-org/synapse/issues/5027), [\#5047](https://github.com/matrix-org/synapse/issues/5047), [\#5073](https://github.com/matrix-org/synapse/issues/5073), [\#5116](https://github.com/matrix-org/synapse/issues/5116)) -- Add support for handling `/versions`, `/voip` and `/push_rules` client endpoints to client_reader worker. ([\#5063](https://github.com/matrix-org/synapse/issues/5063), [\#5065](https://github.com/matrix-org/synapse/issues/5065), [\#5070](https://github.com/matrix-org/synapse/issues/5070)) -- Add a configuration option to require authentication on /publicRooms and /profile endpoints. ([\#5083](https://github.com/matrix-org/synapse/issues/5083)) -- Move admin APIs to `/_synapse/admin/v1`. (The old paths are retained for backwards-compatibility, for now). ([\#5119](https://github.com/matrix-org/synapse/issues/5119)) -- Implement an admin API for sending server notices. Many thanks to @krombel who provided a foundation for this work. ([\#5121](https://github.com/matrix-org/synapse/issues/5121), [\#5142](https://github.com/matrix-org/synapse/issues/5142)) - - -Bugfixes --------- - -- Avoid redundant URL encoding of redirect URL for SSO login in the fallback login page. Fixes a regression introduced in [#4220](https://github.com/matrix-org/synapse/pull/4220). Contributed by Marcel Fabian Krüger ("[zaugin](https://github.com/zauguin)"). ([\#4555](https://github.com/matrix-org/synapse/issues/4555)) -- Fix bug where presence updates were sent to all servers in a room when a new server joined, rather than to just the new server. ([\#4942](https://github.com/matrix-org/synapse/issues/4942), [\#5103](https://github.com/matrix-org/synapse/issues/5103)) -- Fix sync bug which made accepting invites unreliable in worker-mode synapses. ([\#4955](https://github.com/matrix-org/synapse/issues/4955), [\#4956](https://github.com/matrix-org/synapse/issues/4956)) -- start.sh: Fix the --no-rate-limit option for messages and make it bypass rate limit on registration and login too. ([\#4981](https://github.com/matrix-org/synapse/issues/4981)) -- Transfer related groups on room upgrade. ([\#4990](https://github.com/matrix-org/synapse/issues/4990)) -- Prevent the ability to kick users from a room they aren't in. ([\#4999](https://github.com/matrix-org/synapse/issues/4999)) -- Fix issue #4596 so synapse_port_db script works with --curses option on Python 3. Contributed by Anders Jensen-Waud . ([\#5003](https://github.com/matrix-org/synapse/issues/5003)) -- Clients timing out/disappearing while downloading from the media repository will now no longer log a spurious "Producer was not unregistered" message. ([\#5009](https://github.com/matrix-org/synapse/issues/5009)) -- Fix "cannot import name execute_batch" error with postgres. ([\#5032](https://github.com/matrix-org/synapse/issues/5032)) -- Fix disappearing exceptions in manhole. ([\#5035](https://github.com/matrix-org/synapse/issues/5035)) -- Workaround bug in twisted where attempting too many concurrent DNS requests could cause it to hang due to running out of file descriptors. ([\#5037](https://github.com/matrix-org/synapse/issues/5037)) -- Make sure we're not registering the same 3pid twice on registration. ([\#5071](https://github.com/matrix-org/synapse/issues/5071)) -- Don't crash on lack of expiry templates. ([\#5077](https://github.com/matrix-org/synapse/issues/5077)) -- Fix the ratelimiting on third party invites. ([\#5104](https://github.com/matrix-org/synapse/issues/5104)) -- Add some missing limitations to room alias creation. ([\#5124](https://github.com/matrix-org/synapse/issues/5124), [\#5128](https://github.com/matrix-org/synapse/issues/5128)) -- Limit the number of EDUs in transactions to 100 as expected by synapse. Thanks to @superboum for this work! ([\#5138](https://github.com/matrix-org/synapse/issues/5138)) - -Internal Changes ----------------- - -- Add test to verify threepid auth check added in #4435. ([\#4474](https://github.com/matrix-org/synapse/issues/4474)) -- Fix/improve some docstrings in the replication code. ([\#4949](https://github.com/matrix-org/synapse/issues/4949)) -- Split synapse.replication.tcp.streams into smaller files. ([\#4953](https://github.com/matrix-org/synapse/issues/4953)) -- Refactor replication row generation/parsing. ([\#4954](https://github.com/matrix-org/synapse/issues/4954)) -- Run `black` to clean up formatting on `synapse/storage/roommember.py` and `synapse/storage/events.py`. ([\#4959](https://github.com/matrix-org/synapse/issues/4959)) -- Remove log line for password via the admin API. ([\#4965](https://github.com/matrix-org/synapse/issues/4965)) -- Fix typo in TLS filenames in docker/README.md. Also add the '-p' commandline option to the 'docker run' example. Contributed by Jurrie Overgoor. ([\#4968](https://github.com/matrix-org/synapse/issues/4968)) -- Refactor room version definitions. ([\#4969](https://github.com/matrix-org/synapse/issues/4969)) -- Reduce log level of .well-known/matrix/client responses. ([\#4972](https://github.com/matrix-org/synapse/issues/4972)) -- Add `config.signing_key_path` that can be read by `synapse.config` utility. ([\#4974](https://github.com/matrix-org/synapse/issues/4974)) -- Track which identity server is used when binding a threepid and use that for unbinding, as per MSC1915. ([\#4982](https://github.com/matrix-org/synapse/issues/4982)) -- Rewrite KeyringTestCase as a HomeserverTestCase. ([\#4985](https://github.com/matrix-org/synapse/issues/4985)) -- README updates: Corrected the default POSTGRES_USER. Added port forwarding hint in TLS section. ([\#4987](https://github.com/matrix-org/synapse/issues/4987)) -- Remove a number of unused tables from the database schema. ([\#4992](https://github.com/matrix-org/synapse/issues/4992), [\#5028](https://github.com/matrix-org/synapse/issues/5028), [\#5033](https://github.com/matrix-org/synapse/issues/5033)) -- Run `black` on the remainder of `synapse/storage/`. ([\#4996](https://github.com/matrix-org/synapse/issues/4996)) -- Fix grammar in get_current_users_in_room and give it a docstring. ([\#4998](https://github.com/matrix-org/synapse/issues/4998)) -- Clean up some code in the server-key Keyring. ([\#5001](https://github.com/matrix-org/synapse/issues/5001)) -- Convert SYNAPSE_NO_TLS Docker variable to boolean for user friendliness. Contributed by Gabriel Eckerson. ([\#5005](https://github.com/matrix-org/synapse/issues/5005)) -- Refactor synapse.storage._base._simple_select_list_paginate. ([\#5007](https://github.com/matrix-org/synapse/issues/5007)) -- Store the notary server name correctly in server_keys_json. ([\#5024](https://github.com/matrix-org/synapse/issues/5024)) -- Rewrite Datastore.get_server_verify_keys to reduce the number of database transactions. ([\#5030](https://github.com/matrix-org/synapse/issues/5030)) -- Remove extraneous period from copyright headers. ([\#5046](https://github.com/matrix-org/synapse/issues/5046)) -- Update documentation for where to get Synapse packages. ([\#5067](https://github.com/matrix-org/synapse/issues/5067)) -- Add workarounds for pep-517 install errors. ([\#5098](https://github.com/matrix-org/synapse/issues/5098)) -- Improve logging when event-signature checks fail. ([\#5100](https://github.com/matrix-org/synapse/issues/5100)) -- Factor out an "assert_requester_is_admin" function. ([\#5120](https://github.com/matrix-org/synapse/issues/5120)) -- Remove the requirement to authenticate for /admin/server_version. ([\#5122](https://github.com/matrix-org/synapse/issues/5122)) -- Prevent an exception from being raised in a IResolutionReceiver and use a more generic error message for blacklisted URL previews. ([\#5155](https://github.com/matrix-org/synapse/issues/5155)) -- Run `black` on the tests directory. ([\#5170](https://github.com/matrix-org/synapse/issues/5170)) -- Fix CI after new release of isort. ([\#5179](https://github.com/matrix-org/synapse/issues/5179)) -- Fix bogus imports in unit tests. ([\#5154](https://github.com/matrix-org/synapse/issues/5154)) - - -Synapse 0.99.3.2 (2019-05-03) -============================= - -Internal Changes ----------------- - -- Ensure that we have `urllib3` <1.25, to resolve incompatibility with `requests`. ([\#5135](https://github.com/matrix-org/synapse/issues/5135)) - - -Synapse 0.99.3.1 (2019-05-03) -============================= - -Security update ---------------- - -This release includes two security fixes: - -- Switch to using a cryptographically-secure random number generator for token strings, ensuring they cannot be predicted by an attacker. Thanks to @opnsec for identifying and responsibly disclosing this issue! ([\#5133](https://github.com/matrix-org/synapse/issues/5133)) -- Blacklist 0.0.0.0 and :: by default for URL previews. Thanks to @opnsec for identifying and responsibly disclosing this issue too! ([\#5134](https://github.com/matrix-org/synapse/issues/5134)) - -Synapse 0.99.3 (2019-04-01) -=========================== - -No significant changes. - - -Synapse 0.99.3rc1 (2019-03-27) -============================== - -Features --------- - -- The user directory has been rewritten to make it faster, with less chance of falling behind on a large server. ([\#4537](https://github.com/matrix-org/synapse/issues/4537), [\#4846](https://github.com/matrix-org/synapse/issues/4846), [\#4864](https://github.com/matrix-org/synapse/issues/4864), [\#4887](https://github.com/matrix-org/synapse/issues/4887), [\#4900](https://github.com/matrix-org/synapse/issues/4900), [\#4944](https://github.com/matrix-org/synapse/issues/4944)) -- Add configurable rate limiting to the /register endpoint. ([\#4735](https://github.com/matrix-org/synapse/issues/4735), [\#4804](https://github.com/matrix-org/synapse/issues/4804)) -- Move server key queries to federation reader. ([\#4757](https://github.com/matrix-org/synapse/issues/4757)) -- Add support for /account/3pid REST endpoint to client_reader worker. ([\#4759](https://github.com/matrix-org/synapse/issues/4759)) -- Add an endpoint to the admin API for querying the server version. Contributed by Joseph Weston. ([\#4772](https://github.com/matrix-org/synapse/issues/4772)) -- Include a default configuration file in the 'docs' directory. ([\#4791](https://github.com/matrix-org/synapse/issues/4791), [\#4801](https://github.com/matrix-org/synapse/issues/4801)) -- Synapse is now permissive about trailing slashes on some of its federation endpoints, allowing zero or more to be present. ([\#4793](https://github.com/matrix-org/synapse/issues/4793)) -- Add support for /keys/query and /keys/changes REST endpoints to client_reader worker. ([\#4796](https://github.com/matrix-org/synapse/issues/4796)) -- Add checks to incoming events over federation for events evading auth (aka "soft fail"). ([\#4814](https://github.com/matrix-org/synapse/issues/4814)) -- Add configurable rate limiting to the /login endpoint. ([\#4821](https://github.com/matrix-org/synapse/issues/4821), [\#4865](https://github.com/matrix-org/synapse/issues/4865)) -- Remove trailing slashes from certain outbound federation requests. Retry if receiving a 404. Context: #3622. ([\#4840](https://github.com/matrix-org/synapse/issues/4840)) -- Allow passing --daemonize flags to workers in the same way as with master. ([\#4853](https://github.com/matrix-org/synapse/issues/4853)) -- Batch up outgoing read-receipts to reduce federation traffic. ([\#4890](https://github.com/matrix-org/synapse/issues/4890), [\#4927](https://github.com/matrix-org/synapse/issues/4927)) -- Add option to disable searching the user directory. ([\#4895](https://github.com/matrix-org/synapse/issues/4895)) -- Add option to disable searching of local and remote public room lists. ([\#4896](https://github.com/matrix-org/synapse/issues/4896)) -- Add ability for password providers to login/register a user via 3PID (email, phone). ([\#4931](https://github.com/matrix-org/synapse/issues/4931)) - - -Bugfixes --------- - -- Fix a bug where media with spaces in the name would get a corrupted name. ([\#2090](https://github.com/matrix-org/synapse/issues/2090)) -- Fix attempting to paginate in rooms where server cannot see any events, to avoid unnecessarily pulling in lots of redacted events. ([\#4699](https://github.com/matrix-org/synapse/issues/4699)) -- 'event_id' is now a required parameter in federated state requests, as per the matrix spec. ([\#4740](https://github.com/matrix-org/synapse/issues/4740)) -- Fix tightloop over connecting to replication server. ([\#4749](https://github.com/matrix-org/synapse/issues/4749)) -- Fix parsing of Content-Disposition headers on remote media requests and URL previews. ([\#4763](https://github.com/matrix-org/synapse/issues/4763)) -- Fix incorrect log about not persisting duplicate state event. ([\#4776](https://github.com/matrix-org/synapse/issues/4776)) -- Fix v4v6 option in HAProxy example config. Contributed by Flakebi. ([\#4790](https://github.com/matrix-org/synapse/issues/4790)) -- Handle batch updates in worker replication protocol. ([\#4792](https://github.com/matrix-org/synapse/issues/4792)) -- Fix bug where we didn't correctly throttle sending of USER_IP commands over replication. ([\#4818](https://github.com/matrix-org/synapse/issues/4818)) -- Fix potential race in handling missing updates in device list updates. ([\#4829](https://github.com/matrix-org/synapse/issues/4829)) -- Fix bug where synapse expected an un-specced `prev_state` field on state events. ([\#4837](https://github.com/matrix-org/synapse/issues/4837)) -- Transfer a user's notification settings (push rules) on room upgrade. ([\#4838](https://github.com/matrix-org/synapse/issues/4838)) -- fix test_auto_create_auto_join_where_no_consent. ([\#4886](https://github.com/matrix-org/synapse/issues/4886)) -- Fix a bug where hs_disabled_message was sometimes not correctly enforced. ([\#4888](https://github.com/matrix-org/synapse/issues/4888)) -- Fix bug in shutdown room admin API where it would fail if a user in the room hadn't consented to the privacy policy. ([\#4904](https://github.com/matrix-org/synapse/issues/4904)) -- Fix bug where blocked world-readable rooms were still peekable. ([\#4908](https://github.com/matrix-org/synapse/issues/4908)) - - -Internal Changes ----------------- - -- Add a systemd setup that supports synapse workers. Contributed by Luca Corbatto. ([\#4662](https://github.com/matrix-org/synapse/issues/4662)) -- Change from TravisCI to Buildkite for CI. ([\#4752](https://github.com/matrix-org/synapse/issues/4752)) -- When presence is disabled don't send over replication. ([\#4757](https://github.com/matrix-org/synapse/issues/4757)) -- Minor docstring fixes for MatrixFederationAgent. ([\#4765](https://github.com/matrix-org/synapse/issues/4765)) -- Optimise EDU transmission for the federation_sender worker. ([\#4770](https://github.com/matrix-org/synapse/issues/4770)) -- Update test_typing to use HomeserverTestCase. ([\#4771](https://github.com/matrix-org/synapse/issues/4771)) -- Update URLs for riot.im icons and logos in the default notification templates. ([\#4779](https://github.com/matrix-org/synapse/issues/4779)) -- Removed unnecessary $ from some federation endpoint path regexes. ([\#4794](https://github.com/matrix-org/synapse/issues/4794)) -- Remove link to deleted title in README. ([\#4795](https://github.com/matrix-org/synapse/issues/4795)) -- Clean up read-receipt handling. ([\#4797](https://github.com/matrix-org/synapse/issues/4797)) -- Add some debug about processing read receipts. ([\#4798](https://github.com/matrix-org/synapse/issues/4798)) -- Clean up some replication code. ([\#4799](https://github.com/matrix-org/synapse/issues/4799)) -- Add some docstrings. ([\#4815](https://github.com/matrix-org/synapse/issues/4815)) -- Add debug logger to try and track down #4422. ([\#4816](https://github.com/matrix-org/synapse/issues/4816)) -- Make shutdown API send explanation message to room after users have been forced joined. ([\#4817](https://github.com/matrix-org/synapse/issues/4817)) -- Update example_log_config.yaml. ([\#4820](https://github.com/matrix-org/synapse/issues/4820)) -- Document the `generate` option for the docker image. ([\#4824](https://github.com/matrix-org/synapse/issues/4824)) -- Fix check-newsfragment for debian-only changes. ([\#4825](https://github.com/matrix-org/synapse/issues/4825)) -- Add some debug logging for device list updates to help with #4828. ([\#4828](https://github.com/matrix-org/synapse/issues/4828)) -- Improve federation documentation, specifically .well-known support. Many thanks to @vaab. ([\#4832](https://github.com/matrix-org/synapse/issues/4832)) -- Disable captcha registration by default in unit tests. ([\#4839](https://github.com/matrix-org/synapse/issues/4839)) -- Add stuff back to the .gitignore. ([\#4843](https://github.com/matrix-org/synapse/issues/4843)) -- Clarify what registration_shared_secret allows for. ([\#4844](https://github.com/matrix-org/synapse/issues/4844)) -- Correctly log expected errors when fetching server keys. ([\#4847](https://github.com/matrix-org/synapse/issues/4847)) -- Update install docs to explicitly state a full-chain (not just the top-level) TLS certificate must be provided to Synapse. This caused some people's Synapse ports to appear correct in a browser but still (rightfully so) upset the federation tester. ([\#4849](https://github.com/matrix-org/synapse/issues/4849)) -- Move client read-receipt processing to federation sender worker. ([\#4852](https://github.com/matrix-org/synapse/issues/4852)) -- Refactor federation TransactionQueue. ([\#4855](https://github.com/matrix-org/synapse/issues/4855)) -- Comment out most options in the generated config. ([\#4863](https://github.com/matrix-org/synapse/issues/4863)) -- Fix yaml library warnings by using safe_load. ([\#4869](https://github.com/matrix-org/synapse/issues/4869)) -- Update Apache setup to remove location syntax. Thanks to @cwmke! ([\#4870](https://github.com/matrix-org/synapse/issues/4870)) -- Reinstate test case that runs unit tests against oldest supported dependencies. ([\#4879](https://github.com/matrix-org/synapse/issues/4879)) -- Update link to federation docs. ([\#4881](https://github.com/matrix-org/synapse/issues/4881)) -- fix test_auto_create_auto_join_where_no_consent. ([\#4886](https://github.com/matrix-org/synapse/issues/4886)) -- Use a regular HomeServerConfig object for unit tests rater than a Mock. ([\#4889](https://github.com/matrix-org/synapse/issues/4889)) -- Add some notes about tuning postgres for larger deployments. ([\#4895](https://github.com/matrix-org/synapse/issues/4895)) -- Add a config option for torture-testing worker replication. ([\#4902](https://github.com/matrix-org/synapse/issues/4902)) -- Log requests which are simulated by the unit tests. ([\#4905](https://github.com/matrix-org/synapse/issues/4905)) -- Allow newsfragments to end with exclamation marks. Exciting! ([\#4912](https://github.com/matrix-org/synapse/issues/4912)) -- Refactor some more tests to use HomeserverTestCase. ([\#4913](https://github.com/matrix-org/synapse/issues/4913)) -- Refactor out the state deltas portion of the user directory store and handler. ([\#4917](https://github.com/matrix-org/synapse/issues/4917)) -- Fix nginx example in ACME doc. ([\#4923](https://github.com/matrix-org/synapse/issues/4923)) -- Use an explicit dbname for postgres connections in the tests. ([\#4928](https://github.com/matrix-org/synapse/issues/4928)) -- Fix `ClientReplicationStreamProtocol.__str__()`. ([\#4929](https://github.com/matrix-org/synapse/issues/4929)) - - -Synapse 0.99.2 (2019-03-01) -=========================== - -Features --------- - -- Added an HAProxy example in the reverse proxy documentation. Contributed by Benoît S. (“Benpro”). ([\#4541](https://github.com/matrix-org/synapse/issues/4541)) -- Add basic optional sentry integration. ([\#4632](https://github.com/matrix-org/synapse/issues/4632), [\#4694](https://github.com/matrix-org/synapse/issues/4694)) -- Transfer bans on room upgrade. ([\#4642](https://github.com/matrix-org/synapse/issues/4642)) -- Add configurable room list publishing rules. ([\#4647](https://github.com/matrix-org/synapse/issues/4647)) -- Support .well-known delegation when issuing certificates through ACME. ([\#4652](https://github.com/matrix-org/synapse/issues/4652)) -- Allow registration and login to be handled by a worker instance. ([\#4666](https://github.com/matrix-org/synapse/issues/4666), [\#4670](https://github.com/matrix-org/synapse/issues/4670), [\#4682](https://github.com/matrix-org/synapse/issues/4682)) -- Reduce the overhead of creating outbound federation connections over TLS by caching the TLS client options. ([\#4674](https://github.com/matrix-org/synapse/issues/4674)) -- Add prometheus metrics for number of outgoing EDUs, by type. ([\#4695](https://github.com/matrix-org/synapse/issues/4695)) -- Return correct error code when inviting a remote user to a room whose homeserver does not support the room version. ([\#4721](https://github.com/matrix-org/synapse/issues/4721)) -- Prevent showing rooms to other servers that were set to not federate. ([\#4746](https://github.com/matrix-org/synapse/issues/4746)) - - -Bugfixes --------- - -- Fix possible exception when paginating. ([\#4263](https://github.com/matrix-org/synapse/issues/4263)) -- The dependency checker now correctly reports a version mismatch for optional - dependencies, instead of reporting the dependency missing. ([\#4450](https://github.com/matrix-org/synapse/issues/4450)) -- Set CORS headers on .well-known requests. ([\#4651](https://github.com/matrix-org/synapse/issues/4651)) -- Fix kicking guest users on guest access revocation in worker mode. ([\#4667](https://github.com/matrix-org/synapse/issues/4667)) -- Fix an issue in the database migration script where the - `e2e_room_keys.is_verified` column wasn't considered as - a boolean. ([\#4680](https://github.com/matrix-org/synapse/issues/4680)) -- Fix TaskStopped exceptions in logs when outbound requests time out. ([\#4690](https://github.com/matrix-org/synapse/issues/4690)) -- Fix ACME config for python 2. ([\#4717](https://github.com/matrix-org/synapse/issues/4717)) -- Fix paginating over federation persisting incorrect state. ([\#4718](https://github.com/matrix-org/synapse/issues/4718)) - - -Internal Changes ----------------- - -- Run `black` to reformat user directory code. ([\#4635](https://github.com/matrix-org/synapse/issues/4635)) -- Reduce number of exceptions we log. ([\#4643](https://github.com/matrix-org/synapse/issues/4643), [\#4668](https://github.com/matrix-org/synapse/issues/4668)) -- Introduce upsert batching functionality in the database layer. ([\#4644](https://github.com/matrix-org/synapse/issues/4644)) -- Fix various spelling mistakes. ([\#4657](https://github.com/matrix-org/synapse/issues/4657)) -- Cleanup request exception logging. ([\#4669](https://github.com/matrix-org/synapse/issues/4669), [\#4737](https://github.com/matrix-org/synapse/issues/4737), [\#4738](https://github.com/matrix-org/synapse/issues/4738)) -- Improve replication performance by reducing cache invalidation traffic. ([\#4671](https://github.com/matrix-org/synapse/issues/4671), [\#4715](https://github.com/matrix-org/synapse/issues/4715), [\#4748](https://github.com/matrix-org/synapse/issues/4748)) -- Test against Postgres 9.5 as well as 9.4. ([\#4676](https://github.com/matrix-org/synapse/issues/4676)) -- Run unit tests against python 3.7. ([\#4677](https://github.com/matrix-org/synapse/issues/4677)) -- Attempt to clarify installation instructions/config. ([\#4681](https://github.com/matrix-org/synapse/issues/4681)) -- Clean up gitignores. ([\#4688](https://github.com/matrix-org/synapse/issues/4688)) -- Minor tweaks to acme docs. ([\#4689](https://github.com/matrix-org/synapse/issues/4689)) -- Improve the logging in the pusher process. ([\#4691](https://github.com/matrix-org/synapse/issues/4691)) -- Better checks on newsfragments. ([\#4698](https://github.com/matrix-org/synapse/issues/4698), [\#4750](https://github.com/matrix-org/synapse/issues/4750)) -- Avoid some redundant work when processing read receipts. ([\#4706](https://github.com/matrix-org/synapse/issues/4706)) -- Run `push_receipts_to_remotes` as background job. ([\#4707](https://github.com/matrix-org/synapse/issues/4707)) -- Add prometheus metrics for number of badge update pushes. ([\#4709](https://github.com/matrix-org/synapse/issues/4709)) -- Reduce pusher logging on startup ([\#4716](https://github.com/matrix-org/synapse/issues/4716)) -- Don't log exceptions when failing to fetch remote server keys. ([\#4722](https://github.com/matrix-org/synapse/issues/4722)) -- Correctly proxy exception in frontend_proxy worker. ([\#4723](https://github.com/matrix-org/synapse/issues/4723)) -- Add database version to phonehome stats. ([\#4753](https://github.com/matrix-org/synapse/issues/4753)) - - -Synapse 0.99.1.1 (2019-02-14) -============================= - -Bugfixes --------- - -- Fix "TypeError: '>' not supported" when starting without an existing certificate. - Fix a bug where an existing certificate would be reprovisoned every day. ([\#4648](https://github.com/matrix-org/synapse/issues/4648)) - - -Synapse 0.99.1 (2019-02-14) -=========================== - -Features --------- - -- Include m.room.encryption on invites by default ([\#3902](https://github.com/matrix-org/synapse/issues/3902)) -- Federation OpenID listener resource can now be activated even if federation is disabled ([\#4420](https://github.com/matrix-org/synapse/issues/4420)) -- Synapse's ACME support will now correctly reprovision a certificate that approaches its expiry while Synapse is running. ([\#4522](https://github.com/matrix-org/synapse/issues/4522)) -- Add ability to update backup versions ([\#4580](https://github.com/matrix-org/synapse/issues/4580)) -- Allow the "unavailable" presence status for /sync. - This change makes Synapse compliant with r0.4.0 of the Client-Server specification. ([\#4592](https://github.com/matrix-org/synapse/issues/4592)) -- There is no longer any need to specify `no_tls`: it is inferred from the absence of TLS listeners ([\#4613](https://github.com/matrix-org/synapse/issues/4613), [\#4615](https://github.com/matrix-org/synapse/issues/4615), [\#4617](https://github.com/matrix-org/synapse/issues/4617), [\#4636](https://github.com/matrix-org/synapse/issues/4636)) -- The default configuration no longer requires TLS certificates. ([\#4614](https://github.com/matrix-org/synapse/issues/4614)) - - -Bugfixes --------- - -- Copy over room federation ability on room upgrade. ([\#4530](https://github.com/matrix-org/synapse/issues/4530)) -- Fix noisy "twisted.internet.task.TaskStopped" errors in logs ([\#4546](https://github.com/matrix-org/synapse/issues/4546)) -- Synapse is now tolerant of the `tls_fingerprints` option being None or not specified. ([\#4589](https://github.com/matrix-org/synapse/issues/4589)) -- Fix 'no unique or exclusion constraint' error ([\#4591](https://github.com/matrix-org/synapse/issues/4591)) -- Transfer Server ACLs on room upgrade. ([\#4608](https://github.com/matrix-org/synapse/issues/4608)) -- Fix failure to start when not TLS certificate was given even if TLS was disabled. ([\#4618](https://github.com/matrix-org/synapse/issues/4618)) -- Fix self-signed cert notice from generate-config. ([\#4625](https://github.com/matrix-org/synapse/issues/4625)) -- Fix performance of `user_ips` table deduplication background update ([\#4626](https://github.com/matrix-org/synapse/issues/4626), [\#4627](https://github.com/matrix-org/synapse/issues/4627)) - - -Internal Changes ----------------- - -- Change the user directory state query to use a filtered call to the db instead of a generic one. ([\#4462](https://github.com/matrix-org/synapse/issues/4462)) -- Reject federation transactions if they include more than 50 PDUs or 100 EDUs. ([\#4513](https://github.com/matrix-org/synapse/issues/4513)) -- Reduce duplication of ``synapse.app`` code. ([\#4567](https://github.com/matrix-org/synapse/issues/4567)) -- Fix docker upload job to push -py2 images. ([\#4576](https://github.com/matrix-org/synapse/issues/4576)) -- Add port configuration information to ACME instructions. ([\#4578](https://github.com/matrix-org/synapse/issues/4578)) -- Update MSC1711 FAQ to calrify .well-known usage ([\#4584](https://github.com/matrix-org/synapse/issues/4584)) -- Clean up default listener configuration ([\#4586](https://github.com/matrix-org/synapse/issues/4586)) -- Clarifications for reverse proxy docs ([\#4607](https://github.com/matrix-org/synapse/issues/4607)) -- Move ClientTLSOptionsFactory init out of `refresh_certificates` ([\#4611](https://github.com/matrix-org/synapse/issues/4611)) -- Fail cleanly if listener config lacks a 'port' ([\#4616](https://github.com/matrix-org/synapse/issues/4616)) -- Remove redundant entries from docker config ([\#4619](https://github.com/matrix-org/synapse/issues/4619)) -- README updates ([\#4621](https://github.com/matrix-org/synapse/issues/4621)) - - -Synapse 0.99.0 (2019-02-05) -=========================== - -Synapse v0.99.x is a precursor to the upcoming Synapse v1.0 release. It contains foundational changes to room architecture and the federation security model necessary to support the upcoming r0 release of the Server to Server API. - -Features --------- - -- Synapse's cipher string has been updated to require ECDH key exchange. Configuring and generating dh_params is no longer required, and they will be ignored. ([\#4229](https://github.com/matrix-org/synapse/issues/4229)) -- Synapse can now automatically provision TLS certificates via ACME (the protocol used by CAs like Let's Encrypt). ([\#4384](https://github.com/matrix-org/synapse/issues/4384), [\#4492](https://github.com/matrix-org/synapse/issues/4492), [\#4525](https://github.com/matrix-org/synapse/issues/4525), [\#4572](https://github.com/matrix-org/synapse/issues/4572), [\#4564](https://github.com/matrix-org/synapse/issues/4564), [\#4566](https://github.com/matrix-org/synapse/issues/4566), [\#4547](https://github.com/matrix-org/synapse/issues/4547), [\#4557](https://github.com/matrix-org/synapse/issues/4557)) -- Implement MSC1708 (.well-known routing for server-server federation) ([\#4408](https://github.com/matrix-org/synapse/issues/4408), [\#4409](https://github.com/matrix-org/synapse/issues/4409), [\#4426](https://github.com/matrix-org/synapse/issues/4426), [\#4427](https://github.com/matrix-org/synapse/issues/4427), [\#4428](https://github.com/matrix-org/synapse/issues/4428), [\#4464](https://github.com/matrix-org/synapse/issues/4464), [\#4468](https://github.com/matrix-org/synapse/issues/4468), [\#4487](https://github.com/matrix-org/synapse/issues/4487), [\#4488](https://github.com/matrix-org/synapse/issues/4488), [\#4489](https://github.com/matrix-org/synapse/issues/4489), [\#4497](https://github.com/matrix-org/synapse/issues/4497), [\#4511](https://github.com/matrix-org/synapse/issues/4511), [\#4516](https://github.com/matrix-org/synapse/issues/4516), [\#4520](https://github.com/matrix-org/synapse/issues/4520), [\#4521](https://github.com/matrix-org/synapse/issues/4521), [\#4539](https://github.com/matrix-org/synapse/issues/4539), [\#4542](https://github.com/matrix-org/synapse/issues/4542), [\#4544](https://github.com/matrix-org/synapse/issues/4544)) -- Search now includes results from predecessor rooms after a room upgrade. ([\#4415](https://github.com/matrix-org/synapse/issues/4415)) -- Config option to disable requesting MSISDN on registration. ([\#4423](https://github.com/matrix-org/synapse/issues/4423)) -- Add a metric for tracking event stream position of the user directory. ([\#4445](https://github.com/matrix-org/synapse/issues/4445)) -- Support exposing server capabilities in CS API (MSC1753, MSC1804) ([\#4472](https://github.com/matrix-org/synapse/issues/4472), [81b7e7eed](https://github.com/matrix-org/synapse/commit/81b7e7eed323f55d6550e7a270a9dc2c4c7b0fe0))) -- Add support for room version 3 ([\#4483](https://github.com/matrix-org/synapse/issues/4483), [\#4499](https://github.com/matrix-org/synapse/issues/4499), [\#4515](https://github.com/matrix-org/synapse/issues/4515), [\#4523](https://github.com/matrix-org/synapse/issues/4523), [\#4535](https://github.com/matrix-org/synapse/issues/4535)) -- Synapse will now reload TLS certificates from disk upon SIGHUP. ([\#4495](https://github.com/matrix-org/synapse/issues/4495), [\#4524](https://github.com/matrix-org/synapse/issues/4524)) -- The matrixdotorg/synapse Docker images now use Python 3 by default. ([\#4558](https://github.com/matrix-org/synapse/issues/4558)) - -Bugfixes --------- - -- Prevent users with access tokens predating the introduction of device IDs from creating spurious entries in the user_ips table. ([\#4369](https://github.com/matrix-org/synapse/issues/4369)) -- Fix typo in ALL_USER_TYPES definition to ensure type is a tuple ([\#4392](https://github.com/matrix-org/synapse/issues/4392)) -- Fix high CPU usage due to remote devicelist updates ([\#4397](https://github.com/matrix-org/synapse/issues/4397)) -- Fix potential bug where creating or joining a room could fail ([\#4404](https://github.com/matrix-org/synapse/issues/4404)) -- Fix bug when rejecting remote invites ([\#4405](https://github.com/matrix-org/synapse/issues/4405), [\#4527](https://github.com/matrix-org/synapse/issues/4527)) -- Fix incorrect logcontexts after a Deferred was cancelled ([\#4407](https://github.com/matrix-org/synapse/issues/4407)) -- Ensure encrypted room state is persisted across room upgrades. ([\#4411](https://github.com/matrix-org/synapse/issues/4411)) -- Copy over whether a room is a direct message and any associated room tags on room upgrade. ([\#4412](https://github.com/matrix-org/synapse/issues/4412)) -- Fix None guard in calling config.server.is_threepid_reserved ([\#4435](https://github.com/matrix-org/synapse/issues/4435)) -- Don't send IP addresses as SNI ([\#4452](https://github.com/matrix-org/synapse/issues/4452)) -- Fix UnboundLocalError in post_urlencoded_get_json ([\#4460](https://github.com/matrix-org/synapse/issues/4460)) -- Add a timeout to filtered room directory queries. ([\#4461](https://github.com/matrix-org/synapse/issues/4461)) -- Workaround for login error when using both LDAP and internal authentication. ([\#4486](https://github.com/matrix-org/synapse/issues/4486)) -- Fix a bug where setting a relative consent directory path would cause a crash. ([\#4512](https://github.com/matrix-org/synapse/issues/4512)) - - -Deprecations and Removals -------------------------- - -- Synapse no longer generates self-signed TLS certificates when generating a configuration file. ([\#4509](https://github.com/matrix-org/synapse/issues/4509)) - - -Improved Documentation ----------------------- - -- Update debian installation instructions ([\#4526](https://github.com/matrix-org/synapse/issues/4526)) - - -Internal Changes ----------------- - -- Synapse will now take advantage of native UPSERT functionality in PostgreSQL 9.5+ and SQLite 3.24+. ([\#4306](https://github.com/matrix-org/synapse/issues/4306), [\#4459](https://github.com/matrix-org/synapse/issues/4459), [\#4466](https://github.com/matrix-org/synapse/issues/4466), [\#4471](https://github.com/matrix-org/synapse/issues/4471), [\#4477](https://github.com/matrix-org/synapse/issues/4477), [\#4505](https://github.com/matrix-org/synapse/issues/4505)) -- Update README to use the new virtualenv everywhere ([\#4342](https://github.com/matrix-org/synapse/issues/4342)) -- Add better logging for unexpected errors while sending transactions ([\#4368](https://github.com/matrix-org/synapse/issues/4368)) -- Apply a unique index to the user_ips table, preventing duplicates. ([\#4370](https://github.com/matrix-org/synapse/issues/4370), [\#4432](https://github.com/matrix-org/synapse/issues/4432), [\#4434](https://github.com/matrix-org/synapse/issues/4434)) -- Silence travis-ci build warnings by removing non-functional python3.6 ([\#4377](https://github.com/matrix-org/synapse/issues/4377)) -- Fix a comment in the generated config file ([\#4387](https://github.com/matrix-org/synapse/issues/4387)) -- Add ground work for implementing future federation API versions ([\#4390](https://github.com/matrix-org/synapse/issues/4390)) -- Update dependencies on msgpack and pymacaroons to use the up-to-date packages. ([\#4399](https://github.com/matrix-org/synapse/issues/4399)) -- Tweak codecov settings to make them less loud. ([\#4400](https://github.com/matrix-org/synapse/issues/4400)) -- Implement server support for MSC1794 - Federation v2 Invite API ([\#4402](https://github.com/matrix-org/synapse/issues/4402)) -- debian package: symlink to explicit python version ([\#4433](https://github.com/matrix-org/synapse/issues/4433)) -- Add infrastructure to support different event formats ([\#4437](https://github.com/matrix-org/synapse/issues/4437), [\#4447](https://github.com/matrix-org/synapse/issues/4447), [\#4448](https://github.com/matrix-org/synapse/issues/4448), [\#4470](https://github.com/matrix-org/synapse/issues/4470), [\#4481](https://github.com/matrix-org/synapse/issues/4481), [\#4482](https://github.com/matrix-org/synapse/issues/4482), [\#4493](https://github.com/matrix-org/synapse/issues/4493), [\#4494](https://github.com/matrix-org/synapse/issues/4494), [\#4496](https://github.com/matrix-org/synapse/issues/4496), [\#4510](https://github.com/matrix-org/synapse/issues/4510), [\#4514](https://github.com/matrix-org/synapse/issues/4514)) -- Generate the debian config during build ([\#4444](https://github.com/matrix-org/synapse/issues/4444)) -- Clarify documentation for the `public_baseurl` config param ([\#4458](https://github.com/matrix-org/synapse/issues/4458), [\#4498](https://github.com/matrix-org/synapse/issues/4498)) -- Fix quoting for allowed_local_3pids example config ([\#4476](https://github.com/matrix-org/synapse/issues/4476)) -- Remove deprecated --process-dependency-links option from UPGRADE.rst ([\#4485](https://github.com/matrix-org/synapse/issues/4485)) -- Make it possible to set the log level for tests via an environment variable ([\#4506](https://github.com/matrix-org/synapse/issues/4506)) -- Reduce the log level of linearizer lock acquirement to DEBUG. ([\#4507](https://github.com/matrix-org/synapse/issues/4507)) -- Fix code to comply with linting in PyFlakes 3.7.1. ([\#4519](https://github.com/matrix-org/synapse/issues/4519)) -- Add some debug for membership syncing issues ([\#4538](https://github.com/matrix-org/synapse/issues/4538)) -- Docker: only copy what we need to the build image ([\#4562](https://github.com/matrix-org/synapse/issues/4562)) - - -Synapse 0.34.1.1 (2019-01-11) -============================= - -This release fixes CVE-2019-5885 and is recommended for all users of Synapse 0.34.1. - -This release is compatible with Python 2.7 and 3.5+. Python 3.7 is fully supported. - -Bugfixes --------- - -- Fix spontaneous logout on upgrade - ([\#4374](https://github.com/matrix-org/synapse/issues/4374)) - - -Synapse 0.34.1 (2019-01-09) -=========================== - -Internal Changes ----------------- - -- Add better logging for unexpected errors while sending transactions ([\#4361](https://github.com/matrix-org/synapse/issues/4361), [\#4362](https://github.com/matrix-org/synapse/issues/4362)) - - -Synapse 0.34.1rc1 (2019-01-08) -============================== - -Features --------- - -- Special-case a support user for use in verifying behaviour of a given server. The support user does not appear in user directory or monthly active user counts. ([\#4141](https://github.com/matrix-org/synapse/issues/4141), [\#4344](https://github.com/matrix-org/synapse/issues/4344)) -- Support for serving .well-known files ([\#4262](https://github.com/matrix-org/synapse/issues/4262)) -- Rework SAML2 authentication ([\#4265](https://github.com/matrix-org/synapse/issues/4265), [\#4267](https://github.com/matrix-org/synapse/issues/4267)) -- SAML2 authentication: Initialise user display name from SAML2 data ([\#4272](https://github.com/matrix-org/synapse/issues/4272)) -- Synapse can now have its conditional/extra dependencies installed by pip. This functionality can be used by using `pip install matrix-synapse[feature]`, where feature is a comma separated list with the possible values `email.enable_notifs`, `matrix-synapse-ldap3`, `postgres`, `resources.consent`, `saml2`, `url_preview`, and `test`. If you want to install all optional dependencies, you can use "all" instead. ([\#4298](https://github.com/matrix-org/synapse/issues/4298), [\#4325](https://github.com/matrix-org/synapse/issues/4325), [\#4327](https://github.com/matrix-org/synapse/issues/4327)) -- Add routes for reading account data. ([\#4303](https://github.com/matrix-org/synapse/issues/4303)) -- Add opt-in support for v2 rooms ([\#4307](https://github.com/matrix-org/synapse/issues/4307)) -- Add a script to generate a clean config file ([\#4315](https://github.com/matrix-org/synapse/issues/4315)) -- Return server data in /login response ([\#4319](https://github.com/matrix-org/synapse/issues/4319)) - - -Bugfixes --------- - -- Fix contains_url check to be consistent with other instances in code-base and check that value is an instance of string. ([\#3405](https://github.com/matrix-org/synapse/issues/3405)) -- Fix CAS login when username is not valid in an MXID ([\#4264](https://github.com/matrix-org/synapse/issues/4264)) -- Send CORS headers for /media/config ([\#4279](https://github.com/matrix-org/synapse/issues/4279)) -- Add 'sandbox' to CSP for media reprository ([\#4284](https://github.com/matrix-org/synapse/issues/4284)) -- Make the new landing page prettier. ([\#4294](https://github.com/matrix-org/synapse/issues/4294)) -- Fix deleting E2E room keys when using old SQLite versions. ([\#4295](https://github.com/matrix-org/synapse/issues/4295)) -- The metric synapse_admin_mau:current previously did not update when config.mau_stats_only was set to True ([\#4305](https://github.com/matrix-org/synapse/issues/4305)) -- Fixed per-room account data filters ([\#4309](https://github.com/matrix-org/synapse/issues/4309)) -- Fix indentation in default config ([\#4313](https://github.com/matrix-org/synapse/issues/4313)) -- Fix synapse:latest docker upload ([\#4316](https://github.com/matrix-org/synapse/issues/4316)) -- Fix test_metric.py compatibility with prometheus_client 0.5. Contributed by Maarten de Vries . ([\#4317](https://github.com/matrix-org/synapse/issues/4317)) -- Avoid packaging _trial_temp directory in -py3 debian packages ([\#4326](https://github.com/matrix-org/synapse/issues/4326)) -- Check jinja version for consent resource ([\#4327](https://github.com/matrix-org/synapse/issues/4327)) -- fix NPE in /messages by checking if all events were filtered out ([\#4330](https://github.com/matrix-org/synapse/issues/4330)) -- Fix `python -m synapse.config` on Python 3. ([\#4356](https://github.com/matrix-org/synapse/issues/4356)) - - -Deprecations and Removals -------------------------- - -- Remove the deprecated v1/register API on Python 2. It was never ported to Python 3. ([\#4334](https://github.com/matrix-org/synapse/issues/4334)) - - -Internal Changes ----------------- - -- Getting URL previews of IP addresses no longer fails on Python 3. ([\#4215](https://github.com/matrix-org/synapse/issues/4215)) -- drop undocumented dependency on dateutil ([\#4266](https://github.com/matrix-org/synapse/issues/4266)) -- Update the example systemd config to use a virtualenv ([\#4273](https://github.com/matrix-org/synapse/issues/4273)) -- Update link to kernel DCO guide ([\#4274](https://github.com/matrix-org/synapse/issues/4274)) -- Make isort tox check print diff when it fails ([\#4283](https://github.com/matrix-org/synapse/issues/4283)) -- Log room_id in Unknown room errors ([\#4297](https://github.com/matrix-org/synapse/issues/4297)) -- Documentation improvements for coturn setup. Contributed by Krithin Sitaram. ([\#4333](https://github.com/matrix-org/synapse/issues/4333)) -- Update pull request template to use absolute links ([\#4341](https://github.com/matrix-org/synapse/issues/4341)) -- Update README to not lie about required restart when updating TLS certificates ([\#4343](https://github.com/matrix-org/synapse/issues/4343)) -- Update debian packaging for compatibility with transitional package ([\#4349](https://github.com/matrix-org/synapse/issues/4349)) -- Fix command hint to generate a config file when trying to start without a config file ([\#4353](https://github.com/matrix-org/synapse/issues/4353)) -- Add better logging for unexpected errors while sending transactions ([\#4358](https://github.com/matrix-org/synapse/issues/4358)) - - -Synapse 0.34.0 (2018-12-20) -=========================== - -Synapse 0.34.0 is the first release to fully support Python 3. Synapse will now -run on Python versions 3.5 or 3.6 (as well as 2.7). Support for Python 3.7 -remains experimental. - -We recommend upgrading to Python 3, but make sure to read the [upgrade -notes](docs/upgrade.md#upgrading-to-v0340) when doing so. - -Features --------- - -- Add 'sandbox' to CSP for media reprository ([\#4284](https://github.com/matrix-org/synapse/issues/4284)) -- Make the new landing page prettier. ([\#4294](https://github.com/matrix-org/synapse/issues/4294)) -- Fix deleting E2E room keys when using old SQLite versions. ([\#4295](https://github.com/matrix-org/synapse/issues/4295)) -- Add a welcome page for the client API port. Credit to @krombel! ([\#4289](https://github.com/matrix-org/synapse/issues/4289)) -- Remove Matrix console from the default distribution ([\#4290](https://github.com/matrix-org/synapse/issues/4290)) -- Add option to track MAU stats (but not limit people) ([\#3830](https://github.com/matrix-org/synapse/issues/3830)) -- Add an option to enable recording IPs for appservice users ([\#3831](https://github.com/matrix-org/synapse/issues/3831)) -- Rename login type `m.login.cas` to `m.login.sso` ([\#4220](https://github.com/matrix-org/synapse/issues/4220)) -- Add an option to disable search for homeservers that may not be interested in it. ([\#4230](https://github.com/matrix-org/synapse/issues/4230)) - - -Bugfixes --------- - -- Pushrules can now again be made with non-ASCII rule IDs. ([\#4165](https://github.com/matrix-org/synapse/issues/4165)) -- The media repository now no longer fails to decode UTF-8 filenames when downloading remote media. ([\#4176](https://github.com/matrix-org/synapse/issues/4176)) -- URL previews now correctly decode non-UTF-8 text if the header contains a ` synapse ([\#3897](https://github.com/matrix-org/synapse/issues/3897)) -- Increase the timeout when filling missing events in federation requests ([\#3903](https://github.com/matrix-org/synapse/issues/3903)) -- Improve the logging when handling a federation transaction ([\#3904](https://github.com/matrix-org/synapse/issues/3904), [\#3966](https://github.com/matrix-org/synapse/issues/3966)) -- Improve logging of outbound federation requests ([\#3906](https://github.com/matrix-org/synapse/issues/3906), [\#3909](https://github.com/matrix-org/synapse/issues/3909)) -- Fix the docker image building on python 3 ([\#3911](https://github.com/matrix-org/synapse/issues/3911)) -- Add a regression test for logging failed HTTP requests on Python 3. ([\#3912](https://github.com/matrix-org/synapse/issues/3912)) -- Comments and interface cleanup for on_receive_pdu ([\#3924](https://github.com/matrix-org/synapse/issues/3924)) -- Fix spurious exceptions when remote http client closes conncetion ([\#3925](https://github.com/matrix-org/synapse/issues/3925)) -- Log exceptions thrown by background tasks ([\#3927](https://github.com/matrix-org/synapse/issues/3927)) -- Add a cache to get_destination_retry_timings ([\#3933](https://github.com/matrix-org/synapse/issues/3933), [\#3991](https://github.com/matrix-org/synapse/issues/3991)) -- Automate pushes to docker hub ([\#3946](https://github.com/matrix-org/synapse/issues/3946)) -- Require attrs 16.0.0 or later ([\#3947](https://github.com/matrix-org/synapse/issues/3947)) -- Fix incompatibility with python3 on alpine ([\#3948](https://github.com/matrix-org/synapse/issues/3948)) -- Run the test suite on the oldest supported versions of our dependencies in CI. ([\#3952](https://github.com/matrix-org/synapse/issues/3952)) -- CircleCI now only runs merged jobs on PRs, and commit jobs on develop, master, and release branches. ([\#3957](https://github.com/matrix-org/synapse/issues/3957)) -- Fix docstrings and add tests for state store methods ([\#3958](https://github.com/matrix-org/synapse/issues/3958)) -- fix docstring for FederationClient.get_state_for_room ([\#3963](https://github.com/matrix-org/synapse/issues/3963)) -- Run notify_app_services as a bg process ([\#3965](https://github.com/matrix-org/synapse/issues/3965)) -- Clarifications in FederationHandler ([\#3967](https://github.com/matrix-org/synapse/issues/3967)) -- Further reduce the docker image size ([\#3972](https://github.com/matrix-org/synapse/issues/3972)) -- Build py3 docker images for docker hub too ([\#3976](https://github.com/matrix-org/synapse/issues/3976)) -- Updated the installation instructions to point to the matrix-synapse package on PyPI. ([\#3985](https://github.com/matrix-org/synapse/issues/3985)) -- Disable USE_FROZEN_DICTS for unittests by default. ([\#3987](https://github.com/matrix-org/synapse/issues/3987)) -- Remove unused Jenkins and development related files from the repo. ([\#3988](https://github.com/matrix-org/synapse/issues/3988)) -- Improve stacktraces in certain exceptions in the logs ([\#3989](https://github.com/matrix-org/synapse/issues/3989)) - - -Synapse 0.33.5.1 (2018-09-25) -============================= - -Internal Changes ----------------- - -- Fix incompatibility with older Twisted version in tests. Thanks @OlegGirko! ([\#3940](https://github.com/matrix-org/synapse/issues/3940)) - - -Synapse 0.33.5 (2018-09-24) -=========================== - -No significant changes. - - -Synapse 0.33.5rc1 (2018-09-17) -============================== - -Features --------- - -- Python 3.5 and 3.6 support is now in beta. ([\#3576](https://github.com/matrix-org/synapse/issues/3576)) -- Implement `event_format` filter param in `/sync` ([\#3790](https://github.com/matrix-org/synapse/issues/3790)) -- Add synapse_admin_mau:registered_reserved_users metric to expose number of real reaserved users ([\#3846](https://github.com/matrix-org/synapse/issues/3846)) - - -Bugfixes --------- - -- Remove connection ID for replication prometheus metrics, as it creates a large number of new series. ([\#3788](https://github.com/matrix-org/synapse/issues/3788)) -- guest users should not be part of mau total ([\#3800](https://github.com/matrix-org/synapse/issues/3800)) -- Bump dependency on pyopenssl 16.x, to avoid incompatibility with recent Twisted. ([\#3804](https://github.com/matrix-org/synapse/issues/3804)) -- Fix existing room tags not coming down sync when joining a room ([\#3810](https://github.com/matrix-org/synapse/issues/3810)) -- Fix jwt import check ([\#3824](https://github.com/matrix-org/synapse/issues/3824)) -- fix VOIP crashes under Python 3 (#3821) ([\#3835](https://github.com/matrix-org/synapse/issues/3835)) -- Fix manhole so that it works with latest openssh clients ([\#3841](https://github.com/matrix-org/synapse/issues/3841)) -- Fix outbound requests occasionally wedging, which can result in federation breaking between servers. ([\#3845](https://github.com/matrix-org/synapse/issues/3845)) -- Show heroes if room name/canonical alias has been deleted ([\#3851](https://github.com/matrix-org/synapse/issues/3851)) -- Fix handling of redacted events from federation ([\#3859](https://github.com/matrix-org/synapse/issues/3859)) -- ([\#3874](https://github.com/matrix-org/synapse/issues/3874)) -- Mitigate outbound federation randomly becoming wedged ([\#3875](https://github.com/matrix-org/synapse/issues/3875)) - - -Internal Changes ----------------- - -- CircleCI tests now run on the potential merge of a PR. ([\#3704](https://github.com/matrix-org/synapse/issues/3704)) -- http/ is now ported to Python 3. ([\#3771](https://github.com/matrix-org/synapse/issues/3771)) -- Improve human readable error messages for threepid registration/account update ([\#3789](https://github.com/matrix-org/synapse/issues/3789)) -- Make /sync slightly faster by avoiding needless copies ([\#3795](https://github.com/matrix-org/synapse/issues/3795)) -- handlers/ is now ported to Python 3. ([\#3803](https://github.com/matrix-org/synapse/issues/3803)) -- Limit the number of PDUs/EDUs per federation transaction ([\#3805](https://github.com/matrix-org/synapse/issues/3805)) -- Only start postgres instance for postgres tests on Travis CI ([\#3806](https://github.com/matrix-org/synapse/issues/3806)) -- tests/ is now ported to Python 3. ([\#3808](https://github.com/matrix-org/synapse/issues/3808)) -- crypto/ is now ported to Python 3. ([\#3822](https://github.com/matrix-org/synapse/issues/3822)) -- rest/ is now ported to Python 3. ([\#3823](https://github.com/matrix-org/synapse/issues/3823)) -- add some logging for the keyring queue ([\#3826](https://github.com/matrix-org/synapse/issues/3826)) -- speed up lazy loading by 2-3x ([\#3827](https://github.com/matrix-org/synapse/issues/3827)) -- Improved Dockerfile to remove build requirements after building reducing the image size. ([\#3834](https://github.com/matrix-org/synapse/issues/3834)) -- Disable lazy loading for incremental syncs for now ([\#3840](https://github.com/matrix-org/synapse/issues/3840)) -- federation/ is now ported to Python 3. ([\#3847](https://github.com/matrix-org/synapse/issues/3847)) -- Log when we retry outbound requests ([\#3853](https://github.com/matrix-org/synapse/issues/3853)) -- Removed some excess logging messages. ([\#3855](https://github.com/matrix-org/synapse/issues/3855)) -- Speed up purge history for rooms that have been previously purged ([\#3856](https://github.com/matrix-org/synapse/issues/3856)) -- Refactor some HTTP timeout code. ([\#3857](https://github.com/matrix-org/synapse/issues/3857)) -- Fix running merged builds on CircleCI ([\#3858](https://github.com/matrix-org/synapse/issues/3858)) -- Fix typo in replication stream exception. ([\#3860](https://github.com/matrix-org/synapse/issues/3860)) -- Add in flight real time metrics for Measure blocks ([\#3871](https://github.com/matrix-org/synapse/issues/3871)) -- Disable buffering and automatic retrying in treq requests to prevent timeouts. ([\#3872](https://github.com/matrix-org/synapse/issues/3872)) -- mention jemalloc in the README ([\#3877](https://github.com/matrix-org/synapse/issues/3877)) -- Remove unmaintained "nuke-room-from-db.sh" script ([\#3888](https://github.com/matrix-org/synapse/issues/3888)) - - -Synapse 0.33.4 (2018-09-07) -=========================== - -Internal Changes ----------------- - -- Unignore synctl in .dockerignore to fix docker builds ([\#3802](https://github.com/matrix-org/synapse/issues/3802)) - - -Synapse 0.33.4rc2 (2018-09-06) -============================== - -Pull in security fixes from v0.33.3.1 - - -Synapse 0.33.3.1 (2018-09-06) -============================= - -SECURITY FIXES --------------- - -- Fix an issue where event signatures were not always correctly validated ([\#3796](https://github.com/matrix-org/synapse/issues/3796)) -- Fix an issue where server_acls could be circumvented for incoming events ([\#3796](https://github.com/matrix-org/synapse/issues/3796)) - - -Internal Changes ----------------- - -- Unignore synctl in .dockerignore to fix docker builds ([\#3802](https://github.com/matrix-org/synapse/issues/3802)) - - -Synapse 0.33.4rc1 (2018-09-04) -============================== - -Features --------- - -- Support profile API endpoints on workers ([\#3659](https://github.com/matrix-org/synapse/issues/3659)) -- Server notices for resource limit blocking ([\#3680](https://github.com/matrix-org/synapse/issues/3680)) -- Allow guests to use /rooms/:roomId/event/:eventId ([\#3724](https://github.com/matrix-org/synapse/issues/3724)) -- Add mau_trial_days config param, so that users only get counted as MAU after N days. ([\#3749](https://github.com/matrix-org/synapse/issues/3749)) -- Require twisted 17.1 or later (fixes [#3741](https://github.com/matrix-org/synapse/issues/3741)). ([\#3751](https://github.com/matrix-org/synapse/issues/3751)) - - -Bugfixes --------- - -- Fix error collecting prometheus metrics when run on dedicated thread due to threading concurrency issues ([\#3722](https://github.com/matrix-org/synapse/issues/3722)) -- Fix bug where we resent "limit exceeded" server notices repeatedly ([\#3747](https://github.com/matrix-org/synapse/issues/3747)) -- Fix bug where we broke sync when using limit_usage_by_mau but hadn't configured server notices ([\#3753](https://github.com/matrix-org/synapse/issues/3753)) -- Fix 'federation_domain_whitelist' such that an empty list correctly blocks all outbound federation traffic ([\#3754](https://github.com/matrix-org/synapse/issues/3754)) -- Fix tagging of server notice rooms ([\#3755](https://github.com/matrix-org/synapse/issues/3755), [\#3756](https://github.com/matrix-org/synapse/issues/3756)) -- Fix 'admin_uri' config variable and error parameter to be 'admin_contact' to match the spec. ([\#3758](https://github.com/matrix-org/synapse/issues/3758)) -- Don't return non-LL-member state in incremental sync state blocks ([\#3760](https://github.com/matrix-org/synapse/issues/3760)) -- Fix bug in sending presence over federation ([\#3768](https://github.com/matrix-org/synapse/issues/3768)) -- Fix bug where preserved threepid user comes to sign up and server is mau blocked ([\#3777](https://github.com/matrix-org/synapse/issues/3777)) - -Internal Changes ----------------- - -- Removed the link to the unmaintained matrix-synapse-auto-deploy project from the readme. ([\#3378](https://github.com/matrix-org/synapse/issues/3378)) -- Refactor state module to support multiple room versions ([\#3673](https://github.com/matrix-org/synapse/issues/3673)) -- The synapse.storage module has been ported to Python 3. ([\#3725](https://github.com/matrix-org/synapse/issues/3725)) -- Split the state_group_cache into member and non-member state events (and so speed up LL /sync) ([\#3726](https://github.com/matrix-org/synapse/issues/3726)) -- Log failure to authenticate remote servers as warnings (without stack traces) ([\#3727](https://github.com/matrix-org/synapse/issues/3727)) -- The CONTRIBUTING guidelines have been updated to mention our use of Markdown and that .misc files have content. ([\#3730](https://github.com/matrix-org/synapse/issues/3730)) -- Reference the need for an HTTP replication port when using the federation_reader worker ([\#3734](https://github.com/matrix-org/synapse/issues/3734)) -- Fix minor spelling error in federation client documentation. ([\#3735](https://github.com/matrix-org/synapse/issues/3735)) -- Remove redundant state resolution function ([\#3737](https://github.com/matrix-org/synapse/issues/3737)) -- The test suite now passes on PostgreSQL. ([\#3740](https://github.com/matrix-org/synapse/issues/3740)) -- Fix MAU cache invalidation due to missing yield ([\#3746](https://github.com/matrix-org/synapse/issues/3746)) -- Make sure that we close db connections opened during init ([\#3764](https://github.com/matrix-org/synapse/issues/3764)) - - -Synapse 0.33.3 (2018-08-22) -=========================== - -Bugfixes --------- - -- Fix bug introduced in v0.33.3rc1 which made the ToS give a 500 error ([\#3732](https://github.com/matrix-org/synapse/issues/3732)) - - -Synapse 0.33.3rc2 (2018-08-21) -============================== - -Bugfixes --------- - -- Fix bug in v0.33.3rc1 which caused infinite loops and OOMs ([\#3723](https://github.com/matrix-org/synapse/issues/3723)) - - -Synapse 0.33.3rc1 (2018-08-21) -============================== - -Features --------- - -- Add support for the SNI extension to federation TLS connections. Thanks to @vojeroen! ([\#3439](https://github.com/matrix-org/synapse/issues/3439)) -- Add /_media/r0/config ([\#3184](https://github.com/matrix-org/synapse/issues/3184)) -- speed up /members API and add `at` and `membership` params as per MSC1227 ([\#3568](https://github.com/matrix-org/synapse/issues/3568)) -- implement `summary` block in /sync response as per MSC688 ([\#3574](https://github.com/matrix-org/synapse/issues/3574)) -- Add lazy-loading support to /messages as per MSC1227 ([\#3589](https://github.com/matrix-org/synapse/issues/3589)) -- Add ability to limit number of monthly active users on the server ([\#3633](https://github.com/matrix-org/synapse/issues/3633)) -- Support more federation endpoints on workers ([\#3653](https://github.com/matrix-org/synapse/issues/3653)) -- Basic support for room versioning ([\#3654](https://github.com/matrix-org/synapse/issues/3654)) -- Ability to disable client/server Synapse via conf toggle ([\#3655](https://github.com/matrix-org/synapse/issues/3655)) -- Ability to whitelist specific threepids against monthly active user limiting ([\#3662](https://github.com/matrix-org/synapse/issues/3662)) -- Add some metrics for the appservice and federation event sending loops ([\#3664](https://github.com/matrix-org/synapse/issues/3664)) -- Where server is disabled, block ability for locked out users to read new messages ([\#3670](https://github.com/matrix-org/synapse/issues/3670)) -- set admin uri via config, to be used in error messages where the user should contact the administrator ([\#3687](https://github.com/matrix-org/synapse/issues/3687)) -- Synapse's presence functionality can now be disabled with the "use_presence" configuration option. ([\#3694](https://github.com/matrix-org/synapse/issues/3694)) -- For resource limit blocked users, prevent writing into rooms ([\#3708](https://github.com/matrix-org/synapse/issues/3708)) - - -Bugfixes --------- - -- Fix occasional glitches in the synapse_event_persisted_position metric ([\#3658](https://github.com/matrix-org/synapse/issues/3658)) -- Fix bug on deleting 3pid when using identity servers that don't support unbind API ([\#3661](https://github.com/matrix-org/synapse/issues/3661)) -- Make the tests pass on Twisted < 18.7.0 ([\#3676](https://github.com/matrix-org/synapse/issues/3676)) -- Don’t ship recaptcha_ajax.js, use it directly from Google ([\#3677](https://github.com/matrix-org/synapse/issues/3677)) -- Fixes test_reap_monthly_active_users so it passes under postgres ([\#3681](https://github.com/matrix-org/synapse/issues/3681)) -- Fix mau blocking calulation bug on login ([\#3689](https://github.com/matrix-org/synapse/issues/3689)) -- Fix missing yield in synapse.storage.monthly_active_users.initialise_reserved_users ([\#3692](https://github.com/matrix-org/synapse/issues/3692)) -- Improve HTTP request logging to include all requests ([\#3700](https://github.com/matrix-org/synapse/issues/3700)) -- Avoid timing out requests while we are streaming back the response ([\#3701](https://github.com/matrix-org/synapse/issues/3701)) -- Support more federation endpoints on workers ([\#3705](https://github.com/matrix-org/synapse/issues/3705), [\#3713](https://github.com/matrix-org/synapse/issues/3713)) -- Fix "Starting db txn 'get_all_updated_receipts' from sentinel context" warning ([\#3710](https://github.com/matrix-org/synapse/issues/3710)) -- Fix bug where `state_cache` cache factor ignored environment variables ([\#3719](https://github.com/matrix-org/synapse/issues/3719)) - - -Deprecations and Removals -------------------------- - -- The Shared-Secret registration method of the legacy v1/register REST endpoint has been removed. For a replacement, please see [the admin/register API documentation](https://github.com/matrix-org/synapse/blob/master/docs/admin_api/register_api.rst). ([\#3703](https://github.com/matrix-org/synapse/issues/3703)) - - -Internal Changes ----------------- - -- The test suite now can run under PostgreSQL. ([\#3423](https://github.com/matrix-org/synapse/issues/3423)) -- Refactor HTTP replication endpoints to reduce code duplication ([\#3632](https://github.com/matrix-org/synapse/issues/3632)) -- Tests now correctly execute on Python 3. ([\#3647](https://github.com/matrix-org/synapse/issues/3647)) -- Sytests can now be run inside a Docker container. ([\#3660](https://github.com/matrix-org/synapse/issues/3660)) -- Port over enough to Python 3 to allow the sytests to start. ([\#3668](https://github.com/matrix-org/synapse/issues/3668)) -- Update docker base image from alpine 3.7 to 3.8. ([\#3669](https://github.com/matrix-org/synapse/issues/3669)) -- Rename synapse.util.async to synapse.util.async_helpers to mitigate async becoming a keyword on Python 3.7. ([\#3678](https://github.com/matrix-org/synapse/issues/3678)) -- Synapse's tests are now formatted with the black autoformatter. ([\#3679](https://github.com/matrix-org/synapse/issues/3679)) -- Implemented a new testing base class to reduce test boilerplate. ([\#3684](https://github.com/matrix-org/synapse/issues/3684)) -- Rename MAU prometheus metrics ([\#3690](https://github.com/matrix-org/synapse/issues/3690)) -- add new error type ResourceLimit ([\#3707](https://github.com/matrix-org/synapse/issues/3707)) -- Logcontexts for replication command handlers ([\#3709](https://github.com/matrix-org/synapse/issues/3709)) -- Update admin register API documentation to reference a real user ID. ([\#3712](https://github.com/matrix-org/synapse/issues/3712)) - - -Synapse 0.33.2 (2018-08-09) -=========================== - -No significant changes. - - -Synapse 0.33.2rc1 (2018-08-07) -============================== - -Features --------- - -- add support for the lazy_loaded_members filter as per MSC1227 ([\#2970](https://github.com/matrix-org/synapse/issues/2970)) -- add support for the include_redundant_members filter param as per MSC1227 ([\#3331](https://github.com/matrix-org/synapse/issues/3331)) -- Add metrics to track resource usage by background processes ([\#3553](https://github.com/matrix-org/synapse/issues/3553), [\#3556](https://github.com/matrix-org/synapse/issues/3556), [\#3604](https://github.com/matrix-org/synapse/issues/3604), [\#3610](https://github.com/matrix-org/synapse/issues/3610)) -- Add `code` label to `synapse_http_server_response_time_seconds` prometheus metric ([\#3554](https://github.com/matrix-org/synapse/issues/3554)) -- Add support for client_reader to handle more APIs ([\#3555](https://github.com/matrix-org/synapse/issues/3555), [\#3597](https://github.com/matrix-org/synapse/issues/3597)) -- make the /context API filter & lazy-load aware as per MSC1227 ([\#3567](https://github.com/matrix-org/synapse/issues/3567)) -- Add ability to limit number of monthly active users on the server ([\#3630](https://github.com/matrix-org/synapse/issues/3630)) -- When we fail to join a room over federation, pass the error code back to the client. ([\#3639](https://github.com/matrix-org/synapse/issues/3639)) -- Add a new /admin/register API for non-interactively creating users. ([\#3415](https://github.com/matrix-org/synapse/issues/3415)) - - -Bugfixes --------- - -- Make /directory/list API return 404 for room not found instead of 400. Thanks to @fuzzmz! ([\#3620](https://github.com/matrix-org/synapse/issues/3620)) -- Default inviter_display_name to mxid for email invites ([\#3391](https://github.com/matrix-org/synapse/issues/3391)) -- Don't generate TURN credentials if no TURN config options are set ([\#3514](https://github.com/matrix-org/synapse/issues/3514)) -- Correctly announce deleted devices over federation ([\#3520](https://github.com/matrix-org/synapse/issues/3520)) -- Catch failures saving metrics captured by Measure, and instead log the faulty metrics information for further analysis. ([\#3548](https://github.com/matrix-org/synapse/issues/3548)) -- Unicode passwords are now normalised before hashing, preventing the instance where two different devices or browsers might send a different UTF-8 sequence for the password. ([\#3569](https://github.com/matrix-org/synapse/issues/3569)) -- Fix potential stack overflow and deadlock under heavy load ([\#3570](https://github.com/matrix-org/synapse/issues/3570)) -- Respond with M_NOT_FOUND when profiles are not found locally or over federation. Fixes #3585 ([\#3585](https://github.com/matrix-org/synapse/issues/3585)) -- Fix failure to persist events over federation under load ([\#3601](https://github.com/matrix-org/synapse/issues/3601)) -- Fix updating of cached remote profiles ([\#3605](https://github.com/matrix-org/synapse/issues/3605)) -- Fix 'tuple index out of range' error ([\#3607](https://github.com/matrix-org/synapse/issues/3607)) -- Only import secrets when available (fix for py < 3.6) ([\#3626](https://github.com/matrix-org/synapse/issues/3626)) - - -Internal Changes ----------------- - -- Remove redundant checks on who_forgot_in_room ([\#3350](https://github.com/matrix-org/synapse/issues/3350)) -- Remove unnecessary event re-signing hacks ([\#3367](https://github.com/matrix-org/synapse/issues/3367)) -- Rewrite cache list decorator ([\#3384](https://github.com/matrix-org/synapse/issues/3384)) -- Move v1-only REST APIs into their own module. ([\#3460](https://github.com/matrix-org/synapse/issues/3460)) -- Replace more instances of Python 2-only iteritems and itervalues uses. ([\#3562](https://github.com/matrix-org/synapse/issues/3562)) -- Refactor EventContext to accept state during init ([\#3577](https://github.com/matrix-org/synapse/issues/3577)) -- Improve Dockerfile and docker-compose instructions ([\#3543](https://github.com/matrix-org/synapse/issues/3543)) -- Release notes are now in the Markdown format. ([\#3552](https://github.com/matrix-org/synapse/issues/3552)) -- add config for pep8 ([\#3559](https://github.com/matrix-org/synapse/issues/3559)) -- Merge Linearizer and Limiter ([\#3571](https://github.com/matrix-org/synapse/issues/3571), [\#3572](https://github.com/matrix-org/synapse/issues/3572)) -- Lazily load state on master process when using workers to reduce DB consumption ([\#3579](https://github.com/matrix-org/synapse/issues/3579), [\#3581](https://github.com/matrix-org/synapse/issues/3581), [\#3582](https://github.com/matrix-org/synapse/issues/3582), [\#3584](https://github.com/matrix-org/synapse/issues/3584)) -- Fixes and optimisations for resolve_state_groups ([\#3586](https://github.com/matrix-org/synapse/issues/3586)) -- Improve logging for exceptions when handling PDUs ([\#3587](https://github.com/matrix-org/synapse/issues/3587)) -- Add some measure blocks to persist_events ([\#3590](https://github.com/matrix-org/synapse/issues/3590)) -- Fix some random logcontext leaks. ([\#3591](https://github.com/matrix-org/synapse/issues/3591), [\#3606](https://github.com/matrix-org/synapse/issues/3606)) -- Speed up calculating state deltas in persist_event loop ([\#3592](https://github.com/matrix-org/synapse/issues/3592)) -- Attempt to reduce amount of state pulled out of DB during persist_events ([\#3595](https://github.com/matrix-org/synapse/issues/3595)) -- Fix a documentation typo in on_make_leave_request ([\#3609](https://github.com/matrix-org/synapse/issues/3609)) -- Make EventStore inherit from EventFederationStore ([\#3612](https://github.com/matrix-org/synapse/issues/3612)) -- Remove some redundant joins on event_edges.room_id ([\#3613](https://github.com/matrix-org/synapse/issues/3613)) -- Stop populating events.content ([\#3614](https://github.com/matrix-org/synapse/issues/3614)) -- Update the /send_leave path registration to use event_id rather than a transaction ID. ([\#3616](https://github.com/matrix-org/synapse/issues/3616)) -- Refactor FederationHandler to move DB writes into separate functions ([\#3621](https://github.com/matrix-org/synapse/issues/3621)) -- Remove unused field "pdu_failures" from transactions. ([\#3628](https://github.com/matrix-org/synapse/issues/3628)) -- rename replication_layer to federation_client ([\#3634](https://github.com/matrix-org/synapse/issues/3634)) -- Factor out exception handling in federation_client ([\#3638](https://github.com/matrix-org/synapse/issues/3638)) -- Refactor location of docker build script. ([\#3644](https://github.com/matrix-org/synapse/issues/3644)) -- Update CONTRIBUTING to mention newsfragments. ([\#3645](https://github.com/matrix-org/synapse/issues/3645)) - - -Synapse 0.33.1 (2018-08-02) -=========================== - -SECURITY FIXES --------------- - -- Fix a potential issue where servers could request events for rooms they have not joined. ([\#3641](https://github.com/matrix-org/synapse/issues/3641)) -- Fix a potential issue where users could see events in private rooms before they joined. ([\#3642](https://github.com/matrix-org/synapse/issues/3642)) - -Synapse 0.33.0 (2018-07-19) -=========================== - -Bugfixes --------- - -- Disable a noisy warning about logcontexts. ([\#3561](https://github.com/matrix-org/synapse/issues/3561)) - -Synapse 0.33.0rc1 (2018-07-18) -============================== - -Features --------- - -- Enforce the specified API for report\_event. ([\#3316](https://github.com/matrix-org/synapse/issues/3316)) -- Include CPU time from database threads in request/block metrics. ([\#3496](https://github.com/matrix-org/synapse/issues/3496), [\#3501](https://github.com/matrix-org/synapse/issues/3501)) -- Add CPU metrics for \_fetch\_event\_list. ([\#3497](https://github.com/matrix-org/synapse/issues/3497)) -- Optimisation to make handling incoming federation requests more efficient. ([\#3541](https://github.com/matrix-org/synapse/issues/3541)) - -Bugfixes --------- - -- Fix a significant performance regression in /sync. ([\#3505](https://github.com/matrix-org/synapse/issues/3505), [\#3521](https://github.com/matrix-org/synapse/issues/3521), [\#3530](https://github.com/matrix-org/synapse/issues/3530), [\#3544](https://github.com/matrix-org/synapse/issues/3544)) -- Use more portable syntax in our use of the attrs package, widening the supported versions. ([\#3498](https://github.com/matrix-org/synapse/issues/3498)) -- Fix queued federation requests being processed in the wrong order. ([\#3533](https://github.com/matrix-org/synapse/issues/3533)) -- Ensure that erasure requests are correctly honoured for publicly accessible rooms when accessed over federation. ([\#3546](https://github.com/matrix-org/synapse/issues/3546)) - -Misc ----- - -- Refactoring to improve testability. ([\#3351](https://github.com/matrix-org/synapse/issues/3351), [\#3499](https://github.com/matrix-org/synapse/issues/3499)) -- Use `isort` to sort imports. ([\#3463](https://github.com/matrix-org/synapse/issues/3463), [\#3464](https://github.com/matrix-org/synapse/issues/3464), [\#3540](https://github.com/matrix-org/synapse/issues/3540)) -- Use parse and asserts from http.servlet. ([\#3534](https://github.com/matrix-org/synapse/issues/3534), [\#3535](https://github.com/matrix-org/synapse/issues/3535)). - -Synapse 0.32.2 (2018-07-07) -=========================== - -Bugfixes --------- - -- Amend the Python dependencies to depend on attrs from PyPI, not attr ([\#3492](https://github.com/matrix-org/synapse/issues/3492)) - -Synapse 0.32.1 (2018-07-06) -=========================== - -Bugfixes --------- - -- Add explicit dependency on netaddr ([\#3488](https://github.com/matrix-org/synapse/issues/3488)) - -Changes in synapse v0.32.0 (2018-07-06) -======================================= - -No changes since 0.32.0rc1 - -Synapse 0.32.0rc1 (2018-07-05) -============================== - -Features --------- - -- Add blacklist & whitelist of servers allowed to send events to a room via `m.room.server_acl` event. -- Cache factor override system for specific caches ([\#3334](https://github.com/matrix-org/synapse/issues/3334)) -- Add metrics to track appservice transactions ([\#3344](https://github.com/matrix-org/synapse/issues/3344)) -- Try to log more helpful info when a sig verification fails ([\#3372](https://github.com/matrix-org/synapse/issues/3372)) -- Synapse now uses the best performing JSON encoder/decoder according to your runtime (simplejson on CPython, stdlib json on PyPy). ([\#3462](https://github.com/matrix-org/synapse/issues/3462)) -- Add optional ip\_range\_whitelist param to AS registration files to lock AS IP access ([\#3465](https://github.com/matrix-org/synapse/issues/3465)) -- Reject invalid server names in federation requests ([\#3480](https://github.com/matrix-org/synapse/issues/3480)) -- Reject invalid server names in homeserver.yaml ([\#3483](https://github.com/matrix-org/synapse/issues/3483)) - -Bugfixes --------- - -- Strip access\_token from outgoing requests ([\#3327](https://github.com/matrix-org/synapse/issues/3327)) -- Redact AS tokens in logs ([\#3349](https://github.com/matrix-org/synapse/issues/3349)) -- Fix federation backfill from SQLite servers ([\#3355](https://github.com/matrix-org/synapse/issues/3355)) -- Fix event-purge-by-ts admin API ([\#3363](https://github.com/matrix-org/synapse/issues/3363)) -- Fix event filtering in get\_missing\_events handler ([\#3371](https://github.com/matrix-org/synapse/issues/3371)) -- Synapse is now stricter regarding accepting events which it cannot retrieve the prev\_events for. ([\#3456](https://github.com/matrix-org/synapse/issues/3456)) -- Fix bug where synapse would explode when receiving unicode in HTTP User-Agent header ([\#3470](https://github.com/matrix-org/synapse/issues/3470)) -- Invalidate cache on correct thread to avoid race ([\#3473](https://github.com/matrix-org/synapse/issues/3473)) - -Improved Documentation ----------------------- - -- `doc/postgres.rst`: fix display of the last command block. Thanks to @ArchangeGabriel! ([\#3340](https://github.com/matrix-org/synapse/issues/3340)) - -Deprecations and Removals -------------------------- - -- Remove was\_forgotten\_at ([\#3324](https://github.com/matrix-org/synapse/issues/3324)) - -Misc ----- - -- [\#3332](https://github.com/matrix-org/synapse/issues/3332), [\#3341](https://github.com/matrix-org/synapse/issues/3341), [\#3347](https://github.com/matrix-org/synapse/issues/3347), [\#3348](https://github.com/matrix-org/synapse/issues/3348), [\#3356](https://github.com/matrix-org/synapse/issues/3356), [\#3385](https://github.com/matrix-org/synapse/issues/3385), [\#3446](https://github.com/matrix-org/synapse/issues/3446), [\#3447](https://github.com/matrix-org/synapse/issues/3447), [\#3467](https://github.com/matrix-org/synapse/issues/3467), [\#3474](https://github.com/matrix-org/synapse/issues/3474) - -Changes in synapse v0.31.2 (2018-06-14) -======================================= - -SECURITY UPDATE: Prevent unauthorised users from setting state events in a room when there is no `m.room.power_levels` event in force in the room. (PR #3397) - -Discussion around the Matrix Spec change proposal for this change can be followed at . - -Changes in synapse v0.31.1 (2018-06-08) -======================================= - -v0.31.1 fixes a security bug in the `get_missing_events` federation API where event visibility rules were not applied correctly. - -We are not aware of it being actively exploited but please upgrade asap. - -Bug Fixes: - -- Fix event filtering in get\_missing\_events handler (PR #3371) - -Changes in synapse v0.31.0 (2018-06-06) -======================================= - -Most notable change from v0.30.0 is to switch to the python prometheus library to improve system stats reporting. WARNING: this changes a number of prometheus metrics in a backwards-incompatible manner. For more details, see [docs/metrics-howto.rst](docs/metrics-howto.rst#removal-of-deprecated-metrics--time-based-counters-becoming-histograms-in-0310). - -Bug Fixes: - -- Fix metric documentation tables (PR #3341) -- Fix LaterGauge error handling (694968f) -- Fix replication metrics (b7e7fd2) - -Changes in synapse v0.31.0-rc1 (2018-06-04) -=========================================== - -Features: - -- Switch to the Python Prometheus library (PR #3256, #3274) -- Let users leave the server notice room after joining (PR #3287) - -Changes: - -- daily user type phone home stats (PR #3264) -- Use iter\* methods for \_filter\_events\_for\_server (PR #3267) -- Docs on consent bits (PR #3268) -- Remove users from user directory on deactivate (PR #3277) -- Avoid sending consent notice to guest users (PR #3288) -- disable CPUMetrics if no /proc/self/stat (PR #3299) -- Consistently use six\'s iteritems and wrap lazy keys/values in list() if they\'re not meant to be lazy (PR #3307) -- Add private IPv6 addresses to example config for url preview blacklist (PR #3317) Thanks to @thegcat! -- Reduce stuck read-receipts: ignore depth when updating (PR #3318) -- Put python\'s logs into Trial when running unit tests (PR #3319) - -Changes, python 3 migration: - -- Replace some more comparisons with six (PR #3243) Thanks to @NotAFile! -- replace some iteritems with six (PR #3244) Thanks to @NotAFile! -- Add batch\_iter to utils (PR #3245) Thanks to @NotAFile! -- use repr, not str (PR #3246) Thanks to @NotAFile! -- Misc Python3 fixes (PR #3247) Thanks to @NotAFile! -- Py3 storage/\_base.py (PR #3278) Thanks to @NotAFile! -- more six iteritems (PR #3279) Thanks to @NotAFile! -- More Misc. py3 fixes (PR #3280) Thanks to @NotAFile! -- remaining isintance fixes (PR #3281) Thanks to @NotAFile! -- py3-ize state.py (PR #3283) Thanks to @NotAFile! -- extend tox testing for py3 to avoid regressions (PR #3302) Thanks to @krombel! -- use memoryview in py3 (PR #3303) Thanks to @NotAFile! - -Bugs: - -- Fix federation backfill bugs (PR #3261) -- federation: fix LaterGauge usage (PR #3328) Thanks to @intelfx! - -Changes in synapse v0.30.0 (2018-05-24) -======================================= - -\'Server Notices\' are a new feature introduced in Synapse 0.30. They provide a channel whereby server administrators can send messages to users on the server. - -They are used as part of communication of the server policies (see `docs/consent_tracking.md`), however the intention is that they may also find a use for features such as \"Message of the day\". - -This feature is specific to Synapse, but uses standard Matrix communication mechanisms, so should work with any Matrix client. For more details see `docs/server_notices.md` - -Further Server Notices/Consent Tracking Support: - -- Allow overriding the server\_notices user\'s avatar (PR #3273) -- Use the localpart in the consent uri (PR #3272) -- Support for putting %(consent\_uri)s in messages (PR #3271) -- Block attempts to send server notices to remote users (PR #3270) -- Docs on consent bits (PR #3268) - -Changes in synapse v0.30.0-rc1 (2018-05-23) -=========================================== - -Server Notices/Consent Tracking Support: - -- ConsentResource to gather policy consent from users (PR #3213) -- Move RoomCreationHandler out of synapse.handlers.Handlers (PR #3225) -- Infrastructure for a server notices room (PR #3232) -- Send users a server notice about consent (PR #3236) -- Reject attempts to send event before privacy consent is given (PR #3257) -- Add a \'has\_consented\' template var to consent forms (PR #3262) -- Fix dependency on jinja2 (PR #3263) - -Features: - -- Cohort analytics (PR #3163, #3241, #3251) -- Add lxml to docker image for web previews (PR #3239) Thanks to @ptman! -- Add in flight request metrics (PR #3252) - -Changes: - -- Remove unused update\_external\_syncs (PR #3233) -- Use stream rather depth ordering for push actions (PR #3212) -- Make purge\_history operate on tokens (PR #3221) -- Don\'t support limitless pagination (PR #3265) - -Bug Fixes: - -- Fix logcontext resource usage tracking (PR #3258) -- Fix error in handling receipts (PR #3235) -- Stop the transaction cache caching failures (PR #3255) - -Changes in synapse v0.29.1 (2018-05-17) -======================================= - -Changes: - -- Update docker documentation (PR #3222) - -Changes in synapse v0.29.0 (2018-05-16) -======================================= - -Not changes since v0.29.0-rc1 - -Changes in synapse v0.29.0-rc1 (2018-05-14) -=========================================== - -Notable changes, a docker file for running Synapse (Thanks to @kaiyou!) and a closed spec bug in the Client Server API. Additionally further prep for Python 3 migration. - -Potentially breaking change: - -- Make Client-Server API return 401 for invalid token (PR #3161). - - This changes the Client-server spec to return a 401 error code instead of 403 when the access token is unrecognised. This is the behaviour required by the specification, but some clients may be relying on the old, incorrect behaviour. - - Thanks to @NotAFile for fixing this. - -Features: - -- Add a Dockerfile for synapse (PR #2846) Thanks to @kaiyou! - -Changes - General: - -- nuke-room-from-db.sh: added postgresql option and help (PR #2337) Thanks to @rubo77! -- Part user from rooms on account deactivate (PR #3201) -- Make \'unexpected logging context\' into warnings (PR #3007) -- Set Server header in SynapseRequest (PR #3208) -- remove duplicates from groups tables (PR #3129) -- Improve exception handling for background processes (PR #3138) -- Add missing consumeErrors to improve exception handling (PR #3139) -- reraise exceptions more carefully (PR #3142) -- Remove redundant call to preserve\_fn (PR #3143) -- Trap exceptions thrown within run\_in\_background (PR #3144) - -Changes - Refactors: - -- Refactor /context to reuse pagination storage functions (PR #3193) -- Refactor recent events func to use pagination func (PR #3195) -- Refactor pagination DB API to return concrete type (PR #3196) -- Refactor get\_recent\_events\_for\_room return type (PR #3198) -- Refactor sync APIs to reuse pagination API (PR #3199) -- Remove unused code path from member change DB func (PR #3200) -- Refactor request handling wrappers (PR #3203) -- transaction\_id, destination defined twice (PR #3209) Thanks to @damir-manapov! -- Refactor event storage to prepare for changes in state calculations (PR #3141) -- Set Server header in SynapseRequest (PR #3208) -- Use deferred.addTimeout instead of time\_bound\_deferred (PR #3127, #3178) -- Use run\_in\_background in preference to preserve\_fn (PR #3140) - -Changes - Python 3 migration: - -- Construct HMAC as bytes on py3 (PR #3156) Thanks to @NotAFile! -- run config tests on py3 (PR #3159) Thanks to @NotAFile! -- Open certificate files as bytes (PR #3084) Thanks to @NotAFile! -- Open config file in non-bytes mode (PR #3085) Thanks to @NotAFile! -- Make event properties raise AttributeError instead (PR #3102) Thanks to @NotAFile! -- Use six.moves.urlparse (PR #3108) Thanks to @NotAFile! -- Add py3 tests to tox with folders that work (PR #3145) Thanks to @NotAFile! -- Don\'t yield in list comprehensions (PR #3150) Thanks to @NotAFile! -- Move more xrange to six (PR #3151) Thanks to @NotAFile! -- make imports local (PR #3152) Thanks to @NotAFile! -- move httplib import to six (PR #3153) Thanks to @NotAFile! -- Replace stringIO imports with six (PR #3154, #3168) Thanks to @NotAFile! -- more bytes strings (PR #3155) Thanks to @NotAFile! - -Bug Fixes: - -- synapse fails to start under Twisted \>= 18.4 (PR #3157) -- Fix a class of logcontext leaks (PR #3170) -- Fix a couple of logcontext leaks in unit tests (PR #3172) -- Fix logcontext leak in media repo (PR #3174) -- Escape label values in prometheus metrics (PR #3175, #3186) -- Fix \'Unhandled Error\' logs with Twisted 18.4 (PR #3182) Thanks to @Half-Shot! -- Fix logcontext leaks in rate limiter (PR #3183) -- notifications: Convert next\_token to string according to the spec (PR #3190) Thanks to @mujx! -- nuke-room-from-db.sh: fix deletion from search table (PR #3194) Thanks to @rubo77! -- add guard for None on purge\_history api (PR #3160) Thanks to @krombel! - -Changes in synapse v0.28.1 (2018-05-01) -======================================= - -SECURITY UPDATE - -- Clamp the allowed values of event depth received over federation to be \[0, 2\^63 - 1\]. This mitigates an attack where malicious events injected with depth = 2\^63 - 1 render rooms unusable. Depth is used to determine the cosmetic ordering of events within a room, and so the ordering of events in such a room will default to using stream\_ordering rather than depth (topological\_ordering). - - This is a temporary solution to mitigate abuse in the wild, whilst a long term solution is being implemented to improve how the depth parameter is used. - - Full details at - -- Pin Twisted to \<18.4 until we stop using the private \_OpenSSLECCurve API. - -Changes in synapse v0.28.0 (2018-04-26) -======================================= - -Bug Fixes: - -- Fix quarantine media admin API and search reindex (PR #3130) -- Fix media admin APIs (PR #3134) - -Changes in synapse v0.28.0-rc1 (2018-04-24) -=========================================== - -Minor performance improvement to federation sending and bug fixes. - -(Note: This release does not include the delta state resolution implementation discussed in matrix live) - -Features: - -- Add metrics for event processing lag (PR #3090) -- Add metrics for ResponseCache (PR #3092) - -Changes: - -- Synapse on PyPy (PR #2760) Thanks to @Valodim! -- move handling of auto\_join\_rooms to RegisterHandler (PR #2996) Thanks to @krombel! -- Improve handling of SRV records for federation connections (PR #3016) Thanks to @silkeh! -- Document the behaviour of ResponseCache (PR #3059) -- Preparation for py3 (PR #3061, #3073, #3074, #3075, #3103, #3104, #3106, #3107, #3109, #3110) Thanks to @NotAFile! -- update prometheus dashboard to use new metric names (PR #3069) Thanks to @krombel! -- use python3-compatible prints (PR #3074) Thanks to @NotAFile! -- Send federation events concurrently (PR #3078) -- Limit concurrent event sends for a room (PR #3079) -- Improve R30 stat definition (PR #3086) -- Send events to ASes concurrently (PR #3088) -- Refactor ResponseCache usage (PR #3093) -- Clarify that SRV may not point to a CNAME (PR #3100) Thanks to @silkeh! -- Use str(e) instead of e.message (PR #3103) Thanks to @NotAFile! -- Use six.itervalues in some places (PR #3106) Thanks to @NotAFile! -- Refactor store.have\_events (PR #3117) - -Bug Fixes: - -- Return 401 for invalid access\_token on logout (PR #2938) Thanks to @dklug! -- Return a 404 rather than a 500 on rejoining empty rooms (PR #3080) -- fix federation\_domain\_whitelist (PR #3099) -- Avoid creating events with huge numbers of prev\_events (PR #3113) -- Reject events which have lots of prev\_events (PR #3118) - -Changes in synapse v0.27.4 (2018-04-13) -======================================= - -Changes: - -- Update canonicaljson dependency (\#3095) - -Changes in synapse v0.27.3 (2018-04-11) -====================================== - -Bug fixes: - -- URL quote path segments over federation (\#3082) - -Changes in synapse v0.27.3-rc2 (2018-04-09) -=========================================== - -v0.27.3-rc1 used a stale version of the develop branch so the changelog overstates the functionality. v0.27.3-rc2 is up to date, rc1 should be ignored. - -Changes in synapse v0.27.3-rc1 (2018-04-09) -=========================================== - -Notable changes include API support for joinability of groups. Also new metrics and phone home stats. Phone home stats include better visibility of system usage so we can tweak synpase to work better for all users rather than our own experience with matrix.org. Also, recording \'r30\' stat which is the measure we use to track overal growth of the Matrix ecosystem. It is defined as:- - -Counts the number of native 30 day retained users, defined as:- \* Users who have created their accounts more than 30 days - -: - Where last seen at most 30 days ago - - Where account creation and last\_seen are \> 30 days\" - -Features: - -- Add joinability for groups (PR #3045) -- Implement group join API (PR #3046) -- Add counter metrics for calculating state delta (PR #3033) -- R30 stats (PR #3041) -- Measure time it takes to calculate state group ID (PR #3043) -- Add basic performance statistics to phone home (PR #3044) -- Add response size metrics (PR #3071) -- phone home cache size configurations (PR #3063) - -Changes: - -- Add a blurb explaining the main synapse worker (PR #2886) Thanks to @turt2live! -- Replace old style error catching with \'as\' keyword (PR #3000) Thanks to @NotAFile! -- Use .iter\* to avoid copies in StateHandler (PR #3006) -- Linearize calls to \_generate\_user\_id (PR #3029) -- Remove last usage of ujson (PR #3030) -- Use simplejson throughout (PR #3048) -- Use static JSONEncoders (PR #3049) -- Remove uses of events.content (PR #3060) -- Improve database cache performance (PR #3068) - -Bug fixes: - -- Add room\_id to the response of rooms/{roomId}/join (PR #2986) Thanks to @jplatte! -- Fix replication after switch to simplejson (PR #3015) -- 404 correctly on missing paths via NoResource (PR #3022) -- Fix error when claiming e2e keys from offline servers (PR #3034) -- fix tests/storage/test\_user\_directory.py (PR #3042) -- use PUT instead of POST for federating groups/m.join\_policy (PR #3070) Thanks to @krombel! -- postgres port script: fix state\_groups\_pkey error (PR #3072) - -Changes in synapse v0.27.2 (2018-03-26) -======================================= - -Bug fixes: - -- Fix bug which broke TCP replication between workers (PR #3015) - -Changes in synapse v0.27.1 (2018-03-26) -======================================= - -Meta release as v0.27.0 temporarily pointed to the wrong commit - -Changes in synapse v0.27.0 (2018-03-26) -======================================= - -No changes since v0.27.0-rc2 - -Changes in synapse v0.27.0-rc2 (2018-03-19) -=========================================== - -Pulls in v0.26.1 - -Bug fixes: - -- Fix bug introduced in v0.27.0-rc1 that causes much increased memory usage in state cache (PR #3005) - -Changes in synapse v0.26.1 (2018-03-15) -======================================= - -Bug fixes: - -- Fix bug where an invalid event caused server to stop functioning correctly, due to parsing and serializing bugs in ujson library (PR #3008) - -Changes in synapse v0.27.0-rc1 (2018-03-14) -=========================================== - -The common case for running Synapse is not to run separate workers, but for those that do, be aware that synctl no longer starts the main synapse when using `-a` option with workers. A new worker file should be added with `worker_app: synapse.app.homeserver`. - -This release also begins the process of renaming a number of the metrics reported to prometheus. See [docs/metrics-howto.rst](docs/metrics-howto.rst#block-and-response-metrics-renamed-for-0-27-0). Note that the v0.28.0 release will remove the deprecated metric names. - -Features: - -- Add ability for ASes to override message send time (PR #2754) -- Add support for custom storage providers for media repository (PR #2867, #2777, #2783, #2789, #2791, #2804, #2812, #2814, #2857, #2868, #2767) -- Add purge API features, see [docs/admin\_api/purge\_history\_api.rst](docs/admin_api/purge_history_api.rst) for full details (PR #2858, #2867, #2882, #2946, #2962, #2943) -- Add support for whitelisting 3PIDs that users can register. (PR #2813) -- Add `/room/{id}/event/{id}` API (PR #2766) -- Add an admin API to get all the media in a room (PR #2818) Thanks to @turt2live! -- Add `federation_domain_whitelist` option (PR #2820, #2821) - -Changes: - -- Continue to factor out processing from main process and into worker processes. See updated [docs/workers.rst](docs/workers.rst) (PR #2892 - \#2904, #2913, #2920 - \#2926, #2947, #2847, #2854, #2872, #2873, #2874, #2928, #2929, #2934, #2856, #2976 - \#2984, #2987 - \#2989, #2991 - \#2993, #2995, #2784) -- Ensure state cache is used when persisting events (PR #2864, #2871, #2802, #2835, #2836, #2841, #2842, #2849) -- Change the default config to bind on both IPv4 and IPv6 on all platforms (PR #2435) Thanks to @silkeh! -- No longer require a specific version of saml2 (PR #2695) Thanks to @okurz! -- Remove `verbosity`/`log_file` from generated config (PR #2755) -- Add and improve metrics and logging (PR #2770, #2778, #2785, #2786, #2787, #2793, #2794, #2795, #2809, #2810, #2833, #2834, #2844, #2965, #2927, #2975, #2790, #2796, #2838) -- When using synctl with workers, don\'t start the main synapse automatically (PR #2774) -- Minor performance improvements (PR #2773, #2792) -- Use a connection pool for non-federation outbound connections (PR #2817) -- Make it possible to run unit tests against postgres (PR #2829) -- Update pynacl dependency to 1.2.1 or higher (PR #2888) Thanks to @bachp! -- Remove ability for AS users to call /events and /sync (PR #2948) -- Use bcrypt.checkpw (PR #2949) Thanks to @krombel! - -Bug fixes: - -- Fix broken `ldap_config` config option (PR #2683) Thanks to @seckrv! -- Fix error message when user is not allowed to unban (PR #2761) Thanks to @turt2live! -- Fix publicised groups GET API (singular) over federation (PR #2772) -- Fix user directory when using `user_directory_search_all_users` config option (PR #2803, #2831) -- Fix error on `/publicRooms` when no rooms exist (PR #2827) -- Fix bug in quarantine\_media (PR #2837) -- Fix url\_previews when no Content-Type is returned from URL (PR #2845) -- Fix rare race in sync API when joining room (PR #2944) -- Fix slow event search, switch back from GIST to GIN indexes (PR #2769, #2848) - -Changes in synapse v0.26.0 (2018-01-05) -======================================= - -No changes since v0.26.0-rc1 - -Changes in synapse v0.26.0-rc1 (2017-12-13) -=========================================== - -Features: - -- Add ability for ASes to publicise groups for their users (PR #2686) -- Add all local users to the user\_directory and optionally search them (PR #2723) -- Add support for custom login types for validating users (PR #2729) - -Changes: - -- Update example Prometheus config to new format (PR #2648) Thanks to @krombel! -- Rename redact\_content option to include\_content in Push API (PR #2650) -- Declare support for r0.3.0 (PR #2677) -- Improve upserts (PR #2684, #2688, #2689, #2713) -- Improve documentation of workers (PR #2700) -- Improve tracebacks on exceptions (PR #2705) -- Allow guest access to group APIs for reading (PR #2715) -- Support for posting content in federation\_client script (PR #2716) -- Delete devices and pushers on logouts etc (PR #2722) - -Bug fixes: - -- Fix database port script (PR #2673) -- Fix internal server error on login with ldap\_auth\_provider (PR #2678) Thanks to @jkolo! -- Fix error on sqlite 3.7 (PR #2697) -- Fix OPTIONS on preview\_url (PR #2707) -- Fix error handling on dns lookup (PR #2711) -- Fix wrong avatars when inviting multiple users when creating room (PR #2717) -- Fix 500 when joining matrix-dev (PR #2719) - -Changes in synapse v0.25.1 (2017-11-17) -======================================= - -Bug fixes: - -- Fix login with LDAP and other password provider modules (PR #2678). Thanks to @jkolo! - -Changes in synapse v0.25.0 (2017-11-15) -======================================= - -Bug fixes: - -- Fix port script (PR #2673) - -Changes in synapse v0.25.0-rc1 (2017-11-14) -=========================================== - -Features: - -- Add is\_public to groups table to allow for private groups (PR #2582) -- Add a route for determining who you are (PR #2668) Thanks to @turt2live! -- Add more features to the password providers (PR #2608, #2610, #2620, #2622, #2623, #2624, #2626, #2628, #2629) -- Add a hook for custom rest endpoints (PR #2627) -- Add API to update group room visibility (PR #2651) - -Changes: - -- Ignore \ tags when generating URL preview descriptions (PR #2576) Thanks to @maximevaillancourt! -- Register some /unstable endpoints in /r0 as well (PR #2579) Thanks to @krombel! -- Support /keys/upload on /r0 as well as /unstable (PR #2585) -- Front-end proxy: pass through auth header (PR #2586) -- Allow ASes to deactivate their own users (PR #2589) -- Remove refresh tokens (PR #2613) -- Automatically set default displayname on register (PR #2617) -- Log login requests (PR #2618) -- Always return is\_public in the /groups/:group\_id/rooms API (PR #2630) -- Avoid no-op media deletes (PR #2637) Thanks to @spantaleev! -- Fix various embarrassing typos around user\_directory and add some doc. (PR #2643) -- Return whether a user is an admin within a group (PR #2647) -- Namespace visibility options for groups (PR #2657) -- Downcase UserIDs on registration (PR #2662) -- Cache failures when fetching URL previews (PR #2669) - -Bug fixes: - -- Fix port script (PR #2577) -- Fix error when running synapse with no logfile (PR #2581) -- Fix UI auth when deleting devices (PR #2591) -- Fix typo when checking if user is invited to group (PR #2599) -- Fix the port script to drop NUL values in all tables (PR #2611) -- Fix appservices being backlogged and not receiving new events due to a bug in notify\_interested\_services (PR #2631) Thanks to @xyzz! -- Fix updating rooms avatar/display name when modified by admin (PR #2636) Thanks to @farialima! -- Fix bug in state group storage (PR #2649) -- Fix 500 on invalid utf-8 in request (PR #2663) - -Changes in synapse v0.24.1 (2017-10-24) -======================================= - -Bug fixes: - -- Fix updating group profiles over federation (PR #2567) - -Changes in synapse v0.24.0 (2017-10-23) -======================================= - -No changes since v0.24.0-rc1 - -Changes in synapse v0.24.0-rc1 (2017-10-19) -=========================================== - -Features: - -- Add Group Server (PR #2352, #2363, #2374, #2377, #2378, #2382, #2410, #2426, #2430, #2454, #2471, #2472, #2544) -- Add support for channel notifications (PR #2501) -- Add basic implementation of backup media store (PR #2538) -- Add config option to auto-join new users to rooms (PR #2545) - -Changes: - -- Make the spam checker a module (PR #2474) -- Delete expired url cache data (PR #2478) -- Ignore incoming events for rooms that we have left (PR #2490) -- Allow spam checker to reject invites too (PR #2492) -- Add room creation checks to spam checker (PR #2495) -- Spam checking: add the invitee to user\_may\_invite (PR #2502) -- Process events from federation for different rooms in parallel (PR #2520) -- Allow error strings from spam checker (PR #2531) -- Improve error handling for missing files in config (PR #2551) - -Bug fixes: - -- Fix handling SERVFAILs when doing AAAA lookups for federation (PR #2477) -- Fix incompatibility with newer versions of ujson (PR #2483) Thanks to @jeremycline! -- Fix notification keywords that start/end with non-word chars (PR #2500) -- Fix stack overflow and logcontexts from linearizer (PR #2532) -- Fix 500 error when fields missing from power\_levels event (PR #2552) -- Fix 500 error when we get an error handling a PDU (PR #2553) - -Changes in synapse v0.23.1 (2017-10-02) -======================================= - -Changes: - -- Make \'affinity\' package optional, as it is not supported on some platforms - -Changes in synapse v0.23.0 (2017-10-02) -======================================= - -No changes since v0.23.0-rc2 - -Changes in synapse v0.23.0-rc2 (2017-09-26) -=========================================== - -Bug fixes: - -- Fix regression in performance of syncs (PR #2470) - -Changes in synapse v0.23.0-rc1 (2017-09-25) -=========================================== - -Features: - -- Add a frontend proxy worker (PR #2344) -- Add support for event\_id\_only push format (PR #2450) -- Add a PoC for filtering spammy events (PR #2456) -- Add a config option to block all room invites (PR #2457) - -Changes: - -- Use bcrypt module instead of py-bcrypt (PR #2288) Thanks to @kyrias! -- Improve performance of generating push notifications (PR #2343, #2357, #2365, #2366, #2371) -- Improve DB performance for device list handling in sync (PR #2362) -- Include a sample prometheus config (PR #2416) -- Document known to work postgres version (PR #2433) Thanks to @ptman! - -Bug fixes: - -- Fix caching error in the push evaluator (PR #2332) -- Fix bug where pusherpool didn\'t start and broke some rooms (PR #2342) -- Fix port script for user directory tables (PR #2375) -- Fix device lists notifications when user rejoins a room (PR #2443, #2449) -- Fix sync to always send down current state events in timeline (PR #2451) -- Fix bug where guest users were incorrectly kicked (PR #2453) -- Fix bug talking to IPv6 only servers using SRV records (PR #2462) - -Changes in synapse v0.22.1 (2017-07-06) -======================================= - -Bug fixes: - -- Fix bug where pusher pool didn\'t start and caused issues when interacting with some rooms (PR #2342) - -Changes in synapse v0.22.0 (2017-07-06) -======================================= - -No changes since v0.22.0-rc2 - -Changes in synapse v0.22.0-rc2 (2017-07-04) -=========================================== - -Changes: - -- Improve performance of storing user IPs (PR #2307, #2308) -- Slightly improve performance of verifying access tokens (PR #2320) -- Slightly improve performance of event persistence (PR #2321) -- Increase default cache factor size from 0.1 to 0.5 (PR #2330) - -Bug fixes: - -- Fix bug with storing registration sessions that caused frequent CPU churn (PR #2319) - -Changes in synapse v0.22.0-rc1 (2017-06-26) -=========================================== - -Features: - -- Add a user directory API (PR #2252, and many more) -- Add shutdown room API to remove room from local server (PR #2291) -- Add API to quarantine media (PR #2292) -- Add new config option to not send event contents to push servers (PR #2301) Thanks to @cjdelisle! - -Changes: - -- Various performance fixes (PR #2177, #2233, #2230, #2238, #2248, #2256, #2274) -- Deduplicate sync filters (PR #2219) Thanks to @krombel! -- Correct a typo in UPGRADE.rst (PR #2231) Thanks to @aaronraimist! -- Add count of one time keys to sync stream (PR #2237) -- Only store event\_auth for state events (PR #2247) -- Store URL cache preview downloads separately (PR #2299) - -Bug fixes: - -- Fix users not getting notifications when AS listened to that user\_id (PR #2216) Thanks to @slipeer! -- Fix users without push set up not getting notifications after joining rooms (PR #2236) -- Fix preview url API to trim long descriptions (PR #2243) -- Fix bug where we used cached but unpersisted state group as prev group, resulting in broken state of restart (PR #2263) -- Fix removing of pushers when using workers (PR #2267) -- Fix CORS headers to allow Authorization header (PR #2285) Thanks to @krombel! - -Changes in synapse v0.21.1 (2017-06-15) -======================================= - -Bug fixes: - -- Fix bug in anonymous usage statistic reporting (PR #2281) - -Changes in synapse v0.21.0 (2017-05-18) -======================================= - -No changes since v0.21.0-rc3 - -Changes in synapse v0.21.0-rc3 (2017-05-17) -=========================================== - -Features: - -- Add per user rate-limiting overrides (PR #2208) -- Add config option to limit maximum number of events requested by `/sync` and `/messages` (PR #2221) Thanks to @psaavedra! - -Changes: - -- Various small performance fixes (PR #2201, #2202, #2224, #2226, #2227, #2228, #2229) -- Update username availability checker API (PR #2209, #2213) -- When purging, don\'t de-delta state groups we\'re about to delete (PR #2214) -- Documentation to check synapse version (PR #2215) Thanks to @hamber-dick! -- Add an index to event\_search to speed up purge history API (PR #2218) - -Bug fixes: - -- Fix API to allow clients to upload one-time-keys with new sigs (PR #2206) - -Changes in synapse v0.21.0-rc2 (2017-05-08) -=========================================== - -Changes: - -- Always mark remotes as up if we receive a signed request from them (PR #2190) - -Bug fixes: - -- Fix bug where users got pushed for rooms they had muted (PR #2200) - -Changes in synapse v0.21.0-rc1 (2017-05-08) -=========================================== - -Features: - -- Add username availability checker API (PR #2183) -- Add read marker API (PR #2120) - -Changes: - -- Enable guest access for the 3pl/3pid APIs (PR #1986) -- Add setting to support TURN for guests (PR #2011) -- Various performance improvements (PR #2075, #2076, #2080, #2083, #2108, #2158, #2176, #2185) -- Make synctl a bit more user friendly (PR #2078, #2127) Thanks @APwhitehat! -- Replace HTTP replication with TCP replication (PR #2082, #2097, #2098, #2099, #2103, #2014, #2016, #2115, #2116, #2117) -- Support authenticated SMTP (PR #2102) Thanks @DanielDent! -- Add a counter metric for successfully-sent transactions (PR #2121) -- Propagate errors sensibly from proxied IS requests (PR #2147) -- Add more granular event send metrics (PR #2178) - -Bug fixes: - -- Fix nuke-room script to work with current schema (PR #1927) Thanks @zuckschwerdt! -- Fix db port script to not assume postgres tables are in the public schema (PR #2024) Thanks @jerrykan! -- Fix getting latest device IP for user with no devices (PR #2118) -- Fix rejection of invites to unreachable servers (PR #2145) -- Fix code for reporting old verify keys in synapse (PR #2156) -- Fix invite state to always include all events (PR #2163) -- Fix bug where synapse would always fetch state for any missing event (PR #2170) -- Fix a leak with timed out HTTP connections (PR #2180) -- Fix bug where we didn\'t time out HTTP requests to ASes (PR #2192) - -Docs: - -- Clarify doc for SQLite to PostgreSQL port (PR #1961) Thanks @benhylau! -- Fix typo in synctl help (PR #2107) Thanks @HarHarLinks! -- `web_client_location` documentation fix (PR #2131) Thanks @matthewjwolff! -- Update README.rst with FreeBSD changes (PR #2132) Thanks @feld! -- Clarify setting up metrics (PR #2149) Thanks @encks! - -Changes in synapse v0.20.0 (2017-04-11) -======================================= - -Bug fixes: - -- Fix joining rooms over federation where not all servers in the room saw the new server had joined (PR #2094) - -Changes in synapse v0.20.0-rc1 (2017-03-30) -=========================================== - -Features: - -- Add delete\_devices API (PR #1993) -- Add phone number registration/login support (PR #1994, #2055) - -Changes: - -- Use JSONSchema for validation of filters. Thanks @pik! (PR #1783) -- Reread log config on SIGHUP (PR #1982) -- Speed up public room list (PR #1989) -- Add helpful texts to logger config options (PR #1990) -- Minor `/sync` performance improvements. (PR #2002, #2013, #2022) -- Add some debug to help diagnose weird federation issue (PR #2035) -- Correctly limit retries for all federation requests (PR #2050, #2061) -- Don\'t lock table when persisting new one time keys (PR #2053) -- Reduce some CPU work on DB threads (PR #2054) -- Cache hosts in room (PR #2060) -- Batch sending of device list pokes (PR #2063) -- Speed up persist event path in certain edge cases (PR #2070) - -Bug fixes: - -- Fix bug where current\_state\_events renamed to current\_state\_ids (PR #1849) -- Fix routing loop when fetching remote media (PR #1992) -- Fix current\_state\_events table to not lie (PR #1996) -- Fix CAS login to handle PartialDownloadError (PR #1997) -- Fix assertion to stop transaction queue getting wedged (PR #2010) -- Fix presence to fallback to last\_active\_ts if it beats the last sync time. Thanks @Half-Shot! (PR #2014) -- Fix bug when federation received a PDU while a room join is in progress (PR #2016) -- Fix resetting state on rejected events (PR #2025) -- Fix installation issues in readme. Thanks @ricco386 (PR #2037) -- Fix caching of remote servers\' signature keys (PR #2042) -- Fix some leaking log context (PR #2048, #2049, #2057, #2058) -- Fix rejection of invites not reaching sync (PR #2056) - -Changes in synapse v0.19.3 (2017-03-20) -======================================= - -No changes since v0.19.3-rc2 - -Changes in synapse v0.19.3-rc2 (2017-03-13) -=========================================== - -Bug fixes: - -- Fix bug in handling of incoming device list updates over federation. - -Changes in synapse v0.19.3-rc1 (2017-03-08) -=========================================== - -Features: - -- Add some administration functionalities. Thanks to morteza-araby! (PR #1784) - -Changes: - -- Reduce database table sizes (PR #1873, #1916, #1923, #1963) -- Update contrib/ to not use syutil. Thanks to andrewshadura! (PR #1907) -- Don\'t fetch current state when sending an event in common case (PR #1955) - -Bug fixes: - -- Fix synapse\_port\_db failure. Thanks to Pneumaticat! (PR #1904) -- Fix caching to not cache error responses (PR #1913) -- Fix APIs to make kick & ban reasons work (PR #1917) -- Fix bugs in the /keys/changes api (PR #1921) -- Fix bug where users couldn\'t forget rooms they were banned from (PR #1922) -- Fix issue with long language values in pushers API (PR #1925) -- Fix a race in transaction queue (PR #1930) -- Fix dynamic thumbnailing to preserve aspect ratio. Thanks to jkolo! (PR #1945) -- Fix device list update to not constantly resync (PR #1964) -- Fix potential for huge memory usage when getting device that have changed (PR #1969) - -Changes in synapse v0.19.2 (2017-02-20) -======================================= - -- Fix bug with event visibility check in /context/ API. Thanks to Tokodomo for pointing it out! (PR #1929) - -Changes in synapse v0.19.1 (2017-02-09) -======================================= - -- Fix bug where state was incorrectly reset in a room when synapse received an event over federation that did not pass auth checks (PR #1892) - -Changes in synapse v0.19.0 (2017-02-04) -======================================= - -No changes since RC 4. - -Changes in synapse v0.19.0-rc4 (2017-02-02) -=========================================== - -- Bump cache sizes for common membership queries (PR #1879) - -Changes in synapse v0.19.0-rc3 (2017-02-02) -=========================================== - -- Fix email push in pusher worker (PR #1875) -- Make presence.get\_new\_events a bit faster (PR #1876) -- Make /keys/changes a bit more performant (PR #1877) - -Changes in synapse v0.19.0-rc2 (2017-02-02) -=========================================== - -- Include newly joined users in /keys/changes API (PR #1872) - -Changes in synapse v0.19.0-rc1 (2017-02-02) -=========================================== - -Features: - -- Add support for specifying multiple bind addresses (PR #1709, #1712, #1795, #1835). Thanks to @kyrias! -- Add /account/3pid/delete endpoint (PR #1714) -- Add config option to configure the Riot URL used in notification emails (PR #1811). Thanks to @aperezdc! -- Add username and password config options for turn server (PR #1832). Thanks to @xsteadfastx! -- Implement device lists updates over federation (PR #1857, #1861, #1864) -- Implement /keys/changes (PR #1869, #1872) - -Changes: - -- Improve IPv6 support (PR #1696). Thanks to @kyrias and @glyph! -- Log which files we saved attachments to in the media\_repository (PR #1791) -- Linearize updates to membership via PUT /state/ to better handle multiple joins (PR #1787) -- Limit number of entries to prefill from cache on startup (PR #1792) -- Remove full\_twisted\_stacktraces option (PR #1802) -- Measure size of some caches by sum of the size of cached values (PR #1815) -- Measure metrics of string\_cache (PR #1821) -- Reduce logging verbosity (PR #1822, #1823, #1824) -- Don\'t clobber a displayname or avatar\_url if provided by an m.room.member event (PR #1852) -- Better handle 401/404 response for federation /send/ (PR #1866, #1871) - -Fixes: - -- Fix ability to change password to a non-ascii one (PR #1711) -- Fix push getting stuck due to looking at the wrong view of state (PR #1820) -- Fix email address comparison to be case insensitive (PR #1827) -- Fix occasional inconsistencies of room membership (PR #1836, #1840) - -Performance: - -- Don\'t block messages sending on bumping presence (PR #1789) -- Change device\_inbox stream index to include user (PR #1793) -- Optimise state resolution (PR #1818) -- Use DB cache of joined users for presence (PR #1862) -- Add an index to make membership queries faster (PR #1867) - -Changes in synapse v0.18.7 (2017-01-09) -======================================= - -No changes from v0.18.7-rc2 - -Changes in synapse v0.18.7-rc2 (2017-01-07) -=========================================== - -Bug fixes: - -- Fix error in rc1\'s discarding invalid inbound traffic logic that was incorrectly discarding missing events - -Changes in synapse v0.18.7-rc1 (2017-01-06) -=========================================== - -Bug fixes: - -- Fix error in \#PR 1764 to actually fix the nightmare \#1753 bug. -- Improve deadlock logging further -- Discard inbound federation traffic from invalid domains, to immunise against \#1753 - -Changes in synapse v0.18.6 (2017-01-06) -======================================= - -Bug fixes: - -- Fix bug when checking if a guest user is allowed to join a room (PR #1772) Thanks to Patrik Oldsberg for diagnosing and the fix! - -Changes in synapse v0.18.6-rc3 (2017-01-05) -=========================================== - -Bug fixes: - -- Fix bug where we failed to send ban events to the banned server (PR #1758) -- Fix bug where we sent event that didn\'t originate on this server to other servers (PR #1764) -- Fix bug where processing an event from a remote server took a long time because we were making long HTTP requests (PR #1765, PR #1744) - -Changes: - -- Improve logging for debugging deadlocks (PR #1766, PR #1767) - -Changes in synapse v0.18.6-rc2 (2016-12-30) -=========================================== - -Bug fixes: - -- Fix memory leak in twisted by initialising logging correctly (PR #1731) -- Fix bug where fetching missing events took an unacceptable amount of time in large rooms (PR #1734) - -Changes in synapse v0.18.6-rc1 (2016-12-29) -=========================================== - -Bug fixes: - -- Make sure that outbound connections are closed (PR #1725) - -Changes in synapse v0.18.5 (2016-12-16) -======================================= - -Bug fixes: - -- Fix federation /backfill returning events it shouldn\'t (PR #1700) -- Fix crash in url preview (PR #1701) - -Changes in synapse v0.18.5-rc3 (2016-12-13) -=========================================== - -Features: - -- Add support for E2E for guests (PR #1653) -- Add new API appservice specific public room list (PR #1676) -- Add new room membership APIs (PR #1680) - -Changes: - -- Enable guest access for private rooms by default (PR #653) -- Limit the number of events that can be created on a given room concurrently (PR #1620) -- Log the args that we have on UI auth completion (PR #1649) -- Stop generating refresh\_tokens (PR #1654) -- Stop putting a time caveat on access tokens (PR #1656) -- Remove unspecced GET endpoints for e2e keys (PR #1694) - -Bug fixes: - -- Fix handling of 500 and 429\'s over federation (PR #1650) -- Fix Content-Type header parsing (PR #1660) -- Fix error when previewing sites that include unicode, thanks to kyrias (PR #1664) -- Fix some cases where we drop read receipts (PR #1678) -- Fix bug where calls to `/sync` didn\'t correctly timeout (PR #1683) -- Fix bug where E2E key query would fail if a single remote host failed (PR #1686) - -Changes in synapse v0.18.5-rc2 (2016-11-24) -=========================================== - -Bug fixes: - -- Don\'t send old events over federation, fixes bug in -rc1. - -Changes in synapse v0.18.5-rc1 (2016-11-24) -=========================================== - -Features: - -- Implement \"event\_fields\" in filters (PR #1638) - -Changes: - -- Use external ldap auth pacakge (PR #1628) -- Split out federation transaction sending to a worker (PR #1635) -- Fail with a coherent error message if /sync?filter= is invalid (PR #1636) -- More efficient notif count queries (PR #1644) - -Changes in synapse v0.18.4 (2016-11-22) -======================================= - -Bug fixes: - -- Add workaround for buggy clients that the fail to register (PR #1632) - -Changes in synapse v0.18.4-rc1 (2016-11-14) -=========================================== - -Changes: - -- Various database efficiency improvements (PR #1188, #1192) -- Update default config to blacklist more internal IPs, thanks to Euan Kemp (PR #1198) -- Allow specifying duration in minutes in config, thanks to Daniel Dent (PR #1625) - -Bug fixes: - -- Fix media repo to set CORs headers on responses (PR #1190) -- Fix registration to not error on non-ascii passwords (PR #1191) -- Fix create event code to limit the number of prev\_events (PR #1615) -- Fix bug in transaction ID deduplication (PR #1624) - -Changes in synapse v0.18.3 (2016-11-08) -======================================= - -SECURITY UPDATE - -Explicitly require authentication when using LDAP3. This is the default on versions of `ldap3` above 1.0, but some distributions will package an older version. - -If you are using LDAP3 login and have a version of `ldap3` older than 1.0 it is **CRITICAL to updgrade**. - -Changes in synapse v0.18.2 (2016-11-01) -======================================= - -No changes since v0.18.2-rc5 - -Changes in synapse v0.18.2-rc5 (2016-10-28) -=========================================== - -Bug fixes: - -- Fix prometheus process metrics in worker processes (PR #1184) - -Changes in synapse v0.18.2-rc4 (2016-10-27) -=========================================== - -Bug fixes: - -- Fix `user_threepids` schema delta, which in some instances prevented startup after upgrade (PR #1183) - -Changes in synapse v0.18.2-rc3 (2016-10-27) -=========================================== - -Changes: - -- Allow clients to supply access tokens as headers (PR #1098) -- Clarify error codes for GET /filter/, thanks to Alexander Maznev (PR #1164) -- Make password reset email field case insensitive (PR #1170) -- Reduce redundant database work in email pusher (PR #1174) -- Allow configurable rate limiting per AS (PR #1175) -- Check whether to ratelimit sooner to avoid work (PR #1176) -- Standardise prometheus metrics (PR #1177) - -Bug fixes: - -- Fix incredibly slow back pagination query (PR #1178) -- Fix infinite typing bug (PR #1179) - -Changes in synapse v0.18.2-rc2 (2016-10-25) -=========================================== - -(This release did not include the changes advertised and was identical to RC1) - -Changes in synapse v0.18.2-rc1 (2016-10-17) -=========================================== - -Changes: - -- Remove redundant event\_auth index (PR #1113) -- Reduce DB hits for replication (PR #1141) -- Implement pluggable password auth (PR #1155) -- Remove rate limiting from app service senders and fix get\_or\_create\_user requester, thanks to Patrik Oldsberg (PR #1157) -- window.postmessage for Interactive Auth fallback (PR #1159) -- Use sys.executable instead of hardcoded python, thanks to Pedro Larroy (PR #1162) -- Add config option for adding additional TLS fingerprints (PR #1167) -- User-interactive auth on delete device (PR #1168) - -Bug fixes: - -- Fix not being allowed to set your own state\_key, thanks to Patrik Oldsberg (PR #1150) -- Fix interactive auth to return 401 from for incorrect password (PR #1160, #1166) -- Fix email push notifs being dropped (PR #1169) - -Changes in synapse v0.18.1 (2016-10-05) -======================================= - -No changes since v0.18.1-rc1 - -Changes in synapse v0.18.1-rc1 (2016-09-30) -=========================================== - -Features: - -- Add total\_room\_count\_estimate to `/publicRooms` (PR #1133) - -Changes: - -- Time out typing over federation (PR #1140) -- Restructure LDAP authentication (PR #1153) - -Bug fixes: - -- Fix 3pid invites when server is already in the room (PR #1136) -- Fix upgrading with SQLite taking lots of CPU for a few days after upgrade (PR #1144) -- Fix upgrading from very old database versions (PR #1145) -- Fix port script to work with recently added tables (PR #1146) - -Changes in synapse v0.18.0 (2016-09-19) -======================================= - -The release includes major changes to the state storage database schemas, which significantly reduce database size. Synapse will attempt to upgrade the current data in the background. Servers with large SQLite database may experience degradation of performance while this upgrade is in progress, therefore you may want to consider migrating to using Postgres before upgrading very large SQLite databases - -Changes: - -- Make public room search case insensitive (PR #1127) - -Bug fixes: - -- Fix and clean up publicRooms pagination (PR #1129) - -Changes in synapse v0.18.0-rc1 (2016-09-16) -=========================================== - -Features: - -- Add `only=highlight` on `/notifications` (PR #1081) -- Add server param to /publicRooms (PR #1082) -- Allow clients to ask for the whole of a single state event (PR #1094) -- Add is\_direct param to /createRoom (PR #1108) -- Add pagination support to publicRooms (PR #1121) -- Add very basic filter API to /publicRooms (PR #1126) -- Add basic direct to device messaging support for E2E (PR #1074, #1084, #1104, #1111) - -Changes: - -- Move to storing state\_groups\_state as deltas, greatly reducing DB size (PR #1065) -- Reduce amount of state pulled out of the DB during common requests (PR #1069) -- Allow PDF to be rendered from media repo (PR #1071) -- Reindex state\_groups\_state after pruning (PR #1085) -- Clobber EDUs in send queue (PR #1095) -- Conform better to the CAS protocol specification (PR #1100) -- Limit how often we ask for keys from dead servers (PR #1114) - -Bug fixes: - -- Fix /notifications API when used with `from` param (PR #1080) -- Fix backfill when cannot find an event. (PR #1107) - -Changes in synapse v0.17.3 (2016-09-09) -======================================= - -This release fixes a major bug that stopped servers from handling rooms with over 1000 members. - -Changes in synapse v0.17.2 (2016-09-08) -======================================= - -This release contains security bug fixes. Please upgrade. - -No changes since v0.17.2-rc1 - -Changes in synapse v0.17.2-rc1 (2016-09-05) -=========================================== - -Features: - -- Start adding store-and-forward direct-to-device messaging (PR #1046, #1050, #1062, #1066) - -Changes: - -- Avoid pulling the full state of a room out so often (PR #1047, #1049, #1063, #1068) -- Don\'t notify for online to online presence transitions. (PR #1054) -- Occasionally persist unpersisted presence updates (PR #1055) -- Allow application services to have an optional \'url\' (PR #1056) -- Clean up old sent transactions from DB (PR #1059) - -Bug fixes: - -- Fix None check in backfill (PR #1043) -- Fix membership changes to be idempotent (PR #1067) -- Fix bug in get\_pdu where it would sometimes return events with incorrect signature - -Changes in synapse v0.17.1 (2016-08-24) -======================================= - -Changes: - -- Delete old received\_transactions rows (PR #1038) -- Pass through user-supplied content in /join/\$room\_id (PR #1039) - -Bug fixes: - -- Fix bug with backfill (PR #1040) - -Changes in synapse v0.17.1-rc1 (2016-08-22) -=========================================== - -Features: - -- Add notification API (PR #1028) - -Changes: - -- Don\'t print stack traces when failing to get remote keys (PR #996) -- Various federation /event/ perf improvements (PR #998) -- Only process one local membership event per room at a time (PR #1005) -- Move default display name push rule (PR #1011, #1023) -- Fix up preview URL API. Add tests. (PR #1015) -- Set `Content-Security-Policy` on media repo (PR #1021) -- Make notify\_interested\_services faster (PR #1022) -- Add usage stats to prometheus monitoring (PR #1037) - -Bug fixes: - -- Fix token login (PR #993) -- Fix CAS login (PR #994, #995) -- Fix /sync to not clobber status\_msg (PR #997) -- Fix redacted state events to include prev\_content (PR #1003) -- Fix some bugs in the auth/ldap handler (PR #1007) -- Fix backfill request to limit URI length, so that remotes don\'t reject the requests due to path length limits (PR #1012) -- Fix AS push code to not send duplicate events (PR #1025) - -Changes in synapse v0.17.0 (2016-08-08) -======================================= - -This release contains significant security bug fixes regarding authenticating events received over federation. PLEASE UPGRADE. - -This release changes the LDAP configuration format in a backwards incompatible way, see PR #843 for details. - -Changes: - -- Add federation /version API (PR #990) -- Make psutil dependency optional (PR #992) - -Bug fixes: - -- Fix URL preview API to exclude HTML comments in description (PR #988) -- Fix error handling of remote joins (PR #991) - -Changes in synapse v0.17.0-rc4 (2016-08-05) -=========================================== - -Changes: - -- Change the way we summarize URLs when previewing (PR #973) -- Add new `/state_ids/` federation API (PR #979) -- Speed up processing of `/state/` response (PR #986) - -Bug fixes: - -- Fix event persistence when event has already been partially persisted (PR #975, #983, #985) -- Fix port script to also copy across backfilled events (PR #982) - -Changes in synapse v0.17.0-rc3 (2016-08-02) -=========================================== - -Changes: - -- Forbid non-ASes from registering users whose names begin with \'\_\' (PR #958) -- Add some basic admin API docs (PR #963) - -Bug fixes: - -- Send the correct host header when fetching keys (PR #941) -- Fix joining a room that has missing auth events (PR #964) -- Fix various push bugs (PR #966, #970) -- Fix adding emails on registration (PR #968) - -Changes in synapse v0.17.0-rc2 (2016-08-02) -=========================================== - -(This release did not include the changes advertised and was identical to RC1) - -Changes in synapse v0.17.0-rc1 (2016-07-28) -=========================================== - -This release changes the LDAP configuration format in a backwards incompatible way, see PR #843 for details. - -Features: - -- Add purge\_media\_cache admin API (PR #902) -- Add deactivate account admin API (PR #903) -- Add optional pepper to password hashing (PR #907, #910 by KentShikama) -- Add an admin option to shared secret registration (breaks backwards compat) (PR #909) -- Add purge local room history API (PR #911, #923, #924) -- Add requestToken endpoints (PR #915) -- Add an /account/deactivate endpoint (PR #921) -- Add filter param to /messages. Add \'contains\_url\' to filter. (PR #922) -- Add device\_id support to /login (PR #929) -- Add device\_id support to /v2/register flow. (PR #937, #942) -- Add GET /devices endpoint (PR #939, #944) -- Add GET /device/{deviceId} (PR #943) -- Add update and delete APIs for devices (PR #949) - -Changes: - -- Rewrite LDAP Authentication against ldap3 (PR #843 by mweinelt) -- Linearize some federation endpoints based on (origin, room\_id) (PR #879) -- Remove the legacy v0 content upload API. (PR #888) -- Use similar naming we use in email notifs for push (PR #894) -- Optionally include password hash in createUser endpoint (PR #905 by KentShikama) -- Use a query that postgresql optimises better for get\_events\_around (PR #906) -- Fall back to \'username\' if \'user\' is not given for appservice registration. (PR #927 by Half-Shot) -- Add metrics for psutil derived memory usage (PR #936) -- Record device\_id in client\_ips (PR #938) -- Send the correct host header when fetching keys (PR #941) -- Log the hostname the reCAPTCHA was completed on (PR #946) -- Make the device id on e2e key upload optional (PR #956) -- Add r0.2.0 to the \"supported versions\" list (PR #960) -- Don\'t include name of room for invites in push (PR #961) - -Bug fixes: - -- Fix substitution failure in mail template (PR #887) -- Put most recent 20 messages in email notif (PR #892) -- Ensure that the guest user is in the database when upgrading accounts (PR #914) -- Fix various edge cases in auth handling (PR #919) -- Fix 500 ISE when sending alias event without a state\_key (PR #925) -- Fix bug where we stored rejections in the state\_group, persist all rejections (PR #948) -- Fix lack of check of if the user is banned when handling 3pid invites (PR #952) -- Fix a couple of bugs in the transaction and keyring code (PR #954, #955) - -Changes in synapse v0.16.1-r1 (2016-07-08) -========================================== - -THIS IS A CRITICAL SECURITY UPDATE. - -This fixes a bug which allowed users\' accounts to be accessed by unauthorised users. - -Changes in synapse v0.16.1 (2016-06-20) -======================================= - -Bug fixes: - -- Fix assorted bugs in `/preview_url` (PR #872) -- Fix TypeError when setting unicode passwords (PR #873) - -Performance improvements: - -- Turn `use_frozen_events` off by default (PR #877) -- Disable responding with canonical json for federation (PR #878) - -Changes in synapse v0.16.1-rc1 (2016-06-15) -=========================================== - -Features: None - -Changes: - -- Log requester for `/publicRoom` endpoints when possible (PR #856) -- 502 on `/thumbnail` when can\'t connect to remote server (PR #862) -- Linearize fetching of gaps on incoming events (PR #871) - -Bugs fixes: - -- Fix bug where rooms where marked as published by default (PR #857) -- Fix bug where joining room with an event with invalid sender (PR #868) -- Fix bug where backfilled events were sent down sync streams (PR #869) -- Fix bug where outgoing connections could wedge indefinitely, causing push notifications to be unreliable (PR #870) - -Performance improvements: - -- Improve `/publicRooms` performance(PR #859) - -Changes in synapse v0.16.0 (2016-06-09) -======================================= - -NB: As of v0.14 all AS config files must have an ID field. - -Bug fixes: - -- Don\'t make rooms published by default (PR #857) - -Changes in synapse v0.16.0-rc2 (2016-06-08) -=========================================== - -Features: - -- Add configuration option for tuning GC via `gc.set_threshold` (PR #849) - -Changes: - -- Record metrics about GC (PR #771, #847, #852) -- Add metric counter for number of persisted events (PR #841) - -Bug fixes: - -- Fix \'From\' header in email notifications (PR #843) -- Fix presence where timeouts were not being fired for the first 8h after restarts (PR #842) -- Fix bug where synapse sent malformed transactions to AS\'s when retrying transactions (Commits 310197b, 8437906) - -Performance improvements: - -- Remove event fetching from DB threads (PR #835) -- Change the way we cache events (PR #836) -- Add events to cache when we persist them (PR #840) - -Changes in synapse v0.16.0-rc1 (2016-06-03) -=========================================== - -Version 0.15 was not released. See v0.15.0-rc1 below for additional changes. - -Features: - -- Add email notifications for missed messages (PR #759, #786, #799, #810, #815, #821) -- Add a `url_preview_ip_range_whitelist` config param (PR #760) -- Add /report endpoint (PR #762) -- Add basic ignore user API (PR #763) -- Add an openidish mechanism for proving that you own a given user\_id (PR #765) -- Allow clients to specify a server\_name to avoid \'No known servers\' (PR #794) -- Add secondary\_directory\_servers option to fetch room list from other servers (PR #808, #813) - -Changes: - -- Report per request metrics for all of the things using request\_handler (PR #756) -- Correctly handle `NULL` password hashes from the database (PR #775) -- Allow receipts for events we haven\'t seen in the db (PR #784) -- Make synctl read a cache factor from config file (PR #785) -- Increment badge count per missed convo, not per msg (PR #793) -- Special case m.room.third\_party\_invite event auth to match invites (PR #814) - -Bug fixes: - -- Fix typo in event\_auth servlet path (PR #757) -- Fix password reset (PR #758) - -Performance improvements: - -- Reduce database inserts when sending transactions (PR #767) -- Queue events by room for persistence (PR #768) -- Add cache to `get_user_by_id` (PR #772) -- Add and use `get_domain_from_id` (PR #773) -- Use tree cache for `get_linearized_receipts_for_room` (PR #779) -- Remove unused indices (PR #782) -- Add caches to `bulk_get_push_rules*` (PR #804) -- Cache `get_event_reference_hashes` (PR #806) -- Add `get_users_with_read_receipts_in_room` cache (PR #809) -- Use state to calculate `get_users_in_room` (PR #811) -- Load push rules in storage layer so that they get cached (PR #825) -- Make `get_joined_hosts_for_room` use get\_users\_in\_room (PR #828) -- Poke notifier on next reactor tick (PR #829) -- Change CacheMetrics to be quicker (PR #830) - -Changes in synapse v0.15.0-rc1 (2016-04-26) -=========================================== - -Features: - -- Add login support for Javascript Web Tokens, thanks to Niklas Riekenbrauck (PR #671,\#687) -- Add URL previewing support (PR #688) -- Add login support for LDAP, thanks to Christoph Witzany (PR #701) -- Add GET endpoint for pushers (PR #716) - -Changes: - -- Never notify for member events (PR #667) -- Deduplicate identical `/sync` requests (PR #668) -- Require user to have left room to forget room (PR #673) -- Use DNS cache if within TTL (PR #677) -- Let users see their own leave events (PR #699) -- Deduplicate membership changes (PR #700) -- Increase performance of pusher code (PR #705) -- Respond with error status 504 if failed to talk to remote server (PR #731) -- Increase search performance on postgres (PR #745) - -Bug fixes: - -- Fix bug where disabling all notifications still resulted in push (PR #678) -- Fix bug where users couldn\'t reject remote invites if remote refused (PR #691) -- Fix bug where synapse attempted to backfill from itself (PR #693) -- Fix bug where profile information was not correctly added when joining remote rooms (PR #703) -- Fix bug where register API required incorrect key name for AS registration (PR #727) - -Changes in synapse v0.14.0 (2016-03-30) -======================================= - -No changes from v0.14.0-rc2 - -Changes in synapse v0.14.0-rc2 (2016-03-23) -=========================================== - -Features: - -- Add published room list API (PR #657) - -Changes: - -- Change various caches to consume less memory (PR #656, #658, #660, #662, #663, #665) -- Allow rooms to be published without requiring an alias (PR #664) -- Intern common strings in caches to reduce memory footprint (\#666) - -Bug fixes: - -- Fix reject invites over federation (PR #646) -- Fix bug where registration was not idempotent (PR #649) -- Update aliases event after deleting aliases (PR #652) -- Fix unread notification count, which was sometimes wrong (PR #661) - -Changes in synapse v0.14.0-rc1 (2016-03-14) -=========================================== - -Features: - -- Add event\_id to response to state event PUT (PR #581) -- Allow guest users access to messages in rooms they have joined (PR #587) -- Add config for what state is included in a room invite (PR #598) -- Send the inviter\'s member event in room invite state (PR #607) -- Add error codes for malformed/bad JSON in /login (PR #608) -- Add support for changing the actions for default rules (PR #609) -- Add environment variable SYNAPSE\_CACHE\_FACTOR, default it to 0.1 (PR #612) -- Add ability for alias creators to delete aliases (PR #614) -- Add profile information to invites (PR #624) - -Changes: - -- Enforce user\_id exclusivity for AS registrations (PR #572) -- Make adding push rules idempotent (PR #587) -- Improve presence performance (PR #582, #586) -- Change presence semantics for `last_active_ago` (PR #582, #586) -- Don\'t allow `m.room.create` to be changed (PR #596) -- Add 800x600 to default list of valid thumbnail sizes (PR #616) -- Always include kicks and bans in full /sync (PR #625) -- Send history visibility on boundary changes (PR #626) -- Register endpoint now returns a refresh\_token (PR #637) - -Bug fixes: - -- Fix bug where we returned incorrect state in /sync (PR #573) -- Always return a JSON object from push rule API (PR #606) -- Fix bug where registering without a user id sometimes failed (PR #610) -- Report size of ExpiringCache in cache size metrics (PR #611) -- Fix rejection of invites to empty rooms (PR #615) -- Fix usage of `bcrypt` to not use `checkpw` (PR #619) -- Pin `pysaml2` dependency (PR #634) -- Fix bug in `/sync` where timeline order was incorrect for backfilled events (PR #635) - -Changes in synapse v0.13.3 (2016-02-11) -======================================= - -- Fix bug where `/sync` would occasionally return events in the wrong room. - -Changes in synapse v0.13.2 (2016-02-11) -======================================= - -- Fix bug where `/events` would fail to skip some events if there had been more events than the limit specified since the last request (PR #570) - -Changes in synapse v0.13.1 (2016-02-10) -======================================= - -- Bump matrix-angular-sdk (matrix web console) dependency to 0.6.8 to pull in the fix for SYWEB-361 so that the default client can display HTML messages again(!) - -Changes in synapse v0.13.0 (2016-02-10) -======================================= - -This version includes an upgrade of the schema, specifically adding an index to the `events` table. This may cause synapse to pause for several minutes the first time it is started after the upgrade. - -Changes: - -- Improve general performance (PR #540, #543. \#544, #54, #549, #567) -- Change guest user ids to be incrementing integers (PR #550) -- Improve performance of public room list API (PR #552) -- Change profile API to omit keys rather than return null (PR #557) -- Add `/media/r0` endpoint prefix, which is equivalent to `/media/v1/` (PR #595) - -Bug fixes: - -- Fix bug with upgrading guest accounts where it would fail if you opened the registration email on a different device (PR #547) -- Fix bug where unread count could be wrong (PR #568) - -Changes in synapse v0.12.1-rc1 (2016-01-29) -=========================================== - -Features: - -- Add unread notification counts in `/sync` (PR #456) -- Add support for inviting 3pids in `/createRoom` (PR #460) -- Add ability for guest accounts to upgrade (PR #462) -- Add `/versions` API (PR #468) -- Add `event` to `/context` API (PR #492) -- Add specific error code for invalid user names in `/register` (PR #499) -- Add support for push badge counts (PR #507) -- Add support for non-guest users to peek in rooms using `/events` (PR #510) - -Changes: - -- Change `/sync` so that guest users only get rooms they\'ve joined (PR #469) -- Change to require unbanning before other membership changes (PR #501) -- Change default push rules to notify for all messages (PR #486) -- Change default push rules to not notify on membership changes (PR #514) -- Change default push rules in one to one rooms to only notify for events that are messages (PR #529) -- Change `/sync` to reject requests with a `from` query param (PR #512) -- Change server manhole to use SSH rather than telnet (PR #473) -- Change server to require AS users to be registered before use (PR #487) -- Change server not to start when ASes are invalidly configured (PR #494) -- Change server to require ID and `as_token` to be unique for AS\'s (PR #496) -- Change maximum pagination limit to 1000 (PR #497) - -Bug fixes: - -- Fix bug where `/sync` didn\'t return when something under the leave key changed (PR #461) -- Fix bug where we returned smaller rather than larger than requested thumbnails when `method=crop` (PR #464) -- Fix thumbnails API to only return cropped thumbnails when asking for a cropped thumbnail (PR #475) -- Fix bug where we occasionally still logged access tokens (PR #477) -- Fix bug where `/events` would always return immediately for guest users (PR #480) -- Fix bug where `/sync` unexpectedly returned old left rooms (PR #481) -- Fix enabling and disabling push rules (PR #498) -- Fix bug where `/register` returned 500 when given unicode username (PR #513) - -Changes in synapse v0.12.0 (2016-01-04) -======================================= - -- Expose `/login` under `r0` (PR #459) - -Changes in synapse v0.12.0-rc3 (2015-12-23) -=========================================== - -- Allow guest accounts access to `/sync` (PR #455) -- Allow filters to include/exclude rooms at the room level rather than just from the components of the sync for each room. (PR #454) -- Include urls for room avatars in the response to `/publicRooms` (PR #453) -- Don\'t set a identicon as the avatar for a user when they register (PR #450) -- Add a `display_name` to third-party invites (PR #449) -- Send more information to the identity server for third-party invites so that it can send richer messages to the invitee (PR #446) -- Cache the responses to `/initialSync` for 5 minutes. If a client retries a request to `/initialSync` before the a response was computed to the first request then the same response is used for both requests (PR #457) -- Fix a bug where synapse would always request the signing keys of remote servers even when the key was cached locally (PR #452) -- Fix 500 when pagination search results (PR #447) -- Fix a bug where synapse was leaking raw email address in third-party invites (PR #448) - -Changes in synapse v0.12.0-rc2 (2015-12-14) -=========================================== - -- Add caches for whether rooms have been forgotten by a user (PR #434) -- Remove instructions to use `--process-dependency-link` since all of the dependencies of synapse are on PyPI (PR #436) -- Parallelise the processing of `/sync` requests (PR #437) -- Fix race updating presence in `/events` (PR #444) -- Fix bug back-populating search results (PR #441) -- Fix bug calculating state in `/sync` requests (PR #442) - -Changes in synapse v0.12.0-rc1 (2015-12-10) -=========================================== - -- Host the client APIs released as r0 by on paths prefixed by `/_matrix/client/r0`. (PR #430, PR #415, PR #400) -- Updates the client APIs to match r0 of the matrix specification. - - All APIs return events in the new event format, old APIs also include the fields needed to parse the event using the old format for compatibility. (PR #402) - - Search results are now given as a JSON array rather than a JSON object (PR #405) - - Miscellaneous changes to search (PR #403, PR #406, PR #412) - - Filter JSON objects may now be passed as query parameters to `/sync` (PR #431) - - Fix implementation of `/admin/whois` (PR #418) - - Only include the rooms that user has left in `/sync` if the client requests them in the filter (PR #423) - - Don\'t push for `m.room.message` by default (PR #411) - - Add API for setting per account user data (PR #392) - - Allow users to forget rooms (PR #385) -- Performance improvements and monitoring: - - Add per-request counters for CPU time spent on the main python thread. (PR #421, PR #420) - - Add per-request counters for time spent in the database (PR #429) - - Make state updates in the C+S API idempotent (PR #416) - - Only fire `user_joined_room` if the user has actually joined. (PR #410) - - Reuse a single http client, rather than creating new ones (PR #413) -- Fixed a bug upgrading from older versions of synapse on postgresql (PR #417) - -Changes in synapse v0.11.1 (2015-11-20) -======================================= - -- Add extra options to search API (PR #394) -- Fix bug where we did not correctly cap federation retry timers. This meant it could take several hours for servers to start talking to ressurected servers, even when they were receiving traffic from them (PR #393) -- Don\'t advertise login token flow unless CAS is enabled. This caused issues where some clients would always use the fallback API if they did not recognize all login flows (PR #391) -- Change /v2 sync API to rename `private_user_data` to `account_data` (PR #386) -- Change /v2 sync API to remove the `event_map` and rename keys in `rooms` object (PR #389) - -Changes in synapse v0.11.0-r2 (2015-11-19) -========================================== - -- Fix bug in database port script (PR #387) - -Changes in synapse v0.11.0-r1 (2015-11-18) -========================================== - -- Retry and fail federation requests more aggressively for requests that block client side requests (PR #384) - -Changes in synapse v0.11.0 (2015-11-17) -======================================= - -- Change CAS login API (PR #349) - -Changes in synapse v0.11.0-rc2 (2015-11-13) -=========================================== - -- Various changes to /sync API response format (PR #373) -- Fix regression when setting display name in newly joined room over federation (PR #368) -- Fix problem where /search was slow when using SQLite (PR #366) - -Changes in synapse v0.11.0-rc1 (2015-11-11) -=========================================== - -- Add Search API (PR #307, #324, #327, #336, #350, #359) -- Add \'archived\' state to v2 /sync API (PR #316) -- Add ability to reject invites (PR #317) -- Add config option to disable password login (PR #322) -- Add the login fallback API (PR #330) -- Add room context API (PR #334) -- Add room tagging support (PR #335) -- Update v2 /sync API to match spec (PR #305, #316, #321, #332, #337, #341) -- Change retry schedule for application services (PR #320) -- Change retry schedule for remote servers (PR #340) -- Fix bug where we hosted static content in the incorrect place (PR #329) -- Fix bug where we didn\'t increment retry interval for remote servers (PR #343) - -Changes in synapse v0.10.1-rc1 (2015-10-15) -=========================================== - -- Add support for CAS, thanks to Steven Hammerton (PR #295, #296) -- Add support for using macaroons for `access_token` (PR #256, #229) -- Add support for `m.room.canonical_alias` (PR #287) -- Add support for viewing the history of rooms that they have left. (PR #276, #294) -- Add support for refresh tokens (PR #240) -- Add flag on creation which disables federation of the room (PR #279) -- Add some room state to invites. (PR #275) -- Atomically persist events when joining a room over federation (PR #283) -- Change default history visibility for private rooms (PR #271) -- Allow users to redact their own sent events (PR #262) -- Use tox for tests (PR #247) -- Split up syutil into separate libraries (PR #243) - -Changes in synapse v0.10.0-r2 (2015-09-16) -========================================== - -- Fix bug where we always fetched remote server signing keys instead of using ones in our cache. -- Fix adding threepids to an existing account. -- Fix bug with invinting over federation where remote server was already in the room. (PR #281, SYN-392) - -Changes in synapse v0.10.0-r1 (2015-09-08) -========================================== - -- Fix bug with python packaging - -Changes in synapse v0.10.0 (2015-09-03) -======================================= - -No change from release candidate. - -Changes in synapse v0.10.0-rc6 (2015-09-02) -=========================================== - -- Remove some of the old database upgrade scripts. -- Fix database port script to work with newly created sqlite databases. - -Changes in synapse v0.10.0-rc5 (2015-08-27) -=========================================== - -- Fix bug that broke downloading files with ascii filenames across federation. - -Changes in synapse v0.10.0-rc4 (2015-08-27) -=========================================== - -- Allow UTF-8 filenames for upload. (PR #259) - -Changes in synapse v0.10.0-rc3 (2015-08-25) -=========================================== - -- Add `--keys-directory` config option to specify where files such as certs and signing keys should be stored in, when using `--generate-config` or `--generate-keys`. (PR #250) -- Allow `--config-path` to specify a directory, causing synapse to use all \*.yaml files in the directory as config files. (PR #249) -- Add `web_client_location` config option to specify static files to be hosted by synapse under `/_matrix/client`. (PR #245) -- Add helper utility to synapse to read and parse the config files and extract the value of a given key. For example: - - $ python -m synapse.config read server_name -c homeserver.yaml - localhost - - (PR #246) - -Changes in synapse v0.10.0-rc2 (2015-08-24) -=========================================== - -- Fix bug where we incorrectly populated the `event_forward_extremities` table, resulting in problems joining large remote rooms (e.g. `#matrix:matrix.org`) -- Reduce the number of times we wake up pushers by not listening for presence or typing events, reducing the CPU cost of each pusher. - -Changes in synapse v0.10.0-rc1 (2015-08-21) -=========================================== - -Also see v0.9.4-rc1 changelog, which has been amalgamated into this release. - -General: - -- Upgrade to Twisted 15 (PR #173) -- Add support for serving and fetching encryption keys over federation. (PR #208) -- Add support for logging in with email address (PR #234) -- Add support for new `m.room.canonical_alias` event. (PR #233) -- Change synapse to treat user IDs case insensitively during registration and login. (If two users already exist with case insensitive matching user ids, synapse will continue to require them to specify their user ids exactly.) -- Error if a user tries to register with an email already in use. (PR #211) -- Add extra and improve existing caches (PR #212, #219, #226, #228) -- Batch various storage request (PR #226, #228) -- Fix bug where we didn\'t correctly log the entity that triggered the request if the request came in via an application service (PR #230) -- Fix bug where we needlessly regenerated the full list of rooms an AS is interested in. (PR #232) -- Add support for AS\'s to use v2\_alpha registration API (PR #210) - -Configuration: - -- Add `--generate-keys` that will generate any missing cert and key files in the configuration files. This is equivalent to running `--generate-config` on an existing configuration file. (PR #220) -- `--generate-config` now no longer requires a `--server-name` parameter when used on existing configuration files. (PR #220) -- Add `--print-pidfile` flag that controls the printing of the pid to stdout of the demonised process. (PR #213) - -Media Repository: - -- Fix bug where we picked a lower resolution image than requested. (PR #205) -- Add support for specifying if a the media repository should dynamically thumbnail images or not. (PR #206) - -Metrics: - -- Add statistics from the reactor to the metrics API. (PR #224, #225) - -Demo Homeservers: - -- Fix starting the demo homeservers without rate-limiting enabled. (PR #182) -- Fix enabling registration on demo homeservers (PR #223) - -Changes in synapse v0.9.4-rc1 (2015-07-21) -========================================== - -General: - -- Add basic implementation of receipts. (SPEC-99) -- Add support for configuration presets in room creation API. (PR #203) -- Add auth event that limits the visibility of history for new users. (SPEC-134) -- Add SAML2 login/registration support. (PR #201. Thanks Muthu Subramanian!) -- Add client side key management APIs for end to end encryption. (PR #198) -- Change power level semantics so that you cannot kick, ban or change power levels of users that have equal or greater power level than you. (SYN-192) -- Improve performance by bulk inserting events where possible. (PR #193) -- Improve performance by bulk verifying signatures where possible. (PR #194) - -Configuration: - -- Add support for including TLS certificate chains. - -Media Repository: - -- Add Content-Disposition headers to content repository responses. (SYN-150) - -Changes in synapse v0.9.3 (2015-07-01) -====================================== - -No changes from v0.9.3 Release Candidate 1. - -Changes in synapse v0.9.3-rc1 (2015-06-23) -========================================== - -General: - -- Fix a memory leak in the notifier. (SYN-412) -- Improve performance of room initial sync. (SYN-418) -- General improvements to logging. -- Remove `access_token` query params from `INFO` level logging. - -Configuration: - -- Add support for specifying and configuring multiple listeners. (SYN-389) - -Application services: - -- Fix bug where synapse failed to send user queries to application services. - -Changes in synapse v0.9.2-r2 (2015-06-15) -========================================= - -Fix packaging so that schema delta python files get included in the package. - -Changes in synapse v0.9.2 (2015-06-12) -====================================== - -General: - -- Use ultrajson for json (de)serialisation when a canonical encoding is not required. Ultrajson is significantly faster than simplejson in certain circumstances. -- Use connection pools for outgoing HTTP connections. -- Process thumbnails on separate threads. - -Configuration: - -- Add option, `gzip_responses`, to disable HTTP response compression. - -Federation: - -- Improve resilience of backfill by ensuring we fetch any missing auth events. -- Improve performance of backfill and joining remote rooms by removing unnecessary computations. This included handling events we\'d previously handled as well as attempting to compute the current state for outliers. - -Changes in synapse v0.9.1 (2015-05-26) -====================================== - -General: - -- Add support for backfilling when a client paginates. This allows servers to request history for a room from remote servers when a client tries to paginate history the server does not have - SYN-36 -- Fix bug where you couldn\'t disable non-default pushrules - SYN-378 -- Fix `register_new_user` script - SYN-359 -- Improve performance of fetching events from the database, this improves both initialSync and sending of events. -- Improve performance of event streams, allowing synapse to handle more simultaneous connected clients. - -Federation: - -- Fix bug with existing backfill implementation where it returned the wrong selection of events in some circumstances. -- Improve performance of joining remote rooms. - -Configuration: - -- Add support for changing the bind host of the metrics listener via the `metrics_bind_host` option. - -Changes in synapse v0.9.0-r5 (2015-05-21) -========================================= - -- Add more database caches to reduce amount of work done for each pusher. This radically reduces CPU usage when multiple pushers are set up in the same room. - -Changes in synapse v0.9.0 (2015-05-07) -====================================== - -General: - -- Add support for using a PostgreSQL database instead of SQLite. See [docs/postgres.rst](docs/postgres.rst) for details. -- Add password change and reset APIs. See [Registration](https://github.com/matrix-org/matrix-doc/blob/master/specification/10_client_server_api.rst#registration) in the spec. -- Fix memory leak due to not releasing stale notifiers - SYN-339. -- Fix race in caches that occasionally caused some presence updates to be dropped - SYN-369. -- Check server name has not changed on restart. -- Add a sample systemd unit file and a logger configuration in contrib/systemd. Contributed Ivan Shapovalov. - -Federation: - -- Add key distribution mechanisms for fetching public keys of unavailable remote homeservers. See [Retrieving Server Keys](https://github.com/matrix-org/matrix-doc/blob/6f2698/specification/30_server_server_api.rst#retrieving-server-keys) in the spec. - -Configuration: - -- Add support for multiple config files. -- Add support for dictionaries in config files. -- Remove support for specifying config options on the command line, except for: - - `--daemonize` - Daemonize the homeserver. - - `--manhole` - Turn on the twisted telnet manhole service on the given port. - - `--database-path` - The path to a sqlite database to use. - - `--verbose` - The verbosity level. - - `--log-file` - File to log to. - - `--log-config` - Python logging config file. - - `--enable-registration` - Enable registration for new users. - -Application services: - -- Reliably retry sending of events from Synapse to application services, as per [Application Services](https://github.com/matrix-org/matrix-doc/blob/0c6bd9/specification/25_application_service_api.rst#home-server---application-service-api) spec. -- Application services can no longer register via the `/register` API, instead their configuration should be saved to a file and listed in the synapse `app_service_config_files` config option. The AS configuration file has the same format as the old `/register` request. See [docs/application\_services.rst](docs/application_services.rst) for more information. - -Changes in synapse v0.8.1 (2015-03-18) -====================================== - -- Disable registration by default. New users can be added using the command `register_new_matrix_user` or by enabling registration in the config. -- Add metrics to synapse. To enable metrics use config options `enable_metrics` and `metrics_port`. -- Fix bug where banning only kicked the user. - -Changes in synapse v0.8.0 (2015-03-06) -====================================== - -General: - -- Add support for registration fallback. This is a page hosted on the server which allows a user to register for an account, regardless of what client they are using (e.g. mobile devices). -- Added new default push rules and made them configurable by clients: - - Suppress all notice messages. - - Notify when invited to a new room. - - Notify for messages that don\'t match any rule. - - Notify on incoming call. - -Federation: - -- Added per host server side rate-limiting of incoming federation requests. -- Added a `/get_missing_events/` API to federation to reduce number of `/events/` requests. - -Configuration: - -- Added configuration option to disable registration: `disable_registration`. -- Added configuration option to change soft limit of number of open file descriptors: `soft_file_limit`. -- Make `tls_private_key_path` optional when running with `no_tls`. - -Application services: - -- Application services can now poll on the CS API `/events` for their events, by providing their application service `access_token`. -- Added exclusive namespace support to application services API. - -Changes in synapse v0.7.1 (2015-02-19) -====================================== - -- Initial alpha implementation of parts of the Application Services API. Including: - - AS Registration / Unregistration - - User Query API - - Room Alias Query API - - Push transport for receiving events. - - User/Alias namespace admin control -- Add cache when fetching events from remote servers to stop repeatedly fetching events with bad signatures. -- Respect the per remote server retry scheme when fetching both events and server keys to reduce the number of times we send requests to dead servers. -- Inform remote servers when the local server fails to handle a received event. -- Turn off python bytecode generation due to problems experienced when upgrading from previous versions. - -Changes in synapse v0.7.0 (2015-02-12) -====================================== - -- Add initial implementation of the query auth federation API, allowing servers to agree on whether an event should be allowed or rejected. -- Persist events we have rejected from federation, fixing the bug where servers would keep requesting the same events. -- Various federation performance improvements, including: - - Add in memory caches on queries such as: - - > - Computing the state of a room at a point in time, used for authorization on federation requests. - > - Fetching events from the database. - > - User\'s room membership, used for authorizing presence updates. - - - Upgraded JSON library to improve parsing and serialisation speeds. - -- Add default avatars to new user accounts using pydenticon library. -- Correctly time out federation requests. -- Retry federation requests against different servers. -- Add support for push and push rules. -- Add alpha versions of proposed new CSv2 APIs, including `/sync` API. - -Changes in synapse 0.6.1 (2015-01-07) -===================================== - -- Major optimizations to improve performance of initial sync and event sending in large rooms (by up to 10x) -- Media repository now includes a Content-Length header on media downloads. -- Improve quality of thumbnails by changing resizing algorithm. - -Changes in synapse 0.6.0 (2014-12-16) -===================================== - -- Add new API for media upload and download that supports thumbnailing. -- Replicate media uploads over multiple homeservers so media is always served to clients from their local homeserver. This obsoletes the \--content-addr parameter and confusion over accessing content directly from remote homeservers. -- Implement exponential backoff when retrying federation requests when sending to remote homeservers which are offline. -- Implement typing notifications. -- Fix bugs where we sent events with invalid signatures due to bugs where we incorrectly persisted events. -- Improve performance of database queries involving retrieving events. - -Changes in synapse 0.5.4a (2014-12-13) -====================================== - -- Fix bug while generating the error message when a file path specified in the config doesn\'t exist. - -Changes in synapse 0.5.4 (2014-12-03) -===================================== - -- Fix presence bug where some rooms did not display presence updates for remote users. -- Do not log SQL timing log lines when started with \"-v\" -- Fix potential memory leak. - -Changes in synapse 0.5.3c (2014-12-02) -====================================== - -- Change the default value for the content\_addr option to use the HTTP listener, as by default the HTTPS listener will be using a self-signed certificate. - -Changes in synapse 0.5.3 (2014-11-27) -===================================== - -- Fix bug that caused joining a remote room to fail if a single event was not signed correctly. -- Fix bug which caused servers to continuously try and fetch events from other servers. - -Changes in synapse 0.5.2 (2014-11-26) -===================================== - -Fix major bug that caused rooms to disappear from peoples initial sync. - -Changes in synapse 0.5.1 (2014-11-26) -===================================== - -See UPGRADES.rst for specific instructions on how to upgrade. - -- Fix bug where we served up an Event that did not match its signatures. -- Fix regression where we no longer correctly handled the case where a homeserver receives an event for a room it doesn\'t recognise (but is in.) - -Changes in synapse 0.5.0 (2014-11-19) -===================================== - -This release includes changes to the federation protocol and client-server API that is not backwards compatible. - -This release also changes the internal database schemas and so requires servers to drop their current history. See UPGRADES.rst for details. - -Homeserver: - -- Add authentication and authorization to the federation protocol. Events are now signed by their originating homeservers. -- Implement the new authorization model for rooms. -- Split out web client into a seperate repository: matrix-angular-sdk. -- Change the structure of PDUs. -- Fix bug where user could not join rooms via an alias containing 4-byte UTF-8 characters. -- Merge concept of PDUs and Events internally. -- Improve logging by adding request ids to log lines. -- Implement a very basic room initial sync API. -- Implement the new invite/join federation APIs. - -Webclient: - -- The webclient has been moved to a seperate repository. - -Changes in synapse 0.4.2 (2014-10-31) -===================================== - -Homeserver: - -- Fix bugs where we did not notify users of correct presence updates. -- Fix bug where we did not handle sub second event stream timeouts. - -Webclient: - -- Add ability to click on messages to see JSON. -- Add ability to redact messages. -- Add ability to view and edit all room state JSON. -- Handle incoming redactions. -- Improve feedback on errors. -- Fix bugs in mobile CSS. -- Fix bugs with desktop notifications. - -Changes in synapse 0.4.1 (2014-10-17) -===================================== - -Webclient: - -- Fix bug with display of timestamps. - -Changes in synpase 0.4.0 (2014-10-17) -===================================== - -This release includes changes to the federation protocol and client-server API that is not backwards compatible. - -The Matrix specification has been moved to a separate git repository: - -You will also need an updated syutil and config. See UPGRADES.rst. - -Homeserver: - -- Sign federation transactions to assert strong identity over federation. -- Rename timestamp keys in PDUs and events from \'ts\' and \'hsob\_ts\' to \'origin\_server\_ts\'. - -Changes in synapse 0.3.4 (2014-09-25) -===================================== - -This version adds support for using a TURN server. See docs/turn-howto.rst on how to set one up. - -Homeserver: - -- Add support for redaction of messages. -- Fix bug where inviting a user on a remote homeserver could take up to 20-30s. -- Implement a get current room state API. -- Add support specifying and retrieving turn server configuration. - -Webclient: - -- Add button to send messages to users from the home page. -- Add support for using TURN for VoIP calls. -- Show display name change messages. -- Fix bug where the client didn\'t get the state of a newly joined room until after it has been refreshed. -- Fix bugs with tab complete. -- Fix bug where holding down the down arrow caused chrome to chew 100% CPU. -- Fix bug where desktop notifications occasionally used \"Undefined\" as the display name. -- Fix more places where we sometimes saw room IDs incorrectly. -- Fix bug which caused lag when entering text in the text box. - -Changes in synapse 0.3.3 (2014-09-22) -===================================== - -Homeserver: - -- Fix bug where you continued to get events for rooms you had left. - -Webclient: - -- Add support for video calls with basic UI. -- Fix bug where one to one chats were named after your display name rather than the other person\'s. -- Fix bug which caused lag when typing in the textarea. -- Refuse to run on browsers we know won\'t work. -- Trigger pagination when joining new rooms. -- Fix bug where we sometimes didn\'t display invitations in recents. -- Automatically join room when accepting a VoIP call. -- Disable outgoing and reject incoming calls on browsers we don\'t support VoIP in. -- Don\'t display desktop notifications for messages in the room you are non-idle and speaking in. - -Changes in synapse 0.3.2 (2014-09-18) -===================================== - -Webclient: - -- Fix bug where an empty \"bing words\" list in old accounts didn\'t send notifications when it should have done. - -Changes in synapse 0.3.1 (2014-09-18) -===================================== - -This is a release to hotfix v0.3.0 to fix two regressions. - -Webclient: - -- Fix a regression where we sometimes displayed duplicate events. -- Fix a regression where we didn\'t immediately remove rooms you were banned in from the recents list. - -Changes in synapse 0.3.0 (2014-09-18) -===================================== - -See UPGRADE for information about changes to the client server API, including breaking backwards compatibility with VoIP calls and registration API. - -Homeserver: - -- When a user changes their displayname or avatar the server will now update all their join states to reflect this. -- The server now adds \"age\" key to events to indicate how old they are. This is clock independent, so at no point does any server or webclient have to assume their clock is in sync with everyone else. -- Fix bug where we didn\'t correctly pull in missing PDUs. -- Fix bug where prev\_content key wasn\'t always returned. -- Add support for password resets. - -Webclient: - -- Improve page content loading. -- Join/parts now trigger desktop notifications. -- Always show room aliases in the UI if one is present. -- No longer show user-count in the recents side panel. -- Add up & down arrow support to the text box for message sending to step through your sent history. -- Don\'t display notifications for our own messages. -- Emotes are now formatted correctly in desktop notifications. -- The recents list now differentiates between public & private rooms. -- Fix bug where when switching between rooms the pagination flickered before the view jumped to the bottom of the screen. -- Add bing word support. - -Registration API: - -- The registration API has been overhauled to function like the login API. In practice, this means registration requests must now include the following: \'type\':\'m.login.password\'. See UPGRADE for more information on this. -- The \'user\_id\' key has been renamed to \'user\' to better match the login API. -- There is an additional login type: \'m.login.email.identity\'. -- The command client and web client have been updated to reflect these changes. - -Changes in synapse 0.2.3 (2014-09-12) -===================================== - -Homeserver: - -- Fix bug where we stopped sending events to remote homeservers if a user from that homeserver left, even if there were some still in the room. -- Fix bugs in the state conflict resolution where it was incorrectly rejecting events. - -Webclient: - -- Display room names and topics. -- Allow setting/editing of room names and topics. -- Display information about rooms on the main page. -- Handle ban and kick events in real time. -- VoIP UI and reliability improvements. -- Add glare support for VoIP. -- Improvements to initial startup speed. -- Don\'t display duplicate join events. -- Local echo of messages. -- Differentiate sending and sent of local echo. -- Various minor bug fixes. - -Changes in synapse 0.2.2 (2014-09-06) -===================================== - -Homeserver: - -- When the server returns state events it now also includes the previous content. -- Add support for inviting people when creating a new room. -- Make the homeserver inform the room via m.room.aliases when a new alias is added for a room. -- Validate m.room.power\_level events. - -Webclient: - -- Add support for captchas on registration. -- Handle m.room.aliases events. -- Asynchronously send messages and show a local echo. -- Inform the UI when a message failed to send. -- Only autoscroll on receiving a new message if the user was already at the bottom of the screen. -- Add support for ban/kick reasons. - -Changes in synapse 0.2.1 (2014-09-03) -===================================== - -Homeserver: - -- Added support for signing up with a third party id. -- Add synctl scripts. -- Added rate limiting. -- Add option to change the external address the content repo uses. -- Presence bug fixes. - -Webclient: - -- Added support for signing up with a third party id. -- Added support for banning and kicking users. -- Added support for displaying and setting ops. -- Added support for room names. -- Fix bugs with room membership event display. - -Changes in synapse 0.2.0 (2014-09-02) -===================================== - -This update changes many configuration options, updates the database schema and mandates SSL for server-server connections. - -Homeserver: - -- Require SSL for server-server connections. -- Add SSL listener for client-server connections. -- Add ability to use config files. -- Add support for kicking/banning and power levels. -- Allow setting of room names and topics on creation. -- Change presence to include last seen time of the user. -- Change url path prefix to /\_matrix/\... -- Bug fixes to presence. - -Webclient: - -- Reskin the CSS for registration and login. -- Various improvements to rooms CSS. -- Support changes in client-server API. -- Bug fixes to VOIP UI. -- Various bug fixes to handling of changes to room member list. - -Changes in synapse 0.1.2 (2014-08-29) -===================================== - -Webclient: - -- Add basic call state UI for VoIP calls. - -Changes in synapse 0.1.1 (2014-08-29) -===================================== - -Homeserver: - -- Fix bug that caused the event stream to not notify some clients about changes. - -Changes in synapse 0.1.0 (2014-08-29) -===================================== - -Presence has been reenabled in this release. - -Homeserver: - -- Update client to server API, including: - - Use a more consistent url scheme. - - Provide more useful information in the initial sync api. -- Change the presence handling to be much more efficient. -- Change the presence server to server API to not require explicit polling of all users who share a room with a user. -- Fix races in the event streaming logic. - -Webclient: - -- Update to use new client to server API. -- Add basic VOIP support. -- Add idle timers that change your status to away. -- Add recent rooms column when viewing a room. -- Various network efficiency improvements. -- Add basic mobile browser support. -- Add a settings page. - -Changes in synapse 0.0.1 (2014-08-22) -===================================== - -Presence has been disabled in this release due to a bug that caused the homeserver to spam other remote homeservers. - -Homeserver: - -- Completely change the database schema to support generic event types. -- Improve presence reliability. -- Improve reliability of joining remote rooms. -- Fix bug where room join events were duplicated. -- Improve initial sync API to return more information to the client. -- Stop generating fake messages for room membership events. - -Webclient: - -- Add tab completion of names. -- Add ability to upload and send images. -- Add profile pages. -- Improve CSS layout of room. -- Disambiguate identical display names. -- Don\'t get remote users display names and avatars individually. -- Use the new initial sync API to reduce number of round trips to the homeserver. -- Change url scheme to use room aliases instead of room ids where known. -- Increase longpoll timeout. - -Changes in synapse 0.0.0 (2014-08-13) -===================================== - -- Initial alpha release From 28a64807b20edd0bbeeaf2bb835f4486934ada00 Mon Sep 17 00:00:00 2001 From: Brendan Abolivier Date: Tue, 29 Mar 2022 11:22:54 +0100 Subject: [PATCH 04/55] Linkify MSCs --- CHANGES.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index d2c49498e9..b1be4ea6b2 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -30,8 +30,8 @@ Improved Documentation Internal Changes ---------------- -- Rename `shared_rooms` to `mutual_rooms` (MSC2666), as per proposal changes. ([\#12036](https://github.com/matrix-org/synapse/issues/12036)) -- Remove check on `update_user_directory` for shared rooms handler (MSC2666), and update/expand documentation. ([\#12038](https://github.com/matrix-org/synapse/issues/12038)) +- Rename `shared_rooms` to `mutual_rooms` ([MSC2666](https://github.com/matrix-org/matrix-doc/pull/2666)), as per proposal changes. ([\#12036](https://github.com/matrix-org/synapse/issues/12036)) +- Remove check on `update_user_directory` for shared rooms handler ([MSC2666](https://github.com/matrix-org/matrix-doc/pull/2666)), and update/expand documentation. ([\#12038](https://github.com/matrix-org/synapse/issues/12038)) - Refactor `create_new_client_event` to use a new parameter, `state_event_ids`, which accurately describes the usage with [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) instead of abusing `auth_event_ids`. ([\#12083](https://github.com/matrix-org/synapse/issues/12083), [\#12304](https://github.com/matrix-org/synapse/issues/12304)) - Refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config flag `enable_registration_without_verification` is set. ([\#12091](https://github.com/matrix-org/synapse/issues/12091)) - Add tests for database transaction callbacks. ([\#12198](https://github.com/matrix-org/synapse/issues/12198)) From e9220adffc470e6da9dc156e48f4e789064a809c Mon Sep 17 00:00:00 2001 From: Brendan Abolivier Date: Tue, 29 Mar 2022 11:25:06 +0100 Subject: [PATCH 05/55] Link to pre-1.0 changelog --- CHANGES.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGES.md b/CHANGES.md index b1be4ea6b2..b00b2fd517 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -6440,3 +6440,5 @@ Internal Changes - Various improvements to debug logging. ([\#5353](https://github.com/matrix-org/synapse/issues/5353)) - Don't run CI build checks until sample config check has passed. ([\#5370](https://github.com/matrix-org/synapse/issues/5370)) - Automatically retry buildkite builds (max twice) when an agent is lost. ([\#5380](https://github.com/matrix-org/synapse/issues/5380)) + +**Changelogs for versions older than 1.0.0 can be found [here](CHANGES-pre-1.0.md) \ No newline at end of file From 512007f829a83f7ed6d387955a1329574e148782 Mon Sep 17 00:00:00 2001 From: Brendan Abolivier Date: Tue, 29 Mar 2022 11:25:49 +0100 Subject: [PATCH 06/55] Fix formatting --- CHANGES.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGES.md b/CHANGES.md index b00b2fd517..2cef96f869 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -6441,4 +6441,4 @@ Internal Changes - Don't run CI build checks until sample config check has passed. ([\#5370](https://github.com/matrix-org/synapse/issues/5370)) - Automatically retry buildkite builds (max twice) when an agent is lost. ([\#5380](https://github.com/matrix-org/synapse/issues/5380)) -**Changelogs for versions older than 1.0.0 can be found [here](CHANGES-pre-1.0.md) \ No newline at end of file +**Changelogs for versions older than 1.0.0 can be found [here](CHANGES-pre-1.0.md)** \ No newline at end of file From 8a5d6911403f4099d1875aeaebfac296aeaace9f Mon Sep 17 00:00:00 2001 From: Brendan Abolivier Date: Tue, 29 Mar 2022 11:28:39 +0100 Subject: [PATCH 07/55] Missing period --- CHANGES.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGES.md b/CHANGES.md index 2cef96f869..ee1eb9a582 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -6441,4 +6441,4 @@ Internal Changes - Don't run CI build checks until sample config check has passed. ([\#5370](https://github.com/matrix-org/synapse/issues/5370)) - Automatically retry buildkite builds (max twice) when an agent is lost. ([\#5380](https://github.com/matrix-org/synapse/issues/5380)) -**Changelogs for versions older than 1.0.0 can be found [here](CHANGES-pre-1.0.md)** \ No newline at end of file +**Changelogs for versions older than 1.0.0 can be found [here](CHANGES-pre-1.0.md).** \ No newline at end of file From 013f3f5e44278c7894e48ccd179ad4bca63bd857 Mon Sep 17 00:00:00 2001 From: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Date: Tue, 29 Mar 2022 14:12:41 +0100 Subject: [PATCH 08/55] Move warning of open registration to v1.56 upgrade notes (#12322) --- CHANGES.md | 13 ++++++++++++- changelog.d/12322.misc | 1 + docs/upgrade.md | 11 +++++------ 3 files changed, 18 insertions(+), 7 deletions(-) create mode 100644 changelog.d/12322.misc diff --git a/CHANGES.md b/CHANGES.md index ee1eb9a582..a9ac4a5d3d 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,6 +1,17 @@ Synapse 1.56.0rc1 (2022-03-29) ============================== +Synapse will now refuse to start up if open registration is enabled, in order to help mitigate +abuse across the federation. If you would like +to provide registration to anyone, consider adding [email](https://github.com/matrix-org/synapse/blob/8a519f8abc6de772167c2cca101d22ee2052fafc/docs/sample_config.yaml#L1285), +[recaptcha](https://matrix-org.github.io/synapse/v1.56/CAPTCHA_SETUP.html) +or [token-based](https://matrix-org.github.io/synapse/v1.56/usage/administration/admin_api/registration_tokens.html) verification +in order to prevent automated registration from bad actors. + +This check can be disabled by setting the `enable_registration_without_verification` option in your +homeserver configuration file to `true`. More details are available in the +[upgrade notes](https://matrix-org.github.io/synapse/v1.56/upgrade.html#open-registration-without-verification-is-now-disabled-by-default). + Features -------- @@ -33,7 +44,7 @@ Internal Changes - Rename `shared_rooms` to `mutual_rooms` ([MSC2666](https://github.com/matrix-org/matrix-doc/pull/2666)), as per proposal changes. ([\#12036](https://github.com/matrix-org/synapse/issues/12036)) - Remove check on `update_user_directory` for shared rooms handler ([MSC2666](https://github.com/matrix-org/matrix-doc/pull/2666)), and update/expand documentation. ([\#12038](https://github.com/matrix-org/synapse/issues/12038)) - Refactor `create_new_client_event` to use a new parameter, `state_event_ids`, which accurately describes the usage with [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) instead of abusing `auth_event_ids`. ([\#12083](https://github.com/matrix-org/synapse/issues/12083), [\#12304](https://github.com/matrix-org/synapse/issues/12304)) -- Refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config flag `enable_registration_without_verification` is set. ([\#12091](https://github.com/matrix-org/synapse/issues/12091)) +- Refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config flag `enable_registration_without_verification` is set to `true`. ([\#12091](https://github.com/matrix-org/synapse/issues/12091)) - Add tests for database transaction callbacks. ([\#12198](https://github.com/matrix-org/synapse/issues/12198)) - Handle cancellation in `DatabasePool.runInteraction`. ([\#12199](https://github.com/matrix-org/synapse/issues/12199)) - Add missing type hints for cache storage. ([\#12216](https://github.com/matrix-org/synapse/issues/12216)) diff --git a/changelog.d/12322.misc b/changelog.d/12322.misc new file mode 100644 index 0000000000..fd595df81c --- /dev/null +++ b/changelog.d/12322.misc @@ -0,0 +1 @@ +Refuse to start if registration is enabled without email, captcha, or token-based verification unless new config flag `enable_registration_without_verification` is set to `true`. \ No newline at end of file diff --git a/docs/upgrade.md b/docs/upgrade.md index 062e823333..10d81b949c 100644 --- a/docs/upgrade.md +++ b/docs/upgrade.md @@ -87,6 +87,11 @@ process, for example: # Upgrading to v1.56.0 +## Open registration without verification is now disabled by default + +Synapse will refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config +flag `enable_registration_without_verification` is set to "true". + ## Groups/communities feature has been deprecated The non-standard groups/communities feature in Synapse has been deprecated and will @@ -108,12 +113,6 @@ for more information and instructions on how to fix a database with incorrect va # Upgrading to v1.55.0 -## Open registration without verification is now disabled by default - -Synapse will refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config -flag `enable_registration_without_verification` is set to "true". - - ## `synctl` script has been moved The `synctl` script From 4d693f9b7964b8be055e83f2154d357ecac6b5f3 Mon Sep 17 00:00:00 2001 From: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Date: Tue, 29 Mar 2022 14:32:50 +0100 Subject: [PATCH 09/55] Add note in changelog of non-C type locale causing Synapse to refuse to start (#12323) --- CHANGES.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/CHANGES.md b/CHANGES.md index a9ac4a5d3d..de40921bff 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -12,6 +12,11 @@ This check can be disabled by setting the `enable_registration_without_verificat homeserver configuration file to `true`. More details are available in the [upgrade notes](https://matrix-org.github.io/synapse/v1.56/upgrade.html#open-registration-without-verification-is-now-disabled-by-default). +Synapse will additionally now refuse to start when using PostgreSQL with a non-`C` values for `COLLATE` and `CTYPE`, unless +the config flag `allow_unsafe_locale`, found in the database section of the configuration file, is set to `true`. See the +[upgrade notes](https://matrix-org.github.io/synapse/v1.56/upgrade#change-in-behaviour-for-postgresql-databases-with-unsafe-locale) +for details. + Features -------- From 11df4ec6c2178bdd25652644a94c213c3064d693 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Thu, 31 Mar 2022 13:38:09 -0400 Subject: [PATCH 10/55] Add more type hints to the main state store. (#12267) --- changelog.d/12267.misc | 1 + mypy.ini | 1 - synapse/storage/databases/main/state.py | 18 +++++++++++------- synapse/util/caches/__init__.py | 6 ++++-- 4 files changed, 16 insertions(+), 10 deletions(-) create mode 100644 changelog.d/12267.misc diff --git a/changelog.d/12267.misc b/changelog.d/12267.misc new file mode 100644 index 0000000000..e43844d44a --- /dev/null +++ b/changelog.d/12267.misc @@ -0,0 +1 @@ +Add missing type hints for storage. diff --git a/mypy.ini b/mypy.ini index cfe7e77c21..d371886a8d 100644 --- a/mypy.ini +++ b/mypy.ini @@ -43,7 +43,6 @@ exclude = (?x) |synapse/storage/databases/main/event_federation.py |synapse/storage/databases/main/push_rule.py |synapse/storage/databases/main/roommember.py - |synapse/storage/databases/main/state.py |synapse/storage/schema/ |tests/api/test_auth.py diff --git a/synapse/storage/databases/main/state.py b/synapse/storage/databases/main/state.py index 28460fd364..6e99a88a80 100644 --- a/synapse/storage/databases/main/state.py +++ b/synapse/storage/databases/main/state.py @@ -12,9 +12,10 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -import collections.abc import logging -from typing import TYPE_CHECKING, Collection, Iterable, Optional, Set, Tuple +from typing import TYPE_CHECKING, Dict, Iterable, Optional, Set, Tuple + +from frozendict import frozendict from synapse.api.constants import EventTypes, Membership from synapse.api.errors import NotFoundError, UnsupportedRoomVersionError @@ -29,7 +30,7 @@ from synapse.storage.database import ( from synapse.storage.databases.main.events_worker import EventsWorkerStore from synapse.storage.databases.main.roommember import RoomMemberWorkerStore from synapse.storage.state import StateFilter -from synapse.types import JsonDict, StateMap +from synapse.types import JsonDict, JsonMapping, StateMap from synapse.util.caches import intern_string from synapse.util.caches.descriptors import cached, cachedList @@ -132,7 +133,7 @@ class StateGroupWorkerStore(EventsWorkerStore, SQLBaseStore): return room_version - async def get_room_predecessor(self, room_id: str) -> Optional[dict]: + async def get_room_predecessor(self, room_id: str) -> Optional[JsonMapping]: """Get the predecessor of an upgraded room if it exists. Otherwise return None. @@ -158,9 +159,10 @@ class StateGroupWorkerStore(EventsWorkerStore, SQLBaseStore): predecessor = create_event.content.get("predecessor", None) # Ensure the key is a dictionary - if not isinstance(predecessor, collections.abc.Mapping): + if not isinstance(predecessor, (dict, frozendict)): return None + # The keys must be strings since the data is JSON. return predecessor async def get_create_event_for_room(self, room_id: str) -> EventBase: @@ -306,7 +308,9 @@ class StateGroupWorkerStore(EventsWorkerStore, SQLBaseStore): list_name="event_ids", num_args=1, ) - async def _get_state_group_for_events(self, event_ids: Collection[str]) -> JsonDict: + async def _get_state_group_for_events( + self, event_ids: Iterable[str] + ) -> Dict[str, int]: """Returns mapping event_id -> state_group""" rows = await self.db_pool.simple_select_many_batch( table="event_to_state_groups", @@ -521,7 +525,7 @@ class MainStateBackgroundUpdateStore(RoomMemberWorkerStore): ) for user_id in potentially_left_users - joined_users: - await self.mark_remote_user_device_list_as_unsubscribed(user_id) + await self.mark_remote_user_device_list_as_unsubscribed(user_id) # type: ignore[attr-defined] return batch_size diff --git a/synapse/util/caches/__init__.py b/synapse/util/caches/__init__.py index 1cbc180eda..42f6abb5e1 100644 --- a/synapse/util/caches/__init__.py +++ b/synapse/util/caches/__init__.py @@ -17,7 +17,7 @@ import logging import typing from enum import Enum, auto from sys import intern -from typing import Any, Callable, Dict, List, Optional, Sized +from typing import Any, Callable, Dict, List, Optional, Sized, TypeVar import attr from prometheus_client.core import Gauge @@ -195,8 +195,10 @@ KNOWN_KEYS = { ) } +T = TypeVar("T", Optional[str], str) -def intern_string(string: Optional[str]) -> Optional[str]: + +def intern_string(string: T) -> T: """Takes a (potentially) unicode string and interns it if it's ascii""" if string is None: return None From 6927d8725430356880e212edf0c61bf32cb071c3 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Thu, 31 Mar 2022 18:39:34 +0100 Subject: [PATCH 11/55] Handle outliers in `/federation/v1/event` (#12332) The intention here is to avoid doing state lookups for outliers in `/_matrix/federation/v1/event`. Unfortunately that's expanded into something of a rewrite of `filter_events_for_server`, which ended up trying to do that operation in a couple of places. --- changelog.d/12332.misc | 1 + synapse/visibility.py | 222 +++++++++++++++++++++------------------ tests/test_visibility.py | 53 ++++++++++ 3 files changed, 176 insertions(+), 100 deletions(-) create mode 100644 changelog.d/12332.misc diff --git a/changelog.d/12332.misc b/changelog.d/12332.misc new file mode 100644 index 0000000000..9f333e718a --- /dev/null +++ b/changelog.d/12332.misc @@ -0,0 +1 @@ +Avoid trying to calculate the state at outlier events. diff --git a/synapse/visibility.py b/synapse/visibility.py index 49519eb8f5..250f073597 100644 --- a/synapse/visibility.py +++ b/synapse/visibility.py @@ -1,4 +1,5 @@ # Copyright 2014 - 2016 OpenMarket Ltd +# Copyright (C) The Matrix.org Foundation C.I.C. 2022 # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -12,7 +13,9 @@ # See the License for the specific language governing permissions and # limitations under the License. import logging -from typing import Dict, FrozenSet, List, Optional +from typing import Collection, Dict, FrozenSet, List, Optional, Tuple + +from typing_extensions import Final from synapse.api.constants import EventTypes, HistoryVisibility, Membership from synapse.events import EventBase @@ -40,6 +43,8 @@ MEMBERSHIP_PRIORITY = ( Membership.BAN, ) +_HISTORY_VIS_KEY: Final[Tuple[str, str]] = (EventTypes.RoomHistoryVisibility, "") + async def filter_events_for_client( storage: Storage, @@ -74,7 +79,7 @@ async def filter_events_for_client( # to clients. events = [e for e in events if not e.internal_metadata.is_soft_failed()] - types = ((EventTypes.RoomHistoryVisibility, ""), (EventTypes.Member, user_id)) + types = (_HISTORY_VIS_KEY, (EventTypes.Member, user_id)) # we exclude outliers at this point, and then handle them separately later event_id_to_state = await storage.state.get_state_for_events( @@ -157,7 +162,7 @@ async def filter_events_for_client( state = event_id_to_state[event.event_id] # get the room_visibility at the time of the event. - visibility_event = state.get((EventTypes.RoomHistoryVisibility, ""), None) + visibility_event = state.get(_HISTORY_VIS_KEY, None) if visibility_event: visibility = visibility_event.content.get( "history_visibility", HistoryVisibility.SHARED @@ -293,67 +298,28 @@ async def filter_events_for_server( return True return False - def check_event_is_visible(event: EventBase, state: StateMap[EventBase]) -> bool: - history = state.get((EventTypes.RoomHistoryVisibility, ""), None) - if history: - visibility = history.content.get( - "history_visibility", HistoryVisibility.SHARED - ) - if visibility in [HistoryVisibility.INVITED, HistoryVisibility.JOINED]: - # We now loop through all state events looking for - # membership states for the requesting server to determine - # if the server is either in the room or has been invited - # into the room. - for ev in state.values(): - if ev.type != EventTypes.Member: - continue - try: - domain = get_domain_from_id(ev.state_key) - except Exception: - continue + def check_event_is_visible( + visibility: str, memberships: StateMap[EventBase] + ) -> bool: + if visibility not in (HistoryVisibility.INVITED, HistoryVisibility.JOINED): + return True - if domain != server_name: - continue + # We now loop through all membership events looking for + # membership states for the requesting server to determine + # if the server is either in the room or has been invited + # into the room. + for ev in memberships.values(): + assert get_domain_from_id(ev.state_key) == server_name - memtype = ev.membership - if memtype == Membership.JOIN: - return True - elif memtype == Membership.INVITE: - if visibility == HistoryVisibility.INVITED: - return True - else: - # server has no users in the room: redact - return False + memtype = ev.membership + if memtype == Membership.JOIN: + return True + elif memtype == Membership.INVITE: + if visibility == HistoryVisibility.INVITED: + return True - return True - - # Lets check to see if all the events have a history visibility - # of "shared" or "world_readable". If that's the case then we don't - # need to check membership (as we know the server is in the room). - event_to_state_ids = await storage.state.get_state_ids_for_events( - frozenset(e.event_id for e in events), - state_filter=StateFilter.from_types( - types=((EventTypes.RoomHistoryVisibility, ""),) - ), - ) - - visibility_ids = set() - for sids in event_to_state_ids.values(): - hist = sids.get((EventTypes.RoomHistoryVisibility, "")) - if hist: - visibility_ids.add(hist) - - # If we failed to find any history visibility events then the default - # is "shared" visibility. - if not visibility_ids: - all_open = True - else: - event_map = await storage.main.get_events(visibility_ids) - all_open = all( - e.content.get("history_visibility") - in (None, HistoryVisibility.SHARED, HistoryVisibility.WORLD_READABLE) - for e in event_map.values() - ) + # server has no users in the room: redact + return False if not check_history_visibility_only: erased_senders = await storage.main.are_users_erased(e.sender for e in events) @@ -362,34 +328,100 @@ async def filter_events_for_server( # to no users having been erased. erased_senders = {} - if all_open: - # all the history_visibility state affecting these events is open, so - # we don't need to filter by membership state. We *do* need to check - # for user erasure, though. - if erased_senders: - to_return = [] - for e in events: - if not is_sender_erased(e, erased_senders): - to_return.append(e) - elif redact: - to_return.append(prune_event(e)) + # Let's check to see if all the events have a history visibility + # of "shared" or "world_readable". If that's the case then we don't + # need to check membership (as we know the server is in the room). + event_to_history_vis = await _event_to_history_vis(storage, events) - return to_return + # for any with restricted vis, we also need the memberships + event_to_memberships = await _event_to_memberships( + storage, + [ + e + for e in events + if event_to_history_vis[e.event_id] + not in (HistoryVisibility.SHARED, HistoryVisibility.WORLD_READABLE) + ], + server_name, + ) - # If there are no erased users then we can just return the given list - # of events without having to copy it. - return events + to_return = [] + for e in events: + erased = is_sender_erased(e, erased_senders) + visible = check_event_is_visible( + event_to_history_vis[e.event_id], event_to_memberships.get(e.event_id, {}) + ) + if visible and not erased: + to_return.append(e) + elif redact: + to_return.append(prune_event(e)) - # Ok, so we're dealing with events that have non-trivial visibility - # rules, so we need to also get the memberships of the room. + return to_return - # first, for each event we're wanting to return, get the event_ids - # of the history vis and membership state at those events. + +async def _event_to_history_vis( + storage: Storage, events: Collection[EventBase] +) -> Dict[str, str]: + """Get the history visibility at each of the given events + + Returns a map from event id to history_visibility setting + """ + + # outliers get special treatment here. We don't have the state at that point in the + # room (and attempting to look it up will raise an exception), so all we can really + # do is assume that the requesting server is allowed to see the event. That's + # equivalent to there not being a history_visibility event, so we just exclude + # any outliers from the query. + event_to_state_ids = await storage.state.get_state_ids_for_events( + frozenset(e.event_id for e in events if not e.internal_metadata.is_outlier()), + state_filter=StateFilter.from_types(types=(_HISTORY_VIS_KEY,)), + ) + + visibility_ids = { + vis_event_id + for vis_event_id in ( + state_ids.get(_HISTORY_VIS_KEY) for state_ids in event_to_state_ids.values() + ) + if vis_event_id + } + vis_events = await storage.main.get_events(visibility_ids) + + result: Dict[str, str] = {} + for event in events: + vis = HistoryVisibility.SHARED + state_ids = event_to_state_ids.get(event.event_id) + + # if we didn't find any state for this event, it's an outlier, and we assume + # it's open + visibility_id = None + if state_ids: + visibility_id = state_ids.get(_HISTORY_VIS_KEY) + + if visibility_id: + vis_event = vis_events[visibility_id] + vis = vis_event.content.get("history_visibility", HistoryVisibility.SHARED) + assert isinstance(vis, str) + + result[event.event_id] = vis + return result + + +async def _event_to_memberships( + storage: Storage, events: Collection[EventBase], server_name: str +) -> Dict[str, StateMap[EventBase]]: + """Get the remote membership list at each of the given events + + Returns a map from event id to state map, which will contain only membership events + for the given server. + """ + + if not events: + return {} + + # for each event, get the event_ids of the membership state at those events. event_to_state_ids = await storage.state.get_state_ids_for_events( frozenset(e.event_id for e in events), - state_filter=StateFilter.from_types( - types=((EventTypes.RoomHistoryVisibility, ""), (EventTypes.Member, None)) - ), + state_filter=StateFilter.from_types(types=((EventTypes.Member, None),)), ) # We only want to pull out member events that correspond to the @@ -405,10 +437,7 @@ async def filter_events_for_server( for key, event_id in key_to_eid.items() } - def include(typ, state_key): - if typ != EventTypes.Member: - return True - + def include(state_key: str) -> bool: # we avoid using get_domain_from_id here for efficiency. idx = state_key.find(":") if idx == -1: @@ -416,10 +445,14 @@ async def filter_events_for_server( return state_key[idx + 1 :] == server_name event_map = await storage.main.get_events( - [e_id for e_id, key in event_id_to_state_key.items() if include(key[0], key[1])] + [ + e_id + for e_id, (_, state_key) in event_id_to_state_key.items() + if include(state_key) + ] ) - event_to_state = { + return { e_id: { key: event_map[inner_e_id] for key, inner_e_id in key_to_eid.items() @@ -427,14 +460,3 @@ async def filter_events_for_server( } for e_id, key_to_eid in event_to_state_ids.items() } - - to_return = [] - for e in events: - erased = is_sender_erased(e, erased_senders) - visible = check_event_is_visible(e, event_to_state[e.event_id]) - if visible and not erased: - to_return.append(e) - elif redact: - to_return.append(prune_event(e)) - - return to_return diff --git a/tests/test_visibility.py b/tests/test_visibility.py index 532e3fe9cd..a02fd4f79a 100644 --- a/tests/test_visibility.py +++ b/tests/test_visibility.py @@ -17,6 +17,7 @@ from unittest.mock import patch from synapse.api.room_versions import RoomVersions from synapse.events import EventBase, make_event_from_dict +from synapse.events.snapshot import EventContext from synapse.types import JsonDict, create_requester from synapse.visibility import filter_events_for_client, filter_events_for_server @@ -73,6 +74,39 @@ class FilterEventsForServerTestCase(unittest.HomeserverTestCase): self.assertEqual(events_to_filter[i].event_id, filtered[i].event_id) self.assertEqual(filtered[i].content["a"], "b") + def test_filter_outlier(self) -> None: + # outlier events must be returned, for the good of the collective federation + self.get_success(self._inject_room_member("@resident:remote_hs")) + self.get_success(self._inject_visibility("@resident:remote_hs", "joined")) + + outlier = self.get_success(self._inject_outlier()) + self.assertEqual( + self.get_success( + filter_events_for_server(self.storage, "remote_hs", [outlier]) + ), + [outlier], + ) + + # it should also work when there are other events in the list + evt = self.get_success(self._inject_message("@unerased:local_hs")) + + filtered = self.get_success( + filter_events_for_server(self.storage, "remote_hs", [outlier, evt]) + ) + self.assertEqual(len(filtered), 2, f"expected 2 results, got: {filtered}") + self.assertEqual(filtered[0], outlier) + self.assertEqual(filtered[1].event_id, evt.event_id) + self.assertEqual(filtered[1].content, evt.content) + + # ... but other servers should only be able to see the outlier (the other should + # be redacted) + filtered = self.get_success( + filter_events_for_server(self.storage, "other_server", [outlier, evt]) + ) + self.assertEqual(filtered[0], outlier) + self.assertEqual(filtered[1].event_id, evt.event_id) + self.assertNotIn("body", filtered[1].content) + def test_erased_user(self) -> None: # 4 message events, from erased and unerased users, with a membership # change in the middle of them. @@ -187,6 +221,25 @@ class FilterEventsForServerTestCase(unittest.HomeserverTestCase): self.get_success(self.storage.persistence.persist_event(event, context)) return event + def _inject_outlier(self) -> EventBase: + builder = self.event_builder_factory.for_room_version( + RoomVersions.V1, + { + "type": "m.room.member", + "sender": "@test:user", + "state_key": "@test:user", + "room_id": TEST_ROOM_ID, + "content": {"membership": "join"}, + }, + ) + + event = self.get_success(builder.build(prev_event_ids=[], auth_event_ids=[])) + event.internal_metadata.outlier = True + self.get_success( + self.storage.persistence.persist_event(event, EventContext.for_outlier()) + ) + return event + class FilterEventsForClientTestCase(unittest.FederatingHomeserverTestCase): def test_out_of_band_invite_rejection(self): From bebf994ee804ef63ce16801c6694713fcd685320 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Thu, 31 Mar 2022 15:05:13 -0400 Subject: [PATCH 12/55] Move MSC2654 support behind an experimental configuration flag. (#12295) To match the current thinking on disabling experimental features by default. --- changelog.d/12295.misc | 1 + synapse/config/experimental.py | 3 +++ synapse/rest/client/sync.py | 4 +++- tests/rest/client/test_sync.py | 5 +++++ 4 files changed, 12 insertions(+), 1 deletion(-) create mode 100644 changelog.d/12295.misc diff --git a/changelog.d/12295.misc b/changelog.d/12295.misc new file mode 100644 index 0000000000..9c34e16909 --- /dev/null +++ b/changelog.d/12295.misc @@ -0,0 +1 @@ +Move [MSC2654](https://github.com/matrix-org/matrix-doc/pull/2654) support behind an experimental configuration flag. diff --git a/synapse/config/experimental.py b/synapse/config/experimental.py index d6bb1f752b..43db5fcdd9 100644 --- a/synapse/config/experimental.py +++ b/synapse/config/experimental.py @@ -78,3 +78,6 @@ class ExperimentalConfig(Config): # The deprecated groups feature. self.groups_enabled: bool = experimental.get("groups_enabled", True) + + # MSC2654: Unread counts + self.msc2654_enabled: bool = experimental.get("msc2654_enabled", False) diff --git a/synapse/rest/client/sync.py b/synapse/rest/client/sync.py index 53c385a86c..0bf32f873b 100644 --- a/synapse/rest/client/sync.py +++ b/synapse/rest/client/sync.py @@ -99,6 +99,7 @@ class SyncRestServlet(RestServlet): self.presence_handler = hs.get_presence_handler() self._server_notices_sender = hs.get_server_notices_sender() self._event_serializer = hs.get_event_client_serializer() + self._msc2654_enabled = hs.config.experimental.msc2654_enabled async def on_GET(self, request: SynapseRequest) -> Tuple[int, JsonDict]: # This will always be set by the time Twisted calls us. @@ -521,7 +522,8 @@ class SyncRestServlet(RestServlet): result["ephemeral"] = {"events": ephemeral_events} result["unread_notifications"] = room.unread_notifications result["summary"] = room.summary - result["org.matrix.msc2654.unread_count"] = room.unread_count + if self._msc2654_enabled: + result["org.matrix.msc2654.unread_count"] = room.unread_count return result diff --git a/tests/rest/client/test_sync.py b/tests/rest/client/test_sync.py index 3cebbd18f0..773c16a54c 100644 --- a/tests/rest/client/test_sync.py +++ b/tests/rest/client/test_sync.py @@ -496,6 +496,11 @@ class UnreadMessagesTestCase(unittest.HomeserverTestCase): receipts.register_servlets, ] + def default_config(self) -> JsonDict: + config = super().default_config() + config["experimental_features"] = {"msc2654_enabled": True} + return config + def prepare(self, reactor: MemoryReactor, clock: Clock, hs: HomeServer) -> None: self.url = "/sync?since=%s" self.next_batch = "s0" From 4e900ece42eb0d28c2773d328b4ab459b0d3aaf3 Mon Sep 17 00:00:00 2001 From: Amanda Graven Date: Fri, 1 Apr 2022 10:31:30 +0200 Subject: [PATCH 13/55] Add set_user_admin function to the module API (#12341) --- changelog.d/12341.feature | 1 + synapse/module_api/__init__.py | 11 +++++++++++ tests/module_api/test_api.py | 14 ++++++++++++++ 3 files changed, 26 insertions(+) create mode 100644 changelog.d/12341.feature diff --git a/changelog.d/12341.feature b/changelog.d/12341.feature new file mode 100644 index 0000000000..ebb96ee486 --- /dev/null +++ b/changelog.d/12341.feature @@ -0,0 +1 @@ +Allow setting user admin status using the module API. Contributed by Famedly. diff --git a/synapse/module_api/__init__.py b/synapse/module_api/__init__.py index 3c7dcca74d..f7f95bae29 100644 --- a/synapse/module_api/__init__.py +++ b/synapse/module_api/__init__.py @@ -515,6 +515,17 @@ class ModuleApi: """ return await self._store.is_server_admin(UserID.from_string(user_id)) + async def set_user_admin(self, user_id: str, admin: bool) -> None: + """Sets if a user is a server admin. + + Added in Synapse v1.56.0. + + Args: + user_id: The Matrix ID of the user to set admin status for. + admin: True iff the user is to be a server admin, false otherwise. + """ + await self._store.set_server_admin(UserID.from_string(user_id), admin) + def get_qualified_user_id(self, username: str) -> str: """Qualify a user id, if necessary diff --git a/tests/module_api/test_api.py b/tests/module_api/test_api.py index 10dd94b549..36dfe5c36a 100644 --- a/tests/module_api/test_api.py +++ b/tests/module_api/test_api.py @@ -96,6 +96,20 @@ class ModuleApiTestCase(HomeserverTestCase): self.assertEqual(found_user.user_id.to_string(), user_id) self.assertIdentical(found_user.is_admin, True) + def test_can_set_admin(self): + user_id = self.get_success( + self.register_user( + "alice_wants_admin", + "1234", + displayname="Alice Powerhungry", + admin=False, + ) + ) + self.get_success(self.module_api.set_user_admin(user_id, True)) + found_user = self.get_success(self.module_api.get_userinfo_by_id(user_id)) + self.assertEqual(found_user.user_id.to_string(), user_id) + self.assertIdentical(found_user.is_admin, True) + def test_get_userinfo_by_id(self): user_id = self.register_user("alice", "1234") found_user = self.get_success(self.module_api.get_userinfo_by_id(user_id)) From e4409301ba91add60d1ee1f1f0e6346228737d70 Mon Sep 17 00:00:00 2001 From: Brendan Abolivier Date: Fri, 1 Apr 2022 11:22:48 +0200 Subject: [PATCH 14/55] Add a module callback to react to account data changes (#12327) Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- changelog.d/12327.feature | 1 + docs/SUMMARY.md | 1 + docs/modules/account_data_callbacks.md | 106 +++++++++++++++++++++++++ docs/modules/writing_a_module.md | 2 +- synapse/handlers/account_data.py | 52 +++++++++++- synapse/module_api/__init__.py | 15 ++++ tests/rest/client/test_account_data.py | 75 +++++++++++++++++ 7 files changed, 250 insertions(+), 2 deletions(-) create mode 100644 changelog.d/12327.feature create mode 100644 docs/modules/account_data_callbacks.md create mode 100644 tests/rest/client/test_account_data.py diff --git a/changelog.d/12327.feature b/changelog.d/12327.feature new file mode 100644 index 0000000000..4fe294f1b1 --- /dev/null +++ b/changelog.d/12327.feature @@ -0,0 +1 @@ +Add a module callback to react to account data changes. diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index 21f80efc99..6aa48e1919 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -45,6 +45,7 @@ - [Account validity callbacks](modules/account_validity_callbacks.md) - [Password auth provider callbacks](modules/password_auth_provider_callbacks.md) - [Background update controller callbacks](modules/background_update_controller_callbacks.md) + - [Account data callbacks](modules/account_data_callbacks.md) - [Porting a legacy module to the new interface](modules/porting_legacy_module.md) - [Workers](workers.md) - [Using `synctl` with Workers](synctl_workers.md) diff --git a/docs/modules/account_data_callbacks.md b/docs/modules/account_data_callbacks.md new file mode 100644 index 0000000000..25de911627 --- /dev/null +++ b/docs/modules/account_data_callbacks.md @@ -0,0 +1,106 @@ +# Account data callbacks + +Account data callbacks allow module developers to react to changes of the account data +of local users. Account data callbacks can be registered using the module API's +`register_account_data_callbacks` method. + +## Callbacks + +The available account data callbacks are: + +### `on_account_data_updated` + +_First introduced in Synapse v1.57.0_ + +```python +async def on_account_data_updated( + user_id: str, + room_id: Optional[str], + account_data_type: str, + content: "synapse.module_api.JsonDict", +) -> None: +``` + +Called after user's account data has been updated. The module is given the +Matrix ID of the user whose account data is changing, the room ID the data is associated +with, the type associated with the change, as well as the new content. If the account +data is not associated with a specific room, then the room ID is `None`. + +This callback is triggered when new account data is added or when the data associated with +a given type (and optionally room) changes. This includes deletion, since in Matrix, +deleting account data consists of replacing the data associated with a given type +(and optionally room) with an empty dictionary (`{}`). + +Note that this doesn't trigger when changing the tags associated with a room, as these are +processed separately by Synapse. + +If multiple modules implement this callback, Synapse runs them all in order. + +## Example + +The example below is a module that implements the `on_account_data_updated` callback, and +sends an event to an audit room when a user changes their account data. + +```python +import json +import attr +from typing import Any, Dict, Optional + +from synapse.module_api import JsonDict, ModuleApi +from synapse.module_api.errors import ConfigError + + +@attr.s(auto_attribs=True) +class CustomAccountDataConfig: + audit_room: str + sender: str + + +class CustomAccountDataModule: + def __init__(self, config: CustomAccountDataConfig, api: ModuleApi): + self.api = api + self.config = config + + self.api.register_account_data_callbacks( + on_account_data_updated=self.log_new_account_data, + ) + + @staticmethod + def parse_config(config: Dict[str, Any]) -> CustomAccountDataConfig: + def check_in_config(param: str): + if param not in config: + raise ConfigError(f"'{param}' is required") + + check_in_config("audit_room") + check_in_config("sender") + + return CustomAccountDataConfig( + audit_room=config["audit_room"], + sender=config["sender"], + ) + + async def log_new_account_data( + self, + user_id: str, + room_id: Optional[str], + account_data_type: str, + content: JsonDict, + ) -> None: + content_raw = json.dumps(content) + msg_content = f"{user_id} has changed their account data for type {account_data_type} to: {content_raw}" + + if room_id is not None: + msg_content += f" (in room {room_id})" + + await self.api.create_and_send_event_into_room( + { + "room_id": self.config.audit_room, + "sender": self.config.sender, + "type": "m.room.message", + "content": { + "msgtype": "m.text", + "body": msg_content + } + } + ) +``` diff --git a/docs/modules/writing_a_module.md b/docs/modules/writing_a_module.md index e7c0ffad58..e6303b739e 100644 --- a/docs/modules/writing_a_module.md +++ b/docs/modules/writing_a_module.md @@ -33,7 +33,7 @@ A module can implement the following static method: ```python @staticmethod -def parse_config(config: dict) -> dict +def parse_config(config: dict) -> Any ``` This method is given a dictionary resulting from parsing the YAML configuration for the diff --git a/synapse/handlers/account_data.py b/synapse/handlers/account_data.py index 177b4f8991..4af9fbc5d1 100644 --- a/synapse/handlers/account_data.py +++ b/synapse/handlers/account_data.py @@ -12,8 +12,9 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. +import logging import random -from typing import TYPE_CHECKING, Collection, List, Optional, Tuple +from typing import TYPE_CHECKING, Awaitable, Callable, Collection, List, Optional, Tuple from synapse.replication.http.account_data import ( ReplicationAddTagRestServlet, @@ -27,6 +28,12 @@ from synapse.types import JsonDict, UserID if TYPE_CHECKING: from synapse.server import HomeServer +logger = logging.getLogger(__name__) + +ON_ACCOUNT_DATA_UPDATED_CALLBACK = Callable[ + [str, Optional[str], str, JsonDict], Awaitable +] + class AccountDataHandler: def __init__(self, hs: "HomeServer"): @@ -40,6 +47,44 @@ class AccountDataHandler: self._remove_tag_client = ReplicationRemoveTagRestServlet.make_client(hs) self._account_data_writers = hs.config.worker.writers.account_data + self._on_account_data_updated_callbacks: List[ + ON_ACCOUNT_DATA_UPDATED_CALLBACK + ] = [] + + def register_module_callbacks( + self, on_account_data_updated: Optional[ON_ACCOUNT_DATA_UPDATED_CALLBACK] = None + ) -> None: + """Register callbacks from modules.""" + if on_account_data_updated is not None: + self._on_account_data_updated_callbacks.append(on_account_data_updated) + + async def _notify_modules( + self, + user_id: str, + room_id: Optional[str], + account_data_type: str, + content: JsonDict, + ) -> None: + """Notifies modules about new account data changes. + + A change can be either a new account data type being added, or the content + associated with a type being changed. Account data for a given type is removed by + changing the associated content to an empty dictionary. + + Note that this is not called when the tags associated with a room change. + + Args: + user_id: The user whose account data is changing. + room_id: The ID of the room the account data change concerns, if any. + account_data_type: The type of the account data. + content: The content that is now associated with this type. + """ + for callback in self._on_account_data_updated_callbacks: + try: + await callback(user_id, room_id, account_data_type, content) + except Exception as e: + logger.exception("Failed to run module callback %s: %s", callback, e) + async def add_account_data_to_room( self, user_id: str, room_id: str, account_data_type: str, content: JsonDict ) -> int: @@ -63,6 +108,8 @@ class AccountDataHandler: "account_data_key", max_stream_id, users=[user_id] ) + await self._notify_modules(user_id, room_id, account_data_type, content) + return max_stream_id else: response = await self._room_data_client( @@ -96,6 +143,9 @@ class AccountDataHandler: self._notifier.on_new_event( "account_data_key", max_stream_id, users=[user_id] ) + + await self._notify_modules(user_id, None, account_data_type, content) + return max_stream_id else: response = await self._user_data_client( diff --git a/synapse/module_api/__init__.py b/synapse/module_api/__init__.py index f7f95bae29..9a61593ff5 100644 --- a/synapse/module_api/__init__.py +++ b/synapse/module_api/__init__.py @@ -65,6 +65,7 @@ from synapse.events.third_party_rules import ( ON_THREEPID_BIND_CALLBACK, ON_USER_DEACTIVATION_STATUS_CHANGED_CALLBACK, ) +from synapse.handlers.account_data import ON_ACCOUNT_DATA_UPDATED_CALLBACK from synapse.handlers.account_validity import ( IS_USER_EXPIRED_CALLBACK, ON_LEGACY_ADMIN_REQUEST, @@ -216,6 +217,7 @@ class ModuleApi: self._third_party_event_rules = hs.get_third_party_event_rules() self._password_auth_provider = hs.get_password_auth_provider() self._presence_router = hs.get_presence_router() + self._account_data_handler = hs.get_account_data_handler() ################################################################################# # The following methods should only be called during the module's initialisation. @@ -376,6 +378,19 @@ class ModuleApi: min_batch_size=min_batch_size, ) + def register_account_data_callbacks( + self, + *, + on_account_data_updated: Optional[ON_ACCOUNT_DATA_UPDATED_CALLBACK] = None, + ) -> None: + """Registers account data callbacks. + + Added in Synapse 1.57.0. + """ + return self._account_data_handler.register_module_callbacks( + on_account_data_updated=on_account_data_updated, + ) + def register_web_resource(self, path: str, resource: Resource) -> None: """Registers a web resource to be served at the given path. diff --git a/tests/rest/client/test_account_data.py b/tests/rest/client/test_account_data.py new file mode 100644 index 0000000000..d5b0640e7a --- /dev/null +++ b/tests/rest/client/test_account_data.py @@ -0,0 +1,75 @@ +# Copyright 2022 The Matrix.org Foundation C.I.C +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +from unittest.mock import Mock + +from synapse.rest import admin +from synapse.rest.client import account_data, login, room + +from tests import unittest +from tests.test_utils import make_awaitable + + +class AccountDataTestCase(unittest.HomeserverTestCase): + servlets = [ + admin.register_servlets, + login.register_servlets, + room.register_servlets, + account_data.register_servlets, + ] + + def test_on_account_data_updated_callback(self) -> None: + """Tests that the on_account_data_updated module callback is called correctly when + a user's account data changes. + """ + mocked_callback = Mock(return_value=make_awaitable(None)) + self.hs.get_account_data_handler()._on_account_data_updated_callbacks.append( + mocked_callback + ) + + user_id = self.register_user("user", "password") + tok = self.login("user", "password") + account_data_type = "org.matrix.foo" + account_data_content = {"bar": "baz"} + + # Change the user's global account data. + channel = self.make_request( + "PUT", + f"/user/{user_id}/account_data/{account_data_type}", + account_data_content, + access_token=tok, + ) + + # Test that the callback is called with the user ID, the new account data, and + # None as the room ID. + self.assertEqual(channel.code, 200, channel.result) + mocked_callback.assert_called_once_with( + user_id, None, account_data_type, account_data_content + ) + + # Change the user's room-specific account data. + room_id = self.helper.create_room_as(user_id, tok=tok) + channel = self.make_request( + "PUT", + f"/user/{user_id}/rooms/{room_id}/account_data/{account_data_type}", + account_data_content, + access_token=tok, + ) + + # Test that the callback is called with the user ID, the room ID and the new + # account data. + self.assertEqual(channel.code, 200, channel.result) + self.assertEqual(mocked_callback.call_count, 2) + mocked_callback.assert_called_with( + user_id, room_id, account_data_type, account_data_content + ) From 9b43df1f7b2977431563b3cda8fed1ed879651ba Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Fri, 1 Apr 2022 12:53:42 +0100 Subject: [PATCH 15/55] Optimise `_get_state_after_missing_prev_event`: use `/state` (#12040) If we're missing most of the events in the room state, then we may as well call the /state endpoint, instead of individually requesting each and every event. --- changelog.d/12040.feature | 1 + synapse/handlers/federation_event.py | 43 +++- .../storage/databases/main/events_worker.py | 8 +- tests/handlers/test_federation_event.py | 225 ++++++++++++++++++ 4 files changed, 268 insertions(+), 9 deletions(-) create mode 100644 changelog.d/12040.feature create mode 100644 tests/handlers/test_federation_event.py diff --git a/changelog.d/12040.feature b/changelog.d/12040.feature new file mode 100644 index 0000000000..0a88c6e80c --- /dev/null +++ b/changelog.d/12040.feature @@ -0,0 +1 @@ +Optimise fetching large quantities of missing room state over federation. diff --git a/synapse/handlers/federation_event.py b/synapse/handlers/federation_event.py index 567afc910f..e7b9f15e13 100644 --- a/synapse/handlers/federation_event.py +++ b/synapse/handlers/federation_event.py @@ -897,10 +897,24 @@ class FederationEventHandler: logger.debug("We are also missing %i auth events", len(missing_auth_events)) missing_events = missing_desired_events | missing_auth_events - logger.debug("Fetching %i events from remote", len(missing_events)) - await self._get_events_and_persist( - destination=destination, room_id=room_id, event_ids=missing_events - ) + + # Making an individual request for each of 1000s of events has a lot of + # overhead. On the other hand, we don't really want to fetch all of the events + # if we already have most of them. + # + # As an arbitrary heuristic, if we are missing more than 10% of the events, then + # we fetch the whole state. + # + # TODO: might it be better to have an API which lets us do an aggregate event + # request + if (len(missing_events) * 10) >= len(auth_event_ids) + len(state_event_ids): + logger.debug("Requesting complete state from remote") + await self._get_state_and_persist(destination, room_id, event_id) + else: + logger.debug("Fetching %i events from remote", len(missing_events)) + await self._get_events_and_persist( + destination=destination, room_id=room_id, event_ids=missing_events + ) # we need to make sure we re-load from the database to get the rejected # state correct. @@ -959,6 +973,27 @@ class FederationEventHandler: return remote_state + async def _get_state_and_persist( + self, destination: str, room_id: str, event_id: str + ) -> None: + """Get the complete room state at a given event, and persist any new events + as outliers""" + room_version = await self._store.get_room_version(room_id) + auth_events, state_events = await self._federation_client.get_room_state( + destination, room_id, event_id=event_id, room_version=room_version + ) + logger.info("/state returned %i events", len(auth_events) + len(state_events)) + + await self._auth_and_persist_outliers( + room_id, itertools.chain(auth_events, state_events) + ) + + # we also need the event itself. + if not await self._store.have_seen_event(room_id, event_id): + await self._get_events_and_persist( + destination=destination, room_id=room_id, event_ids=(event_id,) + ) + async def _process_received_pdu( self, origin: str, diff --git a/synapse/storage/databases/main/events_worker.py b/synapse/storage/databases/main/events_worker.py index 59454a47df..a60e3f4fdd 100644 --- a/synapse/storage/databases/main/events_worker.py +++ b/synapse/storage/databases/main/events_worker.py @@ -22,7 +22,6 @@ from typing import ( Dict, Iterable, List, - NoReturn, Optional, Set, Tuple, @@ -1330,10 +1329,9 @@ class EventsWorkerStore(SQLBaseStore): return results @cached(max_entries=100000, tree=True) - async def have_seen_event(self, room_id: str, event_id: str) -> NoReturn: - # this only exists for the benefit of the @cachedList descriptor on - # _have_seen_events_dict - raise NotImplementedError() + async def have_seen_event(self, room_id: str, event_id: str) -> bool: + res = await self._have_seen_events_dict(((room_id, event_id),)) + return res[(room_id, event_id)] def _get_current_state_event_counts_txn( self, txn: LoggingTransaction, room_id: str diff --git a/tests/handlers/test_federation_event.py b/tests/handlers/test_federation_event.py new file mode 100644 index 0000000000..489ba57736 --- /dev/null +++ b/tests/handlers/test_federation_event.py @@ -0,0 +1,225 @@ +# Copyright 2022 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +from unittest import mock + +from synapse.events import make_event_from_dict +from synapse.events.snapshot import EventContext +from synapse.federation.transport.client import StateRequestResponse +from synapse.logging.context import LoggingContext +from synapse.rest import admin +from synapse.rest.client import login, room + +from tests import unittest +from tests.test_utils import event_injection, make_awaitable + + +class FederationEventHandlerTests(unittest.FederatingHomeserverTestCase): + servlets = [ + admin.register_servlets, + login.register_servlets, + room.register_servlets, + ] + + def make_homeserver(self, reactor, clock): + # mock out the federation transport client + self.mock_federation_transport_client = mock.Mock( + spec=["get_room_state_ids", "get_room_state", "get_event"] + ) + return super().setup_test_homeserver( + federation_transport_client=self.mock_federation_transport_client + ) + + def test_process_pulled_event_with_missing_state(self) -> None: + """Ensure that we correctly handle pulled events with lots of missing state + + In this test, we pretend we are processing a "pulled" event (eg, via backfill + or get_missing_events). The pulled event has a prev_event we haven't previously + seen, so the server requests the state at that prev_event. There is a lot + of state we don't have, so we expect the server to make a /state request. + + We check that the pulled event is correctly persisted, and that the state is + as we expect. + """ + return self._test_process_pulled_event_with_missing_state(False) + + def test_process_pulled_event_with_missing_state_where_prev_is_outlier( + self, + ) -> None: + """Ensure that we correctly handle pulled events with lots of missing state + + A slight modification to test_process_pulled_event_with_missing_state. Again + we have a "pulled" event which refers to a prev_event with lots of state, + but in this case we already have the prev_event (as an outlier, obviously - + if it were a regular event, we wouldn't need to request the state). + """ + return self._test_process_pulled_event_with_missing_state(True) + + def _test_process_pulled_event_with_missing_state( + self, prev_exists_as_outlier: bool + ) -> None: + OTHER_USER = f"@user:{self.OTHER_SERVER_NAME}" + main_store = self.hs.get_datastores().main + state_storage = self.hs.get_storage().state + + # create the room + user_id = self.register_user("kermit", "test") + tok = self.login("kermit", "test") + room_id = self.helper.create_room_as(room_creator=user_id, tok=tok) + room_version = self.get_success(main_store.get_room_version(room_id)) + + # allow the remote user to send state events + self.helper.send_state( + room_id, + "m.room.power_levels", + {"events_default": 0, "state_default": 0}, + tok=tok, + ) + + # add the remote user to the room + member_event = self.get_success( + event_injection.inject_member_event(self.hs, room_id, OTHER_USER, "join") + ) + + initial_state_map = self.get_success(main_store.get_current_state_ids(room_id)) + + auth_event_ids = [ + initial_state_map[("m.room.create", "")], + initial_state_map[("m.room.power_levels", "")], + initial_state_map[("m.room.join_rules", "")], + member_event.event_id, + ] + + # mock up a load of state events which we are missing + state_events = [ + make_event_from_dict( + self.add_hashes_and_signatures( + { + "type": "test_state_type", + "state_key": f"state_{i}", + "room_id": room_id, + "sender": OTHER_USER, + "prev_events": [member_event.event_id], + "auth_events": auth_event_ids, + "origin_server_ts": 1, + "depth": 10, + "content": {"body": f"state_{i}"}, + } + ), + room_version, + ) + for i in range(1, 10) + ] + + # this is the state that we are going to claim is active at the prev_event. + state_at_prev_event = state_events + self.get_success( + main_store.get_events_as_list(initial_state_map.values()) + ) + + # mock up a prev event. + # Depending on the test, we either persist this upfront (as an outlier), + # or let the server request it. + prev_event = make_event_from_dict( + self.add_hashes_and_signatures( + { + "type": "test_regular_type", + "room_id": room_id, + "sender": OTHER_USER, + "prev_events": [], + "auth_events": auth_event_ids, + "origin_server_ts": 1, + "depth": 11, + "content": {"body": "missing_prev"}, + } + ), + room_version, + ) + if prev_exists_as_outlier: + prev_event.internal_metadata.outlier = True + persistence = self.hs.get_storage().persistence + self.get_success( + persistence.persist_event(prev_event, EventContext.for_outlier()) + ) + else: + + async def get_event(destination: str, event_id: str, timeout=None): + self.assertEqual(destination, self.OTHER_SERVER_NAME) + self.assertEqual(event_id, prev_event.event_id) + return {"pdus": [prev_event.get_pdu_json()]} + + self.mock_federation_transport_client.get_event.side_effect = get_event + + # mock up a regular event to pass into _process_pulled_event + pulled_event = make_event_from_dict( + self.add_hashes_and_signatures( + { + "type": "test_regular_type", + "room_id": room_id, + "sender": OTHER_USER, + "prev_events": [prev_event.event_id], + "auth_events": auth_event_ids, + "origin_server_ts": 1, + "depth": 12, + "content": {"body": "pulled"}, + } + ), + room_version, + ) + + # we expect an outbound request to /state_ids, so stub that out + self.mock_federation_transport_client.get_room_state_ids.return_value = ( + make_awaitable( + { + "pdu_ids": [e.event_id for e in state_at_prev_event], + "auth_chain_ids": [], + } + ) + ) + + # we also expect an outbound request to /state + self.mock_federation_transport_client.get_room_state.return_value = ( + make_awaitable( + StateRequestResponse(auth_events=[], state=state_at_prev_event) + ) + ) + + # we have to bump the clock a bit, to keep the retry logic in + # FederationClient.get_pdu happy + self.reactor.advance(60000) + + # Finally, the call under test: send the pulled event into _process_pulled_event + with LoggingContext("test"): + self.get_success( + self.hs.get_federation_event_handler()._process_pulled_event( + self.OTHER_SERVER_NAME, pulled_event, backfilled=False + ) + ) + + # check that the event is correctly persisted + persisted = self.get_success(main_store.get_event(pulled_event.event_id)) + self.assertIsNotNone(persisted, "pulled event was not persisted at all") + self.assertFalse( + persisted.internal_metadata.is_outlier(), "pulled event was an outlier" + ) + + # check that the state at that event is as expected + state = self.get_success( + state_storage.get_state_ids_for_event(pulled_event.event_id) + ) + expected_state = { + (e.type, e.state_key): e.event_id for e in state_at_prev_event + } + self.assertEqual(state, expected_state) + + if prev_exists_as_outlier: + self.mock_federation_transport_client.get_event.assert_not_called() From 319a805cd35fd25ff5ab6a6d85c4a4bfd5cb1aea Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Fri, 1 Apr 2022 13:01:49 +0100 Subject: [PATCH 16/55] Raise an exception when getting state at an outlier (#12191) It seems like calling `_get_state_group_for_events` for an event where the state is unknown is an error. Accordingly, let's raise an exception rather than silently returning an empty result. --- changelog.d/12191.misc | 1 + synapse/storage/databases/main/state.py | 16 ++++-- synapse/storage/state.py | 20 +++++++ tests/handlers/test_federation.py | 72 ++++++++++++++++++------- 4 files changed, 87 insertions(+), 22 deletions(-) create mode 100644 changelog.d/12191.misc diff --git a/changelog.d/12191.misc b/changelog.d/12191.misc new file mode 100644 index 0000000000..9f333e718a --- /dev/null +++ b/changelog.d/12191.misc @@ -0,0 +1 @@ +Avoid trying to calculate the state at outlier events. diff --git a/synapse/storage/databases/main/state.py b/synapse/storage/databases/main/state.py index 6e99a88a80..4a461a0abb 100644 --- a/synapse/storage/databases/main/state.py +++ b/synapse/storage/databases/main/state.py @@ -13,7 +13,7 @@ # See the License for the specific language governing permissions and # limitations under the License. import logging -from typing import TYPE_CHECKING, Dict, Iterable, Optional, Set, Tuple +from typing import TYPE_CHECKING, Collection, Dict, Iterable, Optional, Set, Tuple from frozendict import frozendict @@ -309,9 +309,13 @@ class StateGroupWorkerStore(EventsWorkerStore, SQLBaseStore): num_args=1, ) async def _get_state_group_for_events( - self, event_ids: Iterable[str] + self, event_ids: Collection[str] ) -> Dict[str, int]: - """Returns mapping event_id -> state_group""" + """Returns mapping event_id -> state_group. + + Raises: + RuntimeError if the state is unknown at any of the given events + """ rows = await self.db_pool.simple_select_many_batch( table="event_to_state_groups", column="event_id", @@ -321,7 +325,11 @@ class StateGroupWorkerStore(EventsWorkerStore, SQLBaseStore): desc="_get_state_group_for_events", ) - return {row["event_id"]: row["state_group"] for row in rows} + res = {row["event_id"]: row["state_group"] for row in rows} + for e in event_ids: + if e not in res: + raise RuntimeError("No state group for unknown or outlier event %s" % e) + return res async def get_referenced_state_groups( self, state_groups: Iterable[int] diff --git a/synapse/storage/state.py b/synapse/storage/state.py index 86f1a5373b..cda194e8c8 100644 --- a/synapse/storage/state.py +++ b/synapse/storage/state.py @@ -571,6 +571,10 @@ class StateGroupStorage: Returns: dict of state_group_id -> (dict of (type, state_key) -> event id) + + Raises: + RuntimeError if we don't have a state group for one or more of the events + (ie they are outliers or unknown) """ if not event_ids: return {} @@ -659,6 +663,10 @@ class StateGroupStorage: Returns: A dict of (event_id) -> (type, state_key) -> [state_events] + + Raises: + RuntimeError if we don't have a state group for one or more of the events + (ie they are outliers or unknown) """ event_to_groups = await self.stores.main._get_state_group_for_events(event_ids) @@ -696,6 +704,10 @@ class StateGroupStorage: Returns: A dict from event_id -> (type, state_key) -> event_id + + Raises: + RuntimeError if we don't have a state group for one or more of the events + (ie they are outliers or unknown) """ event_to_groups = await self.stores.main._get_state_group_for_events(event_ids) @@ -723,6 +735,10 @@ class StateGroupStorage: Returns: A dict from (type, state_key) -> state_event + + Raises: + RuntimeError if we don't have a state group for the event (ie it is an + outlier or is unknown) """ state_map = await self.get_state_for_events( [event_id], state_filter or StateFilter.all() @@ -741,6 +757,10 @@ class StateGroupStorage: Returns: A dict from (type, state_key) -> state_event_id + + Raises: + RuntimeError if we don't have a state group for the event (ie it is an + outlier or is unknown) """ state_map = await self.get_state_ids_for_events( [event_id], state_filter or StateFilter.all() diff --git a/tests/handlers/test_federation.py b/tests/handlers/test_federation.py index 4d65639a1e..0fa5045301 100644 --- a/tests/handlers/test_federation.py +++ b/tests/handlers/test_federation.py @@ -20,17 +20,17 @@ from twisted.test.proto_helpers import MemoryReactor from synapse.api.constants import EventTypes from synapse.api.errors import AuthError, Codes, LimitExceededError, SynapseError from synapse.api.room_versions import RoomVersions -from synapse.events import EventBase +from synapse.events import EventBase, make_event_from_dict from synapse.federation.federation_base import event_from_pdu_json from synapse.logging.context import LoggingContext, run_in_background from synapse.rest import admin from synapse.rest.client import login, room from synapse.server import HomeServer -from synapse.types import create_requester from synapse.util import Clock from synapse.util.stringutils import random_string from tests import unittest +from tests.test_utils import event_injection logger = logging.getLogger(__name__) @@ -39,7 +39,7 @@ def generate_fake_event_id() -> str: return "$fake_" + random_string(43) -class FederationTestCase(unittest.HomeserverTestCase): +class FederationTestCase(unittest.FederatingHomeserverTestCase): servlets = [ admin.register_servlets, login.register_servlets, @@ -219,40 +219,76 @@ class FederationTestCase(unittest.HomeserverTestCase): # create the room user_id = self.register_user("kermit", "test") tok = self.login("kermit", "test") - requester = create_requester(user_id) room_id = self.helper.create_room_as(room_creator=user_id, tok=tok) + room_version = self.get_success(self.store.get_room_version(room_id)) - ev1 = self.helper.send(room_id, "first message", tok=tok) + # we need a user on the remote server to be a member, so that we can send + # extremity-causing events. + self.get_success( + event_injection.inject_member_event( + self.hs, room_id, f"@user:{self.OTHER_SERVER_NAME}", "join" + ) + ) + + send_result = self.helper.send(room_id, "first message", tok=tok) + ev1 = self.get_success( + self.store.get_event(send_result["event_id"], allow_none=False) + ) + current_state = self.get_success( + self.store.get_events_as_list( + (self.get_success(self.store.get_current_state_ids(room_id))).values() + ) + ) # Create "many" backward extremities. The magic number we're trying to # create more than is 5 which corresponds to the number of backward # extremities we slice off in `_maybe_backfill_inner` + federation_event_handler = self.hs.get_federation_event_handler() for _ in range(0, 8): - event_handler = self.hs.get_event_creation_handler() - event, context = self.get_success( - event_handler.create_event( - requester, + event = make_event_from_dict( + self.add_hashes_and_signatures( { + "origin_server_ts": 1, "type": "m.room.message", "content": { "msgtype": "m.text", "body": "message connected to fake event", }, "room_id": room_id, - "sender": user_id, + "sender": f"@user:{self.OTHER_SERVER_NAME}", + "prev_events": [ + ev1.event_id, + # We're creating an backward extremity each time thanks + # to this fake event + generate_fake_event_id(), + ], + # lazy: *everything* is an auth event + "auth_events": [ev.event_id for ev in current_state], + "depth": ev1.depth + 1, }, - prev_event_ids=[ - ev1["event_id"], - # We're creating an backward extremity each time thanks - # to this fake event - generate_fake_event_id(), - ], + room_version, + ), + room_version, + ) + + # we poke this directly into _process_received_pdu, to avoid the + # federation handler wanting to backfill the fake event. + self.get_success( + federation_event_handler._process_received_pdu( + self.OTHER_SERVER_NAME, event, state=current_state ) ) - self.get_success( - event_handler.handle_new_client_event(requester, event, context) + + # we should now have 8 backwards extremities. + backwards_extremities = self.get_success( + self.store.db_pool.simple_select_list( + "event_backward_extremities", + keyvalues={"room_id": room_id}, + retcols=["event_id"], ) + ) + self.assertEqual(len(backwards_extremities), 8) current_depth = 1 limit = 100 From f8712228801d9c4753e65819fa4241818b56953b Mon Sep 17 00:00:00 2001 From: reivilibre Date: Fri, 1 Apr 2022 13:08:55 +0100 Subject: [PATCH 17/55] Move `update_client_ip` background job from the main process to the background worker. (#12251) --- changelog.d/12251.feature | 1 + synapse/app/admin_cmd.py | 2 - synapse/app/generic_worker.py | 2 - .../replication/slave/storage/client_ips.py | 59 ---------- synapse/replication/tcp/commands.py | 8 +- synapse/replication/tcp/handler.py | 46 +++++--- synapse/storage/databases/main/__init__.py | 8 +- synapse/storage/databases/main/client_ips.py | 101 +++++++++++------- .../databases/main/monthly_active_users.py | 60 ++++++----- .../storage/databases/main/registration.py | 24 ++--- 10 files changed, 159 insertions(+), 152 deletions(-) create mode 100644 changelog.d/12251.feature delete mode 100644 synapse/replication/slave/storage/client_ips.py diff --git a/changelog.d/12251.feature b/changelog.d/12251.feature new file mode 100644 index 0000000000..ba9ede03c6 --- /dev/null +++ b/changelog.d/12251.feature @@ -0,0 +1 @@ +Offload the `update_client_ip` background job from the main process to the background worker, when using Redis-based replication. diff --git a/synapse/app/admin_cmd.py b/synapse/app/admin_cmd.py index 6f8e33a156..2b0d92cbae 100644 --- a/synapse/app/admin_cmd.py +++ b/synapse/app/admin_cmd.py @@ -33,7 +33,6 @@ from synapse.handlers.admin import ExfiltrationWriter from synapse.replication.slave.storage._base import BaseSlavedStore from synapse.replication.slave.storage.account_data import SlavedAccountDataStore from synapse.replication.slave.storage.appservice import SlavedApplicationServiceStore -from synapse.replication.slave.storage.client_ips import SlavedClientIpStore from synapse.replication.slave.storage.deviceinbox import SlavedDeviceInboxStore from synapse.replication.slave.storage.devices import SlavedDeviceStore from synapse.replication.slave.storage.events import SlavedEventStore @@ -61,7 +60,6 @@ class AdminCmdSlavedStore( SlavedDeviceStore, SlavedPushRuleStore, SlavedEventStore, - SlavedClientIpStore, BaseSlavedStore, RoomWorkerStore, ): diff --git a/synapse/app/generic_worker.py b/synapse/app/generic_worker.py index b6f510ed30..1865c671f4 100644 --- a/synapse/app/generic_worker.py +++ b/synapse/app/generic_worker.py @@ -53,7 +53,6 @@ from synapse.replication.http import REPLICATION_PREFIX, ReplicationRestResource from synapse.replication.slave.storage._base import BaseSlavedStore from synapse.replication.slave.storage.account_data import SlavedAccountDataStore from synapse.replication.slave.storage.appservice import SlavedApplicationServiceStore -from synapse.replication.slave.storage.client_ips import SlavedClientIpStore from synapse.replication.slave.storage.deviceinbox import SlavedDeviceInboxStore from synapse.replication.slave.storage.devices import SlavedDeviceStore from synapse.replication.slave.storage.directory import DirectoryStore @@ -247,7 +246,6 @@ class GenericWorkerSlavedStore( SlavedApplicationServiceStore, SlavedRegistrationStore, SlavedProfileStore, - SlavedClientIpStore, SlavedFilteringStore, MonthlyActiveUsersWorkerStore, MediaRepositoryStore, diff --git a/synapse/replication/slave/storage/client_ips.py b/synapse/replication/slave/storage/client_ips.py deleted file mode 100644 index 14706a0817..0000000000 --- a/synapse/replication/slave/storage/client_ips.py +++ /dev/null @@ -1,59 +0,0 @@ -# Copyright 2017 Vector Creations Ltd -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -from typing import TYPE_CHECKING - -from synapse.storage.database import DatabasePool, LoggingDatabaseConnection -from synapse.storage.databases.main.client_ips import LAST_SEEN_GRANULARITY -from synapse.util.caches.lrucache import LruCache - -from ._base import BaseSlavedStore - -if TYPE_CHECKING: - from synapse.server import HomeServer - - -class SlavedClientIpStore(BaseSlavedStore): - def __init__( - self, - database: DatabasePool, - db_conn: LoggingDatabaseConnection, - hs: "HomeServer", - ): - super().__init__(database, db_conn, hs) - - self.client_ip_last_seen: LruCache[tuple, int] = LruCache( - cache_name="client_ip_last_seen", max_size=50000 - ) - - async def insert_client_ip( - self, user_id: str, access_token: str, ip: str, user_agent: str, device_id: str - ) -> None: - now = int(self._clock.time_msec()) - key = (user_id, access_token, ip) - - try: - last_seen = self.client_ip_last_seen.get(key) - except KeyError: - last_seen = None - - # Rate-limited inserts - if last_seen is not None and (now - last_seen) < LAST_SEEN_GRANULARITY: - return - - self.client_ip_last_seen.set(key, now) - - self.hs.get_replication_command_handler().send_user_ip( - user_id, access_token, ip, user_agent, device_id, now - ) diff --git a/synapse/replication/tcp/commands.py b/synapse/replication/tcp/commands.py index 3654f6c03c..fe34948168 100644 --- a/synapse/replication/tcp/commands.py +++ b/synapse/replication/tcp/commands.py @@ -356,7 +356,7 @@ class UserIpCommand(Command): access_token: str, ip: str, user_agent: str, - device_id: str, + device_id: Optional[str], last_seen: int, ): self.user_id = user_id @@ -389,6 +389,12 @@ class UserIpCommand(Command): ) ) + def __repr__(self) -> str: + return ( + f"UserIpCommand({self.user_id!r}, .., {self.ip!r}, " + f"{self.user_agent!r}, {self.device_id!r}, {self.last_seen})" + ) + class RemoteServerUpCommand(_SimpleCommand): """Sent when a worker has detected that a remote server is no longer diff --git a/synapse/replication/tcp/handler.py b/synapse/replication/tcp/handler.py index b217c35f99..615f1828dd 100644 --- a/synapse/replication/tcp/handler.py +++ b/synapse/replication/tcp/handler.py @@ -235,6 +235,14 @@ class ReplicationCommandHandler: if self._is_master: self._server_notices_sender = hs.get_server_notices_sender() + if hs.config.redis.redis_enabled: + # If we're using Redis, it's the background worker that should + # receive USER_IP commands and store the relevant client IPs. + self._should_insert_client_ips = hs.config.worker.run_background_tasks + else: + # If we're NOT using Redis, this must be handled by the master + self._should_insert_client_ips = hs.get_instance_name() == "master" + def _add_command_to_stream_queue( self, conn: IReplicationConnection, cmd: Union[RdataCommand, PositionCommand] ) -> None: @@ -401,23 +409,37 @@ class ReplicationCommandHandler: ) -> Optional[Awaitable[None]]: user_ip_cache_counter.inc() - if self._is_master: + if self._is_master or self._should_insert_client_ips: + # We make a point of only returning an awaitable if there's actually + # something to do; on_USER_IP is not an async function, but + # _handle_user_ip is. + # If on_USER_IP returns an awaitable, it gets scheduled as a + # background process (see `BaseReplicationStreamProtocol.handle_command`). return self._handle_user_ip(cmd) else: + # Returning None when this process definitely has nothing to do + # reduces the overhead of handling the USER_IP command, which is + # currently broadcast to all workers regardless of utility. return None async def _handle_user_ip(self, cmd: UserIpCommand) -> None: - await self._store.insert_client_ip( - cmd.user_id, - cmd.access_token, - cmd.ip, - cmd.user_agent, - cmd.device_id, - cmd.last_seen, - ) + """ + Handles a User IP, branching depending on whether we are the main process + and/or the background worker. + """ + if self._is_master: + assert self._server_notices_sender is not None + await self._server_notices_sender.on_user_ip(cmd.user_id) - assert self._server_notices_sender is not None - await self._server_notices_sender.on_user_ip(cmd.user_id) + if self._should_insert_client_ips: + await self._store.insert_client_ip( + cmd.user_id, + cmd.access_token, + cmd.ip, + cmd.user_agent, + cmd.device_id, + cmd.last_seen, + ) def on_RDATA(self, conn: IReplicationConnection, cmd: RdataCommand) -> None: if cmd.instance_name == self._instance_name: @@ -698,7 +720,7 @@ class ReplicationCommandHandler: access_token: str, ip: str, user_agent: str, - device_id: str, + device_id: Optional[str], last_seen: int, ) -> None: """Tell the master that the user made a request.""" diff --git a/synapse/storage/databases/main/__init__.py b/synapse/storage/databases/main/__init__.py index f024761ba7..1ea0b2aa6f 100644 --- a/synapse/storage/databases/main/__init__.py +++ b/synapse/storage/databases/main/__init__.py @@ -33,7 +33,7 @@ from .account_data import AccountDataStore from .appservice import ApplicationServiceStore, ApplicationServiceTransactionStore from .cache import CacheInvalidationWorkerStore from .censor_events import CensorEventsStore -from .client_ips import ClientIpStore +from .client_ips import ClientIpWorkerStore from .deviceinbox import DeviceInboxStore from .devices import DeviceStore from .directory import DirectoryStore @@ -49,7 +49,7 @@ from .keys import KeyStore from .lock import LockStore from .media_repository import MediaRepositoryStore from .metrics import ServerMetricsStore -from .monthly_active_users import MonthlyActiveUsersStore +from .monthly_active_users import MonthlyActiveUsersWorkerStore from .openid import OpenIdStore from .presence import PresenceStore from .profile import ProfileStore @@ -112,13 +112,13 @@ class DataStore( AccountDataStore, EventPushActionsStore, OpenIdStore, - ClientIpStore, + ClientIpWorkerStore, DeviceStore, DeviceInboxStore, UserDirectoryStore, GroupServerStore, UserErasureStore, - MonthlyActiveUsersStore, + MonthlyActiveUsersWorkerStore, StatsStore, RelationsStore, CensorEventsStore, diff --git a/synapse/storage/databases/main/client_ips.py b/synapse/storage/databases/main/client_ips.py index 8b0c614ece..8480ea4e1c 100644 --- a/synapse/storage/databases/main/client_ips.py +++ b/synapse/storage/databases/main/client_ips.py @@ -25,7 +25,9 @@ from synapse.storage.database import ( LoggingTransaction, make_tuple_comparison_clause, ) -from synapse.storage.databases.main.monthly_active_users import MonthlyActiveUsersStore +from synapse.storage.databases.main.monthly_active_users import ( + MonthlyActiveUsersWorkerStore, +) from synapse.types import JsonDict, UserID from synapse.util.caches.lrucache import LruCache @@ -397,7 +399,7 @@ class ClientIpBackgroundUpdateStore(SQLBaseStore): return updated -class ClientIpWorkerStore(ClientIpBackgroundUpdateStore): +class ClientIpWorkerStore(ClientIpBackgroundUpdateStore, MonthlyActiveUsersWorkerStore): def __init__( self, database: DatabasePool, @@ -406,11 +408,40 @@ class ClientIpWorkerStore(ClientIpBackgroundUpdateStore): ): super().__init__(database, db_conn, hs) + if hs.config.redis.redis_enabled: + # If we're using Redis, we can shift this update process off to + # the background worker + self._update_on_this_worker = hs.config.worker.run_background_tasks + else: + # If we're NOT using Redis, this must be handled by the master + self._update_on_this_worker = hs.get_instance_name() == "master" + self.user_ips_max_age = hs.config.server.user_ips_max_age + # (user_id, access_token, ip,) -> last_seen + self.client_ip_last_seen = LruCache[Tuple[str, str, str], int]( + cache_name="client_ip_last_seen", max_size=50000 + ) + if hs.config.worker.run_background_tasks and self.user_ips_max_age: self._clock.looping_call(self._prune_old_user_ips, 5 * 1000) + if self._update_on_this_worker: + # This is the designated worker that can write to the client IP + # tables. + + # (user_id, access_token, ip,) -> (user_agent, device_id, last_seen) + self._batch_row_update: Dict[ + Tuple[str, str, str], Tuple[str, Optional[str], int] + ] = {} + + self._client_ip_looper = self._clock.looping_call( + self._update_client_ips_batch, 5 * 1000 + ) + self.hs.get_reactor().addSystemEventTrigger( + "before", "shutdown", self._update_client_ips_batch + ) + @wrap_as_background_process("prune_old_user_ips") async def _prune_old_user_ips(self) -> None: """Removes entries in user IPs older than the configured period.""" @@ -456,7 +487,7 @@ class ClientIpWorkerStore(ClientIpBackgroundUpdateStore): "_prune_old_user_ips", _prune_old_user_ips_txn ) - async def get_last_client_ip_by_device( + async def _get_last_client_ip_by_device_from_database( self, user_id: str, device_id: Optional[str] ) -> Dict[Tuple[str, str], DeviceLastConnectionInfo]: """For each device_id listed, give the user_ip it was last seen on. @@ -487,7 +518,7 @@ class ClientIpWorkerStore(ClientIpBackgroundUpdateStore): return {(d["user_id"], d["device_id"]): d for d in res} - async def get_user_ip_and_agents( + async def _get_user_ip_and_agents_from_database( self, user: UserID, since_ts: int = 0 ) -> List[LastConnectionInfo]: """Fetch the IPs and user agents for a user since the given timestamp. @@ -539,34 +570,6 @@ class ClientIpWorkerStore(ClientIpBackgroundUpdateStore): for access_token, ip, user_agent, last_seen in rows ] - -class ClientIpStore(ClientIpWorkerStore, MonthlyActiveUsersStore): - def __init__( - self, - database: DatabasePool, - db_conn: LoggingDatabaseConnection, - hs: "HomeServer", - ): - - # (user_id, access_token, ip,) -> last_seen - self.client_ip_last_seen = LruCache[Tuple[str, str, str], int]( - cache_name="client_ip_last_seen", max_size=50000 - ) - - super().__init__(database, db_conn, hs) - - # (user_id, access_token, ip,) -> (user_agent, device_id, last_seen) - self._batch_row_update: Dict[ - Tuple[str, str, str], Tuple[str, Optional[str], int] - ] = {} - - self._client_ip_looper = self._clock.looping_call( - self._update_client_ips_batch, 5 * 1000 - ) - self.hs.get_reactor().addSystemEventTrigger( - "before", "shutdown", self._update_client_ips_batch - ) - async def insert_client_ip( self, user_id: str, @@ -584,17 +587,27 @@ class ClientIpStore(ClientIpWorkerStore, MonthlyActiveUsersStore): last_seen = self.client_ip_last_seen.get(key) except KeyError: last_seen = None - await self.populate_monthly_active_users(user_id) + # Rate-limited inserts if last_seen is not None and (now - last_seen) < LAST_SEEN_GRANULARITY: return self.client_ip_last_seen.set(key, now) - self._batch_row_update[key] = (user_agent, device_id, now) + if self._update_on_this_worker: + await self.populate_monthly_active_users(user_id) + self._batch_row_update[key] = (user_agent, device_id, now) + else: + # We are not the designated writer-worker, so stream over replication + self.hs.get_replication_command_handler().send_user_ip( + user_id, access_token, ip, user_agent, device_id, now + ) @wrap_as_background_process("update_client_ips") async def _update_client_ips_batch(self) -> None: + assert ( + self._update_on_this_worker + ), "This worker is not designated to update client IPs" # If the DB pool has already terminated, don't try updating if not self.db_pool.is_running(): @@ -612,6 +625,10 @@ class ClientIpStore(ClientIpWorkerStore, MonthlyActiveUsersStore): txn: LoggingTransaction, to_update: Mapping[Tuple[str, str, str], Tuple[str, Optional[str], int]], ) -> None: + assert ( + self._update_on_this_worker + ), "This worker is not designated to update client IPs" + if "user_ips" in self.db_pool._unsafe_to_upsert_tables or ( not self.database_engine.can_native_upsert ): @@ -662,7 +679,12 @@ class ClientIpStore(ClientIpWorkerStore, MonthlyActiveUsersStore): A dictionary mapping a tuple of (user_id, device_id) to dicts, with keys giving the column names from the devices table. """ - ret = await super().get_last_client_ip_by_device(user_id, device_id) + ret = await self._get_last_client_ip_by_device_from_database(user_id, device_id) + + if not self._update_on_this_worker: + # Only the writing-worker has additional in-memory data to enhance + # the result + return ret # Update what is retrieved from the database with data which is pending # insertion, as if it has already been stored in the database. @@ -707,9 +729,16 @@ class ClientIpStore(ClientIpWorkerStore, MonthlyActiveUsersStore): Only the latest user agent for each access token and IP address combination is available. """ + rows_from_db = await self._get_user_ip_and_agents_from_database(user, since_ts) + + if not self._update_on_this_worker: + # Only the writing-worker has additional in-memory data to enhance + # the result + return rows_from_db + results: Dict[Tuple[str, str], LastConnectionInfo] = { (connection["access_token"], connection["ip"]): connection - for connection in await super().get_user_ip_and_agents(user, since_ts) + for connection in rows_from_db } # Overlay data that is pending insertion on top of the results from the diff --git a/synapse/storage/databases/main/monthly_active_users.py b/synapse/storage/databases/main/monthly_active_users.py index 216622964a..4f1c22c71b 100644 --- a/synapse/storage/databases/main/monthly_active_users.py +++ b/synapse/storage/databases/main/monthly_active_users.py @@ -15,7 +15,6 @@ import logging from typing import TYPE_CHECKING, Dict, List, Optional, Tuple, cast from synapse.metrics.background_process_metrics import wrap_as_background_process -from synapse.storage._base import SQLBaseStore from synapse.storage.database import ( DatabasePool, LoggingDatabaseConnection, @@ -36,7 +35,7 @@ logger = logging.getLogger(__name__) LAST_SEEN_GRANULARITY = 60 * 60 * 1000 -class MonthlyActiveUsersWorkerStore(SQLBaseStore): +class MonthlyActiveUsersWorkerStore(RegistrationWorkerStore): def __init__( self, database: DatabasePool, @@ -47,9 +46,30 @@ class MonthlyActiveUsersWorkerStore(SQLBaseStore): self._clock = hs.get_clock() self.hs = hs + if hs.config.redis.redis_enabled: + # If we're using Redis, we can shift this update process off to + # the background worker + self._update_on_this_worker = hs.config.worker.run_background_tasks + else: + # If we're NOT using Redis, this must be handled by the master + self._update_on_this_worker = hs.get_instance_name() == "master" + self._limit_usage_by_mau = hs.config.server.limit_usage_by_mau self._max_mau_value = hs.config.server.max_mau_value + self._mau_stats_only = hs.config.server.mau_stats_only + + if self._update_on_this_worker: + # Do not add more reserved users than the total allowable number + self.db_pool.new_transaction( + db_conn, + "initialise_mau_threepids", + [], + [], + self._initialise_reserved_users, + hs.config.server.mau_limits_reserved_threepids[: self._max_mau_value], + ) + @cached(num_args=0) async def get_monthly_active_count(self) -> int: """Generates current count of monthly active users @@ -222,28 +242,6 @@ class MonthlyActiveUsersWorkerStore(SQLBaseStore): "reap_monthly_active_users", _reap_users, reserved_users ) - -class MonthlyActiveUsersStore(MonthlyActiveUsersWorkerStore, RegistrationWorkerStore): - def __init__( - self, - database: DatabasePool, - db_conn: LoggingDatabaseConnection, - hs: "HomeServer", - ): - super().__init__(database, db_conn, hs) - - self._mau_stats_only = hs.config.server.mau_stats_only - - # Do not add more reserved users than the total allowable number - self.db_pool.new_transaction( - db_conn, - "initialise_mau_threepids", - [], - [], - self._initialise_reserved_users, - hs.config.server.mau_limits_reserved_threepids[: self._max_mau_value], - ) - def _initialise_reserved_users( self, txn: LoggingTransaction, threepids: List[dict] ) -> None: @@ -254,6 +252,9 @@ class MonthlyActiveUsersStore(MonthlyActiveUsersWorkerStore, RegistrationWorkerS txn: threepids: List of threepid dicts to reserve """ + assert ( + self._update_on_this_worker + ), "This worker is not designated to update MAUs" # XXX what is this function trying to achieve? It upserts into # monthly_active_users for each *registered* reserved mau user, but why? @@ -287,6 +288,10 @@ class MonthlyActiveUsersStore(MonthlyActiveUsersWorkerStore, RegistrationWorkerS Args: user_id: user to add/update """ + assert ( + self._update_on_this_worker + ), "This worker is not designated to update MAUs" + # Support user never to be included in MAU stats. Note I can't easily call this # from upsert_monthly_active_user_txn because then I need a _txn form of # is_support_user which is complicated because I want to cache the result. @@ -322,6 +327,9 @@ class MonthlyActiveUsersStore(MonthlyActiveUsersWorkerStore, RegistrationWorkerS txn (cursor): user_id (str): user to add/update """ + assert ( + self._update_on_this_worker + ), "This worker is not designated to update MAUs" # Am consciously deciding to lock the table on the basis that is ought # never be a big table and alternative approaches (batching multiple @@ -349,6 +357,10 @@ class MonthlyActiveUsersStore(MonthlyActiveUsersWorkerStore, RegistrationWorkerS Args: user_id(str): the user_id to query """ + assert ( + self._update_on_this_worker + ), "This worker is not designated to update MAUs" + if self._limit_usage_by_mau or self._mau_stats_only: # Trial users and guests should not be included as part of MAU group is_guest = await self.is_guest(user_id) # type: ignore[attr-defined] diff --git a/synapse/storage/databases/main/registration.py b/synapse/storage/databases/main/registration.py index 7f3d190e94..c7634c92fd 100644 --- a/synapse/storage/databases/main/registration.py +++ b/synapse/storage/databases/main/registration.py @@ -1745,6 +1745,18 @@ class RegistrationWorkerStore(CacheInvalidationWorkerStore): "replace_refresh_token", _replace_refresh_token_txn ) + @cached() + async def is_guest(self, user_id: str) -> bool: + res = await self.db_pool.simple_select_one_onecol( + table="users", + keyvalues={"name": user_id}, + retcol="is_guest", + allow_none=True, + desc="is_guest", + ) + + return res if res else False + class RegistrationBackgroundUpdateStore(RegistrationWorkerStore): def __init__( @@ -1887,18 +1899,6 @@ class RegistrationBackgroundUpdateStore(RegistrationWorkerStore): self._invalidate_cache_and_stream(txn, self.get_user_by_id, (user_id,)) txn.call_after(self.is_guest.invalidate, (user_id,)) - @cached() - async def is_guest(self, user_id: str) -> bool: - res = await self.db_pool.simple_select_one_onecol( - table="users", - keyvalues={"name": user_id}, - retcol="is_guest", - allow_none=True, - desc="is_guest", - ) - - return res if res else False - class RegistrationStore(StatsStore, RegistrationBackgroundUpdateStore): def __init__( From b7762b0c9f007e92a5b6a29add1c7f1603431889 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Fri, 1 Apr 2022 13:48:08 +0100 Subject: [PATCH 18/55] Move single-use methods out of `TestCase` (#12348) These methods are only used by a single testcase, so they shouldn't be cluttering up the base `TestCase` class. --- changelog.d/12348.misc | 1 + tests/rest/client/test_account.py | 11 +++++++++++ tests/storage/test_cleanup_extrems.py | 16 ++++++++++++++++ tests/unittest.py | 27 --------------------------- 4 files changed, 28 insertions(+), 27 deletions(-) create mode 100644 changelog.d/12348.misc diff --git a/changelog.d/12348.misc b/changelog.d/12348.misc new file mode 100644 index 0000000000..2bfeadb7f8 --- /dev/null +++ b/changelog.d/12348.misc @@ -0,0 +1 @@ +Move single-use methods out of `TestCase`. diff --git a/tests/rest/client/test_account.py b/tests/rest/client/test_account.py index 27946febff..e00b5c171c 100644 --- a/tests/rest/client/test_account.py +++ b/tests/rest/client/test_account.py @@ -89,6 +89,17 @@ class PasswordResetTestCase(unittest.HomeserverTestCase): self.store = hs.get_datastores().main self.submit_token_resource = PasswordResetSubmitTokenResource(hs) + def attempt_wrong_password_login(self, username: str, password: str) -> None: + """Attempts to login as the user with the given password, asserting + that the attempt *fails*. + """ + body = {"type": "m.login.password", "user": username, "password": password} + + channel = self.make_request( + "POST", "/_matrix/client/r0/login", json.dumps(body).encode("utf8") + ) + self.assertEqual(channel.code, 403, channel.result) + def test_basic_password_reset(self) -> None: """Test basic password reset flow""" old_password = "monkey" diff --git a/tests/storage/test_cleanup_extrems.py b/tests/storage/test_cleanup_extrems.py index ce89c96912..fe91c3fed8 100644 --- a/tests/storage/test_cleanup_extrems.py +++ b/tests/storage/test_cleanup_extrems.py @@ -68,6 +68,22 @@ class CleanupExtremBackgroundUpdateStoreTestCase(HomeserverTestCase): self.wait_for_background_updates() + def add_extremity(self, room_id: str, event_id: str) -> None: + """ + Add the given event as an extremity to the room. + """ + self.get_success( + self.hs.get_datastores().main.db_pool.simple_insert( + table="event_forward_extremities", + values={"room_id": room_id, "event_id": event_id}, + desc="test_add_extremity", + ) + ) + + self.hs.get_datastores().main.get_latest_event_ids_in_room.invalidate( + (room_id,) + ) + def test_soft_failed_extremities_handled_correctly(self): """Test that extremities are correctly calculated in the presence of soft failed events. diff --git a/tests/unittest.py b/tests/unittest.py index 326895f4c9..cbe215ee83 100644 --- a/tests/unittest.py +++ b/tests/unittest.py @@ -717,33 +717,6 @@ class HomeserverTestCase(TestCase): return event.event_id - def add_extremity(self, room_id, event_id): - """ - Add the given event as an extremity to the room. - """ - self.get_success( - self.hs.get_datastores().main.db_pool.simple_insert( - table="event_forward_extremities", - values={"room_id": room_id, "event_id": event_id}, - desc="test_add_extremity", - ) - ) - - self.hs.get_datastores().main.get_latest_event_ids_in_room.invalidate( - (room_id,) - ) - - def attempt_wrong_password_login(self, username, password): - """Attempts to login as the user with the given password, asserting - that the attempt *fails*. - """ - body = {"type": "m.login.password", "user": username, "password": password} - - channel = self.make_request( - "POST", "/_matrix/client/r0/login", json.dumps(body).encode("utf8") - ) - self.assertEqual(channel.code, 403, channel.result) - def inject_room_member(self, room: str, user: str, membership: Membership) -> None: """ Inject a membership event into a room. From 21351820e01e4ec1095707d3b9b34746429d9828 Mon Sep 17 00:00:00 2001 From: David Robertson Date: Fri, 1 Apr 2022 14:05:21 +0100 Subject: [PATCH 19/55] Remove `list_url_patterns` dev script (#12349) --- changelog.d/12349.misc | 1 + mypy.ini | 1 - scripts-dev/list_url_patterns.py | 60 -------------------------------- 3 files changed, 1 insertion(+), 61 deletions(-) create mode 100644 changelog.d/12349.misc delete mode 100755 scripts-dev/list_url_patterns.py diff --git a/changelog.d/12349.misc b/changelog.d/12349.misc new file mode 100644 index 0000000000..2b96e2d938 --- /dev/null +++ b/changelog.d/12349.misc @@ -0,0 +1 @@ +Remove unused and broken dev script `list_url_patterns.py`. \ No newline at end of file diff --git a/mypy.ini b/mypy.ini index d371886a8d..33c7d3e638 100644 --- a/mypy.ini +++ b/mypy.ini @@ -28,7 +28,6 @@ exclude = (?x) |scripts-dev/definitions.py |scripts-dev/federation_client.py |scripts-dev/hash_history.py - |scripts-dev/list_url_patterns.py |scripts-dev/release.py |scripts-dev/tail-synapse.py diff --git a/scripts-dev/list_url_patterns.py b/scripts-dev/list_url_patterns.py deleted file mode 100755 index e85420dea8..0000000000 --- a/scripts-dev/list_url_patterns.py +++ /dev/null @@ -1,60 +0,0 @@ -#! /usr/bin/python - -import argparse -import ast -import os -import sys - -import yaml - -PATTERNS_V1 = [] -PATTERNS_V2 = [] - -RESULT = {"v1": PATTERNS_V1, "v2": PATTERNS_V2} - - -class CallVisitor(ast.NodeVisitor): - def visit_Call(self, node): - if isinstance(node.func, ast.Name): - name = node.func.id - else: - return - - if name == "client_patterns": - PATTERNS_V2.append(node.args[0].s) - - -def find_patterns_in_code(input_code): - input_ast = ast.parse(input_code) - visitor = CallVisitor() - visitor.visit(input_ast) - - -def find_patterns_in_file(filepath): - with open(filepath) as f: - find_patterns_in_code(f.read()) - - -parser = argparse.ArgumentParser(description="Find url patterns.") - -parser.add_argument( - "directories", - nargs="+", - metavar="DIR", - help="Directories to search for definitions", -) - -args = parser.parse_args() - - -for directory in args.directories: - for root, _, files in os.walk(directory): - for filename in files: - if filename.endswith(".py"): - filepath = os.path.join(root, filename) - find_patterns_in_file(filepath) - -PATTERNS_V1.sort() -PATTERNS_V2.sort() - -yaml.dump(RESULT, sys.stdout, default_flow_style=False) From 993d90f82ba8faace30cbdaace5a8c5a4468b32a Mon Sep 17 00:00:00 2001 From: Nick Mills-Barrett Date: Fri, 1 Apr 2022 14:33:25 +0100 Subject: [PATCH 20/55] Use a sequence to generate AS transaction IDs, drop `last_txn` AS state (#12209) Switching to a sequence means there's no need to track `last_txn` on the AS state table to generate new TXN IDs. This also means that there is no longer contention between the AS scheduler and AS handler on updates to the `application_services_state` table, which will prevent serialization errors during the complete AS txn transaction. --- changelog.d/12209.misc | 1 + docs/upgrade.md | 13 ++++ synapse/storage/databases/main/appservice.py | 62 +++++----------- synapse/storage/schema/__init__.py | 5 +- .../schema/main/delta/69/01as_txn_seq.py | 44 ++++++++++++ tests/storage/test_appservice.py | 71 +------------------ 6 files changed, 83 insertions(+), 113 deletions(-) create mode 100644 changelog.d/12209.misc create mode 100644 synapse/storage/schema/main/delta/69/01as_txn_seq.py diff --git a/changelog.d/12209.misc b/changelog.d/12209.misc new file mode 100644 index 0000000000..d145b5eb04 --- /dev/null +++ b/changelog.d/12209.misc @@ -0,0 +1 @@ +Switch to using a sequence to generate AS transaction IDs. Contributed by Nick Beeper. If running synapse with a dedicated appservice worker, this MUST be stopped before upgrading the main process and database. diff --git a/docs/upgrade.md b/docs/upgrade.md index 062e823333..f6d226526a 100644 --- a/docs/upgrade.md +++ b/docs/upgrade.md @@ -85,6 +85,19 @@ process, for example: dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb ``` +# Upgrading to v1.57.0 + +## Changes to database schema for application services + +Synapse v1.57.0 includes a [change](https://github.com/matrix-org/synapse/pull/12209) to the +way transaction IDs are managed for application services. If your deployment uses a dedicated +worker for application service traffic, **it must be stopped** when the database is upgraded +(which normally happens when the main process is upgraded), to ensure the change is made safely +without any risk of reusing transaction IDs. + +Deployments which do not use separate worker processes can be upgraded as normal. Similarly, +deployments where no applciation services are in use can be upgraded as normal. + # Upgrading to v1.56.0 ## Groups/communities feature has been deprecated diff --git a/synapse/storage/databases/main/appservice.py b/synapse/storage/databases/main/appservice.py index 55e1ab099d..eb32c34a85 100644 --- a/synapse/storage/databases/main/appservice.py +++ b/synapse/storage/databases/main/appservice.py @@ -29,6 +29,8 @@ from synapse.storage._base import db_to_json from synapse.storage.database import DatabasePool, LoggingDatabaseConnection from synapse.storage.databases.main.events_worker import EventsWorkerStore from synapse.storage.databases.main.roommember import RoomMemberWorkerStore +from synapse.storage.types import Cursor +from synapse.storage.util.sequence import build_sequence_generator from synapse.types import DeviceListUpdates, JsonDict from synapse.util import json_encoder from synapse.util.caches.descriptors import _CacheContext, cached @@ -72,6 +74,22 @@ class ApplicationServiceWorkerStore(RoomMemberWorkerStore): ) self.exclusive_user_regex = _make_exclusive_regex(self.services_cache) + def get_max_as_txn_id(txn: Cursor) -> int: + logger.warning("Falling back to slow query, you should port to postgres") + txn.execute( + "SELECT COALESCE(max(txn_id), 0) FROM application_services_txns" + ) + return txn.fetchone()[0] # type: ignore + + self._as_txn_seq_gen = build_sequence_generator( + db_conn, + database.engine, + get_max_as_txn_id, + "application_services_txn_id_seq", + table="application_services_txns", + id_column="txn_id", + ) + super().__init__(database, db_conn, hs) def get_app_services(self): @@ -239,21 +257,7 @@ class ApplicationServiceTransactionWorkerStore( """ def _create_appservice_txn(txn): - # work out new txn id (highest txn id for this service += 1) - # The highest id may be the last one sent (in which case it is last_txn) - # or it may be the highest in the txns list (which are waiting to be/are - # being sent) - last_txn_id = self._get_last_txn(txn, service.id) - - txn.execute( - "SELECT MAX(txn_id) FROM application_services_txns WHERE as_id=?", - (service.id,), - ) - highest_txn_id = txn.fetchone()[0] - if highest_txn_id is None: - highest_txn_id = 0 - - new_txn_id = max(highest_txn_id, last_txn_id) + 1 + new_txn_id = self._as_txn_seq_gen.get_next_id_txn(txn) # Insert new txn into txn table event_ids = json_encoder.encode([e.event_id for e in events]) @@ -286,25 +290,8 @@ class ApplicationServiceTransactionWorkerStore( txn_id: The transaction ID being completed. service: The application service which was sent this transaction. """ - txn_id = int(txn_id) def _complete_appservice_txn(txn): - # Debugging query: Make sure the txn being completed is EXACTLY +1 from - # what was there before. If it isn't, we've got problems (e.g. the AS - # has probably missed some events), so whine loudly but still continue, - # since it shouldn't fail completion of the transaction. - last_txn_id = self._get_last_txn(txn, service.id) - if (last_txn_id + 1) != txn_id: - logger.error( - "appservice: Completing a transaction which has an ID > 1 from " - "the last ID sent to this AS. We've either dropped events or " - "sent it to the AS out of order. FIX ME. last_txn=%s " - "completing_txn=%s service_id=%s", - last_txn_id, - txn_id, - service.id, - ) - # Set current txn_id for AS to 'txn_id' self.db_pool.simple_upsert_txn( txn, @@ -376,17 +363,6 @@ class ApplicationServiceTransactionWorkerStore( device_list_summary=DeviceListUpdates(), ) - def _get_last_txn(self, txn, service_id: Optional[str]) -> int: - txn.execute( - "SELECT last_txn FROM application_services_state WHERE as_id=?", - (service_id,), - ) - last_txn_id = txn.fetchone() - if last_txn_id is None or last_txn_id[0] is None: # no row exists - return 0 - else: - return int(last_txn_id[0]) # select 'last_txn' col - async def set_appservice_last_pos(self, pos: int) -> None: def set_appservice_last_pos_txn(txn): txn.execute( diff --git a/synapse/storage/schema/__init__.py b/synapse/storage/schema/__init__.py index 7b21c1b96d..ea900e0f3d 100644 --- a/synapse/storage/schema/__init__.py +++ b/synapse/storage/schema/__init__.py @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -SCHEMA_VERSION = 68 # remember to update the list below when updating +SCHEMA_VERSION = 69 # remember to update the list below when updating """Represents the expectations made by the codebase about the database schema This should be incremented whenever the codebase changes its requirements on the @@ -58,6 +58,9 @@ Changes in SCHEMA_VERSION = 68: - event_reference_hashes is no longer read. - `events` has `state_key` and `rejection_reason` columns, which are populated for new events. + +Changes in SCHEMA_VERSION = 69: + - Use sequence to generate future `application_services_txns.txn_id`s """ diff --git a/synapse/storage/schema/main/delta/69/01as_txn_seq.py b/synapse/storage/schema/main/delta/69/01as_txn_seq.py new file mode 100644 index 0000000000..24bd4b391e --- /dev/null +++ b/synapse/storage/schema/main/delta/69/01as_txn_seq.py @@ -0,0 +1,44 @@ +# Copyright 2022 Beeper +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +""" +Adds a postgres SEQUENCE for generating application service transaction IDs. +""" + +from synapse.storage.engines import PostgresEngine + + +def run_create(cur, database_engine, *args, **kwargs): + if isinstance(database_engine, PostgresEngine): + # If we already have some AS TXNs we want to start from the current + # maximum value. There are two potential places this is stored - the + # actual TXNs themselves *and* the AS state table. At time of migration + # it is possible the TXNs table is empty so we must include the AS state + # last_txn as a potential option, and pick the maximum. + + cur.execute("SELECT COALESCE(max(txn_id), 0) FROM application_services_txns") + row = cur.fetchone() + txn_max = row[0] + + cur.execute("SELECT COALESCE(max(last_txn), 0) FROM application_services_state") + row = cur.fetchone() + last_txn_max = row[0] + + start_val = max(last_txn_max, txn_max) + 1 + + cur.execute( + "CREATE SEQUENCE application_services_txn_id_seq START WITH %s", + (start_val,), + ) diff --git a/tests/storage/test_appservice.py b/tests/storage/test_appservice.py index 97de9a59e0..08078d38e2 100644 --- a/tests/storage/test_appservice.py +++ b/tests/storage/test_appservice.py @@ -169,15 +169,6 @@ class ApplicationServiceTransactionStoreTestCase(unittest.HomeserverTestCase): (as_id, txn_id, json.dumps([e.event_id for e in events])), ) - def _set_last_txn(self, as_id, txn_id): - return self.db_pool.runOperation( - self.engine.convert_param_style( - "INSERT INTO application_services_state(as_id, last_txn, state) " - "VALUES(?,?,?)" - ), - (as_id, txn_id, ApplicationServiceState.UP.value), - ) - def test_get_appservice_state_none( self, ) -> None: @@ -277,64 +268,6 @@ class ApplicationServiceTransactionStoreTestCase(unittest.HomeserverTestCase): self.assertEqual(txn.events, events) self.assertEqual(txn.service, service) - def test_create_appservice_txn_older_last_txn( - self, - ) -> None: - service = Mock(id=self.as_list[0]["id"]) - events = cast(List[EventBase], [Mock(event_id="e1"), Mock(event_id="e2")]) - self.get_success(self._set_last_txn(service.id, 9643)) # AS is falling behind - self.get_success(self._insert_txn(service.id, 9644, events)) - self.get_success(self._insert_txn(service.id, 9645, events)) - txn = self.get_success( - self.store.create_appservice_txn( - service, events, [], [], {}, {}, DeviceListUpdates() - ) - ) - self.assertEqual(txn.id, 9646) - self.assertEqual(txn.events, events) - self.assertEqual(txn.service, service) - - def test_create_appservice_txn_up_to_date_last_txn( - self, - ) -> None: - service = Mock(id=self.as_list[0]["id"]) - events = cast(List[EventBase], [Mock(event_id="e1"), Mock(event_id="e2")]) - self.get_success(self._set_last_txn(service.id, 9643)) - txn = self.get_success( - self.store.create_appservice_txn( - service, events, [], [], {}, {}, DeviceListUpdates() - ) - ) - self.assertEqual(txn.id, 9644) - self.assertEqual(txn.events, events) - self.assertEqual(txn.service, service) - - def test_create_appservice_txn_up_fuzzing( - self, - ) -> None: - service = Mock(id=self.as_list[0]["id"]) - events = cast(List[EventBase], [Mock(event_id="e1"), Mock(event_id="e2")]) - self.get_success(self._set_last_txn(service.id, 9643)) - - # dump in rows with higher IDs to make sure the queries aren't wrong. - self.get_success(self._set_last_txn(self.as_list[1]["id"], 119643)) - self.get_success(self._set_last_txn(self.as_list[2]["id"], 9)) - self.get_success(self._set_last_txn(self.as_list[3]["id"], 9643)) - self.get_success(self._insert_txn(self.as_list[1]["id"], 119644, events)) - self.get_success(self._insert_txn(self.as_list[1]["id"], 119645, events)) - self.get_success(self._insert_txn(self.as_list[1]["id"], 119646, events)) - self.get_success(self._insert_txn(self.as_list[2]["id"], 10, events)) - self.get_success(self._insert_txn(self.as_list[3]["id"], 9643, events)) - - txn = self.get_success( - self.store.create_appservice_txn( - service, events, [], [], {}, {}, DeviceListUpdates() - ) - ) - self.assertEqual(txn.id, 9644) - self.assertEqual(txn.events, events) - self.assertEqual(txn.service, service) - def test_complete_appservice_txn_first_txn( self, ) -> None: @@ -368,13 +301,13 @@ class ApplicationServiceTransactionStoreTestCase(unittest.HomeserverTestCase): ) self.assertEqual(0, len(res)) - def test_complete_appservice_txn_existing_in_state_table( + def test_complete_appservice_txn_updates_last_txn_state( self, ) -> None: service = Mock(id=self.as_list[0]["id"]) events = [Mock(event_id="e1"), Mock(event_id="e2")] txn_id = 5 - self.get_success(self._set_last_txn(service.id, 4)) + self._set_state(self.as_list[0]["id"], ApplicationServiceState.UP) self.get_success(self._insert_txn(service.id, txn_id, events)) self.get_success( self.store.complete_appservice_txn(txn_id=txn_id, service=service) From 336bff1104453d83e8286385e56a1e986cd375ed Mon Sep 17 00:00:00 2001 From: David Robertson Date: Fri, 1 Apr 2022 14:41:42 +0100 Subject: [PATCH 21/55] Burn `check_signature` dev script. (#12351) --- changelog.d/12349.misc | 2 +- changelog.d/12351.misc | 1 + mypy.ini | 1 - scripts-dev/check_signature.py | 72 ---------------------------------- 4 files changed, 2 insertions(+), 74 deletions(-) create mode 100644 changelog.d/12351.misc delete mode 100644 scripts-dev/check_signature.py diff --git a/changelog.d/12349.misc b/changelog.d/12349.misc index 2b96e2d938..b2d83a7f78 100644 --- a/changelog.d/12349.misc +++ b/changelog.d/12349.misc @@ -1 +1 @@ -Remove unused and broken dev script `list_url_patterns.py`. \ No newline at end of file +Remove broken and unused development scripts. diff --git a/changelog.d/12351.misc b/changelog.d/12351.misc new file mode 100644 index 0000000000..b1dd1e2696 --- /dev/null +++ b/changelog.d/12351.misc @@ -0,0 +1 @@ +Remove broken and unused development scripts. \ No newline at end of file diff --git a/mypy.ini b/mypy.ini index 33c7d3e638..84e6b8646e 100644 --- a/mypy.ini +++ b/mypy.ini @@ -24,7 +24,6 @@ files = exclude = (?x) ^( |scripts-dev/build_debian_packages.py - |scripts-dev/check_signature.py |scripts-dev/definitions.py |scripts-dev/federation_client.py |scripts-dev/hash_history.py diff --git a/scripts-dev/check_signature.py b/scripts-dev/check_signature.py deleted file mode 100644 index 6755bc5282..0000000000 --- a/scripts-dev/check_signature.py +++ /dev/null @@ -1,72 +0,0 @@ -import argparse -import json -import logging -import sys - -import dns.resolver -import urllib2 -from signedjson.key import decode_verify_key_bytes, write_signing_keys -from signedjson.sign import verify_signed_json -from unpaddedbase64 import decode_base64 - - -def get_targets(server_name): - if ":" in server_name: - target, port = server_name.split(":") - yield (target, int(port)) - return - try: - answers = dns.resolver.query("_matrix._tcp." + server_name, "SRV") - for srv in answers: - yield (srv.target, srv.port) - except dns.resolver.NXDOMAIN: - yield (server_name, 8448) - - -def get_server_keys(server_name, target, port): - url = "https://%s:%i/_matrix/key/v1" % (target, port) - keys = json.load(urllib2.urlopen(url)) - verify_keys = {} - for key_id, key_base64 in keys["verify_keys"].items(): - verify_key = decode_verify_key_bytes(key_id, decode_base64(key_base64)) - verify_signed_json(keys, server_name, verify_key) - verify_keys[key_id] = verify_key - return verify_keys - - -def main(): - - parser = argparse.ArgumentParser() - parser.add_argument("signature_name") - parser.add_argument( - "input_json", nargs="?", type=argparse.FileType("r"), default=sys.stdin - ) - - args = parser.parse_args() - logging.basicConfig() - - server_name = args.signature_name - keys = {} - for target, port in get_targets(server_name): - try: - keys = get_server_keys(server_name, target, port) - print("Using keys from https://%s:%s/_matrix/key/v1" % (target, port)) - write_signing_keys(sys.stdout, keys.values()) - break - except Exception: - logging.exception("Error talking to %s:%s", target, port) - - json_to_check = json.load(args.input_json) - print("Checking JSON:") - for key_id in json_to_check["signatures"][args.signature_name]: - try: - key = keys[key_id] - verify_signed_json(json_to_check, args.signature_name, key) - print("PASS %s" % (key_id,)) - except Exception: - logging.exception("Check for key %s failed" % (key_id,)) - print("FAIL %s" % (key_id,)) - - -if __name__ == "__main__": - main() From c4cf916ed7d09d13d84f964dd683e1ecfd21815b Mon Sep 17 00:00:00 2001 From: reivilibre Date: Fri, 1 Apr 2022 15:55:09 +0100 Subject: [PATCH 22/55] Default to `private` room visibility rather than `public` when a client does not specify one, according to spec. (#12350) --- changelog.d/12350.bugfix | 1 + synapse/handlers/room.py | 4 +++- tests/module_api/test_api.py | 2 +- tests/rest/client/utils.py | 11 +++++++---- tests/storage/test_cleanup_extrems.py | 4 +++- 5 files changed, 15 insertions(+), 7 deletions(-) create mode 100644 changelog.d/12350.bugfix diff --git a/changelog.d/12350.bugfix b/changelog.d/12350.bugfix new file mode 100644 index 0000000000..9cbdc28038 --- /dev/null +++ b/changelog.d/12350.bugfix @@ -0,0 +1 @@ +Default to `private` room visibility rather than `public` when a client does not specify one, according to spec. \ No newline at end of file diff --git a/synapse/handlers/room.py b/synapse/handlers/room.py index 092e185c99..51a08fd2c0 100644 --- a/synapse/handlers/room.py +++ b/synapse/handlers/room.py @@ -771,7 +771,9 @@ class RoomCreationHandler: % (user_id,), ) - visibility = config.get("visibility", None) + # The spec says rooms should default to private visibility if + # `visibility` is not specified. + visibility = config.get("visibility", "private") is_public = visibility == "public" room_id = await self._generate_room_id( diff --git a/tests/module_api/test_api.py b/tests/module_api/test_api.py index 36dfe5c36a..dee248801d 100644 --- a/tests/module_api/test_api.py +++ b/tests/module_api/test_api.py @@ -292,7 +292,7 @@ class ModuleApiTestCase(HomeserverTestCase): # Create a user and room to play with user_id = self.register_user("kermit", "monkey") tok = self.login("kermit", "monkey") - room_id = self.helper.create_room_as(user_id, tok=tok) + room_id = self.helper.create_room_as(user_id, tok=tok, is_public=False) # The room should not currently be in the public rooms directory is_in_public_rooms = self.get_success( diff --git a/tests/rest/client/utils.py b/tests/rest/client/utils.py index 28663826fc..a0788b1bb0 100644 --- a/tests/rest/client/utils.py +++ b/tests/rest/client/utils.py @@ -88,7 +88,7 @@ class RestHelper: def create_room_as( self, room_creator: Optional[str] = None, - is_public: Optional[bool] = None, + is_public: Optional[bool] = True, room_version: Optional[str] = None, tok: Optional[str] = None, expect_code: int = HTTPStatus.OK, @@ -101,9 +101,12 @@ class RestHelper: Args: room_creator: The user ID to create the room with. is_public: If True, the `visibility` parameter will be set to - "public". If False, it will be set to "private". If left - unspecified, the server will set it to an appropriate default - (which should be "private" as per the CS spec). + "public". If False, it will be set to "private". + If None, doesn't specify the `visibility` parameter in which + case the server is supposed to make the room private according to + the CS API. + Defaults to public, since that is commonly needed in tests + for convenience where room privacy is not a problem. room_version: The room version to create the room as. Defaults to Synapse's default room version. tok: The access token to use in the request. diff --git a/tests/storage/test_cleanup_extrems.py b/tests/storage/test_cleanup_extrems.py index fe91c3fed8..b998ad42d9 100644 --- a/tests/storage/test_cleanup_extrems.py +++ b/tests/storage/test_cleanup_extrems.py @@ -266,7 +266,9 @@ class CleanupExtremDummyEventsTestCase(HomeserverTestCase): self.user = UserID.from_string(self.register_user("user1", "password")) self.token1 = self.login("user1", "password") self.requester = create_requester(self.user) - info, _ = self.get_success(self.room_creator.create_room(self.requester, {})) + info, _ = self.get_success( + self.room_creator.create_room(self.requester, {"visibility": "public"}) + ) self.room_id = info["room_id"] self.event_creator = homeserver.get_event_creation_handler() homeserver.config.consent.user_consent_version = self.CONSENT_VERSION From 33ebee47e4e96a2b6fdf72091769e59034dc550f Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Fri, 1 Apr 2022 16:10:31 +0100 Subject: [PATCH 23/55] Remove redundant `get_success` calls in test code (#12346) There are a bunch of places we call get_success on an immediate value, which is unnecessary. Let's rip them out, and remove the redundant functionality in get_success and friends. --- changelog.d/12346.misc | 1 + tests/handlers/test_deactivate_account.py | 25 ++++--- tests/handlers/test_profile.py | 2 +- tests/handlers/test_sync.py | 4 +- tests/module_api/test_api.py | 20 +++--- .../replication/slave/storage/test_events.py | 4 +- tests/rest/admin/test_server_notice.py | 12 +--- tests/rest/client/test_rooms.py | 2 +- tests/storage/test_id_generators.py | 12 ++-- tests/storage/test_redaction.py | 66 +++++++------------ tests/storage/test_stream.py | 4 +- tests/test_visibility.py | 26 ++++---- tests/unittest.py | 21 ++---- 13 files changed, 76 insertions(+), 123 deletions(-) create mode 100644 changelog.d/12346.misc diff --git a/changelog.d/12346.misc b/changelog.d/12346.misc new file mode 100644 index 0000000000..6561b3be82 --- /dev/null +++ b/changelog.d/12346.misc @@ -0,0 +1 @@ +Remove redundant `get_success` calls in test code. diff --git a/tests/handlers/test_deactivate_account.py b/tests/handlers/test_deactivate_account.py index 3a10791226..7586e472b5 100644 --- a/tests/handlers/test_deactivate_account.py +++ b/tests/handlers/test_deactivate_account.py @@ -44,21 +44,20 @@ class DeactivateAccountTestCase(HomeserverTestCase): Deactivates the account `self.user` using `self.token` and asserts that it returns a 200 success code. """ - req = self.get_success( - self.make_request( - "POST", - "account/deactivate", - { - "auth": { - "type": "m.login.password", - "user": self.user, - "password": "pass", - }, - "erase": True, + req = self.make_request( + "POST", + "account/deactivate", + { + "auth": { + "type": "m.login.password", + "user": self.user, + "password": "pass", }, - access_token=self.token, - ) + "erase": True, + }, + access_token=self.token, ) + self.assertEqual(req.code, HTTPStatus.OK, req) def test_global_account_data_deleted_upon_deactivation(self) -> None: diff --git a/tests/handlers/test_profile.py b/tests/handlers/test_profile.py index 1ec105c373..f88c725a42 100644 --- a/tests/handlers/test_profile.py +++ b/tests/handlers/test_profile.py @@ -59,7 +59,7 @@ class ProfileTestCase(unittest.HomeserverTestCase): self.bob = UserID.from_string("@4567:test") self.alice = UserID.from_string("@alice:remote") - self.get_success(self.register_user(self.frank.localpart, "frankpassword")) + self.register_user(self.frank.localpart, "frankpassword") self.handler = hs.get_profile_handler() diff --git a/tests/handlers/test_sync.py b/tests/handlers/test_sync.py index 3aedc0767b..865b8b7e47 100644 --- a/tests/handlers/test_sync.py +++ b/tests/handlers/test_sync.py @@ -158,9 +158,7 @@ class SyncTestCase(tests.unittest.HomeserverTestCase): ) # Blow away caches (supported room versions can only change due to a restart). - self.get_success( - self.store.get_rooms_for_user_with_stream_ordering.invalidate_all() - ) + self.store.get_rooms_for_user_with_stream_ordering.invalidate_all() self.store._get_event_cache.clear() # The rooms should be excluded from the sync response. diff --git a/tests/module_api/test_api.py b/tests/module_api/test_api.py index dee248801d..9fd5d59c55 100644 --- a/tests/module_api/test_api.py +++ b/tests/module_api/test_api.py @@ -87,24 +87,22 @@ class ModuleApiTestCase(HomeserverTestCase): self.assertEqual(displayname, "Bobberino") def test_can_register_admin_user(self): - user_id = self.get_success( - self.register_user( - "bob_module_admin", "1234", displayname="Bobberino Admin", admin=True - ) + user_id = self.register_user( + "bob_module_admin", "1234", displayname="Bobberino Admin", admin=True ) + found_user = self.get_success(self.module_api.get_userinfo_by_id(user_id)) self.assertEqual(found_user.user_id.to_string(), user_id) self.assertIdentical(found_user.is_admin, True) def test_can_set_admin(self): - user_id = self.get_success( - self.register_user( - "alice_wants_admin", - "1234", - displayname="Alice Powerhungry", - admin=False, - ) + user_id = self.register_user( + "alice_wants_admin", + "1234", + displayname="Alice Powerhungry", + admin=False, ) + self.get_success(self.module_api.set_user_admin(user_id, True)) found_user = self.get_success(self.module_api.get_userinfo_by_id(user_id)) self.assertEqual(found_user.user_id.to_string(), user_id) diff --git a/tests/replication/slave/storage/test_events.py b/tests/replication/slave/storage/test_events.py index 17dc42fd37..297a9e77f8 100644 --- a/tests/replication/slave/storage/test_events.py +++ b/tests/replication/slave/storage/test_events.py @@ -268,7 +268,7 @@ class SlavedEventStoreTestCase(BaseSlavedStoreTestCase): event_source = RoomEventSource(self.hs) event_source.store = self.slaved_store - current_token = self.get_success(event_source.get_current_key()) + current_token = event_source.get_current_key() # gradually stream out the replication while repl_transport.buffer: @@ -277,7 +277,7 @@ class SlavedEventStoreTestCase(BaseSlavedStoreTestCase): self.pump(0) prev_token = current_token - current_token = self.get_success(event_source.get_current_key()) + current_token = event_source.get_current_key() # attempt to replicate the behaviour of the sync handler. # diff --git a/tests/rest/admin/test_server_notice.py b/tests/rest/admin/test_server_notice.py index 2c855bff99..a53463c9ba 100644 --- a/tests/rest/admin/test_server_notice.py +++ b/tests/rest/admin/test_server_notice.py @@ -214,9 +214,7 @@ class ServerNoticeTestCase(unittest.HomeserverTestCase): self.assertEqual(messages[0]["sender"], "@notices:test") # invalidate cache of server notices room_ids - self.get_success( - self.server_notices_manager.get_or_create_notice_room_for_user.invalidate_all() - ) + self.server_notices_manager.get_or_create_notice_room_for_user.invalidate_all() # send second message channel = self.make_request( @@ -291,9 +289,7 @@ class ServerNoticeTestCase(unittest.HomeserverTestCase): # invalidate cache of server notices room_ids # if server tries to send to a cached room_id the user gets the message # in old room - self.get_success( - self.server_notices_manager.get_or_create_notice_room_for_user.invalidate_all() - ) + self.server_notices_manager.get_or_create_notice_room_for_user.invalidate_all() # send second message channel = self.make_request( @@ -380,9 +376,7 @@ class ServerNoticeTestCase(unittest.HomeserverTestCase): # invalidate cache of server notices room_ids # if server tries to send to a cached room_id it gives an error - self.get_success( - self.server_notices_manager.get_or_create_notice_room_for_user.invalidate_all() - ) + self.server_notices_manager.get_or_create_notice_room_for_user.invalidate_all() # send second message channel = self.make_request( diff --git a/tests/rest/client/test_rooms.py b/tests/rest/client/test_rooms.py index 3a9617d6da..6ff79b9e2e 100644 --- a/tests/rest/client/test_rooms.py +++ b/tests/rest/client/test_rooms.py @@ -982,7 +982,7 @@ class RoomJoinRatelimitTestCase(RoomBase): super().prepare(reactor, clock, hs) # profile changes expect that the user is actually registered user = UserID.from_string(self.user_id) - self.get_success(self.register_user(user.localpart, "supersecretpassword")) + self.register_user(user.localpart, "supersecretpassword") @unittest.override_config( {"rc_joins": {"local": {"per_second": 0.5, "burst_count": 3}}} diff --git a/tests/storage/test_id_generators.py b/tests/storage/test_id_generators.py index 395396340b..2d8d1f860f 100644 --- a/tests/storage/test_id_generators.py +++ b/tests/storage/test_id_generators.py @@ -157,10 +157,10 @@ class MultiWriterIdGeneratorTestCase(HomeserverTestCase): self.assertEqual(id_gen.get_positions(), {"master": 7}) self.assertEqual(id_gen.get_current_token_for_writer("master"), 7) - ctx1 = self.get_success(id_gen.get_next()) - ctx2 = self.get_success(id_gen.get_next()) - ctx3 = self.get_success(id_gen.get_next()) - ctx4 = self.get_success(id_gen.get_next()) + ctx1 = id_gen.get_next() + ctx2 = id_gen.get_next() + ctx3 = id_gen.get_next() + ctx4 = id_gen.get_next() s1 = self.get_success(ctx1.__aenter__()) s2 = self.get_success(ctx2.__aenter__()) @@ -362,8 +362,8 @@ class MultiWriterIdGeneratorTestCase(HomeserverTestCase): self.assertEqual(id_gen.get_current_token_for_writer("master"), 7) # Persist two rows at once - ctx1 = self.get_success(id_gen.get_next()) - ctx2 = self.get_success(id_gen.get_next()) + ctx1 = id_gen.get_next() + ctx2 = id_gen.get_next() s1 = self.get_success(ctx1.__aenter__()) s2 = self.get_success(ctx2.__aenter__()) diff --git a/tests/storage/test_redaction.py b/tests/storage/test_redaction.py index 03e9cc7d4a..d8d17ef379 100644 --- a/tests/storage/test_redaction.py +++ b/tests/storage/test_redaction.py @@ -119,11 +119,9 @@ class RedactionTestCase(unittest.HomeserverTestCase): return event def test_redact(self): - self.get_success( - self.inject_room_member(self.room1, self.u_alice, Membership.JOIN) - ) + self.inject_room_member(self.room1, self.u_alice, Membership.JOIN) - msg_event = self.get_success(self.inject_message(self.room1, self.u_alice, "t")) + msg_event = self.inject_message(self.room1, self.u_alice, "t") # Check event has not been redacted: event = self.get_success(self.store.get_event(msg_event.event_id)) @@ -141,9 +139,7 @@ class RedactionTestCase(unittest.HomeserverTestCase): # Redact event reason = "Because I said so" - self.get_success( - self.inject_redaction(self.room1, msg_event.event_id, self.u_alice, reason) - ) + self.inject_redaction(self.room1, msg_event.event_id, self.u_alice, reason) event = self.get_success(self.store.get_event(msg_event.event_id)) @@ -170,14 +166,10 @@ class RedactionTestCase(unittest.HomeserverTestCase): ) def test_redact_join(self): - self.get_success( - self.inject_room_member(self.room1, self.u_alice, Membership.JOIN) - ) + self.inject_room_member(self.room1, self.u_alice, Membership.JOIN) - msg_event = self.get_success( - self.inject_room_member( - self.room1, self.u_bob, Membership.JOIN, extra_content={"blue": "red"} - ) + msg_event = self.inject_room_member( + self.room1, self.u_bob, Membership.JOIN, extra_content={"blue": "red"} ) event = self.get_success(self.store.get_event(msg_event.event_id)) @@ -195,9 +187,7 @@ class RedactionTestCase(unittest.HomeserverTestCase): # Redact event reason = "Because I said so" - self.get_success( - self.inject_redaction(self.room1, msg_event.event_id, self.u_alice, reason) - ) + self.inject_redaction(self.room1, msg_event.event_id, self.u_alice, reason) # Check redaction @@ -311,11 +301,9 @@ class RedactionTestCase(unittest.HomeserverTestCase): def test_redact_censor(self): """Test that a redacted event gets censored in the DB after a month""" - self.get_success( - self.inject_room_member(self.room1, self.u_alice, Membership.JOIN) - ) + self.inject_room_member(self.room1, self.u_alice, Membership.JOIN) - msg_event = self.get_success(self.inject_message(self.room1, self.u_alice, "t")) + msg_event = self.inject_message(self.room1, self.u_alice, "t") # Check event has not been redacted: event = self.get_success(self.store.get_event(msg_event.event_id)) @@ -333,9 +321,7 @@ class RedactionTestCase(unittest.HomeserverTestCase): # Redact event reason = "Because I said so" - self.get_success( - self.inject_redaction(self.room1, msg_event.event_id, self.u_alice, reason) - ) + self.inject_redaction(self.room1, msg_event.event_id, self.u_alice, reason) event = self.get_success(self.store.get_event(msg_event.event_id)) @@ -381,25 +367,19 @@ class RedactionTestCase(unittest.HomeserverTestCase): def test_redact_redaction(self): """Tests that we can redact a redaction and can fetch it again.""" - self.get_success( - self.inject_room_member(self.room1, self.u_alice, Membership.JOIN) + self.inject_room_member(self.room1, self.u_alice, Membership.JOIN) + + msg_event = self.inject_message(self.room1, self.u_alice, "t") + + first_redact_event = self.inject_redaction( + self.room1, msg_event.event_id, self.u_alice, "Redacting message" ) - msg_event = self.get_success(self.inject_message(self.room1, self.u_alice, "t")) - - first_redact_event = self.get_success( - self.inject_redaction( - self.room1, msg_event.event_id, self.u_alice, "Redacting message" - ) - ) - - self.get_success( - self.inject_redaction( - self.room1, - first_redact_event.event_id, - self.u_alice, - "Redacting redaction", - ) + self.inject_redaction( + self.room1, + first_redact_event.event_id, + self.u_alice, + "Redacting redaction", ) # Now lets jump to the future where we have censored the redaction event @@ -414,9 +394,7 @@ class RedactionTestCase(unittest.HomeserverTestCase): def test_store_redacted_redaction(self): """Tests that we can store a redacted redaction.""" - self.get_success( - self.inject_room_member(self.room1, self.u_alice, Membership.JOIN) - ) + self.inject_room_member(self.room1, self.u_alice, Membership.JOIN) builder = self.event_builder_factory.for_room_version( RoomVersions.V1, diff --git a/tests/storage/test_stream.py b/tests/storage/test_stream.py index eaa0d7d749..52e41cdab4 100644 --- a/tests/storage/test_stream.py +++ b/tests/storage/test_stream.py @@ -110,9 +110,7 @@ class PaginationTestCase(HomeserverTestCase): def _filter_messages(self, filter: JsonDict) -> List[EventBase]: """Make a request to /messages with a filter, returns the chunk of events.""" - from_token = self.get_success( - self.hs.get_event_sources().get_current_token_for_pagination() - ) + from_token = self.hs.get_event_sources().get_current_token_for_pagination() events, next_key = self.get_success( self.hs.get_datastores().main.paginate_room_events( diff --git a/tests/test_visibility.py b/tests/test_visibility.py index a02fd4f79a..d0230f9ebb 100644 --- a/tests/test_visibility.py +++ b/tests/test_visibility.py @@ -48,17 +48,15 @@ class FilterEventsForServerTestCase(unittest.HomeserverTestCase): # # before we do that, we persist some other events to act as state. - self.get_success(self._inject_visibility("@admin:hs", "joined")) + self._inject_visibility("@admin:hs", "joined") for i in range(0, 10): - self.get_success(self._inject_room_member("@resident%i:hs" % i)) + self._inject_room_member("@resident%i:hs" % i) events_to_filter = [] for i in range(0, 10): user = "@user%i:%s" % (i, "test_server" if i == 5 else "other_server") - evt = self.get_success( - self._inject_room_member(user, extra_content={"a": "b"}) - ) + evt = self._inject_room_member(user, extra_content={"a": "b"}) events_to_filter.append(evt) filtered = self.get_success( @@ -76,10 +74,10 @@ class FilterEventsForServerTestCase(unittest.HomeserverTestCase): def test_filter_outlier(self) -> None: # outlier events must be returned, for the good of the collective federation - self.get_success(self._inject_room_member("@resident:remote_hs")) - self.get_success(self._inject_visibility("@resident:remote_hs", "joined")) + self._inject_room_member("@resident:remote_hs") + self._inject_visibility("@resident:remote_hs", "joined") - outlier = self.get_success(self._inject_outlier()) + outlier = self._inject_outlier() self.assertEqual( self.get_success( filter_events_for_server(self.storage, "remote_hs", [outlier]) @@ -88,7 +86,7 @@ class FilterEventsForServerTestCase(unittest.HomeserverTestCase): ) # it should also work when there are other events in the list - evt = self.get_success(self._inject_message("@unerased:local_hs")) + evt = self._inject_message("@unerased:local_hs") filtered = self.get_success( filter_events_for_server(self.storage, "remote_hs", [outlier, evt]) @@ -112,19 +110,19 @@ class FilterEventsForServerTestCase(unittest.HomeserverTestCase): # change in the middle of them. events_to_filter = [] - evt = self.get_success(self._inject_message("@unerased:local_hs")) + evt = self._inject_message("@unerased:local_hs") events_to_filter.append(evt) - evt = self.get_success(self._inject_message("@erased:local_hs")) + evt = self._inject_message("@erased:local_hs") events_to_filter.append(evt) - evt = self.get_success(self._inject_room_member("@joiner:remote_hs")) + evt = self._inject_room_member("@joiner:remote_hs") events_to_filter.append(evt) - evt = self.get_success(self._inject_message("@unerased:local_hs")) + evt = self._inject_message("@unerased:local_hs") events_to_filter.append(evt) - evt = self.get_success(self._inject_message("@erased:local_hs")) + evt = self._inject_message("@erased:local_hs") events_to_filter.append(evt) # the erasey user gets erased diff --git a/tests/unittest.py b/tests/unittest.py index cbe215ee83..5b19065c71 100644 --- a/tests/unittest.py +++ b/tests/unittest.py @@ -16,7 +16,6 @@ import gc import hashlib import hmac -import inspect import json import logging import secrets @@ -519,33 +518,23 @@ class HomeserverTestCase(TestCase): self.reactor.pump([by] * 100) def get_success(self, d, by=0.0): - if inspect.isawaitable(d): - d = ensureDeferred(d) - if not isinstance(d, Deferred): - return d + deferred: Deferred[TV] = ensureDeferred(d) self.pump(by=by) - return self.successResultOf(d) + return self.successResultOf(deferred) def get_failure(self, d, exc): """ Run a Deferred and get a Failure from it. The failure must be of the type `exc`. """ - if inspect.isawaitable(d): - d = ensureDeferred(d) - if not isinstance(d, Deferred): - return d + deferred: Deferred[Any] = ensureDeferred(d) self.pump() - return self.failureResultOf(d, exc) + return self.failureResultOf(deferred, exc) def get_success_or_raise(self, d, by=0.0): """Drive deferred to completion and return result or raise exception on failure. """ - - if inspect.isawaitable(d): - deferred = ensureDeferred(d) - if not isinstance(deferred, Deferred): - return d + deferred: Deferred[TV] = ensureDeferred(d) results: list = [] deferred.addBoth(results.append) From f0b03186d96305fd44d74a89bf4230beec0c5c31 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Fri, 1 Apr 2022 17:04:16 +0100 Subject: [PATCH 24/55] Add type hints for `tests/unittest.py`. (#12347) In particular, add type hints for get_success and friends, which are then helpful in a bunch of places. --- changelog.d/12347.misc | 1 + mypy.ini | 1 - tests/handlers/test_e2e_keys.py | 6 +- tests/handlers/test_federation.py | 5 +- tests/handlers/test_oidc.py | 7 +- tests/handlers/test_user_directory.py | 2 + tests/rest/admin/test_media.py | 8 +++ tests/rest/admin/test_user.py | 15 ++-- tests/server.py | 6 +- tests/storage/databases/main/test_lock.py | 8 +-- tests/storage/test_appservice.py | 1 + tests/unittest.py | 85 +++++++++++++++-------- 12 files changed, 97 insertions(+), 48 deletions(-) create mode 100644 changelog.d/12347.misc diff --git a/changelog.d/12347.misc b/changelog.d/12347.misc new file mode 100644 index 0000000000..1f6f584e6d --- /dev/null +++ b/changelog.d/12347.misc @@ -0,0 +1 @@ +Add type annotations for `tests/unittest.py`. diff --git a/mypy.ini b/mypy.ini index 84e6b8646e..85291099ac 100644 --- a/mypy.ini +++ b/mypy.ini @@ -83,7 +83,6 @@ exclude = (?x) |tests/test_server.py |tests/test_state.py |tests/test_terms_auth.py - |tests/unittest.py |tests/util/caches/test_cached_call.py |tests/util/caches/test_deferred_cache.py |tests/util/caches/test_descriptors.py diff --git a/tests/handlers/test_e2e_keys.py b/tests/handlers/test_e2e_keys.py index ac21a28c43..8c74ed1fcf 100644 --- a/tests/handlers/test_e2e_keys.py +++ b/tests/handlers/test_e2e_keys.py @@ -463,8 +463,10 @@ class E2eKeysHandlerTestCase(unittest.HomeserverTestCase): res = e.value.code self.assertEqual(res, 400) - res = self.get_success(self.handler.query_local_devices({local_user: None})) - self.assertDictEqual(res, {local_user: {}}) + query_res = self.get_success( + self.handler.query_local_devices({local_user: None}) + ) + self.assertDictEqual(query_res, {local_user: {}}) def test_upload_signatures(self) -> None: """should check signatures that are uploaded""" diff --git a/tests/handlers/test_federation.py b/tests/handlers/test_federation.py index 0fa5045301..060ba5f517 100644 --- a/tests/handlers/test_federation.py +++ b/tests/handlers/test_federation.py @@ -375,7 +375,8 @@ class FederationTestCase(unittest.FederatingHomeserverTestCase): member_event.signatures = member_event_dict["signatures"] # Add the new member_event to the StateMap - prev_state_map[ + updated_state_map = dict(prev_state_map) + updated_state_map[ (member_event.type, member_event.state_key) ] = member_event.event_id auth_events.append(member_event) @@ -399,7 +400,7 @@ class FederationTestCase(unittest.FederatingHomeserverTestCase): prev_event_ids=message_event_dict["prev_events"], auth_event_ids=self._event_auth_handler.compute_auth_events( builder, - prev_state_map, + updated_state_map, for_verification=False, ), depth=message_event_dict["depth"], diff --git a/tests/handlers/test_oidc.py b/tests/handlers/test_oidc.py index 014815db6e..9684120c70 100644 --- a/tests/handlers/test_oidc.py +++ b/tests/handlers/test_oidc.py @@ -354,10 +354,11 @@ class OidcHandlerTestCase(HomeserverTestCase): req = Mock(spec=["cookies"]) req.cookies = [] - url = self.get_success( - self.provider.handle_redirect_request(req, b"http://client/redirect") + url = urlparse( + self.get_success( + self.provider.handle_redirect_request(req, b"http://client/redirect") + ) ) - url = urlparse(url) auth_endpoint = urlparse(AUTHORIZATION_ENDPOINT) self.assertEqual(url.scheme, auth_endpoint.scheme) diff --git a/tests/handlers/test_user_directory.py b/tests/handlers/test_user_directory.py index 92012cd6f7..c6e501c7be 100644 --- a/tests/handlers/test_user_directory.py +++ b/tests/handlers/test_user_directory.py @@ -351,6 +351,7 @@ class UserDirectoryTestCase(unittest.HomeserverTestCase): self.handler.handle_local_profile_change(regular_user_id, profile_info) ) profile = self.get_success(self.store.get_user_in_directory(regular_user_id)) + assert profile is not None self.assertTrue(profile["display_name"] == display_name) def test_handle_local_profile_change_with_deactivated_user(self) -> None: @@ -369,6 +370,7 @@ class UserDirectoryTestCase(unittest.HomeserverTestCase): # profile is in directory profile = self.get_success(self.store.get_user_in_directory(r_user_id)) + assert profile is not None self.assertTrue(profile["display_name"] == display_name) # deactivate user diff --git a/tests/rest/admin/test_media.py b/tests/rest/admin/test_media.py index 0d47dd0aff..e909e444ac 100644 --- a/tests/rest/admin/test_media.py +++ b/tests/rest/admin/test_media.py @@ -702,6 +702,7 @@ class QuarantineMediaByIDTestCase(unittest.HomeserverTestCase): """ media_info = self.get_success(self.store.get_local_media(self.media_id)) + assert media_info is not None self.assertFalse(media_info["quarantined_by"]) # quarantining @@ -715,6 +716,7 @@ class QuarantineMediaByIDTestCase(unittest.HomeserverTestCase): self.assertFalse(channel.json_body) media_info = self.get_success(self.store.get_local_media(self.media_id)) + assert media_info is not None self.assertTrue(media_info["quarantined_by"]) # remove from quarantine @@ -728,6 +730,7 @@ class QuarantineMediaByIDTestCase(unittest.HomeserverTestCase): self.assertFalse(channel.json_body) media_info = self.get_success(self.store.get_local_media(self.media_id)) + assert media_info is not None self.assertFalse(media_info["quarantined_by"]) def test_quarantine_protected_media(self) -> None: @@ -740,6 +743,7 @@ class QuarantineMediaByIDTestCase(unittest.HomeserverTestCase): # verify protection media_info = self.get_success(self.store.get_local_media(self.media_id)) + assert media_info is not None self.assertTrue(media_info["safe_from_quarantine"]) # quarantining @@ -754,6 +758,7 @@ class QuarantineMediaByIDTestCase(unittest.HomeserverTestCase): # verify that is not in quarantine media_info = self.get_success(self.store.get_local_media(self.media_id)) + assert media_info is not None self.assertFalse(media_info["quarantined_by"]) @@ -830,6 +835,7 @@ class ProtectMediaByIDTestCase(unittest.HomeserverTestCase): """ media_info = self.get_success(self.store.get_local_media(self.media_id)) + assert media_info is not None self.assertFalse(media_info["safe_from_quarantine"]) # protect @@ -843,6 +849,7 @@ class ProtectMediaByIDTestCase(unittest.HomeserverTestCase): self.assertFalse(channel.json_body) media_info = self.get_success(self.store.get_local_media(self.media_id)) + assert media_info is not None self.assertTrue(media_info["safe_from_quarantine"]) # unprotect @@ -856,6 +863,7 @@ class ProtectMediaByIDTestCase(unittest.HomeserverTestCase): self.assertFalse(channel.json_body) media_info = self.get_success(self.store.get_local_media(self.media_id)) + assert media_info is not None self.assertFalse(media_info["safe_from_quarantine"]) diff --git a/tests/rest/admin/test_user.py b/tests/rest/admin/test_user.py index bef911d5df..0cdf1dec40 100644 --- a/tests/rest/admin/test_user.py +++ b/tests/rest/admin/test_user.py @@ -1590,10 +1590,9 @@ class UserRestTestCase(unittest.HomeserverTestCase): self.assertEqual("email", channel.json_body["threepids"][0]["medium"]) self.assertEqual("bob@bob.bob", channel.json_body["threepids"][0]["address"]) - pushers = self.get_success( - self.store.get_pushers_by({"user_name": "@bob:test"}) + pushers = list( + self.get_success(self.store.get_pushers_by({"user_name": "@bob:test"})) ) - pushers = list(pushers) self.assertEqual(len(pushers), 1) self.assertEqual("@bob:test", pushers[0].user_name) @@ -1632,10 +1631,9 @@ class UserRestTestCase(unittest.HomeserverTestCase): self.assertEqual("email", channel.json_body["threepids"][0]["medium"]) self.assertEqual("bob@bob.bob", channel.json_body["threepids"][0]["address"]) - pushers = self.get_success( - self.store.get_pushers_by({"user_name": "@bob:test"}) + pushers = list( + self.get_success(self.store.get_pushers_by({"user_name": "@bob:test"})) ) - pushers = list(pushers) self.assertEqual(len(pushers), 0) def test_set_password(self) -> None: @@ -2144,6 +2142,7 @@ class UserRestTestCase(unittest.HomeserverTestCase): # is in user directory profile = self.get_success(self.store.get_user_in_directory(self.other_user)) + assert profile is not None self.assertTrue(profile["display_name"] == "User") # Deactivate user @@ -2711,6 +2710,7 @@ class PushersRestTestCase(unittest.HomeserverTestCase): user_tuple = self.get_success( self.store.get_user_by_access_token(other_user_token) ) + assert user_tuple is not None token_id = user_tuple.token_id self.get_success( @@ -3676,6 +3676,7 @@ class ShadowBanRestTestCase(unittest.HomeserverTestCase): # The user starts off as not shadow-banned. other_user_token = self.login("user", "pass") result = self.get_success(self.store.get_user_by_access_token(other_user_token)) + assert result is not None self.assertFalse(result.shadow_banned) channel = self.make_request("POST", self.url, access_token=self.admin_user_tok) @@ -3684,6 +3685,7 @@ class ShadowBanRestTestCase(unittest.HomeserverTestCase): # Ensure the user is shadow-banned (and the cache was cleared). result = self.get_success(self.store.get_user_by_access_token(other_user_token)) + assert result is not None self.assertTrue(result.shadow_banned) # Un-shadow-ban the user. @@ -3695,6 +3697,7 @@ class ShadowBanRestTestCase(unittest.HomeserverTestCase): # Ensure the user is no longer shadow-banned (and the cache was cleared). result = self.get_success(self.store.get_user_by_access_token(other_user_token)) + assert result is not None self.assertFalse(result.shadow_banned) diff --git a/tests/server.py b/tests/server.py index 6ce2a17bf4..aaa5ca3e74 100644 --- a/tests/server.py +++ b/tests/server.py @@ -22,7 +22,6 @@ import warnings from collections import deque from io import SEEK_END, BytesIO from typing import ( - AnyStr, Callable, Dict, Iterable, @@ -86,6 +85,9 @@ from tests.utils import ( logger = logging.getLogger(__name__) +# the type of thing that can be passed into `make_request` in the headers list +CustomHeaderType = Tuple[Union[str, bytes], Union[str, bytes]] + class TimedOutException(Exception): """ @@ -260,7 +262,7 @@ def make_request( federation_auth_origin: Optional[bytes] = None, content_is_form: bool = False, await_result: bool = True, - custom_headers: Optional[Iterable[Tuple[AnyStr, AnyStr]]] = None, + custom_headers: Optional[Iterable[CustomHeaderType]] = None, client_ip: str = "127.0.0.1", ) -> FakeChannel: """ diff --git a/tests/storage/databases/main/test_lock.py b/tests/storage/databases/main/test_lock.py index 3ac4646969..74c6224eb6 100644 --- a/tests/storage/databases/main/test_lock.py +++ b/tests/storage/databases/main/test_lock.py @@ -28,7 +28,7 @@ class LockTestCase(unittest.HomeserverTestCase): """ # First to acquire this lock, so it should complete lock = self.get_success(self.store.try_acquire_lock("name", "key")) - self.assertIsNotNone(lock) + assert lock is not None # Enter the context manager self.get_success(lock.__aenter__()) @@ -45,7 +45,7 @@ class LockTestCase(unittest.HomeserverTestCase): # We can now acquire the lock again. lock3 = self.get_success(self.store.try_acquire_lock("name", "key")) - self.assertIsNotNone(lock3) + assert lock3 is not None self.get_success(lock3.__aenter__()) self.get_success(lock3.__aexit__(None, None, None)) @@ -53,7 +53,7 @@ class LockTestCase(unittest.HomeserverTestCase): """Test that we don't time out locks while they're still active""" lock = self.get_success(self.store.try_acquire_lock("name", "key")) - self.assertIsNotNone(lock) + assert lock is not None self.get_success(lock.__aenter__()) @@ -69,7 +69,7 @@ class LockTestCase(unittest.HomeserverTestCase): """Test that we time out locks if they're not updated for ages""" lock = self.get_success(self.store.try_acquire_lock("name", "key")) - self.assertIsNotNone(lock) + assert lock is not None self.get_success(lock.__aenter__()) diff --git a/tests/storage/test_appservice.py b/tests/storage/test_appservice.py index 08078d38e2..1bf93e79a7 100644 --- a/tests/storage/test_appservice.py +++ b/tests/storage/test_appservice.py @@ -358,6 +358,7 @@ class ApplicationServiceTransactionStoreTestCase(unittest.HomeserverTestCase): self.get_success(self._insert_txn(service.id, 12, other_events)) txn = self.get_success(self.store.get_oldest_unsent_txn(service)) + assert txn is not None self.assertEqual(service, txn.service) self.assertEqual(10, txn.id) self.assertEqual(events, txn.events) diff --git a/tests/unittest.py b/tests/unittest.py index 5b19065c71..9afa68c164 100644 --- a/tests/unittest.py +++ b/tests/unittest.py @@ -22,10 +22,11 @@ import secrets import time from typing import ( Any, - AnyStr, + Awaitable, Callable, ClassVar, Dict, + Generic, Iterable, List, Optional, @@ -39,6 +40,7 @@ from unittest.mock import Mock, patch import canonicaljson import signedjson.key import unpaddedbase64 +from typing_extensions import Protocol from twisted.internet.defer import Deferred, ensureDeferred from twisted.python.failure import Failure @@ -49,7 +51,7 @@ from twisted.web.resource import Resource from twisted.web.server import Request from synapse import events -from synapse.api.constants import EventTypes, Membership +from synapse.api.constants import EventTypes from synapse.api.room_versions import KNOWN_ROOM_VERSIONS, RoomVersion from synapse.config.homeserver import HomeServerConfig from synapse.config.server import DEFAULT_ROOM_VERSION @@ -70,7 +72,13 @@ from synapse.types import JsonDict, UserID, create_requester from synapse.util import Clock from synapse.util.httpresourcetree import create_resource_tree -from tests.server import FakeChannel, get_clock, make_request, setup_test_homeserver +from tests.server import ( + CustomHeaderType, + FakeChannel, + get_clock, + make_request, + setup_test_homeserver, +) from tests.test_utils import event_injection, setup_awaitable_errors from tests.test_utils.logging_setup import setup_logging from tests.utils import default_config, setupdb @@ -78,6 +86,17 @@ from tests.utils import default_config, setupdb setupdb() setup_logging() +TV = TypeVar("TV") +_ExcType = TypeVar("_ExcType", bound=BaseException, covariant=True) + + +class _TypedFailure(Generic[_ExcType], Protocol): + """Extension to twisted.Failure, where the 'value' has a certain type.""" + + @property + def value(self) -> _ExcType: + ... + def around(target): """A CLOS-style 'around' modifier, which wraps the original method of the @@ -276,6 +295,7 @@ class HomeserverTestCase(TestCase): if hasattr(self, "user_id"): if self.hijack_auth: + assert self.helper.auth_user_id is not None # We need a valid token ID to satisfy foreign key constraints. token_id = self.get_success( @@ -288,6 +308,7 @@ class HomeserverTestCase(TestCase): ) async def get_user_by_access_token(token=None, allow_guest=False): + assert self.helper.auth_user_id is not None return { "user": UserID.from_string(self.helper.auth_user_id), "token_id": token_id, @@ -295,6 +316,7 @@ class HomeserverTestCase(TestCase): } async def get_user_by_req(request, allow_guest=False, rights="access"): + assert self.helper.auth_user_id is not None return create_requester( UserID.from_string(self.helper.auth_user_id), token_id, @@ -311,7 +333,7 @@ class HomeserverTestCase(TestCase): ) if self.needs_threadpool: - self.reactor.threadpool = ThreadPool() + self.reactor.threadpool = ThreadPool() # type: ignore[assignment] self.addCleanup(self.reactor.threadpool.stop) self.reactor.threadpool.start() @@ -426,7 +448,7 @@ class HomeserverTestCase(TestCase): federation_auth_origin: Optional[bytes] = None, content_is_form: bool = False, await_result: bool = True, - custom_headers: Optional[Iterable[Tuple[AnyStr, AnyStr]]] = None, + custom_headers: Optional[Iterable[CustomHeaderType]] = None, client_ip: str = "127.0.0.1", ) -> FakeChannel: """ @@ -511,30 +533,36 @@ class HomeserverTestCase(TestCase): return hs - def pump(self, by=0.0): + def pump(self, by: float = 0.0) -> None: """ Pump the reactor enough that Deferreds will fire. """ self.reactor.pump([by] * 100) - def get_success(self, d, by=0.0): - deferred: Deferred[TV] = ensureDeferred(d) + def get_success( + self, + d: Awaitable[TV], + by: float = 0.0, + ) -> TV: + deferred: Deferred[TV] = ensureDeferred(d) # type: ignore[arg-type] self.pump(by=by) return self.successResultOf(deferred) - def get_failure(self, d, exc): + def get_failure( + self, d: Awaitable[Any], exc: Type[_ExcType] + ) -> _TypedFailure[_ExcType]: """ Run a Deferred and get a Failure from it. The failure must be of the type `exc`. """ - deferred: Deferred[Any] = ensureDeferred(d) + deferred: Deferred[Any] = ensureDeferred(d) # type: ignore[arg-type] self.pump() return self.failureResultOf(deferred, exc) - def get_success_or_raise(self, d, by=0.0): + def get_success_or_raise(self, d: Awaitable[TV], by: float = 0.0) -> TV: """Drive deferred to completion and return result or raise exception on failure. """ - deferred: Deferred[TV] = ensureDeferred(d) + deferred: Deferred[TV] = ensureDeferred(d) # type: ignore[arg-type] results: list = [] deferred.addBoth(results.append) @@ -642,11 +670,11 @@ class HomeserverTestCase(TestCase): def login( self, - username, - password, - device_id=None, - custom_headers: Optional[Iterable[Tuple[AnyStr, AnyStr]]] = None, - ): + username: str, + password: str, + device_id: Optional[str] = None, + custom_headers: Optional[Iterable[CustomHeaderType]] = None, + ) -> str: """ Log in a user, and get an access token. Requires the Login API be registered. @@ -668,18 +696,22 @@ class HomeserverTestCase(TestCase): return access_token def create_and_send_event( - self, room_id, user, soft_failed=False, prev_event_ids=None - ): + self, + room_id: str, + user: UserID, + soft_failed: bool = False, + prev_event_ids: Optional[List[str]] = None, + ) -> str: """ Create and send an event. Args: - soft_failed (bool): Whether to create a soft failed event or not - prev_event_ids (list[str]|None): Explicitly set the prev events, + soft_failed: Whether to create a soft failed event or not + prev_event_ids: Explicitly set the prev events, or if None just use the default Returns: - str: The new event's ID. + The new event's ID. """ event_creator = self.hs.get_event_creation_handler() requester = create_requester(user) @@ -706,7 +738,7 @@ class HomeserverTestCase(TestCase): return event.event_id - def inject_room_member(self, room: str, user: str, membership: Membership) -> None: + def inject_room_member(self, room: str, user: str, membership: str) -> None: """ Inject a membership event into a room. @@ -766,7 +798,7 @@ class FederatingHomeserverTestCase(HomeserverTestCase): path: str, content: Optional[JsonDict] = None, await_result: bool = True, - custom_headers: Optional[Iterable[Tuple[AnyStr, AnyStr]]] = None, + custom_headers: Optional[Iterable[CustomHeaderType]] = None, client_ip: str = "127.0.0.1", ) -> FakeChannel: """Make an inbound signed federation request to this server @@ -799,7 +831,7 @@ class FederatingHomeserverTestCase(HomeserverTestCase): self.site, method=method, path=path, - content=content, + content=content or "", shorthand=False, await_result=await_result, custom_headers=custom_headers, @@ -878,9 +910,6 @@ def override_config(extra_config): return decorator -TV = TypeVar("TV") - - def skip_unless(condition: bool, reason: str) -> Callable[[TV], TV]: """A test decorator which will skip the decorated test unless a condition is set From 80839a44f1752669c8c4e0de440b1e49ba909aeb Mon Sep 17 00:00:00 2001 From: David Robertson Date: Sun, 3 Apr 2022 21:21:08 +0100 Subject: [PATCH 25/55] Remove more dead/broken dev scripts (#12355) --- changelog.d/12355.misc | 1 + mypy.ini | 3 - scripts-dev/definitions.py | 208 ------------------------------------ scripts-dev/hash_history.py | 81 -------------- scripts-dev/tail-synapse.py | 67 ------------ 5 files changed, 1 insertion(+), 359 deletions(-) create mode 100644 changelog.d/12355.misc delete mode 100755 scripts-dev/definitions.py delete mode 100644 scripts-dev/hash_history.py delete mode 100644 scripts-dev/tail-synapse.py diff --git a/changelog.d/12355.misc b/changelog.d/12355.misc new file mode 100644 index 0000000000..b1dd1e2696 --- /dev/null +++ b/changelog.d/12355.misc @@ -0,0 +1 @@ +Remove broken and unused development scripts. \ No newline at end of file diff --git a/mypy.ini b/mypy.ini index 85291099ac..ac2a5c753d 100644 --- a/mypy.ini +++ b/mypy.ini @@ -24,11 +24,8 @@ files = exclude = (?x) ^( |scripts-dev/build_debian_packages.py - |scripts-dev/definitions.py |scripts-dev/federation_client.py - |scripts-dev/hash_history.py |scripts-dev/release.py - |scripts-dev/tail-synapse.py |synapse/_scripts/export_signing_key.py |synapse/_scripts/move_remote_media_to_new_store.py diff --git a/scripts-dev/definitions.py b/scripts-dev/definitions.py deleted file mode 100755 index c82ddd9677..0000000000 --- a/scripts-dev/definitions.py +++ /dev/null @@ -1,208 +0,0 @@ -#! /usr/bin/python - -import argparse -import ast -import os -import re -import sys - -import yaml - - -class DefinitionVisitor(ast.NodeVisitor): - def __init__(self): - super().__init__() - self.functions = {} - self.classes = {} - self.names = {} - self.attrs = set() - self.definitions = { - "def": self.functions, - "class": self.classes, - "names": self.names, - "attrs": self.attrs, - } - - def visit_Name(self, node): - self.names.setdefault(type(node.ctx).__name__, set()).add(node.id) - - def visit_Attribute(self, node): - self.attrs.add(node.attr) - for child in ast.iter_child_nodes(node): - self.visit(child) - - def visit_ClassDef(self, node): - visitor = DefinitionVisitor() - self.classes[node.name] = visitor.definitions - for child in ast.iter_child_nodes(node): - visitor.visit(child) - - def visit_FunctionDef(self, node): - visitor = DefinitionVisitor() - self.functions[node.name] = visitor.definitions - for child in ast.iter_child_nodes(node): - visitor.visit(child) - - -def non_empty(defs): - functions = {name: non_empty(f) for name, f in defs["def"].items()} - classes = {name: non_empty(f) for name, f in defs["class"].items()} - result = {} - if functions: - result["def"] = functions - if classes: - result["class"] = classes - names = defs["names"] - uses = [] - for name in names.get("Load", ()): - if name not in names.get("Param", ()) and name not in names.get("Store", ()): - uses.append(name) - uses.extend(defs["attrs"]) - if uses: - result["uses"] = uses - result["names"] = names - result["attrs"] = defs["attrs"] - return result - - -def definitions_in_code(input_code): - input_ast = ast.parse(input_code) - visitor = DefinitionVisitor() - visitor.visit(input_ast) - definitions = non_empty(visitor.definitions) - return definitions - - -def definitions_in_file(filepath): - with open(filepath) as f: - return definitions_in_code(f.read()) - - -def defined_names(prefix, defs, names): - for name, funcs in defs.get("def", {}).items(): - names.setdefault(name, {"defined": []})["defined"].append(prefix + name) - defined_names(prefix + name + ".", funcs, names) - - for name, funcs in defs.get("class", {}).items(): - names.setdefault(name, {"defined": []})["defined"].append(prefix + name) - defined_names(prefix + name + ".", funcs, names) - - -def used_names(prefix, item, defs, names): - for name, funcs in defs.get("def", {}).items(): - used_names(prefix + name + ".", name, funcs, names) - - for name, funcs in defs.get("class", {}).items(): - used_names(prefix + name + ".", name, funcs, names) - - path = prefix.rstrip(".") - for used in defs.get("uses", ()): - if used in names: - if item: - names[item].setdefault("uses", []).append(used) - names[used].setdefault("used", {}).setdefault(item, []).append(path) - - -if __name__ == "__main__": - - parser = argparse.ArgumentParser(description="Find definitions.") - parser.add_argument( - "--unused", action="store_true", help="Only list unused definitions" - ) - parser.add_argument( - "--ignore", action="append", metavar="REGEXP", help="Ignore a pattern" - ) - parser.add_argument( - "--pattern", action="append", metavar="REGEXP", help="Search for a pattern" - ) - parser.add_argument( - "directories", - nargs="+", - metavar="DIR", - help="Directories to search for definitions", - ) - parser.add_argument( - "--referrers", - default=0, - type=int, - help="Include referrers up to the given depth", - ) - parser.add_argument( - "--referred", - default=0, - type=int, - help="Include referred down to the given depth", - ) - parser.add_argument( - "--format", default="yaml", help="Output format, one of 'yaml' or 'dot'" - ) - args = parser.parse_args() - - definitions = {} - for directory in args.directories: - for root, _, files in os.walk(directory): - for filename in files: - if filename.endswith(".py"): - filepath = os.path.join(root, filename) - definitions[filepath] = definitions_in_file(filepath) - - names = {} - for filepath, defs in definitions.items(): - defined_names(filepath + ":", defs, names) - - for filepath, defs in definitions.items(): - used_names(filepath + ":", None, defs, names) - - patterns = [re.compile(pattern) for pattern in args.pattern or ()] - ignore = [re.compile(pattern) for pattern in args.ignore or ()] - - result = {} - for name, definition in names.items(): - if patterns and not any(pattern.match(name) for pattern in patterns): - continue - if ignore and any(pattern.match(name) for pattern in ignore): - continue - if args.unused and definition.get("used"): - continue - result[name] = definition - - referrer_depth = args.referrers - referrers = set() - while referrer_depth: - referrer_depth -= 1 - for entry in result.values(): - for used_by in entry.get("used", ()): - referrers.add(used_by) - for name, definition in names.items(): - if name not in referrers: - continue - if ignore and any(pattern.match(name) for pattern in ignore): - continue - result[name] = definition - - referred_depth = args.referred - referred = set() - while referred_depth: - referred_depth -= 1 - for entry in result.values(): - for uses in entry.get("uses", ()): - referred.add(uses) - for name, definition in names.items(): - if name not in referred: - continue - if ignore and any(pattern.match(name) for pattern in ignore): - continue - result[name] = definition - - if args.format == "yaml": - yaml.dump(result, sys.stdout, default_flow_style=False) - elif args.format == "dot": - print("digraph {") - for name, entry in result.items(): - print(name) - for used_by in entry.get("used", ()): - if used_by in result: - print(used_by, "->", name) - print("}") - else: - raise ValueError("Unknown format %r" % (args.format)) diff --git a/scripts-dev/hash_history.py b/scripts-dev/hash_history.py deleted file mode 100644 index 8d6c3d24db..0000000000 --- a/scripts-dev/hash_history.py +++ /dev/null @@ -1,81 +0,0 @@ -import sqlite3 -import sys - -from unpaddedbase64 import decode_base64, encode_base64 - -from synapse.crypto.event_signing import ( - add_event_pdu_content_hash, - compute_pdu_event_reference_hash, -) -from synapse.federation.units import Pdu -from synapse.storage._base import SQLBaseStore -from synapse.storage.pdu import PduStore -from synapse.storage.signatures import SignatureStore - - -class Store: - _get_pdu_tuples = PduStore.__dict__["_get_pdu_tuples"] - _get_pdu_content_hashes_txn = SignatureStore.__dict__["_get_pdu_content_hashes_txn"] - _get_prev_pdu_hashes_txn = SignatureStore.__dict__["_get_prev_pdu_hashes_txn"] - _get_pdu_origin_signatures_txn = SignatureStore.__dict__[ - "_get_pdu_origin_signatures_txn" - ] - _store_pdu_content_hash_txn = SignatureStore.__dict__["_store_pdu_content_hash_txn"] - _store_pdu_reference_hash_txn = SignatureStore.__dict__[ - "_store_pdu_reference_hash_txn" - ] - _store_prev_pdu_hash_txn = SignatureStore.__dict__["_store_prev_pdu_hash_txn"] - simple_insert_txn = SQLBaseStore.__dict__["simple_insert_txn"] - - -store = Store() - - -def select_pdus(cursor): - cursor.execute("SELECT pdu_id, origin FROM pdus ORDER BY depth ASC") - - ids = cursor.fetchall() - - pdu_tuples = store._get_pdu_tuples(cursor, ids) - - pdus = [Pdu.from_pdu_tuple(p) for p in pdu_tuples] - - reference_hashes = {} - - for pdu in pdus: - try: - if pdu.prev_pdus: - print("PROCESS", pdu.pdu_id, pdu.origin, pdu.prev_pdus) - for pdu_id, origin, hashes in pdu.prev_pdus: - ref_alg, ref_hsh = reference_hashes[(pdu_id, origin)] - hashes[ref_alg] = encode_base64(ref_hsh) - store._store_prev_pdu_hash_txn( - cursor, pdu.pdu_id, pdu.origin, pdu_id, origin, ref_alg, ref_hsh - ) - print("SUCCESS", pdu.pdu_id, pdu.origin, pdu.prev_pdus) - pdu = add_event_pdu_content_hash(pdu) - ref_alg, ref_hsh = compute_pdu_event_reference_hash(pdu) - reference_hashes[(pdu.pdu_id, pdu.origin)] = (ref_alg, ref_hsh) - store._store_pdu_reference_hash_txn( - cursor, pdu.pdu_id, pdu.origin, ref_alg, ref_hsh - ) - - for alg, hsh_base64 in pdu.hashes.items(): - print(alg, hsh_base64) - store._store_pdu_content_hash_txn( - cursor, pdu.pdu_id, pdu.origin, alg, decode_base64(hsh_base64) - ) - - except Exception: - print("FAILED_", pdu.pdu_id, pdu.origin, pdu.prev_pdus) - - -def main(): - conn = sqlite3.connect(sys.argv[1]) - cursor = conn.cursor() - select_pdus(cursor) - conn.commit() - - -if __name__ == "__main__": - main() diff --git a/scripts-dev/tail-synapse.py b/scripts-dev/tail-synapse.py deleted file mode 100644 index 44e3a6dbf1..0000000000 --- a/scripts-dev/tail-synapse.py +++ /dev/null @@ -1,67 +0,0 @@ -import collections -import json -import sys -import time - -import requests - -Entry = collections.namedtuple("Entry", "name position rows") - -ROW_TYPES = {} - - -def row_type_for_columns(name, column_names): - column_names = tuple(column_names) - row_type = ROW_TYPES.get((name, column_names)) - if row_type is None: - row_type = collections.namedtuple(name, column_names) - ROW_TYPES[(name, column_names)] = row_type - return row_type - - -def parse_response(content): - streams = json.loads(content) - result = {} - for name, value in streams.items(): - row_type = row_type_for_columns(name, value["field_names"]) - position = value["position"] - rows = [row_type(*row) for row in value["rows"]] - result[name] = Entry(name, position, rows) - return result - - -def replicate(server, streams): - return parse_response( - requests.get( - server + "/_synapse/replication", verify=False, params=streams - ).content - ) - - -def main(): - server = sys.argv[1] - - streams = None - while not streams: - try: - streams = { - row.name: row.position - for row in replicate(server, {"streams": "-1"})["streams"].rows - } - except requests.exceptions.ConnectionError: - time.sleep(0.1) - - while True: - try: - results = replicate(server, streams) - except Exception: - sys.stdout.write("connection_lost(" + repr(streams) + ")\n") - break - for update in results.values(): - for row in update.rows: - sys.stdout.write(repr(row) + "\n") - streams[update.name] = update.position - - -if __name__ == "__main__": - main() From 5c9e39e6192e952ba8a5bb8e5485bc6067f91699 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Mon, 4 Apr 2022 15:25:20 +0100 Subject: [PATCH 26/55] Track device list updates per room. (#12321) This is a first step in dealing with #7721. The idea is basically that rather than calculating the full set of users a device list update needs to be sent to up front, we instead simply record the rooms the user was in at the time of the change. This will allow a few things: 1. we can defer calculating the set of remote servers that need to be poked about the change; and 2. during `/sync` and `/keys/changes` we can avoid also avoid calculating users who share rooms with other users, and instead just look at the rooms that have changed. However, care needs to be taken to correctly handle server downgrades. As such this PR writes to both `device_lists_changes_in_room` and the `device_lists_outbound_pokes` table synchronously. In a future release we can then bump the database schema compat version to `69` and then we can assume that the new `device_lists_changes_in_room` exists and is handled. There is a temporary option to disable writing to `device_lists_outbound_pokes` synchronously, allowing us to test the new code path does work (and by implication upgrading to a future release and downgrading to this one will work correctly). Note: Ideally we'd do the calculation of room to servers on a worker (e.g. the background worker), but currently only master can write to the `device_list_outbound_pokes` table. --- changelog.d/12321.misc | 1 + synapse/_scripts/synapse_port_db.py | 1 + synapse/config/server.py | 8 + synapse/handlers/device.py | 132 ++++++++++- synapse/replication/slave/storage/devices.py | 1 + synapse/storage/databases/main/__init__.py | 1 + synapse/storage/databases/main/devices.py | 217 ++++++++++++++++-- synapse/storage/schema/__init__.py | 1 + .../69/01device_list_oubound_by_room.sql | 38 +++ tests/federation/test_federation_sender.py | 23 +- tests/storage/test_devices.py | 14 +- 11 files changed, 390 insertions(+), 47 deletions(-) create mode 100644 changelog.d/12321.misc create mode 100644 synapse/storage/schema/main/delta/69/01device_list_oubound_by_room.sql diff --git a/changelog.d/12321.misc b/changelog.d/12321.misc new file mode 100644 index 0000000000..200e7c44fe --- /dev/null +++ b/changelog.d/12321.misc @@ -0,0 +1 @@ +Add ground work for speeding up device list updates for users in large numbers of rooms. diff --git a/synapse/_scripts/synapse_port_db.py b/synapse/_scripts/synapse_port_db.py index c38666da18..6324df883b 100755 --- a/synapse/_scripts/synapse_port_db.py +++ b/synapse/_scripts/synapse_port_db.py @@ -97,6 +97,7 @@ BOOLEAN_COLUMNS = { "users": ["shadow_banned"], "e2e_fallback_keys_json": ["used"], "access_tokens": ["used"], + "device_lists_changes_in_room": ["converted_to_destinations"], } diff --git a/synapse/config/server.py b/synapse/config/server.py index 0f90302c95..b3a9e50752 100644 --- a/synapse/config/server.py +++ b/synapse/config/server.py @@ -680,6 +680,14 @@ class ServerConfig(Config): config.get("use_account_validity_in_account_status") or False ) + # This is a temporary option that enables fully using the new + # `device_lists_changes_in_room` without the backwards compat code. This + # is primarily for testing. If enabled the server should *not* be + # downgraded, as it may lead to missing device list updates. + self.use_new_device_lists_changes_in_room = ( + config.get("use_new_device_lists_changes_in_room") or False + ) + self.rooms_to_exclude_from_sync: List[str] = ( config.get("exclude_rooms_from_sync") or [] ) diff --git a/synapse/handlers/device.py b/synapse/handlers/device.py index d5ccaa0c37..c710c02cf9 100644 --- a/synapse/handlers/device.py +++ b/synapse/handlers/device.py @@ -37,7 +37,10 @@ from synapse.api.errors import ( SynapseError, ) from synapse.logging.opentracing import log_kv, set_tag, trace -from synapse.metrics.background_process_metrics import run_as_background_process +from synapse.metrics.background_process_metrics import ( + run_as_background_process, + wrap_as_background_process, +) from synapse.types import ( JsonDict, StreamToken, @@ -278,6 +281,22 @@ class DeviceHandler(DeviceWorkerHandler): hs.get_distributor().observe("user_left_room", self.user_left_room) + # Whether `_handle_new_device_update_async` is currently processing. + self._handle_new_device_update_is_processing = False + + # If a new device update may have happened while the loop was + # processing. + self._handle_new_device_update_new_data = False + + # On start up check if there are any updates pending. + hs.get_reactor().callWhenRunning(self._handle_new_device_update_async) + + # Used to decide if we calculate outbound pokes up front or not. By + # default we do to allow safely downgrading Synapse. + self.use_new_device_lists_changes_in_room = ( + hs.config.server.use_new_device_lists_changes_in_room + ) + def _check_device_name_length(self, name: Optional[str]) -> None: """ Checks whether a device name is longer than the maximum allowed length. @@ -469,19 +488,26 @@ class DeviceHandler(DeviceWorkerHandler): # No changes to notify about, so this is a no-op. return - users_who_share_room = await self.store.get_users_who_share_room_with_user( - user_id - ) + room_ids = await self.store.get_rooms_for_user(user_id) - hosts: Set[str] = set() - if self.hs.is_mine_id(user_id): - hosts.update(get_domain_from_id(u) for u in users_who_share_room) - hosts.discard(self.server_name) + hosts: Optional[Set[str]] = None + if not self.use_new_device_lists_changes_in_room: + hosts = set() - set_tag("target_hosts", hosts) + if self.hs.is_mine_id(user_id): + for room_id in room_ids: + joined_users = await self.store.get_users_in_room(room_id) + hosts.update(get_domain_from_id(u) for u in joined_users) + + set_tag("target_hosts", hosts) + + hosts.discard(self.server_name) position = await self.store.add_device_change_to_streams( - user_id, device_ids, list(hosts) + user_id, + device_ids, + hosts=hosts, + room_ids=room_ids, ) if not position: @@ -495,9 +521,12 @@ class DeviceHandler(DeviceWorkerHandler): # specify the user ID too since the user should always get their own device list # updates, even if they aren't in any rooms. - users_to_notify = users_who_share_room.union({user_id}) + self.notifier.on_new_event( + "device_list_key", position, users={user_id}, rooms=room_ids + ) - self.notifier.on_new_event("device_list_key", position, users=users_to_notify) + # We may need to do some processing asynchronously. + self._handle_new_device_update_async() if hosts: logger.info( @@ -614,6 +643,85 @@ class DeviceHandler(DeviceWorkerHandler): return {"success": True} + @wrap_as_background_process("_handle_new_device_update_async") + async def _handle_new_device_update_async(self) -> None: + """Called when we have a new local device list update that we need to + send out over federation. + + This happens in the background so as not to block the original request + that generated the device update. + """ + if self._handle_new_device_update_is_processing: + self._handle_new_device_update_new_data = True + return + + self._handle_new_device_update_is_processing = True + + # The stream ID we processed previous iteration (if any), and the set of + # hosts we've already poked about for this update. This is so that we + # don't poke the same remote server about the same update repeatedly. + current_stream_id = None + hosts_already_sent_to: Set[str] = set() + + try: + while True: + self._handle_new_device_update_new_data = False + rows = await self.store.get_uncoverted_outbound_room_pokes() + if not rows: + # If the DB returned nothing then there is nothing left to + # do, *unless* a new device list update happened during the + # DB query. + if self._handle_new_device_update_new_data: + continue + else: + return + + for user_id, device_id, room_id, stream_id, opentracing_context in rows: + joined_user_ids = await self.store.get_users_in_room(room_id) + hosts = {get_domain_from_id(u) for u in joined_user_ids} + hosts.discard(self.server_name) + + # Check if we've already sent this update to some hosts + if current_stream_id == stream_id: + hosts -= hosts_already_sent_to + + await self.store.add_device_list_outbound_pokes( + user_id=user_id, + device_id=device_id, + room_id=room_id, + stream_id=stream_id, + hosts=hosts, + context=opentracing_context, + ) + + # Notify replication that we've updated the device list stream. + self.notifier.notify_replication() + + if hosts: + logger.info( + "Sending device list update notif for %r to: %r", + user_id, + hosts, + ) + for host in hosts: + self.federation_sender.send_device_messages( + host, immediate=False + ) + log_kv( + {"message": "sent device update to host", "host": host} + ) + + if current_stream_id != stream_id: + # Clear the set of hosts we've already sent to as we're + # processing a new update. + hosts_already_sent_to.clear() + + hosts_already_sent_to.update(hosts) + current_stream_id = stream_id + + finally: + self._handle_new_device_update_is_processing = False + def _update_device_from_client_ips( device: JsonDict, client_ips: Mapping[Tuple[str, str], Mapping[str, Any]] diff --git a/synapse/replication/slave/storage/devices.py b/synapse/replication/slave/storage/devices.py index 0ffd34f1da..f040e33bfb 100644 --- a/synapse/replication/slave/storage/devices.py +++ b/synapse/replication/slave/storage/devices.py @@ -44,6 +44,7 @@ class SlavedDeviceStore(EndToEndKeyWorkerStore, DeviceWorkerStore, BaseSlavedSto extra_tables=[ ("user_signature_stream", "stream_id"), ("device_lists_outbound_pokes", "stream_id"), + ("device_lists_changes_in_room", "stream_id"), ], ) device_list_max = self._device_list_id_gen.get_current_token() diff --git a/synapse/storage/databases/main/__init__.py b/synapse/storage/databases/main/__init__.py index 1ea0b2aa6f..cdbe3872fa 100644 --- a/synapse/storage/databases/main/__init__.py +++ b/synapse/storage/databases/main/__init__.py @@ -146,6 +146,7 @@ class DataStore( extra_tables=[ ("user_signature_stream", "stream_id"), ("device_lists_outbound_pokes", "stream_id"), + ("device_lists_changes_in_room", "stream_id"), ], ) diff --git a/synapse/storage/databases/main/devices.py b/synapse/storage/databases/main/devices.py index f08f7834d3..07eea4b3d2 100644 --- a/synapse/storage/databases/main/devices.py +++ b/synapse/storage/databases/main/devices.py @@ -810,6 +810,7 @@ class DeviceWorkerStore(SQLBaseStore): SELECT stream_id, destination AS entity FROM device_lists_outbound_pokes ) AS e WHERE ? < stream_id AND stream_id <= ? + ORDER BY stream_id ASC LIMIT ? """ @@ -1528,7 +1529,11 @@ class DeviceStore(DeviceWorkerStore, DeviceBackgroundUpdateStore): ) async def add_device_change_to_streams( - self, user_id: str, device_ids: Collection[str], hosts: Collection[str] + self, + user_id: str, + device_ids: Collection[str], + hosts: Optional[Collection[str]], + room_ids: Collection[str], ) -> Optional[int]: """Persist that a user's devices have been updated, and which hosts (if any) should be poked. @@ -1537,7 +1542,10 @@ class DeviceStore(DeviceWorkerStore, DeviceBackgroundUpdateStore): user_id: The ID of the user whose device changed. device_ids: The IDs of any changed devices. If empty, this function will return None. - hosts: The remote destinations that should be notified of the change. + hosts: The remote destinations that should be notified of the change. If + None then the set of hosts have *not* been calculated, and will be + calculated later by a background task. + room_ids: The rooms that the user is in Returns: The maximum stream ID of device list updates that were added to the database, or @@ -1546,34 +1554,62 @@ class DeviceStore(DeviceWorkerStore, DeviceBackgroundUpdateStore): if not device_ids: return None - async with self._device_list_id_gen.get_next_mult( - len(device_ids) - ) as stream_ids: - await self.db_pool.runInteraction( - "add_device_change_to_stream", - self._add_device_change_to_stream_txn, + context = get_active_span_text_map() + + def add_device_changes_txn( + txn, stream_ids_for_device_change, stream_ids_for_outbound_pokes + ): + self._add_device_change_to_stream_txn( + txn, user_id, device_ids, - stream_ids, + stream_ids_for_device_change, ) - if not hosts: - return stream_ids[-1] + self._add_device_outbound_room_poke_txn( + txn, + user_id, + device_ids, + room_ids, + stream_ids_for_device_change, + context, + hosts_have_been_calculated=hosts is not None, + ) - context = get_active_span_text_map() - async with self._device_list_id_gen.get_next_mult( - len(hosts) * len(device_ids) - ) as stream_ids: - await self.db_pool.runInteraction( - "add_device_outbound_poke_to_stream", - self._add_device_outbound_poke_to_stream_txn, + # If the set of hosts to send to has not been calculated yet (and so + # `hosts` is None) or there are no `hosts` to send to, then skip + # trying to persist them to the DB. + if not hosts: + return + + self._add_device_outbound_poke_to_stream_txn( + txn, user_id, device_ids, hosts, - stream_ids, + stream_ids_for_outbound_pokes, context, ) + # `device_lists_stream` wants a stream ID per device update. + num_stream_ids = len(device_ids) + + if hosts: + # `device_lists_outbound_pokes` wants a different stream ID for + # each row, which is a row per host per device update. + num_stream_ids += len(hosts) * len(device_ids) + + async with self._device_list_id_gen.get_next_mult(num_stream_ids) as stream_ids: + stream_ids_for_device_change = stream_ids[: len(device_ids)] + stream_ids_for_outbound_pokes = stream_ids[len(device_ids) :] + + await self.db_pool.runInteraction( + "add_device_change_to_stream", + add_device_changes_txn, + stream_ids_for_device_change, + stream_ids_for_outbound_pokes, + ) + return stream_ids[-1] def _add_device_change_to_stream_txn( @@ -1617,7 +1653,7 @@ class DeviceStore(DeviceWorkerStore, DeviceBackgroundUpdateStore): user_id: str, device_ids: Iterable[str], hosts: Collection[str], - stream_ids: List[str], + stream_ids: List[int], context: Dict[str, str], ) -> None: for host in hosts: @@ -1628,8 +1664,9 @@ class DeviceStore(DeviceWorkerStore, DeviceBackgroundUpdateStore): ) now = self._clock.time_msec() - next_stream_id = iter(stream_ids) + stream_id_iterator = iter(stream_ids) + encoded_context = json_encoder.encode(context) self.db_pool.simple_insert_many_txn( txn, table="device_lists_outbound_pokes", @@ -1645,16 +1682,146 @@ class DeviceStore(DeviceWorkerStore, DeviceBackgroundUpdateStore): values=[ ( destination, - next(next_stream_id), + next(stream_id_iterator), user_id, device_id, False, now, - json_encoder.encode(context) - if whitelisted_homeserver(destination) - else "{}", + encoded_context if whitelisted_homeserver(destination) else "{}", ) for destination in hosts for device_id in device_ids ], ) + + def _add_device_outbound_room_poke_txn( + self, + txn: LoggingTransaction, + user_id: str, + device_ids: Iterable[str], + room_ids: Collection[str], + stream_ids: List[str], + context: Dict[str, str], + hosts_have_been_calculated: bool, + ) -> None: + """Record the user in the room has updated their device. + + Args: + hosts_have_been_calculated: True if `device_lists_outbound_pokes` + has been updated already with the updates. + """ + + # We only need to convert to outbound pokes if they are our user. + converted_to_destinations = ( + hosts_have_been_calculated or not self.hs.is_mine_id(user_id) + ) + + encoded_context = json_encoder.encode(context) + + # The `device_lists_changes_in_room.stream_id` column matches the + # corresponding `stream_id` of the update in the `device_lists_stream` + # table, i.e. all rows persisted for the same device update will have + # the same `stream_id` (but different room IDs). + self.db_pool.simple_insert_many_txn( + txn, + table="device_lists_changes_in_room", + keys=( + "user_id", + "device_id", + "room_id", + "stream_id", + "converted_to_destinations", + "opentracing_context", + ), + values=[ + ( + user_id, + device_id, + room_id, + stream_id, + converted_to_destinations, + encoded_context, + ) + for room_id in room_ids + for device_id, stream_id in zip(device_ids, stream_ids) + ], + ) + + async def get_uncoverted_outbound_room_pokes( + self, limit: int = 10 + ) -> List[Tuple[str, str, str, int, Optional[Dict[str, str]]]]: + """Get device list changes by room that have not yet been handled and + written to `device_lists_outbound_pokes`. + + Returns: + A list of user ID, device ID, room ID, stream ID and optional opentracing context. + """ + + sql = """ + SELECT user_id, device_id, room_id, stream_id, opentracing_context + FROM device_lists_changes_in_room + WHERE NOT converted_to_destinations + ORDER BY stream_id + LIMIT ? + """ + + def get_uncoverted_outbound_room_pokes_txn(txn): + txn.execute(sql, (limit,)) + return txn.fetchall() + + return await self.db_pool.runInteraction( + "get_uncoverted_outbound_room_pokes", get_uncoverted_outbound_room_pokes_txn + ) + + async def add_device_list_outbound_pokes( + self, + user_id: str, + device_id: str, + room_id: str, + stream_id: int, + hosts: Collection[str], + context: Optional[Dict[str, str]], + ) -> None: + """Queue the device update to be sent to the given set of hosts, + calculated from the room ID. + + Marks the associated row in `device_lists_changes_in_room` as handled. + """ + + def add_device_list_outbound_pokes_txn(txn, stream_ids: List[int]): + if hosts: + self._add_device_outbound_poke_to_stream_txn( + txn, + user_id=user_id, + device_ids=[device_id], + hosts=hosts, + stream_ids=stream_ids, + context=context, + ) + + self.db_pool.simple_update_txn( + txn, + table="device_lists_changes_in_room", + keyvalues={ + "user_id": user_id, + "device_id": device_id, + "stream_id": stream_id, + "room_id": room_id, + }, + updatevalues={"converted_to_destinations": True}, + ) + + if not hosts: + # If there are no hosts then we don't try and generate stream IDs. + return await self.db_pool.runInteraction( + "add_device_list_outbound_pokes", + add_device_list_outbound_pokes_txn, + [], + ) + + async with self._device_list_id_gen.get_next_mult(len(hosts)) as stream_ids: + return await self.db_pool.runInteraction( + "add_device_list_outbound_pokes", + add_device_list_outbound_pokes_txn, + stream_ids, + ) diff --git a/synapse/storage/schema/__init__.py b/synapse/storage/schema/__init__.py index ea900e0f3d..151f2aa9bb 100644 --- a/synapse/storage/schema/__init__.py +++ b/synapse/storage/schema/__init__.py @@ -60,6 +60,7 @@ Changes in SCHEMA_VERSION = 68: new events. Changes in SCHEMA_VERSION = 69: + - We now write to `device_lists_changes_in_room` table. - Use sequence to generate future `application_services_txns.txn_id`s """ diff --git a/synapse/storage/schema/main/delta/69/01device_list_oubound_by_room.sql b/synapse/storage/schema/main/delta/69/01device_list_oubound_by_room.sql new file mode 100644 index 0000000000..b5b1782b2a --- /dev/null +++ b/synapse/storage/schema/main/delta/69/01device_list_oubound_by_room.sql @@ -0,0 +1,38 @@ +/* Copyright 2022 The Matrix.org Foundation C.I.C + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +CREATE TABLE device_lists_changes_in_room ( + user_id TEXT NOT NULL, + device_id TEXT NOT NULL, + room_id TEXT NOT NULL, + + -- This initially matches `device_lists_stream.stream_id`. Note that we + -- delete older values from `device_lists_stream`, so we can't use a foreign + -- constraint here. + -- + -- The table will contain rows with the same `stream_id` but different + -- `room_id`, as for each device update we store a row per room the user is + -- joined to. Therefore `(stream_id, room_id)` gives a unique index. + stream_id BIGINT NOT NULL, + + -- We have a background process which goes through this table and converts + -- entries into rows in `device_lists_outbound_pokes`. Once we have processed + -- a row, we mark it as such by setting `converted_to_destinations=TRUE`. + converted_to_destinations BOOLEAN NOT NULL, + opentracing_context TEXT +); + +CREATE UNIQUE INDEX device_lists_changes_in_stream_id ON device_lists_changes_in_room(stream_id, room_id); +CREATE INDEX device_lists_changes_in_stream_id_unconverted ON device_lists_changes_in_room(stream_id) WHERE NOT converted_to_destinations; diff --git a/tests/federation/test_federation_sender.py b/tests/federation/test_federation_sender.py index e90592855a..a6e91956af 100644 --- a/tests/federation/test_federation_sender.py +++ b/tests/federation/test_federation_sender.py @@ -14,6 +14,7 @@ from typing import Optional from unittest.mock import Mock +from parameterized import parameterized_class from signedjson import key, sign from signedjson.types import BaseKey, SigningKey @@ -154,6 +155,12 @@ class FederationSenderReceiptsTestCases(HomeserverTestCase): ) +@parameterized_class( + [ + {"enable_room_poke_code_path": False}, + {"enable_room_poke_code_path": True}, + ] +) class FederationSenderDevicesTestCases(HomeserverTestCase): servlets = [ admin.register_servlets, @@ -168,17 +175,21 @@ class FederationSenderDevicesTestCases(HomeserverTestCase): def default_config(self): c = super().default_config() c["send_federation"] = True + c["use_new_device_lists_changes_in_room"] = self.enable_room_poke_code_path return c def prepare(self, reactor, clock, hs): - # stub out get_users_who_share_room_with_user so that it claims that - # `@user2:host2` is in the room - def get_users_who_share_room_with_user(user_id): + # stub out `get_rooms_for_user` and `get_users_in_room` so that the + # server thinks the user shares a room with `@user2:host2` + def get_rooms_for_user(user_id): + return defer.succeed({"!room:host1"}) + + hs.get_datastores().main.get_rooms_for_user = get_rooms_for_user + + def get_users_in_room(room_id): return defer.succeed({"@user2:host2"}) - hs.get_datastores().main.get_users_who_share_room_with_user = ( - get_users_who_share_room_with_user - ) + hs.get_datastores().main.get_users_in_room = get_users_in_room # whenever send_transaction is called, record the edu data self.edus = [] diff --git a/tests/storage/test_devices.py b/tests/storage/test_devices.py index 21ffc5a909..d1227dd4ac 100644 --- a/tests/storage/test_devices.py +++ b/tests/storage/test_devices.py @@ -96,7 +96,9 @@ class DeviceStoreTestCase(HomeserverTestCase): # Add two device updates with sequential `stream_id`s self.get_success( - self.store.add_device_change_to_streams("user_id", device_ids, ["somehost"]) + self.store.add_device_change_to_streams( + "user_id", device_ids, ["somehost"], ["!some:room"] + ) ) # Get all device updates ever meant for this remote @@ -122,7 +124,9 @@ class DeviceStoreTestCase(HomeserverTestCase): "device_id5", ] self.get_success( - self.store.add_device_change_to_streams("user_id", device_ids, ["somehost"]) + self.store.add_device_change_to_streams( + "user_id", device_ids, ["somehost"], ["!some:room"] + ) ) # Get device updates meant for this remote @@ -144,7 +148,9 @@ class DeviceStoreTestCase(HomeserverTestCase): # Add some more device updates to ensure it still resumes properly device_ids = ["device_id6", "device_id7"] self.get_success( - self.store.add_device_change_to_streams("user_id", device_ids, ["somehost"]) + self.store.add_device_change_to_streams( + "user_id", device_ids, ["somehost"], ["!some:room"] + ) ) # Get the next batch of device updates @@ -220,7 +226,7 @@ class DeviceStoreTestCase(HomeserverTestCase): self.get_success( self.store.add_device_change_to_streams( - "@user_id:test", device_ids, ["somehost"] + "@user_id:test", device_ids, ["somehost"], ["!some:room"] ) ) From b446c99ac9ecf5d06e868b6da363b113c7aecaf5 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Mon, 4 Apr 2022 20:12:25 +0100 Subject: [PATCH 27/55] Prefill the device_list_stream_cache (#12367) * Prefill the device_list_stream_cache * Newsfile * Newsfile --- changelog.d/12367.feature | 1 + synapse/replication/slave/storage/devices.py | 12 +++++++++++- synapse/storage/databases/main/__init__.py | 12 +++++++++++- 3 files changed, 23 insertions(+), 2 deletions(-) create mode 100644 changelog.d/12367.feature diff --git a/changelog.d/12367.feature b/changelog.d/12367.feature new file mode 100644 index 0000000000..34bb60e966 --- /dev/null +++ b/changelog.d/12367.feature @@ -0,0 +1 @@ +Reduce overhead of restarting synchrotrons. diff --git a/synapse/replication/slave/storage/devices.py b/synapse/replication/slave/storage/devices.py index f040e33bfb..00a634d3a9 100644 --- a/synapse/replication/slave/storage/devices.py +++ b/synapse/replication/slave/storage/devices.py @@ -48,8 +48,18 @@ class SlavedDeviceStore(EndToEndKeyWorkerStore, DeviceWorkerStore, BaseSlavedSto ], ) device_list_max = self._device_list_id_gen.get_current_token() + device_list_prefill, min_device_list_id = self.db_pool.get_cache_dict( + db_conn, + "device_lists_stream", + entity_column="user_id", + stream_column="stream_id", + max_value=device_list_max, + limit=1000, + ) self._device_list_stream_cache = StreamChangeCache( - "DeviceListStreamChangeCache", device_list_max + "DeviceListStreamChangeCache", + min_device_list_id, + prefilled_cache=device_list_prefill, ) self._user_signature_stream_cache = StreamChangeCache( "UserSignatureStreamChangeCache", device_list_max diff --git a/synapse/storage/databases/main/__init__.py b/synapse/storage/databases/main/__init__.py index cdbe3872fa..d4a38daa9a 100644 --- a/synapse/storage/databases/main/__init__.py +++ b/synapse/storage/databases/main/__init__.py @@ -184,8 +184,18 @@ class DataStore( super().__init__(database, db_conn, hs) device_list_max = self._device_list_id_gen.get_current_token() + device_list_prefill, min_device_list_id = self.db_pool.get_cache_dict( + db_conn, + "device_lists_stream", + entity_column="user_id", + stream_column="stream_id", + max_value=device_list_max, + limit=1000, + ) self._device_list_stream_cache = StreamChangeCache( - "DeviceListStreamChangeCache", device_list_max + "DeviceListStreamChangeCache", + min_device_list_id, + prefilled_cache=device_list_prefill, ) self._user_signature_stream_cache = StreamChangeCache( "UserSignatureStreamChangeCache", device_list_max From 9633eb2162017009ef7307034d179e1dc23d1e85 Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Mon, 4 Apr 2022 14:54:50 -0500 Subject: [PATCH 28/55] Allow non-member state sent in room batch to resolve for historic events (MSC2716) (#12329) Part of https://github.com/matrix-org/synapse/issues/12110 Complement test: https://github.com/matrix-org/complement/pull/354 Previously, they didn't resolve because async `filter_events_for_client` removes all outlier state except for out-of-band membership. And fundamentally, we have the state at these events so they shouldn't be marked as outliers. --- changelog.d/12329.bugfix | 1 + synapse/handlers/room_batch.py | 38 +++++++++++----------------------- 2 files changed, 13 insertions(+), 26 deletions(-) create mode 100644 changelog.d/12329.bugfix diff --git a/changelog.d/12329.bugfix b/changelog.d/12329.bugfix new file mode 100644 index 0000000000..aef4117343 --- /dev/null +++ b/changelog.d/12329.bugfix @@ -0,0 +1 @@ +Fix non-member state events not resolving for historical events when used in [MSC2716](https://github.com/matrix-org/matrix-spec-proposals/pull/2716) `/batch_send` `state_events_at_start`. diff --git a/synapse/handlers/room_batch.py b/synapse/handlers/room_batch.py index a0255bd143..78e299d3a5 100644 --- a/synapse/handlers/room_batch.py +++ b/synapse/handlers/room_batch.py @@ -156,8 +156,8 @@ class RoomBatchHandler: ) -> List[str]: """Takes all `state_events_at_start` event dictionaries and creates/persists them in a floating state event chain which don't resolve into the current room - state. They are floating because they reference no prev_events and are marked - as outliers which disconnects them from the normal DAG. + state. They are floating because they reference no prev_events which disconnects + them from the normal DAG. Args: state_events_at_start: @@ -213,31 +213,23 @@ class RoomBatchHandler: room_id=room_id, action=membership, content=event_dict["content"], - # Mark as an outlier to disconnect it from the normal DAG - # and not show up between batches of history. - outlier=True, historical=True, # Only the first event in the state chain should be floating. # The rest should hang off each other in a chain. allow_no_prev_events=index == 0, prev_event_ids=prev_event_ids_for_state_chain, - # Since each state event is marked as an outlier, the - # `EventContext.for_outlier()` won't have any `state_ids` - # set and therefore can't derive any state even though the - # prev_events are set. Also since the first event in the - # state chain is floating with no `prev_events`, it can't - # derive state from anywhere automatically. So we need to - # set some state explicitly. + # The first event in the state chain is floating with no + # `prev_events` which means it can't derive state from + # anywhere automatically. So we need to set some state + # explicitly. # # Make sure to use a copy of this list because we modify it # later in the loop here. Otherwise it will be the same - # reference and also update in the event when we append later. + # reference and also update in the event when we append + # later. state_event_ids=state_event_ids.copy(), ) else: - # TODO: Add some complement tests that adds state that is not member joins - # and will use this code path. Maybe we only want to support join state events - # and can get rid of this `else`? ( event, _, @@ -246,21 +238,15 @@ class RoomBatchHandler: state_event["sender"], app_service_requester.app_service ), event_dict, - # Mark as an outlier to disconnect it from the normal DAG - # and not show up between batches of history. - outlier=True, historical=True, # Only the first event in the state chain should be floating. # The rest should hang off each other in a chain. allow_no_prev_events=index == 0, prev_event_ids=prev_event_ids_for_state_chain, - # Since each state event is marked as an outlier, the - # `EventContext.for_outlier()` won't have any `state_ids` - # set and therefore can't derive any state even though the - # prev_events are set. Also since the first event in the - # state chain is floating with no `prev_events`, it can't - # derive state from anywhere automatically. So we need to - # set some state explicitly. + # The first event in the state chain is floating with no + # `prev_events` which means it can't derive state from + # anywhere automatically. So we need to set some state + # explicitly. # # Make sure to use a copy of this list because we modify it # later in the loop here. Otherwise it will be the same From f608e6c8cf48d8d5327c5ac04746f4f7f3992e31 Mon Sep 17 00:00:00 2001 From: Shay Date: Mon, 4 Apr 2022 17:40:31 -0700 Subject: [PATCH 29/55] Update broken link in `workers.md` (#12369) --- changelog.d/12369.doc | 1 + docs/workers.md | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) create mode 100644 changelog.d/12369.doc diff --git a/changelog.d/12369.doc b/changelog.d/12369.doc new file mode 100644 index 0000000000..c34271b2ce --- /dev/null +++ b/changelog.d/12369.doc @@ -0,0 +1 @@ +Update the link to Redis pub/sub documentation in the workers documentation.. \ No newline at end of file diff --git a/docs/workers.md b/docs/workers.md index 8ac95e39bb..caef44b614 100644 --- a/docs/workers.md +++ b/docs/workers.md @@ -27,7 +27,7 @@ feeds streams of newly written data between processes so they can be kept in sync with the database state. When configured to do so, Synapse uses a -[Redis pub/sub channel](https://redis.io/topics/pubsub) to send the replication +[Redis pub/sub channel](https://redis.io/docs/manual/pubsub/) to send the replication stream between all configured Synapse processes. Additionally, processes may make HTTP requests to each other, primarily for operations which need to wait for a reply ─ such as sending an event. From 5218fe7670eb13a1b5590c9eea270101ec200b84 Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Tue, 5 Apr 2022 04:57:09 -0500 Subject: [PATCH 30/55] Explain how to decipher live and historic pagination tokens (#12317) --- changelog.d/12317.misc | 1 + synapse/types.py | 96 +++++++++++++++++++++++++++++++++++++----- 2 files changed, 86 insertions(+), 11 deletions(-) create mode 100644 changelog.d/12317.misc diff --git a/changelog.d/12317.misc b/changelog.d/12317.misc new file mode 100644 index 0000000000..1dfee496d8 --- /dev/null +++ b/changelog.d/12317.misc @@ -0,0 +1 @@ +Update docstrings to explain how to decipher live and historic pagination tokens. diff --git a/synapse/types.py b/synapse/types.py index 500597b3a4..6bbefb6faa 100644 --- a/synapse/types.py +++ b/synapse/types.py @@ -422,22 +422,44 @@ class RoomStreamToken: s0 s1 | | - [0] V [1] V [2] + [0] ▼ [1] ▼ [2] Tokens can either be a point in the live event stream or a cursor going through historic events. - When traversing the live event stream events are ordered by when they - arrived at the homeserver. + When traversing the live event stream, events are ordered by + `stream_ordering` (when they arrived at the homeserver). - When traversing historic events the events are ordered by their depth in - the event graph "topological_ordering" and then by when they arrived at the - homeserver "stream_ordering". + When traversing historic events, events are first ordered by their `depth` + (`topological_ordering` in the event graph) and tie-broken by + `stream_ordering` (when the event arrived at the homeserver). - Live tokens start with an "s" followed by the "stream_ordering" id of the - event it comes after. Historic tokens start with a "t" followed by the - "topological_ordering" id of the event it comes after, followed by "-", - followed by the "stream_ordering" id of the event it comes after. + If you're looking for more info about what a token with all of the + underscores means, ex. + `s2633508_17_338_6732159_1082514_541479_274711_265584_1`, see the docstring + for `StreamToken` below. + + --- + + Live tokens start with an "s" followed by the `stream_ordering` of the event + that comes before the position of the token. Said another way: + `stream_ordering` uniquely identifies a persisted event. The live token + means "the position just after the event identified by `stream_ordering`". + An example token is: + + s2633508 + + --- + + Historic tokens start with a "t" followed by the `depth` + (`topological_ordering` in the event graph) of the event that comes before + the position of the token, followed by "-", followed by the + `stream_ordering` of the event that comes before the position of the token. + An example token is: + + t426-2633508 + + --- There is also a third mode for live tokens where the token starts with "m", which is sometimes used when using sharded event persisters. In this case @@ -464,6 +486,8 @@ class RoomStreamToken: Note: The `RoomStreamToken` cannot have both a topological part and an instance map. + --- + For caching purposes, `RoomStreamToken`s and by extension, all their attributes, must be hashable. """ @@ -600,7 +624,57 @@ class RoomStreamToken: @attr.s(slots=True, frozen=True, auto_attribs=True) class StreamToken: - """A collection of positions within multiple streams. + """A collection of keys joined together by underscores in the following + order and which represent the position in their respective streams. + + ex. `s2633508_17_338_6732159_1082514_541479_274711_265584_1` + 1. `room_key`: `s2633508` which is a `RoomStreamToken` + - `RoomStreamToken`'s can also look like `t426-2633508` or `m56~2.58~3.59` + - See the docstring for `RoomStreamToken` for more details. + 2. `presence_key`: `17` + 3. `typing_key`: `338` + 4. `receipt_key`: `6732159` + 5. `account_data_key`: `1082514` + 6. `push_rules_key`: `541479` + 7. `to_device_key`: `274711` + 8. `device_list_key`: `265584` + 9. `groups_key`: `1` + + You can see how many of these keys correspond to the various + fields in a "/sync" response: + ```json + { + "next_batch": "s12_4_0_1_1_1_1_4_1", + "presence": { + "events": [] + }, + "device_lists": { + "changed": [] + }, + "rooms": { + "join": { + "!QrZlfIDQLNLdZHqTnt:hs1": { + "timeline": { + "events": [], + "prev_batch": "s10_4_0_1_1_1_1_4_1", + "limited": false + }, + "state": { + "events": [] + }, + "account_data": { + "events": [] + }, + "ephemeral": { + "events": [] + } + } + } + } + } + ``` + + --- For caching purposes, `StreamToken`s and by extension, all their attributes, must be hashable. From a7293ef16fb3ca9f1234877f1496b30cdeeb01cd Mon Sep 17 00:00:00 2001 From: David Robertson Date: Tue, 29 Mar 2022 11:41:19 +0100 Subject: [PATCH 31/55] Bump `black` and `click` versions (#12320) --- changelog.d/12320.misc | 1 + contrib/jitsimeetbridge/jitsimeetbridge.py | 15 +++++++++------ setup.py | 4 ++-- synapse/api/constants.py | 2 +- synapse/appservice/scheduler.py | 2 +- synapse/crypto/keyring.py | 2 +- synapse/events/utils.py | 2 +- synapse/http/proxyagent.py | 2 +- synapse/replication/http/_base.py | 2 +- synapse/rest/media/v1/media_storage.py | 2 +- synapse/rest/media/v1/preview_html.py | 4 ++-- synapse/storage/database.py | 2 +- synapse/util/patch_inline_callbacks.py | 17 ++++++++++------- synapse/util/retryutils.py | 2 +- tests/handlers/test_federation.py | 6 +++--- tests/replication/_base.py | 2 +- 16 files changed, 37 insertions(+), 30 deletions(-) create mode 100644 changelog.d/12320.misc diff --git a/changelog.d/12320.misc b/changelog.d/12320.misc new file mode 100644 index 0000000000..7b4748d230 --- /dev/null +++ b/changelog.d/12320.misc @@ -0,0 +1 @@ +Bump the version of `black` for compatibility with the latest `click` release. diff --git a/contrib/jitsimeetbridge/jitsimeetbridge.py b/contrib/jitsimeetbridge/jitsimeetbridge.py index 495fd4e10a..b3de468687 100644 --- a/contrib/jitsimeetbridge/jitsimeetbridge.py +++ b/contrib/jitsimeetbridge/jitsimeetbridge.py @@ -193,12 +193,15 @@ class TrivialXmppClient: time.sleep(7) print("SSRC spammer started") while self.running: - ssrcMsg = "%(nick)s" % { - "tojid": "%s@%s/%s" % (ROOMNAME, ROOMDOMAIN, self.shortJid), - "nick": self.userId, - "assrc": self.ssrcs["audio"], - "vssrc": self.ssrcs["video"], - } + ssrcMsg = ( + "%(nick)s" + % { + "tojid": "%s@%s/%s" % (ROOMNAME, ROOMDOMAIN, self.shortJid), + "nick": self.userId, + "assrc": self.ssrcs["audio"], + "vssrc": self.ssrcs["video"], + } + ) res = self.sendIq(ssrcMsg) print("reply from ssrc announce: ", res) time.sleep(10) diff --git a/setup.py b/setup.py index 63da71ad7b..48bd418bb8 100755 --- a/setup.py +++ b/setup.py @@ -95,7 +95,7 @@ CONDITIONAL_REQUIREMENTS["all"] = list(ALL_OPTIONAL_REQUIREMENTS) # We pin black so that our tests don't start failing on new releases. CONDITIONAL_REQUIREMENTS["lint"] = [ "isort==5.7.0", - "black==21.12b0", + "black==22.3.0", "flake8-comprehensions", "flake8-bugbear==21.3.2", "flake8", @@ -128,7 +128,7 @@ CONDITIONAL_REQUIREMENTS["dev"] = ( + CONDITIONAL_REQUIREMENTS["test"] + [ # The following are used by the release script - "click==7.1.2", + "click==8.1.0", "redbaron==0.9.2", "GitPython==3.1.14", "commonmark==0.9.1", diff --git a/synapse/api/constants.py b/synapse/api/constants.py index b0c08a074d..92907415e6 100644 --- a/synapse/api/constants.py +++ b/synapse/api/constants.py @@ -23,7 +23,7 @@ from typing_extensions import Final MAX_PDU_SIZE = 65536 # the "depth" field on events is limited to 2**63 - 1 -MAX_DEPTH = 2 ** 63 - 1 +MAX_DEPTH = 2**63 - 1 # the maximum length for a room alias is 255 characters MAX_ALIAS_LENGTH = 255 diff --git a/synapse/appservice/scheduler.py b/synapse/appservice/scheduler.py index 72417151ba..a6084b9c35 100644 --- a/synapse/appservice/scheduler.py +++ b/synapse/appservice/scheduler.py @@ -428,7 +428,7 @@ class _Recoverer: "as-recoverer-%s" % (self.service.id,), self.retry ) - delay = 2 ** self.backoff_counter + delay = 2**self.backoff_counter logger.info("Scheduling retries on %s in %fs", self.service.id, delay) self.clock.call_later(delay, _retry) diff --git a/synapse/crypto/keyring.py b/synapse/crypto/keyring.py index 93d56c077a..6cf384f6a1 100644 --- a/synapse/crypto/keyring.py +++ b/synapse/crypto/keyring.py @@ -182,7 +182,7 @@ class Keyring: vk = get_verify_key(hs.signing_key) self._local_verify_keys[f"{vk.alg}:{vk.version}"] = FetchKeyResult( verify_key=vk, - valid_until_ts=2 ** 63, # fake future timestamp + valid_until_ts=2**63, # fake future timestamp ) async def verify_json_for_server( diff --git a/synapse/events/utils.py b/synapse/events/utils.py index 7120062127..918e87ed9c 100644 --- a/synapse/events/utils.py +++ b/synapse/events/utils.py @@ -49,7 +49,7 @@ if TYPE_CHECKING: # the literal fields "foo\" and "bar" but will instead be treated as "foo\\.bar" SPLIT_FIELD_REGEX = re.compile(r"(? cls.RETRY_ON_CONNECT_ERROR_ATTEMPTS: raise - delay = 2 ** attempts + delay = 2**attempts logger.warning( "%s request connection failed; retrying in %ds: %r", cls.NAME, diff --git a/synapse/rest/media/v1/media_storage.py b/synapse/rest/media/v1/media_storage.py index 9f6c251caf..604f18bf52 100644 --- a/synapse/rest/media/v1/media_storage.py +++ b/synapse/rest/media/v1/media_storage.py @@ -352,7 +352,7 @@ class ReadableFileWrapper: `IConsumer`. """ - CHUNK_SIZE = 2 ** 14 + CHUNK_SIZE = 2**14 clock: Clock path: str diff --git a/synapse/rest/media/v1/preview_html.py b/synapse/rest/media/v1/preview_html.py index 4cc9c66fbe..ca73965fc2 100644 --- a/synapse/rest/media/v1/preview_html.py +++ b/synapse/rest/media/v1/preview_html.py @@ -23,10 +23,10 @@ if TYPE_CHECKING: logger = logging.getLogger(__name__) _charset_match = re.compile( - br'<\s*meta[^>]*charset\s*=\s*"?([a-z0-9_-]+)"?', flags=re.I + rb'<\s*meta[^>]*charset\s*=\s*"?([a-z0-9_-]+)"?', flags=re.I ) _xml_encoding_match = re.compile( - br'\s*<\s*\?\s*xml[^>]*encoding="([a-z0-9_-]+)"', flags=re.I + rb'\s*<\s*\?\s*xml[^>]*encoding="([a-z0-9_-]+)"', flags=re.I ) _content_type_match = re.compile(r'.*; *charset="?(.*?)"?(;|$)', flags=re.I) diff --git a/synapse/storage/database.py b/synapse/storage/database.py index 72fef1533f..3ef2bdd74b 100644 --- a/synapse/storage/database.py +++ b/synapse/storage/database.py @@ -63,7 +63,7 @@ if TYPE_CHECKING: from synapse.server import HomeServer # python 3 does not have a maximum int value -MAX_TXN_ID = 2 ** 63 - 1 +MAX_TXN_ID = 2**63 - 1 logger = logging.getLogger(__name__) diff --git a/synapse/util/patch_inline_callbacks.py b/synapse/util/patch_inline_callbacks.py index 6d4b0b7c5a..dace68666c 100644 --- a/synapse/util/patch_inline_callbacks.py +++ b/synapse/util/patch_inline_callbacks.py @@ -217,13 +217,16 @@ def _check_yield_points( # We don't raise here as its perfectly valid for contexts to # change in a function, as long as it sets the correct context # on resolving (which is checked separately). - err = "%s changed context from %s to %s, happened between lines %d and %d in %s" % ( - frame.f_code.co_name, - expected_context, - current_context(), - last_yield_line_no, - frame.f_lineno, - frame.f_code.co_filename, + err = ( + "%s changed context from %s to %s, happened between lines %d and %d in %s" + % ( + frame.f_code.co_name, + expected_context, + current_context(), + last_yield_line_no, + frame.f_lineno, + frame.f_code.co_filename, + ) ) changes.append(err) diff --git a/synapse/util/retryutils.py b/synapse/util/retryutils.py index 648d9a95a7..d81f2527d7 100644 --- a/synapse/util/retryutils.py +++ b/synapse/util/retryutils.py @@ -30,7 +30,7 @@ MIN_RETRY_INTERVAL = 10 * 60 * 1000 RETRY_MULTIPLIER = 5 # a cap on the backoff. (Essentially none) -MAX_RETRY_INTERVAL = 2 ** 62 +MAX_RETRY_INTERVAL = 2**62 class NotRetryingDestination(Exception): diff --git a/tests/handlers/test_federation.py b/tests/handlers/test_federation.py index 89078fc637..4d65639a1e 100644 --- a/tests/handlers/test_federation.py +++ b/tests/handlers/test_federation.py @@ -496,8 +496,8 @@ class EventFromPduTestCase(TestCase): def test_invalid_numbers(self) -> None: """Invalid values for an integer should be rejected, all floats should be rejected.""" for value in [ - -(2 ** 53), - 2 ** 53, + -(2**53), + 2**53, 1.0, float("inf"), float("-inf"), @@ -524,7 +524,7 @@ class EventFromPduTestCase(TestCase): event_from_pdu_json( { "type": EventTypes.Message, - "content": {"foo": [{"bar": 2 ** 56}]}, + "content": {"foo": [{"bar": 2**56}]}, "room_id": "!room:test", "sender": "@user:test", "depth": 1, diff --git a/tests/replication/_base.py b/tests/replication/_base.py index 9c5df266bd..a0589b6d6a 100644 --- a/tests/replication/_base.py +++ b/tests/replication/_base.py @@ -206,7 +206,7 @@ class BaseStreamTestCase(unittest.HomeserverTestCase): path: bytes = request.path # type: ignore self.assertRegex( path, - br"^/_synapse/replication/get_repl_stream_updates/%s/[^/]+$" + rb"^/_synapse/replication/get_repl_stream_updates/%s/[^/]+$" % (stream_name.encode("ascii"),), ) From ca7e34cb57a93ea48e7bb00b89c63b6f14989736 Mon Sep 17 00:00:00 2001 From: Sean Quah Date: Tue, 5 Apr 2022 12:13:44 +0100 Subject: [PATCH 32/55] Pin signedjson to <= 1.1.1 as a temporary workaround for #12324 To be reverted after the Synapse 1.56 release. Signed-off-by: Sean Quah --- synapse/python_dependencies.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/synapse/python_dependencies.py b/synapse/python_dependencies.py index 8419ab3aca..d02cca0bbb 100644 --- a/synapse/python_dependencies.py +++ b/synapse/python_dependencies.py @@ -48,7 +48,7 @@ REQUIREMENTS = [ "unpaddedbase64>=1.1.0", "canonicaljson>=1.4.0", # we use the type definitions added in signedjson 1.1. - "signedjson>=1.1.0", + "signedjson>=1.1.0,<=1.1.1", "pynacl>=1.2.1", "idna>=2.5", # validating SSL certs for IP addresses requires service_identity 18.1. From 7a95e80418d7cc79a0fe4800a279b71fa84279f1 Mon Sep 17 00:00:00 2001 From: Sean Quah Date: Tue, 5 Apr 2022 12:44:00 +0100 Subject: [PATCH 33/55] 1.56.0 --- CHANGES.md | 16 +++++++++++++--- changelog.d/12320.misc | 1 - changelog.d/12322.misc | 1 - debian/changelog | 6 ++++++ synapse/__init__.py | 2 +- 5 files changed, 20 insertions(+), 6 deletions(-) delete mode 100644 changelog.d/12320.misc delete mode 100644 changelog.d/12322.misc diff --git a/CHANGES.md b/CHANGES.md index de40921bff..4cbe804d04 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,5 +1,5 @@ -Synapse 1.56.0rc1 (2022-03-29) -============================== +Synapse 1.56.0 (2022-04-05) +=========================== Synapse will now refuse to start up if open registration is enabled, in order to help mitigate abuse across the federation. If you would like @@ -17,6 +17,16 @@ the config flag `allow_unsafe_locale`, found in the database section of the conf [upgrade notes](https://matrix-org.github.io/synapse/v1.56/upgrade#change-in-behaviour-for-postgresql-databases-with-unsafe-locale) for details. +Internal Changes +---------------- + +- Bump the version of `black` for compatibility with the latest `click` release. ([\#12320](https://github.com/matrix-org/synapse/issues/12320)) +- Refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config flag `enable_registration_without_verification` is set to `true`. ([\#12322](https://github.com/matrix-org/synapse/issues/12322)) + + +Synapse 1.56.0rc1 (2022-03-29) +============================== + Features -------- @@ -6457,4 +6467,4 @@ Internal Changes - Don't run CI build checks until sample config check has passed. ([\#5370](https://github.com/matrix-org/synapse/issues/5370)) - Automatically retry buildkite builds (max twice) when an agent is lost. ([\#5380](https://github.com/matrix-org/synapse/issues/5380)) -**Changelogs for versions older than 1.0.0 can be found [here](CHANGES-pre-1.0.md).** \ No newline at end of file +**Changelogs for versions older than 1.0.0 can be found [here](CHANGES-pre-1.0.md).** diff --git a/changelog.d/12320.misc b/changelog.d/12320.misc deleted file mode 100644 index 7b4748d230..0000000000 --- a/changelog.d/12320.misc +++ /dev/null @@ -1 +0,0 @@ -Bump the version of `black` for compatibility with the latest `click` release. diff --git a/changelog.d/12322.misc b/changelog.d/12322.misc deleted file mode 100644 index fd595df81c..0000000000 --- a/changelog.d/12322.misc +++ /dev/null @@ -1 +0,0 @@ -Refuse to start if registration is enabled without email, captcha, or token-based verification unless new config flag `enable_registration_without_verification` is set to `true`. \ No newline at end of file diff --git a/debian/changelog b/debian/changelog index d04954457d..903d98af02 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +matrix-synapse-py3 (1.56.0) stable; urgency=medium + + * New synapse release 1.56.0. + + -- Synapse Packaging team Tue, 05 Apr 2022 12:38:39 +0100 + matrix-synapse-py3 (1.56.0~rc1) stable; urgency=medium * New synapse release 1.56.0~rc1. diff --git a/synapse/__init__.py b/synapse/__init__.py index 0960305d79..2e6510537f 100644 --- a/synapse/__init__.py +++ b/synapse/__init__.py @@ -68,7 +68,7 @@ try: except ImportError: pass -__version__ = "1.56.0rc1" +__version__ = "1.56.0" if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)): # We import here so that we don't have to install a bunch of deps when From 708d88b1a22d422c294c4d06f2896a24f2a0251d Mon Sep 17 00:00:00 2001 From: reivilibre Date: Tue, 5 Apr 2022 12:44:48 +0100 Subject: [PATCH 34/55] Allow specifying the Postgres database's port when running unit tests with Postgres. (#12376) --- changelog.d/12376.misc | 1 + docs/development/contributing_guide.md | 14 +++++++++++--- tests/server.py | 4 ++++ tests/utils.py | 8 ++++++++ 4 files changed, 24 insertions(+), 3 deletions(-) create mode 100644 changelog.d/12376.misc diff --git a/changelog.d/12376.misc b/changelog.d/12376.misc new file mode 100644 index 0000000000..191f84bbd4 --- /dev/null +++ b/changelog.d/12376.misc @@ -0,0 +1 @@ +Allow specifying the Postgres database's port when running unit tests with Postgres. \ No newline at end of file diff --git a/docs/development/contributing_guide.md b/docs/development/contributing_guide.md index 9db9352c9e..0d9cf60196 100644 --- a/docs/development/contributing_guide.md +++ b/docs/development/contributing_guide.md @@ -206,9 +206,10 @@ To do so, [configure Postgres](../postgres.md) and run `trial` with the following environment variables matching your configuration: - `SYNAPSE_POSTGRES` to anything nonempty -- `SYNAPSE_POSTGRES_HOST` -- `SYNAPSE_POSTGRES_USER` -- `SYNAPSE_POSTGRES_PASSWORD` +- `SYNAPSE_POSTGRES_HOST` (optional if it's the default: UNIX socket) +- `SYNAPSE_POSTGRES_PORT` (optional if it's the default: 5432) +- `SYNAPSE_POSTGRES_USER` (optional if using a UNIX socket) +- `SYNAPSE_POSTGRES_PASSWORD` (optional if using a UNIX socket) For example: @@ -220,6 +221,13 @@ export SYNAPSE_POSTGRES_PASSWORD=mydevenvpassword trial ``` +You don't need to specify the host, user, port or password if your Postgres +server is set to authenticate you over the UNIX socket (i.e. if the `psql` command +works without further arguments). + +Your Postgres account needs to be able to create databases. + + ## Run the integration tests ([Sytest](https://github.com/matrix-org/sytest)). The integration tests are a more comprehensive suite of tests. They diff --git a/tests/server.py b/tests/server.py index aaa5ca3e74..16559d2588 100644 --- a/tests/server.py +++ b/tests/server.py @@ -76,6 +76,7 @@ from tests.utils import ( POSTGRES_BASE_DB, POSTGRES_HOST, POSTGRES_PASSWORD, + POSTGRES_PORT, POSTGRES_USER, SQLITE_PERSIST_DB, USE_POSTGRES_FOR_TESTS, @@ -747,6 +748,7 @@ def setup_test_homeserver( "host": POSTGRES_HOST, "password": POSTGRES_PASSWORD, "user": POSTGRES_USER, + "port": POSTGRES_PORT, "cp_min": 1, "cp_max": 5, }, @@ -786,6 +788,7 @@ def setup_test_homeserver( database=POSTGRES_BASE_DB, user=POSTGRES_USER, host=POSTGRES_HOST, + port=POSTGRES_PORT, password=POSTGRES_PASSWORD, ) db_conn.autocommit = True @@ -833,6 +836,7 @@ def setup_test_homeserver( database=POSTGRES_BASE_DB, user=POSTGRES_USER, host=POSTGRES_HOST, + port=POSTGRES_PORT, password=POSTGRES_PASSWORD, ) db_conn.autocommit = True diff --git a/tests/utils.py b/tests/utils.py index f6b1d60371..d4ba3a9b99 100644 --- a/tests/utils.py +++ b/tests/utils.py @@ -35,6 +35,11 @@ LEAVE_DB = os.environ.get("SYNAPSE_LEAVE_DB", False) POSTGRES_USER = os.environ.get("SYNAPSE_POSTGRES_USER", None) POSTGRES_HOST = os.environ.get("SYNAPSE_POSTGRES_HOST", None) POSTGRES_PASSWORD = os.environ.get("SYNAPSE_POSTGRES_PASSWORD", None) +POSTGRES_PORT = ( + int(os.environ["SYNAPSE_POSTGRES_PORT"]) + if "SYNAPSE_POSTGRES_PORT" in os.environ + else None +) POSTGRES_BASE_DB = "_synapse_unit_tests_base_%s" % (os.getpid(),) # When debugging a specific test, it's occasionally useful to write the @@ -55,6 +60,7 @@ def setupdb(): db_conn = db_engine.module.connect( user=POSTGRES_USER, host=POSTGRES_HOST, + port=POSTGRES_PORT, password=POSTGRES_PASSWORD, dbname=POSTGRES_DBNAME_FOR_INITIAL_CREATE, ) @@ -73,6 +79,7 @@ def setupdb(): database=POSTGRES_BASE_DB, user=POSTGRES_USER, host=POSTGRES_HOST, + port=POSTGRES_PORT, password=POSTGRES_PASSWORD, ) db_conn = LoggingDatabaseConnection(db_conn, db_engine, "tests") @@ -83,6 +90,7 @@ def setupdb(): db_conn = db_engine.module.connect( user=POSTGRES_USER, host=POSTGRES_HOST, + port=POSTGRES_PORT, password=POSTGRES_PASSWORD, dbname=POSTGRES_DBNAME_FOR_INITIAL_CREATE, ) From 42d8710f38d465904dfe2823d132c299b94f7a3a Mon Sep 17 00:00:00 2001 From: reivilibre Date: Tue, 5 Apr 2022 12:45:36 +0100 Subject: [PATCH 35/55] Fix a spec compliance issue where requests to the `/publicRooms` federation API would specify `limit` as a string. (#12364) --- changelog.d/12364.bugfix | 1 + synapse/federation/transport/client.py | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) create mode 100644 changelog.d/12364.bugfix diff --git a/changelog.d/12364.bugfix b/changelog.d/12364.bugfix new file mode 100644 index 0000000000..d042425d20 --- /dev/null +++ b/changelog.d/12364.bugfix @@ -0,0 +1 @@ +Fix a spec compliance issue where requests to the `/publicRooms` federation API would specify `limit` as a string. \ No newline at end of file diff --git a/synapse/federation/transport/client.py b/synapse/federation/transport/client.py index de6e5f44fe..d4b3cb3e98 100644 --- a/synapse/federation/transport/client.py +++ b/synapse/federation/transport/client.py @@ -481,7 +481,7 @@ class TransportLayerClient: if third_party_instance_id: data["third_party_instance_id"] = third_party_instance_id if limit: - data["limit"] = str(limit) + data["limit"] = limit if since_token: data["since"] = since_token @@ -509,7 +509,7 @@ class TransportLayerClient: if third_party_instance_id: args["third_party_instance_id"] = (third_party_instance_id,) if limit: - args["limit"] = [str(limit)] + args["limit"] = [limit] if since_token: args["since"] = [since_token] From ac80bfba4252c3bb8138cc3711271be63182eed8 Mon Sep 17 00:00:00 2001 From: Sean Quah Date: Tue, 5 Apr 2022 12:55:15 +0100 Subject: [PATCH 36/55] Tweak CHANGES.md --- CHANGES.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index 4cbe804d04..518781f931 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -7,7 +7,6 @@ to provide registration to anyone, consider adding [email](https://github.com/ma [recaptcha](https://matrix-org.github.io/synapse/v1.56/CAPTCHA_SETUP.html) or [token-based](https://matrix-org.github.io/synapse/v1.56/usage/administration/admin_api/registration_tokens.html) verification in order to prevent automated registration from bad actors. - This check can be disabled by setting the `enable_registration_without_verification` option in your homeserver configuration file to `true`. More details are available in the [upgrade notes](https://matrix-org.github.io/synapse/v1.56/upgrade.html#open-registration-without-verification-is-now-disabled-by-default). @@ -21,7 +20,6 @@ Internal Changes ---------------- - Bump the version of `black` for compatibility with the latest `click` release. ([\#12320](https://github.com/matrix-org/synapse/issues/12320)) -- Refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config flag `enable_registration_without_verification` is set to `true`. ([\#12322](https://github.com/matrix-org/synapse/issues/12322)) Synapse 1.56.0rc1 (2022-03-29) @@ -59,7 +57,7 @@ Internal Changes - Rename `shared_rooms` to `mutual_rooms` ([MSC2666](https://github.com/matrix-org/matrix-doc/pull/2666)), as per proposal changes. ([\#12036](https://github.com/matrix-org/synapse/issues/12036)) - Remove check on `update_user_directory` for shared rooms handler ([MSC2666](https://github.com/matrix-org/matrix-doc/pull/2666)), and update/expand documentation. ([\#12038](https://github.com/matrix-org/synapse/issues/12038)) - Refactor `create_new_client_event` to use a new parameter, `state_event_ids`, which accurately describes the usage with [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) instead of abusing `auth_event_ids`. ([\#12083](https://github.com/matrix-org/synapse/issues/12083), [\#12304](https://github.com/matrix-org/synapse/issues/12304)) -- Refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config flag `enable_registration_without_verification` is set to `true`. ([\#12091](https://github.com/matrix-org/synapse/issues/12091)) +- Refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config flag `enable_registration_without_verification` is set to `true`. ([\#12091](https://github.com/matrix-org/synapse/issues/12091), [\#12322](https://github.com/matrix-org/synapse/issues/12322)) - Add tests for database transaction callbacks. ([\#12198](https://github.com/matrix-org/synapse/issues/12198)) - Handle cancellation in `DatabasePool.runInteraction`. ([\#12199](https://github.com/matrix-org/synapse/issues/12199)) - Add missing type hints for cache storage. ([\#12216](https://github.com/matrix-org/synapse/issues/12216)) From d666fc02fab7421efb39a33800a83791f88bf9b2 Mon Sep 17 00:00:00 2001 From: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com> Date: Tue, 5 Apr 2022 14:54:41 +0200 Subject: [PATCH 37/55] Add type hints to some tests files (#12371) --- changelog.d/12371.misc | 1 + mypy.ini | 1 - tests/storage/test__base.py | 14 +++++-- tests/storage/test_monthly_active_users.py | 47 ++++++++++++---------- tests/storage/test_roommember.py | 25 +++++++----- tests/test_phone_home.py | 12 +++--- 6 files changed, 58 insertions(+), 42 deletions(-) create mode 100644 changelog.d/12371.misc diff --git a/changelog.d/12371.misc b/changelog.d/12371.misc new file mode 100644 index 0000000000..c5b6356799 --- /dev/null +++ b/changelog.d/12371.misc @@ -0,0 +1 @@ +Add type hints to tests files. \ No newline at end of file diff --git a/mypy.ini b/mypy.ini index ac2a5c753d..1f33246417 100644 --- a/mypy.ini +++ b/mypy.ini @@ -76,7 +76,6 @@ exclude = (?x) |tests/storage/test_base.py |tests/storage/test_roommember.py |tests/test_metrics.py - |tests/test_phone_home.py |tests/test_server.py |tests/test_state.py |tests/test_terms_auth.py diff --git a/tests/storage/test__base.py b/tests/storage/test__base.py index 4899cd5c36..366398e39d 100644 --- a/tests/storage/test__base.py +++ b/tests/storage/test__base.py @@ -14,12 +14,18 @@ # limitations under the License. import secrets +from typing import Any, Dict, Generator, List, Tuple + +from twisted.test.proto_helpers import MemoryReactor + +from synapse.server import HomeServer +from synapse.util import Clock from tests import unittest class UpsertManyTests(unittest.HomeserverTestCase): - def prepare(self, reactor, clock, hs): + def prepare(self, reactor: MemoryReactor, clock: Clock, hs: HomeServer) -> None: self.storage = hs.get_datastores().main self.table_name = "table_" + secrets.token_hex(6) @@ -40,11 +46,13 @@ class UpsertManyTests(unittest.HomeserverTestCase): ) ) - def _dump_to_tuple(self, res): + def _dump_to_tuple( + self, res: List[Dict[str, Any]] + ) -> Generator[Tuple[int, str, str], None, None]: for i in res: yield (i["id"], i["username"], i["value"]) - def test_upsert_many(self): + def test_upsert_many(self) -> None: """ Upsert_many will perform the upsert operation across a batch of data. """ diff --git a/tests/storage/test_monthly_active_users.py b/tests/storage/test_monthly_active_users.py index 79648d45db..60c8d37594 100644 --- a/tests/storage/test_monthly_active_users.py +++ b/tests/storage/test_monthly_active_users.py @@ -11,11 +11,15 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. +from typing import Any, Dict, List from unittest.mock import Mock from twisted.internet import defer +from twisted.test.proto_helpers import MemoryReactor from synapse.api.constants import UserTypes +from synapse.server import HomeServer +from synapse.util import Clock from tests import unittest from tests.test_utils import make_awaitable @@ -24,7 +28,7 @@ from tests.unittest import default_config, override_config FORTY_DAYS = 40 * 24 * 60 * 60 -def gen_3pids(count): +def gen_3pids(count: int) -> List[Dict[str, Any]]: """Generate `count` threepids as a list.""" return [ {"medium": "email", "address": "user%i@matrix.org" % i} for i in range(count) @@ -32,7 +36,7 @@ def gen_3pids(count): class MonthlyActiveUsersTestCase(unittest.HomeserverTestCase): - def default_config(self): + def default_config(self) -> Dict[str, Any]: config = default_config("test") config.update({"limit_usage_by_mau": True, "max_mau_value": 50}) @@ -44,10 +48,10 @@ class MonthlyActiveUsersTestCase(unittest.HomeserverTestCase): return config - def prepare(self, reactor, clock, homeserver): - self.store = homeserver.get_datastores().main + def prepare(self, reactor: MemoryReactor, clock: Clock, hs: HomeServer) -> None: + self.store = hs.get_datastores().main # Advance the clock a bit - reactor.advance(FORTY_DAYS) + self.reactor.advance(FORTY_DAYS) @override_config({"max_mau_value": 3, "mau_limit_reserved_threepids": gen_3pids(3)}) def test_initialise_reserved_users(self): @@ -245,7 +249,7 @@ class MonthlyActiveUsersTestCase(unittest.HomeserverTestCase): ) self.get_success(d) - self.store.upsert_monthly_active_user = Mock(return_value=make_awaitable(None)) + self.store.upsert_monthly_active_user = Mock(return_value=make_awaitable(None)) # type: ignore[assignment] d = self.store.populate_monthly_active_users(user_id) self.get_success(d) @@ -253,9 +257,9 @@ class MonthlyActiveUsersTestCase(unittest.HomeserverTestCase): self.store.upsert_monthly_active_user.assert_not_called() def test_populate_monthly_users_should_update(self): - self.store.upsert_monthly_active_user = Mock(return_value=make_awaitable(None)) + self.store.upsert_monthly_active_user = Mock(return_value=make_awaitable(None)) # type: ignore[assignment] - self.store.is_trial_user = Mock(return_value=defer.succeed(False)) + self.store.is_trial_user = Mock(return_value=defer.succeed(False)) # type: ignore[assignment] self.store.user_last_seen_monthly_active = Mock( return_value=defer.succeed(None) @@ -266,9 +270,9 @@ class MonthlyActiveUsersTestCase(unittest.HomeserverTestCase): self.store.upsert_monthly_active_user.assert_called_once() def test_populate_monthly_users_should_not_update(self): - self.store.upsert_monthly_active_user = Mock(return_value=make_awaitable(None)) + self.store.upsert_monthly_active_user = Mock(return_value=make_awaitable(None)) # type: ignore[assignment] - self.store.is_trial_user = Mock(return_value=defer.succeed(False)) + self.store.is_trial_user = Mock(return_value=defer.succeed(False)) # type: ignore[assignment] self.store.user_last_seen_monthly_active = Mock( return_value=defer.succeed(self.hs.get_clock().time_msec()) ) @@ -324,16 +328,15 @@ class MonthlyActiveUsersTestCase(unittest.HomeserverTestCase): count = self.get_success(self.store.get_monthly_active_count()) self.assertEqual(count, 0) - d = self.store.register_user( - user_id=support_user_id, password_hash=None, user_type=UserTypes.SUPPORT + self.get_success( + self.store.register_user( + user_id=support_user_id, password_hash=None, user_type=UserTypes.SUPPORT + ) ) - self.get_success(d) - d = self.store.upsert_monthly_active_user(support_user_id) - self.get_success(d) + self.get_success(self.store.upsert_monthly_active_user(support_user_id)) - d = self.store.get_monthly_active_count() - count = self.get_success(d) + count = self.get_success(self.store.get_monthly_active_count()) self.assertEqual(count, 0) # Note that the max_mau_value setting should not matter. @@ -352,7 +355,7 @@ class MonthlyActiveUsersTestCase(unittest.HomeserverTestCase): @override_config({"limit_usage_by_mau": False, "mau_stats_only": False}) def test_no_users_when_not_tracking(self): - self.store.upsert_monthly_active_user = Mock(return_value=make_awaitable(None)) + self.store.upsert_monthly_active_user = Mock(return_value=make_awaitable(None)) # type: ignore[assignment] self.get_success(self.store.populate_monthly_active_users("@user:sever")) @@ -388,16 +391,16 @@ class MonthlyActiveUsersTestCase(unittest.HomeserverTestCase): self.store.register_user(user_id=native_user1, password_hash=None) ) - count = self.get_success(self.store.get_monthly_active_count_by_service()) - self.assertEqual(count, {}) + count1 = self.get_success(self.store.get_monthly_active_count_by_service()) + self.assertEqual(count1, {}) self.get_success(self.store.upsert_monthly_active_user(native_user1)) self.get_success(self.store.upsert_monthly_active_user(appservice1_user1)) self.get_success(self.store.upsert_monthly_active_user(appservice1_user2)) self.get_success(self.store.upsert_monthly_active_user(appservice2_user1)) - count = self.get_success(self.store.get_monthly_active_count()) - self.assertEqual(count, 1) + count2 = self.get_success(self.store.get_monthly_active_count()) + self.assertEqual(count2, 1) d = self.store.get_monthly_active_count_by_service() result = self.get_success(d) diff --git a/tests/storage/test_roommember.py b/tests/storage/test_roommember.py index b8f09a8ee0..a2a9c05f24 100644 --- a/tests/storage/test_roommember.py +++ b/tests/storage/test_roommember.py @@ -12,11 +12,14 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. +from twisted.test.proto_helpers import MemoryReactor from synapse.api.constants import Membership from synapse.rest.admin import register_servlets_for_client_rest_resource from synapse.rest.client import login, room +from synapse.server import HomeServer from synapse.types import UserID, create_requester +from synapse.util import Clock from tests import unittest from tests.server import TestHomeServer @@ -31,7 +34,7 @@ class RoomMemberStoreTestCase(unittest.HomeserverTestCase): room.register_servlets, ] - def prepare(self, reactor, clock, hs: TestHomeServer): + def prepare(self, reactor: MemoryReactor, clock: Clock, hs: TestHomeServer) -> None: # We can't test the RoomMemberStore on its own without the other event # storage logic @@ -44,7 +47,7 @@ class RoomMemberStoreTestCase(unittest.HomeserverTestCase): # User elsewhere on another host self.u_charlie = UserID.from_string("@charlie:elsewhere") - def test_one_member(self): + def test_one_member(self) -> None: # Alice creates the room, and is automatically joined self.room = self.helper.create_room_as(self.u_alice, tok=self.t_alice) @@ -57,7 +60,7 @@ class RoomMemberStoreTestCase(unittest.HomeserverTestCase): self.assertEqual([self.room], [m.room_id for m in rooms_for_user]) - def test_count_known_servers(self): + def test_count_known_servers(self) -> None: """ _count_known_servers will calculate how many servers are in a room. """ @@ -68,7 +71,7 @@ class RoomMemberStoreTestCase(unittest.HomeserverTestCase): servers = self.get_success(self.store._count_known_servers()) self.assertEqual(servers, 2) - def test_count_known_servers_stat_counter_disabled(self): + def test_count_known_servers_stat_counter_disabled(self) -> None: """ If enabled, the metrics for how many servers are known will be counted. """ @@ -85,7 +88,7 @@ class RoomMemberStoreTestCase(unittest.HomeserverTestCase): @unittest.override_config( {"enable_metrics": True, "metrics_flags": {"known_servers": True}} ) - def test_count_known_servers_stat_counter_enabled(self): + def test_count_known_servers_stat_counter_enabled(self) -> None: """ If enabled, the metrics for how many servers are known will be counted. """ @@ -107,7 +110,7 @@ class RoomMemberStoreTestCase(unittest.HomeserverTestCase): # It now knows about Charlie's server. self.assertEqual(self.store._known_servers_count, 2) - def test_get_joined_users_from_context(self): + def test_get_joined_users_from_context(self) -> None: room = self.helper.create_room_as(self.u_alice, tok=self.t_alice) bob_event = self.get_success( event_injection.inject_member_event( @@ -161,7 +164,7 @@ class RoomMemberStoreTestCase(unittest.HomeserverTestCase): ) self.assertEqual(users.keys(), {self.u_alice, self.u_bob}) - def test__null_byte_in_display_name_properly_handled(self): + def test__null_byte_in_display_name_properly_handled(self) -> None: room = self.helper.create_room_as(self.u_alice, tok=self.t_alice) res = self.get_success( @@ -211,11 +214,11 @@ class RoomMemberStoreTestCase(unittest.HomeserverTestCase): class CurrentStateMembershipUpdateTestCase(unittest.HomeserverTestCase): - def prepare(self, reactor, clock, homeserver): - self.store = homeserver.get_datastores().main - self.room_creator = homeserver.get_room_creation_handler() + def prepare(self, reactor: MemoryReactor, clock: Clock, hs: HomeServer) -> None: + self.store = hs.get_datastores().main + self.room_creator = hs.get_room_creation_handler() - def test_can_rerun_update(self): + def test_can_rerun_update(self) -> None: # First make sure we have completed all updates. self.wait_for_background_updates() diff --git a/tests/test_phone_home.py b/tests/test_phone_home.py index 09707a74d7..b01cae6e5d 100644 --- a/tests/test_phone_home.py +++ b/tests/test_phone_home.py @@ -16,23 +16,24 @@ import resource from unittest import mock from synapse.app.phone_stats_home import phone_stats_home +from synapse.types import JsonDict from tests.unittest import HomeserverTestCase class PhoneHomeStatsTestCase(HomeserverTestCase): - def test_performance_frozen_clock(self): + def test_performance_frozen_clock(self) -> None: """ If time doesn't move, don't error out. """ past_stats = [ (self.hs.get_clock().time(), resource.getrusage(resource.RUSAGE_SELF)) ] - stats = {} + stats: JsonDict = {} self.get_success(phone_stats_home(self.hs, stats, past_stats)) self.assertEqual(stats["cpu_average"], 0) - def test_performance_100(self): + def test_performance_100(self) -> None: """ 1 second of usage over 1 second is 100% CPU usage. """ @@ -43,7 +44,8 @@ class PhoneHomeStatsTestCase(HomeserverTestCase): old_resource.ru_maxrss = real_res.ru_maxrss past_stats = [(self.hs.get_clock().time(), old_resource)] - stats = {} + stats: JsonDict = {} self.reactor.advance(1) - self.get_success(phone_stats_home(self.hs, stats, past_stats)) + # `old_resource` has type `Mock` instead of `struct_rusage` + self.get_success(phone_stats_home(self.hs, stats, past_stats)) # type: ignore[arg-type] self.assertApproximates(stats["cpu_average"], 100, tolerance=2.5) From 66053b6bfbc27a6e281655ebca8f2abbee730135 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Tue, 5 Apr 2022 14:26:41 +0100 Subject: [PATCH 38/55] Prefill more stream change caches. (#12372) --- changelog.d/12372.feature | 1 + synapse/replication/slave/storage/devices.py | 25 +--------- synapse/storage/database.py | 43 ++++++++++------- synapse/storage/databases/main/__init__.py | 21 -------- synapse/storage/databases/main/devices.py | 50 ++++++++++++++++++++ synapse/storage/databases/main/receipts.py | 13 ++++- 6 files changed, 92 insertions(+), 61 deletions(-) create mode 100644 changelog.d/12372.feature diff --git a/changelog.d/12372.feature b/changelog.d/12372.feature new file mode 100644 index 0000000000..34bb60e966 --- /dev/null +++ b/changelog.d/12372.feature @@ -0,0 +1 @@ +Reduce overhead of restarting synchrotrons. diff --git a/synapse/replication/slave/storage/devices.py b/synapse/replication/slave/storage/devices.py index 00a634d3a9..30717c2bd0 100644 --- a/synapse/replication/slave/storage/devices.py +++ b/synapse/replication/slave/storage/devices.py @@ -20,7 +20,6 @@ from synapse.replication.tcp.streams._base import DeviceListsStream, UserSignatu from synapse.storage.database import DatabasePool, LoggingDatabaseConnection from synapse.storage.databases.main.devices import DeviceWorkerStore from synapse.storage.databases.main.end_to_end_keys import EndToEndKeyWorkerStore -from synapse.util.caches.stream_change_cache import StreamChangeCache if TYPE_CHECKING: from synapse.server import HomeServer @@ -33,8 +32,6 @@ class SlavedDeviceStore(EndToEndKeyWorkerStore, DeviceWorkerStore, BaseSlavedSto db_conn: LoggingDatabaseConnection, hs: "HomeServer", ): - super().__init__(database, db_conn, hs) - self.hs = hs self._device_list_id_gen = SlavedIdTracker( @@ -47,26 +44,8 @@ class SlavedDeviceStore(EndToEndKeyWorkerStore, DeviceWorkerStore, BaseSlavedSto ("device_lists_changes_in_room", "stream_id"), ], ) - device_list_max = self._device_list_id_gen.get_current_token() - device_list_prefill, min_device_list_id = self.db_pool.get_cache_dict( - db_conn, - "device_lists_stream", - entity_column="user_id", - stream_column="stream_id", - max_value=device_list_max, - limit=1000, - ) - self._device_list_stream_cache = StreamChangeCache( - "DeviceListStreamChangeCache", - min_device_list_id, - prefilled_cache=device_list_prefill, - ) - self._user_signature_stream_cache = StreamChangeCache( - "UserSignatureStreamChangeCache", device_list_max - ) - self._device_list_federation_stream_cache = StreamChangeCache( - "DeviceListFederationStreamChangeCache", device_list_max - ) + + super().__init__(database, db_conn, hs) def get_device_stream_token(self) -> int: return self._device_list_id_gen.get_current_token() diff --git a/synapse/storage/database.py b/synapse/storage/database.py index 0264dea61d..12750d9b89 100644 --- a/synapse/storage/database.py +++ b/synapse/storage/database.py @@ -2030,29 +2030,40 @@ class DatabasePool: max_value: int, limit: int = 100000, ) -> Tuple[Dict[Any, int], int]: - # Fetch a mapping of room_id -> max stream position for "recent" rooms. - # It doesn't really matter how many we get, the StreamChangeCache will - # do the right thing to ensure it respects the max size of cache. - sql = ( - "SELECT %(entity)s, MAX(%(stream)s) FROM %(table)s" - " WHERE %(stream)s > ? - %(limit)s" - " GROUP BY %(entity)s" - ) % { - "table": table, - "entity": entity_column, - "stream": stream_column, - "limit": limit, - } + """Gets roughly the last N changes in the given stream table as a + map from entity to the stream ID of the most recent change. + + Also returns the minimum stream ID. + """ + + # This may return many rows for the same entity, but the `limit` is only + # a suggestion so we don't care that much. + # + # Note: Some stream tables can have multiple rows with the same stream + # ID. Instead of handling this with complicated SQL, we instead simply + # add one to the returned minimum stream ID to ensure correctness. + sql = f""" + SELECT {entity_column}, {stream_column} + FROM {table} + ORDER BY {stream_column} DESC + LIMIT ? + """ txn = db_conn.cursor(txn_name="get_cache_dict") - txn.execute(sql, (int(max_value),)) + txn.execute(sql, (limit,)) - cache = {row[0]: int(row[1]) for row in txn} + # The rows come out in reverse stream ID order, so we want to keep the + # stream ID of the first row for each entity. + cache: Dict[Any, int] = {} + for row in txn: + cache.setdefault(row[0], int(row[1])) txn.close() if cache: - min_val = min(cache.values()) + # We add one here as we don't know if we have all rows for the + # minimum stream ID. + min_val = min(cache.values()) + 1 else: min_val = max_value diff --git a/synapse/storage/databases/main/__init__.py b/synapse/storage/databases/main/__init__.py index d4a38daa9a..951031af50 100644 --- a/synapse/storage/databases/main/__init__.py +++ b/synapse/storage/databases/main/__init__.py @@ -183,27 +183,6 @@ class DataStore( super().__init__(database, db_conn, hs) - device_list_max = self._device_list_id_gen.get_current_token() - device_list_prefill, min_device_list_id = self.db_pool.get_cache_dict( - db_conn, - "device_lists_stream", - entity_column="user_id", - stream_column="stream_id", - max_value=device_list_max, - limit=1000, - ) - self._device_list_stream_cache = StreamChangeCache( - "DeviceListStreamChangeCache", - min_device_list_id, - prefilled_cache=device_list_prefill, - ) - self._user_signature_stream_cache = StreamChangeCache( - "UserSignatureStreamChangeCache", device_list_max - ) - self._device_list_federation_stream_cache = StreamChangeCache( - "DeviceListFederationStreamChangeCache", device_list_max - ) - events_max = self._stream_id_gen.get_current_token() curr_state_delta_prefill, min_curr_state_delta_id = self.db_pool.get_cache_dict( db_conn, diff --git a/synapse/storage/databases/main/devices.py b/synapse/storage/databases/main/devices.py index 07eea4b3d2..dc8009b23d 100644 --- a/synapse/storage/databases/main/devices.py +++ b/synapse/storage/databases/main/devices.py @@ -46,6 +46,7 @@ from synapse.types import JsonDict, get_verify_key_from_cross_signing_key from synapse.util import json_decoder, json_encoder from synapse.util.caches.descriptors import cached, cachedList from synapse.util.caches.lrucache import LruCache +from synapse.util.caches.stream_change_cache import StreamChangeCache from synapse.util.iterutils import batch_iter from synapse.util.stringutils import shortstr @@ -71,6 +72,55 @@ class DeviceWorkerStore(SQLBaseStore): ): super().__init__(database, db_conn, hs) + device_list_max = self._device_list_id_gen.get_current_token() + device_list_prefill, min_device_list_id = self.db_pool.get_cache_dict( + db_conn, + "device_lists_stream", + entity_column="user_id", + stream_column="stream_id", + max_value=device_list_max, + limit=10000, + ) + self._device_list_stream_cache = StreamChangeCache( + "DeviceListStreamChangeCache", + min_device_list_id, + prefilled_cache=device_list_prefill, + ) + + ( + user_signature_stream_prefill, + user_signature_stream_list_id, + ) = self.db_pool.get_cache_dict( + db_conn, + "user_signature_stream", + entity_column="from_user_id", + stream_column="stream_id", + max_value=device_list_max, + limit=1000, + ) + self._user_signature_stream_cache = StreamChangeCache( + "UserSignatureStreamChangeCache", + user_signature_stream_list_id, + prefilled_cache=user_signature_stream_prefill, + ) + + ( + device_list_federation_prefill, + device_list_federation_list_id, + ) = self.db_pool.get_cache_dict( + db_conn, + "device_lists_outbound_pokes", + entity_column="destination", + stream_column="stream_id", + max_value=device_list_max, + limit=10000, + ) + self._device_list_federation_stream_cache = StreamChangeCache( + "DeviceListFederationStreamChangeCache", + device_list_federation_list_id, + prefilled_cache=device_list_federation_prefill, + ) + if hs.config.worker.run_background_tasks: self._clock.looping_call( self._prune_old_outbound_device_pokes, 60 * 60 * 1000 diff --git a/synapse/storage/databases/main/receipts.py b/synapse/storage/databases/main/receipts.py index e6f97aeece..332e901dda 100644 --- a/synapse/storage/databases/main/receipts.py +++ b/synapse/storage/databases/main/receipts.py @@ -98,8 +98,19 @@ class ReceiptsWorkerStore(SQLBaseStore): super().__init__(database, db_conn, hs) + max_receipts_stream_id = self.get_max_receipt_stream_id() + receipts_stream_prefill, min_receipts_stream_id = self.db_pool.get_cache_dict( + db_conn, + "receipts_linearized", + entity_column="room_id", + stream_column="stream_id", + max_value=max_receipts_stream_id, + limit=10000, + ) self._receipts_stream_cache = StreamChangeCache( - "ReceiptsRoomChangeCache", self.get_max_receipt_stream_id() + "ReceiptsRoomChangeCache", + min_receipts_stream_id, + prefilled_cache=receipts_stream_prefill, ) def get_max_receipt_stream_id(self) -> int: From 41b5f72677ea9763f3cf920d4f6df507653222f2 Mon Sep 17 00:00:00 2001 From: Sean Quah <8349537+squahtx@users.noreply.github.com> Date: Tue, 5 Apr 2022 14:56:09 +0100 Subject: [PATCH 39/55] Convert `Linearizer` tests from `inlineCallbacks` to async (#12353) Signed-off-by: Sean Quah --- changelog.d/12353.misc | 1 + tests/util/test_linearizer.py | 222 ++++++++++++++++++++-------------- 2 files changed, 133 insertions(+), 90 deletions(-) create mode 100644 changelog.d/12353.misc diff --git a/changelog.d/12353.misc b/changelog.d/12353.misc new file mode 100644 index 0000000000..1d681fb0e3 --- /dev/null +++ b/changelog.d/12353.misc @@ -0,0 +1 @@ +Convert `Linearizer` tests from `inlineCallbacks` to async. diff --git a/tests/util/test_linearizer.py b/tests/util/test_linearizer.py index c4a3917b23..fa132391a1 100644 --- a/tests/util/test_linearizer.py +++ b/tests/util/test_linearizer.py @@ -13,160 +13,202 @@ # See the License for the specific language governing permissions and # limitations under the License. +from typing import Callable, Hashable, Tuple + from twisted.internet import defer, reactor -from twisted.internet.defer import CancelledError +from twisted.internet.base import ReactorBase +from twisted.internet.defer import CancelledError, Deferred from synapse.logging.context import LoggingContext, current_context -from synapse.util import Clock from synapse.util.async_helpers import Linearizer from tests import unittest class LinearizerTestCase(unittest.TestCase): - @defer.inlineCallbacks - def test_linearizer(self): + def _start_task( + self, linearizer: Linearizer, key: Hashable + ) -> Tuple["Deferred[None]", "Deferred[None]", Callable[[], None]]: + """Starts a task which acquires the linearizer lock, blocks, then completes. + + Args: + linearizer: The `Linearizer`. + key: The `Linearizer` key. + + Returns: + A tuple containing: + * A cancellable `Deferred` for the entire task. + * A `Deferred` that resolves once the task acquires the lock. + * A function that unblocks the task. Must be called by the caller + to allow the task to release the lock and complete. + """ + acquired_d: "Deferred[None]" = Deferred() + unblock_d: "Deferred[None]" = Deferred() + + async def task() -> None: + with await linearizer.queue(key): + acquired_d.callback(None) + await unblock_d + + d = defer.ensureDeferred(task()) + + def unblock() -> None: + unblock_d.callback(None) + # The next task, if it exists, will acquire the lock and require a kick of + # the reactor to advance. + self._pump() + + return d, acquired_d, unblock + + def _pump(self) -> None: + """Pump the reactor to advance `Linearizer`s.""" + assert isinstance(reactor, ReactorBase) + while reactor.getDelayedCalls(): + reactor.runUntilCurrent() + + def test_linearizer(self) -> None: + """Tests that a task is queued up behind an earlier task.""" linearizer = Linearizer() key = object() - d1 = linearizer.queue(key) - cm1 = yield d1 + _, acquired_d1, unblock1 = self._start_task(linearizer, key) + self.assertTrue(acquired_d1.called) - d2 = linearizer.queue(key) - self.assertFalse(d2.called) + _, acquired_d2, unblock2 = self._start_task(linearizer, key) + self.assertFalse(acquired_d2.called) - with cm1: - self.assertFalse(d2.called) + # Once the first task is done, the second task can continue. + unblock1() + self.assertTrue(acquired_d2.called) - with (yield d2): - pass + unblock2() - @defer.inlineCallbacks - def test_linearizer_is_queued(self): + def test_linearizer_is_queued(self) -> None: + """Tests `Linearizer.is_queued`. + + Runs through the same scenario as `test_linearizer`. + """ linearizer = Linearizer() key = object() - d1 = linearizer.queue(key) - cm1 = yield d1 + _, acquired_d1, unblock1 = self._start_task(linearizer, key) + self.assertTrue(acquired_d1.called) - # Since d1 gets called immediately, "is_queued" should return false. + # Since the first task acquires the lock immediately, "is_queued" should return + # false. self.assertFalse(linearizer.is_queued(key)) - d2 = linearizer.queue(key) - self.assertFalse(d2.called) + _, acquired_d2, unblock2 = self._start_task(linearizer, key) + self.assertFalse(acquired_d2.called) - # Now d2 is queued up behind successful completion of cm1 + # Now the second task is queued up behind the first. self.assertTrue(linearizer.is_queued(key)) - with cm1: - self.assertFalse(d2.called) + unblock1() - # cm1 still not done, so d2 still queued. - self.assertTrue(linearizer.is_queued(key)) - - # And now d2 is called and nothing is in the queue again + # And now the second task acquires the lock and nothing is in the queue again. + self.assertTrue(acquired_d2.called) self.assertFalse(linearizer.is_queued(key)) - with (yield d2): - self.assertFalse(linearizer.is_queued(key)) - + unblock2() self.assertFalse(linearizer.is_queued(key)) - def test_lots_of_queued_things(self): - # we have one slow thing, and lots of fast things queued up behind it. - # it should *not* explode the stack. + def test_lots_of_queued_things(self) -> None: + """Tests lots of fast things queued up behind a slow thing. + + The stack should *not* explode when the slow thing completes. + """ linearizer = Linearizer() + key = "" - @defer.inlineCallbacks - def func(i, sleep=False): + async def func(i: int) -> None: with LoggingContext("func(%s)" % i) as lc: - with (yield linearizer.queue("")): + with (await linearizer.queue(key)): self.assertEqual(current_context(), lc) - if sleep: - yield Clock(reactor).sleep(0) self.assertEqual(current_context(), lc) - func(0, sleep=True) + _, _, unblock = self._start_task(linearizer, key) for i in range(1, 100): - func(i) + defer.ensureDeferred(func(i)) - return func(1000) + d = defer.ensureDeferred(func(1000)) + unblock() + self.successResultOf(d) - @defer.inlineCallbacks - def test_multiple_entries(self): + def test_multiple_entries(self) -> None: + """Tests a `Linearizer` with a concurrency above 1.""" limiter = Linearizer(max_count=3) key = object() - d1 = limiter.queue(key) - cm1 = yield d1 + _, acquired_d1, unblock1 = self._start_task(limiter, key) + self.assertTrue(acquired_d1.called) - d2 = limiter.queue(key) - cm2 = yield d2 + _, acquired_d2, unblock2 = self._start_task(limiter, key) + self.assertTrue(acquired_d2.called) - d3 = limiter.queue(key) - cm3 = yield d3 + _, acquired_d3, unblock3 = self._start_task(limiter, key) + self.assertTrue(acquired_d3.called) - d4 = limiter.queue(key) - self.assertFalse(d4.called) + # These next two tasks have to wait. + _, acquired_d4, unblock4 = self._start_task(limiter, key) + self.assertFalse(acquired_d4.called) - d5 = limiter.queue(key) - self.assertFalse(d5.called) + _, acquired_d5, unblock5 = self._start_task(limiter, key) + self.assertFalse(acquired_d5.called) - with cm1: - self.assertFalse(d4.called) - self.assertFalse(d5.called) + # Once the first task completes, the fourth task can continue. + unblock1() + self.assertTrue(acquired_d4.called) + self.assertFalse(acquired_d5.called) - cm4 = yield d4 - self.assertFalse(d5.called) + # Once the third task completes, the fifth task can continue. + unblock3() + self.assertTrue(acquired_d5.called) - with cm3: - self.assertFalse(d5.called) + # Make all tasks finish. + unblock2() + unblock4() + unblock5() - cm5 = yield d5 + # The next task shouldn't have to wait. + _, acquired_d6, unblock6 = self._start_task(limiter, key) + self.assertTrue(acquired_d6) + unblock6() - with cm2: - pass - - with cm4: - pass - - with cm5: - pass - - d6 = limiter.queue(key) - with (yield d6): - pass - - @defer.inlineCallbacks - def test_cancellation(self): + def test_cancellation(self) -> None: + """Tests cancellation while waiting for a `Linearizer`.""" linearizer = Linearizer() key = object() - d1 = linearizer.queue(key) - cm1 = yield d1 + d1, acquired_d1, unblock1 = self._start_task(linearizer, key) + self.assertTrue(acquired_d1.called) - d2 = linearizer.queue(key) - self.assertFalse(d2.called) + # Create a second task, waiting for the first task. + d2, acquired_d2, _ = self._start_task(linearizer, key) + self.assertFalse(acquired_d2.called) - d3 = linearizer.queue(key) - self.assertFalse(d3.called) + # Create a third task, waiting for the second task. + d3, acquired_d3, unblock3 = self._start_task(linearizer, key) + self.assertFalse(acquired_d3.called) + # Cancel the waiting second task. d2.cancel() - with cm1: - pass + unblock1() + self.successResultOf(d1) self.assertTrue(d2.called) - try: - yield d2 - self.fail("Expected d2 to raise CancelledError") - except CancelledError: - pass + self.failureResultOf(d2, CancelledError) - with (yield d3): - pass + # The third task should continue running. + self.assertTrue( + acquired_d3.called, + "Third task did not get the lock after the second task was cancelled", + ) + unblock3() + self.successResultOf(d3) From 800ba87cc881856adae19ec40485578356398639 Mon Sep 17 00:00:00 2001 From: Sean Quah <8349537+squahtx@users.noreply.github.com> Date: Tue, 5 Apr 2022 15:43:52 +0100 Subject: [PATCH 40/55] Refactor and convert `Linearizer` to async (#12357) Refactor and convert `Linearizer` to async. This makes a `Linearizer` cancellation bug easier to fix. Also refactor to use an async context manager, which eliminates an unlikely footgun where code that doesn't immediately use the context manager could forget to release the lock. Signed-off-by: Sean Quah --- changelog.d/12357.misc | 1 + synapse/federation/federation_server.py | 10 +- synapse/handlers/appservice.py | 6 +- synapse/handlers/device.py | 2 +- synapse/handlers/e2e_keys.py | 4 +- synapse/handlers/e2e_room_keys.py | 14 +- synapse/handlers/federation.py | 2 +- synapse/handlers/federation_event.py | 2 +- synapse/handlers/message.py | 2 +- synapse/handlers/presence.py | 4 +- synapse/handlers/read_marker.py | 2 +- synapse/handlers/room.py | 2 +- synapse/handlers/room_member.py | 4 +- synapse/handlers/sso.py | 2 +- synapse/push/bulk_push_rule_evaluator.py | 2 +- synapse/replication/tcp/client.py | 2 +- synapse/rest/media/v1/media_repository.py | 6 +- synapse/state/__init__.py | 2 +- synapse/storage/databases/main/roommember.py | 2 +- synapse/util/async_helpers.py | 144 +++++++++---------- tests/util/test_linearizer.py | 4 +- 21 files changed, 104 insertions(+), 115 deletions(-) create mode 100644 changelog.d/12357.misc diff --git a/changelog.d/12357.misc b/changelog.d/12357.misc new file mode 100644 index 0000000000..d571ae034c --- /dev/null +++ b/changelog.d/12357.misc @@ -0,0 +1 @@ +Refactor `Linearizer`, convert methods to async and use an async context manager. diff --git a/synapse/federation/federation_server.py b/synapse/federation/federation_server.py index c7400c737b..69d833585f 100644 --- a/synapse/federation/federation_server.py +++ b/synapse/federation/federation_server.py @@ -188,7 +188,7 @@ class FederationServer(FederationBase): async def on_backfill_request( self, origin: str, room_id: str, versions: List[str], limit: int ) -> Tuple[int, Dict[str, Any]]: - with (await self._server_linearizer.queue((origin, room_id))): + async with self._server_linearizer.queue((origin, room_id)): origin_host, _ = parse_server_name(origin) await self.check_server_matches_acl(origin_host, room_id) @@ -218,7 +218,7 @@ class FederationServer(FederationBase): Tuple indicating the response status code and dictionary response body including `event_id`. """ - with (await self._server_linearizer.queue((origin, room_id))): + async with self._server_linearizer.queue((origin, room_id)): origin_host, _ = parse_server_name(origin) await self.check_server_matches_acl(origin_host, room_id) @@ -529,7 +529,7 @@ class FederationServer(FederationBase): # in the cache so we could return it without waiting for the linearizer # - but that's non-trivial to get right, and anyway somewhat defeats # the point of the linearizer. - with (await self._server_linearizer.queue((origin, room_id))): + async with self._server_linearizer.queue((origin, room_id)): resp: JsonDict = dict( await self._state_resp_cache.wrap( (room_id, event_id), @@ -883,7 +883,7 @@ class FederationServer(FederationBase): async def on_event_auth( self, origin: str, room_id: str, event_id: str ) -> Tuple[int, Dict[str, Any]]: - with (await self._server_linearizer.queue((origin, room_id))): + async with self._server_linearizer.queue((origin, room_id)): origin_host, _ = parse_server_name(origin) await self.check_server_matches_acl(origin_host, room_id) @@ -945,7 +945,7 @@ class FederationServer(FederationBase): latest_events: List[str], limit: int, ) -> Dict[str, list]: - with (await self._server_linearizer.queue((origin, room_id))): + async with self._server_linearizer.queue((origin, room_id)): origin_host, _ = parse_server_name(origin) await self.check_server_matches_acl(origin_host, room_id) diff --git a/synapse/handlers/appservice.py b/synapse/handlers/appservice.py index 316c4b677c..1b57840506 100644 --- a/synapse/handlers/appservice.py +++ b/synapse/handlers/appservice.py @@ -330,10 +330,8 @@ class ApplicationServicesHandler: continue # Since we read/update the stream position for this AS/stream - with ( - await self._ephemeral_events_linearizer.queue( - (service.id, stream_key) - ) + async with self._ephemeral_events_linearizer.queue( + (service.id, stream_key) ): if stream_key == "receipt_key": events = await self._handle_receipts(service, new_token) diff --git a/synapse/handlers/device.py b/synapse/handlers/device.py index c710c02cf9..ffa28b2a30 100644 --- a/synapse/handlers/device.py +++ b/synapse/handlers/device.py @@ -833,7 +833,7 @@ class DeviceListUpdater: async def _handle_device_updates(self, user_id: str) -> None: "Actually handle pending updates." - with (await self._remote_edu_linearizer.queue(user_id)): + async with self._remote_edu_linearizer.queue(user_id): pending_updates = self._pending_updates.pop(user_id, []) if not pending_updates: # This can happen since we batch updates diff --git a/synapse/handlers/e2e_keys.py b/synapse/handlers/e2e_keys.py index d96456cd40..d6714228ef 100644 --- a/synapse/handlers/e2e_keys.py +++ b/synapse/handlers/e2e_keys.py @@ -118,7 +118,7 @@ class E2eKeysHandler: from_device_id: the device making the query. This is used to limit the number of in-flight queries at a time. """ - with await self._query_devices_linearizer.queue((from_user_id, from_device_id)): + async with self._query_devices_linearizer.queue((from_user_id, from_device_id)): device_keys_query: Dict[str, Iterable[str]] = query_body.get( "device_keys", {} ) @@ -1386,7 +1386,7 @@ class SigningKeyEduUpdater: device_handler = self.e2e_keys_handler.device_handler device_list_updater = device_handler.device_list_updater - with (await self._remote_edu_linearizer.queue(user_id)): + async with self._remote_edu_linearizer.queue(user_id): pending_updates = self._pending_updates.pop(user_id, []) if not pending_updates: # This can happen since we batch updates diff --git a/synapse/handlers/e2e_room_keys.py b/synapse/handlers/e2e_room_keys.py index 52e44a2d42..446f509bdc 100644 --- a/synapse/handlers/e2e_room_keys.py +++ b/synapse/handlers/e2e_room_keys.py @@ -83,7 +83,7 @@ class E2eRoomKeysHandler: # we deliberately take the lock to get keys so that changing the version # works atomically - with (await self._upload_linearizer.queue(user_id)): + async with self._upload_linearizer.queue(user_id): # make sure the backup version exists try: await self.store.get_e2e_room_keys_version_info(user_id, version) @@ -126,7 +126,7 @@ class E2eRoomKeysHandler: """ # lock for consistency with uploading - with (await self._upload_linearizer.queue(user_id)): + async with self._upload_linearizer.queue(user_id): # make sure the backup version exists try: version_info = await self.store.get_e2e_room_keys_version_info( @@ -187,7 +187,7 @@ class E2eRoomKeysHandler: # TODO: Validate the JSON to make sure it has the right keys. # XXX: perhaps we should use a finer grained lock here? - with (await self._upload_linearizer.queue(user_id)): + async with self._upload_linearizer.queue(user_id): # Check that the version we're trying to upload is the current version try: @@ -332,7 +332,7 @@ class E2eRoomKeysHandler: # TODO: Validate the JSON to make sure it has the right keys. # lock everyone out until we've switched version - with (await self._upload_linearizer.queue(user_id)): + async with self._upload_linearizer.queue(user_id): new_version = await self.store.create_e2e_room_keys_version( user_id, version_info ) @@ -359,7 +359,7 @@ class E2eRoomKeysHandler: } """ - with (await self._upload_linearizer.queue(user_id)): + async with self._upload_linearizer.queue(user_id): try: res = await self.store.get_e2e_room_keys_version_info(user_id, version) except StoreError as e: @@ -383,7 +383,7 @@ class E2eRoomKeysHandler: NotFoundError: if this backup version doesn't exist """ - with (await self._upload_linearizer.queue(user_id)): + async with self._upload_linearizer.queue(user_id): try: await self.store.delete_e2e_room_keys_version(user_id, version) except StoreError as e: @@ -413,7 +413,7 @@ class E2eRoomKeysHandler: raise SynapseError( 400, "Version in body does not match", Codes.INVALID_PARAM ) - with (await self._upload_linearizer.queue(user_id)): + async with self._upload_linearizer.queue(user_id): try: old_info = await self.store.get_e2e_room_keys_version_info( user_id, version diff --git a/synapse/handlers/federation.py b/synapse/handlers/federation.py index 350ec9c03a..78d149905f 100644 --- a/synapse/handlers/federation.py +++ b/synapse/handlers/federation.py @@ -151,7 +151,7 @@ class FederationHandler: return. This is used as part of the heuristic to decide if we should back paginate. """ - with (await self._room_backfill.queue(room_id)): + async with self._room_backfill.queue(room_id): return await self._maybe_backfill_inner(room_id, current_depth, limit) async def _maybe_backfill_inner( diff --git a/synapse/handlers/federation_event.py b/synapse/handlers/federation_event.py index e7b9f15e13..03c1197c99 100644 --- a/synapse/handlers/federation_event.py +++ b/synapse/handlers/federation_event.py @@ -224,7 +224,7 @@ class FederationEventHandler: len(missing_prevs), shortstr(missing_prevs), ) - with (await self._room_pdu_linearizer.queue(pdu.room_id)): + async with self._room_pdu_linearizer.queue(pdu.room_id): logger.info( "Acquired room lock to fetch %d missing prev_events", len(missing_prevs), diff --git a/synapse/handlers/message.py b/synapse/handlers/message.py index 766f597a55..7db6905c61 100644 --- a/synapse/handlers/message.py +++ b/synapse/handlers/message.py @@ -851,7 +851,7 @@ class EventCreationHandler: # a situation where event persistence can't keep up, causing # extremities to pile up, which in turn leads to state resolution # taking longer. - with (await self.limiter.queue(event_dict["room_id"])): + async with self.limiter.queue(event_dict["room_id"]): if txn_id and requester.access_token_id: existing_event_id = await self.store.get_event_id_from_transaction_id( event_dict["room_id"], diff --git a/synapse/handlers/presence.py b/synapse/handlers/presence.py index dace31d87e..209a4b0e52 100644 --- a/synapse/handlers/presence.py +++ b/synapse/handlers/presence.py @@ -1030,7 +1030,7 @@ class PresenceHandler(BasePresenceHandler): is_syncing: Whether or not the user is now syncing sync_time_msec: Time in ms when the user was last syncing """ - with (await self.external_sync_linearizer.queue(process_id)): + async with self.external_sync_linearizer.queue(process_id): prev_state = await self.current_state_for_user(user_id) process_presence = self.external_process_to_current_syncs.setdefault( @@ -1071,7 +1071,7 @@ class PresenceHandler(BasePresenceHandler): Used when the process has stopped/disappeared. """ - with (await self.external_sync_linearizer.queue(process_id)): + async with self.external_sync_linearizer.queue(process_id): process_presence = self.external_process_to_current_syncs.pop( process_id, set() ) diff --git a/synapse/handlers/read_marker.py b/synapse/handlers/read_marker.py index bad1acc634..05122fd5a6 100644 --- a/synapse/handlers/read_marker.py +++ b/synapse/handlers/read_marker.py @@ -40,7 +40,7 @@ class ReadMarkerHandler: the read marker has changed. """ - with await self.read_marker_linearizer.queue((room_id, user_id)): + async with self.read_marker_linearizer.queue((room_id, user_id)): existing_read_marker = await self.store.get_account_data_for_room_and_type( user_id, room_id, "m.fully_read" ) diff --git a/synapse/handlers/room.py b/synapse/handlers/room.py index 51a08fd2c0..65d4aea9af 100644 --- a/synapse/handlers/room.py +++ b/synapse/handlers/room.py @@ -883,7 +883,7 @@ class RoomCreationHandler: # # we also don't need to check the requester's shadow-ban here, as we # have already done so above (and potentially emptied invite_list). - with (await self.room_member_handler.member_linearizer.queue((room_id,))): + async with self.room_member_handler.member_linearizer.queue((room_id,)): content = {} is_direct = config.get("is_direct", None) if is_direct: diff --git a/synapse/handlers/room_member.py b/synapse/handlers/room_member.py index 0785e31114..802e57c4d0 100644 --- a/synapse/handlers/room_member.py +++ b/synapse/handlers/room_member.py @@ -515,8 +515,8 @@ class RoomMemberHandler(metaclass=abc.ABCMeta): # We first linearise by the application service (to try to limit concurrent joins # by application services), and then by room ID. - with (await self.member_as_limiter.queue(as_id)): - with (await self.member_linearizer.queue(key)): + async with self.member_as_limiter.queue(as_id): + async with self.member_linearizer.queue(key): result = await self.update_membership_locked( requester, target, diff --git a/synapse/handlers/sso.py b/synapse/handlers/sso.py index 4f02a060d9..e4fe94e557 100644 --- a/synapse/handlers/sso.py +++ b/synapse/handlers/sso.py @@ -430,7 +430,7 @@ class SsoHandler: # grab a lock while we try to find a mapping for this user. This seems... # optimistic, especially for implementations that end up redirecting to # interstitial pages. - with await self._mapping_lock.queue(auth_provider_id): + async with self._mapping_lock.queue(auth_provider_id): # first of all, check if we already have a mapping for this user user_id = await self.get_sso_user_by_remote_user_id( auth_provider_id, diff --git a/synapse/push/bulk_push_rule_evaluator.py b/synapse/push/bulk_push_rule_evaluator.py index a402a3e403..b07cf2eee7 100644 --- a/synapse/push/bulk_push_rule_evaluator.py +++ b/synapse/push/bulk_push_rule_evaluator.py @@ -397,7 +397,7 @@ class RulesForRoom: self.room_push_rule_cache_metrics.inc_hits() return self.data.rules_by_user - with (await self.linearizer.queue(self.room_id)): + async with self.linearizer.queue(self.room_id): if state_group and self.data.state_group == state_group: logger.debug("Using cached rules for %r", self.room_id) self.room_push_rule_cache_metrics.inc_hits() diff --git a/synapse/replication/tcp/client.py b/synapse/replication/tcp/client.py index deeaaec4e6..122892c7bc 100644 --- a/synapse/replication/tcp/client.py +++ b/synapse/replication/tcp/client.py @@ -451,7 +451,7 @@ class FederationSenderHandler: # service for robustness? Or could we replace it with an assertion that # we're not being re-entered? - with (await self._fed_position_linearizer.queue(None)): + async with self._fed_position_linearizer.queue(None): # We persist and ack the same position, so we take a copy of it # here as otherwise it can get modified from underneath us. current_position = self.federation_position diff --git a/synapse/rest/media/v1/media_repository.py b/synapse/rest/media/v1/media_repository.py index 6c414402bd..3e5d6c6294 100644 --- a/synapse/rest/media/v1/media_repository.py +++ b/synapse/rest/media/v1/media_repository.py @@ -258,7 +258,7 @@ class MediaRepository: # We linearize here to ensure that we don't try and download remote # media multiple times concurrently key = (server_name, media_id) - with (await self.remote_media_linearizer.queue(key)): + async with self.remote_media_linearizer.queue(key): responder, media_info = await self._get_remote_media_impl( server_name, media_id ) @@ -294,7 +294,7 @@ class MediaRepository: # We linearize here to ensure that we don't try and download remote # media multiple times concurrently key = (server_name, media_id) - with (await self.remote_media_linearizer.queue(key)): + async with self.remote_media_linearizer.queue(key): responder, media_info = await self._get_remote_media_impl( server_name, media_id ) @@ -850,7 +850,7 @@ class MediaRepository: # TODO: Should we delete from the backup store - with (await self.remote_media_linearizer.queue(key)): + async with self.remote_media_linearizer.queue(key): full_path = self.filepaths.remote_media_filepath(origin, file_id) try: os.remove(full_path) diff --git a/synapse/state/__init__.py b/synapse/state/__init__.py index 21888cc8c5..fbf7ba4600 100644 --- a/synapse/state/__init__.py +++ b/synapse/state/__init__.py @@ -573,7 +573,7 @@ class StateResolutionHandler: """ group_names = frozenset(state_groups_ids.keys()) - with (await self.resolve_linearizer.queue(group_names)): + async with self.resolve_linearizer.queue(group_names): cache = self._state_cache.get(group_names, None) if cache: return cache diff --git a/synapse/storage/databases/main/roommember.py b/synapse/storage/databases/main/roommember.py index 98d09b3736..48e83592e7 100644 --- a/synapse/storage/databases/main/roommember.py +++ b/synapse/storage/databases/main/roommember.py @@ -888,7 +888,7 @@ class RoomMemberWorkerStore(EventsWorkerStore): return frozenset(cache.hosts_to_joined_users) # Since we'll mutate the cache we need to lock. - with (await self._joined_host_linearizer.queue(room_id)): + async with self._joined_host_linearizer.queue(room_id): if state_entry.state_group == cache.state_group: # Same state group, so nothing to do. We've already checked for # this above, but the cache may have changed while waiting on diff --git a/synapse/util/async_helpers.py b/synapse/util/async_helpers.py index 6a8e844d63..4b2a16a6a9 100644 --- a/synapse/util/async_helpers.py +++ b/synapse/util/async_helpers.py @@ -18,7 +18,7 @@ import collections import inspect import itertools import logging -from contextlib import asynccontextmanager, contextmanager +from contextlib import asynccontextmanager from typing import ( Any, AsyncIterator, @@ -29,7 +29,6 @@ from typing import ( Generic, Hashable, Iterable, - Iterator, List, Optional, Set, @@ -342,7 +341,7 @@ class Linearizer: Example: - with await limiter.queue("test_key"): + async with limiter.queue("test_key"): # do some work. """ @@ -383,95 +382,53 @@ class Linearizer: # non-empty. return bool(entry.deferreds) - def queue(self, key: Hashable) -> defer.Deferred: - # we avoid doing defer.inlineCallbacks here, so that cancellation works correctly. - # (https://twistedmatrix.com/trac/ticket/4632 meant that cancellations were not - # propagated inside inlineCallbacks until Twisted 18.7) + def queue(self, key: Hashable) -> AsyncContextManager[None]: + @asynccontextmanager + async def _ctx_manager() -> AsyncIterator[None]: + entry = await self._acquire_lock(key) + try: + yield + finally: + self._release_lock(key, entry) + + return _ctx_manager() + + async def _acquire_lock(self, key: Hashable) -> _LinearizerEntry: + """Acquires a linearizer lock, waiting if necessary. + + Returns once we have secured the lock. + """ entry = self.key_to_defer.setdefault( key, _LinearizerEntry(0, collections.OrderedDict()) ) - # If the number of things executing is greater than the maximum - # then add a deferred to the list of blocked items - # When one of the things currently executing finishes it will callback - # this item so that it can continue executing. - if entry.count >= self.max_count: - res = self._await_lock(key) - else: + if entry.count < self.max_count: + # The number of things executing is less than the maximum. logger.debug( "Acquired uncontended linearizer lock %r for key %r", self.name, key ) entry.count += 1 - res = defer.succeed(None) - - # once we successfully get the lock, we need to return a context manager which - # will release the lock. - - @contextmanager - def _ctx_manager(_: None) -> Iterator[None]: - try: - yield - finally: - logger.debug("Releasing linearizer lock %r for key %r", self.name, key) - - # We've finished executing so check if there are any things - # blocked waiting to execute and start one of them - entry.count -= 1 - - if entry.deferreds: - (next_def, _) = entry.deferreds.popitem(last=False) - - # we need to run the next thing in the sentinel context. - with PreserveLoggingContext(): - next_def.callback(None) - elif entry.count == 0: - # We were the last thing for this key: remove it from the - # map. - del self.key_to_defer[key] - - res.addCallback(_ctx_manager) - return res - - def _await_lock(self, key: Hashable) -> defer.Deferred: - """Helper for queue: adds a deferred to the queue - - Assumes that we've already checked that we've reached the limit of the number - of lock-holders we allow. Creates a new deferred which is added to the list, and - adds some management around cancellations. - - Returns the deferred, which will callback once we have secured the lock. - - """ - entry = self.key_to_defer[key] + return entry + # Otherwise, the number of things executing is at the maximum and we have to + # add a deferred to the list of blocked items. + # When one of the things currently executing finishes it will callback + # this item so that it can continue executing. logger.debug("Waiting to acquire linearizer lock %r for key %r", self.name, key) new_defer: "defer.Deferred[None]" = make_deferred_yieldable(defer.Deferred()) entry.deferreds[new_defer] = 1 - def cb(_r: None) -> "defer.Deferred[None]": - logger.debug("Acquired linearizer lock %r for key %r", self.name, key) - entry.count += 1 - - # if the code holding the lock completes synchronously, then it - # will recursively run the next claimant on the list. That can - # relatively rapidly lead to stack exhaustion. This is essentially - # the same problem as http://twistedmatrix.com/trac/ticket/9304. - # - # In order to break the cycle, we add a cheeky sleep(0) here to - # ensure that we fall back to the reactor between each iteration. - # - # (This needs to happen while we hold the lock, and the context manager's exit - # code must be synchronous, so this is the only sensible place.) - return self._clock.sleep(0) - - def eb(e: Failure) -> Failure: + try: + await new_defer + except Exception as e: logger.info("defer %r got err %r", new_defer, e) if isinstance(e, CancelledError): logger.debug( - "Cancelling wait for linearizer lock %r for key %r", self.name, key + "Cancelling wait for linearizer lock %r for key %r", + self.name, + key, ) - else: logger.warning( "Unexpected exception waiting for linearizer lock %r for key %r", @@ -481,10 +438,43 @@ class Linearizer: # we just have to take ourselves back out of the queue. del entry.deferreds[new_defer] - return e + raise - new_defer.addCallbacks(cb, eb) - return new_defer + logger.debug("Acquired linearizer lock %r for key %r", self.name, key) + entry.count += 1 + + # if the code holding the lock completes synchronously, then it + # will recursively run the next claimant on the list. That can + # relatively rapidly lead to stack exhaustion. This is essentially + # the same problem as http://twistedmatrix.com/trac/ticket/9304. + # + # In order to break the cycle, we add a cheeky sleep(0) here to + # ensure that we fall back to the reactor between each iteration. + # + # This needs to happen while we hold the lock. We could put it on the + # exit path, but that would slow down the uncontended case. + await self._clock.sleep(0) + + return entry + + def _release_lock(self, key: Hashable, entry: _LinearizerEntry) -> None: + """Releases a held linearizer lock.""" + logger.debug("Releasing linearizer lock %r for key %r", self.name, key) + + # We've finished executing so check if there are any things + # blocked waiting to execute and start one of them + entry.count -= 1 + + if entry.deferreds: + (next_def, _) = entry.deferreds.popitem(last=False) + + # we need to run the next thing in the sentinel context. + with PreserveLoggingContext(): + next_def.callback(None) + elif entry.count == 0: + # We were the last thing for this key: remove it from the + # map. + del self.key_to_defer[key] class ReadWriteLock: diff --git a/tests/util/test_linearizer.py b/tests/util/test_linearizer.py index fa132391a1..c2a209e637 100644 --- a/tests/util/test_linearizer.py +++ b/tests/util/test_linearizer.py @@ -46,7 +46,7 @@ class LinearizerTestCase(unittest.TestCase): unblock_d: "Deferred[None]" = Deferred() async def task() -> None: - with await linearizer.queue(key): + async with linearizer.queue(key): acquired_d.callback(None) await unblock_d @@ -125,7 +125,7 @@ class LinearizerTestCase(unittest.TestCase): async def func(i: int) -> None: with LoggingContext("func(%s)" % i) as lc: - with (await linearizer.queue(key)): + async with linearizer.queue(key): self.assertEqual(current_context(), lc) self.assertEqual(current_context(), lc) From 9c4c49991de7e0f176864029580bebc7e6849fb1 Mon Sep 17 00:00:00 2001 From: Sean Quah <8349537+squahtx@users.noreply.github.com> Date: Tue, 5 Apr 2022 16:54:40 +0100 Subject: [PATCH 41/55] Update docstrings for `ReadWriteLock` tests (#12354) Signed-off-by: Sean Quah --- changelog.d/12354.misc | 1 + tests/util/test_rwlock.py | 12 ++++++------ 2 files changed, 7 insertions(+), 6 deletions(-) create mode 100644 changelog.d/12354.misc diff --git a/changelog.d/12354.misc b/changelog.d/12354.misc new file mode 100644 index 0000000000..e3b8950faa --- /dev/null +++ b/changelog.d/12354.misc @@ -0,0 +1 @@ +Update docstrings for `ReadWriteLock` tests. diff --git a/tests/util/test_rwlock.py b/tests/util/test_rwlock.py index 0c84226197..5da04362a9 100644 --- a/tests/util/test_rwlock.py +++ b/tests/util/test_rwlock.py @@ -40,8 +40,8 @@ class ReadWriteLockTestCase(unittest.TestCase): Returns: A tuple of three `Deferred`s: - * A `Deferred` that resolves with `return_value` once the reader or writer - completes successfully. + * A cancellable `Deferred` for the entire read or write operation that + resolves with `return_value` on successful completion. * A `Deferred` that resolves once the reader or writer acquires the lock. * A `Deferred` that blocks the reader or writer. Must be resolved by the caller to allow the reader or writer to release the lock and complete. @@ -87,8 +87,8 @@ class ReadWriteLockTestCase(unittest.TestCase): Returns: A tuple of two `Deferred`s: - * A `Deferred` that resolves with `return_value` once the reader completes - successfully. + * A cancellable `Deferred` for the entire read operation that resolves with + `return_value` on successful completion. * A `Deferred` that resolves once the reader acquires the lock. """ d, acquired_d, unblock_d = self._start_reader_or_writer( @@ -106,8 +106,8 @@ class ReadWriteLockTestCase(unittest.TestCase): Returns: A tuple of two `Deferred`s: - * A `Deferred` that resolves with `return_value` once the writer completes - successfully. + * A cancellable `Deferred` for the entire write operation that resolves + with `return_value` on successful completion. * A `Deferred` that resolves once the writer acquires the lock. """ d, acquired_d, unblock_d = self._start_reader_or_writer( From 31c1209c50c08929fe7c6f2f7531fb9def422c94 Mon Sep 17 00:00:00 2001 From: Sean Quah <8349537+squahtx@users.noreply.github.com> Date: Tue, 5 Apr 2022 16:56:52 +0100 Subject: [PATCH 42/55] Make `StreamToken` and `RoomStreamToken` methods propagate cancellations (#12366) `StreamToken.from_string` and `RoomStreamToken.parse` are both async methods that could be cancelled. These methods must not replace `CancelledError`s with `SynapseError`s. Signed-off-by: Sean Quah --- changelog.d/12366.misc | 1 + synapse/types.py | 5 +++++ 2 files changed, 6 insertions(+) create mode 100644 changelog.d/12366.misc diff --git a/changelog.d/12366.misc b/changelog.d/12366.misc new file mode 100644 index 0000000000..33d8e6c712 --- /dev/null +++ b/changelog.d/12366.misc @@ -0,0 +1 @@ +Make `StreamToken.from_string` and `RoomStreamToken.parse` propagate cancellations instead of replacing them with `SynapseError`s. diff --git a/synapse/types.py b/synapse/types.py index 6bbefb6faa..9ac688b23b 100644 --- a/synapse/types.py +++ b/synapse/types.py @@ -39,6 +39,7 @@ from typing_extensions import TypedDict from unpaddedbase64 import decode_base64 from zope.interface import Interface +from twisted.internet.defer import CancelledError from twisted.internet.interfaces import ( IReactorCore, IReactorPluggableNameResolver, @@ -540,6 +541,8 @@ class RoomStreamToken: stream=stream, instance_map=frozendict(instance_map), ) + except CancelledError: + raise except Exception: pass raise SynapseError(400, "Invalid room stream token %r" % (string,)) @@ -705,6 +708,8 @@ class StreamToken: return cls( await RoomStreamToken.parse(store, keys[0]), *(int(k) for k in keys[1:]) ) + except CancelledError: + raise except Exception: raise SynapseError(400, "Invalid stream token") From 79e7c2c42648b189b800b80fcdb61949ab9be51b Mon Sep 17 00:00:00 2001 From: Sean Quah <8349537+squahtx@users.noreply.github.com> Date: Tue, 5 Apr 2022 17:19:16 +0100 Subject: [PATCH 43/55] Fix edge case where a `Linearizer` could get stuck (#12358) Just after a task acquires a contended `Linearizer` lock, it sleeps. If the task is cancelled during this sleep, we need to release the lock. Signed-off-by: Sean Quah --- changelog.d/12358.misc | 1 + synapse/util/async_helpers.py | 6 ++++- tests/util/test_linearizer.py | 51 ++++++++++++++++++++++++++++++++--- 3 files changed, 53 insertions(+), 5 deletions(-) create mode 100644 changelog.d/12358.misc diff --git a/changelog.d/12358.misc b/changelog.d/12358.misc new file mode 100644 index 0000000000..fcacbcba5c --- /dev/null +++ b/changelog.d/12358.misc @@ -0,0 +1 @@ +Fix a long-standing bug where `Linearizer`s could get stuck if a cancellation were to happen at the wrong time. diff --git a/synapse/util/async_helpers.py b/synapse/util/async_helpers.py index 4b2a16a6a9..650e44de22 100644 --- a/synapse/util/async_helpers.py +++ b/synapse/util/async_helpers.py @@ -453,7 +453,11 @@ class Linearizer: # # This needs to happen while we hold the lock. We could put it on the # exit path, but that would slow down the uncontended case. - await self._clock.sleep(0) + try: + await self._clock.sleep(0) + except CancelledError: + self._release_lock(key, entry) + raise return entry diff --git a/tests/util/test_linearizer.py b/tests/util/test_linearizer.py index c2a209e637..47a1cfbdc1 100644 --- a/tests/util/test_linearizer.py +++ b/tests/util/test_linearizer.py @@ -13,7 +13,9 @@ # See the License for the specific language governing permissions and # limitations under the License. -from typing import Callable, Hashable, Tuple +from typing import Hashable, Tuple + +from typing_extensions import Protocol from twisted.internet import defer, reactor from twisted.internet.base import ReactorBase @@ -25,10 +27,15 @@ from synapse.util.async_helpers import Linearizer from tests import unittest +class UnblockFunction(Protocol): + def __call__(self, pump_reactor: bool = True) -> None: + ... + + class LinearizerTestCase(unittest.TestCase): def _start_task( self, linearizer: Linearizer, key: Hashable - ) -> Tuple["Deferred[None]", "Deferred[None]", Callable[[], None]]: + ) -> Tuple["Deferred[None]", "Deferred[None]", UnblockFunction]: """Starts a task which acquires the linearizer lock, blocks, then completes. Args: @@ -52,11 +59,12 @@ class LinearizerTestCase(unittest.TestCase): d = defer.ensureDeferred(task()) - def unblock() -> None: + def unblock(pump_reactor: bool = True) -> None: unblock_d.callback(None) # The next task, if it exists, will acquire the lock and require a kick of # the reactor to advance. - self._pump() + if pump_reactor: + self._pump() return d, acquired_d, unblock @@ -212,3 +220,38 @@ class LinearizerTestCase(unittest.TestCase): ) unblock3() self.successResultOf(d3) + + def test_cancellation_during_sleep(self) -> None: + """Tests cancellation during the sleep just after waiting for a `Linearizer`.""" + linearizer = Linearizer() + + key = object() + + d1, acquired_d1, unblock1 = self._start_task(linearizer, key) + self.assertTrue(acquired_d1.called) + + # Create a second task, waiting for the first task. + d2, acquired_d2, _ = self._start_task(linearizer, key) + self.assertFalse(acquired_d2.called) + + # Create a third task, waiting for the second task. + d3, acquired_d3, unblock3 = self._start_task(linearizer, key) + self.assertFalse(acquired_d3.called) + + # Once the first task completes, cancel the waiting second task while it is + # sleeping just after acquiring the lock. + unblock1(pump_reactor=False) + self.successResultOf(d1) + d2.cancel() + self._pump() + + self.assertTrue(d2.called) + self.failureResultOf(d2, CancelledError) + + # The third task should continue running. + self.assertTrue( + acquired_d3.called, + "Third task did not get the lock after the second task was cancelled", + ) + unblock3() + self.successResultOf(d3) From 163fd686b5db10eab2c70855dedef23204c6f18c Mon Sep 17 00:00:00 2001 From: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Date: Tue, 5 Apr 2022 17:21:58 +0100 Subject: [PATCH 44/55] Update mdbook to 0.4.17 (#12339) Update mdbook (the tool used to render the documentation website) to version 0.4.17. --- .github/workflows/docs.yaml | 2 +- changelog.d/12339.doc | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) create mode 100644 changelog.d/12339.doc diff --git a/.github/workflows/docs.yaml b/.github/workflows/docs.yaml index 2bf32e376b..b366eb8667 100644 --- a/.github/workflows/docs.yaml +++ b/.github/workflows/docs.yaml @@ -22,7 +22,7 @@ jobs: - name: Setup mdbook uses: peaceiris/actions-mdbook@4b5ef36b314c2599664ca107bb8c02412548d79d # v1.1.14 with: - mdbook-version: '0.4.9' + mdbook-version: '0.4.17' - name: Build the documentation # mdbook will only create an index.html if we're including docs/README.md in SUMMARY.md. diff --git a/changelog.d/12339.doc b/changelog.d/12339.doc new file mode 100644 index 0000000000..cdbd55bb34 --- /dev/null +++ b/changelog.d/12339.doc @@ -0,0 +1 @@ +Upgrade the version of `mdbook` in CI to 0.4.17. From fd1e7d0fc2c65e2cebc7c7b8d1d4d49b05d99168 Mon Sep 17 00:00:00 2001 From: Sean Quah <8349537+squahtx@users.noreply.github.com> Date: Tue, 5 Apr 2022 17:35:20 +0100 Subject: [PATCH 45/55] Post 1.56.0 release tidy up (#12379) --- changelog.d/12320.misc | 1 - changelog.d/12379.misc | 1 + synapse/python_dependencies.py | 2 +- 3 files changed, 2 insertions(+), 2 deletions(-) delete mode 100644 changelog.d/12320.misc create mode 100644 changelog.d/12379.misc diff --git a/changelog.d/12320.misc b/changelog.d/12320.misc deleted file mode 100644 index 7b4748d230..0000000000 --- a/changelog.d/12320.misc +++ /dev/null @@ -1 +0,0 @@ -Bump the version of `black` for compatibility with the latest `click` release. diff --git a/changelog.d/12379.misc b/changelog.d/12379.misc new file mode 100644 index 0000000000..b55778577d --- /dev/null +++ b/changelog.d/12379.misc @@ -0,0 +1 @@ +Remove temporary pin of signedjson<=1.1.1 that was added in Synapse 1.56.0. diff --git a/synapse/python_dependencies.py b/synapse/python_dependencies.py index d02cca0bbb..8419ab3aca 100644 --- a/synapse/python_dependencies.py +++ b/synapse/python_dependencies.py @@ -48,7 +48,7 @@ REQUIREMENTS = [ "unpaddedbase64>=1.1.0", "canonicaljson>=1.4.0", # we use the type definitions added in signedjson 1.1. - "signedjson>=1.1.0,<=1.1.1", + "signedjson>=1.1.0", "pynacl>=1.2.1", "idna>=2.5", # validating SSL certs for IP addresses requires service_identity 18.1. From 7ec9b06303e0f78ef4a82d804798a3826ddffca2 Mon Sep 17 00:00:00 2001 From: David Robertson Date: Tue, 5 Apr 2022 18:40:56 +0100 Subject: [PATCH 46/55] Poetry: initial pyproject.toml and poetry.lock (#12381) Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Co-authored-by: Brendan Abolivier --- .gitignore | 4 + changelog.d/12381.misc | 1 + poetry.lock | 2831 ++++++++++++++++++++++++++++++++++++++++ pyproject.toml | 226 ++++ 4 files changed, 3062 insertions(+) create mode 100644 changelog.d/12381.misc create mode 100644 poetry.lock diff --git a/.gitignore b/.gitignore index 61bbd2dfa0..c011cd27a4 100644 --- a/.gitignore +++ b/.gitignore @@ -15,6 +15,10 @@ _trial_temp*/ .DS_Store __pycache__/ +# We do want the poetry lockfile. TODO: is there a good reason for ignoring +# '*.lock' above? If not, let's nuke it. +!poetry.lock + # stuff that is likely to exist when you run a server locally /*.db /*.log diff --git a/changelog.d/12381.misc b/changelog.d/12381.misc new file mode 100644 index 0000000000..a1d8d60ebb --- /dev/null +++ b/changelog.d/12381.misc @@ -0,0 +1 @@ +Lay groundwork for using `poetry` to manage Synapse's dependencies. \ No newline at end of file diff --git a/poetry.lock b/poetry.lock new file mode 100644 index 0000000000..d4bf972c43 --- /dev/null +++ b/poetry.lock @@ -0,0 +1,2831 @@ +[[package]] +name = "appdirs" +version = "1.4.4" +description = "A small Python module for determining appropriate platform-specific dirs, e.g. a \"user data dir\"." +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "attrs" +version = "21.4.0" +description = "Classes Without Boilerplate" +category = "main" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" + +[package.extras] +dev = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "mypy", "pytest-mypy-plugins", "zope.interface", "furo", "sphinx", "sphinx-notfound-page", "pre-commit", "cloudpickle"] +docs = ["furo", "sphinx", "zope.interface", "sphinx-notfound-page"] +tests = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "mypy", "pytest-mypy-plugins", "zope.interface", "cloudpickle"] +tests_no_zope = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "mypy", "pytest-mypy-plugins", "cloudpickle"] + +[[package]] +name = "authlib" +version = "0.15.5" +description = "The ultimate Python library in building OAuth and OpenID Connect servers." +category = "main" +optional = true +python-versions = "*" + +[package.dependencies] +cryptography = "*" + +[package.extras] +client = ["requests"] + +[[package]] +name = "automat" +version = "20.2.0" +description = "Self-service finite-state machines for the programmer on the go." +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +attrs = ">=19.2.0" +six = "*" + +[package.extras] +visualize = ["graphviz (>0.5.1)", "Twisted (>=16.1.1)"] + +[[package]] +name = "baron" +version = "0.10.1" +description = "Full Syntax Tree for python to make writing refactoring code a realist task" +category = "dev" +optional = false +python-versions = "*" + +[package.dependencies] +rply = "*" + +[[package]] +name = "bcrypt" +version = "3.2.0" +description = "Modern password hashing for your software and your servers" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +cffi = ">=1.1" +six = ">=1.4.1" + +[package.extras] +tests = ["pytest (>=3.2.1,!=3.3.0)"] +typecheck = ["mypy"] + +[[package]] +name = "black" +version = "22.3.0" +description = "The uncompromising code formatter." +category = "dev" +optional = false +python-versions = ">=3.6.2" + +[package.dependencies] +click = ">=8.0.0" +mypy-extensions = ">=0.4.3" +pathspec = ">=0.9.0" +platformdirs = ">=2" +tomli = {version = ">=1.1.0", markers = "python_version < \"3.11\""} +typed-ast = {version = ">=1.4.2", markers = "python_version < \"3.8\" and implementation_name == \"cpython\""} +typing-extensions = {version = ">=3.10.0.0", markers = "python_version < \"3.10\""} + +[package.extras] +colorama = ["colorama (>=0.4.3)"] +d = ["aiohttp (>=3.7.4)"] +jupyter = ["ipython (>=7.8.0)", "tokenize-rt (>=3.2.0)"] +uvloop = ["uvloop (>=0.15.2)"] + +[[package]] +name = "bleach" +version = "4.1.0" +description = "An easy safelist-based HTML-sanitizing tool." +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +packaging = "*" +six = ">=1.9.0" +webencodings = "*" + +[[package]] +name = "canonicaljson" +version = "1.6.0" +description = "Canonical JSON" +category = "main" +optional = false +python-versions = "~=3.7" + +[package.dependencies] +simplejson = ">=3.14.0" + +[package.extras] +frozendict = ["frozendict (>=1.0)"] + +[[package]] +name = "certifi" +version = "2021.10.8" +description = "Python package for providing Mozilla's CA Bundle." +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "cffi" +version = "1.15.0" +description = "Foreign Function Interface for Python calling C code." +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +pycparser = "*" + +[[package]] +name = "charset-normalizer" +version = "2.0.12" +description = "The Real First Universal Charset Detector. Open, modern and actively maintained alternative to Chardet." +category = "main" +optional = false +python-versions = ">=3.5.0" + +[package.extras] +unicode_backport = ["unicodedata2"] + +[[package]] +name = "click" +version = "8.1.0" +description = "Composable command line interface toolkit" +category = "dev" +optional = false +python-versions = ">=3.7" + +[package.dependencies] +colorama = {version = "*", markers = "platform_system == \"Windows\""} +importlib-metadata = {version = "*", markers = "python_version < \"3.8\""} + +[[package]] +name = "click-default-group" +version = "1.2.2" +description = "Extends click.Group to invoke a command without explicit subcommand name" +category = "dev" +optional = false +python-versions = "*" + +[package.dependencies] +click = "*" + +[[package]] +name = "colorama" +version = "0.4.4" +description = "Cross-platform colored terminal text." +category = "dev" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" + +[[package]] +name = "commonmark" +version = "0.9.1" +description = "Python parser for the CommonMark Markdown spec" +category = "dev" +optional = false +python-versions = "*" + +[package.extras] +test = ["flake8 (==3.7.8)", "hypothesis (==3.55.3)"] + +[[package]] +name = "constantly" +version = "15.1.0" +description = "Symbolic constants in Python" +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "cryptography" +version = "36.0.1" +description = "cryptography is a package which provides cryptographic recipes and primitives to Python developers." +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +cffi = ">=1.12" + +[package.extras] +docs = ["sphinx (>=1.6.5,!=1.8.0,!=3.1.0,!=3.1.1)", "sphinx-rtd-theme"] +docstest = ["pyenchant (>=1.6.11)", "twine (>=1.12.0)", "sphinxcontrib-spelling (>=4.0.1)"] +pep8test = ["black", "flake8", "flake8-import-order", "pep8-naming"] +sdist = ["setuptools_rust (>=0.11.4)"] +ssh = ["bcrypt (>=3.1.5)"] +test = ["pytest (>=6.2.0)", "pytest-cov", "pytest-subtests", "pytest-xdist", "pretend", "iso8601", "pytz", "hypothesis (>=1.11.4,!=3.79.2)"] + +[[package]] +name = "defusedxml" +version = "0.7.1" +description = "XML bomb protection for Python stdlib modules" +category = "main" +optional = true +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" + +[[package]] +name = "deprecated" +version = "1.2.13" +description = "Python @deprecated decorator to deprecate old python classes, functions or methods." +category = "dev" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" + +[package.dependencies] +wrapt = ">=1.10,<2" + +[package.extras] +dev = ["tox", "bump2version (<1)", "sphinx (<2)", "importlib-metadata (<3)", "importlib-resources (<4)", "configparser (<5)", "sphinxcontrib-websupport (<2)", "zipp (<2)", "PyTest (<5)", "PyTest-Cov (<2.6)", "pytest", "pytest-cov"] + +[[package]] +name = "docutils" +version = "0.18.1" +description = "Docutils -- Python Documentation Utilities" +category = "dev" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" + +[[package]] +name = "elementpath" +version = "2.5.0" +description = "XPath 1.0/2.0 parsers and selectors for ElementTree and lxml" +category = "main" +optional = true +python-versions = ">=3.7" + +[package.extras] +dev = ["tox", "coverage", "lxml", "xmlschema (>=1.8.0)", "sphinx", "memory-profiler", "flake8", "mypy (==0.910)"] + +[[package]] +name = "flake8" +version = "4.0.1" +description = "the modular source code checker: pep8 pyflakes and co" +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +importlib-metadata = {version = "<4.3", markers = "python_version < \"3.8\""} +mccabe = ">=0.6.0,<0.7.0" +pycodestyle = ">=2.8.0,<2.9.0" +pyflakes = ">=2.4.0,<2.5.0" + +[[package]] +name = "flake8-bugbear" +version = "21.3.2" +description = "A plugin for flake8 finding likely bugs and design problems in your program. Contains warnings that don't belong in pyflakes and pycodestyle." +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +attrs = ">=19.2.0" +flake8 = ">=3.0.0" + +[package.extras] +dev = ["coverage", "black", "hypothesis", "hypothesmith"] + +[[package]] +name = "flake8-comprehensions" +version = "3.8.0" +description = "A flake8 plugin to help you write better list/set/dict comprehensions." +category = "dev" +optional = false +python-versions = ">=3.7" + +[package.dependencies] +flake8 = ">=3.0,<3.2.0 || >3.2.0" +importlib-metadata = {version = "*", markers = "python_version < \"3.8\""} + +[[package]] +name = "frozendict" +version = "2.3.0" +description = "A simple immutable dictionary" +category = "main" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "gitdb" +version = "4.0.9" +description = "Git Object Database" +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +smmap = ">=3.0.1,<6" + +[[package]] +name = "gitpython" +version = "3.1.14" +description = "Python Git Library" +category = "dev" +optional = false +python-versions = ">=3.4" + +[package.dependencies] +gitdb = ">=4.0.1,<5" + +[[package]] +name = "hiredis" +version = "2.0.0" +description = "Python wrapper for hiredis" +category = "main" +optional = true +python-versions = ">=3.6" + +[[package]] +name = "hyperlink" +version = "21.0.0" +description = "A featureful, immutable, and correct URL for Python." +category = "main" +optional = false +python-versions = ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" + +[package.dependencies] +idna = ">=2.5" + +[[package]] +name = "idna" +version = "3.3" +description = "Internationalized Domain Names in Applications (IDNA)" +category = "main" +optional = false +python-versions = ">=3.5" + +[[package]] +name = "ijson" +version = "3.1.4" +description = "Iterative JSON parser with standard Python iterator interfaces" +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "importlib-metadata" +version = "4.2.0" +description = "Read metadata from Python packages" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +typing-extensions = {version = ">=3.6.4", markers = "python_version < \"3.8\""} +zipp = ">=0.5" + +[package.extras] +docs = ["sphinx", "jaraco.packaging (>=8.2)", "rst.linker (>=1.9)"] +testing = ["pytest (>=4.6)", "pytest-checkdocs (>=2.4)", "pytest-flake8", "pytest-cov", "pytest-enabler (>=1.0.1)", "packaging", "pep517", "pyfakefs", "flufl.flake8", "pytest-black (>=0.3.7)", "pytest-mypy", "importlib-resources (>=1.3)"] + +[[package]] +name = "importlib-resources" +version = "5.4.0" +description = "Read resources from Python packages" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +zipp = {version = ">=3.1.0", markers = "python_version < \"3.10\""} + +[package.extras] +docs = ["sphinx", "jaraco.packaging (>=8.2)", "rst.linker (>=1.9)"] +testing = ["pytest (>=6)", "pytest-checkdocs (>=2.4)", "pytest-flake8", "pytest-cov", "pytest-enabler (>=1.0.1)", "pytest-black (>=0.3.7)", "pytest-mypy"] + +[[package]] +name = "incremental" +version = "21.3.0" +description = "A small library that versions your Python projects." +category = "main" +optional = false +python-versions = "*" + +[package.extras] +scripts = ["click (>=6.0)", "twisted (>=16.4.0)"] + +[[package]] +name = "isort" +version = "5.7.0" +description = "A Python utility / library to sort Python imports." +category = "dev" +optional = false +python-versions = ">=3.6,<4.0" + +[package.extras] +pipfile_deprecated_finder = ["pipreqs", "requirementslib"] +requirements_deprecated_finder = ["pipreqs", "pip-api"] +colors = ["colorama (>=0.4.3,<0.5.0)"] + +[[package]] +name = "jaeger-client" +version = "4.8.0" +description = "Jaeger Python OpenTracing Tracer implementation" +category = "main" +optional = true +python-versions = ">=3.7" + +[package.dependencies] +opentracing = ">=2.1,<3.0" +threadloop = ">=1,<2" +thrift = "*" +tornado = ">=4.3" + +[package.extras] +tests = ["mock", "pycurl", "pytest", "pytest-cov", "coverage", "pytest-timeout", "pytest-tornado", "pytest-benchmark", "pytest-localserver", "flake8", "flake8-quotes", "flake8-typing-imports", "codecov", "tchannel (==2.1.0)", "opentracing_instrumentation (>=3,<4)", "prometheus_client (==0.11.0)", "mypy"] + +[[package]] +name = "jeepney" +version = "0.7.1" +description = "Low-level, pure Python DBus protocol wrapper." +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.extras] +test = ["pytest", "pytest-trio", "pytest-asyncio", "testpath", "trio", "async-timeout"] +trio = ["trio", "async-generator"] + +[[package]] +name = "jinja2" +version = "3.0.3" +description = "A very fast and expressive template engine." +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +MarkupSafe = ">=2.0" + +[package.extras] +i18n = ["Babel (>=2.7)"] + +[[package]] +name = "jsonschema" +version = "4.4.0" +description = "An implementation of JSON Schema validation for Python" +category = "main" +optional = false +python-versions = ">=3.7" + +[package.dependencies] +attrs = ">=17.4.0" +importlib-metadata = {version = "*", markers = "python_version < \"3.8\""} +importlib-resources = {version = ">=1.4.0", markers = "python_version < \"3.9\""} +pyrsistent = ">=0.14.0,<0.17.0 || >0.17.0,<0.17.1 || >0.17.1,<0.17.2 || >0.17.2" +typing-extensions = {version = "*", markers = "python_version < \"3.8\""} + +[package.extras] +format = ["fqdn", "idna", "isoduration", "jsonpointer (>1.13)", "rfc3339-validator", "rfc3987", "uri-template", "webcolors (>=1.11)"] +format_nongpl = ["fqdn", "idna", "isoduration", "jsonpointer (>1.13)", "rfc3339-validator", "rfc3986-validator (>0.1.0)", "uri-template", "webcolors (>=1.11)"] + +[[package]] +name = "keyring" +version = "23.5.0" +description = "Store and access your passwords safely." +category = "dev" +optional = false +python-versions = ">=3.7" + +[package.dependencies] +importlib-metadata = ">=3.6" +jeepney = {version = ">=0.4.2", markers = "sys_platform == \"linux\""} +pywin32-ctypes = {version = "<0.1.0 || >0.1.0,<0.1.1 || >0.1.1", markers = "sys_platform == \"win32\""} +SecretStorage = {version = ">=3.2", markers = "sys_platform == \"linux\""} + +[package.extras] +docs = ["sphinx", "jaraco.packaging (>=8.2)", "rst.linker (>=1.9)", "jaraco.tidelift (>=1.4)"] +testing = ["pytest (>=6)", "pytest-checkdocs (>=2.4)", "pytest-flake8", "pytest-cov", "pytest-enabler (>=1.0.1)", "pytest-black (>=0.3.7)", "pytest-mypy"] + +[[package]] +name = "ldap3" +version = "2.9.1" +description = "A strictly RFC 4510 conforming LDAP V3 pure Python client library" +category = "main" +optional = true +python-versions = "*" + +[package.dependencies] +pyasn1 = ">=0.4.6" + +[[package]] +name = "lxml" +version = "4.8.0" +description = "Powerful and Pythonic XML processing library combining libxml2/libxslt with the ElementTree API." +category = "main" +optional = true +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, != 3.4.*" + +[package.extras] +cssselect = ["cssselect (>=0.7)"] +html5 = ["html5lib"] +htmlsoup = ["beautifulsoup4"] +source = ["Cython (>=0.29.7)"] + +[[package]] +name = "markupsafe" +version = "2.1.0" +description = "Safely add untrusted strings to HTML/XML markup." +category = "main" +optional = false +python-versions = ">=3.7" + +[[package]] +name = "matrix-common" +version = "1.1.0" +description = "Common utilities for Synapse, Sydent and Sygnal" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +attrs = "*" +importlib-metadata = {version = ">=1.4", markers = "python_version < \"3.8\""} + +[package.extras] +dev = ["tox", "twisted", "aiounittest", "mypy (==0.910)", "black (==21.9b0)", "flake8 (==4.0.1)", "isort (==5.9.3)"] +test = ["tox", "twisted", "aiounittest"] + +[[package]] +name = "matrix-synapse-ldap3" +version = "0.1.5" +description = "An LDAP3 auth provider for Synapse" +category = "main" +optional = true +python-versions = "*" + +[package.dependencies] +ldap3 = ">=2.8" +service_identity = "*" +Twisted = ">=15.1.0" + +[[package]] +name = "mccabe" +version = "0.6.1" +description = "McCabe checker, plugin for flake8" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "msgpack" +version = "1.0.3" +description = "MessagePack (de)serializer." +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "mypy" +version = "0.931" +description = "Optional static typing for Python" +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +mypy-extensions = ">=0.4.3" +tomli = ">=1.1.0" +typed-ast = {version = ">=1.4.0,<2", markers = "python_version < \"3.8\""} +typing-extensions = ">=3.10" + +[package.extras] +dmypy = ["psutil (>=4.0)"] +python2 = ["typed-ast (>=1.4.0,<2)"] + +[[package]] +name = "mypy-extensions" +version = "0.4.3" +description = "Experimental type system extensions for programs checked with the mypy typechecker." +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "mypy-zope" +version = "0.3.5" +description = "Plugin for mypy to support zope interfaces" +category = "dev" +optional = false +python-versions = "*" + +[package.dependencies] +mypy = "0.931" +"zope.interface" = "*" +"zope.schema" = "*" + +[package.extras] +test = ["pytest (>=4.6)", "pytest-cov", "lxml"] + +[[package]] +name = "netaddr" +version = "0.8.0" +description = "A network address manipulation library for Python" +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "opentracing" +version = "2.4.0" +description = "OpenTracing API for Python. See documentation at http://opentracing.io" +category = "main" +optional = true +python-versions = "*" + +[package.extras] +tests = ["doubles", "flake8", "flake8-quotes", "mock", "pytest", "pytest-cov", "pytest-mock", "sphinx", "sphinx-rtd-theme", "six (>=1.10.0,<2.0)", "gevent", "tornado"] + +[[package]] +name = "packaging" +version = "21.3" +description = "Core utilities for Python packages" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +pyparsing = ">=2.0.2,<3.0.5 || >3.0.5" + +[[package]] +name = "parameterized" +version = "0.8.1" +description = "Parameterized testing with any Python test framework" +category = "main" +optional = false +python-versions = "*" + +[package.extras] +dev = ["jinja2"] + +[[package]] +name = "pathspec" +version = "0.9.0" +description = "Utility library for gitignore style pattern matching of file paths." +category = "dev" +optional = false +python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7" + +[[package]] +name = "phonenumbers" +version = "8.12.44" +description = "Python version of Google's common library for parsing, formatting, storing and validating international phone numbers." +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "pillow" +version = "9.0.1" +description = "Python Imaging Library (Fork)" +category = "main" +optional = false +python-versions = ">=3.7" + +[[package]] +name = "pkginfo" +version = "1.8.2" +description = "Query metadatdata from sdists / bdists / installed packages." +category = "dev" +optional = false +python-versions = "*" + +[package.extras] +testing = ["coverage", "nose"] + +[[package]] +name = "platformdirs" +version = "2.5.1" +description = "A small Python module for determining appropriate platform-specific dirs, e.g. a \"user data dir\"." +category = "dev" +optional = false +python-versions = ">=3.7" + +[package.extras] +docs = ["Sphinx (>=4)", "furo (>=2021.7.5b38)", "proselint (>=0.10.2)", "sphinx-autodoc-typehints (>=1.12)"] +test = ["appdirs (==1.4.4)", "pytest (>=6)", "pytest-cov (>=2.7)", "pytest-mock (>=3.6)"] + +[[package]] +name = "prometheus-client" +version = "0.13.1" +description = "Python client for the Prometheus monitoring system." +category = "main" +optional = false +python-versions = ">=3.6" + +[package.extras] +twisted = ["twisted"] + +[[package]] +name = "psycopg2" +version = "2.9.3" +description = "psycopg2 - Python-PostgreSQL Database Adapter" +category = "main" +optional = true +python-versions = ">=3.6" + +[[package]] +name = "psycopg2cffi" +version = "2.9.0" +description = ".. image:: https://travis-ci.org/chtd/psycopg2cffi.svg?branch=master" +category = "main" +optional = true +python-versions = "*" + +[package.dependencies] +cffi = ">=1.0" +six = "*" + +[[package]] +name = "psycopg2cffi-compat" +version = "1.1" +description = "A Simple library to enable psycopg2 compatability" +category = "main" +optional = true +python-versions = "*" + +[package.dependencies] +psycopg2 = "*" + +[[package]] +name = "pyasn1" +version = "0.4.8" +description = "ASN.1 types and codecs" +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "pyasn1-modules" +version = "0.2.8" +description = "A collection of ASN.1-based protocols modules." +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +pyasn1 = ">=0.4.6,<0.5.0" + +[[package]] +name = "pycodestyle" +version = "2.8.0" +description = "Python style guide checker" +category = "dev" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" + +[[package]] +name = "pycparser" +version = "2.21" +description = "C parser in Python" +category = "main" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" + +[[package]] +name = "pyflakes" +version = "2.4.0" +description = "passive checker of Python programs" +category = "dev" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" + +[[package]] +name = "pygithub" +version = "1.55" +description = "Use the full Github API v3" +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +deprecated = "*" +pyjwt = ">=2.0" +pynacl = ">=1.4.0" +requests = ">=2.14.0" + +[package.extras] +integrations = ["cryptography"] + +[[package]] +name = "pygments" +version = "2.11.2" +description = "Pygments is a syntax highlighting package written in Python." +category = "dev" +optional = false +python-versions = ">=3.5" + +[[package]] +name = "pyjwt" +version = "2.3.0" +description = "JSON Web Token implementation in Python" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.extras] +crypto = ["cryptography (>=3.3.1)"] +dev = ["sphinx", "sphinx-rtd-theme", "zope.interface", "cryptography (>=3.3.1)", "pytest (>=6.0.0,<7.0.0)", "coverage[toml] (==5.0.4)", "mypy", "pre-commit"] +docs = ["sphinx", "sphinx-rtd-theme", "zope.interface"] +tests = ["pytest (>=6.0.0,<7.0.0)", "coverage[toml] (==5.0.4)"] + +[[package]] +name = "pymacaroons" +version = "0.13.0" +description = "Macaroon library for Python" +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +PyNaCl = ">=1.1.2,<2.0" +six = ">=1.8.0" + +[[package]] +name = "pympler" +version = "1.0.1" +description = "A development tool to measure, monitor and analyze the memory behavior of Python objects." +category = "main" +optional = true +python-versions = ">=3.6" + +[[package]] +name = "pynacl" +version = "1.5.0" +description = "Python binding to the Networking and Cryptography (NaCl) library" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +cffi = ">=1.4.1" + +[package.extras] +docs = ["sphinx (>=1.6.5)", "sphinx-rtd-theme"] +tests = ["pytest (>=3.2.1,!=3.3.0)", "hypothesis (>=3.27.0)"] + +[[package]] +name = "pyopenssl" +version = "22.0.0" +description = "Python wrapper module around the OpenSSL library" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +cryptography = ">=35.0" + +[package.extras] +docs = ["sphinx", "sphinx-rtd-theme"] +test = ["flaky", "pretend", "pytest (>=3.0.1)"] + +[[package]] +name = "pyparsing" +version = "3.0.7" +description = "Python parsing module" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.extras] +diagrams = ["jinja2", "railroad-diagrams"] + +[[package]] +name = "pyrsistent" +version = "0.18.1" +description = "Persistent/Functional/Immutable data structures" +category = "main" +optional = false +python-versions = ">=3.7" + +[[package]] +name = "pysaml2" +version = "7.1.2" +description = "Python implementation of SAML Version 2 Standard" +category = "main" +optional = true +python-versions = "<4,>=3.6" + +[package.dependencies] +cryptography = ">=1.4" +defusedxml = "*" +importlib-resources = {version = "*", markers = "python_version < \"3.9\""} +pyOpenSSL = "*" +python-dateutil = "*" +pytz = "*" +requests = ">=1.0.0" +six = "*" +xmlschema = ">=1.2.1" + +[package.extras] +s2repoze = ["paste", "zope.interface", "repoze.who"] + +[[package]] +name = "python-dateutil" +version = "2.8.2" +description = "Extensions to the standard Python datetime module" +category = "main" +optional = true +python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,>=2.7" + +[package.dependencies] +six = ">=1.5" + +[[package]] +name = "pytz" +version = "2021.3" +description = "World timezone definitions, modern and historical" +category = "main" +optional = true +python-versions = "*" + +[[package]] +name = "pywin32-ctypes" +version = "0.2.0" +description = "" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "pyyaml" +version = "6.0" +description = "YAML parser and emitter for Python" +category = "main" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "readme-renderer" +version = "33.0" +description = "readme_renderer is a library for rendering \"readme\" descriptions for Warehouse" +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +bleach = ">=2.1.0" +docutils = ">=0.13.1" +Pygments = ">=2.5.1" + +[package.extras] +md = ["cmarkgfm (>=0.8.0)"] + +[[package]] +name = "redbaron" +version = "0.9.2" +description = "Abstraction on top of baron, a FST for python to make writing refactoring code a realistic task" +category = "dev" +optional = false +python-versions = "*" + +[package.dependencies] +baron = ">=0.7" + +[package.extras] +notebook = ["pygments"] + +[[package]] +name = "requests" +version = "2.27.1" +description = "Python HTTP for Humans." +category = "main" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*" + +[package.dependencies] +certifi = ">=2017.4.17" +charset-normalizer = {version = ">=2.0.0,<2.1.0", markers = "python_version >= \"3\""} +idna = {version = ">=2.5,<4", markers = "python_version >= \"3\""} +urllib3 = ">=1.21.1,<1.27" + +[package.extras] +socks = ["PySocks (>=1.5.6,!=1.5.7)", "win-inet-pton"] +use_chardet_on_py3 = ["chardet (>=3.0.2,<5)"] + +[[package]] +name = "requests-toolbelt" +version = "0.9.1" +description = "A utility belt for advanced users of python-requests" +category = "dev" +optional = false +python-versions = "*" + +[package.dependencies] +requests = ">=2.0.1,<3.0.0" + +[[package]] +name = "rfc3986" +version = "2.0.0" +description = "Validating URI References per RFC 3986" +category = "dev" +optional = false +python-versions = ">=3.7" + +[package.extras] +idna2008 = ["idna"] + +[[package]] +name = "rply" +version = "0.7.8" +description = "A pure Python Lex/Yacc that works with RPython" +category = "dev" +optional = false +python-versions = "*" + +[package.dependencies] +appdirs = "*" + +[[package]] +name = "secretstorage" +version = "3.3.1" +description = "Python bindings to FreeDesktop.org Secret Service API" +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +cryptography = ">=2.0" +jeepney = ">=0.6" + +[[package]] +name = "sentry-sdk" +version = "1.5.7" +description = "Python client for Sentry (https://sentry.io)" +category = "main" +optional = true +python-versions = "*" + +[package.dependencies] +certifi = "*" +urllib3 = ">=1.10.0" + +[package.extras] +aiohttp = ["aiohttp (>=3.5)"] +beam = ["apache-beam (>=2.12)"] +bottle = ["bottle (>=0.12.13)"] +celery = ["celery (>=3)"] +chalice = ["chalice (>=1.16.0)"] +django = ["django (>=1.8)"] +falcon = ["falcon (>=1.4)"] +flask = ["flask (>=0.11)", "blinker (>=1.1)"] +httpx = ["httpx (>=0.16.0)"] +pure_eval = ["pure-eval", "executing", "asttokens"] +pyspark = ["pyspark (>=2.4.4)"] +quart = ["quart (>=0.16.1)", "blinker (>=1.1)"] +rq = ["rq (>=0.6)"] +sanic = ["sanic (>=0.8)"] +sqlalchemy = ["sqlalchemy (>=1.2)"] +tornado = ["tornado (>=5)"] + +[[package]] +name = "service-identity" +version = "21.1.0" +description = "Service identity verification for pyOpenSSL & cryptography." +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +attrs = ">=19.1.0" +cryptography = "*" +pyasn1 = "*" +pyasn1-modules = "*" +six = "*" + +[package.extras] +dev = ["coverage[toml] (>=5.0.2)", "pytest", "sphinx", "furo", "idna", "pyopenssl"] +docs = ["sphinx", "furo"] +idna = ["idna"] +tests = ["coverage[toml] (>=5.0.2)", "pytest"] + +[[package]] +name = "signedjson" +version = "1.1.4" +description = "Sign JSON with Ed25519 signatures" +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +canonicaljson = ">=1.0.0" +importlib-metadata = {version = "*", markers = "python_version < \"3.8\""} +pynacl = ">=0.3.0" +typing-extensions = {version = ">=3.5", markers = "python_version < \"3.8\""} +unpaddedbase64 = ">=1.0.1" + +[package.extras] +dev = ["typing-extensions (>=3.5)"] + +[[package]] +name = "simplejson" +version = "3.17.6" +description = "Simple, fast, extensible JSON encoder/decoder for Python" +category = "main" +optional = false +python-versions = ">=2.5, !=3.0.*, !=3.1.*, !=3.2.*" + +[[package]] +name = "six" +version = "1.16.0" +description = "Python 2 and 3 compatibility utilities" +category = "main" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*" + +[[package]] +name = "smmap" +version = "5.0.0" +description = "A pure Python implementation of a sliding window memory map manager" +category = "dev" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "sortedcontainers" +version = "2.4.0" +description = "Sorted Containers -- Sorted List, Sorted Dict, Sorted Set" +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "systemd-python" +version = "234" +description = "Python interface for libsystemd" +category = "main" +optional = true +python-versions = "*" + +[[package]] +name = "threadloop" +version = "1.0.2" +description = "Tornado IOLoop Backed Concurrent Futures" +category = "main" +optional = true +python-versions = "*" + +[package.dependencies] +tornado = "*" + +[[package]] +name = "thrift" +version = "0.15.0" +description = "Python bindings for the Apache Thrift RPC system" +category = "main" +optional = true +python-versions = "*" + +[package.dependencies] +six = ">=1.7.2" + +[package.extras] +all = ["tornado (>=4.0)", "twisted"] +tornado = ["tornado (>=4.0)"] +twisted = ["twisted"] + +[[package]] +name = "tomli" +version = "1.2.3" +description = "A lil' TOML parser" +category = "dev" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "tornado" +version = "6.1" +description = "Tornado is a Python web framework and asynchronous networking library, originally developed at FriendFeed." +category = "main" +optional = true +python-versions = ">= 3.5" + +[[package]] +name = "towncrier" +version = "21.9.0" +description = "Building newsfiles for your project." +category = "dev" +optional = false +python-versions = "*" + +[package.dependencies] +click = "*" +click-default-group = "*" +incremental = "*" +jinja2 = "*" +tomli = {version = "*", markers = "python_version >= \"3.6\""} + +[package.extras] +dev = ["packaging"] + +[[package]] +name = "tqdm" +version = "4.63.0" +description = "Fast, Extensible Progress Meter" +category = "dev" +optional = false +python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7" + +[package.dependencies] +colorama = {version = "*", markers = "platform_system == \"Windows\""} + +[package.extras] +dev = ["py-make (>=0.1.0)", "twine", "wheel"] +notebook = ["ipywidgets (>=6)"] +telegram = ["requests"] + +[[package]] +name = "treq" +version = "22.2.0" +description = "High-level Twisted HTTP Client API" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +attrs = "*" +hyperlink = ">=21.0.0" +incremental = "*" +requests = ">=2.1.0" +Twisted = {version = ">=18.7.0", extras = ["tls"]} + +[package.extras] +dev = ["pep8", "pyflakes", "httpbin (==0.5.0)"] +docs = ["sphinx (>=1.4.8)"] + +[[package]] +name = "twine" +version = "3.8.0" +description = "Collection of utilities for publishing packages on PyPI" +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +colorama = ">=0.4.3" +importlib-metadata = ">=3.6" +keyring = ">=15.1" +pkginfo = ">=1.8.1" +readme-renderer = ">=21.0" +requests = ">=2.20" +requests-toolbelt = ">=0.8.0,<0.9.0 || >0.9.0" +rfc3986 = ">=1.4.0" +tqdm = ">=4.14" +urllib3 = ">=1.26.0" + +[[package]] +name = "twisted" +version = "22.2.0" +description = "An asynchronous networking framework written in Python" +category = "main" +optional = false +python-versions = ">=3.6.7" + +[package.dependencies] +attrs = ">=19.2.0" +Automat = ">=0.8.0" +constantly = ">=15.1" +hyperlink = ">=17.1.1" +idna = {version = ">=2.4", optional = true, markers = "extra == \"tls\""} +incremental = ">=21.3.0" +pyopenssl = {version = ">=16.0.0", optional = true, markers = "extra == \"tls\""} +service-identity = {version = ">=18.1.0", optional = true, markers = "extra == \"tls\""} +twisted-iocpsupport = {version = ">=1.0.2,<2", markers = "platform_system == \"Windows\""} +typing-extensions = ">=3.6.5" +"zope.interface" = ">=4.4.2" + +[package.extras] +all_non_platform = ["cython-test-exception-raiser (>=1.0.2,<2)", "PyHamcrest (>=1.9.0)", "pyopenssl (>=16.0.0)", "service-identity (>=18.1.0)", "idna (>=2.4)", "pyasn1", "cryptography (>=2.6)", "appdirs (>=1.4.0)", "bcrypt (>=3.0.0)", "pyserial (>=3.0)", "h2 (>=3.0,<4.0)", "priority (>=1.1.0,<2.0)", "pywin32 (!=226)", "contextvars (>=2.4,<3)"] +conch = ["pyasn1", "cryptography (>=2.6)", "appdirs (>=1.4.0)", "bcrypt (>=3.0.0)"] +contextvars = ["contextvars (>=2.4,<3)"] +dev = ["towncrier (>=19.2,<20.0)", "sphinx-rtd-theme (>=0.5,<1.0)", "readthedocs-sphinx-ext (>=2.1,<3.0)", "sphinx (>=4.1.2,<6)", "pyflakes (>=2.2,<3.0)", "twistedchecker (>=0.7,<1.0)", "coverage (>=6b1,<7)", "python-subunit (>=1.4,<2.0)", "pydoctor (>=21.9.0,<21.10.0)"] +dev_release = ["towncrier (>=19.2,<20.0)", "sphinx-rtd-theme (>=0.5,<1.0)", "readthedocs-sphinx-ext (>=2.1,<3.0)", "sphinx (>=4.1.2,<6)", "pydoctor (>=21.9.0,<21.10.0)"] +http2 = ["h2 (>=3.0,<4.0)", "priority (>=1.1.0,<2.0)"] +macos_platform = ["pyobjc-core", "pyobjc-framework-cfnetwork", "pyobjc-framework-cocoa", "cython-test-exception-raiser (>=1.0.2,<2)", "PyHamcrest (>=1.9.0)", "pyopenssl (>=16.0.0)", "service-identity (>=18.1.0)", "idna (>=2.4)", "pyasn1", "cryptography (>=2.6)", "appdirs (>=1.4.0)", "bcrypt (>=3.0.0)", "pyserial (>=3.0)", "h2 (>=3.0,<4.0)", "priority (>=1.1.0,<2.0)", "pywin32 (!=226)", "contextvars (>=2.4,<3)"] +mypy = ["mypy (==0.930)", "mypy-zope (==0.3.4)", "types-setuptools", "types-pyopenssl", "towncrier (>=19.2,<20.0)", "sphinx-rtd-theme (>=0.5,<1.0)", "readthedocs-sphinx-ext (>=2.1,<3.0)", "sphinx (>=4.1.2,<6)", "pyflakes (>=2.2,<3.0)", "twistedchecker (>=0.7,<1.0)", "coverage (>=6b1,<7)", "cython-test-exception-raiser (>=1.0.2,<2)", "PyHamcrest (>=1.9.0)", "pyopenssl (>=16.0.0)", "service-identity (>=18.1.0)", "idna (>=2.4)", "pyasn1", "cryptography (>=2.6)", "appdirs (>=1.4.0)", "bcrypt (>=3.0.0)", "pyserial (>=3.0)", "h2 (>=3.0,<4.0)", "priority (>=1.1.0,<2.0)", "pywin32 (!=226)", "python-subunit (>=1.4,<2.0)", "contextvars (>=2.4,<3)", "pydoctor (>=21.9.0,<21.10.0)"] +osx_platform = ["pyobjc-core", "pyobjc-framework-cfnetwork", "pyobjc-framework-cocoa", "cython-test-exception-raiser (>=1.0.2,<2)", "PyHamcrest (>=1.9.0)", "pyopenssl (>=16.0.0)", "service-identity (>=18.1.0)", "idna (>=2.4)", "pyasn1", "cryptography (>=2.6)", "appdirs (>=1.4.0)", "bcrypt (>=3.0.0)", "pyserial (>=3.0)", "h2 (>=3.0,<4.0)", "priority (>=1.1.0,<2.0)", "pywin32 (!=226)", "contextvars (>=2.4,<3)"] +serial = ["pyserial (>=3.0)", "pywin32 (!=226)"] +test = ["cython-test-exception-raiser (>=1.0.2,<2)", "PyHamcrest (>=1.9.0)"] +tls = ["pyopenssl (>=16.0.0)", "service-identity (>=18.1.0)", "idna (>=2.4)"] +windows_platform = ["pywin32 (!=226)", "cython-test-exception-raiser (>=1.0.2,<2)", "PyHamcrest (>=1.9.0)", "pyopenssl (>=16.0.0)", "service-identity (>=18.1.0)", "idna (>=2.4)", "pyasn1", "cryptography (>=2.6)", "appdirs (>=1.4.0)", "bcrypt (>=3.0.0)", "pyserial (>=3.0)", "h2 (>=3.0,<4.0)", "priority (>=1.1.0,<2.0)", "pywin32 (!=226)", "contextvars (>=2.4,<3)"] + +[[package]] +name = "twisted-iocpsupport" +version = "1.0.2" +description = "An extension for use in the twisted I/O Completion Ports reactor." +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "txredisapi" +version = "1.4.7" +description = "non-blocking redis client for python" +category = "main" +optional = true +python-versions = "*" + +[package.dependencies] +six = "*" +twisted = "*" + +[[package]] +name = "typed-ast" +version = "1.5.2" +description = "a fork of Python 2 and 3 ast modules with type comment support" +category = "dev" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "types-bleach" +version = "4.1.4" +description = "Typing stubs for bleach" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "types-cryptography" +version = "3.3.15" +description = "Typing stubs for cryptography" +category = "dev" +optional = false +python-versions = "*" + +[package.dependencies] +types-enum34 = "*" +types-ipaddress = "*" + +[[package]] +name = "types-enum34" +version = "1.1.8" +description = "Typing stubs for enum34" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "types-ipaddress" +version = "1.0.8" +description = "Typing stubs for ipaddress" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "types-jsonschema" +version = "4.4.1" +description = "Typing stubs for jsonschema" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "types-opentracing" +version = "2.4.7" +description = "Typing stubs for opentracing" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "types-pillow" +version = "9.0.6" +description = "Typing stubs for Pillow" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "types-psycopg2" +version = "2.9.9" +description = "Typing stubs for psycopg2" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "types-pyopenssl" +version = "22.0.0" +description = "Typing stubs for pyOpenSSL" +category = "dev" +optional = false +python-versions = "*" + +[package.dependencies] +types-cryptography = "*" + +[[package]] +name = "types-pyyaml" +version = "6.0.4" +description = "Typing stubs for PyYAML" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "types-requests" +version = "2.27.11" +description = "Typing stubs for requests" +category = "dev" +optional = false +python-versions = "*" + +[package.dependencies] +types-urllib3 = "<1.27" + +[[package]] +name = "types-setuptools" +version = "57.4.9" +description = "Typing stubs for setuptools" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "types-urllib3" +version = "1.26.10" +description = "Typing stubs for urllib3" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "typing-extensions" +version = "4.1.1" +description = "Backported and Experimental Type Hints for Python 3.6+" +category = "main" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "unpaddedbase64" +version = "2.1.0" +description = "Encode and decode Base64 without \"=\" padding" +category = "main" +optional = false +python-versions = ">=3.6,<4.0" + +[[package]] +name = "urllib3" +version = "1.26.8" +description = "HTTP library with thread-safe connection pooling, file post, and more." +category = "main" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, <4" + +[package.extras] +brotli = ["brotlipy (>=0.6.0)"] +secure = ["pyOpenSSL (>=0.14)", "cryptography (>=1.3.4)", "idna (>=2.0.0)", "certifi", "ipaddress"] +socks = ["PySocks (>=1.5.6,!=1.5.7,<2.0)"] + +[[package]] +name = "webencodings" +version = "0.5.1" +description = "Character encoding aliases for legacy web content" +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "wrapt" +version = "1.13.3" +description = "Module for decorators, wrappers and monkey patching." +category = "dev" +optional = false +python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7" + +[[package]] +name = "xmlschema" +version = "1.10.0" +description = "An XML Schema validator and decoder" +category = "main" +optional = true +python-versions = ">=3.7" + +[package.dependencies] +elementpath = ">=2.5.0,<3.0.0" + +[package.extras] +codegen = ["elementpath (>=2.5.0,<3.0.0)", "jinja2"] +dev = ["tox", "coverage", "lxml", "elementpath (>=2.5.0,<3.0.0)", "memory-profiler", "sphinx", "sphinx-rtd-theme", "jinja2", "flake8", "mypy", "lxml-stubs"] +docs = ["elementpath (>=2.5.0,<3.0.0)", "sphinx", "sphinx-rtd-theme", "jinja2"] + +[[package]] +name = "zipp" +version = "3.7.0" +description = "Backport of pathlib-compatible object wrapper for zip files" +category = "main" +optional = false +python-versions = ">=3.7" + +[package.extras] +docs = ["sphinx", "jaraco.packaging (>=8.2)", "rst.linker (>=1.9)"] +testing = ["pytest (>=6)", "pytest-checkdocs (>=2.4)", "pytest-flake8", "pytest-cov", "pytest-enabler (>=1.0.1)", "jaraco.itertools", "func-timeout", "pytest-black (>=0.3.7)", "pytest-mypy"] + +[[package]] +name = "zope.event" +version = "4.5.0" +description = "Very basic event publishing system" +category = "dev" +optional = false +python-versions = "*" + +[package.extras] +docs = ["sphinx"] +test = ["zope.testrunner"] + +[[package]] +name = "zope.interface" +version = "5.4.0" +description = "Interfaces for Python" +category = "main" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" + +[package.extras] +docs = ["sphinx", "repoze.sphinx.autointerface"] +test = ["coverage (>=5.0.3)", "zope.event", "zope.testing"] +testing = ["coverage (>=5.0.3)", "zope.event", "zope.testing"] + +[[package]] +name = "zope.schema" +version = "6.2.0" +description = "zope.interface extension for defining data schemas" +category = "dev" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" + +[package.dependencies] +"zope.event" = "*" +"zope.interface" = ">=5.0.0" + +[package.extras] +docs = ["sphinx", "repoze.sphinx.autointerface"] +test = ["zope.i18nmessageid", "zope.testing", "zope.testrunner"] + +[extras] +all = ["matrix-synapse-ldap3", "psycopg2", "psycopg2cffi", "psycopg2cffi-compat", "pysaml2", "authlib", "lxml", "sentry-sdk", "jaeger-client", "opentracing", "pyjwt", "txredisapi", "hiredis"] +cache_memory = ["Pympler"] +jwt = ["pyjwt"] +matrix-synapse-ldap3 = ["matrix-synapse-ldap3"] +oidc = ["authlib"] +opentracing = ["jaeger-client", "opentracing"] +postgres = ["psycopg2", "psycopg2cffi", "psycopg2cffi-compat"] +redis = ["txredisapi", "hiredis"] +saml2 = ["pysaml2"] +sentry = ["sentry-sdk"] +systemd = ["systemd-python"] +test = ["parameterized"] +url_preview = ["lxml"] + +[metadata] +lock-version = "1.1" +python-versions = "^3.7" +content-hash = "92465e65bef59c43c6112f332e5cf987740899801dbd34c19245db15b5c6362d" + +[metadata.files] +appdirs = [ + {file = "appdirs-1.4.4-py2.py3-none-any.whl", hash = "sha256:a841dacd6b99318a741b166adb07e19ee71a274450e68237b4650ca1055ab128"}, + {file = "appdirs-1.4.4.tar.gz", hash = "sha256:7d5d0167b2b1ba821647616af46a749d1c653740dd0d2415100fe26e27afdf41"}, +] +attrs = [ + {file = "attrs-21.4.0-py2.py3-none-any.whl", hash = "sha256:2d27e3784d7a565d36ab851fe94887c5eccd6a463168875832a1be79c82828b4"}, + {file = "attrs-21.4.0.tar.gz", hash = "sha256:626ba8234211db98e869df76230a137c4c40a12d72445c45d5f5b716f076e2fd"}, +] +authlib = [ + {file = "Authlib-0.15.5-py2.py3-none-any.whl", hash = "sha256:ecf4a7a9f2508c0bb07e93a752dd3c495cfaffc20e864ef0ffc95e3f40d2abaf"}, + {file = "Authlib-0.15.5.tar.gz", hash = "sha256:b83cf6360c8e92b0e9df0d1f32d675790bcc4e3c03977499b1eed24dcdef4252"}, +] +automat = [ + {file = "Automat-20.2.0-py2.py3-none-any.whl", hash = "sha256:b6feb6455337df834f6c9962d6ccf771515b7d939bca142b29c20c2376bc6111"}, + {file = "Automat-20.2.0.tar.gz", hash = "sha256:7979803c74610e11ef0c0d68a2942b152df52da55336e0c9d58daf1831cbdf33"}, +] +baron = [ + {file = "baron-0.10.1-py2.py3-none-any.whl", hash = "sha256:befb33f4b9e832c7cd1e3cf0eafa6dd3cb6ed4cb2544245147c019936f4e0a8a"}, + {file = "baron-0.10.1.tar.gz", hash = "sha256:af822ad44d4eb425c8516df4239ac4fdba9fdb398ef77e4924cd7c9b4045bc2f"}, +] +bcrypt = [ + {file = "bcrypt-3.2.0-cp36-abi3-macosx_10_10_universal2.whl", hash = "sha256:b589229207630484aefe5899122fb938a5b017b0f4349f769b8c13e78d99a8fd"}, + {file = "bcrypt-3.2.0-cp36-abi3-macosx_10_9_x86_64.whl", hash = "sha256:c95d4cbebffafcdd28bd28bb4e25b31c50f6da605c81ffd9ad8a3d1b2ab7b1b6"}, + {file = "bcrypt-3.2.0-cp36-abi3-manylinux1_x86_64.whl", hash = "sha256:63d4e3ff96188e5898779b6057878fecf3f11cfe6ec3b313ea09955d587ec7a7"}, + {file = "bcrypt-3.2.0-cp36-abi3-manylinux2010_x86_64.whl", hash = "sha256:cd1ea2ff3038509ea95f687256c46b79f5fc382ad0aa3664d200047546d511d1"}, + {file = "bcrypt-3.2.0-cp36-abi3-manylinux2014_aarch64.whl", hash = "sha256:cdcdcb3972027f83fe24a48b1e90ea4b584d35f1cc279d76de6fc4b13376239d"}, + {file = "bcrypt-3.2.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl", hash = "sha256:a0584a92329210fcd75eb8a3250c5a941633f8bfaf2a18f81009b097732839b7"}, + {file = "bcrypt-3.2.0-cp36-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:56e5da069a76470679f312a7d3d23deb3ac4519991a0361abc11da837087b61d"}, + {file = "bcrypt-3.2.0-cp36-abi3-win32.whl", hash = "sha256:a67fb841b35c28a59cebed05fbd3e80eea26e6d75851f0574a9273c80f3e9b55"}, + {file = "bcrypt-3.2.0-cp36-abi3-win_amd64.whl", hash = "sha256:81fec756feff5b6818ea7ab031205e1d323d8943d237303baca2c5f9c7846f34"}, + {file = "bcrypt-3.2.0.tar.gz", hash = "sha256:5b93c1726e50a93a033c36e5ca7fdcd29a5c7395af50a6892f5d9e7c6cfbfb29"}, +] +black = [ + {file = "black-22.3.0-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:2497f9c2386572e28921fa8bec7be3e51de6801f7459dffd6e62492531c47e09"}, + {file = "black-22.3.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:5795a0375eb87bfe902e80e0c8cfaedf8af4d49694d69161e5bd3206c18618bb"}, + {file = "black-22.3.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:e3556168e2e5c49629f7b0f377070240bd5511e45e25a4497bb0073d9dda776a"}, + {file = "black-22.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:67c8301ec94e3bcc8906740fe071391bce40a862b7be0b86fb5382beefecd968"}, + {file = "black-22.3.0-cp310-cp310-win_amd64.whl", hash = "sha256:fd57160949179ec517d32ac2ac898b5f20d68ed1a9c977346efbac9c2f1e779d"}, + {file = "black-22.3.0-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:cc1e1de68c8e5444e8f94c3670bb48a2beef0e91dddfd4fcc29595ebd90bb9ce"}, + {file = "black-22.3.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:6d2fc92002d44746d3e7db7cf9313cf4452f43e9ea77a2c939defce3b10b5c82"}, + {file = "black-22.3.0-cp36-cp36m-win_amd64.whl", hash = "sha256:a6342964b43a99dbc72f72812bf88cad8f0217ae9acb47c0d4f141a6416d2d7b"}, + {file = "black-22.3.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:328efc0cc70ccb23429d6be184a15ce613f676bdfc85e5fe8ea2a9354b4e9015"}, + {file = "black-22.3.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:06f9d8846f2340dfac80ceb20200ea5d1b3f181dd0556b47af4e8e0b24fa0a6b"}, + {file = "black-22.3.0-cp37-cp37m-win_amd64.whl", hash = "sha256:ad4efa5fad66b903b4a5f96d91461d90b9507a812b3c5de657d544215bb7877a"}, + {file = "black-22.3.0-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:e8477ec6bbfe0312c128e74644ac8a02ca06bcdb8982d4ee06f209be28cdf163"}, + {file = "black-22.3.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:637a4014c63fbf42a692d22b55d8ad6968a946b4a6ebc385c5505d9625b6a464"}, + {file = "black-22.3.0-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:863714200ada56cbc366dc9ae5291ceb936573155f8bf8e9de92aef51f3ad0f0"}, + {file = "black-22.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:10dbe6e6d2988049b4655b2b739f98785a884d4d6b85bc35133a8fb9a2233176"}, + {file = "black-22.3.0-cp38-cp38-win_amd64.whl", hash = "sha256:cee3e11161dde1b2a33a904b850b0899e0424cc331b7295f2a9698e79f9a69a0"}, + {file = "black-22.3.0-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:5891ef8abc06576985de8fa88e95ab70641de6c1fca97e2a15820a9b69e51b20"}, + {file = "black-22.3.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:30d78ba6bf080eeaf0b7b875d924b15cd46fec5fd044ddfbad38c8ea9171043a"}, + {file = "black-22.3.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:ee8f1f7228cce7dffc2b464f07ce769f478968bfb3dd1254a4c2eeed84928aad"}, + {file = "black-22.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:6ee227b696ca60dd1c507be80a6bc849a5a6ab57ac7352aad1ffec9e8b805f21"}, + {file = "black-22.3.0-cp39-cp39-win_amd64.whl", hash = "sha256:9b542ced1ec0ceeff5b37d69838106a6348e60db7b8fdd245294dc1d26136265"}, + {file = "black-22.3.0-py3-none-any.whl", hash = "sha256:bc58025940a896d7e5356952228b68f793cf5fcb342be703c3a2669a1488cb72"}, + {file = "black-22.3.0.tar.gz", hash = "sha256:35020b8886c022ced9282b51b5a875b6d1ab0c387b31a065b84db7c33085ca79"}, +] +bleach = [ + {file = "bleach-4.1.0-py2.py3-none-any.whl", hash = "sha256:4d2651ab93271d1129ac9cbc679f524565cc8a1b791909c4a51eac4446a15994"}, + {file = "bleach-4.1.0.tar.gz", hash = "sha256:0900d8b37eba61a802ee40ac0061f8c2b5dee29c1927dd1d233e075ebf5a71da"}, +] +canonicaljson = [ + {file = "canonicaljson-1.6.0-py3-none-any.whl", hash = "sha256:7230c2a2a3db07874f622af84effe41a655e07bf23734830e18a454e65d5b998"}, + {file = "canonicaljson-1.6.0.tar.gz", hash = "sha256:8739d5fd91aca7281d425660ae65af7663808c8177778965f67e90b16a2b2427"}, +] +certifi = [ + {file = "certifi-2021.10.8-py2.py3-none-any.whl", hash = "sha256:d62a0163eb4c2344ac042ab2bdf75399a71a2d8c7d47eac2e2ee91b9d6339569"}, + {file = "certifi-2021.10.8.tar.gz", hash = "sha256:78884e7c1d4b00ce3cea67b44566851c4343c120abd683433ce934a68ea58872"}, +] +cffi = [ + {file = "cffi-1.15.0-cp27-cp27m-macosx_10_9_x86_64.whl", hash = "sha256:c2502a1a03b6312837279c8c1bd3ebedf6c12c4228ddbad40912d671ccc8a962"}, + {file = "cffi-1.15.0-cp27-cp27m-manylinux1_i686.whl", hash = "sha256:23cfe892bd5dd8941608f93348c0737e369e51c100d03718f108bf1add7bd6d0"}, + {file = "cffi-1.15.0-cp27-cp27m-manylinux1_x86_64.whl", hash = "sha256:41d45de54cd277a7878919867c0f08b0cf817605e4eb94093e7516505d3c8d14"}, + {file = "cffi-1.15.0-cp27-cp27m-win32.whl", hash = "sha256:4a306fa632e8f0928956a41fa8e1d6243c71e7eb59ffbd165fc0b41e316b2474"}, + {file = "cffi-1.15.0-cp27-cp27m-win_amd64.whl", hash = "sha256:e7022a66d9b55e93e1a845d8c9eba2a1bebd4966cd8bfc25d9cd07d515b33fa6"}, + {file = "cffi-1.15.0-cp27-cp27mu-manylinux1_i686.whl", hash = "sha256:14cd121ea63ecdae71efa69c15c5543a4b5fbcd0bbe2aad864baca0063cecf27"}, + {file = "cffi-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl", hash = "sha256:d4d692a89c5cf08a8557fdeb329b82e7bf609aadfaed6c0d79f5a449a3c7c023"}, + {file = "cffi-1.15.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:0104fb5ae2391d46a4cb082abdd5c69ea4eab79d8d44eaaf79f1b1fd806ee4c2"}, + {file = "cffi-1.15.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:91ec59c33514b7c7559a6acda53bbfe1b283949c34fe7440bcf917f96ac0723e"}, + {file = "cffi-1.15.0-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:f5c7150ad32ba43a07c4479f40241756145a1f03b43480e058cfd862bf5041c7"}, + {file = "cffi-1.15.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:00c878c90cb53ccfaae6b8bc18ad05d2036553e6d9d1d9dbcf323bbe83854ca3"}, + {file = "cffi-1.15.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:abb9a20a72ac4e0fdb50dae135ba5e77880518e742077ced47eb1499e29a443c"}, + {file = "cffi-1.15.0-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:a5263e363c27b653a90078143adb3d076c1a748ec9ecc78ea2fb916f9b861962"}, + {file = "cffi-1.15.0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:f54a64f8b0c8ff0b64d18aa76675262e1700f3995182267998c31ae974fbc382"}, + {file = "cffi-1.15.0-cp310-cp310-win32.whl", hash = "sha256:c21c9e3896c23007803a875460fb786118f0cdd4434359577ea25eb556e34c55"}, + {file = "cffi-1.15.0-cp310-cp310-win_amd64.whl", hash = "sha256:5e069f72d497312b24fcc02073d70cb989045d1c91cbd53979366077959933e0"}, + {file = "cffi-1.15.0-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:64d4ec9f448dfe041705426000cc13e34e6e5bb13736e9fd62e34a0b0c41566e"}, + {file = "cffi-1.15.0-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2756c88cbb94231c7a147402476be2c4df2f6078099a6f4a480d239a8817ae39"}, + {file = "cffi-1.15.0-cp36-cp36m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3b96a311ac60a3f6be21d2572e46ce67f09abcf4d09344c49274eb9e0bf345fc"}, + {file = "cffi-1.15.0-cp36-cp36m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:75e4024375654472cc27e91cbe9eaa08567f7fbdf822638be2814ce059f58032"}, + {file = "cffi-1.15.0-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:59888172256cac5629e60e72e86598027aca6bf01fa2465bdb676d37636573e8"}, + {file = "cffi-1.15.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:27c219baf94952ae9d50ec19651a687b826792055353d07648a5695413e0c605"}, + {file = "cffi-1.15.0-cp36-cp36m-win32.whl", hash = "sha256:4958391dbd6249d7ad855b9ca88fae690783a6be9e86df65865058ed81fc860e"}, + {file = "cffi-1.15.0-cp36-cp36m-win_amd64.whl", hash = "sha256:f6f824dc3bce0edab5f427efcfb1d63ee75b6fcb7282900ccaf925be84efb0fc"}, + {file = "cffi-1.15.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:06c48159c1abed75c2e721b1715c379fa3200c7784271b3c46df01383b593636"}, + {file = "cffi-1.15.0-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:c2051981a968d7de9dd2d7b87bcb9c939c74a34626a6e2f8181455dd49ed69e4"}, + {file = "cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:fd8a250edc26254fe5b33be00402e6d287f562b6a5b2152dec302fa15bb3e997"}, + {file = "cffi-1.15.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:91d77d2a782be4274da750752bb1650a97bfd8f291022b379bb8e01c66b4e96b"}, + {file = "cffi-1.15.0-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:45db3a33139e9c8f7c09234b5784a5e33d31fd6907800b316decad50af323ff2"}, + {file = "cffi-1.15.0-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:263cc3d821c4ab2213cbe8cd8b355a7f72a8324577dc865ef98487c1aeee2bc7"}, + {file = "cffi-1.15.0-cp37-cp37m-win32.whl", hash = "sha256:17771976e82e9f94976180f76468546834d22a7cc404b17c22df2a2c81db0c66"}, + {file = "cffi-1.15.0-cp37-cp37m-win_amd64.whl", hash = "sha256:3415c89f9204ee60cd09b235810be700e993e343a408693e80ce7f6a40108029"}, + {file = "cffi-1.15.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:4238e6dab5d6a8ba812de994bbb0a79bddbdf80994e4ce802b6f6f3142fcc880"}, + {file = "cffi-1.15.0-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:0808014eb713677ec1292301ea4c81ad277b6cdf2fdd90fd540af98c0b101d20"}, + {file = "cffi-1.15.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:57e9ac9ccc3101fac9d6014fba037473e4358ef4e89f8e181f8951a2c0162024"}, + {file = "cffi-1.15.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8b6c2ea03845c9f501ed1313e78de148cd3f6cad741a75d43a29b43da27f2e1e"}, + {file = "cffi-1.15.0-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:10dffb601ccfb65262a27233ac273d552ddc4d8ae1bf93b21c94b8511bffe728"}, + {file = "cffi-1.15.0-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:786902fb9ba7433aae840e0ed609f45c7bcd4e225ebb9c753aa39725bb3e6ad6"}, + {file = "cffi-1.15.0-cp38-cp38-win32.whl", hash = "sha256:da5db4e883f1ce37f55c667e5c0de439df76ac4cb55964655906306918e7363c"}, + {file = "cffi-1.15.0-cp38-cp38-win_amd64.whl", hash = "sha256:181dee03b1170ff1969489acf1c26533710231c58f95534e3edac87fff06c443"}, + {file = "cffi-1.15.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:45e8636704eacc432a206ac7345a5d3d2c62d95a507ec70d62f23cd91770482a"}, + {file = "cffi-1.15.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:31fb708d9d7c3f49a60f04cf5b119aeefe5644daba1cd2a0fe389b674fd1de37"}, + {file = "cffi-1.15.0-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:6dc2737a3674b3e344847c8686cf29e500584ccad76204efea14f451d4cc669a"}, + {file = "cffi-1.15.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:74fdfdbfdc48d3f47148976f49fab3251e550a8720bebc99bf1483f5bfb5db3e"}, + {file = "cffi-1.15.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ffaa5c925128e29efbde7301d8ecaf35c8c60ffbcd6a1ffd3a552177c8e5e796"}, + {file = "cffi-1.15.0-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3f7d084648d77af029acb79a0ff49a0ad7e9d09057a9bf46596dac9514dc07df"}, + {file = "cffi-1.15.0-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:ef1f279350da2c586a69d32fc8733092fd32cc8ac95139a00377841f59a3f8d8"}, + {file = "cffi-1.15.0-cp39-cp39-win32.whl", hash = "sha256:2a23af14f408d53d5e6cd4e3d9a24ff9e05906ad574822a10563efcef137979a"}, + {file = "cffi-1.15.0-cp39-cp39-win_amd64.whl", hash = "sha256:3773c4d81e6e818df2efbc7dd77325ca0dcb688116050fb2b3011218eda36139"}, + {file = "cffi-1.15.0.tar.gz", hash = "sha256:920f0d66a896c2d99f0adbb391f990a84091179542c205fa53ce5787aff87954"}, +] +charset-normalizer = [ + {file = "charset-normalizer-2.0.12.tar.gz", hash = "sha256:2857e29ff0d34db842cd7ca3230549d1a697f96ee6d3fb071cfa6c7393832597"}, + {file = "charset_normalizer-2.0.12-py3-none-any.whl", hash = "sha256:6881edbebdb17b39b4eaaa821b438bf6eddffb4468cf344f09f89def34a8b1df"}, +] +click = [ + {file = "click-8.1.0-py3-none-any.whl", hash = "sha256:19a4baa64da924c5e0cd889aba8e947f280309f1a2ce0947a3e3a7bcb7cc72d6"}, + {file = "click-8.1.0.tar.gz", hash = "sha256:977c213473c7665d3aa092b41ff12063227751c41d7b17165013e10069cc5cd2"}, +] +click-default-group = [ + {file = "click-default-group-1.2.2.tar.gz", hash = "sha256:d9560e8e8dfa44b3562fbc9425042a0fd6d21956fcc2db0077f63f34253ab904"}, +] +colorama = [ + {file = "colorama-0.4.4-py2.py3-none-any.whl", hash = "sha256:9f47eda37229f68eee03b24b9748937c7dc3868f906e8ba69fbcbdd3bc5dc3e2"}, + {file = "colorama-0.4.4.tar.gz", hash = "sha256:5941b2b48a20143d2267e95b1c2a7603ce057ee39fd88e7329b0c292aa16869b"}, +] +commonmark = [ + {file = "commonmark-0.9.1-py2.py3-none-any.whl", hash = "sha256:da2f38c92590f83de410ba1a3cbceafbc74fee9def35f9251ba9a971d6d66fd9"}, + {file = "commonmark-0.9.1.tar.gz", hash = "sha256:452f9dc859be7f06631ddcb328b6919c67984aca654e5fefb3914d54691aed60"}, +] +constantly = [ + {file = "constantly-15.1.0-py2.py3-none-any.whl", hash = "sha256:dd2fa9d6b1a51a83f0d7dd76293d734046aa176e384bf6e33b7e44880eb37c5d"}, + {file = "constantly-15.1.0.tar.gz", hash = "sha256:586372eb92059873e29eba4f9dec8381541b4d3834660707faf8ba59146dfc35"}, +] +cryptography = [ + {file = "cryptography-36.0.1-cp36-abi3-macosx_10_10_universal2.whl", hash = "sha256:73bc2d3f2444bcfeac67dd130ff2ea598ea5f20b40e36d19821b4df8c9c5037b"}, + {file = "cryptography-36.0.1-cp36-abi3-macosx_10_10_x86_64.whl", hash = "sha256:2d87cdcb378d3cfed944dac30596da1968f88fb96d7fc34fdae30a99054b2e31"}, + {file = "cryptography-36.0.1-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:74d6c7e80609c0f4c2434b97b80c7f8fdfaa072ca4baab7e239a15d6d70ed73a"}, + {file = "cryptography-36.0.1-cp36-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl", hash = "sha256:6c0c021f35b421ebf5976abf2daacc47e235f8b6082d3396a2fe3ccd537ab173"}, + {file = "cryptography-36.0.1-cp36-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5d59a9d55027a8b88fd9fd2826c4392bd487d74bf628bb9d39beecc62a644c12"}, + {file = "cryptography-36.0.1-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0a817b961b46894c5ca8a66b599c745b9a3d9f822725221f0e0fe49dc043a3a3"}, + {file = "cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl", hash = "sha256:94ae132f0e40fe48f310bba63f477f14a43116f05ddb69d6fa31e93f05848ae2"}, + {file = "cryptography-36.0.1-cp36-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:7be0eec337359c155df191d6ae00a5e8bbb63933883f4f5dffc439dac5348c3f"}, + {file = "cryptography-36.0.1-cp36-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:e0344c14c9cb89e76eb6a060e67980c9e35b3f36691e15e1b7a9e58a0a6c6dc3"}, + {file = "cryptography-36.0.1-cp36-abi3-win32.whl", hash = "sha256:4caa4b893d8fad33cf1964d3e51842cd78ba87401ab1d2e44556826df849a8ca"}, + {file = "cryptography-36.0.1-cp36-abi3-win_amd64.whl", hash = "sha256:391432971a66cfaf94b21c24ab465a4cc3e8bf4a939c1ca5c3e3a6e0abebdbcf"}, + {file = "cryptography-36.0.1-pp37-pypy37_pp73-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:bb5829d027ff82aa872d76158919045a7c1e91fbf241aec32cb07956e9ebd3c9"}, + {file = "cryptography-36.0.1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ebc15b1c22e55c4d5566e3ca4db8689470a0ca2babef8e3a9ee057a8b82ce4b1"}, + {file = "cryptography-36.0.1-pp37-pypy37_pp73-manylinux_2_24_x86_64.whl", hash = "sha256:596f3cd67e1b950bc372c33f1a28a0692080625592ea6392987dba7f09f17a94"}, + {file = "cryptography-36.0.1-pp38-pypy38_pp73-macosx_10_10_x86_64.whl", hash = "sha256:30ee1eb3ebe1644d1c3f183d115a8c04e4e603ed6ce8e394ed39eea4a98469ac"}, + {file = "cryptography-36.0.1-pp38-pypy38_pp73-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:ec63da4e7e4a5f924b90af42eddf20b698a70e58d86a72d943857c4c6045b3ee"}, + {file = "cryptography-36.0.1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ca238ceb7ba0bdf6ce88c1b74a87bffcee5afbfa1e41e173b1ceb095b39add46"}, + {file = "cryptography-36.0.1-pp38-pypy38_pp73-manylinux_2_24_x86_64.whl", hash = "sha256:ca28641954f767f9822c24e927ad894d45d5a1e501767599647259cbf030b903"}, + {file = "cryptography-36.0.1-pp38-pypy38_pp73-win_amd64.whl", hash = "sha256:39bdf8e70eee6b1c7b289ec6e5d84d49a6bfa11f8b8646b5b3dfe41219153316"}, + {file = "cryptography-36.0.1.tar.gz", hash = "sha256:53e5c1dc3d7a953de055d77bef2ff607ceef7a2aac0353b5d630ab67f7423638"}, +] +defusedxml = [ + {file = "defusedxml-0.7.1-py2.py3-none-any.whl", hash = "sha256:a352e7e428770286cc899e2542b6cdaedb2b4953ff269a210103ec58f6198a61"}, + {file = "defusedxml-0.7.1.tar.gz", hash = "sha256:1bb3032db185915b62d7c6209c5a8792be6a32ab2fedacc84e01b52c51aa3e69"}, +] +deprecated = [ + {file = "Deprecated-1.2.13-py2.py3-none-any.whl", hash = "sha256:64756e3e14c8c5eea9795d93c524551432a0be75629f8f29e67ab8caf076c76d"}, + {file = "Deprecated-1.2.13.tar.gz", hash = "sha256:43ac5335da90c31c24ba028af536a91d41d53f9e6901ddb021bcc572ce44e38d"}, +] +docutils = [ + {file = "docutils-0.18.1-py2.py3-none-any.whl", hash = "sha256:23010f129180089fbcd3bc08cfefccb3b890b0050e1ca00c867036e9d161b98c"}, + {file = "docutils-0.18.1.tar.gz", hash = "sha256:679987caf361a7539d76e584cbeddc311e3aee937877c87346f31debc63e9d06"}, +] +elementpath = [ + {file = "elementpath-2.5.0-py3-none-any.whl", hash = "sha256:2a432775e37a19e4362443078130a7dbfc457d7d093cd421c03958d9034cc08b"}, + {file = "elementpath-2.5.0.tar.gz", hash = "sha256:3a27aaf3399929fccda013899cb76d3ff111734abf4281e5f9d3721ba0b9ffa3"}, +] +flake8 = [ + {file = "flake8-4.0.1-py2.py3-none-any.whl", hash = "sha256:479b1304f72536a55948cb40a32dce8bb0ffe3501e26eaf292c7e60eb5e0428d"}, + {file = "flake8-4.0.1.tar.gz", hash = "sha256:806e034dda44114815e23c16ef92f95c91e4c71100ff52813adf7132a6ad870d"}, +] +flake8-bugbear = [ + {file = "flake8-bugbear-21.3.2.tar.gz", hash = "sha256:cadce434ceef96463b45a7c3000f23527c04ea4b531d16c7ac8886051f516ca0"}, + {file = "flake8_bugbear-21.3.2-py36.py37.py38-none-any.whl", hash = "sha256:5d6ccb0c0676c738a6e066b4d50589c408dcc1c5bf1d73b464b18b73cd6c05c2"}, +] +flake8-comprehensions = [ + {file = "flake8-comprehensions-3.8.0.tar.gz", hash = "sha256:8e108707637b1d13734f38e03435984f6b7854fa6b5a4e34f93e69534be8e521"}, + {file = "flake8_comprehensions-3.8.0-py3-none-any.whl", hash = "sha256:9406314803abe1193c064544ab14fdc43c58424c0882f6ff8a581eb73fc9bb58"}, +] +frozendict = [ + {file = "frozendict-2.3.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:e18e2abd144a9433b0a8334582843b2aa0d3b9ac8b209aaa912ad365115fe2e1"}, + {file = "frozendict-2.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:96dc7a02e78da5725e5e642269bb7ae792e0c9f13f10f2e02689175ebbfedb35"}, + {file = "frozendict-2.3.0-cp310-cp310-win_amd64.whl", hash = "sha256:752a6dcfaf9bb20a7ecab24980e4dbe041f154509c989207caf185522ef85461"}, + {file = "frozendict-2.3.0-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:5346d9fc1c936c76d33975a9a9f1a067342963105d9a403a99e787c939cc2bb2"}, + {file = "frozendict-2.3.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:60dd2253f1bacb63a7c486ec541a968af4f985ffb06602ee8954a3d39ec6bd2e"}, + {file = "frozendict-2.3.0-cp36-cp36m-win_amd64.whl", hash = "sha256:b2e044602ce17e5cd86724add46660fb9d80169545164e763300a3b839cb1b79"}, + {file = "frozendict-2.3.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:a27a69b1ac3591e4258325108aee62b53c0eeb6ad0a993ae68d3c7eaea980420"}, + {file = "frozendict-2.3.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:4f45ef5f6b184d84744fff97b61f6b9a855e24d36b713ea2352fc723a047afa5"}, + {file = "frozendict-2.3.0-cp37-cp37m-win_amd64.whl", hash = "sha256:2d3f5016650c0e9a192f5024e68fb4d63f670d0ee58b099ed3f5b4c62ea30ecb"}, + {file = "frozendict-2.3.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:6cf605916f50aabaaba5624c81eb270200f6c2c466c46960237a125ec8fe3ae0"}, + {file = "frozendict-2.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f6da06e44904beae4412199d7e49be4f85c6cc168ab06b77c735ea7da5ce3454"}, + {file = "frozendict-2.3.0-cp38-cp38-win_amd64.whl", hash = "sha256:1f34793fb409c4fa70ffd25bea87b01f3bd305fb1c6b09e7dff085b126302206"}, + {file = "frozendict-2.3.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:fd72494a559bdcd28aa71f4aa81860269cd0b7c45fff3e2614a0a053ecfd2a13"}, + {file = "frozendict-2.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:00ea9166aa68cc5feed05986206fdbf35e838a09cb3feef998cf35978ff8a803"}, + {file = "frozendict-2.3.0-cp39-cp39-win_amd64.whl", hash = "sha256:9ffaf440648b44e0bc694c1a4701801941378ba3ba6541e17750ae4b4aeeb116"}, + {file = "frozendict-2.3.0-py3-none-any.whl", hash = "sha256:8578fe06815fcdcc672bd5603eebc98361a5317c1c3a13b28c6c810f6ea3b323"}, + {file = "frozendict-2.3.0.tar.gz", hash = "sha256:da4231adefc5928e7810da2732269d3ad7b5616295b3e693746392a8205ea0b5"}, +] +gitdb = [ + {file = "gitdb-4.0.9-py3-none-any.whl", hash = "sha256:8033ad4e853066ba6ca92050b9df2f89301b8fc8bf7e9324d412a63f8bf1a8fd"}, + {file = "gitdb-4.0.9.tar.gz", hash = "sha256:bac2fd45c0a1c9cf619e63a90d62bdc63892ef92387424b855792a6cabe789aa"}, +] +gitpython = [ + {file = "GitPython-3.1.14-py3-none-any.whl", hash = "sha256:3283ae2fba31c913d857e12e5ba5f9a7772bbc064ae2bb09efafa71b0dd4939b"}, + {file = "GitPython-3.1.14.tar.gz", hash = "sha256:be27633e7509e58391f10207cd32b2a6cf5b908f92d9cd30da2e514e1137af61"}, +] +hiredis = [ + {file = "hiredis-2.0.0-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:b4c8b0bc5841e578d5fb32a16e0c305359b987b850a06964bd5a62739d688048"}, + {file = "hiredis-2.0.0-cp36-cp36m-manylinux1_i686.whl", hash = "sha256:0adea425b764a08270820531ec2218d0508f8ae15a448568109ffcae050fee26"}, + {file = "hiredis-2.0.0-cp36-cp36m-manylinux1_x86_64.whl", hash = "sha256:3d55e36715ff06cdc0ab62f9591607c4324297b6b6ce5b58cb9928b3defe30ea"}, + {file = "hiredis-2.0.0-cp36-cp36m-manylinux2010_i686.whl", hash = "sha256:5d2a48c80cf5a338d58aae3c16872f4d452345e18350143b3bf7216d33ba7b99"}, + {file = "hiredis-2.0.0-cp36-cp36m-manylinux2010_x86_64.whl", hash = "sha256:240ce6dc19835971f38caf94b5738092cb1e641f8150a9ef9251b7825506cb05"}, + {file = "hiredis-2.0.0-cp36-cp36m-manylinux2014_aarch64.whl", hash = "sha256:5dc7a94bb11096bc4bffd41a3c4f2b958257085c01522aa81140c68b8bf1630a"}, + {file = "hiredis-2.0.0-cp36-cp36m-win32.whl", hash = "sha256:139705ce59d94eef2ceae9fd2ad58710b02aee91e7fa0ccb485665ca0ecbec63"}, + {file = "hiredis-2.0.0-cp36-cp36m-win_amd64.whl", hash = "sha256:c39c46d9e44447181cd502a35aad2bb178dbf1b1f86cf4db639d7b9614f837c6"}, + {file = "hiredis-2.0.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:adf4dd19d8875ac147bf926c727215a0faf21490b22c053db464e0bf0deb0485"}, + {file = "hiredis-2.0.0-cp37-cp37m-manylinux1_i686.whl", hash = "sha256:0f41827028901814c709e744060843c77e78a3aca1e0d6875d2562372fcb405a"}, + {file = "hiredis-2.0.0-cp37-cp37m-manylinux1_x86_64.whl", hash = "sha256:508999bec4422e646b05c95c598b64bdbef1edf0d2b715450a078ba21b385bcc"}, + {file = "hiredis-2.0.0-cp37-cp37m-manylinux2010_i686.whl", hash = "sha256:0d5109337e1db373a892fdcf78eb145ffb6bbd66bb51989ec36117b9f7f9b579"}, + {file = "hiredis-2.0.0-cp37-cp37m-manylinux2010_x86_64.whl", hash = "sha256:04026461eae67fdefa1949b7332e488224eac9e8f2b5c58c98b54d29af22093e"}, + {file = "hiredis-2.0.0-cp37-cp37m-manylinux2014_aarch64.whl", hash = "sha256:a00514362df15af041cc06e97aebabf2895e0a7c42c83c21894be12b84402d79"}, + {file = "hiredis-2.0.0-cp37-cp37m-win32.whl", hash = "sha256:09004096e953d7ebd508cded79f6b21e05dff5d7361771f59269425108e703bc"}, + {file = "hiredis-2.0.0-cp37-cp37m-win_amd64.whl", hash = "sha256:f8196f739092a78e4f6b1b2172679ed3343c39c61a3e9d722ce6fcf1dac2824a"}, + {file = "hiredis-2.0.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:294a6697dfa41a8cba4c365dd3715abc54d29a86a40ec6405d677ca853307cfb"}, + {file = "hiredis-2.0.0-cp38-cp38-manylinux1_i686.whl", hash = "sha256:3dddf681284fe16d047d3ad37415b2e9ccdc6c8986c8062dbe51ab9a358b50a5"}, + {file = "hiredis-2.0.0-cp38-cp38-manylinux1_x86_64.whl", hash = "sha256:dcef843f8de4e2ff5e35e96ec2a4abbdf403bd0f732ead127bd27e51f38ac298"}, + {file = "hiredis-2.0.0-cp38-cp38-manylinux2010_i686.whl", hash = "sha256:87c7c10d186f1743a8fd6a971ab6525d60abd5d5d200f31e073cd5e94d7e7a9d"}, + {file = "hiredis-2.0.0-cp38-cp38-manylinux2010_x86_64.whl", hash = "sha256:7f0055f1809b911ab347a25d786deff5e10e9cf083c3c3fd2dd04e8612e8d9db"}, + {file = "hiredis-2.0.0-cp38-cp38-manylinux2014_aarch64.whl", hash = "sha256:11d119507bb54e81f375e638225a2c057dda748f2b1deef05c2b1a5d42686048"}, + {file = "hiredis-2.0.0-cp38-cp38-win32.whl", hash = "sha256:7492af15f71f75ee93d2a618ca53fea8be85e7b625e323315169977fae752426"}, + {file = "hiredis-2.0.0-cp38-cp38-win_amd64.whl", hash = "sha256:65d653df249a2f95673976e4e9dd7ce10de61cfc6e64fa7eeaa6891a9559c581"}, + {file = "hiredis-2.0.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:ae8427a5e9062ba66fc2c62fb19a72276cf12c780e8db2b0956ea909c48acff5"}, + {file = "hiredis-2.0.0-cp39-cp39-manylinux1_i686.whl", hash = "sha256:3f5f7e3a4ab824e3de1e1700f05ad76ee465f5f11f5db61c4b297ec29e692b2e"}, + {file = "hiredis-2.0.0-cp39-cp39-manylinux1_x86_64.whl", hash = "sha256:e3447d9e074abf0e3cd85aef8131e01ab93f9f0e86654db7ac8a3f73c63706ce"}, + {file = "hiredis-2.0.0-cp39-cp39-manylinux2010_i686.whl", hash = "sha256:8b42c0dc927b8d7c0eb59f97e6e34408e53bc489f9f90e66e568f329bff3e443"}, + {file = "hiredis-2.0.0-cp39-cp39-manylinux2010_x86_64.whl", hash = "sha256:b84f29971f0ad4adaee391c6364e6f780d5aae7e9226d41964b26b49376071d0"}, + {file = "hiredis-2.0.0-cp39-cp39-manylinux2014_aarch64.whl", hash = "sha256:0b39ec237459922c6544d071cdcf92cbb5bc6685a30e7c6d985d8a3e3a75326e"}, + {file = "hiredis-2.0.0-cp39-cp39-win32.whl", hash = "sha256:a7928283143a401e72a4fad43ecc85b35c27ae699cf5d54d39e1e72d97460e1d"}, + {file = "hiredis-2.0.0-cp39-cp39-win_amd64.whl", hash = "sha256:a4ee8000454ad4486fb9f28b0cab7fa1cd796fc36d639882d0b34109b5b3aec9"}, + {file = "hiredis-2.0.0-pp36-pypy36_pp73-macosx_10_9_x86_64.whl", hash = "sha256:1f03d4dadd595f7a69a75709bc81902673fa31964c75f93af74feac2f134cc54"}, + {file = "hiredis-2.0.0-pp36-pypy36_pp73-manylinux1_x86_64.whl", hash = "sha256:04927a4c651a0e9ec11c68e4427d917e44ff101f761cd3b5bc76f86aaa431d27"}, + {file = "hiredis-2.0.0-pp36-pypy36_pp73-manylinux2010_x86_64.whl", hash = "sha256:a39efc3ade8c1fb27c097fd112baf09d7fd70b8cb10ef1de4da6efbe066d381d"}, + {file = "hiredis-2.0.0-pp36-pypy36_pp73-win32.whl", hash = "sha256:07bbf9bdcb82239f319b1f09e8ef4bdfaec50ed7d7ea51a56438f39193271163"}, + {file = "hiredis-2.0.0-pp37-pypy37_pp73-macosx_10_9_x86_64.whl", hash = "sha256:807b3096205c7cec861c8803a6738e33ed86c9aae76cac0e19454245a6bbbc0a"}, + {file = "hiredis-2.0.0-pp37-pypy37_pp73-manylinux1_x86_64.whl", hash = "sha256:1233e303645f468e399ec906b6b48ab7cd8391aae2d08daadbb5cad6ace4bd87"}, + {file = "hiredis-2.0.0-pp37-pypy37_pp73-manylinux2010_x86_64.whl", hash = "sha256:cb2126603091902767d96bcb74093bd8b14982f41809f85c9b96e519c7e1dc41"}, + {file = "hiredis-2.0.0-pp37-pypy37_pp73-win32.whl", hash = "sha256:f52010e0a44e3d8530437e7da38d11fb822acfb0d5b12e9cd5ba655509937ca0"}, + {file = "hiredis-2.0.0.tar.gz", hash = "sha256:81d6d8e39695f2c37954d1011c0480ef7cf444d4e3ae24bc5e89ee5de360139a"}, +] +hyperlink = [ + {file = "hyperlink-21.0.0-py2.py3-none-any.whl", hash = "sha256:e6b14c37ecb73e89c77d78cdb4c2cc8f3fb59a885c5b3f819ff4ed80f25af1b4"}, + {file = "hyperlink-21.0.0.tar.gz", hash = "sha256:427af957daa58bc909471c6c40f74c5450fa123dd093fc53efd2e91d2705a56b"}, +] +idna = [ + {file = "idna-3.3-py3-none-any.whl", hash = "sha256:84d9dd047ffa80596e0f246e2eab0b391788b0503584e8945f2368256d2735ff"}, + {file = "idna-3.3.tar.gz", hash = "sha256:9d643ff0a55b762d5cdb124b8eaa99c66322e2157b69160bc32796e824360e6d"}, +] +ijson = [ + {file = "ijson-3.1.4-cp27-cp27m-macosx_10_9_x86_64.whl", hash = "sha256:6c1a777096be5f75ffebb335c6d2ebc0e489b231496b7f2ca903aa061fe7d381"}, + {file = "ijson-3.1.4-cp27-cp27m-manylinux1_i686.whl", hash = "sha256:475fc25c3d2a86230b85777cae9580398b42eed422506bf0b6aacfa936f7bfcd"}, + {file = "ijson-3.1.4-cp27-cp27m-manylinux1_x86_64.whl", hash = "sha256:f587699b5a759e30accf733e37950cc06c4118b72e3e146edcea77dded467426"}, + {file = "ijson-3.1.4-cp27-cp27m-manylinux2010_i686.whl", hash = "sha256:339b2b4c7bbd64849dd69ef94ee21e29dcd92c831f47a281fdd48122bb2a715a"}, + {file = "ijson-3.1.4-cp27-cp27m-manylinux2010_x86_64.whl", hash = "sha256:446ef8980504da0af8d20d3cb6452c4dc3d8aa5fd788098985e899b913191fe6"}, + {file = "ijson-3.1.4-cp27-cp27mu-manylinux1_i686.whl", hash = "sha256:3997a2fdb28bc04b9ab0555db5f3b33ed28d91e9d42a3bf2c1842d4990beb158"}, + {file = "ijson-3.1.4-cp27-cp27mu-manylinux1_x86_64.whl", hash = "sha256:fa10a1d88473303ec97aae23169d77c5b92657b7fb189f9c584974c00a79f383"}, + {file = "ijson-3.1.4-cp27-cp27mu-manylinux2010_i686.whl", hash = "sha256:9a5bf5b9d8f2ceaca131ee21fc7875d0f34b95762f4f32e4d65109ca46472147"}, + {file = "ijson-3.1.4-cp27-cp27mu-manylinux2010_x86_64.whl", hash = "sha256:81cc8cee590c8a70cca3c9aefae06dd7cb8e9f75f3a7dc12b340c2e332d33a2a"}, + {file = "ijson-3.1.4-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:4ea5fc50ba158f72943d5174fbc29ebefe72a2adac051c814c87438dc475cf78"}, + {file = "ijson-3.1.4-cp35-cp35m-macosx_10_9_x86_64.whl", hash = "sha256:3b98861a4280cf09d267986cefa46c3bd80af887eae02aba07488d80eb798afa"}, + {file = "ijson-3.1.4-cp35-cp35m-manylinux1_i686.whl", hash = "sha256:068c692efba9692406b86736dcc6803e4a0b6280d7f0b7534bff3faec677ff38"}, + {file = "ijson-3.1.4-cp35-cp35m-manylinux1_x86_64.whl", hash = "sha256:86884ac06ac69cea6d89ab7b84683b3b4159c4013e4a20276d3fc630fe9b7588"}, + {file = "ijson-3.1.4-cp35-cp35m-manylinux2010_i686.whl", hash = "sha256:41e5886ff6fade26f10b87edad723d2db14dcbb1178717790993fcbbb8ccd333"}, + {file = "ijson-3.1.4-cp35-cp35m-manylinux2010_x86_64.whl", hash = "sha256:24b58933bf777d03dc1caa3006112ec7f9e6f6db6ffe1f5f5bd233cb1281f719"}, + {file = "ijson-3.1.4-cp35-cp35m-manylinux2014_aarch64.whl", hash = "sha256:13f80aad0b84d100fb6a88ced24bade21dc6ddeaf2bba3294b58728463194f50"}, + {file = "ijson-3.1.4-cp35-cp35m-win32.whl", hash = "sha256:fa9a25d0bd32f9515e18a3611690f1de12cb7d1320bd93e9da835936b41ad3ff"}, + {file = "ijson-3.1.4-cp35-cp35m-win_amd64.whl", hash = "sha256:c4c1bf98aaab4c8f60d238edf9bcd07c896cfcc51c2ca84d03da22aad88957c5"}, + {file = "ijson-3.1.4-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:f0f2a87c423e8767368aa055310024fa28727f4454463714fef22230c9717f64"}, + {file = "ijson-3.1.4-cp36-cp36m-manylinux1_i686.whl", hash = "sha256:15507de59d74d21501b2a076d9c49abf927eb58a51a01b8f28a0a0565db0a99f"}, + {file = "ijson-3.1.4-cp36-cp36m-manylinux1_x86_64.whl", hash = "sha256:2e6bd6ad95ab40c858592b905e2bbb4fe79bbff415b69a4923dafe841ffadcb4"}, + {file = "ijson-3.1.4-cp36-cp36m-manylinux2010_i686.whl", hash = "sha256:68e295bb12610d086990cedc89fb8b59b7c85740d66e9515aed062649605d0bf"}, + {file = "ijson-3.1.4-cp36-cp36m-manylinux2010_x86_64.whl", hash = "sha256:3bb461352c0f0f2ec460a4b19400a665b8a5a3a2da663a32093df1699642ee3f"}, + {file = "ijson-3.1.4-cp36-cp36m-manylinux2014_aarch64.whl", hash = "sha256:f91c75edd6cf1a66f02425bafc59a22ec29bc0adcbc06f4bfd694d92f424ceb3"}, + {file = "ijson-3.1.4-cp36-cp36m-win32.whl", hash = "sha256:4c53cc72f79a4c32d5fc22efb85aa22f248e8f4f992707a84bdc896cc0b1ecf9"}, + {file = "ijson-3.1.4-cp36-cp36m-win_amd64.whl", hash = "sha256:ac9098470c1ff6e5c23ec0946818bc102bfeeeea474554c8d081dc934be20988"}, + {file = "ijson-3.1.4-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:dcd6f04df44b1945b859318010234651317db2c4232f75e3933f8bb41c4fa055"}, + {file = "ijson-3.1.4-cp37-cp37m-manylinux1_i686.whl", hash = "sha256:5a2f40c053c837591636dc1afb79d85e90b9a9d65f3d9963aae31d1eb11bfed2"}, + {file = "ijson-3.1.4-cp37-cp37m-manylinux1_x86_64.whl", hash = "sha256:f50337e3b8e72ec68441b573c2848f108a8976a57465c859b227ebd2a2342901"}, + {file = "ijson-3.1.4-cp37-cp37m-manylinux2010_i686.whl", hash = "sha256:454918f908abbed3c50a0a05c14b20658ab711b155e4f890900e6f60746dd7cc"}, + {file = "ijson-3.1.4-cp37-cp37m-manylinux2010_x86_64.whl", hash = "sha256:387c2ec434cc1bc7dc9bd33ec0b70d95d443cc1e5934005f26addc2284a437ab"}, + {file = "ijson-3.1.4-cp37-cp37m-manylinux2014_aarch64.whl", hash = "sha256:179ed6fd42e121d252b43a18833df2de08378fac7bce380974ef6f5e522afefa"}, + {file = "ijson-3.1.4-cp37-cp37m-win32.whl", hash = "sha256:26a6a550b270df04e3f442e2bf0870c9362db4912f0e7bdfd300f30ea43115a2"}, + {file = "ijson-3.1.4-cp37-cp37m-win_amd64.whl", hash = "sha256:ff8cf7507d9d8939264068c2cff0a23f99703fa2f31eb3cb45a9a52798843586"}, + {file = "ijson-3.1.4-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:09c9d7913c88a6059cd054ff854958f34d757402b639cf212ffbec201a705a0d"}, + {file = "ijson-3.1.4-cp38-cp38-manylinux1_i686.whl", hash = "sha256:702ba9a732116d659a5e950ee176be6a2e075998ef1bcde11cbf79a77ed0f717"}, + {file = "ijson-3.1.4-cp38-cp38-manylinux1_x86_64.whl", hash = "sha256:667841591521158770adc90793c2bdbb47c94fe28888cb802104b8bbd61f3d51"}, + {file = "ijson-3.1.4-cp38-cp38-manylinux2010_i686.whl", hash = "sha256:df641dd07b38c63eecd4f454db7b27aa5201193df160f06b48111ba97ab62504"}, + {file = "ijson-3.1.4-cp38-cp38-manylinux2010_x86_64.whl", hash = "sha256:9348e7d507eb40b52b12eecff3d50934fcc3d2a15a2f54ec1127a36063b9ba8f"}, + {file = "ijson-3.1.4-cp38-cp38-manylinux2014_aarch64.whl", hash = "sha256:93455902fdc33ba9485c7fae63ac95d96e0ab8942224a357113174bbeaff92e9"}, + {file = "ijson-3.1.4-cp38-cp38-win32.whl", hash = "sha256:5b725f2e984ce70d464b195f206fa44bebbd744da24139b61fec72de77c03a16"}, + {file = "ijson-3.1.4-cp38-cp38-win_amd64.whl", hash = "sha256:a5965c315fbb2dc9769dfdf046eb07daf48ae20b637da95ec8d62b629be09df4"}, + {file = "ijson-3.1.4-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:b8ee7dbb07cec9ba29d60cfe4954b3cc70adb5f85bba1f72225364b59c1cf82b"}, + {file = "ijson-3.1.4-cp39-cp39-manylinux1_i686.whl", hash = "sha256:d9e01c55d501e9c3d686b6ee3af351c9c0c8c3e45c5576bd5601bee3e1300b09"}, + {file = "ijson-3.1.4-cp39-cp39-manylinux1_x86_64.whl", hash = "sha256:297f26f27a04cd0d0a2f865d154090c48ea11b239cabe0a17a6c65f0314bd1ca"}, + {file = "ijson-3.1.4-cp39-cp39-manylinux2010_i686.whl", hash = "sha256:9239973100338a4138d09d7a4602bd289861e553d597cd67390c33bfc452253e"}, + {file = "ijson-3.1.4-cp39-cp39-manylinux2010_x86_64.whl", hash = "sha256:2a64c66a08f56ed45a805691c2fd2e1caef00edd6ccf4c4e5eff02cd94ad8364"}, + {file = "ijson-3.1.4-cp39-cp39-manylinux2014_aarch64.whl", hash = "sha256:d17fd199f0d0a4ab6e0d541b4eec1b68b5bd5bb5d8104521e22243015b51049b"}, + {file = "ijson-3.1.4-cp39-cp39-win32.whl", hash = "sha256:70ee3c8fa0eba18c80c5911639c01a8de4089a4361bad2862a9949e25ec9b1c8"}, + {file = "ijson-3.1.4-cp39-cp39-win_amd64.whl", hash = "sha256:6bf2b64304321705d03fa5e403ec3f36fa5bb27bf661849ad62e0a3a49bc23e3"}, + {file = "ijson-3.1.4-pp27-pypy_73-macosx_10_9_x86_64.whl", hash = "sha256:5d7e3fcc3b6de76a9dba1e9fc6ca23dad18f0fa6b4e6499415e16b684b2e9af1"}, + {file = "ijson-3.1.4-pp27-pypy_73-manylinux1_x86_64.whl", hash = "sha256:a72eb0359ebff94754f7a2f00a6efe4c57716f860fc040c606dedcb40f49f233"}, + {file = "ijson-3.1.4-pp27-pypy_73-manylinux2010_x86_64.whl", hash = "sha256:28fc168f5faf5759fdfa2a63f85f1f7a148bbae98f34404a6ba19f3d08e89e87"}, + {file = "ijson-3.1.4-pp36-pypy36_pp73-macosx_10_9_x86_64.whl", hash = "sha256:2844d4a38d27583897ed73f7946e205b16926b4cab2525d1ce17e8b08064c706"}, + {file = "ijson-3.1.4-pp36-pypy36_pp73-manylinux1_x86_64.whl", hash = "sha256:252defd1f139b5fb8c764d78d5e3a6df81543d9878c58992a89b261369ea97a7"}, + {file = "ijson-3.1.4-pp36-pypy36_pp73-manylinux2010_x86_64.whl", hash = "sha256:15d5356b4d090c699f382c8eb6a2bcd5992a8c8e8b88c88bc6e54f686018328a"}, + {file = "ijson-3.1.4-pp36-pypy36_pp73-win32.whl", hash = "sha256:6774ec0a39647eea70d35fb76accabe3d71002a8701c0545b9120230c182b75b"}, + {file = "ijson-3.1.4-pp37-pypy37_pp73-macosx_10_9_x86_64.whl", hash = "sha256:f11da15ec04cc83ff0f817a65a3392e169be8d111ba81f24d6e09236597bb28c"}, + {file = "ijson-3.1.4-pp37-pypy37_pp73-manylinux1_x86_64.whl", hash = "sha256:ee13ceeed9b6cf81b3b8197ef15595fc43fd54276842ed63840ddd49db0603da"}, + {file = "ijson-3.1.4-pp37-pypy37_pp73-manylinux2010_x86_64.whl", hash = "sha256:97e4df67235fae40d6195711223520d2c5bf1f7f5087c2963fcde44d72ebf448"}, + {file = "ijson-3.1.4-pp37-pypy37_pp73-win32.whl", hash = "sha256:3d10eee52428f43f7da28763bb79f3d90bbbeea1accb15de01e40a00885b6e89"}, + {file = "ijson-3.1.4.tar.gz", hash = "sha256:1d1003ae3c6115ec9b587d29dd136860a81a23c7626b682e2b5b12c9fd30e4ea"}, +] +importlib-metadata = [ + {file = "importlib_metadata-4.2.0-py3-none-any.whl", hash = "sha256:057e92c15bc8d9e8109738a48db0ccb31b4d9d5cfbee5a8670879a30be66304b"}, + {file = "importlib_metadata-4.2.0.tar.gz", hash = "sha256:b7e52a1f8dec14a75ea73e0891f3060099ca1d8e6a462a4dff11c3e119ea1b31"}, +] +importlib-resources = [ + {file = "importlib_resources-5.4.0-py3-none-any.whl", hash = "sha256:33a95faed5fc19b4bc16b29a6eeae248a3fe69dd55d4d229d2b480e23eeaad45"}, + {file = "importlib_resources-5.4.0.tar.gz", hash = "sha256:d756e2f85dd4de2ba89be0b21dba2a3bbec2e871a42a3a16719258a11f87506b"}, +] +incremental = [ + {file = "incremental-21.3.0-py2.py3-none-any.whl", hash = "sha256:92014aebc6a20b78a8084cdd5645eeaa7f74b8933f70fa3ada2cfbd1e3b54321"}, + {file = "incremental-21.3.0.tar.gz", hash = "sha256:02f5de5aff48f6b9f665d99d48bfc7ec03b6e3943210de7cfc88856d755d6f57"}, +] +isort = [ + {file = "isort-5.7.0-py3-none-any.whl", hash = "sha256:fff4f0c04e1825522ce6949973e83110a6e907750cd92d128b0d14aaaadbffdc"}, + {file = "isort-5.7.0.tar.gz", hash = "sha256:c729845434366216d320e936b8ad6f9d681aab72dc7cbc2d51bedc3582f3ad1e"}, +] +jaeger-client = [ + {file = "jaeger-client-4.8.0.tar.gz", hash = "sha256:3157836edab8e2c209bd2d6ae61113db36f7ee399e66b1dcbb715d87ab49bfe0"}, +] +jeepney = [ + {file = "jeepney-0.7.1-py3-none-any.whl", hash = "sha256:1b5a0ea5c0e7b166b2f5895b91a08c14de8915afda4407fb5022a195224958ac"}, + {file = "jeepney-0.7.1.tar.gz", hash = "sha256:fa9e232dfa0c498bd0b8a3a73b8d8a31978304dcef0515adc859d4e096f96f4f"}, +] +jinja2 = [ + {file = "Jinja2-3.0.3-py3-none-any.whl", hash = "sha256:077ce6014f7b40d03b47d1f1ca4b0fc8328a692bd284016f806ed0eaca390ad8"}, + {file = "Jinja2-3.0.3.tar.gz", hash = "sha256:611bb273cd68f3b993fabdc4064fc858c5b47a973cb5aa7999ec1ba405c87cd7"}, +] +jsonschema = [ + {file = "jsonschema-4.4.0-py3-none-any.whl", hash = "sha256:77281a1f71684953ee8b3d488371b162419767973789272434bbc3f29d9c8823"}, + {file = "jsonschema-4.4.0.tar.gz", hash = "sha256:636694eb41b3535ed608fe04129f26542b59ed99808b4f688aa32dcf55317a83"}, +] +keyring = [ + {file = "keyring-23.5.0-py3-none-any.whl", hash = "sha256:b0d28928ac3ec8e42ef4cc227822647a19f1d544f21f96457965dc01cf555261"}, + {file = "keyring-23.5.0.tar.gz", hash = "sha256:9012508e141a80bd1c0b6778d5c610dd9f8c464d75ac6774248500503f972fb9"}, +] +ldap3 = [ + {file = "ldap3-2.9.1-py2.6.egg", hash = "sha256:5ab7febc00689181375de40c396dcad4f2659cd260fc5e94c508b6d77c17e9d5"}, + {file = "ldap3-2.9.1-py2.7.egg", hash = "sha256:2bc966556fc4d4fa9f445a1c31dc484ee81d44a51ab0e2d0fd05b62cac75daa6"}, + {file = "ldap3-2.9.1-py2.py3-none-any.whl", hash = "sha256:5869596fc4948797020d3f03b7939da938778a0f9e2009f7a072ccf92b8e8d70"}, + {file = "ldap3-2.9.1-py3.9.egg", hash = "sha256:5630d1383e09ba94839e253e013f1aa1a2cf7a547628ba1265cb7b9a844b5687"}, + {file = "ldap3-2.9.1.tar.gz", hash = "sha256:f3e7fc4718e3f09dda568b57100095e0ce58633bcabbed8667ce3f8fbaa4229f"}, +] +lxml = [ + {file = "lxml-4.8.0-cp27-cp27m-macosx_10_14_x86_64.whl", hash = "sha256:e1ab2fac607842ac36864e358c42feb0960ae62c34aa4caaf12ada0a1fb5d99b"}, + {file = "lxml-4.8.0-cp27-cp27m-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:28d1af847786f68bec57961f31221125c29d6f52d9187c01cd34dc14e2b29430"}, + {file = "lxml-4.8.0-cp27-cp27m-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:b92d40121dcbd74831b690a75533da703750f7041b4bf951befc657c37e5695a"}, + {file = "lxml-4.8.0-cp27-cp27m-win32.whl", hash = "sha256:e01f9531ba5420838c801c21c1b0f45dbc9607cb22ea2cf132844453bec863a5"}, + {file = "lxml-4.8.0-cp27-cp27m-win_amd64.whl", hash = "sha256:6259b511b0f2527e6d55ad87acc1c07b3cbffc3d5e050d7e7bcfa151b8202df9"}, + {file = "lxml-4.8.0-cp27-cp27mu-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:1010042bfcac2b2dc6098260a2ed022968dbdfaf285fc65a3acf8e4eb1ffd1bc"}, + {file = "lxml-4.8.0-cp27-cp27mu-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:fa56bb08b3dd8eac3a8c5b7d075c94e74f755fd9d8a04543ae8d37b1612dd170"}, + {file = "lxml-4.8.0-cp310-cp310-macosx_10_15_x86_64.whl", hash = "sha256:31ba2cbc64516dcdd6c24418daa7abff989ddf3ba6d3ea6f6ce6f2ed6e754ec9"}, + {file = "lxml-4.8.0-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_24_i686.whl", hash = "sha256:31499847fc5f73ee17dbe1b8e24c6dafc4e8d5b48803d17d22988976b0171f03"}, + {file = "lxml-4.8.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl", hash = "sha256:5f7d7d9afc7b293147e2d506a4596641d60181a35279ef3aa5778d0d9d9123fe"}, + {file = "lxml-4.8.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl", hash = "sha256:a3c5f1a719aa11866ffc530d54ad965063a8cbbecae6515acbd5f0fae8f48eaa"}, + {file = "lxml-4.8.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:6268e27873a3d191849204d00d03f65c0e343b3bcb518a6eaae05677c95621d1"}, + {file = "lxml-4.8.0-cp310-cp310-win32.whl", hash = "sha256:330bff92c26d4aee79c5bc4d9967858bdbe73fdbdbacb5daf623a03a914fe05b"}, + {file = "lxml-4.8.0-cp310-cp310-win_amd64.whl", hash = "sha256:b2582b238e1658c4061ebe1b4df53c435190d22457642377fd0cb30685cdfb76"}, + {file = "lxml-4.8.0-cp35-cp35m-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:a2bfc7e2a0601b475477c954bf167dee6d0f55cb167e3f3e7cefad906e7759f6"}, + {file = "lxml-4.8.0-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:a1547ff4b8a833511eeaceacbcd17b043214fcdb385148f9c1bc5556ca9623e2"}, + {file = "lxml-4.8.0-cp35-cp35m-win32.whl", hash = "sha256:a9f1c3489736ff8e1c7652e9dc39f80cff820f23624f23d9eab6e122ac99b150"}, + {file = "lxml-4.8.0-cp35-cp35m-win_amd64.whl", hash = "sha256:530f278849031b0eb12f46cca0e5db01cfe5177ab13bd6878c6e739319bae654"}, + {file = "lxml-4.8.0-cp36-cp36m-macosx_10_14_x86_64.whl", hash = "sha256:078306d19a33920004addeb5f4630781aaeabb6a8d01398045fcde085091a169"}, + {file = "lxml-4.8.0-cp36-cp36m-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_24_i686.whl", hash = "sha256:86545e351e879d0b72b620db6a3b96346921fa87b3d366d6c074e5a9a0b8dadb"}, + {file = "lxml-4.8.0-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:24f5c5ae618395ed871b3d8ebfcbb36e3f1091fd847bf54c4de623f9107942f3"}, + {file = "lxml-4.8.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl", hash = "sha256:bbab6faf6568484707acc052f4dfc3802bdb0cafe079383fbaa23f1cdae9ecd4"}, + {file = "lxml-4.8.0-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:7993232bd4044392c47779a3c7e8889fea6883be46281d45a81451acfd704d7e"}, + {file = "lxml-4.8.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:6d6483b1229470e1d8835e52e0ff3c6973b9b97b24cd1c116dca90b57a2cc613"}, + {file = "lxml-4.8.0-cp36-cp36m-musllinux_1_1_x86_64.whl", hash = "sha256:ad4332a532e2d5acb231a2e5d33f943750091ee435daffca3fec0a53224e7e33"}, + {file = "lxml-4.8.0-cp36-cp36m-win32.whl", hash = "sha256:db3535733f59e5605a88a706824dfcb9bd06725e709ecb017e165fc1d6e7d429"}, + {file = "lxml-4.8.0-cp36-cp36m-win_amd64.whl", hash = "sha256:5f148b0c6133fb928503cfcdfdba395010f997aa44bcf6474fcdd0c5398d9b63"}, + {file = "lxml-4.8.0-cp37-cp37m-macosx_10_14_x86_64.whl", hash = "sha256:8a31f24e2a0b6317f33aafbb2f0895c0bce772980ae60c2c640d82caac49628a"}, + {file = "lxml-4.8.0-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_24_i686.whl", hash = "sha256:719544565c2937c21a6f76d520e6e52b726d132815adb3447ccffbe9f44203c4"}, + {file = "lxml-4.8.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl", hash = "sha256:c0b88ed1ae66777a798dc54f627e32d3b81c8009967c63993c450ee4cbcbec15"}, + {file = "lxml-4.8.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl", hash = "sha256:fa9b7c450be85bfc6cd39f6df8c5b8cbd76b5d6fc1f69efec80203f9894b885f"}, + {file = "lxml-4.8.0-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:e9f84ed9f4d50b74fbc77298ee5c870f67cb7e91dcdc1a6915cb1ff6a317476c"}, + {file = "lxml-4.8.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:1d650812b52d98679ed6c6b3b55cbb8fe5a5460a0aef29aeb08dc0b44577df85"}, + {file = "lxml-4.8.0-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:80bbaddf2baab7e6de4bc47405e34948e694a9efe0861c61cdc23aa774fcb141"}, + {file = "lxml-4.8.0-cp37-cp37m-win32.whl", hash = "sha256:6f7b82934c08e28a2d537d870293236b1000d94d0b4583825ab9649aef7ddf63"}, + {file = "lxml-4.8.0-cp37-cp37m-win_amd64.whl", hash = "sha256:e1fd7d2fe11f1cb63d3336d147c852f6d07de0d0020d704c6031b46a30b02ca8"}, + {file = "lxml-4.8.0-cp38-cp38-macosx_10_14_x86_64.whl", hash = "sha256:5045ee1ccd45a89c4daec1160217d363fcd23811e26734688007c26f28c9e9e7"}, + {file = "lxml-4.8.0-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_24_i686.whl", hash = "sha256:0c1978ff1fd81ed9dcbba4f91cf09faf1f8082c9d72eb122e92294716c605428"}, + {file = "lxml-4.8.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl", hash = "sha256:52cbf2ff155b19dc4d4100f7442f6a697938bf4493f8d3b0c51d45568d5666b5"}, + {file = "lxml-4.8.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl", hash = "sha256:ce13d6291a5f47c1c8dbd375baa78551053bc6b5e5c0e9bb8e39c0a8359fd52f"}, + {file = "lxml-4.8.0-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:e11527dc23d5ef44d76fef11213215c34f36af1608074561fcc561d983aeb870"}, + {file = "lxml-4.8.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:60d2f60bd5a2a979df28ab309352cdcf8181bda0cca4529769a945f09aba06f9"}, + {file = "lxml-4.8.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:62f93eac69ec0f4be98d1b96f4d6b964855b8255c345c17ff12c20b93f247b68"}, + {file = "lxml-4.8.0-cp38-cp38-win32.whl", hash = "sha256:20b8a746a026017acf07da39fdb10aa80ad9877046c9182442bf80c84a1c4696"}, + {file = "lxml-4.8.0-cp38-cp38-win_amd64.whl", hash = "sha256:891dc8f522d7059ff0024cd3ae79fd224752676447f9c678f2a5c14b84d9a939"}, + {file = "lxml-4.8.0-cp39-cp39-macosx_10_15_x86_64.whl", hash = "sha256:b6fc2e2fb6f532cf48b5fed57567ef286addcef38c28874458a41b7837a57807"}, + {file = "lxml-4.8.0-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_24_i686.whl", hash = "sha256:74eb65ec61e3c7c019d7169387d1b6ffcfea1b9ec5894d116a9a903636e4a0b1"}, + {file = "lxml-4.8.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl", hash = "sha256:627e79894770783c129cc5e89b947e52aa26e8e0557c7e205368a809da4b7939"}, + {file = "lxml-4.8.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl", hash = "sha256:545bd39c9481f2e3f2727c78c169425efbfb3fbba6e7db4f46a80ebb249819ca"}, + {file = "lxml-4.8.0-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:5a58d0b12f5053e270510bf12f753a76aaf3d74c453c00942ed7d2c804ca845c"}, + {file = "lxml-4.8.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:ec4b4e75fc68da9dc0ed73dcdb431c25c57775383fec325d23a770a64e7ebc87"}, + {file = "lxml-4.8.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:5804e04feb4e61babf3911c2a974a5b86f66ee227cc5006230b00ac6d285b3a9"}, + {file = "lxml-4.8.0-cp39-cp39-win32.whl", hash = "sha256:aa0cf4922da7a3c905d000b35065df6184c0dc1d866dd3b86fd961905bbad2ea"}, + {file = "lxml-4.8.0-cp39-cp39-win_amd64.whl", hash = "sha256:dd10383f1d6b7edf247d0960a3db274c07e96cf3a3fc7c41c8448f93eac3fb1c"}, + {file = "lxml-4.8.0-pp37-pypy37_pp73-macosx_10_14_x86_64.whl", hash = "sha256:2403a6d6fb61c285969b71f4a3527873fe93fd0abe0832d858a17fe68c8fa507"}, + {file = "lxml-4.8.0-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_24_i686.whl", hash = "sha256:986b7a96228c9b4942ec420eff37556c5777bfba6758edcb95421e4a614b57f9"}, + {file = "lxml-4.8.0-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl", hash = "sha256:6fe4ef4402df0250b75ba876c3795510d782def5c1e63890bde02d622570d39e"}, + {file = "lxml-4.8.0-pp38-pypy38_pp73-macosx_10_14_x86_64.whl", hash = "sha256:f10ce66fcdeb3543df51d423ede7e238be98412232fca5daec3e54bcd16b8da0"}, + {file = "lxml-4.8.0-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_24_i686.whl", hash = "sha256:730766072fd5dcb219dd2b95c4c49752a54f00157f322bc6d71f7d2a31fecd79"}, + {file = "lxml-4.8.0-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl", hash = "sha256:8b99ec73073b37f9ebe8caf399001848fced9c08064effdbfc4da2b5a8d07b93"}, + {file = "lxml-4.8.0.tar.gz", hash = "sha256:f63f62fc60e6228a4ca9abae28228f35e1bd3ce675013d1dfb828688d50c6e23"}, +] +markupsafe = [ + {file = "MarkupSafe-2.1.0-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:3028252424c72b2602a323f70fbf50aa80a5d3aa616ea6add4ba21ae9cc9da4c"}, + {file = "MarkupSafe-2.1.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:290b02bab3c9e216da57c1d11d2ba73a9f73a614bbdcc027d299a60cdfabb11a"}, + {file = "MarkupSafe-2.1.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:6e104c0c2b4cd765b4e83909cde7ec61a1e313f8a75775897db321450e928cce"}, + {file = "MarkupSafe-2.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:24c3be29abb6b34052fd26fc7a8e0a49b1ee9d282e3665e8ad09a0a68faee5b3"}, + {file = "MarkupSafe-2.1.0-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:204730fd5fe2fe3b1e9ccadb2bd18ba8712b111dcabce185af0b3b5285a7c989"}, + {file = "MarkupSafe-2.1.0-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:d3b64c65328cb4cd252c94f83e66e3d7acf8891e60ebf588d7b493a55a1dbf26"}, + {file = "MarkupSafe-2.1.0-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:96de1932237abe0a13ba68b63e94113678c379dca45afa040a17b6e1ad7ed076"}, + {file = "MarkupSafe-2.1.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:75bb36f134883fdbe13d8e63b8675f5f12b80bb6627f7714c7d6c5becf22719f"}, + {file = "MarkupSafe-2.1.0-cp310-cp310-win32.whl", hash = "sha256:4056f752015dfa9828dce3140dbadd543b555afb3252507348c493def166d454"}, + {file = "MarkupSafe-2.1.0-cp310-cp310-win_amd64.whl", hash = "sha256:d4e702eea4a2903441f2735799d217f4ac1b55f7d8ad96ab7d4e25417cb0827c"}, + {file = "MarkupSafe-2.1.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:f0eddfcabd6936558ec020130f932d479930581171368fd728efcfb6ef0dd357"}, + {file = "MarkupSafe-2.1.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5ddea4c352a488b5e1069069f2f501006b1a4362cb906bee9a193ef1245a7a61"}, + {file = "MarkupSafe-2.1.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:09c86c9643cceb1d87ca08cdc30160d1b7ab49a8a21564868921959bd16441b8"}, + {file = "MarkupSafe-2.1.0-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:a0a0abef2ca47b33fb615b491ce31b055ef2430de52c5b3fb19a4042dbc5cadb"}, + {file = "MarkupSafe-2.1.0-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:736895a020e31b428b3382a7887bfea96102c529530299f426bf2e636aacec9e"}, + {file = "MarkupSafe-2.1.0-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:679cbb78914ab212c49c67ba2c7396dc599a8479de51b9a87b174700abd9ea49"}, + {file = "MarkupSafe-2.1.0-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:84ad5e29bf8bab3ad70fd707d3c05524862bddc54dc040982b0dbcff36481de7"}, + {file = "MarkupSafe-2.1.0-cp37-cp37m-win32.whl", hash = "sha256:8da5924cb1f9064589767b0f3fc39d03e3d0fb5aa29e0cb21d43106519bd624a"}, + {file = "MarkupSafe-2.1.0-cp37-cp37m-win_amd64.whl", hash = "sha256:454ffc1cbb75227d15667c09f164a0099159da0c1f3d2636aa648f12675491ad"}, + {file = "MarkupSafe-2.1.0-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:142119fb14a1ef6d758912b25c4e803c3ff66920635c44078666fe7cc3f8f759"}, + {file = "MarkupSafe-2.1.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:b2a5a856019d2833c56a3dcac1b80fe795c95f401818ea963594b345929dffa7"}, + {file = "MarkupSafe-2.1.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1d1fb9b2eec3c9714dd936860850300b51dbaa37404209c8d4cb66547884b7ed"}, + {file = "MarkupSafe-2.1.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:62c0285e91414f5c8f621a17b69fc0088394ccdaa961ef469e833dbff64bd5ea"}, + {file = "MarkupSafe-2.1.0-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:fc3150f85e2dbcf99e65238c842d1cfe69d3e7649b19864c1cc043213d9cd730"}, + {file = "MarkupSafe-2.1.0-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:f02cf7221d5cd915d7fa58ab64f7ee6dd0f6cddbb48683debf5d04ae9b1c2cc1"}, + {file = "MarkupSafe-2.1.0-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:d5653619b3eb5cbd35bfba3c12d575db2a74d15e0e1c08bf1db788069d410ce8"}, + {file = "MarkupSafe-2.1.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:7d2f5d97fcbd004c03df8d8fe2b973fe2b14e7bfeb2cfa012eaa8759ce9a762f"}, + {file = "MarkupSafe-2.1.0-cp38-cp38-win32.whl", hash = "sha256:3cace1837bc84e63b3fd2dfce37f08f8c18aeb81ef5cf6bb9b51f625cb4e6cd8"}, + {file = "MarkupSafe-2.1.0-cp38-cp38-win_amd64.whl", hash = "sha256:fabbe18087c3d33c5824cb145ffca52eccd053061df1d79d4b66dafa5ad2a5ea"}, + {file = "MarkupSafe-2.1.0-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:023af8c54fe63530545f70dd2a2a7eed18d07a9a77b94e8bf1e2ff7f252db9a3"}, + {file = "MarkupSafe-2.1.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:d66624f04de4af8bbf1c7f21cc06649c1c69a7f84109179add573ce35e46d448"}, + {file = "MarkupSafe-2.1.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c532d5ab79be0199fa2658e24a02fce8542df196e60665dd322409a03db6a52c"}, + {file = "MarkupSafe-2.1.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e67ec74fada3841b8c5f4c4f197bea916025cb9aa3fe5abf7d52b655d042f956"}, + {file = "MarkupSafe-2.1.0-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:30c653fde75a6e5eb814d2a0a89378f83d1d3f502ab710904ee585c38888816c"}, + {file = "MarkupSafe-2.1.0-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:961eb86e5be7d0973789f30ebcf6caab60b844203f4396ece27310295a6082c7"}, + {file = "MarkupSafe-2.1.0-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:598b65d74615c021423bd45c2bc5e9b59539c875a9bdb7e5f2a6b92dfcfc268d"}, + {file = "MarkupSafe-2.1.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:599941da468f2cf22bf90a84f6e2a65524e87be2fce844f96f2dd9a6c9d1e635"}, + {file = "MarkupSafe-2.1.0-cp39-cp39-win32.whl", hash = "sha256:e6f7f3f41faffaea6596da86ecc2389672fa949bd035251eab26dc6697451d05"}, + {file = "MarkupSafe-2.1.0-cp39-cp39-win_amd64.whl", hash = "sha256:b8811d48078d1cf2a6863dafb896e68406c5f513048451cd2ded0473133473c7"}, + {file = "MarkupSafe-2.1.0.tar.gz", hash = "sha256:80beaf63ddfbc64a0452b841d8036ca0611e049650e20afcb882f5d3c266d65f"}, +] +matrix-common = [ + {file = "matrix_common-1.1.0-py3-none-any.whl", hash = "sha256:5d6dfd777503b2f3a031b566e6af25b6e95f9c0818ef57d954c3190fce5eb407"}, + {file = "matrix_common-1.1.0.tar.gz", hash = "sha256:a8238748afc2b37079818367fed5156f355771b07c8ff0a175934f47e0ff3276"}, +] +matrix-synapse-ldap3 = [ + {file = "matrix-synapse-ldap3-0.1.5.tar.gz", hash = "sha256:9fdf8df7c8ec756642aa0fea53b31c0b2f1924f70d7f049a2090b523125456fe"}, +] +mccabe = [ + {file = "mccabe-0.6.1-py2.py3-none-any.whl", hash = "sha256:ab8a6258860da4b6677da4bd2fe5dc2c659cff31b3ee4f7f5d64e79735b80d42"}, + {file = "mccabe-0.6.1.tar.gz", hash = "sha256:dd8d182285a0fe56bace7f45b5e7d1a6ebcbf524e8f3bd87eb0f125271b8831f"}, +] +msgpack = [ + {file = "msgpack-1.0.3-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:96acc674bb9c9be63fa8b6dabc3248fdc575c4adc005c440ad02f87ca7edd079"}, + {file = "msgpack-1.0.3-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:2c3ca57c96c8e69c1a0d2926a6acf2d9a522b41dc4253a8945c4c6cd4981a4e3"}, + {file = "msgpack-1.0.3-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b0a792c091bac433dfe0a70ac17fc2087d4595ab835b47b89defc8bbabcf5c73"}, + {file = "msgpack-1.0.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1c58cdec1cb5fcea8c2f1771d7b5fec79307d056874f746690bd2bdd609ab147"}, + {file = "msgpack-1.0.3-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:2f97c0f35b3b096a330bb4a1a9247d0bd7e1f3a2eba7ab69795501504b1c2c39"}, + {file = "msgpack-1.0.3-cp310-cp310-win32.whl", hash = "sha256:36a64a10b16c2ab31dcd5f32d9787ed41fe68ab23dd66957ca2826c7f10d0b85"}, + {file = "msgpack-1.0.3-cp310-cp310-win_amd64.whl", hash = "sha256:c1ba333b4024c17c7591f0f372e2daa3c31db495a9b2af3cf664aef3c14354f7"}, + {file = "msgpack-1.0.3-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:c2140cf7a3ec475ef0938edb6eb363fa704159e0bf71dde15d953bacc1cf9d7d"}, + {file = "msgpack-1.0.3-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:6f4c22717c74d44bcd7af353024ce71c6b55346dad5e2cc1ddc17ce8c4507c6b"}, + {file = "msgpack-1.0.3-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:47d733a15ade190540c703de209ffbc42a3367600421b62ac0c09fde594da6ec"}, + {file = "msgpack-1.0.3-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c7e03b06f2982aa98d4ddd082a210c3db200471da523f9ac197f2828e80e7770"}, + {file = "msgpack-1.0.3-cp36-cp36m-win32.whl", hash = "sha256:3d875631ecab42f65f9dce6f55ce6d736696ced240f2634633188de2f5f21af9"}, + {file = "msgpack-1.0.3-cp36-cp36m-win_amd64.whl", hash = "sha256:40fb89b4625d12d6027a19f4df18a4de5c64f6f3314325049f219683e07e678a"}, + {file = "msgpack-1.0.3-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:6eef0cf8db3857b2b556213d97dd82de76e28a6524853a9beb3264983391dc1a"}, + {file = "msgpack-1.0.3-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0d8c332f53ffff01953ad25131272506500b14750c1d0ce8614b17d098252fbc"}, + {file = "msgpack-1.0.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9c0903bd93cbd34653dd63bbfcb99d7539c372795201f39d16fdfde4418de43a"}, + {file = "msgpack-1.0.3-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:bf1e6bfed4860d72106f4e0a1ab519546982b45689937b40257cfd820650b920"}, + {file = "msgpack-1.0.3-cp37-cp37m-win32.whl", hash = "sha256:d02cea2252abc3756b2ac31f781f7a98e89ff9759b2e7450a1c7a0d13302ff50"}, + {file = "msgpack-1.0.3-cp37-cp37m-win_amd64.whl", hash = "sha256:2f30dd0dc4dfe6231ad253b6f9f7128ac3202ae49edd3f10d311adc358772dba"}, + {file = "msgpack-1.0.3-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:f201d34dc89342fabb2a10ed7c9a9aaaed9b7af0f16a5923f1ae562b31258dea"}, + {file = "msgpack-1.0.3-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:bb87f23ae7d14b7b3c21009c4b1705ec107cb21ee71975992f6aca571fb4a42a"}, + {file = "msgpack-1.0.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8a3a5c4b16e9d0edb823fe54b59b5660cc8d4782d7bf2c214cb4b91a1940a8ef"}, + {file = "msgpack-1.0.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f74da1e5fcf20ade12c6bf1baa17a2dc3604958922de8dc83cbe3eff22e8b611"}, + {file = "msgpack-1.0.3-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:73a80bd6eb6bcb338c1ec0da273f87420829c266379c8c82fa14c23fb586cfa1"}, + {file = "msgpack-1.0.3-cp38-cp38-win32.whl", hash = "sha256:9fce00156e79af37bb6db4e7587b30d11e7ac6a02cb5bac387f023808cd7d7f4"}, + {file = "msgpack-1.0.3-cp38-cp38-win_amd64.whl", hash = "sha256:9b6f2d714c506e79cbead331de9aae6837c8dd36190d02da74cb409b36162e8a"}, + {file = "msgpack-1.0.3-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:89908aea5f46ee1474cc37fbc146677f8529ac99201bc2faf4ef8edc023c2bf3"}, + {file = "msgpack-1.0.3-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:973ad69fd7e31159eae8f580f3f707b718b61141838321c6fa4d891c4a2cca52"}, + {file = "msgpack-1.0.3-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:da24375ab4c50e5b7486c115a3198d207954fe10aaa5708f7b65105df09109b2"}, + {file = "msgpack-1.0.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a598d0685e4ae07a0672b59792d2cc767d09d7a7f39fd9bd37ff84e060b1a996"}, + {file = "msgpack-1.0.3-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:e4c309a68cb5d6bbd0c50d5c71a25ae81f268c2dc675c6f4ea8ab2feec2ac4e2"}, + {file = "msgpack-1.0.3-cp39-cp39-win32.whl", hash = "sha256:494471d65b25a8751d19c83f1a482fd411d7ca7a3b9e17d25980a74075ba0e88"}, + {file = "msgpack-1.0.3-cp39-cp39-win_amd64.whl", hash = "sha256:f01b26c2290cbd74316990ba84a14ac3d599af9cebefc543d241a66e785cf17d"}, + {file = "msgpack-1.0.3.tar.gz", hash = "sha256:51fdc7fb93615286428ee7758cecc2f374d5ff363bdd884c7ea622a7a327a81e"}, +] +mypy = [ + {file = "mypy-0.931-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:3c5b42d0815e15518b1f0990cff7a705805961613e701db60387e6fb663fe78a"}, + {file = "mypy-0.931-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:c89702cac5b302f0c5d33b172d2b55b5df2bede3344a2fbed99ff96bddb2cf00"}, + {file = "mypy-0.931-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:300717a07ad09525401a508ef5d105e6b56646f7942eb92715a1c8d610149714"}, + {file = "mypy-0.931-cp310-cp310-win_amd64.whl", hash = "sha256:7b3f6f557ba4afc7f2ce6d3215d5db279bcf120b3cfd0add20a5d4f4abdae5bc"}, + {file = "mypy-0.931-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:1bf752559797c897cdd2c65f7b60c2b6969ffe458417b8d947b8340cc9cec08d"}, + {file = "mypy-0.931-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:4365c60266b95a3f216a3047f1d8e3f895da6c7402e9e1ddfab96393122cc58d"}, + {file = "mypy-0.931-cp36-cp36m-win_amd64.whl", hash = "sha256:1b65714dc296a7991000b6ee59a35b3f550e0073411ac9d3202f6516621ba66c"}, + {file = "mypy-0.931-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:e839191b8da5b4e5d805f940537efcaa13ea5dd98418f06dc585d2891d228cf0"}, + {file = "mypy-0.931-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:50c7346a46dc76a4ed88f3277d4959de8a2bd0a0fa47fa87a4cde36fe247ac05"}, + {file = "mypy-0.931-cp37-cp37m-win_amd64.whl", hash = "sha256:d8f1ff62f7a879c9fe5917b3f9eb93a79b78aad47b533911b853a757223f72e7"}, + {file = "mypy-0.931-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:f9fe20d0872b26c4bba1c1be02c5340de1019530302cf2dcc85c7f9fc3252ae0"}, + {file = "mypy-0.931-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:1b06268df7eb53a8feea99cbfff77a6e2b205e70bf31743e786678ef87ee8069"}, + {file = "mypy-0.931-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:8c11003aaeaf7cc2d0f1bc101c1cc9454ec4cc9cb825aef3cafff8a5fdf4c799"}, + {file = "mypy-0.931-cp38-cp38-win_amd64.whl", hash = "sha256:d9d2b84b2007cea426e327d2483238f040c49405a6bf4074f605f0156c91a47a"}, + {file = "mypy-0.931-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:ff3bf387c14c805ab1388185dd22d6b210824e164d4bb324b195ff34e322d166"}, + {file = "mypy-0.931-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:5b56154f8c09427bae082b32275a21f500b24d93c88d69a5e82f3978018a0266"}, + {file = "mypy-0.931-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:8ca7f8c4b1584d63c9a0f827c37ba7a47226c19a23a753d52e5b5eddb201afcd"}, + {file = "mypy-0.931-cp39-cp39-win_amd64.whl", hash = "sha256:74f7eccbfd436abe9c352ad9fb65872cc0f1f0a868e9d9c44db0893440f0c697"}, + {file = "mypy-0.931-py3-none-any.whl", hash = "sha256:1171f2e0859cfff2d366da2c7092b06130f232c636a3f7301e3feb8b41f6377d"}, + {file = "mypy-0.931.tar.gz", hash = "sha256:0038b21890867793581e4cb0d810829f5fd4441aa75796b53033af3aa30430ce"}, +] +mypy-extensions = [ + {file = "mypy_extensions-0.4.3-py2.py3-none-any.whl", hash = "sha256:090fedd75945a69ae91ce1303b5824f428daf5a028d2f6ab8a299250a846f15d"}, + {file = "mypy_extensions-0.4.3.tar.gz", hash = "sha256:2d82818f5bb3e369420cb3c4060a7970edba416647068eb4c5343488a6c604a8"}, +] +mypy-zope = [ + {file = "mypy-zope-0.3.5.tar.gz", hash = "sha256:489e7da1c2af887f2cfe3496995fc247f296512b495b57817edddda9d22308f3"}, + {file = "mypy_zope-0.3.5-py3-none-any.whl", hash = "sha256:3bd0cc9a3e5933b02931af4b214ba32a4f4ff98adb30c979ce733857db91a18b"}, +] +netaddr = [ + {file = "netaddr-0.8.0-py2.py3-none-any.whl", hash = "sha256:9666d0232c32d2656e5e5f8d735f58fd6c7457ce52fc21c98d45f2af78f990ac"}, + {file = "netaddr-0.8.0.tar.gz", hash = "sha256:d6cc57c7a07b1d9d2e917aa8b36ae8ce61c35ba3fcd1b83ca31c5a0ee2b5a243"}, +] +opentracing = [ + {file = "opentracing-2.4.0.tar.gz", hash = "sha256:a173117e6ef580d55874734d1fa7ecb6f3655160b8b8974a2a1e98e5ec9c840d"}, +] +packaging = [ + {file = "packaging-21.3-py3-none-any.whl", hash = "sha256:ef103e05f519cdc783ae24ea4e2e0f508a9c99b2d4969652eed6a2e1ea5bd522"}, + {file = "packaging-21.3.tar.gz", hash = "sha256:dd47c42927d89ab911e606518907cc2d3a1f38bbd026385970643f9c5b8ecfeb"}, +] +parameterized = [ + {file = "parameterized-0.8.1-py2.py3-none-any.whl", hash = "sha256:9cbb0b69a03e8695d68b3399a8a5825200976536fe1cb79db60ed6a4c8c9efe9"}, + {file = "parameterized-0.8.1.tar.gz", hash = "sha256:41bbff37d6186430f77f900d777e5bb6a24928a1c46fb1de692f8b52b8833b5c"}, +] +pathspec = [ + {file = "pathspec-0.9.0-py2.py3-none-any.whl", hash = "sha256:7d15c4ddb0b5c802d161efc417ec1a2558ea2653c2e8ad9c19098201dc1c993a"}, + {file = "pathspec-0.9.0.tar.gz", hash = "sha256:e564499435a2673d586f6b2130bb5b95f04a3ba06f81b8f895b651a3c76aabb1"}, +] +phonenumbers = [ + {file = "phonenumbers-8.12.44-py2.py3-none-any.whl", hash = "sha256:cc1299cf37b309ecab6214297663ab86cb3d64ae37fd5b88e904fe7983a874a6"}, + {file = "phonenumbers-8.12.44.tar.gz", hash = "sha256:26cfd0257d1704fe2f88caff2caabb70d16a877b1e65b6aae51f9fbbe10aa8ce"}, +] +pillow = [ + {file = "Pillow-9.0.1-1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:a5d24e1d674dd9d72c66ad3ea9131322819ff86250b30dc5821cbafcfa0b96b4"}, + {file = "Pillow-9.0.1-1-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:2632d0f846b7c7600edf53c48f8f9f1e13e62f66a6dbc15191029d950bfed976"}, + {file = "Pillow-9.0.1-1-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:b9618823bd237c0d2575283f2939655f54d51b4527ec3972907a927acbcc5bfc"}, + {file = "Pillow-9.0.1-cp310-cp310-macosx_10_10_universal2.whl", hash = "sha256:9bfdb82cdfeccec50aad441afc332faf8606dfa5e8efd18a6692b5d6e79f00fd"}, + {file = "Pillow-9.0.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:5100b45a4638e3c00e4d2320d3193bdabb2d75e79793af7c3eb139e4f569f16f"}, + {file = "Pillow-9.0.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:528a2a692c65dd5cafc130de286030af251d2ee0483a5bf50c9348aefe834e8a"}, + {file = "Pillow-9.0.1-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0f29d831e2151e0b7b39981756d201f7108d3d215896212ffe2e992d06bfe049"}, + {file = "Pillow-9.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:855c583f268edde09474b081e3ddcd5cf3b20c12f26e0d434e1386cc5d318e7a"}, + {file = "Pillow-9.0.1-cp310-cp310-win32.whl", hash = "sha256:d9d7942b624b04b895cb95af03a23407f17646815495ce4547f0e60e0b06f58e"}, + {file = "Pillow-9.0.1-cp310-cp310-win_amd64.whl", hash = "sha256:81c4b81611e3a3cb30e59b0cf05b888c675f97e3adb2c8672c3154047980726b"}, + {file = "Pillow-9.0.1-cp37-cp37m-macosx_10_10_x86_64.whl", hash = "sha256:413ce0bbf9fc6278b2d63309dfeefe452835e1c78398efb431bab0672fe9274e"}, + {file = "Pillow-9.0.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:80fe64a6deb6fcfdf7b8386f2cf216d329be6f2781f7d90304351811fb591360"}, + {file = "Pillow-9.0.1-cp37-cp37m-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:cef9c85ccbe9bee00909758936ea841ef12035296c748aaceee535969e27d31b"}, + {file = "Pillow-9.0.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1d19397351f73a88904ad1aee421e800fe4bbcd1aeee6435fb62d0a05ccd1030"}, + {file = "Pillow-9.0.1-cp37-cp37m-win32.whl", hash = "sha256:d21237d0cd37acded35154e29aec853e945950321dd2ffd1a7d86fe686814669"}, + {file = "Pillow-9.0.1-cp37-cp37m-win_amd64.whl", hash = "sha256:ede5af4a2702444a832a800b8eb7f0a7a1c0eed55b644642e049c98d589e5092"}, + {file = "Pillow-9.0.1-cp38-cp38-macosx_10_10_x86_64.whl", hash = "sha256:b5b3f092fe345c03bca1e0b687dfbb39364b21ebb8ba90e3fa707374b7915204"}, + {file = "Pillow-9.0.1-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:335ace1a22325395c4ea88e00ba3dc89ca029bd66bd5a3c382d53e44f0ccd77e"}, + {file = "Pillow-9.0.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:db6d9fac65bd08cea7f3540b899977c6dee9edad959fa4eaf305940d9cbd861c"}, + {file = "Pillow-9.0.1-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f154d173286a5d1863637a7dcd8c3437bb557520b01bddb0be0258dcb72696b5"}, + {file = "Pillow-9.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:14d4b1341ac07ae07eb2cc682f459bec932a380c3b122f5540432d8977e64eae"}, + {file = "Pillow-9.0.1-cp38-cp38-win32.whl", hash = "sha256:effb7749713d5317478bb3acb3f81d9d7c7f86726d41c1facca068a04cf5bb4c"}, + {file = "Pillow-9.0.1-cp38-cp38-win_amd64.whl", hash = "sha256:7f7609a718b177bf171ac93cea9fd2ddc0e03e84d8fa4e887bdfc39671d46b00"}, + {file = "Pillow-9.0.1-cp39-cp39-macosx_10_10_x86_64.whl", hash = "sha256:80ca33961ced9c63358056bd08403ff866512038883e74f3a4bf88ad3eb66838"}, + {file = "Pillow-9.0.1-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:1c3c33ac69cf059bbb9d1a71eeaba76781b450bc307e2291f8a4764d779a6b28"}, + {file = "Pillow-9.0.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:12875d118f21cf35604176872447cdb57b07126750a33748bac15e77f90f1f9c"}, + {file = "Pillow-9.0.1-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:514ceac913076feefbeaf89771fd6febde78b0c4c1b23aaeab082c41c694e81b"}, + {file = "Pillow-9.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d3c5c79ab7dfce6d88f1ba639b77e77a17ea33a01b07b99840d6ed08031cb2a7"}, + {file = "Pillow-9.0.1-cp39-cp39-win32.whl", hash = "sha256:718856856ba31f14f13ba885ff13874be7fefc53984d2832458f12c38205f7f7"}, + {file = "Pillow-9.0.1-cp39-cp39-win_amd64.whl", hash = "sha256:f25ed6e28ddf50de7e7ea99d7a976d6a9c415f03adcaac9c41ff6ff41b6d86ac"}, + {file = "Pillow-9.0.1-pp37-pypy37_pp73-macosx_10_10_x86_64.whl", hash = "sha256:011233e0c42a4a7836498e98c1acf5e744c96a67dd5032a6f666cc1fb97eab97"}, + {file = "Pillow-9.0.1-pp37-pypy37_pp73-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:253e8a302a96df6927310a9d44e6103055e8fb96a6822f8b7f514bb7ef77de56"}, + {file = "Pillow-9.0.1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:6295f6763749b89c994fcb6d8a7f7ce03c3992e695f89f00b741b4580b199b7e"}, + {file = "Pillow-9.0.1-pp38-pypy38_pp73-win_amd64.whl", hash = "sha256:a9f44cd7e162ac6191491d7249cceb02b8116b0f7e847ee33f739d7cb1ea1f70"}, + {file = "Pillow-9.0.1.tar.gz", hash = "sha256:6c8bc8238a7dfdaf7a75f5ec5a663f4173f8c367e5a39f87e720495e1eed75fa"}, +] +pkginfo = [ + {file = "pkginfo-1.8.2-py2.py3-none-any.whl", hash = "sha256:c24c487c6a7f72c66e816ab1796b96ac6c3d14d49338293d2141664330b55ffc"}, + {file = "pkginfo-1.8.2.tar.gz", hash = "sha256:542e0d0b6750e2e21c20179803e40ab50598d8066d51097a0e382cba9eb02bff"}, +] +platformdirs = [ + {file = "platformdirs-2.5.1-py3-none-any.whl", hash = "sha256:bcae7cab893c2d310a711b70b24efb93334febe65f8de776ee320b517471e227"}, + {file = "platformdirs-2.5.1.tar.gz", hash = "sha256:7535e70dfa32e84d4b34996ea99c5e432fa29a708d0f4e394bbcb2a8faa4f16d"}, +] +prometheus-client = [ + {file = "prometheus_client-0.13.1-py3-none-any.whl", hash = "sha256:357a447fd2359b0a1d2e9b311a0c5778c330cfbe186d880ad5a6b39884652316"}, + {file = "prometheus_client-0.13.1.tar.gz", hash = "sha256:ada41b891b79fca5638bd5cfe149efa86512eaa55987893becd2c6d8d0a5dfc5"}, +] +psycopg2 = [ + {file = "psycopg2-2.9.3-cp310-cp310-win32.whl", hash = "sha256:083707a696e5e1c330af2508d8fab36f9700b26621ccbcb538abe22e15485362"}, + {file = "psycopg2-2.9.3-cp310-cp310-win_amd64.whl", hash = "sha256:d3ca6421b942f60c008f81a3541e8faf6865a28d5a9b48544b0ee4f40cac7fca"}, + {file = "psycopg2-2.9.3-cp36-cp36m-win32.whl", hash = "sha256:9572e08b50aed176ef6d66f15a21d823bb6f6d23152d35e8451d7d2d18fdac56"}, + {file = "psycopg2-2.9.3-cp36-cp36m-win_amd64.whl", hash = "sha256:a81e3866f99382dfe8c15a151f1ca5fde5815fde879348fe5a9884a7c092a305"}, + {file = "psycopg2-2.9.3-cp37-cp37m-win32.whl", hash = "sha256:cb10d44e6694d763fa1078a26f7f6137d69f555a78ec85dc2ef716c37447e4b2"}, + {file = "psycopg2-2.9.3-cp37-cp37m-win_amd64.whl", hash = "sha256:4295093a6ae3434d33ec6baab4ca5512a5082cc43c0505293087b8a46d108461"}, + {file = "psycopg2-2.9.3-cp38-cp38-win32.whl", hash = "sha256:34b33e0162cfcaad151f249c2649fd1030010c16f4bbc40a604c1cb77173dcf7"}, + {file = "psycopg2-2.9.3-cp38-cp38-win_amd64.whl", hash = "sha256:0762c27d018edbcb2d34d51596e4346c983bd27c330218c56c4dc25ef7e819bf"}, + {file = "psycopg2-2.9.3-cp39-cp39-win32.whl", hash = "sha256:8cf3878353cc04b053822896bc4922b194792df9df2f1ad8da01fb3043602126"}, + {file = "psycopg2-2.9.3-cp39-cp39-win_amd64.whl", hash = "sha256:06f32425949bd5fe8f625c49f17ebb9784e1e4fe928b7cce72edc36fb68e4c0c"}, + {file = "psycopg2-2.9.3.tar.gz", hash = "sha256:8e841d1bf3434da985cc5ef13e6f75c8981ced601fd70cc6bf33351b91562981"}, +] +psycopg2cffi = [ + {file = "psycopg2cffi-2.9.0.tar.gz", hash = "sha256:7e272edcd837de3a1d12b62185eb85c45a19feda9e62fa1b120c54f9e8d35c52"}, +] +psycopg2cffi-compat = [ + {file = "psycopg2cffi-compat-1.1.tar.gz", hash = "sha256:d25e921748475522b33d13420aad5c2831c743227dc1f1f2585e0fdb5c914e05"}, +] +pyasn1 = [ + {file = "pyasn1-0.4.8-py2.4.egg", hash = "sha256:fec3e9d8e36808a28efb59b489e4528c10ad0f480e57dcc32b4de5c9d8c9fdf3"}, + {file = "pyasn1-0.4.8-py2.5.egg", hash = "sha256:0458773cfe65b153891ac249bcf1b5f8f320b7c2ce462151f8fa74de8934becf"}, + {file = "pyasn1-0.4.8-py2.6.egg", hash = "sha256:5c9414dcfede6e441f7e8f81b43b34e834731003427e5b09e4e00e3172a10f00"}, + {file = "pyasn1-0.4.8-py2.7.egg", hash = "sha256:6e7545f1a61025a4e58bb336952c5061697da694db1cae97b116e9c46abcf7c8"}, + {file = "pyasn1-0.4.8-py2.py3-none-any.whl", hash = "sha256:39c7e2ec30515947ff4e87fb6f456dfc6e84857d34be479c9d4a4ba4bf46aa5d"}, + {file = "pyasn1-0.4.8-py3.1.egg", hash = "sha256:78fa6da68ed2727915c4767bb386ab32cdba863caa7dbe473eaae45f9959da86"}, + {file = "pyasn1-0.4.8-py3.2.egg", hash = "sha256:08c3c53b75eaa48d71cf8c710312316392ed40899cb34710d092e96745a358b7"}, + {file = "pyasn1-0.4.8-py3.3.egg", hash = "sha256:03840c999ba71680a131cfaee6fab142e1ed9bbd9c693e285cc6aca0d555e576"}, + {file = "pyasn1-0.4.8-py3.4.egg", hash = "sha256:7ab8a544af125fb704feadb008c99a88805126fb525280b2270bb25cc1d78a12"}, + {file = "pyasn1-0.4.8-py3.5.egg", hash = "sha256:e89bf84b5437b532b0803ba5c9a5e054d21fec423a89952a74f87fa2c9b7bce2"}, + {file = "pyasn1-0.4.8-py3.6.egg", hash = "sha256:014c0e9976956a08139dc0712ae195324a75e142284d5f87f1a87ee1b068a359"}, + {file = "pyasn1-0.4.8-py3.7.egg", hash = "sha256:99fcc3c8d804d1bc6d9a099921e39d827026409a58f2a720dcdb89374ea0c776"}, + {file = "pyasn1-0.4.8.tar.gz", hash = "sha256:aef77c9fb94a3ac588e87841208bdec464471d9871bd5050a287cc9a475cd0ba"}, +] +pyasn1-modules = [ + {file = "pyasn1-modules-0.2.8.tar.gz", hash = "sha256:905f84c712230b2c592c19470d3ca8d552de726050d1d1716282a1f6146be65e"}, + {file = "pyasn1_modules-0.2.8-py2.4.egg", hash = "sha256:0fe1b68d1e486a1ed5473f1302bd991c1611d319bba158e98b106ff86e1d7199"}, + {file = "pyasn1_modules-0.2.8-py2.5.egg", hash = "sha256:fe0644d9ab041506b62782e92b06b8c68cca799e1a9636ec398675459e031405"}, + {file = "pyasn1_modules-0.2.8-py2.6.egg", hash = "sha256:a99324196732f53093a84c4369c996713eb8c89d360a496b599fb1a9c47fc3eb"}, + {file = "pyasn1_modules-0.2.8-py2.7.egg", hash = "sha256:0845a5582f6a02bb3e1bde9ecfc4bfcae6ec3210dd270522fee602365430c3f8"}, + {file = "pyasn1_modules-0.2.8-py2.py3-none-any.whl", hash = "sha256:a50b808ffeb97cb3601dd25981f6b016cbb3d31fbf57a8b8a87428e6158d0c74"}, + {file = "pyasn1_modules-0.2.8-py3.1.egg", hash = "sha256:f39edd8c4ecaa4556e989147ebf219227e2cd2e8a43c7e7fcb1f1c18c5fd6a3d"}, + {file = "pyasn1_modules-0.2.8-py3.2.egg", hash = "sha256:b80486a6c77252ea3a3e9b1e360bc9cf28eaac41263d173c032581ad2f20fe45"}, + {file = "pyasn1_modules-0.2.8-py3.3.egg", hash = "sha256:65cebbaffc913f4fe9e4808735c95ea22d7a7775646ab690518c056784bc21b4"}, + {file = "pyasn1_modules-0.2.8-py3.4.egg", hash = "sha256:15b7c67fabc7fc240d87fb9aabf999cf82311a6d6fb2c70d00d3d0604878c811"}, + {file = "pyasn1_modules-0.2.8-py3.5.egg", hash = "sha256:426edb7a5e8879f1ec54a1864f16b882c2837bfd06eee62f2c982315ee2473ed"}, + {file = "pyasn1_modules-0.2.8-py3.6.egg", hash = "sha256:cbac4bc38d117f2a49aeedec4407d23e8866ea4ac27ff2cf7fb3e5b570df19e0"}, + {file = "pyasn1_modules-0.2.8-py3.7.egg", hash = "sha256:c29a5e5cc7a3f05926aff34e097e84f8589cd790ce0ed41b67aed6857b26aafd"}, +] +pycodestyle = [ + {file = "pycodestyle-2.8.0-py2.py3-none-any.whl", hash = "sha256:720f8b39dde8b293825e7ff02c475f3077124006db4f440dcbc9a20b76548a20"}, + {file = "pycodestyle-2.8.0.tar.gz", hash = "sha256:eddd5847ef438ea1c7870ca7eb78a9d47ce0cdb4851a5523949f2601d0cbbe7f"}, +] +pycparser = [ + {file = "pycparser-2.21-py2.py3-none-any.whl", hash = "sha256:8ee45429555515e1f6b185e78100aea234072576aa43ab53aefcae078162fca9"}, + {file = "pycparser-2.21.tar.gz", hash = "sha256:e644fdec12f7872f86c58ff790da456218b10f863970249516d60a5eaca77206"}, +] +pyflakes = [ + {file = "pyflakes-2.4.0-py2.py3-none-any.whl", hash = "sha256:3bb3a3f256f4b7968c9c788781e4ff07dce46bdf12339dcda61053375426ee2e"}, + {file = "pyflakes-2.4.0.tar.gz", hash = "sha256:05a85c2872edf37a4ed30b0cce2f6093e1d0581f8c19d7393122da7e25b2b24c"}, +] +pygithub = [ + {file = "PyGithub-1.55-py3-none-any.whl", hash = "sha256:2caf0054ea079b71e539741ae56c5a95e073b81fa472ce222e81667381b9601b"}, + {file = "PyGithub-1.55.tar.gz", hash = "sha256:1bbfff9372047ff3f21d5cd8e07720f3dbfdaf6462fcaed9d815f528f1ba7283"}, +] +pygments = [ + {file = "Pygments-2.11.2-py3-none-any.whl", hash = "sha256:44238f1b60a76d78fc8ca0528ee429702aae011c265fe6a8dd8b63049ae41c65"}, + {file = "Pygments-2.11.2.tar.gz", hash = "sha256:4e426f72023d88d03b2fa258de560726ce890ff3b630f88c21cbb8b2503b8c6a"}, +] +pyjwt = [ + {file = "PyJWT-2.3.0-py3-none-any.whl", hash = "sha256:e0c4bb8d9f0af0c7f5b1ec4c5036309617d03d56932877f2f7a0beeb5318322f"}, + {file = "PyJWT-2.3.0.tar.gz", hash = "sha256:b888b4d56f06f6dcd777210c334e69c737be74755d3e5e9ee3fe67dc18a0ee41"}, +] +pymacaroons = [ + {file = "pymacaroons-0.13.0-py2.py3-none-any.whl", hash = "sha256:3e14dff6a262fdbf1a15e769ce635a8aea72e6f8f91e408f9a97166c53b91907"}, + {file = "pymacaroons-0.13.0.tar.gz", hash = "sha256:1e6bba42a5f66c245adf38a5a4006a99dcc06a0703786ea636098667d42903b8"}, +] +pympler = [ + {file = "Pympler-1.0.1-py3-none-any.whl", hash = "sha256:d260dda9ae781e1eab6ea15bacb84015849833ba5555f141d2d9b7b7473b307d"}, + {file = "Pympler-1.0.1.tar.gz", hash = "sha256:993f1a3599ca3f4fcd7160c7545ad06310c9e12f70174ae7ae8d4e25f6c5d3fa"}, +] +pynacl = [ + {file = "PyNaCl-1.5.0-cp36-abi3-macosx_10_10_universal2.whl", hash = "sha256:401002a4aaa07c9414132aaed7f6836ff98f59277a234704ff66878c2ee4a0d1"}, + {file = "PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl", hash = "sha256:52cb72a79269189d4e0dc537556f4740f7f0a9ec41c1322598799b0bdad4ef92"}, + {file = "PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a36d4a9dda1f19ce6e03c9a784a2921a4b726b02e1c736600ca9c22029474394"}, + {file = "PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl", hash = "sha256:0c84947a22519e013607c9be43706dd42513f9e6ae5d39d3613ca1e142fba44d"}, + {file = "PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:06b8f6fa7f5de8d5d2f7573fe8c863c051225a27b61e6860fd047b1775807858"}, + {file = "PyNaCl-1.5.0-cp36-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:a422368fc821589c228f4c49438a368831cb5bbc0eab5ebe1d7fac9dded6567b"}, + {file = "PyNaCl-1.5.0-cp36-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:61f642bf2378713e2c2e1de73444a3778e5f0a38be6fee0fe532fe30060282ff"}, + {file = "PyNaCl-1.5.0-cp36-abi3-win32.whl", hash = "sha256:e46dae94e34b085175f8abb3b0aaa7da40767865ac82c928eeb9e57e1ea8a543"}, + {file = "PyNaCl-1.5.0-cp36-abi3-win_amd64.whl", hash = "sha256:20f42270d27e1b6a29f54032090b972d97f0a1b0948cc52392041ef7831fee93"}, + {file = "PyNaCl-1.5.0.tar.gz", hash = "sha256:8ac7448f09ab85811607bdd21ec2464495ac8b7c66d146bf545b0f08fb9220ba"}, +] +pyopenssl = [ + {file = "pyOpenSSL-22.0.0-py2.py3-none-any.whl", hash = "sha256:ea252b38c87425b64116f808355e8da644ef9b07e429398bfece610f893ee2e0"}, + {file = "pyOpenSSL-22.0.0.tar.gz", hash = "sha256:660b1b1425aac4a1bea1d94168a85d99f0b3144c869dd4390d27629d0087f1bf"}, +] +pyparsing = [ + {file = "pyparsing-3.0.7-py3-none-any.whl", hash = "sha256:a6c06a88f252e6c322f65faf8f418b16213b51bdfaece0524c1c1bc30c63c484"}, + {file = "pyparsing-3.0.7.tar.gz", hash = "sha256:18ee9022775d270c55187733956460083db60b37d0d0fb357445f3094eed3eea"}, +] +pyrsistent = [ + {file = "pyrsistent-0.18.1-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:df46c854f490f81210870e509818b729db4488e1f30f2a1ce1698b2295a878d1"}, + {file = "pyrsistent-0.18.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5d45866ececf4a5fff8742c25722da6d4c9e180daa7b405dc0a2a2790d668c26"}, + {file = "pyrsistent-0.18.1-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:4ed6784ceac462a7d6fcb7e9b663e93b9a6fb373b7f43594f9ff68875788e01e"}, + {file = "pyrsistent-0.18.1-cp310-cp310-win32.whl", hash = "sha256:e4f3149fd5eb9b285d6bfb54d2e5173f6a116fe19172686797c056672689daf6"}, + {file = "pyrsistent-0.18.1-cp310-cp310-win_amd64.whl", hash = "sha256:636ce2dc235046ccd3d8c56a7ad54e99d5c1cd0ef07d9ae847306c91d11b5fec"}, + {file = "pyrsistent-0.18.1-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:e92a52c166426efbe0d1ec1332ee9119b6d32fc1f0bbfd55d5c1088070e7fc1b"}, + {file = "pyrsistent-0.18.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d7a096646eab884bf8bed965bad63ea327e0d0c38989fc83c5ea7b8a87037bfc"}, + {file = "pyrsistent-0.18.1-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:cdfd2c361b8a8e5d9499b9082b501c452ade8bbf42aef97ea04854f4a3f43b22"}, + {file = "pyrsistent-0.18.1-cp37-cp37m-win32.whl", hash = "sha256:7ec335fc998faa4febe75cc5268a9eac0478b3f681602c1f27befaf2a1abe1d8"}, + {file = "pyrsistent-0.18.1-cp37-cp37m-win_amd64.whl", hash = "sha256:6455fc599df93d1f60e1c5c4fe471499f08d190d57eca040c0ea182301321286"}, + {file = "pyrsistent-0.18.1-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:fd8da6d0124efa2f67d86fa70c851022f87c98e205f0594e1fae044e7119a5a6"}, + {file = "pyrsistent-0.18.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7bfe2388663fd18bd8ce7db2c91c7400bf3e1a9e8bd7d63bf7e77d39051b85ec"}, + {file = "pyrsistent-0.18.1-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0e3e1fcc45199df76053026a51cc59ab2ea3fc7c094c6627e93b7b44cdae2c8c"}, + {file = "pyrsistent-0.18.1-cp38-cp38-win32.whl", hash = "sha256:b568f35ad53a7b07ed9b1b2bae09eb15cdd671a5ba5d2c66caee40dbf91c68ca"}, + {file = "pyrsistent-0.18.1-cp38-cp38-win_amd64.whl", hash = "sha256:d1b96547410f76078eaf66d282ddca2e4baae8964364abb4f4dcdde855cd123a"}, + {file = "pyrsistent-0.18.1-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:f87cc2863ef33c709e237d4b5f4502a62a00fab450c9e020892e8e2ede5847f5"}, + {file = "pyrsistent-0.18.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:6bc66318fb7ee012071b2792024564973ecc80e9522842eb4e17743604b5e045"}, + {file = "pyrsistent-0.18.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:914474c9f1d93080338ace89cb2acee74f4f666fb0424896fcfb8d86058bf17c"}, + {file = "pyrsistent-0.18.1-cp39-cp39-win32.whl", hash = "sha256:1b34eedd6812bf4d33814fca1b66005805d3640ce53140ab8bbb1e2651b0d9bc"}, + {file = "pyrsistent-0.18.1-cp39-cp39-win_amd64.whl", hash = "sha256:e24a828f57e0c337c8d8bb9f6b12f09dfdf0273da25fda9e314f0b684b415a07"}, + {file = "pyrsistent-0.18.1.tar.gz", hash = "sha256:d4d61f8b993a7255ba714df3aca52700f8125289f84f704cf80916517c46eb96"}, +] +pysaml2 = [ + {file = "pysaml2-7.1.2-py2.py3-none-any.whl", hash = "sha256:d915961aaa4d4d97d952b30fe5d18d64cf053465acf3e38d8090b36c5ff08325"}, + {file = "pysaml2-7.1.2.tar.gz", hash = "sha256:1ec94442306511b93fe7a5710f224e05e0aba948682d506614d1e04f3232f827"}, +] +python-dateutil = [ + {file = "python-dateutil-2.8.2.tar.gz", hash = "sha256:0123cacc1627ae19ddf3c27a5de5bd67ee4586fbdd6440d9748f8abb483d3e86"}, + {file = "python_dateutil-2.8.2-py2.py3-none-any.whl", hash = "sha256:961d03dc3453ebbc59dbdea9e4e11c5651520a876d0f4db161e8674aae935da9"}, +] +pytz = [ + {file = "pytz-2021.3-py2.py3-none-any.whl", hash = "sha256:3672058bc3453457b622aab7a1c3bfd5ab0bdae451512f6cf25f64ed37f5b87c"}, + {file = "pytz-2021.3.tar.gz", hash = "sha256:acad2d8b20a1af07d4e4c9d2e9285c5ed9104354062f275f3fcd88dcef4f1326"}, +] +pywin32-ctypes = [ + {file = "pywin32-ctypes-0.2.0.tar.gz", hash = "sha256:24ffc3b341d457d48e8922352130cf2644024a4ff09762a2261fd34c36ee5942"}, + {file = "pywin32_ctypes-0.2.0-py2.py3-none-any.whl", hash = "sha256:9dc2d991b3479cc2df15930958b674a48a227d5361d413827a4cfd0b5876fc98"}, +] +pyyaml = [ + {file = "PyYAML-6.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:d4db7c7aef085872ef65a8fd7d6d09a14ae91f691dec3e87ee5ee0539d516f53"}, + {file = "PyYAML-6.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:9df7ed3b3d2e0ecfe09e14741b857df43adb5a3ddadc919a2d94fbdf78fea53c"}, + {file = "PyYAML-6.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:77f396e6ef4c73fdc33a9157446466f1cff553d979bd00ecb64385760c6babdc"}, + {file = "PyYAML-6.0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:a80a78046a72361de73f8f395f1f1e49f956c6be882eed58505a15f3e430962b"}, + {file = "PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:f84fbc98b019fef2ee9a1cb3ce93e3187a6df0b2538a651bfb890254ba9f90b5"}, + {file = "PyYAML-6.0-cp310-cp310-win32.whl", hash = "sha256:2cd5df3de48857ed0544b34e2d40e9fac445930039f3cfe4bcc592a1f836d513"}, + {file = "PyYAML-6.0-cp310-cp310-win_amd64.whl", hash = "sha256:daf496c58a8c52083df09b80c860005194014c3698698d1a57cbcfa182142a3a"}, + {file = "PyYAML-6.0-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:897b80890765f037df3403d22bab41627ca8811ae55e9a722fd0392850ec4d86"}, + {file = "PyYAML-6.0-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:50602afada6d6cbfad699b0c7bb50d5ccffa7e46a3d738092afddc1f9758427f"}, + {file = "PyYAML-6.0-cp36-cp36m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:48c346915c114f5fdb3ead70312bd042a953a8ce5c7106d5bfb1a5254e47da92"}, + {file = "PyYAML-6.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:98c4d36e99714e55cfbaaee6dd5badbc9a1ec339ebfc3b1f52e293aee6bb71a4"}, + {file = "PyYAML-6.0-cp36-cp36m-win32.whl", hash = "sha256:0283c35a6a9fbf047493e3a0ce8d79ef5030852c51e9d911a27badfde0605293"}, + {file = "PyYAML-6.0-cp36-cp36m-win_amd64.whl", hash = "sha256:07751360502caac1c067a8132d150cf3d61339af5691fe9e87803040dbc5db57"}, + {file = "PyYAML-6.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:819b3830a1543db06c4d4b865e70ded25be52a2e0631ccd2f6a47a2822f2fd7c"}, + {file = "PyYAML-6.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:473f9edb243cb1935ab5a084eb238d842fb8f404ed2193a915d1784b5a6b5fc0"}, + {file = "PyYAML-6.0-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:0ce82d761c532fe4ec3f87fc45688bdd3a4c1dc5e0b4a19814b9009a29baefd4"}, + {file = "PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:231710d57adfd809ef5d34183b8ed1eeae3f76459c18fb4a0b373ad56bedcdd9"}, + {file = "PyYAML-6.0-cp37-cp37m-win32.whl", hash = "sha256:c5687b8d43cf58545ade1fe3e055f70eac7a5a1a0bf42824308d868289a95737"}, + {file = "PyYAML-6.0-cp37-cp37m-win_amd64.whl", hash = "sha256:d15a181d1ecd0d4270dc32edb46f7cb7733c7c508857278d3d378d14d606db2d"}, + {file = "PyYAML-6.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:0b4624f379dab24d3725ffde76559cff63d9ec94e1736b556dacdfebe5ab6d4b"}, + {file = "PyYAML-6.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:213c60cd50106436cc818accf5baa1aba61c0189ff610f64f4a3e8c6726218ba"}, + {file = "PyYAML-6.0-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:9fa600030013c4de8165339db93d182b9431076eb98eb40ee068700c9c813e34"}, + {file = "PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:277a0ef2981ca40581a47093e9e2d13b3f1fbbeffae064c1d21bfceba2030287"}, + {file = "PyYAML-6.0-cp38-cp38-win32.whl", hash = "sha256:d4eccecf9adf6fbcc6861a38015c2a64f38b9d94838ac1810a9023a0609e1b78"}, + {file = "PyYAML-6.0-cp38-cp38-win_amd64.whl", hash = "sha256:1e4747bc279b4f613a09eb64bba2ba602d8a6664c6ce6396a4d0cd413a50ce07"}, + {file = "PyYAML-6.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:055d937d65826939cb044fc8c9b08889e8c743fdc6a32b33e2390f66013e449b"}, + {file = "PyYAML-6.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:e61ceaab6f49fb8bdfaa0f92c4b57bcfbea54c09277b1b4f7ac376bfb7a7c174"}, + {file = "PyYAML-6.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d67d839ede4ed1b28a4e8909735fc992a923cdb84e618544973d7dfc71540803"}, + {file = "PyYAML-6.0-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:cba8c411ef271aa037d7357a2bc8f9ee8b58b9965831d9e51baf703280dc73d3"}, + {file = "PyYAML-6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:40527857252b61eacd1d9af500c3337ba8deb8fc298940291486c465c8b46ec0"}, + {file = "PyYAML-6.0-cp39-cp39-win32.whl", hash = "sha256:b5b9eccad747aabaaffbc6064800670f0c297e52c12754eb1d976c57e4f74dcb"}, + {file = "PyYAML-6.0-cp39-cp39-win_amd64.whl", hash = "sha256:b3d267842bf12586ba6c734f89d1f5b871df0273157918b0ccefa29deb05c21c"}, + {file = "PyYAML-6.0.tar.gz", hash = "sha256:68fb519c14306fec9720a2a5b45bc9f0c8d1b9c72adf45c37baedfcd949c35a2"}, +] +readme-renderer = [ + {file = "readme_renderer-33.0-py3-none-any.whl", hash = "sha256:f02cee0c4de9636b5a62b6be50c9742427ba1b956aad1d938bfb087d0d72ccdf"}, + {file = "readme_renderer-33.0.tar.gz", hash = "sha256:e3b53bc84bd6af054e4cc1fe3567dc1ae19f554134221043a3f8c674e22209db"}, +] +redbaron = [ + {file = "redbaron-0.9.2-py2.py3-none-any.whl", hash = "sha256:d01032b6a848b5521a8d6ef72486315c2880f420956870cdd742e2b5a09b9bab"}, + {file = "redbaron-0.9.2.tar.gz", hash = "sha256:472d0739ca6b2240bb2278ae428604a75472c9c12e86c6321e8c016139c0132f"}, +] +requests = [ + {file = "requests-2.27.1-py2.py3-none-any.whl", hash = "sha256:f22fa1e554c9ddfd16e6e41ac79759e17be9e492b3587efa038054674760e72d"}, + {file = "requests-2.27.1.tar.gz", hash = "sha256:68d7c56fd5a8999887728ef304a6d12edc7be74f1cfa47714fc8b414525c9a61"}, +] +requests-toolbelt = [ + {file = "requests-toolbelt-0.9.1.tar.gz", hash = "sha256:968089d4584ad4ad7c171454f0a5c6dac23971e9472521ea3b6d49d610aa6fc0"}, + {file = "requests_toolbelt-0.9.1-py2.py3-none-any.whl", hash = "sha256:380606e1d10dc85c3bd47bf5a6095f815ec007be7a8b69c878507068df059e6f"}, +] +rfc3986 = [ + {file = "rfc3986-2.0.0-py2.py3-none-any.whl", hash = "sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd"}, + {file = "rfc3986-2.0.0.tar.gz", hash = "sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c"}, +] +rply = [ + {file = "rply-0.7.8-py2.py3-none-any.whl", hash = "sha256:28ffd11d656c48aeb8c508eb382acd6a0bd906662624b34388751732a27807e7"}, + {file = "rply-0.7.8.tar.gz", hash = "sha256:2a808ac25a4580a9991fc304d64434e299a8fc75760574492f242cbb5bb301c9"}, +] +secretstorage = [ + {file = "SecretStorage-3.3.1-py3-none-any.whl", hash = "sha256:422d82c36172d88d6a0ed5afdec956514b189ddbfb72fefab0c8a1cee4eaf71f"}, + {file = "SecretStorage-3.3.1.tar.gz", hash = "sha256:fd666c51a6bf200643495a04abb261f83229dcb6fd8472ec393df7ffc8b6f195"}, +] +sentry-sdk = [ + {file = "sentry-sdk-1.5.7.tar.gz", hash = "sha256:aa52da941c56b5a76fd838f8e9e92a850bf893a9eb1e33ffce6c21431d07ee30"}, + {file = "sentry_sdk-1.5.7-py2.py3-none-any.whl", hash = "sha256:411a8495bd18cf13038e5749e4710beb4efa53da6351f67b4c2f307c2d9b6d49"}, +] +service-identity = [ + {file = "service-identity-21.1.0.tar.gz", hash = "sha256:6e6c6086ca271dc11b033d17c3a8bea9f24ebff920c587da090afc9519419d34"}, + {file = "service_identity-21.1.0-py2.py3-none-any.whl", hash = "sha256:f0b0caac3d40627c3c04d7a51b6e06721857a0e10a8775f2d1d7e72901b3a7db"}, +] +signedjson = [ + {file = "signedjson-1.1.4-py3-none-any.whl", hash = "sha256:45569ec54241c65d2403fe3faf7169be5322547706a231e884ca2b427f23d228"}, + {file = "signedjson-1.1.4.tar.gz", hash = "sha256:cd91c56af53f169ef032c62e9c4a3292dc158866933318d0592e3462db3d6492"}, +] +simplejson = [ + {file = "simplejson-3.17.6-cp27-cp27m-macosx_10_9_x86_64.whl", hash = "sha256:a89acae02b2975b1f8e4974cb8cdf9bf9f6c91162fb8dec50c259ce700f2770a"}, + {file = "simplejson-3.17.6-cp27-cp27m-manylinux1_i686.whl", hash = "sha256:82ff356ff91be0ab2293fc6d8d262451eb6ac4fd999244c4b5f863e049ba219c"}, + {file = "simplejson-3.17.6-cp27-cp27m-manylinux1_x86_64.whl", hash = "sha256:0de783e9c2b87bdd75b57efa2b6260c24b94605b5c9843517577d40ee0c3cc8a"}, + {file = "simplejson-3.17.6-cp27-cp27m-manylinux2010_i686.whl", hash = "sha256:d24a9e61df7a7787b338a58abfba975414937b609eb6b18973e25f573bc0eeeb"}, + {file = "simplejson-3.17.6-cp27-cp27m-manylinux2010_x86_64.whl", hash = "sha256:e8603e691580487f11306ecb066c76f1f4a8b54fb3bdb23fa40643a059509366"}, + {file = "simplejson-3.17.6-cp27-cp27mu-manylinux1_i686.whl", hash = "sha256:9b01e7b00654115965a206e3015f0166674ec1e575198a62a977355597c0bef5"}, + {file = "simplejson-3.17.6-cp27-cp27mu-manylinux1_x86_64.whl", hash = "sha256:37bc0cf0e5599f36072077e56e248f3336917ded1d33d2688624d8ed3cefd7d2"}, + {file = "simplejson-3.17.6-cp27-cp27mu-manylinux2010_i686.whl", hash = "sha256:cf6e7d5fe2aeb54898df18db1baf479863eae581cce05410f61f6b4188c8ada1"}, + {file = "simplejson-3.17.6-cp27-cp27mu-manylinux2010_x86_64.whl", hash = "sha256:bdfc54b4468ed4cd7415928cbe782f4d782722a81aeb0f81e2ddca9932632211"}, + {file = "simplejson-3.17.6-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:dd16302d39c4d6f4afde80edd0c97d4db643327d355a312762ccd9bd2ca515ed"}, + {file = "simplejson-3.17.6-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:deac4bdafa19bbb89edfb73b19f7f69a52d0b5bd3bb0c4ad404c1bbfd7b4b7fd"}, + {file = "simplejson-3.17.6-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:a8bbdb166e2fb816e43ab034c865147edafe28e1b19c72433147789ac83e2dda"}, + {file = "simplejson-3.17.6-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a7854326920d41c3b5d468154318fe6ba4390cb2410480976787c640707e0180"}, + {file = "simplejson-3.17.6-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:04e31fa6ac8e326480703fb6ded1488bfa6f1d3f760d32e29dbf66d0838982ce"}, + {file = "simplejson-3.17.6-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:f63600ec06982cdf480899026f4fda622776f5fabed9a869fdb32d72bc17e99a"}, + {file = "simplejson-3.17.6-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:e03c3b8cc7883a54c3f34a6a135c4a17bc9088a33f36796acdb47162791b02f6"}, + {file = "simplejson-3.17.6-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:a2d30d6c1652140181dc6861f564449ad71a45e4f165a6868c27d36745b65d40"}, + {file = "simplejson-3.17.6-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:a1aa6e4cae8e3b8d5321be4f51c5ce77188faf7baa9fe1e78611f93a8eed2882"}, + {file = "simplejson-3.17.6-cp310-cp310-win32.whl", hash = "sha256:97202f939c3ff341fc3fa84d15db86156b1edc669424ba20b0a1fcd4a796a045"}, + {file = "simplejson-3.17.6-cp310-cp310-win_amd64.whl", hash = "sha256:80d3bc9944be1d73e5b1726c3bbfd2628d3d7fe2880711b1eb90b617b9b8ac70"}, + {file = "simplejson-3.17.6-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:9fa621b3c0c05d965882c920347b6593751b7ab20d8fa81e426f1735ca1a9fc7"}, + {file = "simplejson-3.17.6-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:dd2fb11922f58df8528adfca123f6a84748ad17d066007e7ac977720063556bd"}, + {file = "simplejson-3.17.6-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:724c1fe135aa437d5126138d977004d165a3b5e2ee98fc4eb3e7c0ef645e7e27"}, + {file = "simplejson-3.17.6-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:4ff4ac6ff3aa8f814ac0f50bf218a2e1a434a17aafad4f0400a57a8cc62ef17f"}, + {file = "simplejson-3.17.6-cp36-cp36m-musllinux_1_1_aarch64.whl", hash = "sha256:67093a526e42981fdd954868062e56c9b67fdd7e712616cc3265ad0c210ecb51"}, + {file = "simplejson-3.17.6-cp36-cp36m-musllinux_1_1_i686.whl", hash = "sha256:5d6b4af7ad7e4ac515bc6e602e7b79e2204e25dbd10ab3aa2beef3c5a9cad2c7"}, + {file = "simplejson-3.17.6-cp36-cp36m-musllinux_1_1_x86_64.whl", hash = "sha256:1c9b1ed7ed282b36571638297525f8ef80f34b3e2d600a56f962c6044f24200d"}, + {file = "simplejson-3.17.6-cp36-cp36m-win32.whl", hash = "sha256:632ecbbd2228575e6860c9e49ea3cc5423764d5aa70b92acc4e74096fb434044"}, + {file = "simplejson-3.17.6-cp36-cp36m-win_amd64.whl", hash = "sha256:4c09868ddb86bf79b1feb4e3e7e4a35cd6e61ddb3452b54e20cf296313622566"}, + {file = "simplejson-3.17.6-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:4b6bd8144f15a491c662f06814bd8eaa54b17f26095bb775411f39bacaf66837"}, + {file = "simplejson-3.17.6-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5decdc78849617917c206b01e9fc1d694fd58caa961be816cb37d3150d613d9a"}, + {file = "simplejson-3.17.6-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:521877c7bd060470806eb6335926e27453d740ac1958eaf0d8c00911bc5e1802"}, + {file = "simplejson-3.17.6-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:65b998193bd7b0c7ecdfffbc825d808eac66279313cb67d8892bb259c9d91494"}, + {file = "simplejson-3.17.6-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:ac786f6cb7aa10d44e9641c7a7d16d7f6e095b138795cd43503769d4154e0dc2"}, + {file = "simplejson-3.17.6-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:3ff5b3464e1ce86a8de8c88e61d4836927d5595c2162cab22e96ff551b916e81"}, + {file = "simplejson-3.17.6-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:69bd56b1d257a91e763256d63606937ae4eb890b18a789b66951c00062afec33"}, + {file = "simplejson-3.17.6-cp37-cp37m-win32.whl", hash = "sha256:b81076552d34c27e5149a40187a8f7e2abb2d3185576a317aaf14aeeedad862a"}, + {file = "simplejson-3.17.6-cp37-cp37m-win_amd64.whl", hash = "sha256:07ecaafc1b1501f275bf5acdee34a4ad33c7c24ede287183ea77a02dc071e0c0"}, + {file = "simplejson-3.17.6-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:068670af975247acbb9fc3d5393293368cda17026db467bf7a51548ee8f17ee1"}, + {file = "simplejson-3.17.6-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:4d1c135af0c72cb28dd259cf7ba218338f4dc027061262e46fe058b4e6a4c6a3"}, + {file = "simplejson-3.17.6-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:23fe704da910ff45e72543cbba152821685a889cf00fc58d5c8ee96a9bad5f94"}, + {file = "simplejson-3.17.6-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f444762fed1bc1fd75187ef14a20ed900c1fbb245d45be9e834b822a0223bc81"}, + {file = "simplejson-3.17.6-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:681eb4d37c9a9a6eb9b3245a5e89d7f7b2b9895590bb08a20aa598c1eb0a1d9d"}, + {file = "simplejson-3.17.6-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:8e8607d8f6b4f9d46fee11447e334d6ab50e993dd4dbfb22f674616ce20907ab"}, + {file = "simplejson-3.17.6-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:b10556817f09d46d420edd982dd0653940b90151d0576f09143a8e773459f6fe"}, + {file = "simplejson-3.17.6-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:e1ec8a9ee0987d4524ffd6299e778c16cc35fef6d1a2764e609f90962f0b293a"}, + {file = "simplejson-3.17.6-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:0b4126cac7d69ac06ff22efd3e0b3328a4a70624fcd6bca4fc1b4e6d9e2e12bf"}, + {file = "simplejson-3.17.6-cp38-cp38-win32.whl", hash = "sha256:35a49ebef25f1ebdef54262e54ae80904d8692367a9f208cdfbc38dbf649e00a"}, + {file = "simplejson-3.17.6-cp38-cp38-win_amd64.whl", hash = "sha256:743cd768affaa508a21499f4858c5b824ffa2e1394ed94eb85caf47ac0732198"}, + {file = "simplejson-3.17.6-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:fb62d517a516128bacf08cb6a86ecd39fb06d08e7c4980251f5d5601d29989ba"}, + {file = "simplejson-3.17.6-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:12133863178a8080a3dccbf5cb2edfab0001bc41e5d6d2446af2a1131105adfe"}, + {file = "simplejson-3.17.6-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:5540fba2d437edaf4aa4fbb80f43f42a8334206ad1ad3b27aef577fd989f20d9"}, + {file = "simplejson-3.17.6-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d74ee72b5071818a1a5dab47338e87f08a738cb938a3b0653b9e4d959ddd1fd9"}, + {file = "simplejson-3.17.6-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:28221620f4dcabdeac310846629b976e599a13f59abb21616356a85231ebd6ad"}, + {file = "simplejson-3.17.6-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:b09bc62e5193e31d7f9876220fb429ec13a6a181a24d897b9edfbbdbcd678851"}, + {file = "simplejson-3.17.6-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:7255a37ff50593c9b2f1afa8fafd6ef5763213c1ed5a9e2c6f5b9cc925ab979f"}, + {file = "simplejson-3.17.6-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:401d40969cee3df7bda211e57b903a534561b77a7ade0dd622a8d1a31eaa8ba7"}, + {file = "simplejson-3.17.6-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:a649d0f66029c7eb67042b15374bd93a26aae202591d9afd71e111dd0006b198"}, + {file = "simplejson-3.17.6-cp39-cp39-win32.whl", hash = "sha256:522fad7be85de57430d6d287c4b635813932946ebf41b913fe7e880d154ade2e"}, + {file = "simplejson-3.17.6-cp39-cp39-win_amd64.whl", hash = "sha256:3fe87570168b2ae018391e2b43fbf66e8593a86feccb4b0500d134c998983ccc"}, + {file = "simplejson-3.17.6.tar.gz", hash = "sha256:cf98038d2abf63a1ada5730e91e84c642ba6c225b0198c3684151b1f80c5f8a6"}, +] +six = [ + {file = "six-1.16.0-py2.py3-none-any.whl", hash = "sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254"}, + {file = "six-1.16.0.tar.gz", hash = "sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926"}, +] +smmap = [ + {file = "smmap-5.0.0-py3-none-any.whl", hash = "sha256:2aba19d6a040e78d8b09de5c57e96207b09ed71d8e55ce0959eeee6c8e190d94"}, + {file = "smmap-5.0.0.tar.gz", hash = "sha256:c840e62059cd3be204b0c9c9f74be2c09d5648eddd4580d9314c3ecde0b30936"}, +] +sortedcontainers = [ + {file = "sortedcontainers-2.4.0-py2.py3-none-any.whl", hash = "sha256:a163dcaede0f1c021485e957a39245190e74249897e2ae4b2aa38595db237ee0"}, + {file = "sortedcontainers-2.4.0.tar.gz", hash = "sha256:25caa5a06cc30b6b83d11423433f65d1f9d76c4c6a0c90e3379eaa43b9bfdb88"}, +] +systemd-python = [ + {file = "systemd-python-234.tar.gz", hash = "sha256:fd0e44bf70eadae45aadc292cb0a7eb5b0b6372cd1b391228047d33895db83e7"}, +] +threadloop = [ + {file = "threadloop-1.0.2-py2-none-any.whl", hash = "sha256:5c90dbefab6ffbdba26afb4829d2a9df8275d13ac7dc58dccb0e279992679599"}, + {file = "threadloop-1.0.2.tar.gz", hash = "sha256:8b180aac31013de13c2ad5c834819771992d350267bddb854613ae77ef571944"}, +] +thrift = [ + {file = "thrift-0.15.0.tar.gz", hash = "sha256:87c8205a71cf8bbb111cb99b1f7495070fbc9cabb671669568854210da5b3e29"}, +] +tomli = [ + {file = "tomli-1.2.3-py3-none-any.whl", hash = "sha256:e3069e4be3ead9668e21cb9b074cd948f7b3113fd9c8bba083f48247aab8b11c"}, + {file = "tomli-1.2.3.tar.gz", hash = "sha256:05b6166bff487dc068d322585c7ea4ef78deed501cc124060e0f238e89a9231f"}, +] +tornado = [ + {file = "tornado-6.1-cp35-cp35m-macosx_10_9_x86_64.whl", hash = "sha256:d371e811d6b156d82aa5f9a4e08b58debf97c302a35714f6f45e35139c332e32"}, + {file = "tornado-6.1-cp35-cp35m-manylinux1_i686.whl", hash = "sha256:0d321a39c36e5f2c4ff12b4ed58d41390460f798422c4504e09eb5678e09998c"}, + {file = "tornado-6.1-cp35-cp35m-manylinux1_x86_64.whl", hash = "sha256:9de9e5188a782be6b1ce866e8a51bc76a0fbaa0e16613823fc38e4fc2556ad05"}, + {file = "tornado-6.1-cp35-cp35m-manylinux2010_i686.whl", hash = "sha256:61b32d06ae8a036a6607805e6720ef00a3c98207038444ba7fd3d169cd998910"}, + {file = "tornado-6.1-cp35-cp35m-manylinux2010_x86_64.whl", hash = "sha256:3e63498f680547ed24d2c71e6497f24bca791aca2fe116dbc2bd0ac7f191691b"}, + {file = "tornado-6.1-cp35-cp35m-manylinux2014_aarch64.whl", hash = "sha256:6c77c9937962577a6a76917845d06af6ab9197702a42e1346d8ae2e76b5e3675"}, + {file = "tornado-6.1-cp35-cp35m-win32.whl", hash = "sha256:6286efab1ed6e74b7028327365cf7346b1d777d63ab30e21a0f4d5b275fc17d5"}, + {file = "tornado-6.1-cp35-cp35m-win_amd64.whl", hash = "sha256:fa2ba70284fa42c2a5ecb35e322e68823288a4251f9ba9cc77be04ae15eada68"}, + {file = "tornado-6.1-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:0a00ff4561e2929a2c37ce706cb8233b7907e0cdc22eab98888aca5dd3775feb"}, + {file = "tornado-6.1-cp36-cp36m-manylinux1_i686.whl", hash = "sha256:748290bf9112b581c525e6e6d3820621ff020ed95af6f17fedef416b27ed564c"}, + {file = "tornado-6.1-cp36-cp36m-manylinux1_x86_64.whl", hash = "sha256:e385b637ac3acaae8022e7e47dfa7b83d3620e432e3ecb9a3f7f58f150e50921"}, + {file = "tornado-6.1-cp36-cp36m-manylinux2010_i686.whl", hash = "sha256:25ad220258349a12ae87ede08a7b04aca51237721f63b1808d39bdb4b2164558"}, + {file = "tornado-6.1-cp36-cp36m-manylinux2010_x86_64.whl", hash = "sha256:65d98939f1a2e74b58839f8c4dab3b6b3c1ce84972ae712be02845e65391ac7c"}, + {file = "tornado-6.1-cp36-cp36m-manylinux2014_aarch64.whl", hash = "sha256:e519d64089b0876c7b467274468709dadf11e41d65f63bba207e04217f47c085"}, + {file = "tornado-6.1-cp36-cp36m-win32.whl", hash = "sha256:b87936fd2c317b6ee08a5741ea06b9d11a6074ef4cc42e031bc6403f82a32575"}, + {file = "tornado-6.1-cp36-cp36m-win_amd64.whl", hash = "sha256:cc0ee35043162abbf717b7df924597ade8e5395e7b66d18270116f8745ceb795"}, + {file = "tornado-6.1-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:7250a3fa399f08ec9cb3f7b1b987955d17e044f1ade821b32e5f435130250d7f"}, + {file = "tornado-6.1-cp37-cp37m-manylinux1_i686.whl", hash = "sha256:ed3ad863b1b40cd1d4bd21e7498329ccaece75db5a5bf58cd3c9f130843e7102"}, + {file = "tornado-6.1-cp37-cp37m-manylinux1_x86_64.whl", hash = "sha256:dcef026f608f678c118779cd6591c8af6e9b4155c44e0d1bc0c87c036fb8c8c4"}, + {file = "tornado-6.1-cp37-cp37m-manylinux2010_i686.whl", hash = "sha256:70dec29e8ac485dbf57481baee40781c63e381bebea080991893cd297742b8fd"}, + {file = "tornado-6.1-cp37-cp37m-manylinux2010_x86_64.whl", hash = "sha256:d3f7594930c423fd9f5d1a76bee85a2c36fd8b4b16921cae7e965f22575e9c01"}, + {file = "tornado-6.1-cp37-cp37m-manylinux2014_aarch64.whl", hash = "sha256:3447475585bae2e77ecb832fc0300c3695516a47d46cefa0528181a34c5b9d3d"}, + {file = "tornado-6.1-cp37-cp37m-win32.whl", hash = "sha256:e7229e60ac41a1202444497ddde70a48d33909e484f96eb0da9baf8dc68541df"}, + {file = "tornado-6.1-cp37-cp37m-win_amd64.whl", hash = "sha256:cb5ec8eead331e3bb4ce8066cf06d2dfef1bfb1b2a73082dfe8a161301b76e37"}, + {file = "tornado-6.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:20241b3cb4f425e971cb0a8e4ffc9b0a861530ae3c52f2b0434e6c1b57e9fd95"}, + {file = "tornado-6.1-cp38-cp38-manylinux1_i686.whl", hash = "sha256:c77da1263aa361938476f04c4b6c8916001b90b2c2fdd92d8d535e1af48fba5a"}, + {file = "tornado-6.1-cp38-cp38-manylinux1_x86_64.whl", hash = "sha256:fba85b6cd9c39be262fcd23865652920832b61583de2a2ca907dbd8e8a8c81e5"}, + {file = "tornado-6.1-cp38-cp38-manylinux2010_i686.whl", hash = "sha256:1e8225a1070cd8eec59a996c43229fe8f95689cb16e552d130b9793cb570a288"}, + {file = "tornado-6.1-cp38-cp38-manylinux2010_x86_64.whl", hash = "sha256:d14d30e7f46a0476efb0deb5b61343b1526f73ebb5ed84f23dc794bdb88f9d9f"}, + {file = "tornado-6.1-cp38-cp38-manylinux2014_aarch64.whl", hash = "sha256:8f959b26f2634a091bb42241c3ed8d3cedb506e7c27b8dd5c7b9f745318ddbb6"}, + {file = "tornado-6.1-cp38-cp38-win32.whl", hash = "sha256:34ca2dac9e4d7afb0bed4677512e36a52f09caa6fded70b4e3e1c89dbd92c326"}, + {file = "tornado-6.1-cp38-cp38-win_amd64.whl", hash = "sha256:6196a5c39286cc37c024cd78834fb9345e464525d8991c21e908cc046d1cc02c"}, + {file = "tornado-6.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:f0ba29bafd8e7e22920567ce0d232c26d4d47c8b5cf4ed7b562b5db39fa199c5"}, + {file = "tornado-6.1-cp39-cp39-manylinux1_i686.whl", hash = "sha256:33892118b165401f291070100d6d09359ca74addda679b60390b09f8ef325ffe"}, + {file = "tornado-6.1-cp39-cp39-manylinux1_x86_64.whl", hash = "sha256:7da13da6f985aab7f6f28debab00c67ff9cbacd588e8477034c0652ac141feea"}, + {file = "tornado-6.1-cp39-cp39-manylinux2010_i686.whl", hash = "sha256:e0791ac58d91ac58f694d8d2957884df8e4e2f6687cdf367ef7eb7497f79eaa2"}, + {file = "tornado-6.1-cp39-cp39-manylinux2010_x86_64.whl", hash = "sha256:66324e4e1beede9ac79e60f88de548da58b1f8ab4b2f1354d8375774f997e6c0"}, + {file = "tornado-6.1-cp39-cp39-manylinux2014_aarch64.whl", hash = "sha256:a48900ecea1cbb71b8c71c620dee15b62f85f7c14189bdeee54966fbd9a0c5bd"}, + {file = "tornado-6.1-cp39-cp39-win32.whl", hash = "sha256:d3d20ea5782ba63ed13bc2b8c291a053c8d807a8fa927d941bd718468f7b950c"}, + {file = "tornado-6.1-cp39-cp39-win_amd64.whl", hash = "sha256:548430be2740e327b3fe0201abe471f314741efcb0067ec4f2d7dcfb4825f3e4"}, + {file = "tornado-6.1.tar.gz", hash = "sha256:33c6e81d7bd55b468d2e793517c909b139960b6c790a60b7991b9b6b76fb9791"}, +] +towncrier = [ + {file = "towncrier-21.9.0-py2.py3-none-any.whl", hash = "sha256:fc5a88a2a54988e3a8ed2b60d553599da8330f65722cc607c839614ed87e0f92"}, + {file = "towncrier-21.9.0.tar.gz", hash = "sha256:9cb6f45c16e1a1eec9d0e7651165e7be60cd0ab81d13a5c96ca97a498ae87f48"}, +] +tqdm = [ + {file = "tqdm-4.63.0-py2.py3-none-any.whl", hash = "sha256:e643e071046f17139dea55b880dc9b33822ce21613b4a4f5ea57f202833dbc29"}, + {file = "tqdm-4.63.0.tar.gz", hash = "sha256:1d9835ede8e394bb8c9dcbffbca02d717217113adc679236873eeaac5bc0b3cd"}, +] +treq = [ + {file = "treq-22.2.0-py3-none-any.whl", hash = "sha256:27d95b07c5c14be3e7b280416139b036087617ad5595be913b1f9b3ce981b9b2"}, + {file = "treq-22.2.0.tar.gz", hash = "sha256:df757e3f141fc782ede076a604521194ffcb40fa2645cf48e5a37060307f52ec"}, +] +twine = [ + {file = "twine-3.8.0-py3-none-any.whl", hash = "sha256:d0550fca9dc19f3d5e8eadfce0c227294df0a2a951251a4385797c8a6198b7c8"}, + {file = "twine-3.8.0.tar.gz", hash = "sha256:8efa52658e0ae770686a13b675569328f1fba9837e5de1867bfe5f46a9aefe19"}, +] +twisted = [ + {file = "Twisted-22.2.0-py3-none-any.whl", hash = "sha256:5c63c149eb6b8fe1e32a0215b1cef96fabdba04f705d8efb9174b1ccf5b49d49"}, + {file = "Twisted-22.2.0.tar.gz", hash = "sha256:57f32b1f6838facb8c004c89467840367ad38e9e535f8252091345dba500b4f2"}, +] +twisted-iocpsupport = [ + {file = "twisted-iocpsupport-1.0.2.tar.gz", hash = "sha256:72068b206ee809c9c596b57b5287259ea41ddb4774d86725b19f35bf56aa32a9"}, + {file = "twisted_iocpsupport-1.0.2-cp310-cp310-win32.whl", hash = "sha256:985c06a33f5c0dae92c71a036d1ea63872ee86a21dd9b01e1f287486f15524b4"}, + {file = "twisted_iocpsupport-1.0.2-cp310-cp310-win_amd64.whl", hash = "sha256:81b3abe3527b367da0220482820cb12a16c661672b7bcfcde328902890d63323"}, + {file = "twisted_iocpsupport-1.0.2-cp36-cp36m-win32.whl", hash = "sha256:9dbb8823b49f06d4de52721b47de4d3b3026064ef4788ce62b1a21c57c3fff6f"}, + {file = "twisted_iocpsupport-1.0.2-cp36-cp36m-win_amd64.whl", hash = "sha256:b9fed67cf0f951573f06d560ac2f10f2a4bbdc6697770113a2fc396ea2cb2565"}, + {file = "twisted_iocpsupport-1.0.2-cp37-cp37m-win32.whl", hash = "sha256:b76b4eed9b27fd63ddb0877efdd2d15835fdcb6baa745cb85b66e5d016ac2878"}, + {file = "twisted_iocpsupport-1.0.2-cp37-cp37m-win_amd64.whl", hash = "sha256:851b3735ca7e8102e661872390e3bce88f8901bece95c25a0c8bb9ecb8a23d32"}, + {file = "twisted_iocpsupport-1.0.2-cp38-cp38-win32.whl", hash = "sha256:bf4133139d77fc706d8f572e6b7d82871d82ec7ef25d685c2351bdacfb701415"}, + {file = "twisted_iocpsupport-1.0.2-cp38-cp38-win_amd64.whl", hash = "sha256:306becd6e22ab6e8e4f36b6bdafd9c92e867c98a5ce517b27fdd27760ee7ae41"}, + {file = "twisted_iocpsupport-1.0.2-cp39-cp39-win32.whl", hash = "sha256:3c61742cb0bc6c1ac117a7e5f422c129832f0c295af49e01d8a6066df8cfc04d"}, + {file = "twisted_iocpsupport-1.0.2-cp39-cp39-win_amd64.whl", hash = "sha256:b435857b9efcbfc12f8c326ef0383f26416272260455bbca2cd8d8eca470c546"}, + {file = "twisted_iocpsupport-1.0.2-pp37-pypy37_pp73-win_amd64.whl", hash = "sha256:7d972cfa8439bdcb35a7be78b7ef86d73b34b808c74be56dfa785c8a93b851bf"}, +] +txredisapi = [ + {file = "txredisapi-1.4.7-py3-none-any.whl", hash = "sha256:34c9eba8d34f452d30661f073b67b8cd42b695e3d31678ec1bbf628a65a0f059"}, + {file = "txredisapi-1.4.7.tar.gz", hash = "sha256:e6cc43f51e35d608abdca8f8c7d20e148fe1d82679f6e584baea613ebec812bb"}, +] +typed-ast = [ + {file = "typed_ast-1.5.2-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:183b183b7771a508395d2cbffd6db67d6ad52958a5fdc99f450d954003900266"}, + {file = "typed_ast-1.5.2-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:676d051b1da67a852c0447621fdd11c4e104827417bf216092ec3e286f7da596"}, + {file = "typed_ast-1.5.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:bc2542e83ac8399752bc16e0b35e038bdb659ba237f4222616b4e83fb9654985"}, + {file = "typed_ast-1.5.2-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:74cac86cc586db8dfda0ce65d8bcd2bf17b58668dfcc3652762f3ef0e6677e76"}, + {file = "typed_ast-1.5.2-cp310-cp310-win_amd64.whl", hash = "sha256:18fe320f354d6f9ad3147859b6e16649a0781425268c4dde596093177660e71a"}, + {file = "typed_ast-1.5.2-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:31d8c6b2df19a777bc8826770b872a45a1f30cfefcfd729491baa5237faae837"}, + {file = "typed_ast-1.5.2-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:963a0ccc9a4188524e6e6d39b12c9ca24cc2d45a71cfdd04a26d883c922b4b78"}, + {file = "typed_ast-1.5.2-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:0eb77764ea470f14fcbb89d51bc6bbf5e7623446ac4ed06cbd9ca9495b62e36e"}, + {file = "typed_ast-1.5.2-cp36-cp36m-win_amd64.whl", hash = "sha256:294a6903a4d087db805a7656989f613371915fc45c8cc0ddc5c5a0a8ad9bea4d"}, + {file = "typed_ast-1.5.2-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:26a432dc219c6b6f38be20a958cbe1abffcc5492821d7e27f08606ef99e0dffd"}, + {file = "typed_ast-1.5.2-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c7407cfcad702f0b6c0e0f3e7ab876cd1d2c13b14ce770e412c0c4b9728a0f88"}, + {file = "typed_ast-1.5.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:f30ddd110634c2d7534b2d4e0e22967e88366b0d356b24de87419cc4410c41b7"}, + {file = "typed_ast-1.5.2-cp37-cp37m-win_amd64.whl", hash = "sha256:8c08d6625bb258179b6e512f55ad20f9dfef019bbfbe3095247401e053a3ea30"}, + {file = "typed_ast-1.5.2-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:90904d889ab8e81a956f2c0935a523cc4e077c7847a836abee832f868d5c26a4"}, + {file = "typed_ast-1.5.2-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:bbebc31bf11762b63bf61aaae232becb41c5bf6b3461b80a4df7e791fabb3aca"}, + {file = "typed_ast-1.5.2-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c29dd9a3a9d259c9fa19d19738d021632d673f6ed9b35a739f48e5f807f264fb"}, + {file = "typed_ast-1.5.2-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:58ae097a325e9bb7a684572d20eb3e1809802c5c9ec7108e85da1eb6c1a3331b"}, + {file = "typed_ast-1.5.2-cp38-cp38-win_amd64.whl", hash = "sha256:da0a98d458010bf4fe535f2d1e367a2e2060e105978873c04c04212fb20543f7"}, + {file = "typed_ast-1.5.2-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:33b4a19ddc9fc551ebabca9765d54d04600c4a50eda13893dadf67ed81d9a098"}, + {file = "typed_ast-1.5.2-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:1098df9a0592dd4c8c0ccfc2e98931278a6c6c53cb3a3e2cf7e9ee3b06153344"}, + {file = "typed_ast-1.5.2-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:42c47c3b43fe3a39ddf8de1d40dbbfca60ac8530a36c9b198ea5b9efac75c09e"}, + {file = "typed_ast-1.5.2-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:f290617f74a610849bd8f5514e34ae3d09eafd521dceaa6cf68b3f4414266d4e"}, + {file = "typed_ast-1.5.2-cp39-cp39-win_amd64.whl", hash = "sha256:df05aa5b241e2e8045f5f4367a9f6187b09c4cdf8578bb219861c4e27c443db5"}, + {file = "typed_ast-1.5.2.tar.gz", hash = "sha256:525a2d4088e70a9f75b08b3f87a51acc9cde640e19cc523c7e41aa355564ae27"}, +] +types-bleach = [ + {file = "types-bleach-4.1.4.tar.gz", hash = "sha256:2d30c2c4fb6854088ac636471352c9a51bf6c089289800d2a8060820a01cd43a"}, + {file = "types_bleach-4.1.4-py3-none-any.whl", hash = "sha256:edffe173ed6d7b6f3543036a96204a9319c3bf6c3645917b14274e43f000cc9b"}, +] +types-cryptography = [ + {file = "types-cryptography-3.3.15.tar.gz", hash = "sha256:a7983a75a7b88a18f88832008f0ef140b8d1097888ec1a0824ec8fb7e105273b"}, + {file = "types_cryptography-3.3.15-py3-none-any.whl", hash = "sha256:d9b0dd5465d7898d400850e7f35e5518aa93a7e23d3e11757cd81b4777089046"}, +] +types-enum34 = [ + {file = "types-enum34-1.1.8.tar.gz", hash = "sha256:6f9c769641d06d73a55e11c14d38ac76fcd37eb545ce79cebb6eec9d50a64110"}, + {file = "types_enum34-1.1.8-py3-none-any.whl", hash = "sha256:05058c7a495f6bfaaca0be4aeac3cce5cdd80a2bad2aab01fd49a20bf4a0209d"}, +] +types-ipaddress = [ + {file = "types-ipaddress-1.0.8.tar.gz", hash = "sha256:a03df3be5935e50ba03fa843daabff539a041a28e73e0fce2c5705bee54d3841"}, + {file = "types_ipaddress-1.0.8-py3-none-any.whl", hash = "sha256:4933b74da157ba877b1a705d64f6fa7742745e9ffd65e51011f370c11ebedb55"}, +] +types-jsonschema = [ + {file = "types-jsonschema-4.4.1.tar.gz", hash = "sha256:bd68b75217ebbb33b0242db10047581dad3b061a963a46ee80d4a9044080663e"}, + {file = "types_jsonschema-4.4.1-py3-none-any.whl", hash = "sha256:ab3ecfdc912d6091cc82f4b7556cfbf1a7cbabc26da0ceaa1cbbc232d1d09971"}, +] +types-opentracing = [ + {file = "types-opentracing-2.4.7.tar.gz", hash = "sha256:be60e9618355aa892571ace002e6b353702538b1c0dc4fbc1c921219d6658830"}, + {file = "types_opentracing-2.4.7-py3-none-any.whl", hash = "sha256:861fb8103b07cf717f501dd400cb274ca9992552314d4d6c7a824b11a215e512"}, +] +types-pillow = [ + {file = "types-Pillow-9.0.6.tar.gz", hash = "sha256:79b350b1188c080c27558429f1e119e69c9f020b877a82df761d9283070e0185"}, + {file = "types_Pillow-9.0.6-py3-none-any.whl", hash = "sha256:bd1e0a844fc718398aa265bf50fcad550fc520cc54f80e5ffeb7b3226b3cc507"}, +] +types-psycopg2 = [ + {file = "types-psycopg2-2.9.9.tar.gz", hash = "sha256:4f9d4d52eeb343dc00fd5ed4f1513a8a5c18efba0a072eb82706d15cf4f20a2e"}, + {file = "types_psycopg2-2.9.9-py3-none-any.whl", hash = "sha256:cec9291d4318ad70b407310f8304b3d40f6d0358f09870448f7a65e3027c80af"}, +] +types-pyopenssl = [ + {file = "types-pyOpenSSL-22.0.0.tar.gz", hash = "sha256:d86dde7f6fe2f1ac9fe0b6282e489f649f480364bdaa9d6a4696d52505f4477e"}, + {file = "types_pyOpenSSL-22.0.0-py3-none-any.whl", hash = "sha256:da685f57b864979f36df0157895139c8244ad4aad19b551f1678206fbad0108a"}, +] +types-pyyaml = [ + {file = "types-PyYAML-6.0.4.tar.gz", hash = "sha256:6252f62d785e730e454dfa0c9f0fb99d8dae254c5c3c686903cf878ea27c04b7"}, + {file = "types_PyYAML-6.0.4-py3-none-any.whl", hash = "sha256:693b01c713464a6851f36ff41077f8adbc6e355eda929addfb4a97208aea9b4b"}, +] +types-requests = [ + {file = "types-requests-2.27.11.tar.gz", hash = "sha256:6a7ed24b21780af4a5b5e24c310b2cd885fb612df5fd95584d03d87e5f2a195a"}, + {file = "types_requests-2.27.11-py3-none-any.whl", hash = "sha256:506279bad570c7b4b19ac1f22e50146538befbe0c133b2cea66a9b04a533a859"}, +] +types-setuptools = [ + {file = "types-setuptools-57.4.9.tar.gz", hash = "sha256:536ef74744f8e1e4be4fc719887f886e74e4cf3c792b4a06984320be4df450b5"}, + {file = "types_setuptools-57.4.9-py3-none-any.whl", hash = "sha256:948dc6863373750e2cd0b223a84f1fb608414cde5e55cf38ea657b93aeb411d2"}, +] +types-urllib3 = [ + {file = "types-urllib3-1.26.10.tar.gz", hash = "sha256:a26898f530e6c3f43f25b907f2b884486868ffd56a9faa94cbf9b3eb6e165d6a"}, + {file = "types_urllib3-1.26.10-py3-none-any.whl", hash = "sha256:d755278d5ecd7a7a6479a190e54230f241f1a99c19b81518b756b19dc69e518c"}, +] +typing-extensions = [ + {file = "typing_extensions-4.1.1-py3-none-any.whl", hash = "sha256:21c85e0fe4b9a155d0799430b0ad741cdce7e359660ccbd8b530613e8df88ce2"}, + {file = "typing_extensions-4.1.1.tar.gz", hash = "sha256:1a9462dcc3347a79b1f1c0271fbe79e844580bb598bafa1ed208b94da3cdcd42"}, +] +unpaddedbase64 = [ + {file = "unpaddedbase64-2.1.0-py3-none-any.whl", hash = "sha256:485eff129c30175d2cd6f0cd8d2310dff51e666f7f36175f738d75dfdbd0b1c6"}, + {file = "unpaddedbase64-2.1.0.tar.gz", hash = "sha256:7273c60c089de39d90f5d6d4a7883a79e319dc9d9b1c8924a7fab96178a5f005"}, +] +urllib3 = [ + {file = "urllib3-1.26.8-py2.py3-none-any.whl", hash = "sha256:000ca7f471a233c2251c6c7023ee85305721bfdf18621ebff4fd17a8653427ed"}, + {file = "urllib3-1.26.8.tar.gz", hash = "sha256:0e7c33d9a63e7ddfcb86780aac87befc2fbddf46c58dbb487e0855f7ceec283c"}, +] +webencodings = [ + {file = "webencodings-0.5.1-py2.py3-none-any.whl", hash = "sha256:a0af1213f3c2226497a97e2b3aa01a7e4bee4f403f95be16fc9acd2947514a78"}, + {file = "webencodings-0.5.1.tar.gz", hash = "sha256:b36a1c245f2d304965eb4e0a82848379241dc04b865afcc4aab16748587e1923"}, +] +wrapt = [ + {file = "wrapt-1.13.3-cp27-cp27m-macosx_10_9_x86_64.whl", hash = "sha256:e05e60ff3b2b0342153be4d1b597bbcfd8330890056b9619f4ad6b8d5c96a81a"}, + {file = "wrapt-1.13.3-cp27-cp27m-manylinux1_i686.whl", hash = "sha256:85148f4225287b6a0665eef08a178c15097366d46b210574a658c1ff5b377489"}, + {file = "wrapt-1.13.3-cp27-cp27m-manylinux1_x86_64.whl", hash = "sha256:2dded5496e8f1592ec27079b28b6ad2a1ef0b9296d270f77b8e4a3a796cf6909"}, + {file = "wrapt-1.13.3-cp27-cp27m-manylinux2010_i686.whl", hash = "sha256:e94b7d9deaa4cc7bac9198a58a7240aaf87fe56c6277ee25fa5b3aa1edebd229"}, + {file = "wrapt-1.13.3-cp27-cp27m-manylinux2010_x86_64.whl", hash = "sha256:498e6217523111d07cd67e87a791f5e9ee769f9241fcf8a379696e25806965af"}, + {file = "wrapt-1.13.3-cp27-cp27mu-manylinux1_i686.whl", hash = "sha256:ec7e20258ecc5174029a0f391e1b948bf2906cd64c198a9b8b281b811cbc04de"}, + {file = "wrapt-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl", hash = "sha256:87883690cae293541e08ba2da22cacaae0a092e0ed56bbba8d018cc486fbafbb"}, + {file = "wrapt-1.13.3-cp27-cp27mu-manylinux2010_i686.whl", hash = "sha256:f99c0489258086308aad4ae57da9e8ecf9e1f3f30fa35d5e170b4d4896554d80"}, + {file = "wrapt-1.13.3-cp27-cp27mu-manylinux2010_x86_64.whl", hash = "sha256:6a03d9917aee887690aa3f1747ce634e610f6db6f6b332b35c2dd89412912bca"}, + {file = "wrapt-1.13.3-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:936503cb0a6ed28dbfa87e8fcd0a56458822144e9d11a49ccee6d9a8adb2ac44"}, + {file = "wrapt-1.13.3-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:f9c51d9af9abb899bd34ace878fbec8bf357b3194a10c4e8e0a25512826ef056"}, + {file = "wrapt-1.13.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:220a869982ea9023e163ba915077816ca439489de6d2c09089b219f4e11b6785"}, + {file = "wrapt-1.13.3-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:0877fe981fd76b183711d767500e6b3111378ed2043c145e21816ee589d91096"}, + {file = "wrapt-1.13.3-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:43e69ffe47e3609a6aec0fe723001c60c65305784d964f5007d5b4fb1bc6bf33"}, + {file = "wrapt-1.13.3-cp310-cp310-win32.whl", hash = "sha256:78dea98c81915bbf510eb6a3c9c24915e4660302937b9ae05a0947164248020f"}, + {file = "wrapt-1.13.3-cp310-cp310-win_amd64.whl", hash = "sha256:ea3e746e29d4000cd98d572f3ee2a6050a4f784bb536f4ac1f035987fc1ed83e"}, + {file = "wrapt-1.13.3-cp35-cp35m-manylinux1_i686.whl", hash = "sha256:8c73c1a2ec7c98d7eaded149f6d225a692caa1bd7b2401a14125446e9e90410d"}, + {file = "wrapt-1.13.3-cp35-cp35m-manylinux1_x86_64.whl", hash = "sha256:086218a72ec7d986a3eddb7707c8c4526d677c7b35e355875a0fe2918b059179"}, + {file = "wrapt-1.13.3-cp35-cp35m-manylinux2010_i686.whl", hash = "sha256:e92d0d4fa68ea0c02d39f1e2f9cb5bc4b4a71e8c442207433d8db47ee79d7aa3"}, + {file = "wrapt-1.13.3-cp35-cp35m-manylinux2010_x86_64.whl", hash = "sha256:d4a5f6146cfa5c7ba0134249665acd322a70d1ea61732723c7d3e8cc0fa80755"}, + {file = "wrapt-1.13.3-cp35-cp35m-win32.whl", hash = "sha256:8aab36778fa9bba1a8f06a4919556f9f8c7b33102bd71b3ab307bb3fecb21851"}, + {file = "wrapt-1.13.3-cp35-cp35m-win_amd64.whl", hash = "sha256:944b180f61f5e36c0634d3202ba8509b986b5fbaf57db3e94df11abee244ba13"}, + {file = "wrapt-1.13.3-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:2ebdde19cd3c8cdf8df3fc165bc7827334bc4e353465048b36f7deeae8ee0918"}, + {file = "wrapt-1.13.3-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:610f5f83dd1e0ad40254c306f4764fcdc846641f120c3cf424ff57a19d5f7ade"}, + {file = "wrapt-1.13.3-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:5601f44a0f38fed36cc07db004f0eedeaadbdcec90e4e90509480e7e6060a5bc"}, + {file = "wrapt-1.13.3-cp36-cp36m-musllinux_1_1_i686.whl", hash = "sha256:e6906d6f48437dfd80464f7d7af1740eadc572b9f7a4301e7dd3d65db285cacf"}, + {file = "wrapt-1.13.3-cp36-cp36m-musllinux_1_1_x86_64.whl", hash = "sha256:766b32c762e07e26f50d8a3468e3b4228b3736c805018e4b0ec8cc01ecd88125"}, + {file = "wrapt-1.13.3-cp36-cp36m-win32.whl", hash = "sha256:5f223101f21cfd41deec8ce3889dc59f88a59b409db028c469c9b20cfeefbe36"}, + {file = "wrapt-1.13.3-cp36-cp36m-win_amd64.whl", hash = "sha256:f122ccd12fdc69628786d0c947bdd9cb2733be8f800d88b5a37c57f1f1d73c10"}, + {file = "wrapt-1.13.3-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:46f7f3af321a573fc0c3586612db4decb7eb37172af1bc6173d81f5b66c2e068"}, + {file = "wrapt-1.13.3-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:778fd096ee96890c10ce96187c76b3e99b2da44e08c9e24d5652f356873f6709"}, + {file = "wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:0cb23d36ed03bf46b894cfec777eec754146d68429c30431c99ef28482b5c1df"}, + {file = "wrapt-1.13.3-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:96b81ae75591a795d8c90edc0bfaab44d3d41ffc1aae4d994c5aa21d9b8e19a2"}, + {file = "wrapt-1.13.3-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:7dd215e4e8514004c8d810a73e342c536547038fb130205ec4bba9f5de35d45b"}, + {file = "wrapt-1.13.3-cp37-cp37m-win32.whl", hash = "sha256:47f0a183743e7f71f29e4e21574ad3fa95676136f45b91afcf83f6a050914829"}, + {file = "wrapt-1.13.3-cp37-cp37m-win_amd64.whl", hash = "sha256:fd76c47f20984b43d93de9a82011bb6e5f8325df6c9ed4d8310029a55fa361ea"}, + {file = "wrapt-1.13.3-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:b73d4b78807bd299b38e4598b8e7bd34ed55d480160d2e7fdaabd9931afa65f9"}, + {file = "wrapt-1.13.3-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:ec9465dd69d5657b5d2fa6133b3e1e989ae27d29471a672416fd729b429eb554"}, + {file = "wrapt-1.13.3-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:dd91006848eb55af2159375134d724032a2d1d13bcc6f81cd8d3ed9f2b8e846c"}, + {file = "wrapt-1.13.3-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:ae9de71eb60940e58207f8e71fe113c639da42adb02fb2bcbcaccc1ccecd092b"}, + {file = "wrapt-1.13.3-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:51799ca950cfee9396a87f4a1240622ac38973b6df5ef7a41e7f0b98797099ce"}, + {file = "wrapt-1.13.3-cp38-cp38-win32.whl", hash = "sha256:4b9c458732450ec42578b5642ac53e312092acf8c0bfce140ada5ca1ac556f79"}, + {file = "wrapt-1.13.3-cp38-cp38-win_amd64.whl", hash = "sha256:7dde79d007cd6dfa65afe404766057c2409316135cb892be4b1c768e3f3a11cb"}, + {file = "wrapt-1.13.3-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:981da26722bebb9247a0601e2922cedf8bb7a600e89c852d063313102de6f2cb"}, + {file = "wrapt-1.13.3-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:705e2af1f7be4707e49ced9153f8d72131090e52be9278b5dbb1498c749a1e32"}, + {file = "wrapt-1.13.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:25b1b1d5df495d82be1c9d2fad408f7ce5ca8a38085e2da41bb63c914baadff7"}, + {file = "wrapt-1.13.3-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:77416e6b17926d953b5c666a3cb718d5945df63ecf922af0ee576206d7033b5e"}, + {file = "wrapt-1.13.3-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:865c0b50003616f05858b22174c40ffc27a38e67359fa1495605f96125f76640"}, + {file = "wrapt-1.13.3-cp39-cp39-win32.whl", hash = "sha256:0a017a667d1f7411816e4bf214646d0ad5b1da2c1ea13dec6c162736ff25a374"}, + {file = "wrapt-1.13.3-cp39-cp39-win_amd64.whl", hash = "sha256:81bd7c90d28a4b2e1df135bfbd7c23aee3050078ca6441bead44c42483f9ebfb"}, + {file = "wrapt-1.13.3.tar.gz", hash = "sha256:1fea9cd438686e6682271d36f3481a9f3636195578bab9ca3382e2f5f01fc185"}, +] +xmlschema = [ + {file = "xmlschema-1.10.0-py3-none-any.whl", hash = "sha256:dbd68bded2fef00c19cf37110ca0565eca34cf0b6c9e1d3b62ad0de8cbb582ca"}, + {file = "xmlschema-1.10.0.tar.gz", hash = "sha256:be1eedce6a4b911fd3a7f4060d0811951820a13410e61f0454b30e9f4e7cf197"}, +] +zipp = [ + {file = "zipp-3.7.0-py3-none-any.whl", hash = "sha256:b47250dd24f92b7dd6a0a8fc5244da14608f3ca90a5efcd37a3b1642fac9a375"}, + {file = "zipp-3.7.0.tar.gz", hash = "sha256:9f50f446828eb9d45b267433fd3e9da8d801f614129124863f9c51ebceafb87d"}, +] +"zope.event" = [ + {file = "zope.event-4.5.0-py2.py3-none-any.whl", hash = "sha256:2666401939cdaa5f4e0c08cf7f20c9b21423b95e88f4675b1443973bdb080c42"}, + {file = "zope.event-4.5.0.tar.gz", hash = "sha256:5e76517f5b9b119acf37ca8819781db6c16ea433f7e2062c4afc2b6fbedb1330"}, +] +"zope.interface" = [ + {file = "zope.interface-5.4.0-cp27-cp27m-macosx_10_14_x86_64.whl", hash = "sha256:7df1e1c05304f26faa49fa752a8c690126cf98b40b91d54e6e9cc3b7d6ffe8b7"}, + {file = "zope.interface-5.4.0-cp27-cp27m-manylinux1_i686.whl", hash = "sha256:2c98384b254b37ce50eddd55db8d381a5c53b4c10ee66e1e7fe749824f894021"}, + {file = "zope.interface-5.4.0-cp27-cp27m-manylinux1_x86_64.whl", hash = "sha256:08f9636e99a9d5410181ba0729e0408d3d8748026ea938f3b970a0249daa8192"}, + {file = "zope.interface-5.4.0-cp27-cp27m-manylinux2010_i686.whl", hash = "sha256:0ea1d73b7c9dcbc5080bb8aaffb776f1c68e807767069b9ccdd06f27a161914a"}, + {file = "zope.interface-5.4.0-cp27-cp27m-manylinux2010_x86_64.whl", hash = "sha256:273f158fabc5ea33cbc936da0ab3d4ba80ede5351babc4f577d768e057651531"}, + {file = "zope.interface-5.4.0-cp27-cp27m-win32.whl", hash = "sha256:a1e6e96217a0f72e2b8629e271e1b280c6fa3fe6e59fa8f6701bec14e3354325"}, + {file = "zope.interface-5.4.0-cp27-cp27m-win_amd64.whl", hash = "sha256:877473e675fdcc113c138813a5dd440da0769a2d81f4d86614e5d62b69497155"}, + {file = "zope.interface-5.4.0-cp27-cp27mu-manylinux1_i686.whl", hash = "sha256:f7ee479e96f7ee350db1cf24afa5685a5899e2b34992fb99e1f7c1b0b758d263"}, + {file = "zope.interface-5.4.0-cp27-cp27mu-manylinux1_x86_64.whl", hash = "sha256:b0297b1e05fd128d26cc2460c810d42e205d16d76799526dfa8c8ccd50e74959"}, + {file = "zope.interface-5.4.0-cp27-cp27mu-manylinux2010_i686.whl", hash = "sha256:af310ec8335016b5e52cae60cda4a4f2a60a788cbb949a4fbea13d441aa5a09e"}, + {file = "zope.interface-5.4.0-cp27-cp27mu-manylinux2010_x86_64.whl", hash = "sha256:9a9845c4c6bb56e508651f005c4aeb0404e518c6f000d5a1123ab077ab769f5c"}, + {file = "zope.interface-5.4.0-cp35-cp35m-manylinux1_i686.whl", hash = "sha256:0b465ae0962d49c68aa9733ba92a001b2a0933c317780435f00be7ecb959c702"}, + {file = "zope.interface-5.4.0-cp35-cp35m-manylinux1_x86_64.whl", hash = "sha256:5dd9ca406499444f4c8299f803d4a14edf7890ecc595c8b1c7115c2342cadc5f"}, + {file = "zope.interface-5.4.0-cp35-cp35m-manylinux2010_i686.whl", hash = "sha256:469e2407e0fe9880ac690a3666f03eb4c3c444411a5a5fddfdabc5d184a79f05"}, + {file = "zope.interface-5.4.0-cp35-cp35m-manylinux2010_x86_64.whl", hash = "sha256:52de7fc6c21b419078008f697fd4103dbc763288b1406b4562554bd47514c004"}, + {file = "zope.interface-5.4.0-cp35-cp35m-manylinux2014_aarch64.whl", hash = "sha256:3dd4952748521205697bc2802e4afac5ed4b02909bb799ba1fe239f77fd4e117"}, + {file = "zope.interface-5.4.0-cp35-cp35m-win32.whl", hash = "sha256:dd93ea5c0c7f3e25335ab7d22a507b1dc43976e1345508f845efc573d3d779d8"}, + {file = "zope.interface-5.4.0-cp35-cp35m-win_amd64.whl", hash = "sha256:3748fac0d0f6a304e674955ab1365d515993b3a0a865e16a11ec9d86fb307f63"}, + {file = "zope.interface-5.4.0-cp36-cp36m-macosx_10_14_x86_64.whl", hash = "sha256:66c0061c91b3b9cf542131148ef7ecbecb2690d48d1612ec386de9d36766058f"}, + {file = "zope.interface-5.4.0-cp36-cp36m-manylinux1_i686.whl", hash = "sha256:d0c1bc2fa9a7285719e5678584f6b92572a5b639d0e471bb8d4b650a1a910920"}, + {file = "zope.interface-5.4.0-cp36-cp36m-manylinux1_x86_64.whl", hash = "sha256:2876246527c91e101184f63ccd1d716ec9c46519cc5f3d5375a3351c46467c46"}, + {file = "zope.interface-5.4.0-cp36-cp36m-manylinux2010_i686.whl", hash = "sha256:334701327f37c47fa628fc8b8d28c7d7730ce7daaf4bda1efb741679c2b087fc"}, + {file = "zope.interface-5.4.0-cp36-cp36m-manylinux2010_x86_64.whl", hash = "sha256:71aace0c42d53abe6fc7f726c5d3b60d90f3c5c055a447950ad6ea9cec2e37d9"}, + {file = "zope.interface-5.4.0-cp36-cp36m-manylinux2014_aarch64.whl", hash = "sha256:5bb3489b4558e49ad2c5118137cfeaf59434f9737fa9c5deefc72d22c23822e2"}, + {file = "zope.interface-5.4.0-cp36-cp36m-win32.whl", hash = "sha256:1c0e316c9add0db48a5b703833881351444398b04111188069a26a61cfb4df78"}, + {file = "zope.interface-5.4.0-cp36-cp36m-win_amd64.whl", hash = "sha256:6f0c02cbb9691b7c91d5009108f975f8ffeab5dff8f26d62e21c493060eff2a1"}, + {file = "zope.interface-5.4.0-cp37-cp37m-macosx_10_14_x86_64.whl", hash = "sha256:7d97a4306898b05404a0dcdc32d9709b7d8832c0c542b861d9a826301719794e"}, + {file = "zope.interface-5.4.0-cp37-cp37m-manylinux1_i686.whl", hash = "sha256:867a5ad16892bf20e6c4ea2aab1971f45645ff3102ad29bd84c86027fa99997b"}, + {file = "zope.interface-5.4.0-cp37-cp37m-manylinux1_x86_64.whl", hash = "sha256:5f931a1c21dfa7a9c573ec1f50a31135ccce84e32507c54e1ea404894c5eb96f"}, + {file = "zope.interface-5.4.0-cp37-cp37m-manylinux2010_i686.whl", hash = "sha256:194d0bcb1374ac3e1e023961610dc8f2c78a0f5f634d0c737691e215569e640d"}, + {file = "zope.interface-5.4.0-cp37-cp37m-manylinux2010_x86_64.whl", hash = "sha256:8270252effc60b9642b423189a2fe90eb6b59e87cbee54549db3f5562ff8d1b8"}, + {file = "zope.interface-5.4.0-cp37-cp37m-manylinux2014_aarch64.whl", hash = "sha256:15e7d1f7a6ee16572e21e3576d2012b2778cbacf75eb4b7400be37455f5ca8bf"}, + {file = "zope.interface-5.4.0-cp37-cp37m-win32.whl", hash = "sha256:8892f89999ffd992208754851e5a052f6b5db70a1e3f7d54b17c5211e37a98c7"}, + {file = "zope.interface-5.4.0-cp37-cp37m-win_amd64.whl", hash = "sha256:2e5a26f16503be6c826abca904e45f1a44ff275fdb7e9d1b75c10671c26f8b94"}, + {file = "zope.interface-5.4.0-cp38-cp38-macosx_10_14_x86_64.whl", hash = "sha256:0f91b5b948686659a8e28b728ff5e74b1be6bf40cb04704453617e5f1e945ef3"}, + {file = "zope.interface-5.4.0-cp38-cp38-manylinux1_i686.whl", hash = "sha256:4de4bc9b6d35c5af65b454d3e9bc98c50eb3960d5a3762c9438df57427134b8e"}, + {file = "zope.interface-5.4.0-cp38-cp38-manylinux1_x86_64.whl", hash = "sha256:bf68f4b2b6683e52bec69273562df15af352e5ed25d1b6641e7efddc5951d1a7"}, + {file = "zope.interface-5.4.0-cp38-cp38-manylinux2010_i686.whl", hash = "sha256:63b82bb63de7c821428d513607e84c6d97d58afd1fe2eb645030bdc185440120"}, + {file = "zope.interface-5.4.0-cp38-cp38-manylinux2010_x86_64.whl", hash = "sha256:db1fa631737dab9fa0b37f3979d8d2631e348c3b4e8325d6873c2541d0ae5a48"}, + {file = "zope.interface-5.4.0-cp38-cp38-manylinux2014_aarch64.whl", hash = "sha256:f44e517131a98f7a76696a7b21b164bcb85291cee106a23beccce454e1f433a4"}, + {file = "zope.interface-5.4.0-cp38-cp38-win32.whl", hash = "sha256:a9506a7e80bcf6eacfff7f804c0ad5350c8c95b9010e4356a4b36f5322f09abb"}, + {file = "zope.interface-5.4.0-cp38-cp38-win_amd64.whl", hash = "sha256:3c02411a3b62668200910090a0dff17c0b25aaa36145082a5a6adf08fa281e54"}, + {file = "zope.interface-5.4.0-cp39-cp39-macosx_10_14_x86_64.whl", hash = "sha256:0cee5187b60ed26d56eb2960136288ce91bcf61e2a9405660d271d1f122a69a4"}, + {file = "zope.interface-5.4.0-cp39-cp39-manylinux1_i686.whl", hash = "sha256:a8156e6a7f5e2a0ff0c5b21d6bcb45145efece1909efcbbbf48c56f8da68221d"}, + {file = "zope.interface-5.4.0-cp39-cp39-manylinux1_x86_64.whl", hash = "sha256:205e40ccde0f37496904572035deea747390a8b7dc65146d30b96e2dd1359a83"}, + {file = "zope.interface-5.4.0-cp39-cp39-manylinux2010_i686.whl", hash = "sha256:3f24df7124c323fceb53ff6168da70dbfbae1442b4f3da439cd441681f54fe25"}, + {file = "zope.interface-5.4.0-cp39-cp39-manylinux2010_x86_64.whl", hash = "sha256:5208ebd5152e040640518a77827bdfcc73773a15a33d6644015b763b9c9febc1"}, + {file = "zope.interface-5.4.0-cp39-cp39-manylinux2014_aarch64.whl", hash = "sha256:17776ecd3a1fdd2b2cd5373e5ef8b307162f581c693575ec62e7c5399d80794c"}, + {file = "zope.interface-5.4.0-cp39-cp39-win32.whl", hash = "sha256:d4d9d6c1a455d4babd320203b918ccc7fcbefe308615c521062bc2ba1aa4d26e"}, + {file = "zope.interface-5.4.0-cp39-cp39-win_amd64.whl", hash = "sha256:0cba8477e300d64a11a9789ed40ee8932b59f9ee05f85276dbb4b59acee5dd09"}, + {file = "zope.interface-5.4.0.tar.gz", hash = "sha256:5dba5f530fec3f0988d83b78cc591b58c0b6eb8431a85edd1569a0539a8a5a0e"}, +] +"zope.schema" = [ + {file = "zope.schema-6.2.0-py2.py3-none-any.whl", hash = "sha256:03150d8670549590b45109e06b7b964f4e751fa9cb5297ec4985c3bc38641b07"}, + {file = "zope.schema-6.2.0.tar.gz", hash = "sha256:2201aef8ad75ee5a881284d7a6acd384661d6dca7bde5e80a22839a77124595b"}, +] diff --git a/pyproject.toml b/pyproject.toml index c9cd0cf6ec..19ffdc7b2c 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -66,3 +66,229 @@ multi_line_output = 3 include_trailing_comma = true combine_as_imports = true +[tool.poetry] +name = "matrix-synapse" +version = "1.56.0" +description = "Homeserver for the Matrix decentralised comms protocol" +authors = ["Matrix.org Team and Contributors "] +license = "Apache-2.0" +readme = "README.rst" +repository = "https://github.com/matrix-org/synapse" +packages = [ + { include = "synapse" }, +] +classifiers = [ + "Development Status :: 5 - Production/Stable", + "Topic :: Communications :: Chat", +] +include = [ + { path = "AUTHORS.rst", format = "sdist" }, + { path = "book.toml", format = "sdist" }, + { path = "changelog.d", format = "sdist" }, + { path = "CHANGES.md", format = "sdist" }, + { path = "CONTRIBUTING.md", format = "sdist" }, + { path = "demo", format = "sdist" }, + { path = "docs", format = "sdist" }, + { path = "INSTALL.md", format = "sdist" }, + { path = "mypy.ini", format = "sdist" }, + { path = "scripts-dev", format = "sdist" }, + { path = "synmark", format="sdist" }, + { path = "sytest-blacklist", format = "sdist" }, + { path = "tests", format = "sdist" }, + { path = "UPGRADE.rst", format = "sdist" }, +] + +[tool.poetry.scripts] +synapse_homeserver = "synapse.app.homeserver:main" +synapse_worker = "synapse.app.generic_worker:main" +synctl = "synapse._scripts.synctl:main" + +export_signing_key = "synapse._scripts.export_signing_key:main" +generate_config = "synapse._scripts.generate_config:main" +generate_log_config = "synapse._scripts.generate_log_config:main" +generate_signing_key = "synapse._scripts.generate_signing_key:main" +hash_password = "synapse._scripts.hash_password:main" +register_new_matrix_user = "synapse._scripts.register_new_matrix_user:main" +synapse_port_db = "synapse._scripts.synapse_port_db:main" +synapse_review_recent_signups = "synapse._scripts.review_recent_signups:main" +update_synapse_database = "synapse._scripts.update_synapse_database:main" + +[tool.poetry.dependencies] +python = "^3.7" + +# Mandatory Dependencies +# ---------------------- +# we use the TYPE_CHECKER.redefine method added in jsonschema 3.0.0 +jsonschema = ">=3.0.0" +# frozendict 2.1.2 is broken on Debian 10: https://github.com/Marco-Sulla/python-frozendict/issues/41 +frozendict = ">=1,!=2.1.2" +# We require 2.1.0 or higher for type hints. Previous guard was >= 1.1.0 +unpaddedbase64 = ">=2.1.0" +canonicaljson = ">=1.4.0" +# we use the type definitions added in signedjson 1.1. +signedjson = ">=1.1.0" +PyNaCl = ">=1.2.1" +idna = ">=2.5" +# validating SSL certs for IP addresses requires service_identity 18.1. +service-identity = ">=18.1.0" +# Twisted 18.9 introduces some logger improvements that the structured +# logger utilises +Twisted = ">=18.9.0" +treq = ">=15.1" +# Twisted has required pyopenssl 16.0 since about Twisted 16.6. +pyOpenSSL = ">=16.0.0" +PyYAML = ">=3.11" +pyasn1 = ">=0.1.9" +pyasn1-modules = ">=0.0.7" +bcrypt = ">=3.1.0" +Pillow = ">=5.4.0" +sortedcontainers = ">=1.4.4" +pymacaroons = ">=0.13.0" +msgpack = ">=0.5.2" +phonenumbers = ">=8.2.0" +# we use GaugeHistogramMetric, which was added in prom-client 0.4.0. +prometheus-client = ">=0.4.0" +# we use `order`, which arrived in attrs 19.2.0. +# Note: 21.1.0 broke `/sync`, see #9936 +attrs = ">=19.2.0,!=21.1.0" +netaddr = ">=0.7.18" +# Jinja 2.x is incompatible with MarkupSafe>=2.1. To ensure that admins do not +# end up with a broken installation, with recent MarkupSafe but old Jinja, we +# add a lower bound to the Jinja2 dependency. +Jinja2 = ">=3.0" +bleach = ">=1.4.3" +# We use `ParamSpec`, which was added in `typing-extensions` 3.10.0.0. +typing-extensions = ">=3.10.0" +# We enforce that we have a `cryptography` version that bundles an `openssl` +# with the latest security patches. +cryptography = ">=3.4.7" +# ijson 3.1.4 fixes a bug with "." in property names +ijson = ">=3.1.4" +matrix-common = "~=1.1.0" +# We need packaging.requirements.Requirement, added in 16.1. +packaging = ">=16.1" + + +# Optional Dependencies +# --------------------- +matrix-synapse-ldap3 = { version = ">=0.1", optional = true } +psycopg2 = { version = ">=2.8", markers = "platform_python_implementation != 'PyPy'", optional = true } +psycopg2cffi = { version = ">=2.8", markers = "platform_python_implementation == 'PyPy'", optional = true } +psycopg2cffi-compat = { version = "==1.1", markers = "platform_python_implementation == 'PyPy'", optional = true } +pysaml2 = { version = ">=4.5.0", optional = true } +authlib = { version = ">=0.14.0", optional = true } +# systemd-python is necessary for logging to the systemd journal via +# `systemd.journal.JournalHandler`, as is documented in +# `contrib/systemd/log_config.yaml`. +# Note: systemd-python 231 appears to have been yanked from pypi +systemd-python = { version = ">=231", optional = true } +lxml = { version = ">=4.2.0", optional = true } +sentry-sdk = { version = ">=0.7.2", optional = true } +opentracing = { version = ">=2.2.0", optional = true } +jaeger-client = { version = ">=4.0.0", optional = true } +pyjwt = { version = ">=1.6.4", optional = true } +txredisapi = { version = ">=1.4.7", optional = true } +hiredis = { version = "*", optional = true } +Pympler = { version = "*", optional = true } +parameterized = { version = ">=0.7.4", optional = true } + +[tool.poetry.extras] +# NB: Packages that should be part of `pip install matrix-synapse[all]` need to be specified +# twice: once here, and once in the `all` extra. +matrix-synapse-ldap3 = ["matrix-synapse-ldap3"] +postgres = ["psycopg2", "psycopg2cffi", "psycopg2cffi-compat"] +saml2 = ["pysaml2"] +oidc = ["authlib"] +# systemd-python is necessary for logging to the systemd journal via +# `systemd.journal.JournalHandler`, as is documented in +# `contrib/systemd/log_config.yaml`. +systemd = ["systemd-python"] +url_preview = ["lxml"] +sentry = ["sentry-sdk"] +opentracing = ["jaeger-client", "opentracing"] +jwt = ["pyjwt"] +# hiredis is not a *strict* dependency, but it makes things much faster. +# (if it is not installed, we fall back to slow code.) +redis = ["txredisapi", "hiredis"] +# Required to use experimental `caches.track_memory_usage` config option. +cache_memory = ["pympler"] +test = ["parameterized"] + +# The duplication here is awful. I hate hate hate hate hate it. However, for now I want +# to ensure you can still `pip install matrix-synapse[all]` like today. Two motivations: +# 1) for new installations, I want instructions in existing documentation and tutorials +# out there to still work. +# 2) I don't want to hard-code a list of extras into CI if I can help it. The ideal +# solution here would be something like https://github.com/python-poetry/poetry/issues/3413 +# Poetry 1.2's dependency groups might make this easier. But I'm not trying that out +# until there's a stable release of 1.2. +# +# NB: the strings in this list must be *package* names, not extra names. +# Some of our extra names _are_ package names, which can lead to great confusion. +all = [ + # matrix-synapse-ldap3 + "matrix-synapse-ldap3", + # postgres + "psycopg2", "psycopg2cffi", "psycopg2cffi-compat", + # saml2 + "pysaml2", + # oidc + "authlib", + # url_preview + "lxml", + # sentry + "sentry-sdk", + # opentracing + "jaeger-client", "opentracing", + # jwt + "pyjwt", + #redis + "txredisapi", "hiredis" + # omitted: + # - cache_memory: this is an experimental option + # - test: it's useful to have this separate from dev deps in the olddeps job + # - systemd: this is a system-based requirement +] + +[tool.poetry.dev-dependencies] +## We pin black so that our tests don't start failing on new releases. +isort = "==5.7.0" +black = "==22.3.0" +flake8-comprehensions = "*" +flake8-bugbear = "==21.3.2" +flake8 = "*" + +# Typechecking +mypy = "==0.931" +mypy-zope = "==0.3.5" +types-bleach = ">=4.1.0" +types-jsonschema = ">=3.2.0" +types-opentracing = ">=2.4.2" +types-Pillow = ">=8.3.4" +types-psycopg2 = ">=2.9.9" +types-pyOpenSSL = ">=20.0.7" +types-PyYAML = ">=5.4.10" +types-requests = ">=2.26.0" +types-setuptools = ">=57.4.0" + +# Dependencies which are exclusively required by unit test code. This is +# NOT a list of all modules that are necessary to run the unit tests. +# Tests assume that all optional dependencies are installed. +# parameterized<0.7.4 can create classes with names that would normally be invalid +# identifiers. trial really does not like this when running with multiple workers. +parameterized = ">=0.7.4" + +# The following are used by the release script +click = "==8.1.0" +redbaron = "==0.9.2" +GitPython = "==3.1.14" +commonmark = "==0.9.1" +pygithub = "==1.55" +# The following are executed as commands by the release script. +twine = "*" +# Towncrier min version comes from #3425. Rationale unclear. +towncrier = ">=18.6.0rc1" + +[build-system] +requires = ["setuptools"] +build-backend = "setuptools.build_meta" \ No newline at end of file From 573cd0f92f78de54cbe5dc9d0bf9f904f9d90011 Mon Sep 17 00:00:00 2001 From: David Robertson Date: Tue, 5 Apr 2022 19:25:56 +0100 Subject: [PATCH 47/55] Add missing dependency on importlib_metadata (#12384) --- changelog.d/12384.misc | 1 + poetry.lock | 2 +- pyproject.toml | 3 +++ synapse/python_dependencies.py | 3 +++ 4 files changed, 8 insertions(+), 1 deletion(-) create mode 100644 changelog.d/12384.misc diff --git a/changelog.d/12384.misc b/changelog.d/12384.misc new file mode 100644 index 0000000000..8a60f32876 --- /dev/null +++ b/changelog.d/12384.misc @@ -0,0 +1 @@ +Make missing `importlib_metadata` dependency explicit. \ No newline at end of file diff --git a/poetry.lock b/poetry.lock index d4bf972c43..17430cbfa6 100644 --- a/poetry.lock +++ b/poetry.lock @@ -1593,7 +1593,7 @@ url_preview = ["lxml"] [metadata] lock-version = "1.1" python-versions = "^3.7" -content-hash = "92465e65bef59c43c6112f332e5cf987740899801dbd34c19245db15b5c6362d" +content-hash = "7ff6d982a9d6675cb595b216b23549ef1942d0e39cb91c97494ff6ed95a9e8d2" [metadata.files] appdirs = [ diff --git a/pyproject.toml b/pyproject.toml index 19ffdc7b2c..92ea302b75 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -167,6 +167,9 @@ ijson = ">=3.1.4" matrix-common = "~=1.1.0" # We need packaging.requirements.Requirement, added in 16.1. packaging = ">=16.1" +# At the time of writing, we only use functions from the version `importlib.metadata` +# which shipped in Python 3.8. This corresponds to version 1.4 of the backport. +importlib_metadata = { version = ">=1.4", python = "<3.8" } # Optional Dependencies diff --git a/synapse/python_dependencies.py b/synapse/python_dependencies.py index 8419ab3aca..cd68aa362e 100644 --- a/synapse/python_dependencies.py +++ b/synapse/python_dependencies.py @@ -89,6 +89,9 @@ REQUIREMENTS = [ "matrix-common~=1.1.0", # We need packaging.requirements.Requirement, added in 16.1. "packaging>=16.1", + # At the time of writing, we only use functions from the version `importlib.metadata` + # which shipped in Python 3.8. This corresponds to version 1.4 of the backport. + "importlib_metadata>=1.4", ] CONDITIONAL_REQUIREMENTS = { From 793d03e2c5c8e688380b8070d53a7e500b7734af Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Wed, 6 Apr 2022 05:40:28 -0500 Subject: [PATCH 48/55] Generate historic pagination token for `/messages` when no `?from` token provided (#12370) --- changelog.d/12370.bugfix | 1 + synapse/handlers/pagination.py | 9 ++++++++- synapse/handlers/room.py | 4 ++-- synapse/storage/databases/main/stream.py | 18 ++++++++++-------- synapse/streams/events.py | 4 ++-- tests/storage/test_stream.py | 4 +++- 6 files changed, 26 insertions(+), 14 deletions(-) create mode 100644 changelog.d/12370.bugfix diff --git a/changelog.d/12370.bugfix b/changelog.d/12370.bugfix new file mode 100644 index 0000000000..97dca97829 --- /dev/null +++ b/changelog.d/12370.bugfix @@ -0,0 +1 @@ +Fix `/messages` returning backfilled and [MSC2716](https://github.com/matrix-org/synapse/pull/12319) historic messages our of order. diff --git a/synapse/handlers/pagination.py b/synapse/handlers/pagination.py index 876b879483..7ee3340373 100644 --- a/synapse/handlers/pagination.py +++ b/synapse/handlers/pagination.py @@ -441,7 +441,14 @@ class PaginationHandler: if pagin_config.from_token: from_token = pagin_config.from_token else: - from_token = self.hs.get_event_sources().get_current_token_for_pagination() + from_token = ( + await self.hs.get_event_sources().get_current_token_for_pagination( + room_id + ) + ) + # We expect `/messages` to use historic pagination tokens by default but + # `/messages` should still works with live tokens when manually provided. + assert from_token.room_key.topological if pagin_config.limit is None: # This shouldn't happen as we've set a default limit before this diff --git a/synapse/handlers/room.py b/synapse/handlers/room.py index 65d4aea9af..b31f00b517 100644 --- a/synapse/handlers/room.py +++ b/synapse/handlers/room.py @@ -1444,8 +1444,8 @@ class RoomEventSource(EventSource[RoomStreamToken, EventBase]): def get_current_key(self) -> RoomStreamToken: return self.store.get_room_max_token() - def get_current_key_for_room(self, room_id: str) -> Awaitable[str]: - return self.store.get_room_events_max_id(room_id) + def get_current_key_for_room(self, room_id: str) -> Awaitable[RoomStreamToken]: + return self.store.get_current_room_stream_token_for_room_id(room_id) class ShutdownRoomResponse(TypedDict): diff --git a/synapse/storage/databases/main/stream.py b/synapse/storage/databases/main/stream.py index 8e764790db..82e9ef02d2 100644 --- a/synapse/storage/databases/main/stream.py +++ b/synapse/storage/databases/main/stream.py @@ -748,21 +748,23 @@ class StreamWorkerStore(EventsWorkerStore, SQLBaseStore): "get_room_event_before_stream_ordering", _f ) - async def get_room_events_max_id(self, room_id: Optional[str] = None) -> str: - """Returns the current token for rooms stream. + async def get_current_room_stream_token_for_room_id( + self, room_id: Optional[str] = None + ) -> RoomStreamToken: + """Returns the current position of the rooms stream. - By default, it returns the current global stream token. Specifying a - `room_id` causes it to return the current room specific topological - token. + By default, it returns a live token with the current global stream + token. Specifying a `room_id` causes it to return a historic token with + the room specific topological token. """ - token = self.get_room_max_stream_ordering() + stream_ordering = self.get_room_max_stream_ordering() if room_id is None: - return "s%d" % (token,) + return RoomStreamToken(None, stream_ordering) else: topo = await self.db_pool.runInteraction( "_get_max_topological_txn", self._get_max_topological_txn, room_id ) - return "t%d-%d" % (topo, token) + return RoomStreamToken(topo, stream_ordering) def get_stream_id_for_event_txn( self, diff --git a/synapse/streams/events.py b/synapse/streams/events.py index fb8fe17295..acf17ba623 100644 --- a/synapse/streams/events.py +++ b/synapse/streams/events.py @@ -69,7 +69,7 @@ class EventSources: ) return token - def get_current_token_for_pagination(self) -> StreamToken: + async def get_current_token_for_pagination(self, room_id: str) -> StreamToken: """Get the current token for a given room to be used to paginate events. @@ -80,7 +80,7 @@ class EventSources: The current token for pagination. """ token = StreamToken( - room_key=self.sources.room.get_current_key(), + room_key=await self.sources.room.get_current_key_for_room(room_id), presence_key=0, typing_key=0, receipt_key=0, diff --git a/tests/storage/test_stream.py b/tests/storage/test_stream.py index 52e41cdab4..78663a53fe 100644 --- a/tests/storage/test_stream.py +++ b/tests/storage/test_stream.py @@ -110,7 +110,9 @@ class PaginationTestCase(HomeserverTestCase): def _filter_messages(self, filter: JsonDict) -> List[EventBase]: """Make a request to /messages with a filter, returns the chunk of events.""" - from_token = self.hs.get_event_sources().get_current_token_for_pagination() + from_token = self.get_success( + self.hs.get_event_sources().get_current_token_for_pagination(self.room_id) + ) events, next_key = self.get_success( self.hs.get_datastores().main.paginate_room_events( From ae01a7edd32c8a4650700294c50a37385fa07984 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Wed, 6 Apr 2022 13:59:04 +0100 Subject: [PATCH 49/55] Update type annotations for compatiblity with prometheus_client 0.14 (#12389) Principally, `prometheus_client.REGISTRY.register` now requires its argument to extend `prometheus_client.Collector`. Additionally, `Gauge.set` is now annotated so that passing `Optional[int]` causes an error. --- changelog.d/12389.misc | 1 + synapse/metrics/__init__.py | 16 +++++----- synapse/metrics/_gc.py | 6 ++-- synapse/metrics/_reactor_metrics.py | 4 ++- synapse/metrics/_types.py | 31 +++++++++++++++++++ synapse/metrics/background_process_metrics.py | 3 +- synapse/metrics/jemalloc.py | 20 +++++++++--- synapse/storage/databases/main/events.py | 4 +-- 8 files changed, 67 insertions(+), 18 deletions(-) create mode 100644 changelog.d/12389.misc create mode 100644 synapse/metrics/_types.py diff --git a/changelog.d/12389.misc b/changelog.d/12389.misc new file mode 100644 index 0000000000..00e3a5d758 --- /dev/null +++ b/changelog.d/12389.misc @@ -0,0 +1 @@ +Update type annotations for compatiblity with prometheus_client 0.14. diff --git a/synapse/metrics/__init__.py b/synapse/metrics/__init__.py index d321946aa2..fffd83546c 100644 --- a/synapse/metrics/__init__.py +++ b/synapse/metrics/__init__.py @@ -1,4 +1,5 @@ # Copyright 2015, 2016 OpenMarket Ltd +# Copyright 2022 The Matrix.org Foundation C.I.C. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -52,12 +53,13 @@ from synapse.metrics._exposition import ( start_http_server, ) from synapse.metrics._gc import MIN_TIME_BETWEEN_GCS, install_gc_manager +from synapse.metrics._types import Collector logger = logging.getLogger(__name__) METRICS_PREFIX = "/_synapse/metrics" -all_gauges: "Dict[str, Union[LaterGauge, InFlightGauge]]" = {} +all_gauges: Dict[str, Collector] = {} HAVE_PROC_SELF_STAT = os.path.exists("/proc/self/stat") @@ -78,11 +80,10 @@ RegistryProxy = cast(CollectorRegistry, _RegistryProxy) @attr.s(slots=True, hash=True, auto_attribs=True) -class LaterGauge: - +class LaterGauge(Collector): name: str desc: str - labels: Optional[Iterable[str]] = attr.ib(hash=False) + labels: Optional[Sequence[str]] = attr.ib(hash=False) # callback: should either return a value (if there are no labels for this metric), # or dict mapping from a label tuple to a value caller: Callable[ @@ -125,7 +126,7 @@ class LaterGauge: MetricsEntry = TypeVar("MetricsEntry") -class InFlightGauge(Generic[MetricsEntry]): +class InFlightGauge(Generic[MetricsEntry], Collector): """Tracks number of things (e.g. requests, Measure blocks, etc) in flight at any given time. @@ -246,7 +247,7 @@ class InFlightGauge(Generic[MetricsEntry]): all_gauges[self.name] = self -class GaugeBucketCollector: +class GaugeBucketCollector(Collector): """Like a Histogram, but the buckets are Gauges which are updated atomically. The data is updated by calling `update_data` with an iterable of measurements. @@ -340,7 +341,7 @@ class GaugeBucketCollector: # -class CPUMetrics: +class CPUMetrics(Collector): def __init__(self) -> None: ticks_per_sec = 100 try: @@ -470,6 +471,7 @@ def register_threadpool(name: str, threadpool: ThreadPool) -> None: __all__ = [ + "Collector", "MetricsResource", "generate_latest", "start_http_server", diff --git a/synapse/metrics/_gc.py b/synapse/metrics/_gc.py index 2bc909efa0..b7d47ce3e7 100644 --- a/synapse/metrics/_gc.py +++ b/synapse/metrics/_gc.py @@ -30,6 +30,8 @@ from prometheus_client.core import ( from twisted.internet import task +from synapse.metrics._types import Collector + """Prometheus metrics for garbage collection""" @@ -71,7 +73,7 @@ gc_time = Histogram( ) -class GCCounts: +class GCCounts(Collector): def collect(self) -> Iterable[Metric]: cm = GaugeMetricFamily("python_gc_counts", "GC object counts", labels=["gen"]) for n, m in enumerate(gc.get_count()): @@ -135,7 +137,7 @@ def install_gc_manager() -> None: # -class PyPyGCStats: +class PyPyGCStats(Collector): def collect(self) -> Iterable[Metric]: # @stats is a pretty-printer object with __str__() returning a nice table, diff --git a/synapse/metrics/_reactor_metrics.py b/synapse/metrics/_reactor_metrics.py index f38f798313..a2c6e6842d 100644 --- a/synapse/metrics/_reactor_metrics.py +++ b/synapse/metrics/_reactor_metrics.py @@ -21,6 +21,8 @@ from prometheus_client.core import REGISTRY, GaugeMetricFamily from twisted.internet import reactor +from synapse.metrics._types import Collector + # # Twisted reactor metrics # @@ -54,7 +56,7 @@ class EpollWrapper: return getattr(self._poller, item) -class ReactorLastSeenMetric: +class ReactorLastSeenMetric(Collector): def __init__(self, epoll_wrapper: EpollWrapper): self._epoll_wrapper = epoll_wrapper diff --git a/synapse/metrics/_types.py b/synapse/metrics/_types.py new file mode 100644 index 0000000000..dc5aa49397 --- /dev/null +++ b/synapse/metrics/_types.py @@ -0,0 +1,31 @@ +# Copyright 2022 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +from abc import ABC, abstractmethod +from typing import Iterable + +from prometheus_client import Metric + +try: + from prometheus_client.registry import Collector +except ImportError: + # prometheus_client.Collector is new as of prometheus 0.14. We redefine it here + # for compatibility with earlier versions. + class _Collector(ABC): + @abstractmethod + def collect(self) -> Iterable[Metric]: + pass + + Collector = _Collector # type: ignore diff --git a/synapse/metrics/background_process_metrics.py b/synapse/metrics/background_process_metrics.py index 53c508af91..f61396bb79 100644 --- a/synapse/metrics/background_process_metrics.py +++ b/synapse/metrics/background_process_metrics.py @@ -46,6 +46,7 @@ from synapse.logging.opentracing import ( noop_context_manager, start_active_span, ) +from synapse.metrics._types import Collector if TYPE_CHECKING: import resource @@ -127,7 +128,7 @@ _background_processes_active_since_last_scrape: "Set[_BackgroundProcess]" = set( _bg_metrics_lock = threading.Lock() -class _Collector: +class _Collector(Collector): """A custom metrics collector for the background process metrics. Ensures that all of the metrics are up-to-date with any in-flight processes diff --git a/synapse/metrics/jemalloc.py b/synapse/metrics/jemalloc.py index 98ed9c0829..6bc329f04a 100644 --- a/synapse/metrics/jemalloc.py +++ b/synapse/metrics/jemalloc.py @@ -16,11 +16,13 @@ import ctypes import logging import os import re -from typing import Iterable, Optional +from typing import Iterable, Optional, overload -from prometheus_client import Metric +from prometheus_client import REGISTRY, Metric +from typing_extensions import Literal -from synapse.metrics import REGISTRY, GaugeMetricFamily +from synapse.metrics import GaugeMetricFamily +from synapse.metrics._types import Collector logger = logging.getLogger(__name__) @@ -59,6 +61,16 @@ def _setup_jemalloc_stats() -> None: jemalloc = ctypes.CDLL(jemalloc_path) + @overload + def _mallctl( + name: str, read: Literal[True] = True, write: Optional[int] = None + ) -> int: + ... + + @overload + def _mallctl(name: str, read: Literal[False], write: Optional[int] = None) -> None: + ... + def _mallctl( name: str, read: bool = True, write: Optional[int] = None ) -> Optional[int]: @@ -134,7 +146,7 @@ def _setup_jemalloc_stats() -> None: except Exception as e: logger.warning("Failed to reload jemalloc stats: %s", e) - class JemallocCollector: + class JemallocCollector(Collector): """Metrics for internal jemalloc stats.""" def collect(self) -> Iterable[Metric]: diff --git a/synapse/storage/databases/main/events.py b/synapse/storage/databases/main/events.py index d253243125..57489c30f1 100644 --- a/synapse/storage/databases/main/events.py +++ b/synapse/storage/databases/main/events.py @@ -200,9 +200,7 @@ class PersistEventsStore: if stream < 0: # backfilled events have negative stream orderings, so we don't # want to set the event_persisted_position to that. - synapse.metrics.event_persisted_position.set( - events_and_contexts[-1][0].internal_metadata.stream_ordering - ) + synapse.metrics.event_persisted_position.set(stream) for event, context in events_and_contexts: if context.app_service: From 6fe757d69ebbc96bf5077db88b1fd3a28c217d11 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Wed, 6 Apr 2022 14:52:39 +0100 Subject: [PATCH 50/55] Fix `synapse_event_persisted_position` metric (#12390) Fixes a bug introduced in #11417 where we would only included backfilled events in `synapse_event_persisted_position` --- changelog.d/12390.bugfix | 1 + synapse/storage/databases/main/events.py | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-) create mode 100644 changelog.d/12390.bugfix diff --git a/changelog.d/12390.bugfix b/changelog.d/12390.bugfix new file mode 100644 index 0000000000..159c6b3c0f --- /dev/null +++ b/changelog.d/12390.bugfix @@ -0,0 +1 @@ +Fix a bug introduced in Synapse 1.49.0 which caused the `synapse_event_persisted_position` metric to have invalid values. diff --git a/synapse/storage/databases/main/events.py b/synapse/storage/databases/main/events.py index 57489c30f1..3fcd5f5b99 100644 --- a/synapse/storage/databases/main/events.py +++ b/synapse/storage/databases/main/events.py @@ -197,9 +197,9 @@ class PersistEventsStore: ) persist_event_counter.inc(len(events_and_contexts)) - if stream < 0: - # backfilled events have negative stream orderings, so we don't - # want to set the event_persisted_position to that. + if not use_negative_stream_ordering: + # we don't want to set the event_persisted_position to a negative + # stream_ordering. synapse.metrics.event_persisted_position.set(stream) for event, context in events_and_contexts: From 2cf74cf2fcde0e00dd583ae626c24ac15e8911bf Mon Sep 17 00:00:00 2001 From: David Robertson Date: Wed, 6 Apr 2022 17:37:56 +0100 Subject: [PATCH 51/55] Narrow the `importlib_metadata` dependency (#12400) --- changelog.d/12400.misc | 1 + synapse/python_dependencies.py | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) create mode 100644 changelog.d/12400.misc diff --git a/changelog.d/12400.misc b/changelog.d/12400.misc new file mode 100644 index 0000000000..8a60f32876 --- /dev/null +++ b/changelog.d/12400.misc @@ -0,0 +1 @@ +Make missing `importlib_metadata` dependency explicit. \ No newline at end of file diff --git a/synapse/python_dependencies.py b/synapse/python_dependencies.py index cd68aa362e..8bd91d5f72 100644 --- a/synapse/python_dependencies.py +++ b/synapse/python_dependencies.py @@ -91,7 +91,7 @@ REQUIREMENTS = [ "packaging>=16.1", # At the time of writing, we only use functions from the version `importlib.metadata` # which shipped in Python 3.8. This corresponds to version 1.4 of the backport. - "importlib_metadata>=1.4", + "importlib_metadata>=1.4 ; python_version < '3.8'", ] CONDITIONAL_REQUIREMENTS = { From 05a37f4008c1f384b2b6bcda9dc8d551e19f6ab1 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 6 Apr 2022 13:27:46 -0400 Subject: [PATCH 52/55] Remove support for the unstable identifier from MSC3288. (#12398) --- changelog.d/12398.misc | 1 + synapse/handlers/identity.py | 2 -- 2 files changed, 1 insertion(+), 2 deletions(-) create mode 100644 changelog.d/12398.misc diff --git a/changelog.d/12398.misc b/changelog.d/12398.misc new file mode 100644 index 0000000000..f94054bdaa --- /dev/null +++ b/changelog.d/12398.misc @@ -0,0 +1 @@ +Remove support for the unstable identifiers specified in [MSC3288](https://github.com/matrix-org/matrix-doc/pull/3288). diff --git a/synapse/handlers/identity.py b/synapse/handlers/identity.py index 57c9fdfe62..c183e9c465 100644 --- a/synapse/handlers/identity.py +++ b/synapse/handlers/identity.py @@ -858,8 +858,6 @@ class IdentityHandler: if room_type is not None: invite_config["room_type"] = room_type - # TODO The unstable field is deprecated and should be removed in the future. - invite_config["org.matrix.msc3288.room_type"] = room_type # If a custom web client location is available, include it in the request. if self._web_client_location: From 6902e9ff2bee4b6e0c2424b88bdac607a2e6b440 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 6 Apr 2022 13:41:14 -0400 Subject: [PATCH 53/55] Remove docs for converting a legacy structured logging config. (#12392) And update the upgrade notes to point at old versions of the documentation. --- changelog.d/12392.doc | 1 + docs/structured_logging.md | 79 -------------------------------------- docs/upgrade.md | 4 +- 3 files changed, 3 insertions(+), 81 deletions(-) create mode 100644 changelog.d/12392.doc diff --git a/changelog.d/12392.doc b/changelog.d/12392.doc new file mode 100644 index 0000000000..6a2f522bf6 --- /dev/null +++ b/changelog.d/12392.doc @@ -0,0 +1 @@ +Remove documentation for converting a legacy structured logging configuration to the new format. diff --git a/docs/structured_logging.md b/docs/structured_logging.md index 805c867653..a6667e1a11 100644 --- a/docs/structured_logging.md +++ b/docs/structured_logging.md @@ -78,82 +78,3 @@ loggers: The above logging config will set Synapse as 'INFO' logging level by default, with the SQL layer at 'WARNING', and will log JSON formatted messages to a remote endpoint at 10.1.2.3:9999. - -## Upgrading from legacy structured logging configuration - -Versions of Synapse prior to v1.54.0 automatically converted the legacy -structured logging configuration, which was deprecated in v1.23.0, to the standard -library logging configuration. - -The following reference can be used to update your configuration. Based on the -drain `type`, we can pick a new handler: - -1. For a type of `console`, `console_json`, or `console_json_terse`: a handler - with a class of `logging.StreamHandler` and a `stream` of `ext://sys.stdout` - or `ext://sys.stderr` should be used. -2. For a type of `file` or `file_json`: a handler of `logging.FileHandler` with - a location of the file path should be used. -3. For a type of `network_json_terse`: a handler of `synapse.logging.RemoteHandler` - with the host and port should be used. - -Then based on the drain `type` we can pick a new formatter: - -1. For a type of `console` or `file` no formatter is necessary. -2. For a type of `console_json` or `file_json`: a formatter of - `synapse.logging.JsonFormatter` should be used. -3. For a type of `console_json_terse` or `network_json_terse`: a formatter of - `synapse.logging.TerseJsonFormatter` should be used. - -For each new handler and formatter they should be added to the logging configuration -and then assigned to either a logger or the root logger. - -An example legacy configuration: - -```yaml -structured: true - -loggers: - synapse: - level: INFO - synapse.storage.SQL: - level: WARNING - -drains: - console: - type: console - location: stdout - file: - type: file_json - location: homeserver.log -``` - -Would be converted into a new configuration: - -```yaml -version: 1 - -formatters: - json: - class: synapse.logging.JsonFormatter - -handlers: - console: - class: logging.StreamHandler - stream: ext://sys.stdout - file: - class: logging.FileHandler - formatter: json - filename: homeserver.log - -loggers: - synapse: - level: INFO - handlers: [console, file] - synapse.storage.SQL: - level: WARNING -``` - -The new logging configuration is a bit more verbose, but significantly more -flexible. It allows for configuration that were not previously possible, such as -sending plain logs over the network, or using different handlers for different -modules. diff --git a/docs/upgrade.md b/docs/upgrade.md index de3444930e..f0c0a3998d 100644 --- a/docs/upgrade.md +++ b/docs/upgrade.md @@ -160,7 +160,7 @@ please upgrade Mjolnir to version 1.3.2 or later before upgrading Synapse. This release removes support for the `structured: true` logging configuration which was deprecated in Synapse v1.23.0. If your logging configuration contains `structured: true` then it should be modified based on the -[structured logging documentation](structured_logging.md). +[structured logging documentation](https://matrix-org.github.io/synapse/v1.56/structured_logging.html#upgrading-from-legacy-structured-logging-configuration). # Upgrading to v1.53.0 @@ -777,7 +777,7 @@ lock down external access to the Admin API endpoints. This release deprecates use of the `structured: true` logging configuration for structured logging. If your logging configuration contains `structured: true` then it should be modified based on the -[structured logging documentation](structured_logging.md). +[structured logging documentation](https://matrix-org.github.io/synapse/v1.56/structured_logging.html#upgrading-from-legacy-structured-logging-configuration). The `structured` and `drains` logging options are now deprecated and should be replaced by standard logging configuration of `handlers` and From f931c0602a1dcfd40696c53779d80dc68c9c9e17 Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Wed, 6 Apr 2022 13:41:15 -0500 Subject: [PATCH 54/55] Update changelog to describe changes from #12370 relative to `develop` (#12401) --- changelog.d/12370.bugfix | 1 - changelog.d/12370.feature | 1 + 2 files changed, 1 insertion(+), 1 deletion(-) delete mode 100644 changelog.d/12370.bugfix create mode 100644 changelog.d/12370.feature diff --git a/changelog.d/12370.bugfix b/changelog.d/12370.bugfix deleted file mode 100644 index 97dca97829..0000000000 --- a/changelog.d/12370.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix `/messages` returning backfilled and [MSC2716](https://github.com/matrix-org/synapse/pull/12319) historic messages our of order. diff --git a/changelog.d/12370.feature b/changelog.d/12370.feature new file mode 100644 index 0000000000..2c695ddaa0 --- /dev/null +++ b/changelog.d/12370.feature @@ -0,0 +1 @@ +Update `/messages` to use historic pagination tokens if no `from` query parameter is given. From 350062661cbc402c089ce7a20b72d83073ed8fec Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Wed, 6 Apr 2022 16:34:33 -0500 Subject: [PATCH 55/55] Clarify that we mark as outliers because we don't have any state for them (#12345) As discussed at https://github.com/matrix-org/synapse/pull/12179#discussion_r837263852 --- changelog.d/12345.doc | 1 + docs/development/room-dag-concepts.md | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) create mode 100644 changelog.d/12345.doc diff --git a/changelog.d/12345.doc b/changelog.d/12345.doc new file mode 100644 index 0000000000..0dcf836e68 --- /dev/null +++ b/changelog.d/12345.doc @@ -0,0 +1 @@ +Updates to the Room DAG concepts development document to clarify that we mark events as outliers because we don't have any state for them. diff --git a/docs/development/room-dag-concepts.md b/docs/development/room-dag-concepts.md index 3eb4d5acc4..76709487f8 100644 --- a/docs/development/room-dag-concepts.md +++ b/docs/development/room-dag-concepts.md @@ -39,7 +39,8 @@ yet correlated to the DAG. Outliers typically arise when we fetch the auth chain or state for a given event. When that happens, we just grab the events in the state/auth chain, without calculating the state at those events, or backfilling their -`prev_events`. +`prev_events`. Since we don't have the state at any events fetched in that +way, we mark them as outliers. So, typically, we won't have the `prev_events` of an `outlier` in the database, (though it's entirely possible that we *might* have them for some other