Compare commits
4 Commits
v0.1
...
anoa/user_
| Author | SHA1 | Date | |
|---|---|---|---|
|
6f98bc6512 | ||
|
|
50a63d5fda | ||
|
|
280bfb15ce | ||
|
|
fff89c33d8 |
@@ -117,10 +117,8 @@ steps:
|
||||
limit: 2
|
||||
|
||||
- label: ":python: 3.5 / :postgres: 9.5"
|
||||
agents:
|
||||
queue: "medium"
|
||||
env:
|
||||
TRIAL_FLAGS: "-j 8"
|
||||
TRIAL_FLAGS: "-j 4"
|
||||
command:
|
||||
- "bash -c 'python -m pip install tox && python -m tox -e py35-postgres,codecov'"
|
||||
plugins:
|
||||
@@ -136,10 +134,8 @@ steps:
|
||||
limit: 2
|
||||
|
||||
- label: ":python: 3.7 / :postgres: 9.5"
|
||||
agents:
|
||||
queue: "medium"
|
||||
env:
|
||||
TRIAL_FLAGS: "-j 8"
|
||||
TRIAL_FLAGS: "-j 4"
|
||||
command:
|
||||
- "bash -c 'python -m pip install tox && python -m tox -e py37-postgres,codecov'"
|
||||
plugins:
|
||||
@@ -155,10 +151,8 @@ steps:
|
||||
limit: 2
|
||||
|
||||
- label: ":python: 3.7 / :postgres: 11"
|
||||
agents:
|
||||
queue: "medium"
|
||||
env:
|
||||
TRIAL_FLAGS: "-j 8"
|
||||
TRIAL_FLAGS: "-j 4"
|
||||
command:
|
||||
- "bash -c 'python -m pip install tox && python -m tox -e py37-postgres,codecov'"
|
||||
plugins:
|
||||
@@ -220,10 +214,8 @@ steps:
|
||||
env:
|
||||
POSTGRES: "1"
|
||||
WORKERS: "1"
|
||||
BLACKLIST: "synapse-blacklist-with-workers"
|
||||
command:
|
||||
- "bash .buildkite/merge_base_branch.sh"
|
||||
- "bash -c 'cat /src/sytest-blacklist /src/.buildkite/worker-blacklist > /src/synapse-blacklist-with-workers'"
|
||||
- "bash /synapse_sytest.sh"
|
||||
plugins:
|
||||
- docker#v3.0.1:
|
||||
@@ -231,6 +223,7 @@ steps:
|
||||
propagate-environment: true
|
||||
always-pull: true
|
||||
workdir: "/src"
|
||||
soft_fail: true
|
||||
retry:
|
||||
automatic:
|
||||
- exit_status: -1
|
||||
|
||||
@@ -1,34 +0,0 @@
|
||||
# This file serves as a blacklist for SyTest tests that we expect will fail in
|
||||
# Synapse when run under worker mode. For more details, see sytest-blacklist.
|
||||
|
||||
Message history can be paginated
|
||||
|
||||
m.room.history_visibility == "world_readable" allows/forbids appropriately for Guest users
|
||||
|
||||
m.room.history_visibility == "world_readable" allows/forbids appropriately for Real users
|
||||
|
||||
Can re-join room if re-invited
|
||||
|
||||
/upgrade creates a new room
|
||||
|
||||
The only membership state included in an initial sync is for all the senders in the timeline
|
||||
|
||||
Local device key changes get to remote servers
|
||||
|
||||
If remote user leaves room we no longer receive device updates
|
||||
|
||||
Forgotten room messages cannot be paginated
|
||||
|
||||
Inbound federation can get public room list
|
||||
|
||||
Members from the gap are included in gappy incr LL sync
|
||||
|
||||
Leaves are present in non-gapped incremental syncs
|
||||
|
||||
Old leaves are present in gapped incremental syncs
|
||||
|
||||
User sees updates to presence from other users in the incremental sync.
|
||||
|
||||
Gapped incremental syncs include all state changes
|
||||
|
||||
Old members are included in gappy incr LL sync if they start speaking
|
||||
122
CHANGES.md
122
CHANGES.md
@@ -1,125 +1,3 @@
|
||||
Synapse 1.2.1 (2019-07-26)
|
||||
==========================
|
||||
|
||||
Security update
|
||||
---------------
|
||||
|
||||
This release includes *four* security fixes:
|
||||
|
||||
- Prevent an attack where a federated server could send redactions for arbitrary events in v1 and v2 rooms. ([\#5767](https://github.com/matrix-org/synapse/issues/5767))
|
||||
- Prevent a denial-of-service attack where cycles of redaction events would make Synapse spin infinitely. Thanks to `@lrizika:matrix.org` for identifying and responsibly disclosing this issue. ([0f2ecb961](https://github.com/matrix-org/synapse/commit/0f2ecb961))
|
||||
- Prevent an attack where users could be joined or parted from public rooms without their consent. Thanks to @dylangerdaly for identifying and responsibly disclosing this issue. ([\#5744](https://github.com/matrix-org/synapse/issues/5744))
|
||||
- Fix a vulnerability where a federated server could spoof read-receipts from
|
||||
users on other servers. Thanks to @dylangerdaly for identifying this issue too. ([\#5743](https://github.com/matrix-org/synapse/issues/5743))
|
||||
|
||||
Additionally, the following fix was in Synapse **1.2.0**, but was not correctly
|
||||
identified during the original release:
|
||||
|
||||
- It was possible for a room moderator to send a redaction for an `m.room.create` event, which would downgrade the room to version 1. Thanks to `/dev/ponies` for identifying and responsibly disclosing this issue! ([\#5701](https://github.com/matrix-org/synapse/issues/5701))
|
||||
|
||||
Synapse 1.2.0 (2019-07-25)
|
||||
==========================
|
||||
|
||||
No significant changes.
|
||||
|
||||
|
||||
Synapse 1.2.0rc2 (2019-07-24)
|
||||
=============================
|
||||
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Fix a regression introduced in v1.2.0rc1 which led to incorrect labels on some prometheus metrics. ([\#5734](https://github.com/matrix-org/synapse/issues/5734))
|
||||
|
||||
|
||||
Synapse 1.2.0rc1 (2019-07-22)
|
||||
=============================
|
||||
|
||||
Security fixes
|
||||
--------------
|
||||
|
||||
This update included a security fix which was initially incorrectly flagged as
|
||||
a regular bug fix.
|
||||
|
||||
- It was possible for a room moderator to send a redaction for an `m.room.create` event, which would downgrade the room to version 1. Thanks to `/dev/ponies` for identifying and responsibly disclosing this issue! ([\#5701](https://github.com/matrix-org/synapse/issues/5701))
|
||||
|
||||
Features
|
||||
--------
|
||||
|
||||
- Add support for opentracing. ([\#5544](https://github.com/matrix-org/synapse/issues/5544), [\#5712](https://github.com/matrix-org/synapse/issues/5712))
|
||||
- Add ability to pull all locally stored events out of synapse that a particular user can see. ([\#5589](https://github.com/matrix-org/synapse/issues/5589))
|
||||
- Add a basic admin command app to allow server operators to run Synapse admin commands separately from the main production instance. ([\#5597](https://github.com/matrix-org/synapse/issues/5597))
|
||||
- Add `sender` and `origin_server_ts` fields to `m.replace`. ([\#5613](https://github.com/matrix-org/synapse/issues/5613))
|
||||
- Add default push rule to ignore reactions. ([\#5623](https://github.com/matrix-org/synapse/issues/5623))
|
||||
- Include the original event when asking for its relations. ([\#5626](https://github.com/matrix-org/synapse/issues/5626))
|
||||
- Implement `session_lifetime` configuration option, after which access tokens will expire. ([\#5660](https://github.com/matrix-org/synapse/issues/5660))
|
||||
- Return "This account has been deactivated" when a deactivated user tries to login. ([\#5674](https://github.com/matrix-org/synapse/issues/5674))
|
||||
- Enable aggregations support by default ([\#5714](https://github.com/matrix-org/synapse/issues/5714))
|
||||
|
||||
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Fix 'utime went backwards' errors on daemonization. ([\#5609](https://github.com/matrix-org/synapse/issues/5609))
|
||||
- Various minor fixes to the federation request rate limiter. ([\#5621](https://github.com/matrix-org/synapse/issues/5621))
|
||||
- Forbid viewing relations on an event once it has been redacted. ([\#5629](https://github.com/matrix-org/synapse/issues/5629))
|
||||
- Fix requests to the `/store_invite` endpoint of identity servers being sent in the wrong format. ([\#5638](https://github.com/matrix-org/synapse/issues/5638))
|
||||
- Fix newly-registered users not being able to lookup their own profile without joining a room. ([\#5644](https://github.com/matrix-org/synapse/issues/5644))
|
||||
- Fix bug in #5626 that prevented the original_event field from actually having the contents of the original event in a call to `/relations`. ([\#5654](https://github.com/matrix-org/synapse/issues/5654))
|
||||
- Fix 3PID bind requests being sent to identity servers as `application/x-form-www-urlencoded` data, which is deprecated. ([\#5658](https://github.com/matrix-org/synapse/issues/5658))
|
||||
- Fix some problems with authenticating redactions in recent room versions. ([\#5699](https://github.com/matrix-org/synapse/issues/5699), [\#5700](https://github.com/matrix-org/synapse/issues/5700), [\#5707](https://github.com/matrix-org/synapse/issues/5707))
|
||||
|
||||
|
||||
Updates to the Docker image
|
||||
---------------------------
|
||||
|
||||
- Base Docker image on a newer Alpine Linux version (3.8 -> 3.10). ([\#5619](https://github.com/matrix-org/synapse/issues/5619))
|
||||
- Add missing space in default logging file format generated by the Docker image. ([\#5620](https://github.com/matrix-org/synapse/issues/5620))
|
||||
|
||||
|
||||
Improved Documentation
|
||||
----------------------
|
||||
|
||||
- Add information about nginx normalisation to reverse_proxy.rst. Contributed by @skalarproduktraum - thanks! ([\#5397](https://github.com/matrix-org/synapse/issues/5397))
|
||||
- --no-pep517 should be --no-use-pep517 in the documentation to setup the development environment. ([\#5651](https://github.com/matrix-org/synapse/issues/5651))
|
||||
- Improvements to Postgres setup instructions. Contributed by @Lrizika - thanks! ([\#5661](https://github.com/matrix-org/synapse/issues/5661))
|
||||
- Minor tweaks to postgres documentation. ([\#5675](https://github.com/matrix-org/synapse/issues/5675))
|
||||
|
||||
|
||||
Deprecations and Removals
|
||||
-------------------------
|
||||
|
||||
- Remove support for the `invite_3pid_guest` configuration setting. ([\#5625](https://github.com/matrix-org/synapse/issues/5625))
|
||||
|
||||
|
||||
Internal Changes
|
||||
----------------
|
||||
|
||||
- Move logging code out of `synapse.util` and into `synapse.logging`. ([\#5606](https://github.com/matrix-org/synapse/issues/5606), [\#5617](https://github.com/matrix-org/synapse/issues/5617))
|
||||
- Add a blacklist file to the repo to blacklist certain sytests from failing CI. ([\#5611](https://github.com/matrix-org/synapse/issues/5611))
|
||||
- Make runtime errors surrounding password reset emails much clearer. ([\#5616](https://github.com/matrix-org/synapse/issues/5616))
|
||||
- Remove dead code for persiting outgoing federation transactions. ([\#5622](https://github.com/matrix-org/synapse/issues/5622))
|
||||
- Add `lint.sh` to the scripts-dev folder which will run all linting steps required by CI. ([\#5627](https://github.com/matrix-org/synapse/issues/5627))
|
||||
- Move RegistrationHandler.get_or_create_user to test code. ([\#5628](https://github.com/matrix-org/synapse/issues/5628))
|
||||
- Add some more common python virtual-environment paths to the black exclusion list. ([\#5630](https://github.com/matrix-org/synapse/issues/5630))
|
||||
- Some counter metrics exposed over Prometheus have been renamed, with the old names preserved for backwards compatibility and deprecated. See `docs/metrics-howto.rst` for details. ([\#5636](https://github.com/matrix-org/synapse/issues/5636))
|
||||
- Unblacklist some user_directory sytests. ([\#5637](https://github.com/matrix-org/synapse/issues/5637))
|
||||
- Factor out some redundant code in the login implementation. ([\#5639](https://github.com/matrix-org/synapse/issues/5639))
|
||||
- Update ModuleApi to avoid register(generate_token=True). ([\#5640](https://github.com/matrix-org/synapse/issues/5640))
|
||||
- Remove access-token support from `RegistrationHandler.register`, and rename it. ([\#5641](https://github.com/matrix-org/synapse/issues/5641))
|
||||
- Remove access-token support from `RegistrationStore.register`, and rename it. ([\#5642](https://github.com/matrix-org/synapse/issues/5642))
|
||||
- Improve logging for auto-join when a new user is created. ([\#5643](https://github.com/matrix-org/synapse/issues/5643))
|
||||
- Remove unused and unnecessary check for FederationDeniedError in _exception_to_failure. ([\#5645](https://github.com/matrix-org/synapse/issues/5645))
|
||||
- Fix a small typo in a code comment. ([\#5655](https://github.com/matrix-org/synapse/issues/5655))
|
||||
- Clean up exception handling around client access tokens. ([\#5656](https://github.com/matrix-org/synapse/issues/5656))
|
||||
- Add a mechanism for per-test homeserver configuration in the unit tests. ([\#5657](https://github.com/matrix-org/synapse/issues/5657))
|
||||
- Inline issue_access_token. ([\#5659](https://github.com/matrix-org/synapse/issues/5659))
|
||||
- Update the sytest BuildKite configuration to checkout Synapse in `/src`. ([\#5664](https://github.com/matrix-org/synapse/issues/5664))
|
||||
- Add a `docker` type to the towncrier configuration. ([\#5673](https://github.com/matrix-org/synapse/issues/5673))
|
||||
- Convert `synapse.federation.transport.server` to `async`. Might improve some stack traces. ([\#5689](https://github.com/matrix-org/synapse/issues/5689))
|
||||
- Documentation for opentracing. ([\#5703](https://github.com/matrix-org/synapse/issues/5703))
|
||||
|
||||
|
||||
Synapse 1.1.0 (2019-07-04)
|
||||
==========================
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@ include demo/README
|
||||
include demo/demo.tls.dh
|
||||
include demo/*.py
|
||||
include demo/*.sh
|
||||
include sytest-blacklist
|
||||
|
||||
recursive-include synapse/storage/schema *.sql
|
||||
recursive-include synapse/storage/schema *.sql.postgres
|
||||
@@ -33,7 +34,6 @@ exclude Dockerfile
|
||||
exclude .dockerignore
|
||||
exclude test_postgresql.sh
|
||||
exclude .editorconfig
|
||||
exclude sytest-blacklist
|
||||
|
||||
include pyproject.toml
|
||||
recursive-include changelog.d *
|
||||
|
||||
@@ -49,13 +49,6 @@ returned by the Client-Server API:
|
||||
# configured on port 443.
|
||||
curl -kv https://<host.name>/_matrix/client/versions 2>&1 | grep "Server:"
|
||||
|
||||
Upgrading to v1.2.0
|
||||
===================
|
||||
|
||||
Some counter metrics have been renamed, with the old names deprecated. See
|
||||
`the metrics documentation <docs/metrics-howto.rst#renaming-of-metrics--deprecation-of-old-names-in-12>`_
|
||||
for details.
|
||||
|
||||
Upgrading to v1.1.0
|
||||
===================
|
||||
|
||||
|
||||
1
changelog.d/5397.doc
Normal file
1
changelog.d/5397.doc
Normal file
@@ -0,0 +1 @@
|
||||
Add information about nginx normalisation to reverse_proxy.rst. Contributed by @skalarproduktraum - thanks!
|
||||
1
changelog.d/5544.misc
Normal file
1
changelog.d/5544.misc
Normal file
@@ -0,0 +1 @@
|
||||
Added opentracing and configuration options.
|
||||
1
changelog.d/5589.feature
Normal file
1
changelog.d/5589.feature
Normal file
@@ -0,0 +1 @@
|
||||
Add ability to pull all locally stored events out of synapse that a particular user can see.
|
||||
1
changelog.d/5606.misc
Normal file
1
changelog.d/5606.misc
Normal file
@@ -0,0 +1 @@
|
||||
Move logging code out of `synapse.util` and into `synapse.logging`.
|
||||
1
changelog.d/5609.bugfix
Normal file
1
changelog.d/5609.bugfix
Normal file
@@ -0,0 +1 @@
|
||||
Fix 'utime went backwards' errors on daemonization.
|
||||
1
changelog.d/5611.misc
Normal file
1
changelog.d/5611.misc
Normal file
@@ -0,0 +1 @@
|
||||
Add a blacklist file to the repo to blacklist certain sytests from failing CI.
|
||||
1
changelog.d/5613.feature
Normal file
1
changelog.d/5613.feature
Normal file
@@ -0,0 +1 @@
|
||||
Add `sender` and `origin_server_ts` fields to `m.replace`.
|
||||
1
changelog.d/5616.misc
Normal file
1
changelog.d/5616.misc
Normal file
@@ -0,0 +1 @@
|
||||
Make runtime errors surrounding password reset emails much clearer.
|
||||
1
changelog.d/5617.misc
Normal file
1
changelog.d/5617.misc
Normal file
@@ -0,0 +1 @@
|
||||
Move logging code out of `synapse.util` and into `synapse.logging`.
|
||||
1
changelog.d/5619.docker
Normal file
1
changelog.d/5619.docker
Normal file
@@ -0,0 +1 @@
|
||||
Base Docker image on a newer Alpine Linux version (3.8 -> 3.10).
|
||||
1
changelog.d/5620.docker
Normal file
1
changelog.d/5620.docker
Normal file
@@ -0,0 +1 @@
|
||||
Add missing space in default logging file format generated by the Docker image.
|
||||
1
changelog.d/5621.bugfix
Normal file
1
changelog.d/5621.bugfix
Normal file
@@ -0,0 +1 @@
|
||||
Various minor fixes to the federation request rate limiter.
|
||||
1
changelog.d/5622.misc
Normal file
1
changelog.d/5622.misc
Normal file
@@ -0,0 +1 @@
|
||||
Remove dead code for persiting outgoing federation transactions.
|
||||
1
changelog.d/5623.feature
Normal file
1
changelog.d/5623.feature
Normal file
@@ -0,0 +1 @@
|
||||
Add default push rule to ignore reactions.
|
||||
1
changelog.d/5625.removal
Normal file
1
changelog.d/5625.removal
Normal file
@@ -0,0 +1 @@
|
||||
Remove support for the `invite_3pid_guest` configuration setting.
|
||||
1
changelog.d/5626.feature
Normal file
1
changelog.d/5626.feature
Normal file
@@ -0,0 +1 @@
|
||||
Include the original event when asking for its relations.
|
||||
1
changelog.d/5627.misc
Normal file
1
changelog.d/5627.misc
Normal file
@@ -0,0 +1 @@
|
||||
Add `lint.sh` to the scripts-dev folder which will run all linting steps required by CI.
|
||||
1
changelog.d/5628.misc
Normal file
1
changelog.d/5628.misc
Normal file
@@ -0,0 +1 @@
|
||||
Move RegistrationHandler.get_or_create_user to test code.
|
||||
1
changelog.d/5630.misc
Normal file
1
changelog.d/5630.misc
Normal file
@@ -0,0 +1 @@
|
||||
Add some more common python virtual-environment paths to the black exclusion list.
|
||||
1
changelog.d/5637.misc
Normal file
1
changelog.d/5637.misc
Normal file
@@ -0,0 +1 @@
|
||||
Unblacklist some user_directory sytests.
|
||||
1
changelog.d/5638.bugfix
Normal file
1
changelog.d/5638.bugfix
Normal file
@@ -0,0 +1 @@
|
||||
Fix requests to the `/store_invite` endpoint of identity servers being sent in the wrong format.
|
||||
1
changelog.d/5639.misc
Normal file
1
changelog.d/5639.misc
Normal file
@@ -0,0 +1 @@
|
||||
Factor out some redundant code in the login implementation.
|
||||
1
changelog.d/5640.misc
Normal file
1
changelog.d/5640.misc
Normal file
@@ -0,0 +1 @@
|
||||
Update ModuleApi to avoid register(generate_token=True).
|
||||
1
changelog.d/5641.misc
Normal file
1
changelog.d/5641.misc
Normal file
@@ -0,0 +1 @@
|
||||
Remove access-token support from RegistrationHandler.register, and rename it.
|
||||
1
changelog.d/5642.misc
Normal file
1
changelog.d/5642.misc
Normal file
@@ -0,0 +1 @@
|
||||
Remove access-token support from `RegistrationStore.register`, and rename it.
|
||||
1
changelog.d/5643.misc
Normal file
1
changelog.d/5643.misc
Normal file
@@ -0,0 +1 @@
|
||||
Improve logging for auto-join when a new user is created.
|
||||
1
changelog.d/5644.bugfix
Normal file
1
changelog.d/5644.bugfix
Normal file
@@ -0,0 +1 @@
|
||||
Fix newly-registered users not being able to lookup their own profile without joining a room.
|
||||
1
changelog.d/5645.misc
Normal file
1
changelog.d/5645.misc
Normal file
@@ -0,0 +1 @@
|
||||
Remove unused and unnecessary check for FederationDeniedError in _exception_to_failure.
|
||||
1
changelog.d/5651.doc
Normal file
1
changelog.d/5651.doc
Normal file
@@ -0,0 +1 @@
|
||||
--no-pep517 should be --no-use-pep517 in the documentation to setup the development environment.
|
||||
1
changelog.d/5654.bugfix
Normal file
1
changelog.d/5654.bugfix
Normal file
@@ -0,0 +1 @@
|
||||
Fix bug in #5626 that prevented the original_event field from actually having the contents of the original event in a call to `/relations`.
|
||||
1
changelog.d/5655.misc
Normal file
1
changelog.d/5655.misc
Normal file
@@ -0,0 +1 @@
|
||||
Fix a small typo in a code comment.
|
||||
1
changelog.d/5656.misc
Normal file
1
changelog.d/5656.misc
Normal file
@@ -0,0 +1 @@
|
||||
Clean up exception handling around client access tokens.
|
||||
1
changelog.d/5657.misc
Normal file
1
changelog.d/5657.misc
Normal file
@@ -0,0 +1 @@
|
||||
Add a mechanism for per-test homeserver configuration in the unit tests.
|
||||
1
changelog.d/5658.bugfix
Normal file
1
changelog.d/5658.bugfix
Normal file
@@ -0,0 +1 @@
|
||||
Fix 3PID bind requests being sent to identity servers as `application/x-form-www-urlencoded` data, which is deprecated.
|
||||
1
changelog.d/5659.misc
Normal file
1
changelog.d/5659.misc
Normal file
@@ -0,0 +1 @@
|
||||
Inline issue_access_token.
|
||||
1
changelog.d/5660.feature
Normal file
1
changelog.d/5660.feature
Normal file
@@ -0,0 +1 @@
|
||||
Implement `session_lifetime` configuration option, after which access tokens will expire.
|
||||
1
changelog.d/5661.doc
Normal file
1
changelog.d/5661.doc
Normal file
@@ -0,0 +1 @@
|
||||
Improvements to Postgres setup instructions. Contributed by @Lrizika - thanks!
|
||||
1
changelog.d/5664.misc
Normal file
1
changelog.d/5664.misc
Normal file
@@ -0,0 +1 @@
|
||||
Update the sytest BuildKite configuration to checkout Synapse in `/src`.
|
||||
1
changelog.d/5673.misc
Normal file
1
changelog.d/5673.misc
Normal file
@@ -0,0 +1 @@
|
||||
Add a `docker` type to the towncrier configuration.
|
||||
1
changelog.d/5674.feature
Normal file
1
changelog.d/5674.feature
Normal file
@@ -0,0 +1 @@
|
||||
Return "This account has been deactivated" when a deactivated user tries to login.
|
||||
@@ -1 +0,0 @@
|
||||
Synapse now no longer accepts the `-v`/`--verbose`, `-f`/`--log-file`, or `--log-config` command line flags, and removes the deprecated `verbose` and `log_file` configuration file options. Users of these options should migrate their options into the dedicated log configuration.
|
||||
1
changelog.d/5686.feature
Normal file
1
changelog.d/5686.feature
Normal file
@@ -0,0 +1 @@
|
||||
Use `M_USER_DEACTIVATED` instead of `M_UNKNOWN` for errcode when a deactivated user attempts to login.
|
||||
@@ -1 +0,0 @@
|
||||
Make Jaeger fully configurable.
|
||||
@@ -1 +0,0 @@
|
||||
Add precautionary measures to prevent future abuse of `window.opener` in default welcome page.
|
||||
@@ -1 +0,0 @@
|
||||
Reduce database IO usage by optimising queries for current membership.
|
||||
@@ -1 +0,0 @@
|
||||
Improve caching when fetching `get_filtered_current_state_ids`.
|
||||
@@ -1 +0,0 @@
|
||||
Don't accept opentracing data from clients.
|
||||
@@ -1 +0,0 @@
|
||||
Speed up PostgreSQL unit tests in CI.
|
||||
@@ -1 +0,0 @@
|
||||
Update the coding style document.
|
||||
@@ -1 +0,0 @@
|
||||
Improve database query performance when recording retry intervals for remote hosts.
|
||||
@@ -1 +0,0 @@
|
||||
Add a set of opentracing utils.
|
||||
@@ -1 +0,0 @@
|
||||
Fix stack overflow in server key lookup code.
|
||||
@@ -1 +0,0 @@
|
||||
start.sh no longer uses deprecated cli option.
|
||||
@@ -1 +0,0 @@
|
||||
Synapse now no longer accepts the `-v`/`--verbose`, `-f`/`--log-file`, or `--log-config` command line flags, and removes the deprecated `verbose` and `log_file` configuration file options. Users of these options should migrate their options into the dedicated log configuration.
|
||||
@@ -1 +0,0 @@
|
||||
Cache result of get_version_string to reduce overhead of `/version` federation requests.
|
||||
@@ -1 +0,0 @@
|
||||
Return 'user_type' in admin API user endpoints results.
|
||||
@@ -1 +0,0 @@
|
||||
Add sd_notify hooks to ease systemd integration and allows usage of Type=Notify.
|
||||
@@ -1 +0,0 @@
|
||||
Don't package the sytest test blacklist file.
|
||||
@@ -1 +0,0 @@
|
||||
Replace uses of returnValue with plain return, as returnValue is not needed on Python 3.
|
||||
@@ -1 +0,0 @@
|
||||
Reduce database IO usage by optimising queries for current membership.
|
||||
@@ -1 +0,0 @@
|
||||
Blacklist some flakey tests in worker mode.
|
||||
@@ -1 +0,0 @@
|
||||
Log when we receive an event receipt from an unexpected origin.
|
||||
@@ -1 +0,0 @@
|
||||
Fix some error cases in the caching layer.
|
||||
@@ -1 +0,0 @@
|
||||
Add a prometheus metric for pending cache lookups.
|
||||
@@ -1 +0,0 @@
|
||||
Stop trying to fetch events with event_id=None.
|
||||
@@ -1 +0,0 @@
|
||||
Convert RedactionTestCase to modern test style.
|
||||
@@ -1 +0,0 @@
|
||||
Allow looping calls to be given arguments.
|
||||
@@ -4,8 +4,7 @@ After=matrix-synapse.service
|
||||
BindsTo=matrix-synapse.service
|
||||
|
||||
[Service]
|
||||
Type=notify
|
||||
NotifyAccess=main
|
||||
Type=simple
|
||||
User=matrix-synapse
|
||||
WorkingDirectory=/var/lib/matrix-synapse
|
||||
EnvironmentFile=/etc/default/matrix-synapse
|
||||
|
||||
@@ -2,8 +2,7 @@
|
||||
Description=Synapse Matrix Homeserver
|
||||
|
||||
[Service]
|
||||
Type=notify
|
||||
NotifyAccess=main
|
||||
Type=simple
|
||||
User=matrix-synapse
|
||||
WorkingDirectory=/var/lib/matrix-synapse
|
||||
EnvironmentFile=/etc/default/matrix-synapse
|
||||
|
||||
@@ -14,9 +14,7 @@
|
||||
Description=Synapse Matrix homeserver
|
||||
|
||||
[Service]
|
||||
Type=notify
|
||||
NotifyAccess=main
|
||||
ExecReload=/bin/kill -HUP $MAINPID
|
||||
Type=simple
|
||||
Restart=on-abort
|
||||
|
||||
User=synapse
|
||||
|
||||
16
debian/changelog
vendored
16
debian/changelog
vendored
@@ -1,21 +1,9 @@
|
||||
matrix-synapse-py3 (1.2.1) stable; urgency=medium
|
||||
|
||||
* New synapse release 1.2.1.
|
||||
|
||||
-- Synapse Packaging team <packages@matrix.org> Fri, 26 Jul 2019 11:32:47 +0100
|
||||
|
||||
matrix-synapse-py3 (1.2.0) stable; urgency=medium
|
||||
matrix-synapse-py3 (1.1.0-1) UNRELEASED; urgency=medium
|
||||
|
||||
[ Amber Brown ]
|
||||
* Update logging config defaults to match API changes in Synapse.
|
||||
|
||||
[ Richard van der Hoff ]
|
||||
* Add Recommends and Depends for some libraries which you probably want.
|
||||
|
||||
[ Synapse Packaging team ]
|
||||
* New synapse release 1.2.0.
|
||||
|
||||
-- Synapse Packaging team <packages@matrix.org> Thu, 25 Jul 2019 14:10:07 +0100
|
||||
-- Erik Johnston <erikj@rae> Thu, 04 Jul 2019 13:59:02 +0100
|
||||
|
||||
matrix-synapse-py3 (1.1.0) stable; urgency=medium
|
||||
|
||||
|
||||
7
debian/control
vendored
7
debian/control
vendored
@@ -2,20 +2,16 @@ Source: matrix-synapse-py3
|
||||
Section: contrib/python
|
||||
Priority: extra
|
||||
Maintainer: Synapse Packaging team <packages@matrix.org>
|
||||
# keep this list in sync with the build dependencies in docker/Dockerfile-dhvirtualenv.
|
||||
Build-Depends:
|
||||
debhelper (>= 9),
|
||||
dh-systemd,
|
||||
dh-virtualenv (>= 1.1),
|
||||
libsystemd-dev,
|
||||
libpq-dev,
|
||||
lsb-release,
|
||||
python3-dev,
|
||||
python3,
|
||||
python3-setuptools,
|
||||
python3-pip,
|
||||
python3-venv,
|
||||
libsqlite3-dev,
|
||||
tar,
|
||||
Standards-Version: 3.9.8
|
||||
Homepage: https://github.com/matrix-org/synapse
|
||||
@@ -32,12 +28,9 @@ Depends:
|
||||
debconf,
|
||||
python3-distutils|libpython3-stdlib (<< 3.6),
|
||||
${misc:Depends},
|
||||
${shlibs:Depends},
|
||||
${synapse:pydepends},
|
||||
# some of our scripts use perl, but none of them are important,
|
||||
# so we put perl:Depends in Suggests rather than Depends.
|
||||
Recommends:
|
||||
${shlibs1:Recommends},
|
||||
Suggests:
|
||||
sqlite3,
|
||||
${perl:Depends},
|
||||
|
||||
14
debian/rules
vendored
14
debian/rules
vendored
@@ -3,29 +3,15 @@
|
||||
# Build Debian package using https://github.com/spotify/dh-virtualenv
|
||||
#
|
||||
|
||||
# assume we only have one package
|
||||
PACKAGE_NAME:=`dh_listpackages`
|
||||
|
||||
override_dh_systemd_enable:
|
||||
dh_systemd_enable --name=matrix-synapse
|
||||
|
||||
override_dh_installinit:
|
||||
dh_installinit --name=matrix-synapse
|
||||
|
||||
# we don't really want to strip the symbols from our object files.
|
||||
override_dh_strip:
|
||||
|
||||
override_dh_shlibdeps:
|
||||
# make the postgres package's dependencies a recommendation
|
||||
# rather than a hard dependency.
|
||||
find debian/$(PACKAGE_NAME)/ -path '*/site-packages/psycopg2/*.so' | \
|
||||
xargs dpkg-shlibdeps -Tdebian/$(PACKAGE_NAME).substvars \
|
||||
-pshlibs1 -dRecommends
|
||||
|
||||
# all the other dependencies can be normal 'Depends' requirements,
|
||||
# except for PIL's, which is self-contained and which confuses
|
||||
# dpkg-shlibdeps.
|
||||
dh_shlibdeps -X site-packages/PIL/.libs -X site-packages/psycopg2
|
||||
|
||||
override_dh_virtualenv:
|
||||
./debian/build_virtualenv
|
||||
|
||||
@@ -29,7 +29,7 @@ for port in 8080 8081 8082; do
|
||||
|
||||
if ! grep -F "Customisation made by demo/start.sh" -q $DIR/etc/$port.config; then
|
||||
printf '\n\n# Customisation made by demo/start.sh\n' >> $DIR/etc/$port.config
|
||||
|
||||
|
||||
echo 'enable_registration: true' >> $DIR/etc/$port.config
|
||||
|
||||
# Warning, this heredoc depends on the interaction of tabs and spaces. Please don't
|
||||
@@ -43,7 +43,7 @@ for port in 8080 8081 8082; do
|
||||
tls: true
|
||||
resources:
|
||||
- names: [client, federation]
|
||||
|
||||
|
||||
- port: $port
|
||||
tls: false
|
||||
bind_addresses: ['::1', '127.0.0.1']
|
||||
@@ -68,7 +68,7 @@ for port in 8080 8081 8082; do
|
||||
|
||||
# Generate tls keys
|
||||
openssl req -x509 -newkey rsa:4096 -keyout $DIR/etc/localhost\:$https_port.tls.key -out $DIR/etc/localhost\:$https_port.tls.crt -days 365 -nodes -subj "/O=matrix"
|
||||
|
||||
|
||||
# Ignore keys from the trusted keys server
|
||||
echo '# Ignore keys from the trusted keys server' >> $DIR/etc/$port.config
|
||||
echo 'trusted_key_servers:' >> $DIR/etc/$port.config
|
||||
@@ -120,6 +120,7 @@ for port in 8080 8081 8082; do
|
||||
python3 -m synapse.app.homeserver \
|
||||
--config-path "$DIR/etc/$port.config" \
|
||||
-D \
|
||||
-vv \
|
||||
|
||||
popd
|
||||
done
|
||||
|
||||
@@ -43,9 +43,6 @@ RUN cd dh-virtualenv-1.1 && dpkg-buildpackage -us -uc -b
|
||||
FROM ${distro}
|
||||
|
||||
# Install the build dependencies
|
||||
#
|
||||
# NB: keep this list in sync with the list of build-deps in debian/control
|
||||
# TODO: it would be nice to do that automatically.
|
||||
RUN apt-get update -qq -o Acquire::Languages=none \
|
||||
&& env DEBIAN_FRONTEND=noninteractive apt-get install \
|
||||
-yqq --no-install-recommends -o Dpkg::Options::=--force-unsafe-io \
|
||||
|
||||
@@ -1,8 +1,4 @@
|
||||
Code Style
|
||||
==========
|
||||
|
||||
Formatting tools
|
||||
----------------
|
||||
# Code Style
|
||||
|
||||
The Synapse codebase uses a number of code formatting tools in order to
|
||||
quickly and automatically check for formatting (and sometimes logical) errors
|
||||
@@ -10,20 +6,20 @@ in code.
|
||||
|
||||
The necessary tools are detailed below.
|
||||
|
||||
- **black**
|
||||
## Formatting tools
|
||||
|
||||
The Synapse codebase uses `black <https://pypi.org/project/black/>`_ as an
|
||||
opinionated code formatter, ensuring all comitted code is properly
|
||||
formatted.
|
||||
The Synapse codebase uses [black](https://pypi.org/project/black/) as an
|
||||
opinionated code formatter, ensuring all comitted code is properly
|
||||
formatted.
|
||||
|
||||
First install ``black`` with::
|
||||
First install ``black`` with::
|
||||
|
||||
pip install --upgrade black
|
||||
pip install --upgrade black
|
||||
|
||||
Have ``black`` auto-format your code (it shouldn't change any functionality)
|
||||
with::
|
||||
Have ``black`` auto-format your code (it shouldn't change any
|
||||
functionality) with::
|
||||
|
||||
black . --exclude="\.tox|build|env"
|
||||
black . --exclude="\.tox|build|env"
|
||||
|
||||
- **flake8**
|
||||
|
||||
@@ -58,16 +54,17 @@ functionality is supported in your editor for a more convenient development
|
||||
workflow. It is not, however, recommended to run ``flake8`` on save as it
|
||||
takes a while and is very resource intensive.
|
||||
|
||||
General rules
|
||||
-------------
|
||||
## General rules
|
||||
|
||||
- **Naming**:
|
||||
|
||||
- Use camel case for class and type names
|
||||
- Use underscores for functions and variables.
|
||||
|
||||
- **Docstrings**: should follow the `google code style
|
||||
<https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings>`_.
|
||||
- Use double quotes ``"foo"`` rather than single quotes ``'foo'``.
|
||||
|
||||
- **Comments**: should follow the `google code style
|
||||
<http://google.github.io/styleguide/pyguide.html?showone=Comments#Comments>`_.
|
||||
This is so that we can generate documentation with `sphinx
|
||||
<http://sphinxcontrib-napoleon.readthedocs.org/en/latest/>`_. See the
|
||||
`examples
|
||||
@@ -76,8 +73,6 @@ General rules
|
||||
|
||||
- **Imports**:
|
||||
|
||||
- Imports should be sorted by ``isort`` as described above.
|
||||
|
||||
- Prefer to import classes and functions rather than packages or modules.
|
||||
|
||||
Example::
|
||||
@@ -97,84 +92,25 @@ General rules
|
||||
This goes against the advice in the Google style guide, but it means that
|
||||
errors in the name are caught early (at import time).
|
||||
|
||||
- Multiple imports from the same package can be combined onto one line::
|
||||
|
||||
from synapse.types import GroupID, RoomID, UserID
|
||||
|
||||
An effort should be made to keep the individual imports in alphabetical
|
||||
order.
|
||||
|
||||
If the list becomes long, wrap it with parentheses and split it over
|
||||
multiple lines.
|
||||
|
||||
- As per `PEP-8 <https://www.python.org/dev/peps/pep-0008/#imports>`_,
|
||||
imports should be grouped in the following order, with a blank line between
|
||||
each group:
|
||||
|
||||
1. standard library imports
|
||||
2. related third party imports
|
||||
3. local application/library specific imports
|
||||
|
||||
- Imports within each group should be sorted alphabetically by module name.
|
||||
|
||||
- Avoid wildcard imports (``from synapse.types import *``) and relative
|
||||
imports (``from .types import UserID``).
|
||||
|
||||
Configuration file format
|
||||
-------------------------
|
||||
|
||||
The `sample configuration file <./sample_config.yaml>`_ acts as a reference to
|
||||
Synapse's configuration options for server administrators. Remember that many
|
||||
readers will be unfamiliar with YAML and server administration in general, so
|
||||
that it is important that the file be as easy to understand as possible, which
|
||||
includes following a consistent format.
|
||||
|
||||
Some guidelines follow:
|
||||
|
||||
* Sections should be separated with a heading consisting of a single line
|
||||
prefixed and suffixed with ``##``. There should be **two** blank lines
|
||||
before the section header, and **one** after.
|
||||
|
||||
* Each option should be listed in the file with the following format:
|
||||
|
||||
* A comment describing the setting. Each line of this comment should be
|
||||
prefixed with a hash (``#``) and a space.
|
||||
|
||||
The comment should describe the default behaviour (ie, what happens if
|
||||
the setting is omitted), as well as what the effect will be if the
|
||||
setting is changed.
|
||||
|
||||
Often, the comment end with something like "uncomment the
|
||||
following to \<do action>".
|
||||
|
||||
* A line consisting of only ``#``.
|
||||
|
||||
* A commented-out example setting, prefixed with only ``#``.
|
||||
|
||||
For boolean (on/off) options, convention is that this example should be
|
||||
the *opposite* to the default (so the comment will end with "Uncomment
|
||||
the following to enable [or disable] \<feature\>." For other options,
|
||||
the example should give some non-default value which is likely to be
|
||||
useful to the reader.
|
||||
|
||||
* There should be a blank line between each option.
|
||||
|
||||
* Where several settings are grouped into a single dict, *avoid* the
|
||||
convention where the whole block is commented out, resulting in comment
|
||||
lines starting ``# #``, as this is hard to read and confusing to
|
||||
edit. Instead, leave the top-level config option uncommented, and follow
|
||||
the conventions above for sub-options. Ensure that your code correctly
|
||||
handles the top-level option being set to ``None`` (as it will be if no
|
||||
sub-options are enabled).
|
||||
|
||||
* Lines should be wrapped at 80 characters.
|
||||
|
||||
Example::
|
||||
|
||||
## Frobnication ##
|
||||
|
||||
# The frobnicator will ensure that all requests are fully frobnicated.
|
||||
# To enable it, uncomment the following.
|
||||
#
|
||||
#frobnicator_enabled: true
|
||||
|
||||
# By default, the frobnicator will frobnicate with the default frobber.
|
||||
# The following will make it use an alternative frobber.
|
||||
#
|
||||
#frobincator_frobber: special_frobber
|
||||
|
||||
# Settings for the frobber
|
||||
#
|
||||
frobber:
|
||||
# frobbing speed. Defaults to 1.
|
||||
#
|
||||
#speed: 10
|
||||
|
||||
# frobbing distance. Defaults to 1000.
|
||||
#
|
||||
#distance: 100
|
||||
|
||||
Note that the sample configuration is generated from the synapse code and is
|
||||
maintained by a script, ``scripts-dev/generate_sample_config``. Making sure
|
||||
that the output from this script matches the desired format is left as an
|
||||
exercise for the reader!
|
||||
|
||||
@@ -148,7 +148,7 @@ call any other functions.
|
||||
d = more_stuff()
|
||||
result = yield d # also fine, of course
|
||||
|
||||
return result
|
||||
defer.returnValue(result)
|
||||
|
||||
def nonInlineCallbacksFun():
|
||||
logger.debug("just a wrapper really")
|
||||
|
||||
@@ -59,108 +59,6 @@ How to monitor Synapse metrics using Prometheus
|
||||
Restart Prometheus.
|
||||
|
||||
|
||||
Renaming of metrics & deprecation of old names in 1.2
|
||||
-----------------------------------------------------
|
||||
|
||||
Synapse 1.2 updates the Prometheus metrics to match the naming convention of the
|
||||
upstream ``prometheus_client``. The old names are considered deprecated and will
|
||||
be removed in a future version of Synapse.
|
||||
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| New Name | Old Name |
|
||||
+=============================================================================+=======================================================================+
|
||||
| python_gc_objects_collected_total | python_gc_objects_collected |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| python_gc_objects_uncollectable_total | python_gc_objects_uncollectable |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| python_gc_collections_total | python_gc_collections |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| process_cpu_seconds_total | process_cpu_seconds |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_federation_client_sent_transactions_total | synapse_federation_client_sent_transactions |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_federation_client_events_processed_total | synapse_federation_client_events_processed |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_event_processing_loop_count_total | synapse_event_processing_loop_count |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_event_processing_loop_room_count_total | synapse_event_processing_loop_room_count |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_util_metrics_block_count_total | synapse_util_metrics_block_count |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_util_metrics_block_time_seconds_total | synapse_util_metrics_block_time_seconds |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_util_metrics_block_ru_utime_seconds_total | synapse_util_metrics_block_ru_utime_seconds |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_util_metrics_block_ru_stime_seconds_total | synapse_util_metrics_block_ru_stime_seconds |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_util_metrics_block_db_txn_count_total | synapse_util_metrics_block_db_txn_count |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_util_metrics_block_db_txn_duration_seconds_total | synapse_util_metrics_block_db_txn_duration_seconds |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_util_metrics_block_db_sched_duration_seconds_total | synapse_util_metrics_block_db_sched_duration_seconds |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_background_process_start_count_total | synapse_background_process_start_count |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_background_process_ru_utime_seconds_total | synapse_background_process_ru_utime_seconds |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_background_process_ru_stime_seconds_total | synapse_background_process_ru_stime_seconds |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_background_process_db_txn_count_total | synapse_background_process_db_txn_count |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_background_process_db_txn_duration_seconds_total | synapse_background_process_db_txn_duration_seconds |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_background_process_db_sched_duration_seconds_total | synapse_background_process_db_sched_duration_seconds |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_storage_events_persisted_events_total | synapse_storage_events_persisted_events |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_storage_events_persisted_events_sep_total | synapse_storage_events_persisted_events_sep |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_storage_events_state_delta_total | synapse_storage_events_state_delta |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_storage_events_state_delta_single_event_total | synapse_storage_events_state_delta_single_event |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_storage_events_state_delta_reuse_delta_total | synapse_storage_events_state_delta_reuse_delta |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_federation_server_received_pdus_total | synapse_federation_server_received_pdus |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_federation_server_received_edus_total | synapse_federation_server_received_edus |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_handler_presence_notified_presence_total | synapse_handler_presence_notified_presence |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_handler_presence_federation_presence_out_total | synapse_handler_presence_federation_presence_out |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_handler_presence_presence_updates_total | synapse_handler_presence_presence_updates |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_handler_presence_timers_fired_total | synapse_handler_presence_timers_fired |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_handler_presence_federation_presence_total | synapse_handler_presence_federation_presence |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_handler_presence_bump_active_time_total | synapse_handler_presence_bump_active_time |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_federation_client_sent_edus_total | synapse_federation_client_sent_edus |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_federation_client_sent_pdu_destinations_count_total | synapse_federation_client_sent_pdu_destinations:count |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_federation_client_sent_pdu_destinations_total | synapse_federation_client_sent_pdu_destinations:total |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_handlers_appservice_events_processed_total | synapse_handlers_appservice_events_processed |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_notifier_notified_events_total | synapse_notifier_notified_events |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_push_bulk_push_rule_evaluator_push_rules_invalidation_counter_total | synapse_push_bulk_push_rule_evaluator_push_rules_invalidation_counter |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_push_bulk_push_rule_evaluator_push_rules_state_size_counter_total | synapse_push_bulk_push_rule_evaluator_push_rules_state_size_counter |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_http_httppusher_http_pushes_processed_total | synapse_http_httppusher_http_pushes_processed |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_http_httppusher_http_pushes_failed_total | synapse_http_httppusher_http_pushes_failed |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_http_httppusher_badge_updates_processed_total | synapse_http_httppusher_badge_updates_processed |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
| synapse_http_httppusher_badge_updates_failed_total | synapse_http_httppusher_badge_updates_failed |
|
||||
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
|
||||
|
||||
|
||||
Removal of deprecated metrics & time based counters becoming histograms in 0.31.0
|
||||
---------------------------------------------------------------------------------
|
||||
|
||||
|
||||
@@ -1,100 +0,0 @@
|
||||
===========
|
||||
OpenTracing
|
||||
===========
|
||||
|
||||
Background
|
||||
----------
|
||||
|
||||
OpenTracing is a semi-standard being adopted by a number of distributed tracing
|
||||
platforms. It is a common api for facilitating vendor-agnostic tracing
|
||||
instrumentation. That is, we can use the OpenTracing api and select one of a
|
||||
number of tracer implementations to do the heavy lifting in the background.
|
||||
Our current selected implementation is Jaeger.
|
||||
|
||||
OpenTracing is a tool which gives an insight into the causal relationship of
|
||||
work done in and between servers. The servers each track events and report them
|
||||
to a centralised server - in Synapse's case: Jaeger. The basic unit used to
|
||||
represent events is the span. The span roughly represents a single piece of work
|
||||
that was done and the time at which it occurred. A span can have child spans,
|
||||
meaning that the work of the child had to be completed for the parent span to
|
||||
complete, or it can have follow-on spans which represent work that is undertaken
|
||||
as a result of the parent but is not depended on by the parent to in order to
|
||||
finish.
|
||||
|
||||
Since this is undertaken in a distributed environment a request to another
|
||||
server, such as an RPC or a simple GET, can be considered a span (a unit or
|
||||
work) for the local server. This causal link is what OpenTracing aims to
|
||||
capture and visualise. In order to do this metadata about the local server's
|
||||
span, i.e the 'span context', needs to be included with the request to the
|
||||
remote.
|
||||
|
||||
It is up to the remote server to decide what it does with the spans
|
||||
it creates. This is called the sampling policy and it can be configured
|
||||
through Jaeger's settings.
|
||||
|
||||
For OpenTracing concepts see
|
||||
https://opentracing.io/docs/overview/what-is-tracing/.
|
||||
|
||||
For more information about Jaeger's implementation see
|
||||
https://www.jaegertracing.io/docs/
|
||||
|
||||
=====================
|
||||
Seting up OpenTracing
|
||||
=====================
|
||||
|
||||
To receive OpenTracing spans, start up a Jaeger server. This can be done
|
||||
using docker like so:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker run -d --name jaeger
|
||||
-p 6831:6831/udp \
|
||||
-p 6832:6832/udp \
|
||||
-p 5778:5778 \
|
||||
-p 16686:16686 \
|
||||
-p 14268:14268 \
|
||||
jaegertracing/all-in-one:1.13
|
||||
|
||||
Latest documentation is probably at
|
||||
https://www.jaegertracing.io/docs/1.13/getting-started/
|
||||
|
||||
|
||||
Enable OpenTracing in Synapse
|
||||
-----------------------------
|
||||
|
||||
OpenTracing is not enabled by default. It must be enabled in the homeserver
|
||||
config by uncommenting the config options under ``opentracing`` as shown in
|
||||
the `sample config <./sample_config.yaml>`_. For example:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
opentracing:
|
||||
tracer_enabled: true
|
||||
homeserver_whitelist:
|
||||
- "mytrustedhomeserver.org"
|
||||
- "*.myotherhomeservers.com"
|
||||
|
||||
Homeserver whitelisting
|
||||
-----------------------
|
||||
|
||||
The homeserver whitelist is configured using regular expressions. A list of regular
|
||||
expressions can be given and their union will be compared when propagating any
|
||||
spans contexts to another homeserver.
|
||||
|
||||
Though it's mostly safe to send and receive span contexts to and from
|
||||
untrusted users since span contexts are usually opaque ids it can lead to
|
||||
two problems, namely:
|
||||
|
||||
- If the span context is marked as sampled by the sending homeserver the receiver will
|
||||
sample it. Therefore two homeservers with wildly different sampling policies
|
||||
could incur higher sampling counts than intended.
|
||||
- Sending servers can attach arbitrary data to spans, known as 'baggage'. For safety this has been disabled in Synapse
|
||||
but that doesn't prevent another server sending you baggage which will be logged
|
||||
to OpenTracing's logs.
|
||||
|
||||
==================
|
||||
Configuring Jaeger
|
||||
==================
|
||||
|
||||
Sampling strategies can be set as in this document:
|
||||
https://www.jaegertracing.io/docs/1.13/sampling/
|
||||
@@ -11,9 +11,7 @@ a postgres database.
|
||||
|
||||
* If you are using the `matrix.org debian/ubuntu
|
||||
packages <../INSTALL.md#matrixorg-packages>`_,
|
||||
the necessary python library will already be installed, but you will need to
|
||||
ensure the low-level postgres library is installed, which you can do with
|
||||
``apt install libpq5``.
|
||||
the necessary libraries will already be installed.
|
||||
|
||||
* For other pre-built packages, please consult the documentation from the
|
||||
relevant package.
|
||||
@@ -36,7 +34,7 @@ Assuming your PostgreSQL database user is called ``postgres``, create a user
|
||||
su - postgres
|
||||
createuser --pwprompt synapse_user
|
||||
|
||||
Before you can authenticate with the ``synapse_user``, you must create a
|
||||
Before you can authenticate with the ``synapse_user``, you must create a
|
||||
database that it can access. To create a database, first connect to the database
|
||||
with your database user::
|
||||
|
||||
@@ -55,7 +53,7 @@ and then run::
|
||||
This would create an appropriate database named ``synapse`` owned by the
|
||||
``synapse_user`` user (which must already have been created as above).
|
||||
|
||||
Note that the PostgreSQL database *must* have the correct encoding set (as
|
||||
Note that the PostgreSQL database *must* have the correct encoding set (as
|
||||
shown above), otherwise it will not be able to store UTF8 strings.
|
||||
|
||||
You may need to enable password authentication so ``synapse_user`` can connect
|
||||
|
||||
@@ -1409,40 +1409,17 @@ password_config:
|
||||
|
||||
|
||||
## Opentracing ##
|
||||
# These settings enable opentracing which implements distributed tracing
|
||||
# This allows you to observe the causal chain of events across servers
|
||||
# including requests, key lookups etc. across any server running
|
||||
# synapse or any other other services which supports opentracing.
|
||||
# (specifically those implemented with jaeger)
|
||||
|
||||
# These settings enable opentracing, which implements distributed tracing.
|
||||
# This allows you to observe the causal chains of events across servers
|
||||
# including requests, key lookups etc., across any server running
|
||||
# synapse or any other other services which supports opentracing
|
||||
# (specifically those implemented with Jaeger).
|
||||
#
|
||||
opentracing:
|
||||
# tracing is disabled by default. Uncomment the following line to enable it.
|
||||
#
|
||||
#enabled: true
|
||||
|
||||
# The list of homeservers we wish to send and receive span contexts and span baggage.
|
||||
# See docs/opentracing.rst
|
||||
# This is a list of regexes which are matched against the server_name of the
|
||||
# homeserver.
|
||||
#
|
||||
# By defult, it is empty, so no servers are matched.
|
||||
#
|
||||
#homeserver_whitelist:
|
||||
# - ".*"
|
||||
|
||||
# Jaeger can be configured to sample traces at different rates.
|
||||
# All configuration options provided by Jaeger can be set here.
|
||||
# Jaeger's configuration mostly related to trace sampling which
|
||||
# is documented here:
|
||||
# https://www.jaegertracing.io/docs/1.13/sampling/.
|
||||
#
|
||||
#jaeger_config:
|
||||
# sampler:
|
||||
# type: const
|
||||
# param: 1
|
||||
|
||||
# Logging whether spans were started and reported
|
||||
#
|
||||
# logging:
|
||||
# false
|
||||
#opentracing:
|
||||
# # Enable / disable tracer
|
||||
# tracer_enabled: false
|
||||
# # The list of homeservers we wish to expose our current traces to.
|
||||
# # The list is a list of regexes which are matched against the
|
||||
# # servername of the homeserver
|
||||
# homeserver_whitelist:
|
||||
# - ".*"
|
||||
|
||||
@@ -35,4 +35,4 @@ try:
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
__version__ = "1.2.1"
|
||||
__version__ = "1.1.0"
|
||||
|
||||
@@ -128,7 +128,7 @@ class Auth(object):
|
||||
)
|
||||
|
||||
self._check_joined_room(member, user_id, room_id)
|
||||
return member
|
||||
defer.returnValue(member)
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def check_user_was_in_room(self, room_id, user_id):
|
||||
@@ -156,13 +156,13 @@ class Auth(object):
|
||||
if forgot:
|
||||
raise AuthError(403, "User %s not in room %s" % (user_id, room_id))
|
||||
|
||||
return member
|
||||
defer.returnValue(member)
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def check_host_in_room(self, room_id, host):
|
||||
with Measure(self.clock, "check_host_in_room"):
|
||||
latest_event_ids = yield self.store.is_host_joined(room_id, host)
|
||||
return latest_event_ids
|
||||
defer.returnValue(latest_event_ids)
|
||||
|
||||
def _check_joined_room(self, member, user_id, room_id):
|
||||
if not member or member.membership != Membership.JOIN:
|
||||
@@ -219,7 +219,9 @@ class Auth(object):
|
||||
device_id="dummy-device", # stubbed
|
||||
)
|
||||
|
||||
return synapse.types.create_requester(user_id, app_service=app_service)
|
||||
defer.returnValue(
|
||||
synapse.types.create_requester(user_id, app_service=app_service)
|
||||
)
|
||||
|
||||
user_info = yield self.get_user_by_access_token(access_token, rights)
|
||||
user = user_info["user"]
|
||||
@@ -260,8 +262,10 @@ class Auth(object):
|
||||
|
||||
request.authenticated_entity = user.to_string()
|
||||
|
||||
return synapse.types.create_requester(
|
||||
user, token_id, is_guest, device_id, app_service=app_service
|
||||
defer.returnValue(
|
||||
synapse.types.create_requester(
|
||||
user, token_id, is_guest, device_id, app_service=app_service
|
||||
)
|
||||
)
|
||||
except KeyError:
|
||||
raise MissingClientTokenError()
|
||||
@@ -272,25 +276,25 @@ class Auth(object):
|
||||
self.get_access_token_from_request(request)
|
||||
)
|
||||
if app_service is None:
|
||||
return (None, None)
|
||||
defer.returnValue((None, None))
|
||||
|
||||
if app_service.ip_range_whitelist:
|
||||
ip_address = IPAddress(self.hs.get_ip_from_request(request))
|
||||
if ip_address not in app_service.ip_range_whitelist:
|
||||
return (None, None)
|
||||
defer.returnValue((None, None))
|
||||
|
||||
if b"user_id" not in request.args:
|
||||
return (app_service.sender, app_service)
|
||||
defer.returnValue((app_service.sender, app_service))
|
||||
|
||||
user_id = request.args[b"user_id"][0].decode("utf8")
|
||||
if app_service.sender == user_id:
|
||||
return (app_service.sender, app_service)
|
||||
defer.returnValue((app_service.sender, app_service))
|
||||
|
||||
if not app_service.is_interested_in_user(user_id):
|
||||
raise AuthError(403, "Application service cannot masquerade as this user.")
|
||||
if not (yield self.store.get_user_by_id(user_id)):
|
||||
raise AuthError(403, "Application service has not registered this user")
|
||||
return (user_id, app_service)
|
||||
defer.returnValue((user_id, app_service))
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def get_user_by_access_token(self, token, rights="access"):
|
||||
@@ -326,7 +330,7 @@ class Auth(object):
|
||||
msg="Access token has expired", soft_logout=True
|
||||
)
|
||||
|
||||
return r
|
||||
defer.returnValue(r)
|
||||
|
||||
# otherwise it needs to be a valid macaroon
|
||||
try:
|
||||
@@ -374,7 +378,7 @@ class Auth(object):
|
||||
}
|
||||
else:
|
||||
raise RuntimeError("Unknown rights setting %s", rights)
|
||||
return ret
|
||||
defer.returnValue(ret)
|
||||
except (
|
||||
_InvalidMacaroonException,
|
||||
pymacaroons.exceptions.MacaroonException,
|
||||
@@ -502,7 +506,7 @@ class Auth(object):
|
||||
def _look_up_user_by_access_token(self, token):
|
||||
ret = yield self.store.get_user_by_access_token(token)
|
||||
if not ret:
|
||||
return None
|
||||
defer.returnValue(None)
|
||||
|
||||
# we use ret.get() below because *lots* of unit tests stub out
|
||||
# get_user_by_access_token in a way where it only returns a couple of
|
||||
@@ -514,7 +518,7 @@ class Auth(object):
|
||||
"device_id": ret.get("device_id"),
|
||||
"valid_until_ms": ret.get("valid_until_ms"),
|
||||
}
|
||||
return user_info
|
||||
defer.returnValue(user_info)
|
||||
|
||||
def get_appservice_by_req(self, request):
|
||||
token = self.get_access_token_from_request(request)
|
||||
@@ -539,7 +543,7 @@ class Auth(object):
|
||||
@defer.inlineCallbacks
|
||||
def compute_auth_events(self, event, current_state_ids, for_verification=False):
|
||||
if event.type == EventTypes.Create:
|
||||
return []
|
||||
defer.returnValue([])
|
||||
|
||||
auth_ids = []
|
||||
|
||||
@@ -600,7 +604,22 @@ class Auth(object):
|
||||
if member_event.content["membership"] == Membership.JOIN:
|
||||
auth_ids.append(member_event.event_id)
|
||||
|
||||
return auth_ids
|
||||
defer.returnValue(auth_ids)
|
||||
|
||||
def check_redaction(self, room_version, event, auth_events):
|
||||
"""Check whether the event sender is allowed to redact the target event.
|
||||
|
||||
Returns:
|
||||
True if the the sender is allowed to redact the target event if the
|
||||
target event was created by them.
|
||||
False if the sender is allowed to redact the target event with no
|
||||
further checks.
|
||||
|
||||
Raises:
|
||||
AuthError if the event sender is definitely not allowed to redact
|
||||
the target event.
|
||||
"""
|
||||
return event_auth.check_redaction(room_version, event, auth_events)
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def check_can_change_room_list(self, room_id, user):
|
||||
@@ -614,7 +633,7 @@ class Auth(object):
|
||||
|
||||
is_admin = yield self.is_server_admin(user)
|
||||
if is_admin:
|
||||
return True
|
||||
defer.returnValue(True)
|
||||
|
||||
user_id = user.to_string()
|
||||
yield self.check_joined_room(room_id, user_id)
|
||||
@@ -708,7 +727,7 @@ class Auth(object):
|
||||
# * The user is a guest user, and has joined the room
|
||||
# else it will throw.
|
||||
member_event = yield self.check_user_was_in_room(room_id, user_id)
|
||||
return (member_event.membership, member_event.event_id)
|
||||
defer.returnValue((member_event.membership, member_event.event_id))
|
||||
except AuthError:
|
||||
visibility = yield self.state.get_current_state(
|
||||
room_id, EventTypes.RoomHistoryVisibility, ""
|
||||
@@ -717,7 +736,7 @@ class Auth(object):
|
||||
visibility
|
||||
and visibility.content["history_visibility"] == "world_readable"
|
||||
):
|
||||
return (Membership.JOIN, None)
|
||||
defer.returnValue((Membership.JOIN, None))
|
||||
return
|
||||
raise AuthError(
|
||||
403, "Guest access not allowed", errcode=Codes.GUEST_ACCESS_FORBIDDEN
|
||||
|
||||
@@ -61,6 +61,7 @@ class Codes(object):
|
||||
INCOMPATIBLE_ROOM_VERSION = "M_INCOMPATIBLE_ROOM_VERSION"
|
||||
WRONG_ROOM_KEYS_VERSION = "M_WRONG_ROOM_KEYS_VERSION"
|
||||
EXPIRED_ACCOUNT = "ORG_MATRIX_EXPIRED_ACCOUNT"
|
||||
USER_DEACTIVATED = "M_USER_DEACTIVATED"
|
||||
|
||||
|
||||
class CodeMessageException(RuntimeError):
|
||||
@@ -151,7 +152,7 @@ class UserDeactivatedError(SynapseError):
|
||||
msg (str): The human-readable error message
|
||||
"""
|
||||
super(UserDeactivatedError, self).__init__(
|
||||
code=http_client.FORBIDDEN, msg=msg, errcode=Codes.UNKNOWN
|
||||
code=http_client.FORBIDDEN, msg=msg, errcode=Codes.USER_DEACTIVATED
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -132,7 +132,7 @@ class Filtering(object):
|
||||
@defer.inlineCallbacks
|
||||
def get_user_filter(self, user_localpart, filter_id):
|
||||
result = yield self.store.get_user_filter(user_localpart, filter_id)
|
||||
return FilterCollection(result)
|
||||
defer.returnValue(FilterCollection(result))
|
||||
|
||||
def add_user_filter(self, user_localpart, user_filter):
|
||||
self.check_valid_filter(user_filter)
|
||||
|
||||
@@ -15,12 +15,10 @@
|
||||
|
||||
import gc
|
||||
import logging
|
||||
import os
|
||||
import signal
|
||||
import sys
|
||||
import traceback
|
||||
|
||||
import sdnotify
|
||||
from daemonize import Daemonize
|
||||
|
||||
from twisted.internet import defer, error, reactor
|
||||
@@ -50,7 +48,7 @@ def register_sighup(func):
|
||||
_sighup_callbacks.append(func)
|
||||
|
||||
|
||||
def start_worker_reactor(appname, config, run_command=reactor.run):
|
||||
def start_worker_reactor(appname, config):
|
||||
""" Run the reactor in the main process
|
||||
|
||||
Daemonizes if necessary, and then configures some resources, before starting
|
||||
@@ -59,7 +57,6 @@ def start_worker_reactor(appname, config, run_command=reactor.run):
|
||||
Args:
|
||||
appname (str): application name which will be sent to syslog
|
||||
config (synapse.config.Config): config object
|
||||
run_command (Callable[]): callable that actually runs the reactor
|
||||
"""
|
||||
|
||||
logger = logging.getLogger(config.worker_app)
|
||||
@@ -72,19 +69,11 @@ def start_worker_reactor(appname, config, run_command=reactor.run):
|
||||
daemonize=config.worker_daemonize,
|
||||
print_pidfile=config.print_pidfile,
|
||||
logger=logger,
|
||||
run_command=run_command,
|
||||
)
|
||||
|
||||
|
||||
def start_reactor(
|
||||
appname,
|
||||
soft_file_limit,
|
||||
gc_thresholds,
|
||||
pid_file,
|
||||
daemonize,
|
||||
print_pidfile,
|
||||
logger,
|
||||
run_command=reactor.run,
|
||||
appname, soft_file_limit, gc_thresholds, pid_file, daemonize, print_pidfile, logger
|
||||
):
|
||||
""" Run the reactor in the main process
|
||||
|
||||
@@ -99,7 +88,6 @@ def start_reactor(
|
||||
daemonize (bool): true to run the reactor in a background process
|
||||
print_pidfile (bool): whether to print the pid file, if daemonize is True
|
||||
logger (logging.Logger): logger instance to pass to Daemonize
|
||||
run_command (Callable[]): callable that actually runs the reactor
|
||||
"""
|
||||
|
||||
install_dns_limiter(reactor)
|
||||
@@ -109,7 +97,7 @@ def start_reactor(
|
||||
change_resource_limit(soft_file_limit)
|
||||
if gc_thresholds:
|
||||
gc.set_threshold(*gc_thresholds)
|
||||
run_command()
|
||||
reactor.run()
|
||||
|
||||
# make sure that we run the reactor with the sentinel log context,
|
||||
# otherwise other PreserveLoggingContext instances will get confused
|
||||
@@ -151,7 +139,8 @@ def listen_metrics(bind_addresses, port):
|
||||
"""
|
||||
Start Prometheus metrics server.
|
||||
"""
|
||||
from synapse.metrics import RegistryProxy, start_http_server
|
||||
from synapse.metrics import RegistryProxy
|
||||
from prometheus_client import start_http_server
|
||||
|
||||
for host in bind_addresses:
|
||||
logger.info("Starting metrics listener on %s:%d", host, port)
|
||||
@@ -244,16 +233,9 @@ def start(hs, listeners=None):
|
||||
if hasattr(signal, "SIGHUP"):
|
||||
|
||||
def handle_sighup(*args, **kwargs):
|
||||
# Tell systemd our state, if we're using it. This will silently fail if
|
||||
# we're not using systemd.
|
||||
sd_channel = sdnotify.SystemdNotifier()
|
||||
sd_channel.notify("RELOADING=1")
|
||||
|
||||
for i in _sighup_callbacks:
|
||||
i(hs)
|
||||
|
||||
sd_channel.notify("READY=1")
|
||||
|
||||
signal.signal(signal.SIGHUP, handle_sighup)
|
||||
|
||||
register_sighup(refresh_certificate)
|
||||
@@ -269,7 +251,6 @@ def start(hs, listeners=None):
|
||||
hs.get_datastore().start_profiling()
|
||||
|
||||
setup_sentry(hs)
|
||||
setup_sdnotify(hs)
|
||||
except Exception:
|
||||
traceback.print_exc(file=sys.stderr)
|
||||
reactor = hs.get_reactor()
|
||||
@@ -302,25 +283,6 @@ def setup_sentry(hs):
|
||||
scope.set_tag("worker_name", name)
|
||||
|
||||
|
||||
def setup_sdnotify(hs):
|
||||
"""Adds process state hooks to tell systemd what we are up to.
|
||||
"""
|
||||
|
||||
# Tell systemd our state, if we're using it. This will silently fail if
|
||||
# we're not using systemd.
|
||||
sd_channel = sdnotify.SystemdNotifier()
|
||||
|
||||
hs.get_reactor().addSystemEventTrigger(
|
||||
"after",
|
||||
"startup",
|
||||
lambda: sd_channel.notify("READY=1\nMAINPID=%s" % (os.getpid())),
|
||||
)
|
||||
|
||||
hs.get_reactor().addSystemEventTrigger(
|
||||
"before", "shutdown", lambda: sd_channel.notify("STOPPING=1")
|
||||
)
|
||||
|
||||
|
||||
def install_dns_limiter(reactor, max_dns_requests_in_flight=100):
|
||||
"""Replaces the resolver with one that limits the number of in flight DNS
|
||||
requests.
|
||||
|
||||
@@ -1,264 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
# Copyright 2019 Matrix.org Foundation C.I.C.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
import argparse
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
from canonicaljson import json
|
||||
|
||||
from twisted.internet import defer, task
|
||||
|
||||
import synapse
|
||||
from synapse.app import _base
|
||||
from synapse.config._base import ConfigError
|
||||
from synapse.config.homeserver import HomeServerConfig
|
||||
from synapse.config.logger import setup_logging
|
||||
from synapse.handlers.admin import ExfiltrationWriter
|
||||
from synapse.replication.slave.storage._base import BaseSlavedStore
|
||||
from synapse.replication.slave.storage.account_data import SlavedAccountDataStore
|
||||
from synapse.replication.slave.storage.appservice import SlavedApplicationServiceStore
|
||||
from synapse.replication.slave.storage.client_ips import SlavedClientIpStore
|
||||
from synapse.replication.slave.storage.deviceinbox import SlavedDeviceInboxStore
|
||||
from synapse.replication.slave.storage.devices import SlavedDeviceStore
|
||||
from synapse.replication.slave.storage.events import SlavedEventStore
|
||||
from synapse.replication.slave.storage.filtering import SlavedFilteringStore
|
||||
from synapse.replication.slave.storage.groups import SlavedGroupServerStore
|
||||
from synapse.replication.slave.storage.presence import SlavedPresenceStore
|
||||
from synapse.replication.slave.storage.push_rule import SlavedPushRuleStore
|
||||
from synapse.replication.slave.storage.receipts import SlavedReceiptsStore
|
||||
from synapse.replication.slave.storage.registration import SlavedRegistrationStore
|
||||
from synapse.replication.slave.storage.room import RoomStore
|
||||
from synapse.replication.tcp.client import ReplicationClientHandler
|
||||
from synapse.server import HomeServer
|
||||
from synapse.storage.engines import create_engine
|
||||
from synapse.util.logcontext import LoggingContext
|
||||
from synapse.util.versionstring import get_version_string
|
||||
|
||||
logger = logging.getLogger("synapse.app.admin_cmd")
|
||||
|
||||
|
||||
class AdminCmdSlavedStore(
|
||||
SlavedReceiptsStore,
|
||||
SlavedAccountDataStore,
|
||||
SlavedApplicationServiceStore,
|
||||
SlavedRegistrationStore,
|
||||
SlavedFilteringStore,
|
||||
SlavedPresenceStore,
|
||||
SlavedGroupServerStore,
|
||||
SlavedDeviceInboxStore,
|
||||
SlavedDeviceStore,
|
||||
SlavedPushRuleStore,
|
||||
SlavedEventStore,
|
||||
SlavedClientIpStore,
|
||||
RoomStore,
|
||||
BaseSlavedStore,
|
||||
):
|
||||
pass
|
||||
|
||||
|
||||
class AdminCmdServer(HomeServer):
|
||||
DATASTORE_CLASS = AdminCmdSlavedStore
|
||||
|
||||
def _listen_http(self, listener_config):
|
||||
pass
|
||||
|
||||
def start_listening(self, listeners):
|
||||
pass
|
||||
|
||||
def build_tcp_replication(self):
|
||||
return AdminCmdReplicationHandler(self)
|
||||
|
||||
|
||||
class AdminCmdReplicationHandler(ReplicationClientHandler):
|
||||
@defer.inlineCallbacks
|
||||
def on_rdata(self, stream_name, token, rows):
|
||||
pass
|
||||
|
||||
def get_streams_to_replicate(self):
|
||||
return {}
|
||||
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def export_data_command(hs, args):
|
||||
"""Export data for a user.
|
||||
|
||||
Args:
|
||||
hs (HomeServer)
|
||||
args (argparse.Namespace)
|
||||
"""
|
||||
|
||||
user_id = args.user_id
|
||||
directory = args.output_directory
|
||||
|
||||
res = yield hs.get_handlers().admin_handler.export_user_data(
|
||||
user_id, FileExfiltrationWriter(user_id, directory=directory)
|
||||
)
|
||||
print(res)
|
||||
|
||||
|
||||
class FileExfiltrationWriter(ExfiltrationWriter):
|
||||
"""An ExfiltrationWriter that writes the users data to a directory.
|
||||
Returns the directory location on completion.
|
||||
|
||||
Note: This writes to disk on the main reactor thread.
|
||||
|
||||
Args:
|
||||
user_id (str): The user whose data is being exfiltrated.
|
||||
directory (str|None): The directory to write the data to, if None then
|
||||
will write to a temporary directory.
|
||||
"""
|
||||
|
||||
def __init__(self, user_id, directory=None):
|
||||
self.user_id = user_id
|
||||
|
||||
if directory:
|
||||
self.base_directory = directory
|
||||
else:
|
||||
self.base_directory = tempfile.mkdtemp(
|
||||
prefix="synapse-exfiltrate__%s__" % (user_id,)
|
||||
)
|
||||
|
||||
os.makedirs(self.base_directory, exist_ok=True)
|
||||
if list(os.listdir(self.base_directory)):
|
||||
raise Exception("Directory must be empty")
|
||||
|
||||
def write_events(self, room_id, events):
|
||||
room_directory = os.path.join(self.base_directory, "rooms", room_id)
|
||||
os.makedirs(room_directory, exist_ok=True)
|
||||
events_file = os.path.join(room_directory, "events")
|
||||
|
||||
with open(events_file, "a") as f:
|
||||
for event in events:
|
||||
print(json.dumps(event.get_pdu_json()), file=f)
|
||||
|
||||
def write_state(self, room_id, event_id, state):
|
||||
room_directory = os.path.join(self.base_directory, "rooms", room_id)
|
||||
state_directory = os.path.join(room_directory, "state")
|
||||
os.makedirs(state_directory, exist_ok=True)
|
||||
|
||||
event_file = os.path.join(state_directory, event_id)
|
||||
|
||||
with open(event_file, "a") as f:
|
||||
for event in state.values():
|
||||
print(json.dumps(event.get_pdu_json()), file=f)
|
||||
|
||||
def write_invite(self, room_id, event, state):
|
||||
self.write_events(room_id, [event])
|
||||
|
||||
# We write the invite state somewhere else as they aren't full events
|
||||
# and are only a subset of the state at the event.
|
||||
room_directory = os.path.join(self.base_directory, "rooms", room_id)
|
||||
os.makedirs(room_directory, exist_ok=True)
|
||||
|
||||
invite_state = os.path.join(room_directory, "invite_state")
|
||||
|
||||
with open(invite_state, "a") as f:
|
||||
for event in state.values():
|
||||
print(json.dumps(event), file=f)
|
||||
|
||||
def finished(self):
|
||||
return self.base_directory
|
||||
|
||||
|
||||
def start(config_options):
|
||||
parser = argparse.ArgumentParser(description="Synapse Admin Command")
|
||||
HomeServerConfig.add_arguments_to_parser(parser)
|
||||
|
||||
subparser = parser.add_subparsers(
|
||||
title="Admin Commands",
|
||||
required=True,
|
||||
dest="command",
|
||||
metavar="<admin_command>",
|
||||
help="The admin command to perform.",
|
||||
)
|
||||
export_data_parser = subparser.add_parser(
|
||||
"export-data", help="Export all data for a user"
|
||||
)
|
||||
export_data_parser.add_argument("user_id", help="User to extra data from")
|
||||
export_data_parser.add_argument(
|
||||
"--output-directory",
|
||||
action="store",
|
||||
metavar="DIRECTORY",
|
||||
required=False,
|
||||
help="The directory to store the exported data in. Must be empty. Defaults"
|
||||
" to creating a temp directory.",
|
||||
)
|
||||
export_data_parser.set_defaults(func=export_data_command)
|
||||
|
||||
try:
|
||||
config, args = HomeServerConfig.load_config_with_parser(parser, config_options)
|
||||
except ConfigError as e:
|
||||
sys.stderr.write("\n" + str(e) + "\n")
|
||||
sys.exit(1)
|
||||
|
||||
if config.worker_app is not None:
|
||||
assert config.worker_app == "synapse.app.admin_cmd"
|
||||
|
||||
# Update the config with some basic overrides so that don't have to specify
|
||||
# a full worker config.
|
||||
config.worker_app = "synapse.app.admin_cmd"
|
||||
|
||||
if (
|
||||
not config.worker_daemonize
|
||||
and not config.worker_log_file
|
||||
and not config.worker_log_config
|
||||
):
|
||||
# Since we're meant to be run as a "command" let's not redirect stdio
|
||||
# unless we've actually set log config.
|
||||
config.no_redirect_stdio = True
|
||||
|
||||
# Explicitly disable background processes
|
||||
config.update_user_directory = False
|
||||
config.start_pushers = False
|
||||
config.send_federation = False
|
||||
|
||||
setup_logging(config, use_worker_options=True)
|
||||
|
||||
synapse.events.USE_FROZEN_DICTS = config.use_frozen_dicts
|
||||
|
||||
database_engine = create_engine(config.database_config)
|
||||
|
||||
ss = AdminCmdServer(
|
||||
config.server_name,
|
||||
db_config=config.database_config,
|
||||
config=config,
|
||||
version_string="Synapse/" + get_version_string(synapse),
|
||||
database_engine=database_engine,
|
||||
)
|
||||
|
||||
ss.setup()
|
||||
|
||||
# We use task.react as the basic run command as it correctly handles tearing
|
||||
# down the reactor when the deferreds resolve and setting the return value.
|
||||
# We also make sure that `_base.start` gets run before we actually run the
|
||||
# command.
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def run(_reactor):
|
||||
with LoggingContext("command"):
|
||||
yield _base.start(ss, [])
|
||||
yield args.func(ss, args)
|
||||
|
||||
_base.start_worker_reactor(
|
||||
"synapse-admin-cmd", config, run_command=lambda: task.react(run)
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
with LoggingContext("main"):
|
||||
start(sys.argv[1:])
|
||||
@@ -27,7 +27,8 @@ from synapse.config.homeserver import HomeServerConfig
|
||||
from synapse.config.logger import setup_logging
|
||||
from synapse.http.site import SynapseSite
|
||||
from synapse.logging.context import LoggingContext, run_in_background
|
||||
from synapse.metrics import METRICS_PREFIX, MetricsResource, RegistryProxy
|
||||
from synapse.metrics import RegistryProxy
|
||||
from synapse.metrics.resource import METRICS_PREFIX, MetricsResource
|
||||
from synapse.replication.slave.storage.appservice import SlavedApplicationServiceStore
|
||||
from synapse.replication.slave.storage.directory import DirectoryStore
|
||||
from synapse.replication.slave.storage.events import SlavedEventStore
|
||||
@@ -168,9 +169,7 @@ def start(config_options):
|
||||
)
|
||||
|
||||
ps.setup()
|
||||
reactor.addSystemEventTrigger(
|
||||
"before", "startup", _base.start, ps, config.worker_listeners
|
||||
)
|
||||
reactor.callWhenRunning(_base.start, ps, config.worker_listeners)
|
||||
|
||||
_base.start_worker_reactor("synapse-appservice", config)
|
||||
|
||||
|
||||
@@ -28,7 +28,8 @@ from synapse.config.logger import setup_logging
|
||||
from synapse.http.server import JsonResource
|
||||
from synapse.http.site import SynapseSite
|
||||
from synapse.logging.context import LoggingContext
|
||||
from synapse.metrics import METRICS_PREFIX, MetricsResource, RegistryProxy
|
||||
from synapse.metrics import RegistryProxy
|
||||
from synapse.metrics.resource import METRICS_PREFIX, MetricsResource
|
||||
from synapse.replication.slave.storage._base import BaseSlavedStore
|
||||
from synapse.replication.slave.storage.account_data import SlavedAccountDataStore
|
||||
from synapse.replication.slave.storage.appservice import SlavedApplicationServiceStore
|
||||
@@ -194,9 +195,7 @@ def start(config_options):
|
||||
)
|
||||
|
||||
ss.setup()
|
||||
reactor.addSystemEventTrigger(
|
||||
"before", "startup", _base.start, ss, config.worker_listeners
|
||||
)
|
||||
reactor.callWhenRunning(_base.start, ss, config.worker_listeners)
|
||||
|
||||
_base.start_worker_reactor("synapse-client-reader", config)
|
||||
|
||||
|
||||
@@ -28,7 +28,8 @@ from synapse.config.logger import setup_logging
|
||||
from synapse.http.server import JsonResource
|
||||
from synapse.http.site import SynapseSite
|
||||
from synapse.logging.context import LoggingContext
|
||||
from synapse.metrics import METRICS_PREFIX, MetricsResource, RegistryProxy
|
||||
from synapse.metrics import RegistryProxy
|
||||
from synapse.metrics.resource import METRICS_PREFIX, MetricsResource
|
||||
from synapse.replication.slave.storage._base import BaseSlavedStore
|
||||
from synapse.replication.slave.storage.account_data import SlavedAccountDataStore
|
||||
from synapse.replication.slave.storage.appservice import SlavedApplicationServiceStore
|
||||
@@ -193,9 +194,7 @@ def start(config_options):
|
||||
)
|
||||
|
||||
ss.setup()
|
||||
reactor.addSystemEventTrigger(
|
||||
"before", "startup", _base.start, ss, config.worker_listeners
|
||||
)
|
||||
reactor.callWhenRunning(_base.start, ss, config.worker_listeners)
|
||||
|
||||
_base.start_worker_reactor("synapse-event-creator", config)
|
||||
|
||||
|
||||
@@ -29,7 +29,8 @@ from synapse.config.logger import setup_logging
|
||||
from synapse.federation.transport.server import TransportLayerServer
|
||||
from synapse.http.site import SynapseSite
|
||||
from synapse.logging.context import LoggingContext
|
||||
from synapse.metrics import METRICS_PREFIX, MetricsResource, RegistryProxy
|
||||
from synapse.metrics import RegistryProxy
|
||||
from synapse.metrics.resource import METRICS_PREFIX, MetricsResource
|
||||
from synapse.replication.slave.storage._base import BaseSlavedStore
|
||||
from synapse.replication.slave.storage.account_data import SlavedAccountDataStore
|
||||
from synapse.replication.slave.storage.appservice import SlavedApplicationServiceStore
|
||||
@@ -175,9 +176,7 @@ def start(config_options):
|
||||
)
|
||||
|
||||
ss.setup()
|
||||
reactor.addSystemEventTrigger(
|
||||
"before", "startup", _base.start, ss, config.worker_listeners
|
||||
)
|
||||
reactor.callWhenRunning(_base.start, ss, config.worker_listeners)
|
||||
|
||||
_base.start_worker_reactor("synapse-federation-reader", config)
|
||||
|
||||
|
||||
@@ -28,8 +28,9 @@ from synapse.config.logger import setup_logging
|
||||
from synapse.federation import send_queue
|
||||
from synapse.http.site import SynapseSite
|
||||
from synapse.logging.context import LoggingContext, run_in_background
|
||||
from synapse.metrics import METRICS_PREFIX, MetricsResource, RegistryProxy
|
||||
from synapse.metrics import RegistryProxy
|
||||
from synapse.metrics.background_process_metrics import run_as_background_process
|
||||
from synapse.metrics.resource import METRICS_PREFIX, MetricsResource
|
||||
from synapse.replication.slave.storage.deviceinbox import SlavedDeviceInboxStore
|
||||
from synapse.replication.slave.storage.devices import SlavedDeviceStore
|
||||
from synapse.replication.slave.storage.events import SlavedEventStore
|
||||
@@ -198,9 +199,7 @@ def start(config_options):
|
||||
)
|
||||
|
||||
ss.setup()
|
||||
reactor.addSystemEventTrigger(
|
||||
"before", "startup", _base.start, ss, config.worker_listeners
|
||||
)
|
||||
reactor.callWhenRunning(_base.start, ss, config.worker_listeners)
|
||||
|
||||
_base.start_worker_reactor("synapse-federation-sender", config)
|
||||
|
||||
|
||||
@@ -30,7 +30,8 @@ from synapse.http.server import JsonResource
|
||||
from synapse.http.servlet import RestServlet, parse_json_object_from_request
|
||||
from synapse.http.site import SynapseSite
|
||||
from synapse.logging.context import LoggingContext
|
||||
from synapse.metrics import METRICS_PREFIX, MetricsResource, RegistryProxy
|
||||
from synapse.metrics import RegistryProxy
|
||||
from synapse.metrics.resource import METRICS_PREFIX, MetricsResource
|
||||
from synapse.replication.slave.storage._base import BaseSlavedStore
|
||||
from synapse.replication.slave.storage.appservice import SlavedApplicationServiceStore
|
||||
from synapse.replication.slave.storage.client_ips import SlavedClientIpStore
|
||||
@@ -70,12 +71,12 @@ class PresenceStatusStubServlet(RestServlet):
|
||||
except HttpResponseException as e:
|
||||
raise e.to_synapse_error()
|
||||
|
||||
return (200, result)
|
||||
defer.returnValue((200, result))
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def on_PUT(self, request, user_id):
|
||||
yield self.auth.get_user_by_req(request)
|
||||
return (200, {})
|
||||
defer.returnValue((200, {}))
|
||||
|
||||
|
||||
class KeyUploadServlet(RestServlet):
|
||||
@@ -126,11 +127,11 @@ class KeyUploadServlet(RestServlet):
|
||||
self.main_uri + request.uri.decode("ascii"), body, headers=headers
|
||||
)
|
||||
|
||||
return (200, result)
|
||||
defer.returnValue((200, result))
|
||||
else:
|
||||
# Just interested in counts.
|
||||
result = yield self.store.count_e2e_one_time_keys(user_id, device_id)
|
||||
return (200, {"one_time_key_counts": result})
|
||||
defer.returnValue((200, {"one_time_key_counts": result}))
|
||||
|
||||
|
||||
class FrontendProxySlavedStore(
|
||||
@@ -247,9 +248,7 @@ def start(config_options):
|
||||
)
|
||||
|
||||
ss.setup()
|
||||
reactor.addSystemEventTrigger(
|
||||
"before", "startup", _base.start, ss, config.worker_listeners
|
||||
)
|
||||
reactor.callWhenRunning(_base.start, ss, config.worker_listeners)
|
||||
|
||||
_base.start_worker_reactor("synapse-frontend-proxy", config)
|
||||
|
||||
|
||||
7
synapse/app/homeserver.py
Normal file → Executable file
7
synapse/app/homeserver.py
Normal file → Executable file
@@ -55,8 +55,9 @@ from synapse.http.additional_resource import AdditionalResource
|
||||
from synapse.http.server import RootRedirect
|
||||
from synapse.http.site import SynapseSite
|
||||
from synapse.logging.context import LoggingContext
|
||||
from synapse.metrics import METRICS_PREFIX, MetricsResource, RegistryProxy
|
||||
from synapse.metrics import RegistryProxy
|
||||
from synapse.metrics.background_process_metrics import run_as_background_process
|
||||
from synapse.metrics.resource import METRICS_PREFIX, MetricsResource
|
||||
from synapse.module_api import ModuleApi
|
||||
from synapse.python_dependencies import check_requirements
|
||||
from synapse.replication.http import REPLICATION_PREFIX, ReplicationRestResource
|
||||
@@ -406,7 +407,7 @@ def setup(config_options):
|
||||
if provision:
|
||||
yield acme.provision_certificate()
|
||||
|
||||
return provision
|
||||
defer.returnValue(provision)
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def reprovision_acme():
|
||||
@@ -447,7 +448,7 @@ def setup(config_options):
|
||||
reactor.stop()
|
||||
sys.exit(1)
|
||||
|
||||
reactor.addSystemEventTrigger("before", "startup", start)
|
||||
reactor.callWhenRunning(start)
|
||||
|
||||
return hs
|
||||
|
||||
|
||||
@@ -28,7 +28,8 @@ from synapse.config.homeserver import HomeServerConfig
|
||||
from synapse.config.logger import setup_logging
|
||||
from synapse.http.site import SynapseSite
|
||||
from synapse.logging.context import LoggingContext
|
||||
from synapse.metrics import METRICS_PREFIX, MetricsResource, RegistryProxy
|
||||
from synapse.metrics import RegistryProxy
|
||||
from synapse.metrics.resource import METRICS_PREFIX, MetricsResource
|
||||
from synapse.replication.slave.storage._base import BaseSlavedStore
|
||||
from synapse.replication.slave.storage.appservice import SlavedApplicationServiceStore
|
||||
from synapse.replication.slave.storage.client_ips import SlavedClientIpStore
|
||||
@@ -161,9 +162,7 @@ def start(config_options):
|
||||
)
|
||||
|
||||
ss.setup()
|
||||
reactor.addSystemEventTrigger(
|
||||
"before", "startup", _base.start, ss, config.worker_listeners
|
||||
)
|
||||
reactor.callWhenRunning(_base.start, ss, config.worker_listeners)
|
||||
|
||||
_base.start_worker_reactor("synapse-media-repository", config)
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user