Newsfile

Correctly retry replication HTTP requests on timeout
2020-09-28 14:33:56 +01:00 · 2020-09-28 14:33:56 +01:00
508 changed files with 7396 additions and 16451 deletions
@@ -1,10 +1,41 @@
 # This file serves as a blacklist for SyTest tests that we expect will fail in
 # Synapse when run under worker mode. For more details, see sytest-blacklist.

+Message history can be paginated
+
 Can re-join room if re-invited

+The only membership state included in an initial sync is for all the senders in the timeline
+
+Local device key changes get to remote servers
+
+If remote user leaves room we no longer receive device updates
+
+Forgotten room messages cannot be paginated
+
+Inbound federation can get public room list
+
+Members from the gap are included in gappy incr LL sync
+
+Leaves are present in non-gapped incremental syncs
+
+Old leaves are present in gapped incremental syncs
+
+User sees updates to presence from other users in the incremental sync.
+
+Gapped incremental syncs include all state changes
+
+Old members are included in gappy incr LL sync if they start speaking
+
 # new failures as of https://github.com/matrix-org/sytest/pull/732
 Device list doesn't change if remote server is down
+Remote servers cannot set power levels in rooms without existing powerlevels
+Remote servers should reject attempts by non-creators to set the power levels

 # https://buildkite.com/matrix-dot-org/synapse/builds/6134#6f67bf47-e234-474d-80e8-c6e1868b15c5
 Server correctly handles incoming m.device_list_update
+
+# this fails reliably with a torture level of 100 due to https://github.com/matrix-org/synapse/issues/6536
+Outbound federation requests missing prev_events and then asks for /state_ids and resolves the state
+
+Can get rooms/{roomId}/members at a given point
@@ -1,36 +1,22 @@
-version: 2.1
+version: 2
 jobs:
  dockerhubuploadrelease:
-    docker:
-      - image: docker:git
+    machine: true
    steps:
      - checkout
-      - setup_remote_docker
-      - docker_prepare
+      - run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:${CIRCLE_TAG} .
      - run: docker login --username $DOCKER_HUB_USERNAME --password $DOCKER_HUB_PASSWORD
-      - docker_build:
-          tag: -t matrixdotorg/synapse:${CIRCLE_TAG}
-          platforms: linux/amd64
-      - docker_build:
-          tag: -t matrixdotorg/synapse:${CIRCLE_TAG}
-          platforms: linux/amd64,linux/arm/v7,linux/arm64
-
+      - run: docker push matrixdotorg/synapse:${CIRCLE_TAG}
  dockerhubuploadlatest:
-    docker:
-      - image: docker:git
+    machine: true
    steps:
      - checkout
-      - setup_remote_docker
-      - docker_prepare
+      - run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:latest .
      - run: docker login --username $DOCKER_HUB_USERNAME --password $DOCKER_HUB_PASSWORD
-      - docker_build:
-          tag: -t matrixdotorg/synapse:latest
-          platforms: linux/amd64
-      - docker_build:
-          tag: -t matrixdotorg/synapse:latest
-          platforms: linux/amd64,linux/arm/v7,linux/arm64
+      - run: docker push matrixdotorg/synapse:latest

 workflows:
+  version: 2
  build:
    jobs:
      - dockerhubuploadrelease:
@@ -43,33 +29,3 @@ workflows:
          filters:
            branches:
              only: master
-
-commands:
-  docker_prepare:
-    description: Downloads the buildx cli plugin and enables multiarch images
-    parameters:
-      buildx_version:
-        type: string
-        default: "v0.4.1"
-    steps:
-      - run: apk add --no-cache curl
-      - run: mkdir -vp ~/.docker/cli-plugins/ ~/dockercache
-      - run: curl --silent -L "https://github.com/docker/buildx/releases/download/<< parameters.buildx_version >>/buildx-<< parameters.buildx_version >>.linux-amd64" > ~/.docker/cli-plugins/docker-buildx
-      - run: chmod a+x ~/.docker/cli-plugins/docker-buildx
-      # install qemu links in /proc/sys/fs/binfmt_misc on the docker instance running the circleci job
-      - run: docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
-      # create a context named `builder` for the builds
-      - run: docker context create builder
-      # create a buildx builder using the new context, and set it as the default
-      - run: docker buildx create builder --use
-
-  docker_build:
-    description: Builds and pushed images to dockerhub using buildx
-    parameters:
-      platforms:
-        type: string
-        default: linux/amd64
-      tag:
-        type: string
-    steps:
-      - run: docker buildx build -f docker/Dockerfile --push --platform << parameters.platforms >> --label gitsha1=${CIRCLE_SHA1} << parameters.tag >> --progress=plain .
@@ -21,7 +21,6 @@ _trial_temp*/
 /.python-version
 /*.signing.key
 /env/
-/.venv*/
 /homeserver*.yaml
 /logs
 /media_store/
@@ -1,390 +1,3 @@
-Synapse 1.23.0rc1 (2020-11-13)
-==============================
-
-Features
--------
-
- Add a push rule that highlights when a jitsi conference is created in a room. ([\#8286](https://github.com/matrix-org/synapse/issues/8286))
- Add an admin api to delete a single file or files that were not used for a defined time from server. Contributed by @dklimpel. ([\#8519](https://github.com/matrix-org/synapse/issues/8519))
- Split admin API for reported events (`GET /_synapse/admin/v1/event_reports`) into detail and list endpoints. This is a breaking change to #8217 which was introduced in Synapse v1.21.0. Those who already use this API should check their scripts. Contributed by @dklimpel. ([\#8539](https://github.com/matrix-org/synapse/issues/8539))
- Support generating structured logs via the standard logging configuration. ([\#8607](https://github.com/matrix-org/synapse/issues/8607), [\#8685](https://github.com/matrix-org/synapse/issues/8685))
- Add an admin API to allow server admins to list users' pushers. Contributed by @dklimpel. ([\#8610](https://github.com/matrix-org/synapse/issues/8610), [\#8689](https://github.com/matrix-org/synapse/issues/8689))
- Add an admin API `GET /_synapse/admin/v1/users/<user_id>/media` to get information about uploaded media. Contributed by @dklimpel. ([\#8647](https://github.com/matrix-org/synapse/issues/8647))
- Add an admin API for local user media statistics. Contributed by @dklimpel. ([\#8700](https://github.com/matrix-org/synapse/issues/8700))
- Add `displayname` to Shared-Secret Registration for admins. ([\#8722](https://github.com/matrix-org/synapse/issues/8722))
-
-
-Bugfixes
--------
-
- Fix fetching of E2E cross signing keys over federation when only one of the master key and device signing key is cached already. ([\#8455](https://github.com/matrix-org/synapse/issues/8455))
- Fix a bug where Synapse would blindly forward bad responses from federation to clients when retrieving profile information. ([\#8580](https://github.com/matrix-org/synapse/issues/8580))
- Fix a bug where the account validity endpoint would silently fail if the user ID did not have an expiration time. It now returns a 400 error. ([\#8620](https://github.com/matrix-org/synapse/issues/8620))
- Fix email notifications for invites without local state. ([\#8627](https://github.com/matrix-org/synapse/issues/8627))
- Fix handling of invalid group IDs to return a 400 rather than log an exception and return a 500. ([\#8628](https://github.com/matrix-org/synapse/issues/8628))
- Fix handling of User-Agent headers that are invalid UTF-8, which caused user agents of users to not get correctly recorded. ([\#8632](https://github.com/matrix-org/synapse/issues/8632))
- Fix a bug in the `joined_rooms` admin API if the user has never joined any rooms. The bug was introduced, along with the API, in v1.21.0. ([\#8643](https://github.com/matrix-org/synapse/issues/8643))
- Fix exception during handling multiple concurrent requests for remote media when using multiple media repositories. ([\#8682](https://github.com/matrix-org/synapse/issues/8682))
- Fix bug that prevented Synapse from recovering after losing connection to the database. ([\#8726](https://github.com/matrix-org/synapse/issues/8726))
- Fix bug where the `/_synapse/admin/v1/send_server_notice` API could send notices to non-notice rooms. ([\#8728](https://github.com/matrix-org/synapse/issues/8728))
- Fix PostgreSQL port script fails when DB has no backfilled events. Broke in v1.21.0. ([\#8729](https://github.com/matrix-org/synapse/issues/8729))
- Fix PostgreSQL port script to correctly handle foreign key constraints. Broke in v1.21.0. ([\#8730](https://github.com/matrix-org/synapse/issues/8730))
- Fix PostgreSQL port script so that it can be run again after a failure. Broke in v1.21.0. ([\#8755](https://github.com/matrix-org/synapse/issues/8755))
-
-
-Improved Documentation
----------------------
-
- Instructions for Azure AD in the OpenID Connect documentation. Contributed by peterk. ([\#8582](https://github.com/matrix-org/synapse/issues/8582))
- Improve the sample configuration for single sign-on providers. ([\#8635](https://github.com/matrix-org/synapse/issues/8635))
- Fix the filepath of Dex's example config and the link to Dex's Getting Started guide in the OpenID Connect docs. ([\#8657](https://github.com/matrix-org/synapse/issues/8657))
- Note support for Python 3.9. ([\#8665](https://github.com/matrix-org/synapse/issues/8665))
- Minor updates to docs on running tests. ([\#8666](https://github.com/matrix-org/synapse/issues/8666))
- Interlink prometheus/grafana documentation. ([\#8667](https://github.com/matrix-org/synapse/issues/8667))
- Notes on SSO logins and media_repository worker. ([\#8701](https://github.com/matrix-org/synapse/issues/8701))
- Document experimental support for running multiple event persisters. ([\#8706](https://github.com/matrix-org/synapse/issues/8706))
- Add information regarding the various sources of, and expected contributions to, Synapse's documentation to `CONTRIBUTING.md`. ([\#8714](https://github.com/matrix-org/synapse/issues/8714))
- Migrate documentation `docs/admin_api/event_reports` to markdown. ([\#8742](https://github.com/matrix-org/synapse/issues/8742))
- Add some helpful hints to the README for new Synapse developers. Contributed by @chagai95. ([\#8746](https://github.com/matrix-org/synapse/issues/8746))
-
-
-Internal Changes
----------------
-
- Optimise `/createRoom` with multiple invited users. ([\#8559](https://github.com/matrix-org/synapse/issues/8559))
- Implement and use an @lru_cache decorator. ([\#8595](https://github.com/matrix-org/synapse/issues/8595))
- Don't instansiate Requester directly. ([\#8614](https://github.com/matrix-org/synapse/issues/8614))
- Type hints for `RegistrationStore`. ([\#8615](https://github.com/matrix-org/synapse/issues/8615))
- Change schema to support access tokens belonging to one user but granting access to another. ([\#8616](https://github.com/matrix-org/synapse/issues/8616))
- Remove unused OPTIONS handlers. ([\#8621](https://github.com/matrix-org/synapse/issues/8621))
- Run `mypy` as part of the lint.sh script. ([\#8633](https://github.com/matrix-org/synapse/issues/8633))
- Correct Synapse's PyPI package name in the OpenID Connect installation instructions. ([\#8634](https://github.com/matrix-org/synapse/issues/8634))
- Catch exceptions during initialization of `password_providers`. Contributed by Nicolai Søborg. ([\#8636](https://github.com/matrix-org/synapse/issues/8636))
- Fix typos and spelling errors in the code. ([\#8639](https://github.com/matrix-org/synapse/issues/8639))
- Reduce number of OpenTracing spans started. ([\#8640](https://github.com/matrix-org/synapse/issues/8640), [\#8668](https://github.com/matrix-org/synapse/issues/8668), [\#8670](https://github.com/matrix-org/synapse/issues/8670))
- Add field `total` to device list in admin API. ([\#8644](https://github.com/matrix-org/synapse/issues/8644))
- Add more type hints to the application services code. ([\#8655](https://github.com/matrix-org/synapse/issues/8655), [\#8693](https://github.com/matrix-org/synapse/issues/8693))
- Tell Black to format code for Python 3.5. ([\#8664](https://github.com/matrix-org/synapse/issues/8664))
- Don't pull event from DB when handling replication traffic. ([\#8669](https://github.com/matrix-org/synapse/issues/8669))
- Abstract some invite-related code in preparation for landing knocking. ([\#8671](https://github.com/matrix-org/synapse/issues/8671), [\#8688](https://github.com/matrix-org/synapse/issues/8688))
- Clarify representation of events in logfiles. ([\#8679](https://github.com/matrix-org/synapse/issues/8679))
- Don't require `hiredis` package to be installed to run unit tests. ([\#8680](https://github.com/matrix-org/synapse/issues/8680))
- Fix typing info on cache call signature to accept `on_invalidate`. ([\#8684](https://github.com/matrix-org/synapse/issues/8684))
- Fail tests if they do not await coroutines. ([\#8690](https://github.com/matrix-org/synapse/issues/8690))
- Improve start time by adding an index to `e2e_cross_signing_keys.stream_id`. ([\#8694](https://github.com/matrix-org/synapse/issues/8694))
- Re-organize the structured logging code to separate the TCP transport handling from the JSON formatting. ([\#8697](https://github.com/matrix-org/synapse/issues/8697))
- Use Python 3.8 in Docker images by default. ([\#8698](https://github.com/matrix-org/synapse/issues/8698))
- Remove the "draft" status of the Room Details Admin API. ([\#8702](https://github.com/matrix-org/synapse/issues/8702))
- Improve the error returned when a non-string displayname or avatar_url is used when updating a user's profile. ([\#8705](https://github.com/matrix-org/synapse/issues/8705))
- Block attempts by clients to send server ACLs, or redactions of server ACLs, that would result in the local server being blocked from the room. ([\#8708](https://github.com/matrix-org/synapse/issues/8708))
- Add metrics the allow the local sysadmin to track 3PID `/requestToken` requests. ([\#8712](https://github.com/matrix-org/synapse/issues/8712))
- Consolidate duplicated lists of purged tables that are checked in tests. ([\#8713](https://github.com/matrix-org/synapse/issues/8713))
- Add some `mdui:UIInfo` element examples for `saml2_config` in the homeserver config. ([\#8718](https://github.com/matrix-org/synapse/issues/8718))
- Improve the error message returned when a remote server incorrectly sets the `Content-Type` header in response to a JSON request. ([\#8719](https://github.com/matrix-org/synapse/issues/8719))
- Speed up repeated state resolutions on the same room by caching event ID to auth event ID lookups. ([\#8752](https://github.com/matrix-org/synapse/issues/8752))
-
-
-Synapse 1.22.1 (2020-10-30)
-===========================
-
-Bugfixes
--------
-
- Fix a bug where an appservice may not be forwarded events for a room it was recently invited to. Broke in v1.22.0. ([\#8676](https://github.com/matrix-org/synapse/issues/8676))
- Fix `Object of type frozendict is not JSON serializable` exceptions when using third-party event rules. Broke in v1.22.0. ([\#8678](https://github.com/matrix-org/synapse/issues/8678))
-
-
-Synapse 1.22.0 (2020-10-27)
-===========================
-
-No significant changes.
-
-
-Synapse 1.22.0rc2 (2020-10-26)
-==============================
-
-Bugfixes
--------
-
- Fix bugs where ephemeral events were not sent to appservices. Broke in v1.22.0rc1. ([\#8648](https://github.com/matrix-org/synapse/issues/8648), [\#8656](https://github.com/matrix-org/synapse/issues/8656))
- Fix `user_daily_visits` table to not have duplicate rows per user/device due to multiple user agents. Broke in v1.22.0rc1. ([\#8654](https://github.com/matrix-org/synapse/issues/8654))
-
-Synapse 1.22.0rc1 (2020-10-22)
-==============================
-
-Features
--------
-
- Add a configuration option for always using the "userinfo endpoint" for OpenID Connect. This fixes support for some identity providers, e.g. GitLab. Contributed by Benjamin Koch. ([\#7658](https://github.com/matrix-org/synapse/issues/7658))
- Add ability for `ThirdPartyEventRules` modules to query and manipulate whether a room is in the public rooms directory. ([\#8292](https://github.com/matrix-org/synapse/issues/8292), [\#8467](https://github.com/matrix-org/synapse/issues/8467))
- Add support for olm fallback keys ([MSC2732](https://github.com/matrix-org/matrix-doc/pull/2732)). ([\#8312](https://github.com/matrix-org/synapse/issues/8312), [\#8501](https://github.com/matrix-org/synapse/issues/8501))
- Add support for running background tasks in a separate worker process. ([\#8369](https://github.com/matrix-org/synapse/issues/8369), [\#8458](https://github.com/matrix-org/synapse/issues/8458), [\#8489](https://github.com/matrix-org/synapse/issues/8489), [\#8513](https://github.com/matrix-org/synapse/issues/8513), [\#8544](https://github.com/matrix-org/synapse/issues/8544), [\#8599](https://github.com/matrix-org/synapse/issues/8599))
- Add support for device dehydration ([MSC2697](https://github.com/matrix-org/matrix-doc/pull/2697)). ([\#8380](https://github.com/matrix-org/synapse/issues/8380))
- Add support for [MSC2409](https://github.com/matrix-org/matrix-doc/pull/2409), which allows sending typing, read receipts, and presence events to appservices. ([\#8437](https://github.com/matrix-org/synapse/issues/8437), [\#8590](https://github.com/matrix-org/synapse/issues/8590))
- Change default room version to "6", per [MSC2788](https://github.com/matrix-org/matrix-doc/pull/2788). ([\#8461](https://github.com/matrix-org/synapse/issues/8461))
- Add the ability to send non-membership events into a room via the `ModuleApi`. ([\#8479](https://github.com/matrix-org/synapse/issues/8479))
- Increase default upload size limit from 10M to 50M. Contributed by @Akkowicz. ([\#8502](https://github.com/matrix-org/synapse/issues/8502))
- Add support for modifying event content in `ThirdPartyRules` modules. ([\#8535](https://github.com/matrix-org/synapse/issues/8535), [\#8564](https://github.com/matrix-org/synapse/issues/8564))
-
-
-Bugfixes
--------
-
- Fix a longstanding bug where invalid ignored users in account data could break clients. ([\#8454](https://github.com/matrix-org/synapse/issues/8454))
- Fix a bug where backfilling a room with an event that was missing the `redacts` field would break. ([\#8457](https://github.com/matrix-org/synapse/issues/8457))
- Don't attempt to respond to some requests if the client has already disconnected. ([\#8465](https://github.com/matrix-org/synapse/issues/8465))
- Fix message duplication if something goes wrong after persisting the event. ([\#8476](https://github.com/matrix-org/synapse/issues/8476))
- Fix incremental sync returning an incorrect `prev_batch` token in timeline section, which when used to paginate returned events that were included in the incremental sync. Broken since v0.16.0. ([\#8486](https://github.com/matrix-org/synapse/issues/8486))
- Expose the `uk.half-shot.msc2778.login.application_service` to clients from the login API. This feature was added in v1.21.0, but was not exposed as a potential login flow. ([\#8504](https://github.com/matrix-org/synapse/issues/8504))
- Fix error code for `/profile/{userId}/displayname` to be `M_BAD_JSON`. ([\#8517](https://github.com/matrix-org/synapse/issues/8517))
- Fix a bug introduced in v1.7.0 that could cause Synapse to insert values from non-state `m.room.retention` events into the `room_retention` database table. ([\#8527](https://github.com/matrix-org/synapse/issues/8527))
- Fix not sending events over federation when using sharded event writers. ([\#8536](https://github.com/matrix-org/synapse/issues/8536))
- Fix a long standing bug where email notifications for encrypted messages were blank. ([\#8545](https://github.com/matrix-org/synapse/issues/8545))
- Fix increase in the number of `There was no active span...` errors logged when using OpenTracing. ([\#8567](https://github.com/matrix-org/synapse/issues/8567))
- Fix a bug that prevented errors encountered during execution of the `synapse_port_db` from being correctly printed. ([\#8585](https://github.com/matrix-org/synapse/issues/8585))
- Fix appservice transactions to only include a maximum of 100 persistent and 100 ephemeral events. ([\#8606](https://github.com/matrix-org/synapse/issues/8606))
-
-
-Updates to the Docker image
---------------------------
-
- Added multi-arch support (arm64,arm/v7) for the docker images. Contributed by @maquis196. ([\#7921](https://github.com/matrix-org/synapse/issues/7921))
- Add support for passing commandline args to the synapse process. Contributed by @samuel-p. ([\#8390](https://github.com/matrix-org/synapse/issues/8390))
-
-
-Improved Documentation
----------------------
-
- Update the directions for using the manhole with coroutines. ([\#8462](https://github.com/matrix-org/synapse/issues/8462))
- Improve readme by adding new shield.io badges. ([\#8493](https://github.com/matrix-org/synapse/issues/8493))
- Added note about docker in manhole.md regarding which ip address to bind to. Contributed by @Maquis196. ([\#8526](https://github.com/matrix-org/synapse/issues/8526))
- Document the new behaviour of the `allowed_lifetime_min` and `allowed_lifetime_max` settings in the room retention configuration. ([\#8529](https://github.com/matrix-org/synapse/issues/8529))
-
-
-Deprecations and Removals
-------------------------
-
- Drop unused `device_max_stream_id` table. ([\#8589](https://github.com/matrix-org/synapse/issues/8589))
-
-
-Internal Changes
----------------
-
- Check for unreachable code with mypy. ([\#8432](https://github.com/matrix-org/synapse/issues/8432))
- Add unit test for event persister sharding. ([\#8433](https://github.com/matrix-org/synapse/issues/8433))
- Allow events to be sent to clients sooner when using sharded event persisters. ([\#8439](https://github.com/matrix-org/synapse/issues/8439), [\#8488](https://github.com/matrix-org/synapse/issues/8488), [\#8496](https://github.com/matrix-org/synapse/issues/8496), [\#8499](https://github.com/matrix-org/synapse/issues/8499))
- Configure `public_baseurl` when using demo scripts. ([\#8443](https://github.com/matrix-org/synapse/issues/8443))
- Add SQL logging on queries that happen during startup. ([\#8448](https://github.com/matrix-org/synapse/issues/8448))
- Speed up unit tests when using PostgreSQL. ([\#8450](https://github.com/matrix-org/synapse/issues/8450))
- Remove redundant database loads of stream_ordering for events we already have. ([\#8452](https://github.com/matrix-org/synapse/issues/8452))
- Reduce inconsistencies between codepaths for membership and non-membership events. ([\#8463](https://github.com/matrix-org/synapse/issues/8463))
- Combine `SpamCheckerApi` with the more generic `ModuleApi`. ([\#8464](https://github.com/matrix-org/synapse/issues/8464))
- Additional testing for `ThirdPartyEventRules`. ([\#8468](https://github.com/matrix-org/synapse/issues/8468))
- Add `-d` option to `./scripts-dev/lint.sh` to lint files that have changed since the last git commit. ([\#8472](https://github.com/matrix-org/synapse/issues/8472))
- Unblacklist some sytests. ([\#8474](https://github.com/matrix-org/synapse/issues/8474))
- Include the log level in the phone home stats. ([\#8477](https://github.com/matrix-org/synapse/issues/8477))
- Remove outdated sphinx documentation, scripts and configuration. ([\#8480](https://github.com/matrix-org/synapse/issues/8480))
- Clarify error message when plugin config parsers raise an error. ([\#8492](https://github.com/matrix-org/synapse/issues/8492))
- Remove the deprecated `Handlers` object. ([\#8494](https://github.com/matrix-org/synapse/issues/8494))
- Fix a threadsafety bug in unit tests. ([\#8497](https://github.com/matrix-org/synapse/issues/8497))
- Add user agent to user_daily_visits table. ([\#8503](https://github.com/matrix-org/synapse/issues/8503))
- Add type hints to various parts of the code base. ([\#8407](https://github.com/matrix-org/synapse/issues/8407), [\#8505](https://github.com/matrix-org/synapse/issues/8505), [\#8507](https://github.com/matrix-org/synapse/issues/8507), [\#8547](https://github.com/matrix-org/synapse/issues/8547), [\#8562](https://github.com/matrix-org/synapse/issues/8562), [\#8609](https://github.com/matrix-org/synapse/issues/8609))
- Remove unused code from the test framework. ([\#8514](https://github.com/matrix-org/synapse/issues/8514))
- Apply some internal fixes to the `HomeServer` class to make its code more idiomatic and statically-verifiable. ([\#8515](https://github.com/matrix-org/synapse/issues/8515))
- Factor out common code between `RoomMemberHandler._locally_reject_invite` and `EventCreationHandler.create_event`. ([\#8537](https://github.com/matrix-org/synapse/issues/8537))
- Improve database performance by executing more queries without starting transactions. ([\#8542](https://github.com/matrix-org/synapse/issues/8542))
- Rename `Cache` to `DeferredCache`, to better reflect its purpose. ([\#8548](https://github.com/matrix-org/synapse/issues/8548))
- Move metric registration code down into `LruCache`. ([\#8561](https://github.com/matrix-org/synapse/issues/8561), [\#8591](https://github.com/matrix-org/synapse/issues/8591))
- Replace `DeferredCache` with the lighter-weight `LruCache` where possible. ([\#8563](https://github.com/matrix-org/synapse/issues/8563))
- Add virtualenv-generated folders to `.gitignore`. ([\#8566](https://github.com/matrix-org/synapse/issues/8566))
- Add `get_immediate` method to `DeferredCache`. ([\#8568](https://github.com/matrix-org/synapse/issues/8568))
- Fix mypy not properly checking across the codebase, additionally, fix a typing assertion error in `handlers/auth.py`. ([\#8569](https://github.com/matrix-org/synapse/issues/8569))
- Fix `synmark` benchmark runner. ([\#8571](https://github.com/matrix-org/synapse/issues/8571))
- Modify `DeferredCache.get()` to return `Deferred`s instead of `ObservableDeferred`s. ([\#8572](https://github.com/matrix-org/synapse/issues/8572))
- Adjust a protocol-type definition to fit `sqlite3` assertions. ([\#8577](https://github.com/matrix-org/synapse/issues/8577))
- Support macOS on the `synmark` benchmark runner. ([\#8578](https://github.com/matrix-org/synapse/issues/8578))
- Update `mypy` static type checker to 0.790. ([\#8583](https://github.com/matrix-org/synapse/issues/8583), [\#8600](https://github.com/matrix-org/synapse/issues/8600))
- Re-organize the structured logging code to separate the TCP transport handling from the JSON formatting. ([\#8587](https://github.com/matrix-org/synapse/issues/8587))
- Remove extraneous unittest logging decorators from unit tests. ([\#8592](https://github.com/matrix-org/synapse/issues/8592))
- Minor optimisations in caching code. ([\#8593](https://github.com/matrix-org/synapse/issues/8593), [\#8594](https://github.com/matrix-org/synapse/issues/8594))
-
-
-Synapse 1.21.2 (2020-10-15)
-===========================
-
-Debian packages and Docker images have been rebuilt using the latest versions of dependency libraries, including authlib 0.15.1. Please see bugfixes below.
-
-Security advisory
-----------------
-
-* HTML pages served via Synapse were vulnerable to cross-site scripting (XSS)
-  attacks. All server administrators are encouraged to upgrade.
-  ([\#8444](https://github.com/matrix-org/synapse/pull/8444))
-  ([CVE-2020-26891](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-26891))
-
-  This fix was originally included in v1.21.0 but was missing a security advisory.
-
-  This was reported by [Denis Kasak](https://github.com/dkasak).
-
-Bugfixes
--------
-
- Fix rare bug where sending an event would fail due to a racey assertion. ([\#8530](https://github.com/matrix-org/synapse/issues/8530))
- An updated version of the authlib dependency is included in the Docker and Debian images to fix an issue using OpenID Connect. See [\#8534](https://github.com/matrix-org/synapse/issues/8534) for details.
-
-
-Synapse 1.21.1 (2020-10-13)
-===========================
-
-This release fixes a regression in v1.21.0 that prevented debian packages from being built.
-It is otherwise identical to v1.21.0.
-
-Synapse 1.21.0 (2020-10-12)
-===========================
-
-No significant changes since v1.21.0rc3.
-
-As [noted in
-v1.20.0](https://github.com/matrix-org/synapse/blob/release-v1.21.0/CHANGES.md#synapse-1200-2020-09-22),
-a future release will drop support for accessing Synapse's
-[Admin API](https://github.com/matrix-org/synapse/tree/master/docs/admin_api) under the
-`/_matrix/client/*` endpoint prefixes. At that point, the Admin API will only
-be accessible under `/_synapse/admin`.
-
-
-Synapse 1.21.0rc3 (2020-10-08)
-==============================
-
-Bugfixes
--------
-
- Fix duplication of events on high traffic servers, caused by PostgreSQL `could not serialize access due to concurrent update` errors. ([\#8456](https://github.com/matrix-org/synapse/issues/8456))
-
-
-Internal Changes
----------------
-
- Add Groovy Gorilla to the list of distributions we build `.deb`s for. ([\#8475](https://github.com/matrix-org/synapse/issues/8475))
-
-
-Synapse 1.21.0rc2 (2020-10-02)
-==============================
-
-Features
--------
-
- Convert additional templates from inline HTML to Jinja2 templates. ([\#8444](https://github.com/matrix-org/synapse/issues/8444))
-
-Bugfixes
--------
-
- Fix a regression in v1.21.0rc1 which broke thumbnails of remote media. ([\#8438](https://github.com/matrix-org/synapse/issues/8438))
- Do not expose the experimental `uk.half-shot.msc2778.login.application_service` flow in the login API, which caused a compatibility problem with Element iOS. ([\#8440](https://github.com/matrix-org/synapse/issues/8440))
- Fix malformed log line in new federation "catch up" logic. ([\#8442](https://github.com/matrix-org/synapse/issues/8442))
- Fix DB query on startup for negative streams which caused long start up times. Introduced in [\#8374](https://github.com/matrix-org/synapse/issues/8374). ([\#8447](https://github.com/matrix-org/synapse/issues/8447))
-
-
-Synapse 1.21.0rc1 (2020-10-01)
-==============================
-
-Features
--------
-
- Require the user to confirm that their password should be reset after clicking the email confirmation link. ([\#8004](https://github.com/matrix-org/synapse/issues/8004))
- Add an admin API `GET /_synapse/admin/v1/event_reports` to read entries of table `event_reports`. Contributed by @dklimpel. ([\#8217](https://github.com/matrix-org/synapse/issues/8217))
- Consolidate the SSO error template across all configuration. ([\#8248](https://github.com/matrix-org/synapse/issues/8248), [\#8405](https://github.com/matrix-org/synapse/issues/8405))
- Add a configuration option to specify a whitelist of domains that a user can be redirected to after validating their email or phone number. ([\#8275](https://github.com/matrix-org/synapse/issues/8275), [\#8417](https://github.com/matrix-org/synapse/issues/8417))
- Add experimental support for sharding event persister. ([\#8294](https://github.com/matrix-org/synapse/issues/8294), [\#8387](https://github.com/matrix-org/synapse/issues/8387), [\#8396](https://github.com/matrix-org/synapse/issues/8396), [\#8419](https://github.com/matrix-org/synapse/issues/8419))
- Add the room topic and avatar to the room details admin API. ([\#8305](https://github.com/matrix-org/synapse/issues/8305))
- Add an admin API for querying rooms where a user is a member. Contributed by @dklimpel. ([\#8306](https://github.com/matrix-org/synapse/issues/8306))
- Add `uk.half-shot.msc2778.login.application_service` login type to allow appservices to login. ([\#8320](https://github.com/matrix-org/synapse/issues/8320))
- Add a configuration option that allows existing users to log in with OpenID Connect. Contributed by @BBBSnowball and @OmmyZhang. ([\#8345](https://github.com/matrix-org/synapse/issues/8345))
- Add prometheus metrics for replication requests. ([\#8406](https://github.com/matrix-org/synapse/issues/8406))
- Support passing additional single sign-on parameters to the client. ([\#8413](https://github.com/matrix-org/synapse/issues/8413))
- Add experimental reporting of metrics on expensive rooms for state-resolution. ([\#8420](https://github.com/matrix-org/synapse/issues/8420))
- Add experimental prometheus metric to track numbers of "large" rooms for state resolutiom. ([\#8425](https://github.com/matrix-org/synapse/issues/8425))
- Add prometheus metrics to track federation delays. ([\#8430](https://github.com/matrix-org/synapse/issues/8430))
-
-
-Bugfixes
--------
-
- Fix a bug in the media repository where remote thumbnails with the same size but different crop methods would overwrite each other. Contributed by @deepbluev7. ([\#7124](https://github.com/matrix-org/synapse/issues/7124))
- Fix inconsistent handling of non-existent push rules, and stop tracking the `enabled` state of removed push rules. ([\#7796](https://github.com/matrix-org/synapse/issues/7796))
- Fix a longstanding bug when storing a media file with an empty `upload_name`. ([\#7905](https://github.com/matrix-org/synapse/issues/7905))
- Fix messages not being sent over federation until an event is sent into the same room. ([\#8230](https://github.com/matrix-org/synapse/issues/8230), [\#8247](https://github.com/matrix-org/synapse/issues/8247), [\#8258](https://github.com/matrix-org/synapse/issues/8258), [\#8272](https://github.com/matrix-org/synapse/issues/8272), [\#8322](https://github.com/matrix-org/synapse/issues/8322))
- Fix a longstanding bug where files that could not be thumbnailed would result in an Internal Server Error. ([\#8236](https://github.com/matrix-org/synapse/issues/8236), [\#8435](https://github.com/matrix-org/synapse/issues/8435))
- Upgrade minimum version of `canonicaljson` to version 1.4.0, to fix an unicode encoding issue. ([\#8262](https://github.com/matrix-org/synapse/issues/8262))
- Fix longstanding bug which could lead to incomplete database upgrades on SQLite. ([\#8265](https://github.com/matrix-org/synapse/issues/8265))
- Fix stack overflow when stderr is redirected to the logging system, and the logging system encounters an error. ([\#8268](https://github.com/matrix-org/synapse/issues/8268))
- Fix a bug which cause the logging system to report errors, if `DEBUG` was enabled and no `context` filter was applied. ([\#8278](https://github.com/matrix-org/synapse/issues/8278))
- Fix edge case where push could get delayed for a user until a later event was pushed. ([\#8287](https://github.com/matrix-org/synapse/issues/8287))
- Fix fetching malformed events from remote servers. ([\#8324](https://github.com/matrix-org/synapse/issues/8324))
- Fix `UnboundLocalError` from occuring when appservices send a malformed register request. ([\#8329](https://github.com/matrix-org/synapse/issues/8329))
- Don't send push notifications to expired user accounts. ([\#8353](https://github.com/matrix-org/synapse/issues/8353))
- Fix a regression in v1.19.0 with reactivating users through the admin API. ([\#8362](https://github.com/matrix-org/synapse/issues/8362))
- Fix a bug where during device registration the length of the device name wasn't limited. ([\#8364](https://github.com/matrix-org/synapse/issues/8364))
- Include `guest_access` in the fields that are checked for null bytes when updating `room_stats_state`. Broke in v1.7.2. ([\#8373](https://github.com/matrix-org/synapse/issues/8373))
- Fix theoretical race condition where events are not sent down `/sync` if the synchrotron worker is restarted without restarting other workers. ([\#8374](https://github.com/matrix-org/synapse/issues/8374))
- Fix a bug which could cause errors in rooms with malformed membership events, on servers using sqlite. ([\#8385](https://github.com/matrix-org/synapse/issues/8385))
- Fix "Re-starting finished log context" warning when receiving an event we already had over federation. ([\#8398](https://github.com/matrix-org/synapse/issues/8398))
- Fix incorrect handling of timeouts on outgoing HTTP requests. ([\#8400](https://github.com/matrix-org/synapse/issues/8400))
- Fix a regression in v1.20.0 in the `synapse_port_db` script regarding the `ui_auth_sessions_ips` table. ([\#8410](https://github.com/matrix-org/synapse/issues/8410))
- Remove unnecessary 3PID registration check when resetting password via an email address. Bug introduced in v0.34.0rc2. ([\#8414](https://github.com/matrix-org/synapse/issues/8414))
-
-
-Improved Documentation
----------------------
-
- Add `/_synapse/client` to the reverse proxy documentation. ([\#8227](https://github.com/matrix-org/synapse/issues/8227))
- Add note to the reverse proxy settings documentation about disabling Apache's mod_security2. Contributed by Julian Fietkau (@jfietkau). ([\#8375](https://github.com/matrix-org/synapse/issues/8375))
- Improve description of `server_name` config option in `homserver.yaml`. ([\#8415](https://github.com/matrix-org/synapse/issues/8415))
-
-
-Deprecations and Removals
-------------------------
-
- Drop support for `prometheus_client` older than 0.4.0. ([\#8426](https://github.com/matrix-org/synapse/issues/8426))
-
-
-Internal Changes
----------------
-
- Fix tests on distros which disable TLSv1.0. Contributed by @danc86. ([\#8208](https://github.com/matrix-org/synapse/issues/8208))
- Simplify the distributor code to avoid unnecessary work. ([\#8216](https://github.com/matrix-org/synapse/issues/8216))
- Remove the `populate_stats_process_rooms_2` background job and restore functionality to `populate_stats_process_rooms`. ([\#8243](https://github.com/matrix-org/synapse/issues/8243))
- Clean up type hints for `PaginationConfig`. ([\#8250](https://github.com/matrix-org/synapse/issues/8250), [\#8282](https://github.com/matrix-org/synapse/issues/8282))
- Track the latest event for every destination and room for catch-up after federation outage. ([\#8256](https://github.com/matrix-org/synapse/issues/8256))
- Fix non-user visible bug in implementation of `MultiWriterIdGenerator.get_current_token_for_writer`. ([\#8257](https://github.com/matrix-org/synapse/issues/8257))
- Switch to the JSON implementation from the standard library. ([\#8259](https://github.com/matrix-org/synapse/issues/8259))
- Add type hints to `synapse.util.async_helpers`. ([\#8260](https://github.com/matrix-org/synapse/issues/8260))
- Simplify tests that mock asynchronous functions. ([\#8261](https://github.com/matrix-org/synapse/issues/8261))
- Add type hints to `StreamToken` and `RoomStreamToken` classes. ([\#8279](https://github.com/matrix-org/synapse/issues/8279))
- Change `StreamToken.room_key` to be a `RoomStreamToken` instance. ([\#8281](https://github.com/matrix-org/synapse/issues/8281))
- Refactor notifier code to correctly use the max event stream position. ([\#8288](https://github.com/matrix-org/synapse/issues/8288))
- Use slotted classes where possible. ([\#8296](https://github.com/matrix-org/synapse/issues/8296))
- Support testing the local Synapse checkout against the [Complement homeserver test suite](https://github.com/matrix-org/complement/). ([\#8317](https://github.com/matrix-org/synapse/issues/8317))
- Update outdated usages of `metaclass` to python 3 syntax. ([\#8326](https://github.com/matrix-org/synapse/issues/8326))
- Move lint-related dependencies to package-extra field, update CONTRIBUTING.md to utilise this. ([\#8330](https://github.com/matrix-org/synapse/issues/8330), [\#8377](https://github.com/matrix-org/synapse/issues/8377))
- Use the `admin_patterns` helper in additional locations. ([\#8331](https://github.com/matrix-org/synapse/issues/8331))
- Fix test logging to allow braces in log output. ([\#8335](https://github.com/matrix-org/synapse/issues/8335))
- Remove `__future__` imports related to Python 2 compatibility. ([\#8337](https://github.com/matrix-org/synapse/issues/8337))
- Simplify `super()` calls to Python 3 syntax. ([\#8344](https://github.com/matrix-org/synapse/issues/8344))
- Fix bad merge from `release-v1.20.0` branch to `develop`. ([\#8354](https://github.com/matrix-org/synapse/issues/8354))
- Factor out a `_send_dummy_event_for_room` method. ([\#8370](https://github.com/matrix-org/synapse/issues/8370))
- Improve logging of state resolution. ([\#8371](https://github.com/matrix-org/synapse/issues/8371))
- Add type annotations to `SimpleHttpClient`. ([\#8372](https://github.com/matrix-org/synapse/issues/8372))
- Refactor ID generators to use `async with` syntax. ([\#8383](https://github.com/matrix-org/synapse/issues/8383))
- Add `EventStreamPosition` type. ([\#8388](https://github.com/matrix-org/synapse/issues/8388))
- Create a mechanism for marking tests "logcontext clean". ([\#8399](https://github.com/matrix-org/synapse/issues/8399))
- A pair of tiny cleanups in the federation request code. ([\#8401](https://github.com/matrix-org/synapse/issues/8401))
- Add checks on startup that PostgreSQL sequences are consistent with their associated tables. ([\#8402](https://github.com/matrix-org/synapse/issues/8402))
- Do not include appservice users when calculating the total MAU for a server. ([\#8404](https://github.com/matrix-org/synapse/issues/8404))
- Typing fixes for `synapse.handlers.federation`. ([\#8422](https://github.com/matrix-org/synapse/issues/8422))
- Various refactors to simplify stream token handling. ([\#8423](https://github.com/matrix-org/synapse/issues/8423))
- Make stream token serializing/deserializing async. ([\#8427](https://github.com/matrix-org/synapse/issues/8427))
-
-
 Synapse 1.20.1 (2020-09-24)
 ===========================

@@ -408,11 +21,9 @@ API](https://github.com/matrix-org/synapse/tree/master/docs) has been
 accessible under the `/_matrix/client/api/v1/admin`,
 `/_matrix/client/unstable/admin`, `/_matrix/client/r0/admin` and
 `/_synapse/admin` prefixes. In a future release, we will be dropping support
-for accessing Synapse's Admin API using the `/_matrix/client/*` prefixes.
-
-From that point, the Admin API will only be accessible under `/_synapse/admin`.
-This makes it easier for homeserver admins to lock down external access to the
-Admin API endpoints.
+for accessing Synapse's Admin API using the `/_matrix/client/*` prefixes. This
+makes it easier for homeserver admins to lock down external access to the Admin
+API endpoints.

 Synapse 1.20.0rc5 (2020-09-18)
 ==============================
@@ -46,7 +46,7 @@ locally. You'll need python 3.6 or later, and to install a number of tools:

 ```
 # Install the dependencies
-pip install -e ".[lint,mypy]"
+pip install -e ".[lint]"

 # Run the linter script
 ./scripts-dev/lint.sh
@@ -63,10 +63,6 @@ run-time:
 ./scripts-dev/lint.sh path/to/file1.py path/to/file2.py path/to/folder
 ```

-You can also provide the `-d` option, which will lint the files that have been
-changed since the last git commit. This will often be significantly faster than
-linting the whole codebase.
-
 Before pushing new changes, ensure they don't produce linting errors. Commit any
 files that were corrected.

@@ -156,24 +152,6 @@ directory, you will need both a regular newsfragment *and* an entry in the
 debian changelog. (Though typically such changes should be submitted as two
 separate pull requests.)

-## Documentation
-
-There is a growing amount of documentation located in the [docs](docs)
-directory. This documentation is intended primarily for sysadmins running their
-own Synapse instance, as well as developers interacting externally with
-Synapse. [docs/dev](docs/dev) exists primarily to house documentation for
-Synapse developers. [docs/admin_api](docs/admin_api) houses documentation
-regarding Synapse's Admin API, which is used mostly by sysadmins and external
-service developers.
-
-New files added to both folders should be written in [Github-Flavoured
-Markdown](https://guides.github.com/features/mastering-markdown/), and attempts
-should be made to migrate existing documents to markdown where possible.
-
-Some documentation also exists in [Synapse's Github
-Wiki](https://github.com/matrix-org/synapse/wiki), although this is primarily
-contributed to by community authors.
-
 ## Sign off

 In order to have a concrete record that your contribution is intentional
@@ -57,7 +57,7 @@ light workloads.
 System requirements:

 - POSIX-compliant system (tested on Linux & OS X)
- Python 3.5.2 or later, up to Python 3.9.
+- Python 3.5.2 or later, up to Python 3.8.
 - At least 1GB of free RAM if you want to join large public rooms like #matrix:matrix.org

 Synapse is written in Python but some of the libraries it uses are written in
@@ -1,6 +1,10 @@
-=========================================================
-Synapse |support| |development| |license| |pypi| |python|
-=========================================================
+================
+Synapse |shield|
+================
+
+.. |shield| image:: https://img.shields.io/matrix/synapse:matrix.org?label=support&logo=matrix
+  :alt: (get support on #synapse:matrix.org)
+  :target: https://matrix.to/#/#synapse:matrix.org

 .. contents::

@@ -256,27 +260,23 @@ directory of your choice::
 Synapse has a number of external dependencies, that are easiest
 to install using pip and a virtualenv::

-    python3 -m venv ./env
-    source ./env/bin/activate
-    pip install -e ".[all,test]"
+    virtualenv -p python3 env
+    source env/bin/activate
+    python -m pip install --no-use-pep517 -e ".[all]"

 This will run a process of downloading and installing all the needed
-dependencies into a virtual env. If any dependencies fail to install,
-try installing the failing modules individually::
+dependencies into a virtual env.

-    pip install -e "module-name"
-
-Once this is done, you may wish to run Synapse's unit tests to
-check that everything is installed correctly::
+Once this is done, you may wish to run Synapse's unit tests, to
+check that everything is installed as it should be::

    python -m twisted.trial tests

-This should end with a 'PASSED' result (note that exact numbers will
-differ)::
+This should end with a 'PASSED' result::

-    Ran 1337 tests in 716.064s
+    Ran 143 tests in 0.601s

-    PASSED (skips=15, successes=1322)
+    PASSED (successes=143)

 Running the Integration Tests
 =============================
@@ -290,6 +290,19 @@ Testing with SyTest is recommended for verifying that changes related to the
 Client-Server API are functioning correctly. See the `installation instructions
 <https://github.com/matrix-org/sytest#installing>`_ for details.

+Building Internal API Documentation
+===================================
+
+Before building internal API documentation install sphinx and
+sphinxcontrib-napoleon::
+
+    pip install sphinx
+    pip install sphinxcontrib-napoleon
+
+Building internal API documentation::
+
+    python setup.py build_sphinx
+
 Troubleshooting
 ===============

@@ -374,23 +387,3 @@ something like the following in their logs::

 This is normally caused by a misconfiguration in your reverse-proxy. See
 `<docs/reverse_proxy.md>`_ and double-check that your settings are correct.
-
-.. |support| image:: https://img.shields.io/matrix/synapse:matrix.org?label=support&logo=matrix
-  :alt: (get support on #synapse:matrix.org)
-  :target: https://matrix.to/#/#synapse:matrix.org
-
-.. |development| image:: https://img.shields.io/matrix/synapse-dev:matrix.org?label=development&logo=matrix
-  :alt: (discuss development on #synapse-dev:matrix.org)
-  :target: https://matrix.to/#/#synapse-dev:matrix.org
-
-.. |license| image:: https://img.shields.io/github/license/matrix-org/synapse
-  :alt: (check license in LICENSE file)
-  :target: LICENSE
-
-.. |pypi| image:: https://img.shields.io/pypi/v/matrix-synapse
-  :alt: (latest version released on PyPi)
-  :target: https://pypi.org/project/matrix-synapse
-
-.. |python| image:: https://img.shields.io/pypi/pyversions/matrix-synapse
-  :alt: (supported python versions)
-  :target: https://pypi.org/project/matrix-synapse
@@ -75,39 +75,6 @@ for example:
     wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb
     dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb

-Upgrading to v1.23.0
-====================
-
-Structured logging configuration breaking changes
-------------------------------------------------
-
-This release deprecates use of the ``structured: true`` logging configuration for
-structured logging. If your logging configuration contains ``structured: true``
-then it should be modified based on the `structured logging documentation
-<https://github.com/matrix-org/synapse/blob/master/docs/structured_logging.md>`_.
-
-The ``structured`` and ``drains`` logging options are now deprecated and should
-be replaced by standard logging configuration of ``handlers`` and ``formatters`.
-
-A future will release of Synapse will make using ``structured: true`` an error.
-
-Upgrading to v1.22.0
-====================
-
-ThirdPartyEventRules breaking changes
-------------------------------------
-
-This release introduces a backwards-incompatible change to modules making use of
-``ThirdPartyEventRules`` in Synapse. If you make use of a module defined under the
-``third_party_event_rules`` config option, please make sure it is updated to handle
-the below change:
-
-The ``http_client`` argument is no longer passed to modules as they are initialised. Instead,
-modules are expected to make use of the ``http_client`` property on the ``ModuleApi`` class.
-Modules are now passed a ``module_api`` argument during initialisation, which is an instance of
-``ModuleApi``. ``ModuleApi`` instances have a ``http_client`` property which acts the same as
-the ``http_client`` argument previously passed to ``ThirdPartyEventRules`` modules.
-
 Upgrading to v1.21.0
 ====================

@@ -0,0 +1 @@
+Fix a bug in the media repository where remote thumbnails with the same size but different crop methods would overwrite each other. Contributed by @deepbluev7.
@@ -0,0 +1 @@
+Fix inconsistent handling of non-existent push rules, and stop tracking the `enabled` state of removed push rules.
@@ -0,0 +1 @@
+Require the user to confirm that their password should be reset after clicking the email confirmation link.
@@ -0,0 +1 @@
+Fix tests on distros which disable TLSv1.0. Contributed by @danc86.
@@ -0,0 +1 @@
+Simplify the distributor code to avoid unnecessary work.
@@ -0,0 +1 @@
+Add an admin API `GET /_synapse/admin/v1/event_reports` to read entries of table `event_reports`. Contributed by @dklimpel.
@@ -0,0 +1 @@
+Add `/_synapse/client` to the reverse proxy documentation.
@@ -0,0 +1 @@
+Fix messages over federation being lost until an event is sent into the same room.
@@ -0,0 +1 @@
+Fix a longstanding bug where files that could not be thumbnailed would result in an Internal Server Error.
@@ -0,0 +1 @@
+Remove the 'populate_stats_process_rooms_2' background job and restore functionality to 'populate_stats_process_rooms'.
@@ -0,0 +1 @@
+Fix messages over federation being lost until an event is sent into the same room.
@@ -0,0 +1 @@
+Consolidate the SSO error template across all configuration.
@@ -0,0 +1 @@
+Clean up type hints for `PaginationConfig`.
@@ -0,0 +1 @@
+Track the latest event for every destination and room for catch-up after federation outage.
@@ -0,0 +1 @@
+Fix non-user visible bug in implementation of `MultiWriterIdGenerator.get_current_token_for_writer`.
@@ -0,0 +1 @@
+Fix messages over federation being lost until an event is sent into the same room.
@@ -0,0 +1 @@
+Switch to the JSON implementation from the standard library.
@@ -0,0 +1 @@
+Add type hints to `synapse.util.async_helpers`.
@@ -0,0 +1 @@
+Simplify tests that mock asynchronous functions.
@@ -0,0 +1 @@
+Upgrade canonicaljson to version 1.4.0 to fix an unicode encoding issue.
@@ -0,0 +1 @@
+Fix logstanding bug which could lead to incomplete database upgrades on SQLite.
@@ -0,0 +1 @@
+Fix stack overflow when stderr is redirected to the logging system, and the logging system encounters an error.
@@ -0,0 +1 @@
+Fix messages over federation being lost until an event is sent into the same room.
@@ -0,0 +1 @@
+Add a config option to specify a whitelist of domains that a user can be redirected to after validating their email or phone number.
@@ -0,0 +1 @@
+Fix a bug which cause the logging system to report errors, if `DEBUG` was enabled and no `context` filter was applied.
@@ -0,0 +1 @@
+Add type hints to `StreamToken` and `RoomStreamToken` classes.
@@ -0,0 +1 @@
+Change `StreamToken.room_key` to be a `RoomStreamToken` instance.
@@ -0,0 +1 @@
+Clean up type hints for `PaginationConfig`.
@@ -0,0 +1 @@
+Fix edge case where push could get delayed for a user until a later event was pushed.
@@ -0,0 +1 @@
+Refactor notifier code to correctly use the max event stream position.
@@ -0,0 +1 @@
+Add experimental support for sharding event persister.
@@ -0,0 +1 @@
+Use slotted classes where possible.
@@ -0,0 +1 @@
+Add the room topic and avatar to the room details admin API.
@@ -0,0 +1 @@
+Add an admin API for querying rooms where a user is a member. Contributed by @dklimpel.
@@ -0,0 +1 @@
+Add `uk.half-shot.msc2778.login.application_service` login type to allow appservices to login.
@@ -0,0 +1 @@
+Fix messages over federation being lost until an event is sent into the same room.
@@ -0,0 +1 @@
+Fix fetching events from remote servers that are malformed.
@@ -0,0 +1 @@
+Update outdated usages of `metaclass` to python 3 syntax.
@@ -0,0 +1 @@
+Fix UnboundLocalError from occuring when appservices send malformed register request.
@@ -0,0 +1 @@
+Move lint-related dependencies to package-extra field, update CONTRIBUTING.md to utilise this.
@@ -0,0 +1 @@
+Use the `admin_patterns` helper in additional locations.
@@ -0,0 +1 @@
+Fix test logging to allow braces in log output.
@@ -0,0 +1 @@
+Remove `__future__` imports related to Python 2 compatibility.
@@ -0,0 +1 @@
+Simplify `super()` calls to Python 3 syntax.
@@ -0,0 +1 @@
+Add a configuration option that allows existing users to log in with OpenID Connect. Contributed by @BBBSnowball and @OmmyZhang.
@@ -0,0 +1 @@
+Don't send push notifications to expired user accounts.
@@ -0,0 +1 @@
+Fix bad merge from `release-v1.20.0` branch to `develop`.
@@ -0,0 +1 @@
+Fixed a regression in v1.19.0 with reactivating users through the admin API.
@@ -0,0 +1,2 @@
+Fix a bug where during device registration the length of the device name wasn't
+limited.
@@ -0,0 +1 @@
+Factor out a `_send_dummy_event_for_room` method.
@@ -0,0 +1 @@
+Improve logging of state resolution.
@@ -0,0 +1 @@
+Add type annotations to `SimpleHttpClient`.
@@ -0,0 +1 @@
+Include `guest_access` in the fields that are checked for null bytes when updating `room_stats_state`. Broke in v1.7.2.
@@ -0,0 +1 @@
+Fix theoretical race condition where events are not sent down `/sync` if the synchrotron worker is restarted without restarting other workers.
@@ -0,0 +1 @@
+Add note to the reverse proxy settings documentation about disabling Apache's mod_security2. Contributed by Julian Fietkau (@jfietkau).
@@ -0,0 +1 @@
+Move lint-related dependencies to package-extra field, update CONTRIBUTING.md to utilise this.
@@ -0,0 +1 @@
+Refactor ID generators to use `async with` syntax.
@@ -0,0 +1 @@
+Fix a bug which could cause errors in rooms with malformed membership events, on servers using sqlite.
@@ -0,0 +1 @@
+Fix a bug introduced in v1.20.0 which caused the `synapse_port_db` script to fail.
@@ -0,0 +1 @@
+Add experimental support for sharding event persister.
@@ -0,0 +1 @@
+Add `EventStreamPosition` type.
@@ -0,0 +1 @@
+Add experimental support for sharding event persister.
@@ -0,0 +1 @@
+Fix "Re-starting finished log context" warning when receiving an event we already had over federation.
@@ -0,0 +1 @@
+A pair of tiny cleanups in the federation request code.
@@ -0,0 +1 @@
+Consolidate the SSO error template across all configuration.
@@ -0,0 +1 @@
+Fix retries of HTTP requests between Synapse workers when the requests time out.
@@ -3,4 +3,4 @@
 0. Set up Prometheus and Grafana. Out of scope for this readme. Useful documentation about using Grafana with Prometheus: http://docs.grafana.org/features/datasources/prometheus/
 1. Have your Prometheus scrape your Synapse. https://github.com/matrix-org/synapse/blob/master/docs/metrics-howto.md
 2. Import dashboard into Grafana. Download `synapse.json`. Import it to Grafana and select the correct Prometheus datasource. http://docs.grafana.org/reference/export_import/
-3. Set up required recording rules. https://github.com/matrix-org/synapse/tree/master/contrib/prometheus
+3. Set up additional recording rules
@@ -42,7 +42,7 @@ dh_virtualenv \
    --preinstall="mock" \
    --extra-pip-arg="--no-cache-dir" \
    --extra-pip-arg="--compile" \
-    --extras="all,systemd,test"
+    --extras="all,systemd"

 PACKAGE_BUILD_DIR="debian/matrix-synapse-py3"
 VIRTUALENV_DIR="${PACKAGE_BUILD_DIR}${DH_VIRTUALENV_INSTALL_ROOT}/matrix-synapse"
@@ -1,38 +1,3 @@
-matrix-synapse-py3 (1.22.1) stable; urgency=medium
-
-  * New synapse release 1.22.1.
-
- -- Synapse Packaging team <packages@matrix.org>  Fri, 30 Oct 2020 15:25:37 +0000
-
-matrix-synapse-py3 (1.22.0) stable; urgency=medium
-
-  * New synapse release 1.22.0.
-
- -- Synapse Packaging team <packages@matrix.org>  Tue, 27 Oct 2020 12:07:12 +0000
-
-matrix-synapse-py3 (1.21.2) stable; urgency=medium
-
-  [ Synapse Packaging team ]
-  * New synapse release 1.21.2.
-
- -- Synapse Packaging team <packages@matrix.org>  Thu, 15 Oct 2020 09:23:27 -0400
-
-matrix-synapse-py3 (1.21.1) stable; urgency=medium
-
-  [ Synapse Packaging team ]
-  * New synapse release 1.21.1.
-
-  [ Andrew Morgan ]
-  * Explicitly install "test" python dependencies.
-
- -- Synapse Packaging team <packages@matrix.org>  Tue, 13 Oct 2020 10:24:13 +0100
-
-matrix-synapse-py3 (1.21.0) stable; urgency=medium
-
-  * New synapse release 1.21.0.
-
- -- Synapse Packaging team <packages@matrix.org>  Mon, 12 Oct 2020 15:47:44 +0100
-
 matrix-synapse-py3 (1.20.1) stable; urgency=medium

  * New synapse release 1.20.1.
@@ -30,8 +30,6 @@ for port in 8080 8081 8082; do
    if ! grep -F "Customisation made by demo/start.sh" -q  $DIR/etc/$port.config; then
        printf '\n\n# Customisation made by demo/start.sh\n' >> $DIR/etc/$port.config

-        echo "public_baseurl: http://localhost:$port/" >> $DIR/etc/$port.config
-
        echo 'enable_registration: true' >> $DIR/etc/$port.config

        # Warning, this heredoc depends on the interaction of tabs and spaces. Please don't
@@ -11,7 +11,7 @@
 #    docker build -f docker/Dockerfile --build-arg PYTHON_VERSION=3.6 .
 #

-ARG PYTHON_VERSION=3.8
+ARG PYTHON_VERSION=3.7

 ###
 ### Stage 0: builder
@@ -83,7 +83,7 @@ docker logs synapse
 If all is well, you should now be able to connect to http://localhost:8008 and
 see a confirmation message.

-The following environment variables are supported in `run` mode:
+The following environment variables are supported in run mode:

 * `SYNAPSE_CONFIG_DIR`: where additional config files are stored. Defaults to
  `/data`.
@@ -94,20 +94,6 @@ The following environment variables are supported in `run` mode:
 * `UID`, `GID`: the user and group id to run Synapse as. Defaults to `991`, `991`.
 * `TZ`: the [timezone](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) the container will run with. Defaults to `UTC`.

-For more complex setups (e.g. for workers) you can also pass your args directly to synapse using `run` mode. For example like this:
-
-```
-docker run -d --name synapse \
-    --mount type=volume,src=synapse-data,dst=/data \
-    -p 8008:8008 \
-    matrixdotorg/synapse:latest run \
-    -m synapse.app.generic_worker \
-    --config-path=/data/homeserver.yaml \
-    --config-path=/data/generic_worker.yaml
-```
-
-If you do not provide `-m`, the value of the `SYNAPSE_WORKER` environment variable is used. If you do not provide at least one `--config-path` or `-c`, the value of the `SYNAPSE_CONFIG_PATH` environment variable is used instead.
-
 ## Generating an (admin) user

 After synapse is running, you may wish to create a user via `register_new_matrix_user`.
@@ -90,7 +90,7 @@ federation_rc_concurrent: 3

 media_store_path: "/data/media"
 uploads_path: "/data/uploads"
-max_upload_size: "{{ SYNAPSE_MAX_UPLOAD_SIZE or "50M" }}"
+max_upload_size: "{{ SYNAPSE_MAX_UPLOAD_SIZE or "10M" }}"
 max_image_pixels: "32M"
 dynamic_thumbnails: false

@@ -179,7 +179,7 @@ def run_generate_config(environ, ownership):


 def main(args, environ):
-    mode = args[1] if len(args) > 1 else "run"
+    mode = args[1] if len(args) > 1 else None
    desired_uid = int(environ.get("UID", "991"))
    desired_gid = int(environ.get("GID", "991"))
    synapse_worker = environ.get("SYNAPSE_WORKER", "synapse.app.homeserver")
@@ -205,47 +205,36 @@ def main(args, environ):
            config_dir, config_path, environ, ownership
        )

-    if mode != "run":
+    if mode is not None:
        error("Unknown execution mode '%s'" % (mode,))

-    args = args[2:]
+    config_dir = environ.get("SYNAPSE_CONFIG_DIR", "/data")
+    config_path = environ.get("SYNAPSE_CONFIG_PATH", config_dir + "/homeserver.yaml")

-    if "-m" not in args:
-        args = ["-m", synapse_worker] + args
-
-    # if there are no config files passed to synapse, try adding the default file
-    if not any(p.startswith("--config-path") or p.startswith("-c") for p in args):
-        config_dir = environ.get("SYNAPSE_CONFIG_DIR", "/data")
-        config_path = environ.get(
-            "SYNAPSE_CONFIG_PATH", config_dir + "/homeserver.yaml"
-        )
-
-        if not os.path.exists(config_path):
-            if "SYNAPSE_SERVER_NAME" in environ:
-                error(
-                    """\
+    if not os.path.exists(config_path):
+        if "SYNAPSE_SERVER_NAME" in environ:
+            error(
+                """\
 Config file '%s' does not exist.

 The synapse docker image no longer supports generating a config file on-the-fly
 based on environment variables. You can migrate to a static config file by
 running with 'migrate_config'. See the README for more details.
 """
-                    % (config_path,)
-                )
-
-            error(
-                "Config file '%s' does not exist. You should either create a new "
-                "config file by running with the `generate` argument (and then edit "
-                "the resulting file before restarting) or specify the path to an "
-                "existing config file with the SYNAPSE_CONFIG_PATH variable."
                % (config_path,)
            )

-        args += ["--config-path", config_path]
+        error(
+            "Config file '%s' does not exist. You should either create a new "
+            "config file by running with the `generate` argument (and then edit "
+            "the resulting file before restarting) or specify the path to an "
+            "existing config file with the SYNAPSE_CONFIG_PATH variable."
+            % (config_path,)
+        )

-    log("Starting synapse with args " + " ".join(args))
+    log("Starting synapse with config file " + config_path)

-    args = ["python"] + args
+    args = ["python", "-m", synapse_worker, "--config-path", config_path]
    if ownership is not None:
        args = ["gosu", ownership] + args
        os.execv("/usr/sbin/gosu", args)
@@ -1,172 +0,0 @@
-# Show reported events
-
-This API returns information about reported events.
-
-The api is:
-```
-GET /_synapse/admin/v1/event_reports?from=0&limit=10
-```
-To use it, you will need to authenticate by providing an `access_token` for a
-server admin: see [README.rst](README.rst).
-
-It returns a JSON body like the following:
-
-```json
-{
-    "event_reports": [
-        {
-            "event_id": "$bNUFCwGzWca1meCGkjp-zwslF-GfVcXukvRLI1_FaVY",
-            "id": 2,
-            "reason": "foo",
-            "score": -100,
-            "received_ts": 1570897107409,
-            "canonical_alias": "#alias1:matrix.org",
-            "room_id": "!ERAgBpSOcCCuTJqQPk:matrix.org",
-            "name": "Matrix HQ",
-            "sender": "@foobar:matrix.org",
-            "user_id": "@foo:matrix.org"
-        },
-        {
-            "event_id": "$3IcdZsDaN_En-S1DF4EMCy3v4gNRKeOJs8W5qTOKj4I",
-            "id": 3,
-            "reason": "bar",
-            "score": -100,
-            "received_ts": 1598889612059,
-            "canonical_alias": "#alias2:matrix.org",
-            "room_id": "!eGvUQuTCkHGVwNMOjv:matrix.org",
-            "name": "Your room name here",
-            "sender": "@foobar:matrix.org",
-            "user_id": "@bar:matrix.org"
-        }
-    ],
-    "next_token": 2,
-    "total": 4
-}
-```
-
-To paginate, check for `next_token` and if present, call the endpoint again with `from`
-set to the value of `next_token`. This will return a new page.
-
-If the endpoint does not return a `next_token` then there are no more reports to
-paginate through.
-
-**URL parameters:**
-
-* `limit`: integer - Is optional but is used for pagination, denoting the maximum number
-  of items to return in this call. Defaults to `100`.
-* `from`: integer - Is optional but used for pagination, denoting the offset in the
-  returned results. This should be treated as an opaque value and not explicitly set to
-  anything other than the return value of `next_token` from a previous call. Defaults to `0`.
-* `dir`: string - Direction of event report order. Whether to fetch the most recent
-  first (`b`) or the oldest first (`f`). Defaults to `b`.
-* `user_id`: string - Is optional and filters to only return users with user IDs that
-  contain this value. This is the user who reported the event and wrote the reason.
-* `room_id`: string - Is optional and filters to only return rooms with room IDs that
-  contain this value.
-
-**Response**
-
-The following fields are returned in the JSON response body:
-
-* `id`: integer - ID of event report.
-* `received_ts`: integer - The timestamp (in milliseconds since the unix epoch) when this
-  report was sent.
-* `room_id`: string - The ID of the room in which the event being reported is located.
-* `name`: string - The name of the room.
-* `event_id`: string - The ID of the reported event.
-* `user_id`: string - This is the user who reported the event and wrote the reason.
-* `reason`: string - Comment made by the `user_id` in this report. May be blank.
-* `score`: integer - Content is reported based upon a negative score, where -100 is
-  "most offensive" and 0 is "inoffensive".
-* `sender`: string - This is the ID of the user who sent the original message/event that
-  was reported.
-* `canonical_alias`: string - The canonical alias of the room. `null` if the room does not
-  have a canonical alias set.
-* `next_token`: integer - Indication for pagination. See above.
-* `total`: integer - Total number of event reports related to the query
-  (`user_id` and `room_id`).
-
-# Show details of a specific event report
-
-This API returns information about a specific event report.
-
-The api is:
-```
-GET /_synapse/admin/v1/event_reports/<report_id>
-```
-To use it, you will need to authenticate by providing an `access_token` for a
-server admin: see [README.rst](README.rst).
-
-It returns a JSON body like the following:
-
-```jsonc
-{
-    "event_id": "$bNUFCwGzWca1meCGkjp-zwslF-GfVcXukvRLI1_FaVY",
-    "event_json": {
-        "auth_events": [
-            "$YK4arsKKcc0LRoe700pS8DSjOvUT4NDv0HfInlMFw2M",
-            "$oggsNXxzPFRE3y53SUNd7nsj69-QzKv03a1RucHu-ws"
-        ],
-        "content": {
-            "body": "matrix.org: This Week in Matrix",
-            "format": "org.matrix.custom.html",
-            "formatted_body": "<strong>matrix.org</strong>:<br><a href=\"https://matrix.org/blog/\"><strong>This Week in Matrix</strong></a>",
-            "msgtype": "m.notice"
-        },
-        "depth": 546,
-        "hashes": {
-            "sha256": "xK1//xnmvHJIOvbgXlkI8eEqdvoMmihVDJ9J4SNlsAw"
-        },
-        "origin": "matrix.org",
-        "origin_server_ts": 1592291711430,
-        "prev_events": [
-            "$YK4arsKKcc0LRoe700pS8DSjOvUT4NDv0HfInlMFw2M"
-        ],
-        "prev_state": [],
-        "room_id": "!ERAgBpSOcCCuTJqQPk:matrix.org",
-        "sender": "@foobar:matrix.org",
-        "signatures": {
-            "matrix.org": {
-                "ed25519:a_JaEG": "cs+OUKW/iHx5pEidbWxh0UiNNHwe46Ai9LwNz+Ah16aWDNszVIe2gaAcVZfvNsBhakQTew51tlKmL2kspXk/Dg"
-            }
-        },
-        "type": "m.room.message",
-        "unsigned": {
-            "age_ts": 1592291711430,
-        }
-    },
-    "id": <report_id>,
-    "reason": "foo",
-    "score": -100,
-    "received_ts": 1570897107409,
-    "canonical_alias": "#alias1:matrix.org",
-    "room_id": "!ERAgBpSOcCCuTJqQPk:matrix.org",
-    "name": "Matrix HQ",
-    "sender": "@foobar:matrix.org",
-    "user_id": "@foo:matrix.org"
-}
-```
-
-**URL parameters:**
-
-* `report_id`: string - The ID of the event report.
-
-**Response**
-
-The following fields are returned in the JSON response body:
-
-* `id`: integer - ID of event report.
-* `received_ts`: integer - The timestamp (in milliseconds since the unix epoch) when this
-  report was sent.
-* `room_id`: string - The ID of the room in which the event being reported is located.
-* `name`: string - The name of the room.
-* `event_id`: string - The ID of the reported event.
-* `user_id`: string - This is the user who reported the event and wrote the reason.
-* `reason`: string - Comment made by the `user_id` in this report. May be blank.
-* `score`: integer - Content is reported based upon a negative score, where -100 is
-  "most offensive" and 0 is "inoffensive".
-* `sender`: string - This is the ID of the user who sent the original message/event that
-  was reported.
-* `canonical_alias`: string - The canonical alias of the room. `null` if the room does not
-  have a canonical alias set.
-* `event_json`: object - Details of the original event that was reported.
@@ -0,0 +1,129 @@
+Show reported events
+====================
+
+This API returns information about reported events.
+
+The api is::
+
+    GET /_synapse/admin/v1/event_reports?from=0&limit=10
+
+To use it, you will need to authenticate by providing an ``access_token`` for a
+server admin: see `README.rst <README.rst>`_.
+
+It returns a JSON body like the following:
+
+.. code:: jsonc
+
+    {
+        "event_reports": [
+            {
+                "content": {
+                    "reason": "foo",
+                    "score": -100
+                },
+                "event_id": "$bNUFCwGzWca1meCGkjp-zwslF-GfVcXukvRLI1_FaVY",
+                "event_json": {
+                    "auth_events": [
+                        "$YK4arsKKcc0LRoe700pS8DSjOvUT4NDv0HfInlMFw2M",
+                        "$oggsNXxzPFRE3y53SUNd7nsj69-QzKv03a1RucHu-ws"
+                    ],
+                    "content": {
+                        "body": "matrix.org: This Week in Matrix",
+                        "format": "org.matrix.custom.html",
+                        "formatted_body": "<strong>matrix.org</strong>:<br><a href=\"https://matrix.org/blog/\"><strong>This Week in Matrix</strong></a>",
+                        "msgtype": "m.notice"
+                    },
+                    "depth": 546,
+                    "hashes": {
+                        "sha256": "xK1//xnmvHJIOvbgXlkI8eEqdvoMmihVDJ9J4SNlsAw"
+                    },
+                    "origin": "matrix.org",
+                    "origin_server_ts": 1592291711430,
+                    "prev_events": [
+                        "$YK4arsKKcc0LRoe700pS8DSjOvUT4NDv0HfInlMFw2M"
+                    ],
+                    "prev_state": [],
+                    "room_id": "!ERAgBpSOcCCuTJqQPk:matrix.org",
+                    "sender": "@foobar:matrix.org",
+                    "signatures": {
+                        "matrix.org": {
+                            "ed25519:a_JaEG": "cs+OUKW/iHx5pEidbWxh0UiNNHwe46Ai9LwNz+Ah16aWDNszVIe2gaAcVZfvNsBhakQTew51tlKmL2kspXk/Dg"
+                        }
+                    },
+                    "type": "m.room.message",
+                    "unsigned": {
+                        "age_ts": 1592291711430,
+                    }
+                },
+                "id": 2,
+                "reason": "foo",
+                "received_ts": 1570897107409,
+                "room_alias": "#alias1:matrix.org",
+                "room_id": "!ERAgBpSOcCCuTJqQPk:matrix.org",
+                "sender": "@foobar:matrix.org",
+                "user_id": "@foo:matrix.org"
+            },
+            {
+                "content": {
+                    "reason": "bar",
+                    "score": -100
+                },
+                "event_id": "$3IcdZsDaN_En-S1DF4EMCy3v4gNRKeOJs8W5qTOKj4I",
+                "event_json": {
+                    // hidden items
+                    // see above
+                },
+                "id": 3,
+                "reason": "bar",
+                "received_ts": 1598889612059,
+                "room_alias": "#alias2:matrix.org",
+                "room_id": "!eGvUQuTCkHGVwNMOjv:matrix.org",
+                "sender": "@foobar:matrix.org",
+                "user_id": "@bar:matrix.org"
+            }
+        ],
+        "next_token": 2,
+        "total": 4
+    }
+
+To paginate, check for ``next_token`` and if present, call the endpoint again
+with ``from`` set to the value of ``next_token``. This will return a new page.
+
+If the endpoint does not return a ``next_token`` then there are no more
+reports to paginate through.
+
+**URL parameters:**
+
+- ``limit``: integer - Is optional but is used for pagination,
+  denoting the maximum number of items to return in this call. Defaults to ``100``.
+- ``from``: integer - Is optional but used for pagination,
+  denoting the offset in the returned results. This should be treated as an opaque value and
+  not explicitly set to anything other than the return value of ``next_token`` from a previous call.
+  Defaults to ``0``.
+- ``dir``: string - Direction of event report order. Whether to fetch the most recent first (``b``) or the
+  oldest first (``f``). Defaults to ``b``.
+- ``user_id``: string - Is optional and filters to only return users with user IDs that contain this value.
+  This is the user who reported the event and wrote the reason.
+- ``room_id``: string - Is optional and filters to only return rooms with room IDs that contain this value.
+
+**Response**
+
+The following fields are returned in the JSON response body:
+
+- ``id``: integer - ID of event report.
+- ``received_ts``: integer - The timestamp (in milliseconds since the unix epoch) when this report was sent.
+- ``room_id``: string - The ID of the room in which the event being reported is located.
+- ``event_id``: string - The ID of the reported event.
+- ``user_id``: string - This is the user who reported the event and wrote the reason.
+- ``reason``: string - Comment made by the ``user_id`` in this report. May be blank.
+- ``content``: object - Content of reported event.
+
+  - ``reason``: string - Comment made by the ``user_id`` in this report. May be blank.
+  - ``score``: integer - Content is reported based upon a negative score, where -100 is "most offensive" and 0 is "inoffensive".
+
+- ``sender``: string - This is the ID of the user who sent the original message/event that was reported.
+- ``room_alias``: string - The alias of the room. ``null`` if the room does not have a canonical alias set.
+- ``event_json``: object - Details of the original event that was reported.
+- ``next_token``: integer - Indication for pagination. See above.
+- ``total``: integer - Total number of event reports related to the query (``user_id`` and ``room_id``).
+
@@ -100,82 +100,3 @@ Response:
  "num_quarantined": 10  # The number of media items successfully quarantined
 }
 ```
-
-# Delete local media
-This API deletes the *local* media from the disk of your own server.
-This includes any local thumbnails and copies of media downloaded from
-remote homeservers.
-This API will not affect media that has been uploaded to external
-media repositories (e.g https://github.com/turt2live/matrix-media-repo/).
-See also [purge_remote_media.rst](purge_remote_media.rst).
-
-## Delete a specific local media
-Delete a specific `media_id`.
-
-Request:
-
-```
-DELETE /_synapse/admin/v1/media/<server_name>/<media_id>
-
-{}
-```
-
-URL Parameters
-
-* `server_name`: string - The name of your local server (e.g `matrix.org`)
-* `media_id`: string - The ID of the media (e.g `abcdefghijklmnopqrstuvwx`)
-
-Response:
-
-```json
-    {
-       "deleted_media": [
-          "abcdefghijklmnopqrstuvwx"
-       ],
-       "total": 1
-    }
-```
-
-The following fields are returned in the JSON response body:
-
-* `deleted_media`: an array of strings - List of deleted `media_id`
-* `total`: integer - Total number of deleted `media_id`
-
-## Delete local media by date or size
-
-Request:
-
-```
-POST /_synapse/admin/v1/media/<server_name>/delete?before_ts=<before_ts>
-
-{}
-```
-
-URL Parameters
-
-* `server_name`: string - The name of your local server (e.g `matrix.org`).
-* `before_ts`: string representing a positive integer - Unix timestamp in ms.
-Files that were last used before this timestamp will be deleted. It is the timestamp of
-last access and not the timestamp creation. 
-* `size_gt`: Optional - string representing a positive integer - Size of the media in bytes.
-Files that are larger will be deleted. Defaults to `0`.
-* `keep_profiles`: Optional - string representing a boolean - Switch to also delete files
-that are still used in image data (e.g user profile, room avatar).
-If `false` these files will be deleted. Defaults to `true`.
-
-Response:
-
-```json
-    {
-       "deleted_media": [
-          "abcdefghijklmnopqrstuvwx",
-          "abcdefghijklmnopqrstuvwz"
-       ],
-       "total": 2
-    }
-```
-
-The following fields are returned in the JSON response body:
-
-* `deleted_media`: an array of strings - List of deleted `media_id`
-* `total`: integer - Total number of deleted `media_id`
@@ -18,8 +18,7 @@ To fetch the nonce, you need to request one from the API::

 Once you have the nonce, you can make a ``POST`` to the same URL with a JSON
 body containing the nonce, username, password, whether they are an admin
-(optional, False by default), and a HMAC digest of the content. Also you can
-set the displayname (optional, ``username`` by default).
+(optional, False by default), and a HMAC digest of the content.

 As an example::

@@ -27,7 +26,6 @@ As an example::
  > {
     "nonce": "thisisanonce",
     "username": "pepper_roni",
-     "displayname": "Pepper Roni",
     "password": "pizza",
     "admin": true,
     "mac": "mac_digest_here"
@@ -265,10 +265,12 @@ Response:
 Once the `next_token` parameter is no longer present, we know we've reached the
 end of the list.

-# Room Details API
+# DRAFT: Room Details API

 The Room Details admin API allows server admins to get all details of a room.

+This API is still a draft and details might change!
+
 The following fields are possible in the JSON response body:

 * `room_id` - The ID of the room.
@@ -1,83 +0,0 @@
-# Users' media usage statistics
-
-Returns information about all local media usage of users. Gives the
-possibility to filter them by time and user.
-
-The API is:
-
-```
-GET /_synapse/admin/v1/statistics/users/media
-```
-
-To use it, you will need to authenticate by providing an `access_token`
-for a server admin: see [README.rst](README.rst).
-
-A response body like the following is returned:
-
-```json
-{
-  "users": [
-    {
-      "displayname": "foo_user_0",
-      "media_count": 2,
-      "media_length": 134,
-      "user_id": "@foo_user_0:test"
-    },
-    {
-      "displayname": "foo_user_1",
-      "media_count": 2,
-      "media_length": 134,
-      "user_id": "@foo_user_1:test"
-    }
-  ],
-  "next_token": 3,
-  "total": 10
-}
-```
-
-To paginate, check for `next_token` and if present, call the endpoint
-again with `from` set to the value of `next_token`. This will return a new page.
-
-If the endpoint does not return a `next_token` then there are no more
-reports to paginate through.
-
-**Parameters**
-
-The following parameters should be set in the URL:
-
-* `limit`: string representing a positive integer - Is optional but is
-  used for pagination, denoting the maximum number of items to return
-  in this call. Defaults to `100`.
-* `from`: string representing a positive integer - Is optional but used for pagination,
-  denoting the offset in the returned results. This should be treated as an opaque value
-  and not explicitly set to anything other than the return value of `next_token` from a
-  previous call. Defaults to `0`.
-* `order_by` - string - The method in which to sort the returned list of users. Valid values are:
-  - `user_id` - Users are ordered alphabetically by `user_id`. This is the default.
-  - `displayname` - Users are ordered alphabetically by `displayname`.
-  - `media_length` - Users are ordered by the total size of uploaded media in bytes.
-    Smallest to largest.
-  - `media_count` - Users are ordered by number of uploaded media. Smallest to largest.
-* `from_ts` - string representing a positive integer - Considers only
-  files created at this timestamp or later. Unix timestamp in ms.
-* `until_ts` - string representing a positive integer - Considers only
-  files created at this timestamp or earlier. Unix timestamp in ms.
-* `search_term` - string - Filter users by their user ID localpart **or** displayname.
-  The search term can be found in any part of the string.
-  Defaults to no filtering.
-* `dir` - string - Direction of order. Either `f` for forwards or `b` for backwards.
-  Setting this value to `b` will reverse the above sort order. Defaults to `f`.
-
-
-**Response**
-
-The following fields are returned in the JSON response body:
-
-* `users` - An array of objects, each containing information
-  about the user and their local media. Objects contain the following fields:
-  - `displayname` - string - Displayname of this user.
-  - `media_count` - integer - Number of uploaded media by this user.
-  - `media_length` - integer - Size of uploaded media in bytes by this user.
-  - `user_id` - string - Fully-qualified user ID (ex. `@user:server.com`).
-* `next_token` - integer - Opaque value used for pagination. See above.
-* `total` - integer - Total number of users after filtering.
@@ -341,89 +341,6 @@ The following fields are returned in the JSON response body:
 - ``total`` - Number of rooms.


-List media of an user
-================================
-Gets a list of all local media that a specific ``user_id`` has created.
-The response is ordered by creation date descending and media ID descending.
-The newest media is on top.
-
-The API is::
-
-  GET /_synapse/admin/v1/users/<user_id>/media
-
-To use it, you will need to authenticate by providing an ``access_token`` for a
-server admin: see `README.rst <README.rst>`_.
-
-A response body like the following is returned:
-
-.. code:: json
-
-    {
-      "media": [
-        {
-          "created_ts": 100400,
-          "last_access_ts": null,
-          "media_id": "qXhyRzulkwLsNHTbpHreuEgo",
-          "media_length": 67,
-          "media_type": "image/png",
-          "quarantined_by": null,
-          "safe_from_quarantine": false,
-          "upload_name": "test1.png"
-        },
-        {
-          "created_ts": 200400,
-          "last_access_ts": null,
-          "media_id": "FHfiSnzoINDatrXHQIXBtahw",
-          "media_length": 67,
-          "media_type": "image/png",
-          "quarantined_by": null,
-          "safe_from_quarantine": false,
-          "upload_name": "test2.png"
-        }
-      ],
-      "next_token": 3,
-      "total": 2
-    }
-
-To paginate, check for ``next_token`` and if present, call the endpoint again
-with ``from`` set to the value of ``next_token``. This will return a new page.
-
-If the endpoint does not return a ``next_token`` then there are no more
-reports to paginate through.
-
-**Parameters**
-
-The following parameters should be set in the URL:
-
- ``user_id`` - string - fully qualified: for example, ``@user:server.com``.
- ``limit``: string representing a positive integer - Is optional but is used for pagination,
-  denoting the maximum number of items to return in this call. Defaults to ``100``.
- ``from``: string representing a positive integer - Is optional but used for pagination,
-  denoting the offset in the returned results. This should be treated as an opaque value and
-  not explicitly set to anything other than the return value of ``next_token`` from a previous call.
-  Defaults to ``0``.
-
-**Response**
-
-The following fields are returned in the JSON response body:
-
- ``media`` - An array of objects, each containing information about a media.
-  Media objects contain the following fields:
-
-  - ``created_ts`` - integer - Timestamp when the content was uploaded in ms.
-  - ``last_access_ts`` - integer - Timestamp when the content was last accessed in ms.
-  - ``media_id`` - string - The id used to refer to the media.
-  - ``media_length`` - integer - Length of the media in bytes.
-  - ``media_type`` - string - The MIME-type of the media.
-  - ``quarantined_by`` - string - The user ID that initiated the quarantine request
-    for this media.
-
-  - ``safe_from_quarantine`` - bool - Status if this media is safe from quarantining.
-  - ``upload_name`` - string - The name the media was uploaded with.
-
- ``next_token``: integer - Indication for pagination. See above.
- ``total`` - integer - Total number of media.
-
 User devices
 ============

@@ -458,8 +375,7 @@ A response body like the following is returned:
          "last_seen_ts": 1474491775025,
          "user_id": "<user_id>"
        }
-      ],
-      "total": 2
+      ]
    }

 **Parameters**
@@ -484,8 +400,6 @@ The following fields are returned in the JSON response body:
    devices was last seen. (May be a few minutes out of date, for efficiency reasons).
  - ``user_id`` - Owner of  device.

- ``total`` - Total number of user's devices.
-
 Delete multiple devices
 ------------------
 Deletes the given devices for a specific ``user_id``, and invalidates
@@ -611,82 +525,3 @@ The following parameters should be set in the URL:

 - ``user_id`` - fully qualified: for example, ``@user:server.com``.
 - ``device_id`` - The device to delete.
-
-List all pushers
-================
-Gets information about all pushers for a specific ``user_id``.
-
-The API is::
-
-  GET /_synapse/admin/v1/users/<user_id>/pushers
-
-To use it, you will need to authenticate by providing an ``access_token`` for a
-server admin: see `README.rst <README.rst>`_.
-
-A response body like the following is returned:
-
-.. code:: json
-
-    {
-      "pushers": [
-        {
-          "app_display_name":"HTTP Push Notifications",
-          "app_id":"m.http",
-          "data": {
-            "url":"example.com"
-          },
-          "device_display_name":"pushy push",
-          "kind":"http",
-          "lang":"None",
-          "profile_tag":"",
-          "pushkey":"a@example.com"
-        }
-      ],
-      "total": 1
-    }
-
-**Parameters**
-
-The following parameters should be set in the URL:
-
- ``user_id`` - fully qualified: for example, ``@user:server.com``.
-
-**Response**
-
-The following fields are returned in the JSON response body:
-
- ``pushers`` - An array containing the current pushers for the user
-
-  - ``app_display_name`` - string - A string that will allow the user to identify
-    what application owns this pusher.
-
-  - ``app_id`` - string - This is a reverse-DNS style identifier for the application.
-    Max length, 64 chars.
-
-  - ``data`` - A dictionary of information for the pusher implementation itself.
-
-    - ``url`` - string - Required if ``kind`` is ``http``. The URL to use to send
-      notifications to.
-
-    - ``format`` - string - The format to use when sending notifications to the
-      Push Gateway.
-
-  - ``device_display_name`` - string -  A string that will allow the user to identify
-    what device owns this pusher.
-
-  - ``profile_tag`` - string - This string determines which set of device specific rules
-    this pusher executes.
-
-  - ``kind`` - string -  The kind of pusher. "http" is a pusher that sends HTTP pokes.
-  - ``lang`` - string - The preferred language for receiving notifications
-    (e.g. 'en' or 'en-US')
-
-  - ``profile_tag`` - string - This string determines which set of device specific rules
-    this pusher executes.
-
-  - ``pushkey`` - string - This is a unique identifier for this pusher.
-    Max length, 512 bytes.
-
- ``total`` - integer - Number of pushers.
-
-See also `Client-Server API Spec <https://matrix.org/docs/spec/client_server/latest#get-matrix-client-r0-pushers>`_
@@ -64,6 +64,8 @@ save as it takes a while and is very resource intensive.
    -   Use underscores for functions and variables.
 -   **Docstrings**: should follow the [google code
    style](https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings).
+    This is so that we can generate documentation with
+    [sphinx](http://sphinxcontrib-napoleon.readthedocs.org/en/latest/).
    See the
    [examples](http://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html)
    in the sphinx documentation.
@@ -5,45 +5,8 @@ The "manhole" allows server administrators to access a Python shell on a running
 Synapse installation. This is a very powerful mechanism for administration and
 debugging.

-**_Security Warning_**
-
-Note that this will give administrative access to synapse to **all users** with
-shell access to the server. It should therefore **not** be enabled in
-environments where untrusted users have shell access.
-
-***
-
 To enable it, first uncomment the `manhole` listener configuration in
-`homeserver.yaml`. The configuration is slightly different if you're using docker.
-
-#### Docker config
-
-If you are using Docker, set `bind_addresses` to `['0.0.0.0']` as shown:
-
-```yaml
-listeners:
-  - port: 9000
-    bind_addresses: ['0.0.0.0']
-    type: manhole
-```
-
-When using `docker run` to start the server, you will then need to change the command to the following to include the
-`manhole` port forwarding. The `-p 127.0.0.1:9000:9000` below is important: it 
-ensures that access to the `manhole` is only possible for local users.
-
-```bash
-docker run -d --name synapse \
-    --mount type=volume,src=synapse-data,dst=/data \
-    -p 8008:8008 \
-    -p 127.0.0.1:9000:9000 \
-    matrixdotorg/synapse:latest
-```
-
-#### Native config
-
-If you are not using docker, set `bind_addresses` to `['::1', '127.0.0.1']` as shown.
-The `bind_addresses` in the example below is important: it ensures that access to the
-`manhole` is only possible for local users).
+`homeserver.yaml`:

 ```yaml
 listeners:
@@ -52,7 +15,12 @@ listeners:
    type: manhole
 ```

-#### Accessing synapse manhole
+(`bind_addresses` in the above is important: it ensures that access to the
+manhole is only possible for local users).
+
+Note that this will give administrative access to synapse to **all users** with
+shell access to the server. It should therefore **not** be enabled in
+environments where untrusted users have shell access.

 Then restart synapse, and point an ssh client at port 9000 on localhost, using
 the username `matrix`:
@@ -67,12 +35,9 @@ This gives a Python REPL in which `hs` gives access to the
 `synapse.server.HomeServer` object - which in turn gives access to many other
 parts of the process.

-Note that any call which returns a coroutine will need to be wrapped in `ensureDeferred`.
-
 As a simple example, retrieving an event from the database:

-```pycon
->>> from twisted.internet import defer
->>> defer.ensureDeferred(hs.get_datastore().get_event('$1416420717069yeQaw:matrix.org'))
+```
+>>> hs.get_datastore().get_event('$1416420717069yeQaw:matrix.org')
 <Deferred at 0x7ff253fc6998 current result: <FrozenEvent event_id='$1416420717069yeQaw:matrix.org', type='m.room.create', state_key=''>>
 ```
@@ -136,34 +136,24 @@ the server's database.

 ### Lifetime limits

-Server admins can set limits on the values of `max_lifetime` to use when
-purging old events in a room. These limits can be defined as such in the
-`retention` section of the configuration file:
+**Note: this feature is mainly useful within a closed federation or on
+servers that don't federate, because there currently is no way to
+enforce these limits in an open federation.**
+
+Server admins can restrict the values their local users are allowed to
+use for both `min_lifetime` and `max_lifetime`. These limits can be
+defined as such in the `retention` section of the configuration file:

 ```yaml
  allowed_lifetime_min: 1d
  allowed_lifetime_max: 1y
 ```

-The limits are considered when running purge jobs. If necessary, the
-effective value of `max_lifetime` will be brought between
-`allowed_lifetime_min` and `allowed_lifetime_max` (inclusive).
-This means that, if the value of `max_lifetime` defined in the room's state
-is lower than `allowed_lifetime_min`, the value of `allowed_lifetime_min`
-will be used instead. Likewise, if the value of `max_lifetime` is higher
-than `allowed_lifetime_max`, the value of `allowed_lifetime_max` will be
-used instead.
-
-In the example above, we ensure Synapse never deletes events that are less
-than one day old, and that it always deletes events that are over a year
-old.
-
-If a default policy is set, and its `max_lifetime` value is lower than
-`allowed_lifetime_min` or higher than `allowed_lifetime_max`, the same
-process applies.
-
-Both parameters are optional; if one is omitted Synapse won't use it to
-adjust the effective value of `max_lifetime`.
+Here, `allowed_lifetime_min` is the lowest value a local user can set
+for both `min_lifetime` and `max_lifetime`, and `allowed_lifetime_max`
+is the highest value. Both parameters are optional (e.g. setting
+`allowed_lifetime_min` but not `allowed_lifetime_max` only enforces a
+minimum and no maximum).

 Like other settings in this section, these parameters can be expressed
 either as a duration or as a number of milliseconds.
@@ -60,8 +60,6 @@

 1.  Restart Prometheus.

-1.  Consider using the [grafana dashboard](https://github.com/matrix-org/synapse/tree/master/contrib/grafana/) and required [recording rules](https://github.com/matrix-org/synapse/tree/master/contrib/prometheus/) 
-
 ## Monitoring workers

 To monitor a Synapse installation using
@@ -37,7 +37,7 @@ as follows:
   provided by `matrix.org` so no further action is needed.

 * If you installed Synapse into a virtualenv, run `/path/to/env/bin/pip
-   install matrix-synapse[oidc]` to install the necessary dependencies.
+   install synapse[oidc]` to install the necessary dependencies.

 * For other installation mechanisms, see the documentation provided by the
   maintainer.
@@ -52,39 +52,14 @@ specific providers.

 Here are a few configs for providers that should work with Synapse.

-### Microsoft Azure Active Directory
-Azure AD can act as an OpenID Connect Provider. Register a new application under 
-*App registrations* in the Azure AD management console. The RedirectURI for your
-application should point to your matrix server: `[synapse public baseurl]/_synapse/oidc/callback`
-
-Go to *Certificates & secrets* and register a new client secret. Make note of your 
-Directory (tenant) ID as it will be used in the Azure links.
-Edit your Synapse config file and change the `oidc_config` section:
-
-```yaml
-oidc_config:
-   enabled: true
-   issuer: "https://login.microsoftonline.com/<tenant id>/v2.0"
-   client_id: "<client id>"
-   client_secret: "<client secret>"
-   scopes: ["openid", "profile"]
-   authorization_endpoint: "https://login.microsoftonline.com/<tenant id>/oauth2/v2.0/authorize"
-   token_endpoint: "https://login.microsoftonline.com/<tenant id>/oauth2/v2.0/token"
-   userinfo_endpoint: "https://graph.microsoft.com/oidc/userinfo"
-
-   user_mapping_provider:
-     config:
-       localpart_template: "{{ user.preferred_username.split('@')[0] }}"
-       display_name_template: "{{ user.name }}"
-```
-
 ### [Dex][dex-idp]

 [Dex][dex-idp] is a simple, open-source, certified OpenID Connect Provider.
 Although it is designed to help building a full-blown provider with an
 external database, it can be configured with static passwords in a config file.

-Follow the [Getting Started guide](https://dexidp.io/docs/getting-started/)
+Follow the [Getting Started
+guide](https://github.com/dexidp/dex/blob/master/Documentation/getting-started.md)
 to install Dex.

 Edit `examples/config-dev.yaml` config file from the Dex repo to add a client:
@@ -98,7 +73,7 @@ staticClients:
  name: 'Synapse'
 ```

-Run with `dex serve examples/config-dev.yaml`.
+Run with `dex serve examples/config-dex.yaml`.

 Synapse config:

@@ -205,7 +180,7 @@ GitHub is a bit special as it is not an OpenID Connect compliant provider, but
 just a regular OAuth2 provider.

 The [`/user` API endpoint](https://developer.github.com/v3/users/#get-the-authenticated-user)
-can be used to retrieve information on the authenticated user. As the Synapse
+can be used to retrieve information on the authenticated user. As the Synaspse
 login mechanism needs an attribute to uniquely identify users, and that endpoint
 does not return a `sub` property, an alternative `subject_claim` has to be set.

@@ -263,36 +238,13 @@ Synapse config:

 ```yaml
 oidc_config:
-  enabled: true
-  issuer: "https://id.twitch.tv/oauth2/"
-  client_id: "your-client-id" # TO BE FILLED
-  client_secret: "your-client-secret" # TO BE FILLED
-  client_auth_method: "client_secret_post"
-  user_mapping_provider:
-    config:
-      localpart_template: "{{ user.preferred_username }}"
-      display_name_template: "{{ user.name }}"
-```
-
-### GitLab
-
-1. Create a [new application](https://gitlab.com/profile/applications).
-2. Add the `read_user` and `openid` scopes.
-3. Add this Callback URL: `[synapse public baseurl]/_synapse/oidc/callback`
-
-Synapse config:
-
-```yaml
-oidc_config:
-  enabled: true
-  issuer: "https://gitlab.com/"
-  client_id: "your-client-id" # TO BE FILLED
-  client_secret: "your-client-secret" # TO BE FILLED
-  client_auth_method: "client_secret_post"
-  scopes: ["openid", "read_user"]
-  user_profile_method: "userinfo_endpoint"
-  user_mapping_provider:
-    config:
-      localpart_template: '{{ user.nickname }}'
-      display_name_template: '{{ user.name }}'
+   enabled: true
+   issuer: "https://id.twitch.tv/oauth2/"
+   client_id: "your-client-id" # TO BE FILLED
+   client_secret: "your-client-secret" # TO BE FILLED
+   client_auth_method: "client_secret_post"
+   user_mapping_provider:
+     config:
+       localpart_template: '{{ user.preferred_username }}'
+       display_name_template: '{{ user.name }}'
 ```
@@ -106,17 +106,6 @@ Note that the above may fail with an error about duplicate rows if corruption
 has already occurred, and such duplicate rows will need to be manually removed.


-## Fixing inconsistent sequences error
-
-Synapse uses Postgres sequences to generate IDs for various tables. A sequence
-and associated table can get out of sync if, for example, Synapse has been
-downgraded and then upgraded again.
-
-To fix the issue shut down Synapse (including any and all workers) and run the
-SQL command included in the error message. Once done Synapse should start
-successfully.
-
-
 ## Tuning Postgres

 The default settings should be fine for most deployments. For larger
@@ -54,7 +54,7 @@ server {
        proxy_set_header X-Forwarded-For $remote_addr;
        # Nginx by default only allows file uploads up to 1M in size
        # Increase client_max_body_size to match max_upload_size defined in homeserver.yaml
-        client_max_body_size 50M;
+        client_max_body_size 10M;
    }
 }
 ```
@@ -33,23 +33,10 @@

 ## Server ##

-# The public-facing domain of the server
-#
-# The server_name name will appear at the end of usernames and room addresses
-# created on this server. For example if the server_name was example.com,
-# usernames on this server would be in the format @user:example.com
-#
-# In most cases you should avoid using a matrix specific subdomain such as
-# matrix.example.com or synapse.example.com as the server_name for the same
-# reasons you wouldn't use user@email.example.com as your email address.
-# See https://github.com/matrix-org/synapse/blob/master/docs/delegate.md
-# for information on how to host Synapse on a subdomain while preserving
-# a clean server_name.
-#
-# The server_name cannot be changed later so it is important to
-# configure this correctly before you start Synapse. It should be all
-# lowercase and may contain an explicit port.
-# Examples: matrix.org, localhost:8080
+# The domain name of the server, with optional explicit port.
+# This is used by remote servers to connect to this server,
+# e.g. matrix.org, localhost:8080, etc.
+# This is also the last part of your UserID.
 #
 server_name: "SERVERNAME"

@@ -119,7 +106,7 @@ pid_file: DATADIR/homeserver.pid
 # For example, for room version 1, default_room_version should be set
 # to "1".
 #
-#default_room_version: "6"
+#default_room_version: "5"

 # The GC threshold parameters to pass to `gc.set_threshold`, if defined
 #
@@ -629,7 +616,6 @@ acme:
 #tls_fingerprints: [{"sha256": "<base64_encoded_sha256_fingerprint>"}]


-## Federation ##

 # Restrict federation to the following whitelist of domains.
 # N.B. we recommend also firewalling your federation listener to limit
@@ -663,17 +649,6 @@ federation_ip_range_blacklist:
  - 'fe80::/64'
  - 'fc00::/7'

-# Report prometheus metrics on the age of PDUs being sent to and received from
-# the following domains. This can be used to give an idea of "delay" on inbound
-# and outbound federation, though be aware that any delay can be due to problems
-# at either end or with the intermediate network.
-#
-# By default, no domains are monitored in this way.
-#
-#federation_metrics_domains:
-#  - matrix.org
-#  - example.com
-

 ## Caching ##

@@ -893,7 +868,7 @@ media_store_path: "DATADIR/media_store"

 # The largest allowed upload size in bytes
 #
-#max_upload_size: 50M
+#max_upload_size: 10M

 # Maximum number of pixels that will be thumbnailed
 #
@@ -1505,8 +1480,10 @@ trusted_key_servers:

 ## Single sign-on integration ##

-# The following settings can be used to make Synapse use a single sign-on
-# provider for authentication, instead of its internal password database.
+# Enable SAML2 for registration and login. Uses pysaml2.
+#
+# At least one of `sp_config` or `config_path` must be set in this section to
+# enable SAML login.
 #
 # You will probably also want to set the following options to `false` to
 # disable the regular login/registration flows:
@@ -1515,11 +1492,6 @@ trusted_key_servers:
 #
 # You will also want to investigate the settings under the "sso" configuration
 # section below.
-
-# Enable SAML2 for registration and login. Uses pysaml2.
-#
-# At least one of `sp_config` or `config_path` must be set in this section to
-# enable SAML login.
 #
 # Once SAML support is enabled, a metadata file will be exposed at
 # https://<server>:<port>/_matrix/saml2/metadata.xml, which you may be able to
@@ -1535,64 +1507,40 @@ saml2_config:
  # so it is not normally necessary to specify them unless you need to
  # override them.
  #
-  sp_config:
-    # Point this to the IdP's metadata. You must provide either a local
-    # file via the `local` attribute or (preferably) a URL via the
-    # `remote` attribute.
-    #
-    #metadata:
-    #  local: ["saml2/idp.xml"]
-    #  remote:
-    #    - url: https://our_idp/metadata.xml
-
-    # By default, the user has to go to our login page first. If you'd like
-    # to allow IdP-initiated login, set 'allow_unsolicited: true' in a
-    # 'service.sp' section:
-    #
-    #service:
-    #  sp:
-    #    allow_unsolicited: true
-
-    # The examples below are just used to generate our metadata xml, and you
-    # may well not need them, depending on your setup. Alternatively you
-    # may need a whole lot more detail - see the pysaml2 docs!
-
-    #description: ["My awesome SP", "en"]
-    #name: ["Test SP", "en"]
-
-    #ui_info:
-    #  display_name:
-    #    - lang: en
-    #      text: "Display Name is the descriptive name of your service."
-    #  description:
-    #    - lang: en
-    #      text: "Description should be a short paragraph explaining the purpose of the service."
-    #  information_url:
-    #    - lang: en
-    #      text: "https://example.com/terms-of-service"
-    #  privacy_statement_url:
-    #    - lang: en
-    #      text: "https://example.com/privacy-policy"
-    #  keywords:
-    #    - lang: en
-    #      text: ["Matrix", "Element"]
-    #  logo:
-    #    - lang: en
-    #      text: "https://example.com/logo.svg"
-    #      width: "200"
-    #      height: "80"
-
-    #organization:
-    #  name: Example com
-    #  display_name:
-    #    - ["Example co", "en"]
-    #  url: "http://example.com"
-
-    #contact_person:
-    #  - given_name: Bob
-    #    sur_name: "the Sysadmin"
-    #    email_address": ["admin@example.com"]
-    #    contact_type": technical
+  #sp_config:
+  #  # point this to the IdP's metadata. You can use either a local file or
+  #  # (preferably) a URL.
+  #  metadata:
+  #    #local: ["saml2/idp.xml"]
+  #    remote:
+  #      - url: https://our_idp/metadata.xml
+  #
+  #  # By default, the user has to go to our login page first. If you'd like
+  #  # to allow IdP-initiated login, set 'allow_unsolicited: true' in a
+  #  # 'service.sp' section:
+  #  #
+  #  #service:
+  #  #  sp:
+  #  #    allow_unsolicited: true
+  #
+  #  # The examples below are just used to generate our metadata xml, and you
+  #  # may well not need them, depending on your setup. Alternatively you
+  #  # may need a whole lot more detail - see the pysaml2 docs!
+  #
+  #  description: ["My awesome SP", "en"]
+  #  name: ["Test SP", "en"]
+  #
+  #  organization:
+  #    name: Example com
+  #    display_name:
+  #      - ["Example co", "en"]
+  #    url: "http://example.com"
+  #
+  #  contact_person:
+  #    - given_name: Bob
+  #      sur_name: "the Sysadmin"
+  #      email_address": ["admin@example.com"]
+  #      contact_type": technical

  # Instead of putting the config inline as above, you can specify a
  # separate pysaml2 configuration file:
@@ -1668,10 +1616,11 @@ saml2_config:
  #    value: "sales"


-# Enable OpenID Connect (OIDC) / OAuth 2.0 for registration and login.
+# OpenID Connect integration. The following settings can be used to make Synapse
+# use an OpenID Connect Provider for authentication, instead of its internal
+# password database.
 #
-# See https://github.com/matrix-org/synapse/blob/master/docs/openid.md
-# for some example configurations.
+# See https://github.com/matrix-org/synapse/blob/master/docs/openid.md.
 #
 oidc_config:
  # Uncomment the following to enable authorization against an OpenID Connect
@@ -1740,14 +1689,6 @@ oidc_config:
  #
  #skip_verification: true

-  # Whether to fetch the user profile from the userinfo endpoint. Valid
-  # values are: "auto" or "userinfo_endpoint".
-  #
-  # Defaults to "auto", which fetches the userinfo endpoint if "openid" is included
-  # in `scopes`. Uncomment the following to always fetch the userinfo endpoint.
-  #
-  #user_profile_method: "userinfo_endpoint"
-
  # Uncomment to allow a user logging in via OIDC to match a pre-existing account instead
  # of failing. This could be used if switching from password logins to OIDC. Defaults to false.
  #
@@ -1794,47 +1735,17 @@ oidc_config:
      #
      #display_name_template: "{{ user.given_name }} {{ user.last_name }}"

-      # Jinja2 templates for extra attributes to send back to the client during
-      # login.
-      #
-      # Note that these are non-standard and clients will ignore them without modifications.
-      #
-      #extra_attributes:
-        #birthdate: "{{ user.birthdate }}"


-
-# Enable Central Authentication Service (CAS) for registration and login.
+# Enable CAS for registration and login.
 #
-cas_config:
-  # Uncomment the following to enable authorization against a CAS server.
-  # Defaults to false.
-  #
-  #enabled: true
-
-  # The URL of the CAS authorization endpoint.
-  #
-  #server_url: "https://cas-server.com"
-
-  # The public URL of the homeserver.
-  #
-  #service_url: "https://homeserver.domain.com:8448"
-
-  # The attribute of the CAS response to use as the display name.
-  #
-  # If unset, no displayname will be set.
-  #
-  #displayname_attribute: name
-
-  # It is possible to configure Synapse to only allow logins if CAS attributes
-  # match particular values. All of the keys in the mapping below must exist
-  # and the values must match the given value. Alternately if the given value
-  # is None then any value is allowed (the attribute just must exist).
-  # All of the listed attributes must match for the login to be permitted.
-  #
-  #required_attributes:
-  #  userGroup: "staff"
-  #  department: None
+#cas_config:
+#   enabled: true
+#   server_url: "https://cas-server.com"
+#   service_url: "https://homeserver.domain.com:8448"
+#   #displayname_attribute: name
+#   #required_attributes:
+#   #    name: value


 # Additional settings to use with single-sign on systems such as OpenID Connect,
@@ -1934,7 +1845,7 @@ sso:
 # and issued at ("iat") claims are validated if present.
 #
 # Note that this is a non-standard login type and client support is
-# expected to be non-existent.
+# expected to be non-existant.
 #
 # See https://github.com/matrix-org/synapse/blob/master/docs/jwt.md.
 #
@@ -2450,7 +2361,7 @@ spam_checker:
 #
 # Options for the rules include:
 #
-#   user_id: Matches against the creator of the alias
+#   user_id: Matches agaisnt the creator of the alias
 #   room_id: Matches against the room ID being published
 #   alias: Matches against any current local or canonical aliases
 #            associated with the room
@@ -2496,7 +2407,7 @@ opentracing:
    # This is a list of regexes which are matched against the server_name of the
    # homeserver.
    #
-    # By default, it is empty, so no servers are matched.
+    # By defult, it is empty, so no servers are matched.
    #
    #homeserver_whitelist:
    #  - ".*"
@@ -2552,11 +2463,6 @@ opentracing:
 #  events: worker1
 #  typing: worker1

-# The worker that is used to run background tasks (e.g. cleaning up expired
-# data). If not provided this defaults to the main process.
-#
-#run_background_tasks_on: worker1
-

 # Configuration for Redis when using workers. This *must* be enabled when
 # using workers (unless using old style direct TCP configuration).
@@ -3,11 +3,7 @@
 # This is a YAML file containing a standard Python logging configuration
 # dictionary. See [1] for details on the valid settings.
 #
-# Synapse also supports structured logging for machine readable logs which can
-# be ingested by ELK stacks. See [2] for details.
-#
 # [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema
-# [2]: https://github.com/matrix-org/synapse/blob/master/docs/structured_logging.md

 version: 1

@@ -63,7 +59,7 @@ root:
    # then write them to a file.
    #
    # Replace "buffer" with "console" to log to stderr instead. (Note that you'll
-    # also need to update the configuration for the `twisted` logger above, in
+    # also need to update the configuation for the `twisted` logger above, in
    # this case.)
    #
    handlers: [buffer]
@@ -11,7 +11,7 @@ able to be imported by the running Synapse.
 The Python class is instantiated with two objects:

 * Any configuration (see below).
-* An instance of `synapse.module_api.ModuleApi`.
+* An instance of `synapse.spam_checker_api.SpamCheckerApi`.

 It then implements methods which return a boolean to alter behavior in Synapse.

@@ -26,8 +26,11 @@ well as some specific methods:
 The details of the each of these methods (as well as their inputs and outputs)
 are documented in the `synapse.events.spamcheck.SpamChecker` class.

-The `ModuleApi` class provides a way for the custom spam checker class to
-call back into the homeserver internals.
+The `SpamCheckerApi` class provides a way for the custom spam checker class to
+call back into the homeserver internals. It currently implements the following
+methods:
+
+* `get_state_events_in_room`

 ### Example

--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Erik Johnston	11149c89c7	Newsfile	2020-09-28 14:33:56 +01:00
Erik Johnston	40c4cfbfb1	Correctly retry replication HTTP requests on timeout	2020-09-28 14:33:56 +01:00
				`@@ -0,0 +1 @@`
				`Fix a bug in the media repository where remote thumbnails with the same size but different crop methods would overwrite each other. Contributed by @deepbluev7.`
				`@@ -0,0 +1 @@`
				Fix inconsistent handling of non-existent push rules, and stop tracking the `enabled` state of removed push rules.
				`@@ -0,0 +1 @@`
				`Require the user to confirm that their password should be reset after clicking the email confirmation link.`
				`@@ -0,0 +1 @@`
				`Fix tests on distros which disable TLSv1.0. Contributed by @danc86.`
				`@@ -0,0 +1 @@`
				`Simplify the distributor code to avoid unnecessary work.`
				`@@ -0,0 +1 @@`
				Add an admin API `GET /_synapse/admin/v1/event_reports` to read entries of table `event_reports`. Contributed by @dklimpel.
				`@@ -0,0 +1 @@`
				Add `/_synapse/client` to the reverse proxy documentation.
				`@@ -0,0 +1 @@`
				`Fix messages over federation being lost until an event is sent into the same room.`
				`@@ -0,0 +1 @@`
				`Fix a longstanding bug where files that could not be thumbnailed would result in an Internal Server Error.`