Tweak CircleCI config to trigger rebuild of v1.23.1

It seems that letting CircleCI use its default docker version (17.09.0-ce,
apparently) did not interact well with multiarch builds: in particular, we saw
weird effects where running an amd64 build at the same time as an arm64 build
caused the arm64 builds to fail with:

   Error while loading /usr/sbin/dpkg-deb: No such file or directory

.circleci/config.yml

@@ -5,53 +5,30 @@ jobs:
       - image: docker:git
     steps:
       - checkout
-      - setup_remote_docker
       - docker_prepare
       - run: docker login --username $DOCKER_HUB_USERNAME --password $DOCKER_HUB_PASSWORD
       - docker_build:
-          tag: -t matrixdotorg/synapse:${CIRCLE_TAG}
-          platforms: linux/amd64
-      - docker_build:
-          tag: -t matrixdotorg/synapse:${CIRCLE_TAG}
+          tag: -t matrixdotorg/synapse:v1.23.1
           platforms: linux/amd64,linux/arm/v7,linux/arm64
 
-  dockerhubuploadlatest:
-    docker:
-      - image: docker:git
-    steps:
-      - checkout
-      - setup_remote_docker
-      - docker_prepare
-      - run: docker login --username $DOCKER_HUB_USERNAME --password $DOCKER_HUB_PASSWORD
-      - docker_build:
-          tag: -t matrixdotorg/synapse:latest
-          platforms: linux/amd64
-      - docker_build:
-          tag: -t matrixdotorg/synapse:latest
-          platforms: linux/amd64,linux/arm/v7,linux/arm64
 
 workflows:
   build:
     jobs:
-      - dockerhubuploadrelease:
-          filters:
-            tags:
-              only: /v[0-9].[0-9]+.[0-9]+.*/
-            branches:
-              ignore: /.*/
-      - dockerhubuploadlatest:
-          filters:
-            branches:
-              only: master
+      - dockerhubuploadrelease
 
 commands:
   docker_prepare:
-    description: Downloads the buildx cli plugin and enables multiarch images
+    description: Sets up a remote docker server, downloads the buildx cli plugin, and enables multiarch images
     parameters:
       buildx_version:
         type: string
         default: "v0.4.1"
     steps:
+      - setup_remote_docker:
+          # 19.03.13 was the most recent available on circleci at the time of
+          # writing.
+          version: 19.03.13
       - run: apk add --no-cache curl
       - run: mkdir -vp ~/.docker/cli-plugins/ ~/dockercache
       - run: curl --silent -L "https://github.com/docker/buildx/releases/download/<< parameters.buildx_version >>/buildx-<< parameters.buildx_version >>.linux-amd64" > ~/.docker/cli-plugins/docker-buildx

CHANGES.md

@@ -1,3 +1,279 @@
+Synapse 1.23.1 (2020-12-09)
+===========================
+
+Due to the two security issues highlighted below, server administrators are
+encouraged to update Synapse. We are not aware of these vulnerabilities being
+exploited in the wild.
+
+Security advisory
+-----------------
+
+The following issues are fixed in v1.23.1 and v1.24.0.
+
+- There is a denial of service attack
+  ([CVE-2020-26257](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-26257))
+  against the federation APIs in which future events will not be correctly sent
+  to other servers over federation. This affects all servers that participate in
+  open federation. (Fixed in [#8776](https://github.com/matrix-org/synapse/pull/8776)).
+
+- Synapse may be affected by OpenSSL
+  [CVE-2020-1971](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-1971).
+  Synapse administrators should ensure that they have the latest versions of
+  the cryptography Python package installed.
+
+To upgrade Synapse along with the cryptography package:
+
+* Administrators using the [`matrix.org` Docker
+  image](https://hub.docker.com/r/matrixdotorg/synapse/) or the [Debian/Ubuntu
+  packages from
+  `matrix.org`](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#matrixorg-packages)
+  should ensure that they have version 1.24.0 or 1.23.1 installed: these images include
+  the updated packages.
+* Administrators who have [installed Synapse from
+  source](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#installing-from-source)
+  should upgrade the cryptography package within their virtualenv by running:
+  ```sh
+  <path_to_virtualenv>/bin/pip install 'cryptography>=3.3'
+  ```
+* Administrators who have installed Synapse from distribution packages should
+  consult the information from their distributions.
+
+Bugfixes
+--------
+
+- Fix a bug in some federation APIs which could lead to unexpected behaviour if different parameters were set in the URI and the request body. ([\#8776](https://github.com/matrix-org/synapse/issues/8776))
+
+
+Internal Changes
+----------------
+
+- Add a maximum version for pysaml2 on Python 3.5. ([\#8898](https://github.com/matrix-org/synapse/issues/8898))
+
+
+Synapse 1.23.0 (2020-11-18)
+===========================
+
+This release changes the way structured logging is configured. See the [upgrade notes](UPGRADE.rst#upgrading-to-v1230) for details.
+
+**Note**: We are aware of a trivially exploitable denial of service vulnerability in versions of Synapse prior to 1.20.0. Complete details will be disclosed on Monday, November 23rd. If you have not upgraded recently, please do so.
+
+Bugfixes
+--------
+
+- Fix a dependency versioning bug in the Dockerfile that prevented Synapse from starting. ([\#8767](https://github.com/matrix-org/synapse/issues/8767))
+
+
+Synapse 1.23.0rc1 (2020-11-13)
+==============================
+
+Features
+--------
+
+- Add a push rule that highlights when a jitsi conference is created in a room. ([\#8286](https://github.com/matrix-org/synapse/issues/8286))
+- Add an admin api to delete a single file or files that were not used for a defined time from server. Contributed by @dklimpel. ([\#8519](https://github.com/matrix-org/synapse/issues/8519))
+- Split admin API for reported events (`GET /_synapse/admin/v1/event_reports`) into detail and list endpoints. This is a breaking change to #8217 which was introduced in Synapse v1.21.0. Those who already use this API should check their scripts. Contributed by @dklimpel. ([\#8539](https://github.com/matrix-org/synapse/issues/8539))
+- Support generating structured logs via the standard logging configuration. ([\#8607](https://github.com/matrix-org/synapse/issues/8607), [\#8685](https://github.com/matrix-org/synapse/issues/8685))
+- Add an admin API to allow server admins to list users' pushers. Contributed by @dklimpel. ([\#8610](https://github.com/matrix-org/synapse/issues/8610), [\#8689](https://github.com/matrix-org/synapse/issues/8689))
+- Add an admin API `GET /_synapse/admin/v1/users/<user_id>/media` to get information about uploaded media. Contributed by @dklimpel. ([\#8647](https://github.com/matrix-org/synapse/issues/8647))
+- Add an admin API for local user media statistics. Contributed by @dklimpel. ([\#8700](https://github.com/matrix-org/synapse/issues/8700))
+- Add `displayname` to Shared-Secret Registration for admins. ([\#8722](https://github.com/matrix-org/synapse/issues/8722))
+
+
+Bugfixes
+--------
+
+- Fix fetching of E2E cross signing keys over federation when only one of the master key and device signing key is cached already. ([\#8455](https://github.com/matrix-org/synapse/issues/8455))
+- Fix a bug where Synapse would blindly forward bad responses from federation to clients when retrieving profile information. ([\#8580](https://github.com/matrix-org/synapse/issues/8580))
+- Fix a bug where the account validity endpoint would silently fail if the user ID did not have an expiration time. It now returns a 400 error. ([\#8620](https://github.com/matrix-org/synapse/issues/8620))
+- Fix email notifications for invites without local state. ([\#8627](https://github.com/matrix-org/synapse/issues/8627))
+- Fix handling of invalid group IDs to return a 400 rather than log an exception and return a 500. ([\#8628](https://github.com/matrix-org/synapse/issues/8628))
+- Fix handling of User-Agent headers that are invalid UTF-8, which caused user agents of users to not get correctly recorded. ([\#8632](https://github.com/matrix-org/synapse/issues/8632))
+- Fix a bug in the `joined_rooms` admin API if the user has never joined any rooms. The bug was introduced, along with the API, in v1.21.0. ([\#8643](https://github.com/matrix-org/synapse/issues/8643))
+- Fix exception during handling multiple concurrent requests for remote media when using multiple media repositories. ([\#8682](https://github.com/matrix-org/synapse/issues/8682))
+- Fix bug that prevented Synapse from recovering after losing connection to the database. ([\#8726](https://github.com/matrix-org/synapse/issues/8726))
+- Fix bug where the `/_synapse/admin/v1/send_server_notice` API could send notices to non-notice rooms. ([\#8728](https://github.com/matrix-org/synapse/issues/8728))
+- Fix PostgreSQL port script fails when DB has no backfilled events. Broke in v1.21.0. ([\#8729](https://github.com/matrix-org/synapse/issues/8729))
+- Fix PostgreSQL port script to correctly handle foreign key constraints. Broke in v1.21.0. ([\#8730](https://github.com/matrix-org/synapse/issues/8730))
+- Fix PostgreSQL port script so that it can be run again after a failure. Broke in v1.21.0. ([\#8755](https://github.com/matrix-org/synapse/issues/8755))
+
+
+Improved Documentation
+----------------------
+
+- Instructions for Azure AD in the OpenID Connect documentation. Contributed by peterk. ([\#8582](https://github.com/matrix-org/synapse/issues/8582))
+- Improve the sample configuration for single sign-on providers. ([\#8635](https://github.com/matrix-org/synapse/issues/8635))
+- Fix the filepath of Dex's example config and the link to Dex's Getting Started guide in the OpenID Connect docs. ([\#8657](https://github.com/matrix-org/synapse/issues/8657))
+- Note support for Python 3.9. ([\#8665](https://github.com/matrix-org/synapse/issues/8665))
+- Minor updates to docs on running tests. ([\#8666](https://github.com/matrix-org/synapse/issues/8666))
+- Interlink prometheus/grafana documentation. ([\#8667](https://github.com/matrix-org/synapse/issues/8667))
+- Notes on SSO logins and media_repository worker. ([\#8701](https://github.com/matrix-org/synapse/issues/8701))
+- Document experimental support for running multiple event persisters. ([\#8706](https://github.com/matrix-org/synapse/issues/8706))
+- Add information regarding the various sources of, and expected contributions to, Synapse's documentation to `CONTRIBUTING.md`. ([\#8714](https://github.com/matrix-org/synapse/issues/8714))
+- Migrate documentation `docs/admin_api/event_reports` to markdown. ([\#8742](https://github.com/matrix-org/synapse/issues/8742))
+- Add some helpful hints to the README for new Synapse developers. Contributed by @chagai95. ([\#8746](https://github.com/matrix-org/synapse/issues/8746))
+
+
+Internal Changes
+----------------
+
+- Optimise `/createRoom` with multiple invited users. ([\#8559](https://github.com/matrix-org/synapse/issues/8559))
+- Implement and use an `@lru_cache` decorator. ([\#8595](https://github.com/matrix-org/synapse/issues/8595))
+- Don't instansiate Requester directly. ([\#8614](https://github.com/matrix-org/synapse/issues/8614))
+- Type hints for `RegistrationStore`. ([\#8615](https://github.com/matrix-org/synapse/issues/8615))
+- Change schema to support access tokens belonging to one user but granting access to another. ([\#8616](https://github.com/matrix-org/synapse/issues/8616))
+- Remove unused OPTIONS handlers. ([\#8621](https://github.com/matrix-org/synapse/issues/8621))
+- Run `mypy` as part of the lint.sh script. ([\#8633](https://github.com/matrix-org/synapse/issues/8633))
+- Correct Synapse's PyPI package name in the OpenID Connect installation instructions. ([\#8634](https://github.com/matrix-org/synapse/issues/8634))
+- Catch exceptions during initialization of `password_providers`. Contributed by Nicolai Søborg. ([\#8636](https://github.com/matrix-org/synapse/issues/8636))
+- Fix typos and spelling errors in the code. ([\#8639](https://github.com/matrix-org/synapse/issues/8639))
+- Reduce number of OpenTracing spans started. ([\#8640](https://github.com/matrix-org/synapse/issues/8640), [\#8668](https://github.com/matrix-org/synapse/issues/8668), [\#8670](https://github.com/matrix-org/synapse/issues/8670))
+- Add field `total` to device list in admin API. ([\#8644](https://github.com/matrix-org/synapse/issues/8644))
+- Add more type hints to the application services code. ([\#8655](https://github.com/matrix-org/synapse/issues/8655), [\#8693](https://github.com/matrix-org/synapse/issues/8693))
+- Tell Black to format code for Python 3.5. ([\#8664](https://github.com/matrix-org/synapse/issues/8664))
+- Don't pull event from DB when handling replication traffic. ([\#8669](https://github.com/matrix-org/synapse/issues/8669))
+- Abstract some invite-related code in preparation for landing knocking. ([\#8671](https://github.com/matrix-org/synapse/issues/8671), [\#8688](https://github.com/matrix-org/synapse/issues/8688))
+- Clarify representation of events in logfiles. ([\#8679](https://github.com/matrix-org/synapse/issues/8679))
+- Don't require `hiredis` package to be installed to run unit tests. ([\#8680](https://github.com/matrix-org/synapse/issues/8680))
+- Fix typing info on cache call signature to accept `on_invalidate`. ([\#8684](https://github.com/matrix-org/synapse/issues/8684))
+- Fail tests if they do not await coroutines. ([\#8690](https://github.com/matrix-org/synapse/issues/8690))
+- Improve start time by adding an index to `e2e_cross_signing_keys.stream_id`. ([\#8694](https://github.com/matrix-org/synapse/issues/8694))
+- Re-organize the structured logging code to separate the TCP transport handling from the JSON formatting. ([\#8697](https://github.com/matrix-org/synapse/issues/8697))
+- Use Python 3.8 in Docker images by default. ([\#8698](https://github.com/matrix-org/synapse/issues/8698))
+- Remove the "draft" status of the Room Details Admin API. ([\#8702](https://github.com/matrix-org/synapse/issues/8702))
+- Improve the error returned when a non-string displayname or avatar_url is used when updating a user's profile. ([\#8705](https://github.com/matrix-org/synapse/issues/8705))
+- Block attempts by clients to send server ACLs, or redactions of server ACLs, that would result in the local server being blocked from the room. ([\#8708](https://github.com/matrix-org/synapse/issues/8708))
+- Add metrics the allow the local sysadmin to track 3PID `/requestToken` requests. ([\#8712](https://github.com/matrix-org/synapse/issues/8712))
+- Consolidate duplicated lists of purged tables that are checked in tests. ([\#8713](https://github.com/matrix-org/synapse/issues/8713))
+- Add some `mdui:UIInfo` element examples for `saml2_config` in the homeserver config. ([\#8718](https://github.com/matrix-org/synapse/issues/8718))
+- Improve the error message returned when a remote server incorrectly sets the `Content-Type` header in response to a JSON request. ([\#8719](https://github.com/matrix-org/synapse/issues/8719))
+- Speed up repeated state resolutions on the same room by caching event ID to auth event ID lookups. ([\#8752](https://github.com/matrix-org/synapse/issues/8752))
+
+
+Synapse 1.22.1 (2020-10-30)
+===========================
+
+Bugfixes
+--------
+
+- Fix a bug where an appservice may not be forwarded events for a room it was recently invited to. Broke in v1.22.0. ([\#8676](https://github.com/matrix-org/synapse/issues/8676))
+- Fix `Object of type frozendict is not JSON serializable` exceptions when using third-party event rules. Broke in v1.22.0. ([\#8678](https://github.com/matrix-org/synapse/issues/8678))
+
+
+Synapse 1.22.0 (2020-10-27)
+===========================
+
+No significant changes.
+
+
+Synapse 1.22.0rc2 (2020-10-26)
+==============================
+
+Bugfixes
+--------
+
+- Fix bugs where ephemeral events were not sent to appservices. Broke in v1.22.0rc1. ([\#8648](https://github.com/matrix-org/synapse/issues/8648), [\#8656](https://github.com/matrix-org/synapse/issues/8656))
+- Fix `user_daily_visits` table to not have duplicate rows per user/device due to multiple user agents. Broke in v1.22.0rc1. ([\#8654](https://github.com/matrix-org/synapse/issues/8654))
+
+Synapse 1.22.0rc1 (2020-10-22)
+==============================
+
+Features
+--------
+
+- Add a configuration option for always using the "userinfo endpoint" for OpenID Connect. This fixes support for some identity providers, e.g. GitLab. Contributed by Benjamin Koch. ([\#7658](https://github.com/matrix-org/synapse/issues/7658))
+- Add ability for `ThirdPartyEventRules` modules to query and manipulate whether a room is in the public rooms directory. ([\#8292](https://github.com/matrix-org/synapse/issues/8292), [\#8467](https://github.com/matrix-org/synapse/issues/8467))
+- Add support for olm fallback keys ([MSC2732](https://github.com/matrix-org/matrix-doc/pull/2732)). ([\#8312](https://github.com/matrix-org/synapse/issues/8312), [\#8501](https://github.com/matrix-org/synapse/issues/8501))
+- Add support for running background tasks in a separate worker process. ([\#8369](https://github.com/matrix-org/synapse/issues/8369), [\#8458](https://github.com/matrix-org/synapse/issues/8458), [\#8489](https://github.com/matrix-org/synapse/issues/8489), [\#8513](https://github.com/matrix-org/synapse/issues/8513), [\#8544](https://github.com/matrix-org/synapse/issues/8544), [\#8599](https://github.com/matrix-org/synapse/issues/8599))
+- Add support for device dehydration ([MSC2697](https://github.com/matrix-org/matrix-doc/pull/2697)). ([\#8380](https://github.com/matrix-org/synapse/issues/8380))
+- Add support for [MSC2409](https://github.com/matrix-org/matrix-doc/pull/2409), which allows sending typing, read receipts, and presence events to appservices. ([\#8437](https://github.com/matrix-org/synapse/issues/8437), [\#8590](https://github.com/matrix-org/synapse/issues/8590))
+- Change default room version to "6", per [MSC2788](https://github.com/matrix-org/matrix-doc/pull/2788). ([\#8461](https://github.com/matrix-org/synapse/issues/8461))
+- Add the ability to send non-membership events into a room via the `ModuleApi`. ([\#8479](https://github.com/matrix-org/synapse/issues/8479))
+- Increase default upload size limit from 10M to 50M. Contributed by @Akkowicz. ([\#8502](https://github.com/matrix-org/synapse/issues/8502))
+- Add support for modifying event content in `ThirdPartyRules` modules. ([\#8535](https://github.com/matrix-org/synapse/issues/8535), [\#8564](https://github.com/matrix-org/synapse/issues/8564))
+
+
+Bugfixes
+--------
+
+- Fix a longstanding bug where invalid ignored users in account data could break clients. ([\#8454](https://github.com/matrix-org/synapse/issues/8454))
+- Fix a bug where backfilling a room with an event that was missing the `redacts` field would break. ([\#8457](https://github.com/matrix-org/synapse/issues/8457))
+- Don't attempt to respond to some requests if the client has already disconnected. ([\#8465](https://github.com/matrix-org/synapse/issues/8465))
+- Fix message duplication if something goes wrong after persisting the event. ([\#8476](https://github.com/matrix-org/synapse/issues/8476))
+- Fix incremental sync returning an incorrect `prev_batch` token in timeline section, which when used to paginate returned events that were included in the incremental sync. Broken since v0.16.0. ([\#8486](https://github.com/matrix-org/synapse/issues/8486))
+- Expose the `uk.half-shot.msc2778.login.application_service` to clients from the login API. This feature was added in v1.21.0, but was not exposed as a potential login flow. ([\#8504](https://github.com/matrix-org/synapse/issues/8504))
+- Fix error code for `/profile/{userId}/displayname` to be `M_BAD_JSON`. ([\#8517](https://github.com/matrix-org/synapse/issues/8517))
+- Fix a bug introduced in v1.7.0 that could cause Synapse to insert values from non-state `m.room.retention` events into the `room_retention` database table. ([\#8527](https://github.com/matrix-org/synapse/issues/8527))
+- Fix not sending events over federation when using sharded event writers. ([\#8536](https://github.com/matrix-org/synapse/issues/8536))
+- Fix a long standing bug where email notifications for encrypted messages were blank. ([\#8545](https://github.com/matrix-org/synapse/issues/8545))
+- Fix increase in the number of `There was no active span...` errors logged when using OpenTracing. ([\#8567](https://github.com/matrix-org/synapse/issues/8567))
+- Fix a bug that prevented errors encountered during execution of the `synapse_port_db` from being correctly printed. ([\#8585](https://github.com/matrix-org/synapse/issues/8585))
+- Fix appservice transactions to only include a maximum of 100 persistent and 100 ephemeral events. ([\#8606](https://github.com/matrix-org/synapse/issues/8606))
+
+
+Updates to the Docker image
+---------------------------
+
+- Added multi-arch support (arm64,arm/v7) for the docker images. Contributed by @maquis196. ([\#7921](https://github.com/matrix-org/synapse/issues/7921))
+- Add support for passing commandline args to the synapse process. Contributed by @samuel-p. ([\#8390](https://github.com/matrix-org/synapse/issues/8390))
+
+
+Improved Documentation
+----------------------
+
+- Update the directions for using the manhole with coroutines. ([\#8462](https://github.com/matrix-org/synapse/issues/8462))
+- Improve readme by adding new shield.io badges. ([\#8493](https://github.com/matrix-org/synapse/issues/8493))
+- Added note about docker in manhole.md regarding which ip address to bind to. Contributed by @Maquis196. ([\#8526](https://github.com/matrix-org/synapse/issues/8526))
+- Document the new behaviour of the `allowed_lifetime_min` and `allowed_lifetime_max` settings in the room retention configuration. ([\#8529](https://github.com/matrix-org/synapse/issues/8529))
+
+
+Deprecations and Removals
+-------------------------
+
+- Drop unused `device_max_stream_id` table. ([\#8589](https://github.com/matrix-org/synapse/issues/8589))
+
+
+Internal Changes
+----------------
+
+- Check for unreachable code with mypy. ([\#8432](https://github.com/matrix-org/synapse/issues/8432))
+- Add unit test for event persister sharding. ([\#8433](https://github.com/matrix-org/synapse/issues/8433))
+- Allow events to be sent to clients sooner when using sharded event persisters. ([\#8439](https://github.com/matrix-org/synapse/issues/8439), [\#8488](https://github.com/matrix-org/synapse/issues/8488), [\#8496](https://github.com/matrix-org/synapse/issues/8496), [\#8499](https://github.com/matrix-org/synapse/issues/8499))
+- Configure `public_baseurl` when using demo scripts. ([\#8443](https://github.com/matrix-org/synapse/issues/8443))
+- Add SQL logging on queries that happen during startup. ([\#8448](https://github.com/matrix-org/synapse/issues/8448))
+- Speed up unit tests when using PostgreSQL. ([\#8450](https://github.com/matrix-org/synapse/issues/8450))
+- Remove redundant database loads of stream_ordering for events we already have. ([\#8452](https://github.com/matrix-org/synapse/issues/8452))
+- Reduce inconsistencies between codepaths for membership and non-membership events. ([\#8463](https://github.com/matrix-org/synapse/issues/8463))
+- Combine `SpamCheckerApi` with the more generic `ModuleApi`. ([\#8464](https://github.com/matrix-org/synapse/issues/8464))
+- Additional testing for `ThirdPartyEventRules`. ([\#8468](https://github.com/matrix-org/synapse/issues/8468))
+- Add `-d` option to `./scripts-dev/lint.sh` to lint files that have changed since the last git commit. ([\#8472](https://github.com/matrix-org/synapse/issues/8472))
+- Unblacklist some sytests. ([\#8474](https://github.com/matrix-org/synapse/issues/8474))
+- Include the log level in the phone home stats. ([\#8477](https://github.com/matrix-org/synapse/issues/8477))
+- Remove outdated sphinx documentation, scripts and configuration. ([\#8480](https://github.com/matrix-org/synapse/issues/8480))
+- Clarify error message when plugin config parsers raise an error. ([\#8492](https://github.com/matrix-org/synapse/issues/8492))
+- Remove the deprecated `Handlers` object. ([\#8494](https://github.com/matrix-org/synapse/issues/8494))
+- Fix a threadsafety bug in unit tests. ([\#8497](https://github.com/matrix-org/synapse/issues/8497))
+- Add user agent to user_daily_visits table. ([\#8503](https://github.com/matrix-org/synapse/issues/8503))
+- Add type hints to various parts of the code base. ([\#8407](https://github.com/matrix-org/synapse/issues/8407), [\#8505](https://github.com/matrix-org/synapse/issues/8505), [\#8507](https://github.com/matrix-org/synapse/issues/8507), [\#8547](https://github.com/matrix-org/synapse/issues/8547), [\#8562](https://github.com/matrix-org/synapse/issues/8562), [\#8609](https://github.com/matrix-org/synapse/issues/8609))
+- Remove unused code from the test framework. ([\#8514](https://github.com/matrix-org/synapse/issues/8514))
+- Apply some internal fixes to the `HomeServer` class to make its code more idiomatic and statically-verifiable. ([\#8515](https://github.com/matrix-org/synapse/issues/8515))
+- Factor out common code between `RoomMemberHandler._locally_reject_invite` and `EventCreationHandler.create_event`. ([\#8537](https://github.com/matrix-org/synapse/issues/8537))
+- Improve database performance by executing more queries without starting transactions. ([\#8542](https://github.com/matrix-org/synapse/issues/8542))
+- Rename `Cache` to `DeferredCache`, to better reflect its purpose. ([\#8548](https://github.com/matrix-org/synapse/issues/8548))
+- Move metric registration code down into `LruCache`. ([\#8561](https://github.com/matrix-org/synapse/issues/8561), [\#8591](https://github.com/matrix-org/synapse/issues/8591))
+- Replace `DeferredCache` with the lighter-weight `LruCache` where possible. ([\#8563](https://github.com/matrix-org/synapse/issues/8563))
+- Add virtualenv-generated folders to `.gitignore`. ([\#8566](https://github.com/matrix-org/synapse/issues/8566))
+- Add `get_immediate` method to `DeferredCache`. ([\#8568](https://github.com/matrix-org/synapse/issues/8568))
+- Fix mypy not properly checking across the codebase, additionally, fix a typing assertion error in `handlers/auth.py`. ([\#8569](https://github.com/matrix-org/synapse/issues/8569))
+- Fix `synmark` benchmark runner. ([\#8571](https://github.com/matrix-org/synapse/issues/8571))
+- Modify `DeferredCache.get()` to return `Deferred`s instead of `ObservableDeferred`s. ([\#8572](https://github.com/matrix-org/synapse/issues/8572))
+- Adjust a protocol-type definition to fit `sqlite3` assertions. ([\#8577](https://github.com/matrix-org/synapse/issues/8577))
+- Support macOS on the `synmark` benchmark runner. ([\#8578](https://github.com/matrix-org/synapse/issues/8578))
+- Update `mypy` static type checker to 0.790. ([\#8583](https://github.com/matrix-org/synapse/issues/8583), [\#8600](https://github.com/matrix-org/synapse/issues/8600))
+- Re-organize the structured logging code to separate the TCP transport handling from the JSON formatting. ([\#8587](https://github.com/matrix-org/synapse/issues/8587))
+- Remove extraneous unittest logging decorators from unit tests. ([\#8592](https://github.com/matrix-org/synapse/issues/8592))
+- Minor optimisations in caching code. ([\#8593](https://github.com/matrix-org/synapse/issues/8593), [\#8594](https://github.com/matrix-org/synapse/issues/8594))
+
+
 Synapse 1.21.2 (2020-10-15)
 ===========================
 

CONTRIBUTING.md

@@ -46,7 +46,7 @@ locally. You'll need python 3.6 or later, and to install a number of tools:
 
 ```
 # Install the dependencies
-pip install -e ".[lint]"
+pip install -e ".[lint,mypy]"
 
 # Run the linter script
 ./scripts-dev/lint.sh
@@ -63,7 +63,7 @@ run-time:
 ./scripts-dev/lint.sh path/to/file1.py path/to/file2.py path/to/folder
 ```
 
-You can also provided the `-d` option, which will lint the files that have been
+You can also provide the `-d` option, which will lint the files that have been
 changed since the last git commit. This will often be significantly faster than
 linting the whole codebase.
 
@@ -156,6 +156,24 @@ directory, you will need both a regular newsfragment *and* an entry in the
 debian changelog. (Though typically such changes should be submitted as two
 separate pull requests.)
 
+## Documentation
+
+There is a growing amount of documentation located in the [docs](docs)
+directory. This documentation is intended primarily for sysadmins running their
+own Synapse instance, as well as developers interacting externally with
+Synapse. [docs/dev](docs/dev) exists primarily to house documentation for
+Synapse developers. [docs/admin_api](docs/admin_api) houses documentation
+regarding Synapse's Admin API, which is used mostly by sysadmins and external
+service developers.
+
+New files added to both folders should be written in [Github-Flavoured
+Markdown](https://guides.github.com/features/mastering-markdown/), and attempts
+should be made to migrate existing documents to markdown where possible.
+
+Some documentation also exists in [Synapse's Github
+Wiki](https://github.com/matrix-org/synapse/wiki), although this is primarily
+contributed to by community authors.
+
 ## Sign off
 
 In order to have a concrete record that your contribution is intentional

INSTALL.md

@@ -57,7 +57,7 @@ light workloads.
 System requirements:
 
 - POSIX-compliant system (tested on Linux & OS X)
-- Python 3.5.2 or later, up to Python 3.8.
+- Python 3.5.2 or later, up to Python 3.9.
 - At least 1GB of free RAM if you want to join large public rooms like #matrix:matrix.org
 
 Synapse is written in Python but some of the libraries it uses are written in

README.rst

@@ -256,23 +256,27 @@ directory of your choice::
 Synapse has a number of external dependencies, that are easiest
 to install using pip and a virtualenv::
 
-    virtualenv -p python3 env
-    source env/bin/activate
-    python -m pip install --no-use-pep517 -e ".[all]"
+    python3 -m venv ./env
+    source ./env/bin/activate
+    pip install -e ".[all,test]"
 
 This will run a process of downloading and installing all the needed
-dependencies into a virtual env.
+dependencies into a virtual env. If any dependencies fail to install,
+try installing the failing modules individually::
 
-Once this is done, you may wish to run Synapse's unit tests, to
-check that everything is installed as it should be::
+    pip install -e "module-name"
+
+Once this is done, you may wish to run Synapse's unit tests to
+check that everything is installed correctly::
 
     python -m twisted.trial tests
 
-This should end with a 'PASSED' result::
+This should end with a 'PASSED' result (note that exact numbers will
+differ)::
 
-    Ran 143 tests in 0.601s
+    Ran 1337 tests in 716.064s
 
-    PASSED (successes=143)
+    PASSED (skips=15, successes=1322)
 
 Running the Integration Tests
 =============================

UPGRADE.rst

@@ -75,6 +75,22 @@ for example:
      wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb
      dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
 
+Upgrading to v1.23.0
+====================
+
+Structured logging configuration breaking changes
+-------------------------------------------------
+
+This release deprecates use of the ``structured: true`` logging configuration for
+structured logging. If your logging configuration contains ``structured: true``
+then it should be modified based on the `structured logging documentation
+<https://github.com/matrix-org/synapse/blob/master/docs/structured_logging.md>`_.
+
+The ``structured`` and ``drains`` logging options are now deprecated and should
+be replaced by standard logging configuration of ``handlers`` and ``formatters``.
+
+A future will release of Synapse will make using ``structured: true`` an error.
+
 Upgrading to v1.22.0
 ====================
 

changelog.d/7658.feature

@@ -1 +0,0 @@
-Add a configuration option for always using the "userinfo endpoint" for OpenID Connect. This fixes support for some identity providers, e.g. GitLab. Contributed by Benjamin Koch.

changelog.d/7921.docker

@@ -1 +0,0 @@
-Added multi-arch support (arm64,arm/v7) for the docker images. Contributed by @maquis196.

changelog.d/8292.feature

@@ -1 +0,0 @@
-Allow `ThirdPartyEventRules` modules to query and manipulate whether a room is in the public rooms directory.

changelog.d/8312.feature

@@ -1 +0,0 @@
-Add support for olm fallback keys ([MSC2732](https://github.com/matrix-org/matrix-doc/pull/2732)).

changelog.d/8369.feature

@@ -1 +0,0 @@
-Allow running background tasks in a separate worker process.

changelog.d/8380.feature

@@ -1 +0,0 @@
-Add support for device dehydration ([MSC2697](https://github.com/matrix-org/matrix-doc/pull/2697)).

changelog.d/8390.docker

@@ -1 +0,0 @@
-Add support for passing commandline args to the synapse process. Contributed by @samuel-p.

changelog.d/8407.misc

@@ -1 +0,0 @@
-Add typing information to the device handler.

changelog.d/8432.misc

@@ -1 +0,0 @@
-Check for unreachable code with mypy.

changelog.d/8433.misc

@@ -1 +0,0 @@
-Add unit test for event persister sharding.

changelog.d/8437.feature

@@ -1 +0,0 @@
-Implement [MSC2409](https://github.com/matrix-org/matrix-doc/pull/2409) to send typing, read receipts, and presence events to appservices.

changelog.d/8439.misc

@@ -1 +0,0 @@
-Allow events to be sent to clients sooner when using sharded event persisters.

changelog.d/8443.misc

@@ -1 +0,0 @@
-Configure `public_baseurl` when using demo scripts.

changelog.d/8448.misc

@@ -1 +0,0 @@
-Add SQL logging on queries that happen during startup.

changelog.d/8450.misc

@@ -1 +0,0 @@
-Speed up unit tests when using PostgreSQL.

changelog.d/8452.misc

@@ -1 +0,0 @@
-Remove redundant databae loads of stream_ordering for events we already have.

changelog.d/8454.bugfix

@@ -1 +0,0 @@
-Fix a longstanding bug where invalid ignored users in account data could break clients.

changelog.d/8457.bugfix

@@ -1 +0,0 @@
-Fix a bug where backfilling a room with an event that was missing the `redacts` field would break.

changelog.d/8458.feature

@@ -1 +0,0 @@
-Allow running background tasks in a separate worker process.

changelog.d/8461.feature

@@ -1 +0,0 @@
-Change default room version to "6", per [MSC2788](https://github.com/matrix-org/matrix-doc/pull/2788).

changelog.d/8462.doc

@@ -1 +0,0 @@
-Update the directions for using the manhole with coroutines.

changelog.d/8463.misc

@@ -1 +0,0 @@
-Reduce inconsistencies between codepaths for membership and non-membership events.

changelog.d/8464.misc

@@ -1 +0,0 @@
-Combine `SpamCheckerApi` with the more generic `ModuleApi`.

changelog.d/8465.bugfix

@@ -1 +0,0 @@
-Don't attempt to respond to some requests if the client has already disconnected.

changelog.d/8467.feature

@@ -1 +0,0 @@
-Allow `ThirdPartyEventRules` modules to query and manipulate whether a room is in the public rooms directory.

changelog.d/8468.misc

@@ -1 +0,0 @@
-Additional testing for `ThirdPartyEventRules`.

changelog.d/8472.misc

@@ -1 +0,0 @@
-Add `-d` option to `./scripts-dev/lint.sh` to lint files that have changed since the last git commit.

changelog.d/8474.misc

@@ -1 +0,0 @@
-Unblacklist some sytests.

changelog.d/8476.bugfix

@@ -1 +0,0 @@
-Fix message duplication if something goes wrong after persisting the event.

changelog.d/8477.misc

@@ -1 +0,0 @@
-Include the log level in the phone home stats.

changelog.d/8479.feature

@@ -1 +0,0 @@
-Add the ability to send non-membership events into a room via the `ModuleApi`.

changelog.d/8480.misc

@@ -1 +0,0 @@
-Remove outdated sphinx documentation, scripts and configuration.

changelog.d/8486.bugfix

@@ -1 +0,0 @@
-Fix incremental sync returning an incorrect `prev_batch` token in timeline section, which when used to paginate returned events that were included in the incremental sync. Broken since v0.16.0.

changelog.d/8488.misc

@@ -1 +0,0 @@
-Allow events to be sent to clients sooner when using sharded event persisters.

changelog.d/8489.feature

@@ -1 +0,0 @@
- Allow running background tasks in a separate worker process.

changelog.d/8492.misc

@@ -1 +0,0 @@
-Clarify error message when plugin config parsers raise an error.

changelog.d/8493.doc

@@ -1 +0,0 @@
-Improve readme by adding new shield.io badges.

changelog.d/8494.misc

@@ -1 +0,0 @@
-Remove the deprecated `Handlers` object.

changelog.d/8496.misc

@@ -1 +0,0 @@
-Allow events to be sent to clients sooner when using sharded event persisters.

changelog.d/8497.misc

@@ -1 +0,0 @@
-Fix a threadsafety bug in unit tests.

changelog.d/8499.misc

@@ -1 +0,0 @@
-Allow events to be sent to clients sooner when using sharded event persisters.

changelog.d/8501.feature

@@ -1 +0,0 @@
-Add support for olm fallback keys ([MSC2732](https://github.com/matrix-org/matrix-doc/pull/2732)).

changelog.d/8502.feature

@@ -1 +0,0 @@
-Increase default upload size limit from 10M to 50M. Contributed by @Akkowicz.

changelog.d/8503.misc

@@ -1 +0,0 @@
-Add user agent to user_daily_visits table.

changelog.d/8504.bugfix

@@ -1 +0,0 @@
-Expose the `uk.half-shot.msc2778.login.application_service` to clients from the login API. This feature was added in v1.21.0, but was not exposed as a potential login flow.

changelog.d/8505.misc

@@ -1 +0,0 @@
-Add type hints to various parts of the code base.

changelog.d/8507.misc

@@ -1 +0,0 @@
- Add type hints to various parts of the code base.

changelog.d/8513.feature

@@ -1 +0,0 @@
-Allow running background tasks in a separate worker process.

changelog.d/8514.misc

@@ -1 +0,0 @@
-Remove unused code from the test framework.

changelog.d/8515.misc

@@ -1 +0,0 @@
-Apply some internal fixes to the `HomeServer` class to make its code more idiomatic and statically-verifiable.

changelog.d/8517.bugfix

@@ -1 +0,0 @@
-Fix error code for `/profile/{userId}/displayname` to be `M_BAD_JSON`.

changelog.d/8526.doc

@@ -1 +0,0 @@
-Added note about docker in manhole.md regarding which ip address to bind to. Contributed by @Maquis196.

changelog.d/8527.bugfix

@@ -1 +0,0 @@
-Fix a bug introduced in v1.7.0 that could cause Synapse to insert values from non-state `m.room.retention` events into the `room_retention` database table.

changelog.d/8529.doc

@@ -1 +0,0 @@
-Document the new behaviour of the `allowed_lifetime_min` and `allowed_lifetime_max` settings in the room retention configuration.

changelog.d/8535.feature

@@ -1 +0,0 @@
-Support modifying event content in `ThirdPartyRules` modules.

changelog.d/8536.bugfix

@@ -1 +0,0 @@
-Fix not sending events over federation when using sharded event writers.

changelog.d/8537.misc

@@ -1 +0,0 @@
-Factor out common code between `RoomMemberHandler._locally_reject_invite` and `EventCreationHandler.create_event`.

changelog.d/8542.misc

@@ -1 +0,0 @@
-Improve database performance by executing more queries without starting transactions.

changelog.d/8544.feature

@@ -1 +0,0 @@
-Allow running background tasks in a separate worker process.

changelog.d/8545.bugfix

@@ -1 +0,0 @@
-Fix a long standing bug where email notifications for encrypted messages were blank.

changelog.d/8547.misc

@@ -1 +0,0 @@
-Enable mypy type checking for `synapse.util.caches`.

changelog.d/8548.misc

@@ -1 +0,0 @@
-Rename `Cache` to `DeferredCache`, to better reflect its purpose.

changelog.d/8561.misc

@@ -1 +0,0 @@
-Move metric registration code down into `LruCache`.

changelog.d/8562.misc

@@ -1 +0,0 @@
-Add type annotations for `LruCache`.

changelog.d/8563.misc

@@ -1 +0,0 @@
-Replace `DeferredCache` with the lighter-weight `LruCache` where possible.

changelog.d/8564.feature

@@ -1 +0,0 @@
-Support modifying event content in `ThirdPartyRules` modules.

changelog.d/8566.misc

@@ -1 +0,0 @@
-Add virtualenv-generated folders to `.gitignore`.

changelog.d/8567.bugfix

@@ -1 +0,0 @@
-Fix increase in the number of `There was no active span...` errors logged when using OpenTracing.

changelog.d/8568.misc

@@ -1 +0,0 @@
-Add `get_immediate` method to `DeferredCache`.

changelog.d/8569.misc

@@ -1 +0,0 @@
-Fix mypy not properly checking across the codebase, additionally, fix a typing assertion error in `handlers/auth.py`.

changelog.d/8571.misc

@@ -1 +0,0 @@
-Fix `synmark` benchmark runner.

changelog.d/8572.misc

@@ -1 +0,0 @@
-Modify `DeferredCache.get()` to return `Deferred`s instead of `ObservableDeferred`s.

changelog.d/8577.misc

@@ -1 +0,0 @@
-Adjust a protocol-type definition to fit `sqlite3` assertions.

changelog.d/8578.misc

@@ -1 +0,0 @@
-Support macOS on the `synmark` benchmark runner.

changelog.d/8583.misc

@@ -1 +0,0 @@
-Update `mypy` static type checker to 0.790.

changelog.d/8585.bugfix

@@ -1 +0,0 @@
-Fix a bug that prevented errors encountered during execution of the `synapse_port_db` from being correctly printed.

changelog.d/8587.misc

@@ -1 +0,0 @@
-Re-organize the structured logging code to separate the TCP transport handling from the JSON formatting.

changelog.d/8589.removal

@@ -1 +0,0 @@
-Drop unused `device_max_stream_id` table.

changelog.d/8590.misc

@@ -1 +0,0 @@
-Implement [MSC2409](https://github.com/matrix-org/matrix-doc/pull/2409) to send typing, read receipts, and presence events to appservices.

changelog.d/8591.misc

@@ -1 +0,0 @@
- Move metric registration code down into `LruCache`.

changelog.d/8592.misc

@@ -1 +0,0 @@
-Remove extraneous unittest logging decorators from unit tests.

changelog.d/8599.feature

@@ -1 +0,0 @@
-Allow running background tasks in a separate worker process.

changelog.d/8600.misc

@@ -1 +0,0 @@
-Update `mypy` static type checker to 0.790.

changelog.d/8606.feature

@@ -1 +0,0 @@
-Limit appservice transactions to 100 persistent and 100 ephemeral events.

changelog.d/8609.misc

@@ -1 +0,0 @@
-Add type hints to profile and base handler.

changelog.d/8906.misc

@@ -0,0 +1 @@
+Fix multiarch docker image builds.

contrib/grafana/README.md

@@ -3,4 +3,4 @@
 0. Set up Prometheus and Grafana. Out of scope for this readme. Useful documentation about using Grafana with Prometheus: http://docs.grafana.org/features/datasources/prometheus/
 1. Have your Prometheus scrape your Synapse. https://github.com/matrix-org/synapse/blob/master/docs/metrics-howto.md
 2. Import dashboard into Grafana. Download `synapse.json`. Import it to Grafana and select the correct Prometheus datasource. http://docs.grafana.org/reference/export_import/
-3. Set up additional recording rules
+3. Set up required recording rules. https://github.com/matrix-org/synapse/tree/master/contrib/prometheus

debian/changelog

@@ -1,3 +1,27 @@
+matrix-synapse-py3 (1.23.1) stable; urgency=medium
+
+  * New synapse release 1.23.1.
+
+ -- Synapse Packaging team <packages@matrix.org>  Wed, 09 Dec 2020 10:40:39 +0000
+
+matrix-synapse-py3 (1.23.0) stable; urgency=medium
+
+  * New synapse release 1.23.0.
+
+ -- Synapse Packaging team <packages@matrix.org>  Wed, 18 Nov 2020 11:41:28 +0000
+
+matrix-synapse-py3 (1.22.1) stable; urgency=medium
+
+  * New synapse release 1.22.1.
+
+ -- Synapse Packaging team <packages@matrix.org>  Fri, 30 Oct 2020 15:25:37 +0000
+
+matrix-synapse-py3 (1.22.0) stable; urgency=medium
+
+  * New synapse release 1.22.0.
+
+ -- Synapse Packaging team <packages@matrix.org>  Tue, 27 Oct 2020 12:07:12 +0000
+
 matrix-synapse-py3 (1.21.2) stable; urgency=medium
 
   [ Synapse Packaging team ]

docker/Dockerfile

@@ -11,7 +11,7 @@
 #    docker build -f docker/Dockerfile --build-arg PYTHON_VERSION=3.6 .
 #
 
-ARG PYTHON_VERSION=3.7
+ARG PYTHON_VERSION=3.8
 
 ###
 ### Stage 0: builder
@@ -36,7 +36,8 @@ RUN pip install --prefix="/install" --no-warn-script-location \
         frozendict \
         jaeger-client \
         opentracing \
-        prometheus-client \
+        # Match the version constraints of Synapse
+        "prometheus_client>=0.4.0,<0.9.0" \
         psycopg2 \
         pycparser \
         pyrsistent \

docs/admin_api/event_reports.md

@@ -0,0 +1,172 @@
+# Show reported events
+
+This API returns information about reported events.
+
+The api is:
+```
+GET /_synapse/admin/v1/event_reports?from=0&limit=10
+```
+To use it, you will need to authenticate by providing an `access_token` for a
+server admin: see [README.rst](README.rst).
+
+It returns a JSON body like the following:
+
+```json
+{
+    "event_reports": [
+        {
+            "event_id": "$bNUFCwGzWca1meCGkjp-zwslF-GfVcXukvRLI1_FaVY",
+            "id": 2,
+            "reason": "foo",
+            "score": -100,
+            "received_ts": 1570897107409,
+            "canonical_alias": "#alias1:matrix.org",
+            "room_id": "!ERAgBpSOcCCuTJqQPk:matrix.org",
+            "name": "Matrix HQ",
+            "sender": "@foobar:matrix.org",
+            "user_id": "@foo:matrix.org"
+        },
+        {
+            "event_id": "$3IcdZsDaN_En-S1DF4EMCy3v4gNRKeOJs8W5qTOKj4I",
+            "id": 3,
+            "reason": "bar",
+            "score": -100,
+            "received_ts": 1598889612059,
+            "canonical_alias": "#alias2:matrix.org",
+            "room_id": "!eGvUQuTCkHGVwNMOjv:matrix.org",
+            "name": "Your room name here",
+            "sender": "@foobar:matrix.org",
+            "user_id": "@bar:matrix.org"
+        }
+    ],
+    "next_token": 2,
+    "total": 4
+}
+```
+
+To paginate, check for `next_token` and if present, call the endpoint again with `from`
+set to the value of `next_token`. This will return a new page.
+
+If the endpoint does not return a `next_token` then there are no more reports to
+paginate through.
+
+**URL parameters:**
+
+* `limit`: integer - Is optional but is used for pagination, denoting the maximum number
+  of items to return in this call. Defaults to `100`.
+* `from`: integer - Is optional but used for pagination, denoting the offset in the
+  returned results. This should be treated as an opaque value and not explicitly set to
+  anything other than the return value of `next_token` from a previous call. Defaults to `0`.
+* `dir`: string - Direction of event report order. Whether to fetch the most recent
+  first (`b`) or the oldest first (`f`). Defaults to `b`.
+* `user_id`: string - Is optional and filters to only return users with user IDs that
+  contain this value. This is the user who reported the event and wrote the reason.
+* `room_id`: string - Is optional and filters to only return rooms with room IDs that
+  contain this value.
+
+**Response**
+
+The following fields are returned in the JSON response body:
+
+* `id`: integer - ID of event report.
+* `received_ts`: integer - The timestamp (in milliseconds since the unix epoch) when this
+  report was sent.
+* `room_id`: string - The ID of the room in which the event being reported is located.
+* `name`: string - The name of the room.
+* `event_id`: string - The ID of the reported event.
+* `user_id`: string - This is the user who reported the event and wrote the reason.
+* `reason`: string - Comment made by the `user_id` in this report. May be blank.
+* `score`: integer - Content is reported based upon a negative score, where -100 is
+  "most offensive" and 0 is "inoffensive".
+* `sender`: string - This is the ID of the user who sent the original message/event that
+  was reported.
+* `canonical_alias`: string - The canonical alias of the room. `null` if the room does not
+  have a canonical alias set.
+* `next_token`: integer - Indication for pagination. See above.
+* `total`: integer - Total number of event reports related to the query
+  (`user_id` and `room_id`).
+
+# Show details of a specific event report
+
+This API returns information about a specific event report.
+
+The api is:
+```
+GET /_synapse/admin/v1/event_reports/<report_id>
+```
+To use it, you will need to authenticate by providing an `access_token` for a
+server admin: see [README.rst](README.rst).
+
+It returns a JSON body like the following:
+
+```jsonc
+{
+    "event_id": "$bNUFCwGzWca1meCGkjp-zwslF-GfVcXukvRLI1_FaVY",
+    "event_json": {
+        "auth_events": [
+            "$YK4arsKKcc0LRoe700pS8DSjOvUT4NDv0HfInlMFw2M",
+            "$oggsNXxzPFRE3y53SUNd7nsj69-QzKv03a1RucHu-ws"
+        ],
+        "content": {
+            "body": "matrix.org: This Week in Matrix",
+            "format": "org.matrix.custom.html",
+            "formatted_body": "<strong>matrix.org</strong>:<br><a href=\"https://matrix.org/blog/\"><strong>This Week in Matrix</strong></a>",
+            "msgtype": "m.notice"
+        },
+        "depth": 546,
+        "hashes": {
+            "sha256": "xK1//xnmvHJIOvbgXlkI8eEqdvoMmihVDJ9J4SNlsAw"
+        },
+        "origin": "matrix.org",
+        "origin_server_ts": 1592291711430,
+        "prev_events": [
+            "$YK4arsKKcc0LRoe700pS8DSjOvUT4NDv0HfInlMFw2M"
+        ],
+        "prev_state": [],
+        "room_id": "!ERAgBpSOcCCuTJqQPk:matrix.org",
+        "sender": "@foobar:matrix.org",
+        "signatures": {
+            "matrix.org": {
+                "ed25519:a_JaEG": "cs+OUKW/iHx5pEidbWxh0UiNNHwe46Ai9LwNz+Ah16aWDNszVIe2gaAcVZfvNsBhakQTew51tlKmL2kspXk/Dg"
+            }
+        },
+        "type": "m.room.message",
+        "unsigned": {
+            "age_ts": 1592291711430,
+        }
+    },
+    "id": <report_id>,
+    "reason": "foo",
+    "score": -100,
+    "received_ts": 1570897107409,
+    "canonical_alias": "#alias1:matrix.org",
+    "room_id": "!ERAgBpSOcCCuTJqQPk:matrix.org",
+    "name": "Matrix HQ",
+    "sender": "@foobar:matrix.org",
+    "user_id": "@foo:matrix.org"
+}
+```
+
+**URL parameters:**
+
+* `report_id`: string - The ID of the event report.
+
+**Response**
+
+The following fields are returned in the JSON response body:
+
+* `id`: integer - ID of event report.
+* `received_ts`: integer - The timestamp (in milliseconds since the unix epoch) when this
+  report was sent.
+* `room_id`: string - The ID of the room in which the event being reported is located.
+* `name`: string - The name of the room.
+* `event_id`: string - The ID of the reported event.
+* `user_id`: string - This is the user who reported the event and wrote the reason.
+* `reason`: string - Comment made by the `user_id` in this report. May be blank.
+* `score`: integer - Content is reported based upon a negative score, where -100 is
+  "most offensive" and 0 is "inoffensive".
+* `sender`: string - This is the ID of the user who sent the original message/event that
+  was reported.
+* `canonical_alias`: string - The canonical alias of the room. `null` if the room does not
+  have a canonical alias set.
+* `event_json`: object - Details of the original event that was reported.

docs/admin_api/event_reports.rst

@@ -1,129 +0,0 @@
-Show reported events
-====================
-
-This API returns information about reported events.
-
-The api is::
-
-    GET /_synapse/admin/v1/event_reports?from=0&limit=10
-
-To use it, you will need to authenticate by providing an ``access_token`` for a
-server admin: see `README.rst <README.rst>`_.
-
-It returns a JSON body like the following:
-
-.. code:: jsonc
-
-    {
-        "event_reports": [
-            {
-                "content": {
-                    "reason": "foo",
-                    "score": -100
-                },
-                "event_id": "$bNUFCwGzWca1meCGkjp-zwslF-GfVcXukvRLI1_FaVY",
-                "event_json": {
-                    "auth_events": [
-                        "$YK4arsKKcc0LRoe700pS8DSjOvUT4NDv0HfInlMFw2M",
-                        "$oggsNXxzPFRE3y53SUNd7nsj69-QzKv03a1RucHu-ws"
-                    ],
-                    "content": {
-                        "body": "matrix.org: This Week in Matrix",
-                        "format": "org.matrix.custom.html",
-                        "formatted_body": "<strong>matrix.org</strong>:<br><a href=\"https://matrix.org/blog/\"><strong>This Week in Matrix</strong></a>",
-                        "msgtype": "m.notice"
-                    },
-                    "depth": 546,
-                    "hashes": {
-                        "sha256": "xK1//xnmvHJIOvbgXlkI8eEqdvoMmihVDJ9J4SNlsAw"
-                    },
-                    "origin": "matrix.org",
-                    "origin_server_ts": 1592291711430,
-                    "prev_events": [
-                        "$YK4arsKKcc0LRoe700pS8DSjOvUT4NDv0HfInlMFw2M"
-                    ],
-                    "prev_state": [],
-                    "room_id": "!ERAgBpSOcCCuTJqQPk:matrix.org",
-                    "sender": "@foobar:matrix.org",
-                    "signatures": {
-                        "matrix.org": {
-                            "ed25519:a_JaEG": "cs+OUKW/iHx5pEidbWxh0UiNNHwe46Ai9LwNz+Ah16aWDNszVIe2gaAcVZfvNsBhakQTew51tlKmL2kspXk/Dg"
-                        }
-                    },
-                    "type": "m.room.message",
-                    "unsigned": {
-                        "age_ts": 1592291711430,
-                    }
-                },
-                "id": 2,
-                "reason": "foo",
-                "received_ts": 1570897107409,
-                "room_alias": "#alias1:matrix.org",
-                "room_id": "!ERAgBpSOcCCuTJqQPk:matrix.org",
-                "sender": "@foobar:matrix.org",
-                "user_id": "@foo:matrix.org"
-            },
-            {
-                "content": {
-                    "reason": "bar",
-                    "score": -100
-                },
-                "event_id": "$3IcdZsDaN_En-S1DF4EMCy3v4gNRKeOJs8W5qTOKj4I",
-                "event_json": {
-                    // hidden items
-                    // see above
-                },
-                "id": 3,
-                "reason": "bar",
-                "received_ts": 1598889612059,
-                "room_alias": "#alias2:matrix.org",
-                "room_id": "!eGvUQuTCkHGVwNMOjv:matrix.org",
-                "sender": "@foobar:matrix.org",
-                "user_id": "@bar:matrix.org"
-            }
-        ],
-        "next_token": 2,
-        "total": 4
-    }
-
-To paginate, check for ``next_token`` and if present, call the endpoint again
-with ``from`` set to the value of ``next_token``. This will return a new page.
-
-If the endpoint does not return a ``next_token`` then there are no more
-reports to paginate through.
-
-**URL parameters:**
-
-- ``limit``: integer - Is optional but is used for pagination,
-  denoting the maximum number of items to return in this call. Defaults to ``100``.
-- ``from``: integer - Is optional but used for pagination,
-  denoting the offset in the returned results. This should be treated as an opaque value and
-  not explicitly set to anything other than the return value of ``next_token`` from a previous call.
-  Defaults to ``0``.
-- ``dir``: string - Direction of event report order. Whether to fetch the most recent first (``b``) or the
-  oldest first (``f``). Defaults to ``b``.
-- ``user_id``: string - Is optional and filters to only return users with user IDs that contain this value.
-  This is the user who reported the event and wrote the reason.
-- ``room_id``: string - Is optional and filters to only return rooms with room IDs that contain this value.
-
-**Response**
-
-The following fields are returned in the JSON response body:
-
-- ``id``: integer - ID of event report.
-- ``received_ts``: integer - The timestamp (in milliseconds since the unix epoch) when this report was sent.
-- ``room_id``: string - The ID of the room in which the event being reported is located.
-- ``event_id``: string - The ID of the reported event.
-- ``user_id``: string - This is the user who reported the event and wrote the reason.
-- ``reason``: string - Comment made by the ``user_id`` in this report. May be blank.
-- ``content``: object - Content of reported event.
-
-  - ``reason``: string - Comment made by the ``user_id`` in this report. May be blank.
-  - ``score``: integer - Content is reported based upon a negative score, where -100 is "most offensive" and 0 is "inoffensive".
-
-- ``sender``: string - This is the ID of the user who sent the original message/event that was reported.
-- ``room_alias``: string - The alias of the room. ``null`` if the room does not have a canonical alias set.
-- ``event_json``: object - Details of the original event that was reported.
-- ``next_token``: integer - Indication for pagination. See above.
-- ``total``: integer - Total number of event reports related to the query (``user_id`` and ``room_id``).
-

docs/admin_api/media_admin_api.md

@@ -100,3 +100,82 @@ Response:
   "num_quarantined": 10  # The number of media items successfully quarantined
 }
 ```
+
+# Delete local media
+This API deletes the *local* media from the disk of your own server.
+This includes any local thumbnails and copies of media downloaded from
+remote homeservers.
+This API will not affect media that has been uploaded to external
+media repositories (e.g https://github.com/turt2live/matrix-media-repo/).
+See also [purge_remote_media.rst](purge_remote_media.rst).
+
+## Delete a specific local media
+Delete a specific `media_id`.
+
+Request:
+
+```
+DELETE /_synapse/admin/v1/media/<server_name>/<media_id>
+
+{}
+```
+
+URL Parameters
+
+* `server_name`: string - The name of your local server (e.g `matrix.org`)
+* `media_id`: string - The ID of the media (e.g `abcdefghijklmnopqrstuvwx`)
+
+Response:
+
+```json
+    {
+       "deleted_media": [
+          "abcdefghijklmnopqrstuvwx"
+       ],
+       "total": 1
+    }
+```
+
+The following fields are returned in the JSON response body:
+
+* `deleted_media`: an array of strings - List of deleted `media_id`
+* `total`: integer - Total number of deleted `media_id`
+
+## Delete local media by date or size
+
+Request:
+
+```
+POST /_synapse/admin/v1/media/<server_name>/delete?before_ts=<before_ts>
+
+{}
+```
+
+URL Parameters
+
+* `server_name`: string - The name of your local server (e.g `matrix.org`).
+* `before_ts`: string representing a positive integer - Unix timestamp in ms.
+Files that were last used before this timestamp will be deleted. It is the timestamp of
+last access and not the timestamp creation. 
+* `size_gt`: Optional - string representing a positive integer - Size of the media in bytes.
+Files that are larger will be deleted. Defaults to `0`.
+* `keep_profiles`: Optional - string representing a boolean - Switch to also delete files
+that are still used in image data (e.g user profile, room avatar).
+If `false` these files will be deleted. Defaults to `true`.
+
+Response:
+
+```json
+    {
+       "deleted_media": [
+          "abcdefghijklmnopqrstuvwx",
+          "abcdefghijklmnopqrstuvwz"
+       ],
+       "total": 2
+    }
+```
+
+The following fields are returned in the JSON response body:
+
+* `deleted_media`: an array of strings - List of deleted `media_id`
+* `total`: integer - Total number of deleted `media_id`

docs/admin_api/register_api.rst

@@ -18,7 +18,8 @@ To fetch the nonce, you need to request one from the API::
 
 Once you have the nonce, you can make a ``POST`` to the same URL with a JSON
 body containing the nonce, username, password, whether they are an admin
-(optional, False by default), and a HMAC digest of the content.
+(optional, False by default), and a HMAC digest of the content. Also you can
+set the displayname (optional, ``username`` by default).
 
 As an example::
 
@@ -26,6 +27,7 @@ As an example::
   > {
      "nonce": "thisisanonce",
      "username": "pepper_roni",
+     "displayname": "Pepper Roni",
      "password": "pizza",
      "admin": true,
      "mac": "mac_digest_here"