1
0
Commit Graph

2382 Commits

Author SHA1 Message Date
Andrew Morgan
e15cca700b Merge commit 'f76194a02' into anoa/dinsic_release_1_21_x
* commit 'f76194a02':
  1.21.0
  Update change log
  1.21.0rc3
  Reduce serialization errors in MultiWriterIdGen (#8456)
  Add Ubuntu 20.10 (Groovy Gorilla) to build scripts. (#8475)
  move #8444 to 'feature'
  linkify changelog
2020-10-21 15:12:45 +01:00
Andrew Morgan
854764073c Merge commit 'cc40a59b4' into anoa/dinsic_release_1_21_x
* commit 'cc40a59b4':
  1.21.0
  Add prometheus metrics to track federation delays (#8430)
  Make token serializing/deserializing async (#8427)
  Allow additional SSO properties to be passed to the client (#8413)
  changelog
  Add an improved "forward extremities" metric
  Rewrite BucketCollector
  Fix _exposition.py to stop stripping samples
  Drop support for ancient prometheus_client (#8426)
  Various clean ups to room stream tokens. (#8423)
  changelog
  Report state res metrics to Prometheus and log
  Move Measure calls into `resolve_events_with_store`
  Expose a `get_resource_usage` method in `Measure`
  Move `resolve_events_with_store` into StateResolutionHandler
2020-10-21 14:53:23 +01:00
Andrew Morgan
e8ed9a6016 Merge commit '8238b55e0' into anoa/dinsic_release_1_21_x
* commit '8238b55e0':
  Update description of server_name config option (#8415)
  Discard an empty upload_name before persisting an uploaded file (#7905)
  Don't table scan events on worker startup (#8419)
  Mypy fixes for `synapse.handlers.federation` (#8422)
2020-10-21 14:53:16 +01:00
Andrew Morgan
ab26b5894a Merge commit 'f43c66d23' into anoa/dinsic_release_1_21_x
* commit 'f43c66d23':
  Add support for running Complement against the local checkout (#8317)
  Filter out appservices from mau count (#8404)
  Only assert valid next_link params when provided (#8417)
  Add metrics to track success/otherwise of replication requests (#8406)
  Fix handling of connection timeouts in outgoing http requests (#8400)
  Changelog
  Don't check whether a 3pid is allowed to register during password reset
  Add checks for postgres sequence consistency (#8402)
  Create a mechanism for marking tests "logcontext clean" (#8399)
  Add `ui_auth_sessions_ips` table to `synapse_port_db` ignore list (#8410)
  A pair of tiny cleanups in the federation request code. (#8401)
  typo
2020-10-21 14:53:14 +01:00
Andrew Morgan
5b0b103521 Merge commit '31acc5c30' into anoa/dinsic_release_1_21_x
* commit '31acc5c30':
  Escape the error description on the sso_error template. (#8405)
  Fix occasional "Re-starting finished log context" from keyring (#8398)
  Allow existing users to login via OpenID Connect. (#8345)
  Fix schema delta for servers that have not backfilled (#8396)
  Fix MultiWriteIdGenerator's handling of restarts. (#8374)
  s/URLs/variables in changelog
  s/accidentally/incorrectly in changelog
  Update changelog wording
  Add type annotations to SimpleHttpClient (#8372)
  Add new sequences to port DB script (#8387)
  Add EventStreamPosition type (#8388)
  Mark the shadow_banned column as boolean in synapse_port_db. (#8386)
2020-10-21 14:51:58 +01:00
Andrew Morgan
3729b15a64 Merge commit '2983049a7' into anoa/dinsic_release_1_21_x
* commit '2983049a7':
  Factor out `_send_dummy_event_for_room` (#8370)
  Improve logging of state resolution (#8371)
  Fix bug which caused failure on join with malformed membership events (#8385)
  Use `async with` for ID gens (#8383)
  Don't push if an user account has expired (#8353)
  Do not check lint/test dependencies at runtime. (#8377)
  Add note to reverse_proxy.md about disabling Apache's mod_security2 (#8375)
  Changelog
2020-10-21 14:15:38 +01:00
Andrew Morgan
e529fa7574 Merge commit '4325be1a5' into anoa/dinsic_release_1_21_x
* commit '4325be1a5':
  Fix missing null character check on guest_access room state
  Fixed a bug with reactivating users with the admin API (#8362)
  Admin API for reported events (#8217)
  Fix wording of deprecation notice in changelog
  Deprecation warning for synapse admin api being accessible under /_matrix
  Create function to check for long names in devices (#8364)
  Add a comment re #1691
  Fix a bad merge from release-v1.20.0. (#8354)
  Admin API for querying rooms where a user is a member (#8306)
  Catch-up after Federation Outage (bonus): Catch-up on Synapse Startup (#8322)
  Simplify super() calls to Python 3 syntax. (#8344)
  Allow appservice users to /login (#8320)
  Update test logging to be able to accept braces (#8335)
  Move lint dependencies to extras_require (#8330)
2020-10-21 14:14:36 +01:00
Andrew Morgan
7176832124 Merge commit '837293c31' into anoa/dinsic_release_1_21_x
* commit '837293c31':
  Remove obsolete __future__ imports (#8337)
  Use admin_patterns for all admin APIs. (#8331)
  Fix a potential bug of UnboundLocalError (#8329)
  Switch metaclass initialization to python 3-compatible syntax (#8326)
  Catch-up after Federation Outage (split, 4): catch-up loop (#8272)
  Use slots in attrs classes where possible (#8296)
  Fix typos in comments.
  Add the topic and avatar to the room details admin API (#8305)
  Improve SAML error messages (#8248)
  Add experimental support for sharding event persister. Again. (#8294)
  Make `StreamToken.room_key` be a `RoomStreamToken` instance. (#8281)
  Use TLSv1.2 for fake servers in tests (#8208)
  Add /_synapse/client to the reverse proxy docs (#8227)
  Clean up `Notifier.on_new_room_event` code path (#8288)
2020-10-20 19:51:28 +01:00
Andrew Morgan
e13218756c Merge commit '2832ef5bb' into anoa/dinsic_release_1_21_x
* commit '2832ef5bb':
  1.20.0rc3
  Fix the exception that is raised when invalid JSON is encountered. (#8291)
  Remove shared rooms info from upgrade/workers doc as it's still experimental (#8290)
2020-10-20 19:28:12 +01:00
Andrew Morgan
8ee3f063a8 Merge commit 'a3a90ee03' into anoa/dinsic_release_1_21_x
* commit 'a3a90ee03':
  Show a confirmation page during user password reset (#8004)
  Do not error when thumbnailing invalid files (#8236)
  Remove some unused distributor signals (#8216)
  Fixup pusher pool notifications (#8287)
  Revert "Fixup pusher pool notifications"
  Fixup pusher pool notifications
2020-10-20 19:27:41 +01:00
Andrew Morgan
45079599ab Merge commit '453dfe210' into anoa/dinsic_release_1_21_x
* commit '453dfe210':
  blacklist MSC2753 sytests until it's implemented in synapse (#8285)
  Don't remember `enabled` of deleted push rules and properly return 404 for missing push rules in `.../actions` and `.../enabled` (#7796)
2020-10-20 19:15:10 +01:00
Andrew Morgan
82c379c20b Merge commit 'e45b83411' into anoa/dinsic_release_1_21_x
* commit 'e45b83411':
  Add types to async_helpers (#8260)
  Fix mypy error on develop (#8282)
  Include method in thumbnail media name (#7124)
  Add types to StreamToken and RoomStreamToken (#8279)
  Add a config option for validating 'next_link' parameters against a domain whitelist (#8275)
  Clean up types for PaginationConfig (#8250)
  Use the right constructor for log records (#8278)
  Fix `MultiWriterIdGenerator.current_position`. (#8257)
2020-10-20 19:12:51 +01:00
Andrew Morgan
061b9895eb Merge commit 'cca03dbec' into anoa/dinsic_release_1_21_x
* commit 'cca03dbec':
  fix typo
  s/fixes/fix/
  Directly import json from the standard library. (#8259)
  Allow for make_awaitable's return value to be re-used. (#8261)
  Rename 'populate_stats_process_rooms_2' background job back to 'populate_stats_process_rooms' again (#8243)
2020-10-20 18:29:12 +01:00
Andrew Morgan
4213fe1613 Merge commit '7586fdf1e' into anoa/dinsic_release_1_21_x
* commit '7586fdf1e':
  Bump canonicaljson to version 1.4.0 (#8262)
  Run database updates in a transaction (#8265)
  Add tests for `last_successful_stream_ordering` (#8258)
2020-10-20 18:10:26 +01:00
Andrew Morgan
47d53b32e3 Merge commit '17fa4c7ca' into anoa/dinsic_release_1_21_x
* commit '17fa4c7ca':
  Catch up after Federation Outage (split, 2): Track last successful stream ordering after transmission (#8247)
  Catch-up after Federation Outage (split, 1) (#8230)
  Fix type signature in simple_select_one_onecol and friends (#8241)
  Stop sub-classing object (#8249)
2020-10-20 18:09:59 +01:00
Andrew Morgan
d7fd0deaf1 Merge commit '112266eaf' into anoa/dinsic_release_1_21_x
* commit '112266eaf':
  Add StreamStore to mypy (#8232)
  Re-implement unread counts (again) (#8059)
2020-10-20 17:51:52 +01:00
Andrew Morgan
56722c69c5 Merge commit '0d4f614fd' into anoa/dinsic_release_1_21_x
* commit '0d4f614fd':
  Refactor `_get_e2e_device_keys_for_federation_query_txn` (#8225)
  Add experimental support for sharding event persister. (#8170)
  Add /user/{user_id}/shared_rooms/ api (#7785)
  Do not try to store invalid data in the stats table (#8226)
  Convert the main methods run by the reactor to async. (#8213)
2020-10-20 17:51:35 +01:00
Andrew Morgan
059e0fdedf Merge commit '5bf8e5f55' into anoa/dinsic_release_1_21_x
* commit '5bf8e5f55':
  Convert the well known resolver to async (#8214)
  Convert additional databases to async/await part 2 (#8200)
  Make MultiWriterIDGenerator work for streams that use negative stream IDs (#8203)
  Do not install setuptools 50.0. (#8212)
  Move and rename `get_devices_with_keys_by_user` (#8204)
  Rename `get_e2e_device_keys` to better reflect its purpose (#8205)
  Add a comment about _LimitedHostnameResolver
2020-10-20 17:42:29 +01:00
Andrew Morgan
d49dd2f643 Merge commit 'd2ac767de' into anoa/dinsic_release_1_21_x
* commit 'd2ac767de':
  Convert ReadWriteLock to async/await. (#8202)
  Fix incorrect return signature
  Fix `wait_for_stream_position` for multiple waiters. (#8196)
2020-10-20 17:42:24 +01:00
Andrew Morgan
b7672ff59e Merge commit 'aec708517' into anoa/dinsic_release_1_21_x
* commit 'aec708517':
  Convert state and stream stores and related code to async (#8194)
  Ensure that the OpenID Connect remote ID is a string. (#8190)
2020-10-20 17:42:15 +01:00
Andrew Morgan
ff8a9a6a28 Merge commit 'e00816ad9' into anoa/dinsic_release_1_21_x
* commit 'e00816ad9':
  Do not yield on awaitables in tests. (#8193)
2020-10-20 17:42:05 +01:00
Andrew Morgan
ed944cb079 Merge commit 'b49a5b930' into anoa/dinsic_release_1_21_x
* commit 'b49a5b930':
  Convert stats and related calls to async/await (#8192)
2020-10-20 17:42:00 +01:00
Andrew Morgan
6961a007dd Merge commit 'b71d4a094' into anoa/dinsic_release_1_21_x
* commit 'b71d4a094':
  Convert simple_delete to async/await. (#8191)
2020-10-20 17:41:55 +01:00
Andrew Morgan
ec50c99609 Merge commit '9b7ac03af' into anoa/dinsic_release_1_21_x
* commit '9b7ac03af':
  Convert calls of async database methods to async (#8166)
  simple_search_list_txn should return None, not 0. (#8187)
  Fix missing _add_persisted_position (#8179)
2020-10-20 17:41:11 +01:00
Andrew Morgan
118f41dba7 Merge commit '30426c706' into anoa/dinsic_release_1_21_x
* commit '30426c706':
  Convert additional database methods to async (select list, search, insert_many, delete_*) (#8168)
2020-10-20 17:33:58 +01:00
Andrew Morgan
a89a160c17 Merge commit '4a739c73b' into anoa/dinsic_release_1_21_x
* commit '4a739c73b':
  Convert simple_update* and simple_select* to async (#8173)
2020-10-20 17:33:47 +01:00
Andrew Morgan
df77da8b4d Merge commit 'a466b6797' into anoa/dinsic_release_1_21_x
* commit 'a466b6797':
  Reduce run-times of tests by advancing the reactor less (#7757)
  Update debian systemd service to use Type=notify (#8169)
  Remove remaining is_guest argument uses from get_room_data calls (#8181)
  Do not propagate typing notifications from shadow-banned users. (#8176)
  Remove unused parameter from, and add safeguard in, get_room_data (#8174)
  Add required Debian dependencies to allow docker builds on the arm platform (#8144)
  Allow running mypy directly. (#8175)
  Update the test federation client to handle streaming responses (#8130)
  Do not propagate profile changes of shadow-banned users into rooms. (#8157)
  Make SlavedIdTracker.advance have same interface as MultiWriterIDGenerator (#8171)
  Convert simple_select_one and simple_select_one_onecol to async (#8162)
2020-10-20 17:13:42 +01:00
Andrew Morgan
b7bb088b84 Merge commit '56efa9ec7' into anoa/dinsic_release_1_21_x
* commit '56efa9ec7': (22 commits)
  Fix rate limiting unit tests. (#8167)
  Add functions to `MultiWriterIdGen` used by events stream (#8164)
  Do not allow send_nonmember_event to be called with shadow-banned users. (#8158)
  Changelog fixes
  Make StreamIdGen `get_next` and `get_next_mult` async  (#8161)
  Wording fixes to 'name' user admin api filter (#8163)
  Fix missing double-backtick in RST document
  Search in columns 'name' and 'displayname' in the admin users endpoint (#7377)
  Add type hints for state. (#8140)
  Stop shadow-banned users from sending non-member events. (#8142)
  Allow capping a room's retention policy (#8104)
  Add healthcheck for default localhost 8008 port on /health endpoint. (#8147)
  Fix flaky shadow-ban tests. (#8152)
  Don't fail /submit_token requests on incorrect session ID if request_token_inhibit_3pid_errors is turned on (#7991)
  Do not apply ratelimiting on joins to appservices (#8139)
  Micro-optimisations to get_auth_chain_ids (#8132)
  Allow denying or shadow banning registrations via the spam checker (#8034)
  Stop shadow-banned users from sending invites. (#8095)
  Be more tolerant of membership events in unknown rooms (#8110)
  Improve the error code when trying to register using a name reserved for guests. (#8135)
  ...
2020-10-20 16:45:58 +01:00
Andrew Morgan
385ab7a5c8 Merge commit 'b79d69796' into anoa/dinsic_release_1_21_x
* commit 'b79d69796':
  1.19.1rc1
  Fix join ratelimiter breaking profile updates and idempotency (#8153)
2020-10-20 16:21:14 +01:00
Andrew Morgan
520810ab25 Merge commit '2df82ae45' into anoa/dinsic_release_1_21_x
* commit '2df82ae45':
  Do not apply ratelimiting on joins to appservices (#8139)
2020-10-20 15:11:13 +01:00
Andrew Morgan
687d30b2ed Merge commit 'c9c544cda' into anoa/dinsic_release_1_21_x
* commit 'c9c544cda':
  Remove `ChainedIdGenerator`. (#8123)
  Switch the JSON byte producer from a pull to a push producer. (#8116)
  Updated docs: Added note about missing 308 redirect support. (#8120)
  Be stricter about JSON that is accepted by Synapse (#8106)
  Convert runWithConnection to async. (#8121)
  Remove the unused inlineCallbacks code-paths in the caching code (#8119)
  Separate `get_current_token` into two. (#8113)
  Convert events worker database to async/await. (#8071)
  Add a link to the matrix-synapse-rest-password-provider. (#8111)
2020-10-19 19:12:39 +01:00
Andrew Morgan
1e9ec2afff Merge commit '3c01724b3' into anoa/dinsic_release_1_21_x
* commit '3c01724b3':
  Fix the return type of send_nonmember_events. (#8112)
  Remove : from allowed client_secret chars (#8101)
  Rename changelog from bugfix to misc.
  Iteratively encode JSON responses to avoid blocking the reactor. (#8013)
  Return the previous stream token if a non-member event is a duplicate. (#8093)
2020-10-19 18:35:07 +01:00
Andrew Morgan
25b077d747 Merge commit '050e20e7c' into anoa/dinsic_release_1_21_x
* commit '050e20e7c':
  Convert some of the general database methods to async (#8100)
2020-10-19 18:34:31 +01:00
Andrew Morgan
29506f5c50 Merge commit 'e04e465b4' into anoa/dinsic_release_1_21_x
* commit 'e04e465b4':
  Use the default templates when a custom template file cannot be found (#8037)
  Changelog changes
  Convert stream database to async/await. (#8074)
  Add a shadow-banned flag to users. (#8092)
  Convert pusher databases to async/await. (#8075)
  Convert receipts and events databases to async/await. (#8076)
2020-10-19 18:27:56 +01:00
Andrew Morgan
464db5f27d Merge commit 'dd8f28bd3' into anoa/dinsic_release_1_21_x
* commit 'dd8f28bd3':
  Fix unawaited coroutine error in tests. (#8072)
2020-10-19 18:24:39 +01:00
Andrew Morgan
1ab32bd9a7 Merge commit 'fbe930dad' into anoa/dinsic_release_1_21_x
* commit 'fbe930dad':
  Convert the roommember database to async/await. (#8070)
2020-10-19 18:24:35 +01:00
Andrew Morgan
78bcfa71db Merge commit '5ecc8b582' into anoa/dinsic_release_1_21_x
* commit '5ecc8b582':
  Convert devices database to async/await. (#8069)
2020-10-19 18:24:30 +01:00
Andrew Morgan
20bfb5390c Merge commit '5dd73d029' into anoa/dinsic_release_1_21_x
* commit '5dd73d029':
  Add type hints to handlers.message and events.builder (#8067)
2020-10-19 18:24:25 +01:00
Andrew Morgan
33d971bd03 Merge commit 'd68e10f30' into anoa/dinsic_release_1_21_x
* commit 'd68e10f30':
  Convert account data, device inbox, and censor events databases to async/await (#8063)
2020-10-19 18:24:11 +01:00
Andrew Morgan
d872d53c4d Merge commit 'a3a59bab7' into anoa/dinsic_release_1_21_x
* commit 'a3a59bab7':
  Convert appservice, group server, profile and more databases to async (#8066)
2020-10-19 18:23:58 +01:00
Andrew Morgan
13c1c20972 Merge commit '04faa0bfa' into anoa/dinsic_release_1_21_x
* commit '04faa0bfa':
  Convert tags and metrics databases to async/await (#8062)
2020-10-19 18:16:34 +01:00
Andrew Morgan
8857135bc6 Merge commit 'a0acdfa9e' into anoa/dinsic_release_1_21_x
* commit 'a0acdfa9e':
  Converts event_federation and registration databases to async/await (#8061)
2020-10-19 18:14:26 +01:00
Andrew Morgan
70c6f5065a Merge commit 'db131b6b2' into anoa/dinsic_release_1_21_x
* commit 'db131b6b2':
  Change the default log config to reduce disk I/O and storage (#8040)
  Implement login blocking based on SAML attributes (#8052)
  Add an assertion on prev_events in create_new_client_event (#8041)
  Typo
  Lint
  why mypy why
  Lint
  Incorporate review
  Incorporate review
  Fix PUT /pushrules to use the right rule IDs
  Back out the database hack and replace it with a temporary config setting
  Fix cache name
  Fix cache invalidation calls
  Lint
  Changelog
  Implement new experimental push rules with a database hack to enable them
2020-10-19 17:08:43 +01:00
Andrew Morgan
c5aaa80a96 Merge commit '7f837959e' into anoa/dinsic_release_1_21_x
* commit '7f837959e':
  Convert directory, e2e_room_keys, end_to_end_keys, monthly_active_users database to async (#8042)
  Convert additional database stores to async/await (#8045)
2020-10-19 17:06:53 +01:00
Andrew Morgan
9dd3b52f1b Merge commit '7620912d8' into anoa/dinsic_release_1_21_x
* commit '7620912d8':
  Add health check endpoint (#8048)
2020-10-19 17:06:32 +01:00
Andrew Morgan
d47db9e9e9 Merge commit '2ffd6783c' into anoa/dinsic_release_1_21_x
* commit '2ffd6783c':
  Revert #7736 (#8039)
2020-10-19 17:06:14 +01:00
Andrew Morgan
9e2cab71fe Merge commit 'fe6cfc80e' into anoa/dinsic_release_1_21_x
* commit 'fe6cfc80e':
  Convert some util functions to async (#8035)
2020-10-19 17:06:09 +01:00
Andrew Morgan
383a87e0e2 Merge commit 'd4a7829b1' into anoa/dinsic_release_1_21_x
* commit 'd4a7829b1':
  Convert synapse.api to async/await (#8031)
2020-10-19 14:59:45 +01:00
Andrew Morgan
9d09ab74bb Merge commit '66f24449d' into anoa/dinsic_release_1_21_x
* commit '66f24449d':
  Improve performance of the register endpoint (#8009)
2020-10-19 14:29:04 +01:00
Andrew Morgan
d174faacc7 Merge commit 'a7bdf98d0' into anoa/dinsic_release_1_21_x
* commit 'a7bdf98d0':
  Rename database classes to make some sense (#8033)
2020-10-16 18:21:05 +01:00