1
0
Commit Graph

2326 Commits

Author SHA1 Message Date
Andrew Morgan
47d53b32e3 Merge commit '17fa4c7ca' into anoa/dinsic_release_1_21_x
* commit '17fa4c7ca':
  Catch up after Federation Outage (split, 2): Track last successful stream ordering after transmission (#8247)
  Catch-up after Federation Outage (split, 1) (#8230)
  Fix type signature in simple_select_one_onecol and friends (#8241)
  Stop sub-classing object (#8249)
2020-10-20 18:09:59 +01:00
Andrew Morgan
d7fd0deaf1 Merge commit '112266eaf' into anoa/dinsic_release_1_21_x
* commit '112266eaf':
  Add StreamStore to mypy (#8232)
  Re-implement unread counts (again) (#8059)
2020-10-20 17:51:52 +01:00
Andrew Morgan
56722c69c5 Merge commit '0d4f614fd' into anoa/dinsic_release_1_21_x
* commit '0d4f614fd':
  Refactor `_get_e2e_device_keys_for_federation_query_txn` (#8225)
  Add experimental support for sharding event persister. (#8170)
  Add /user/{user_id}/shared_rooms/ api (#7785)
  Do not try to store invalid data in the stats table (#8226)
  Convert the main methods run by the reactor to async. (#8213)
2020-10-20 17:51:35 +01:00
Andrew Morgan
059e0fdedf Merge commit '5bf8e5f55' into anoa/dinsic_release_1_21_x
* commit '5bf8e5f55':
  Convert the well known resolver to async (#8214)
  Convert additional databases to async/await part 2 (#8200)
  Make MultiWriterIDGenerator work for streams that use negative stream IDs (#8203)
  Do not install setuptools 50.0. (#8212)
  Move and rename `get_devices_with_keys_by_user` (#8204)
  Rename `get_e2e_device_keys` to better reflect its purpose (#8205)
  Add a comment about _LimitedHostnameResolver
2020-10-20 17:42:29 +01:00
Andrew Morgan
d49dd2f643 Merge commit 'd2ac767de' into anoa/dinsic_release_1_21_x
* commit 'd2ac767de':
  Convert ReadWriteLock to async/await. (#8202)
  Fix incorrect return signature
  Fix `wait_for_stream_position` for multiple waiters. (#8196)
2020-10-20 17:42:24 +01:00
Andrew Morgan
b7672ff59e Merge commit 'aec708517' into anoa/dinsic_release_1_21_x
* commit 'aec708517':
  Convert state and stream stores and related code to async (#8194)
  Ensure that the OpenID Connect remote ID is a string. (#8190)
2020-10-20 17:42:15 +01:00
Andrew Morgan
ff8a9a6a28 Merge commit 'e00816ad9' into anoa/dinsic_release_1_21_x
* commit 'e00816ad9':
  Do not yield on awaitables in tests. (#8193)
2020-10-20 17:42:05 +01:00
Andrew Morgan
ed944cb079 Merge commit 'b49a5b930' into anoa/dinsic_release_1_21_x
* commit 'b49a5b930':
  Convert stats and related calls to async/await (#8192)
2020-10-20 17:42:00 +01:00
Andrew Morgan
6961a007dd Merge commit 'b71d4a094' into anoa/dinsic_release_1_21_x
* commit 'b71d4a094':
  Convert simple_delete to async/await. (#8191)
2020-10-20 17:41:55 +01:00
Andrew Morgan
ec50c99609 Merge commit '9b7ac03af' into anoa/dinsic_release_1_21_x
* commit '9b7ac03af':
  Convert calls of async database methods to async (#8166)
  simple_search_list_txn should return None, not 0. (#8187)
  Fix missing _add_persisted_position (#8179)
2020-10-20 17:41:11 +01:00
Andrew Morgan
118f41dba7 Merge commit '30426c706' into anoa/dinsic_release_1_21_x
* commit '30426c706':
  Convert additional database methods to async (select list, search, insert_many, delete_*) (#8168)
2020-10-20 17:33:58 +01:00
Andrew Morgan
a89a160c17 Merge commit '4a739c73b' into anoa/dinsic_release_1_21_x
* commit '4a739c73b':
  Convert simple_update* and simple_select* to async (#8173)
2020-10-20 17:33:47 +01:00
Andrew Morgan
df77da8b4d Merge commit 'a466b6797' into anoa/dinsic_release_1_21_x
* commit 'a466b6797':
  Reduce run-times of tests by advancing the reactor less (#7757)
  Update debian systemd service to use Type=notify (#8169)
  Remove remaining is_guest argument uses from get_room_data calls (#8181)
  Do not propagate typing notifications from shadow-banned users. (#8176)
  Remove unused parameter from, and add safeguard in, get_room_data (#8174)
  Add required Debian dependencies to allow docker builds on the arm platform (#8144)
  Allow running mypy directly. (#8175)
  Update the test federation client to handle streaming responses (#8130)
  Do not propagate profile changes of shadow-banned users into rooms. (#8157)
  Make SlavedIdTracker.advance have same interface as MultiWriterIDGenerator (#8171)
  Convert simple_select_one and simple_select_one_onecol to async (#8162)
2020-10-20 17:13:42 +01:00
Andrew Morgan
b7bb088b84 Merge commit '56efa9ec7' into anoa/dinsic_release_1_21_x
* commit '56efa9ec7': (22 commits)
  Fix rate limiting unit tests. (#8167)
  Add functions to `MultiWriterIdGen` used by events stream (#8164)
  Do not allow send_nonmember_event to be called with shadow-banned users. (#8158)
  Changelog fixes
  Make StreamIdGen `get_next` and `get_next_mult` async  (#8161)
  Wording fixes to 'name' user admin api filter (#8163)
  Fix missing double-backtick in RST document
  Search in columns 'name' and 'displayname' in the admin users endpoint (#7377)
  Add type hints for state. (#8140)
  Stop shadow-banned users from sending non-member events. (#8142)
  Allow capping a room's retention policy (#8104)
  Add healthcheck for default localhost 8008 port on /health endpoint. (#8147)
  Fix flaky shadow-ban tests. (#8152)
  Don't fail /submit_token requests on incorrect session ID if request_token_inhibit_3pid_errors is turned on (#7991)
  Do not apply ratelimiting on joins to appservices (#8139)
  Micro-optimisations to get_auth_chain_ids (#8132)
  Allow denying or shadow banning registrations via the spam checker (#8034)
  Stop shadow-banned users from sending invites. (#8095)
  Be more tolerant of membership events in unknown rooms (#8110)
  Improve the error code when trying to register using a name reserved for guests. (#8135)
  ...
2020-10-20 16:45:58 +01:00
Andrew Morgan
385ab7a5c8 Merge commit 'b79d69796' into anoa/dinsic_release_1_21_x
* commit 'b79d69796':
  1.19.1rc1
  Fix join ratelimiter breaking profile updates and idempotency (#8153)
2020-10-20 16:21:14 +01:00
Andrew Morgan
520810ab25 Merge commit '2df82ae45' into anoa/dinsic_release_1_21_x
* commit '2df82ae45':
  Do not apply ratelimiting on joins to appservices (#8139)
2020-10-20 15:11:13 +01:00
Andrew Morgan
687d30b2ed Merge commit 'c9c544cda' into anoa/dinsic_release_1_21_x
* commit 'c9c544cda':
  Remove `ChainedIdGenerator`. (#8123)
  Switch the JSON byte producer from a pull to a push producer. (#8116)
  Updated docs: Added note about missing 308 redirect support. (#8120)
  Be stricter about JSON that is accepted by Synapse (#8106)
  Convert runWithConnection to async. (#8121)
  Remove the unused inlineCallbacks code-paths in the caching code (#8119)
  Separate `get_current_token` into two. (#8113)
  Convert events worker database to async/await. (#8071)
  Add a link to the matrix-synapse-rest-password-provider. (#8111)
2020-10-19 19:12:39 +01:00
Andrew Morgan
1e9ec2afff Merge commit '3c01724b3' into anoa/dinsic_release_1_21_x
* commit '3c01724b3':
  Fix the return type of send_nonmember_events. (#8112)
  Remove : from allowed client_secret chars (#8101)
  Rename changelog from bugfix to misc.
  Iteratively encode JSON responses to avoid blocking the reactor. (#8013)
  Return the previous stream token if a non-member event is a duplicate. (#8093)
2020-10-19 18:35:07 +01:00
Andrew Morgan
25b077d747 Merge commit '050e20e7c' into anoa/dinsic_release_1_21_x
* commit '050e20e7c':
  Convert some of the general database methods to async (#8100)
2020-10-19 18:34:31 +01:00
Andrew Morgan
29506f5c50 Merge commit 'e04e465b4' into anoa/dinsic_release_1_21_x
* commit 'e04e465b4':
  Use the default templates when a custom template file cannot be found (#8037)
  Changelog changes
  Convert stream database to async/await. (#8074)
  Add a shadow-banned flag to users. (#8092)
  Convert pusher databases to async/await. (#8075)
  Convert receipts and events databases to async/await. (#8076)
2020-10-19 18:27:56 +01:00
Andrew Morgan
464db5f27d Merge commit 'dd8f28bd3' into anoa/dinsic_release_1_21_x
* commit 'dd8f28bd3':
  Fix unawaited coroutine error in tests. (#8072)
2020-10-19 18:24:39 +01:00
Andrew Morgan
1ab32bd9a7 Merge commit 'fbe930dad' into anoa/dinsic_release_1_21_x
* commit 'fbe930dad':
  Convert the roommember database to async/await. (#8070)
2020-10-19 18:24:35 +01:00
Andrew Morgan
78bcfa71db Merge commit '5ecc8b582' into anoa/dinsic_release_1_21_x
* commit '5ecc8b582':
  Convert devices database to async/await. (#8069)
2020-10-19 18:24:30 +01:00
Andrew Morgan
20bfb5390c Merge commit '5dd73d029' into anoa/dinsic_release_1_21_x
* commit '5dd73d029':
  Add type hints to handlers.message and events.builder (#8067)
2020-10-19 18:24:25 +01:00
Andrew Morgan
33d971bd03 Merge commit 'd68e10f30' into anoa/dinsic_release_1_21_x
* commit 'd68e10f30':
  Convert account data, device inbox, and censor events databases to async/await (#8063)
2020-10-19 18:24:11 +01:00
Andrew Morgan
d872d53c4d Merge commit 'a3a59bab7' into anoa/dinsic_release_1_21_x
* commit 'a3a59bab7':
  Convert appservice, group server, profile and more databases to async (#8066)
2020-10-19 18:23:58 +01:00
Andrew Morgan
13c1c20972 Merge commit '04faa0bfa' into anoa/dinsic_release_1_21_x
* commit '04faa0bfa':
  Convert tags and metrics databases to async/await (#8062)
2020-10-19 18:16:34 +01:00
Andrew Morgan
8857135bc6 Merge commit 'a0acdfa9e' into anoa/dinsic_release_1_21_x
* commit 'a0acdfa9e':
  Converts event_federation and registration databases to async/await (#8061)
2020-10-19 18:14:26 +01:00
Andrew Morgan
70c6f5065a Merge commit 'db131b6b2' into anoa/dinsic_release_1_21_x
* commit 'db131b6b2':
  Change the default log config to reduce disk I/O and storage (#8040)
  Implement login blocking based on SAML attributes (#8052)
  Add an assertion on prev_events in create_new_client_event (#8041)
  Typo
  Lint
  why mypy why
  Lint
  Incorporate review
  Incorporate review
  Fix PUT /pushrules to use the right rule IDs
  Back out the database hack and replace it with a temporary config setting
  Fix cache name
  Fix cache invalidation calls
  Lint
  Changelog
  Implement new experimental push rules with a database hack to enable them
2020-10-19 17:08:43 +01:00
Andrew Morgan
c5aaa80a96 Merge commit '7f837959e' into anoa/dinsic_release_1_21_x
* commit '7f837959e':
  Convert directory, e2e_room_keys, end_to_end_keys, monthly_active_users database to async (#8042)
  Convert additional database stores to async/await (#8045)
2020-10-19 17:06:53 +01:00
Andrew Morgan
9dd3b52f1b Merge commit '7620912d8' into anoa/dinsic_release_1_21_x
* commit '7620912d8':
  Add health check endpoint (#8048)
2020-10-19 17:06:32 +01:00
Andrew Morgan
d47db9e9e9 Merge commit '2ffd6783c' into anoa/dinsic_release_1_21_x
* commit '2ffd6783c':
  Revert #7736 (#8039)
2020-10-19 17:06:14 +01:00
Andrew Morgan
9e2cab71fe Merge commit 'fe6cfc80e' into anoa/dinsic_release_1_21_x
* commit 'fe6cfc80e':
  Convert some util functions to async (#8035)
2020-10-19 17:06:09 +01:00
Andrew Morgan
383a87e0e2 Merge commit 'd4a7829b1' into anoa/dinsic_release_1_21_x
* commit 'd4a7829b1':
  Convert synapse.api to async/await (#8031)
2020-10-19 14:59:45 +01:00
Andrew Morgan
9d09ab74bb Merge commit '66f24449d' into anoa/dinsic_release_1_21_x
* commit '66f24449d':
  Improve performance of the register endpoint (#8009)
2020-10-19 14:29:04 +01:00
Andrew Morgan
d174faacc7 Merge commit 'a7bdf98d0' into anoa/dinsic_release_1_21_x
* commit 'a7bdf98d0':
  Rename database classes to make some sense (#8033)
2020-10-16 18:21:05 +01:00
Andrew Morgan
04fef33237 Merge commit 'e19de43eb' into anoa/dinsic_release_1_21_x
* commit 'e19de43eb':
  Convert streams to async. (#8014)
2020-10-16 17:27:24 +01:00
Andrew Morgan
73381abc23 Merge commit '5d92a1428' into anoa/dinsic_release_1_21_x
* commit '5d92a1428':
  Prevent join->join membership transitions changing member count (#7977)
2020-10-16 17:21:08 +01:00
Andrew Morgan
3442b28a1b Merge commit '681250980' into anoa/dinsic_release_1_21_x
* commit '681250980':
  Implement handling of HTTP HEAD requests. (#7999)
2020-10-16 17:21:03 +01:00
Andrew Morgan
cb0ee5e1aa Merge commit '2a89ce8cd' into anoa/dinsic_release_1_21_x
* commit '2a89ce8cd':
  Convert the crypto module to async/await. (#8003)
2020-10-16 17:20:59 +01:00
Andrew Morgan
479fc9d9b5 Merge commit '394be6a0e' into anoa/dinsic_release_1_21_x
* commit '394be6a0e':
  Newsfile
  Add ratelimiting on joins
  Add docs for undoing room shutdowns (#7998)
2020-10-16 17:20:16 +01:00
Andrew Morgan
4ae35a3c7b Merge commit 'c978f6c45' into anoa/dinsic_release_1_21_x
* commit 'c978f6c45':
  Convert federation client to async/await. (#7975)
2020-10-16 17:08:04 +01:00
Andrew Morgan
6d02e38937 Merge commit '4cce8ef74' into anoa/dinsic_release_1_21_x
* commit '4cce8ef74':
  Convert appservice to async. (#7973)
2020-10-16 17:07:55 +01:00
Andrew Morgan
c078f9ee16 Merge commit 'b3a97d6da' into anoa/dinsic_release_1_21_x
* commit 'b3a97d6da':
  Convert some of the data store to async. (#7976)
2020-10-16 17:07:47 +01:00
Andrew Morgan
684991eff0 Merge commit '3950ae51e' into anoa/dinsic_release_1_21_x
* commit '3950ae51e':
  Ensure that remove_pusher is always async (#7981)
  Ensure the msg property of HttpResponseException is a string. (#7979)
  Remove from the event_relations table when purging historical events. (#7978)
  Add additional logging for SAML sessions. (#7971)
  Add MSC reference to changelog for #7736
  Re-implement unread counts (#7736)
  Various improvements to the docs (#7899)
  Convert storage layer to async/await. (#7963)
  Add an option to disable purge in delete room admin API (#7964)
  Move some log lines from default logger to sql/transaction loggers (#7952)
  Use the JSON module from the std library instead of simplejson. (#7936)
  Fix exit code for `check_line_terminators.sh` (#7970)
  Option to allow server admins to join complex rooms (#7902)
  Fix typo in metrics docs (#7966)
  Add script for finding files with unix line terminators (#7965)
  Convert the remaining media repo code to async / await. (#7947)
  Convert a synapse.events to async/await. (#7949)
  Convert groups and visibility code to async / await. (#7951)
  Convert push to async/await. (#7948)
2020-10-16 17:07:28 +01:00
Andrew Morgan
a2b8233a4e Remember mappings when we bind a 3pid using the internal sydent bind API (#66)
https://github.com/matrix-org/synapse-dinsic/pull/51 added an option that would automatically bind a user's threepid to a configured identity server after they had registered. Unfortunately, when you bind threepids, ideally you would store that mapping in the database so that later on you can remove those mappings when you deactivate an account.

We found that due the fact that we did not store these mappings, threepids were not unbound upon user account deactivation.

This PR fixes the issue by creating the mappings again, meaning they will again be removed upon account deactivation.
2020-10-14 11:18:29 +01:00
Andrew Morgan
722e1c016a "Freeze" a room when the last admin of that room leaves (#59)
If the last admin of a room departs, and thus the room no longer has any admins within it, we "freeze" the room. Freezing a room means that the power level required to do anything in the room (sending messages, inviting others etc) will require power level 100.

At the moment, an admin can come back and unfreeze the room manually. The plan is to eventually make unfreezing of the room automatic on admin rejoin, though that will be in a separate PR.

This *could* work in mainline, however if the admin who leaves is on a homeserver without this functionality, then the room isn't frozen. I imagine this would probably be pretty confusing to people. Part of this feature was allowing Synapse modules to send events, which has been implemented in mainline at  https://github.com/matrix-org/synapse/pull/8479, and cherry-picked to the `dinsic` fork in 62c7b10. The actual freezing logic has been implemented here in the RoomAccessRules module.
2020-10-13 15:49:50 +01:00
Andrew Morgan
62c7b10ea5 Allow modules to create and send events into rooms (#8479)
This PR allows Synapse modules making use of the `ModuleApi` to create and send non-membership events into a room. This can useful to have modules send messages, or change power levels in a room etc. Note that they must send event through a user that's already in the room.

The non-membership event limitation is currently arbitrary, as it's another chunk of work and not necessary at the moment.

This commit has been cherry-picked from mainline.
2020-10-12 13:24:15 +01:00
Andrew Morgan
8b234fb216 Swap method calls in RoomAccessTestCase.test_change_rules (#64)
Swap these calls around, as the check for changing `restricted` to `unrestricted` will actually change `self.restricted_room` to an unrestricted room.

Do that last, instead of first. Additionally add a comment with a warning.
2020-09-18 11:37:21 +01:00
Mathieu Velten
b8d7ca9946 Make all rooms noisy by default (#60) 2020-09-18 11:35:41 +01:00