1
0
Commit Graph

541 Commits

Author SHA1 Message Date
Andrew Morgan
93db8f1992 Merge commit 'cf7d3c90d' into dinsic 2021-04-16 12:33:45 +01:00
Andrew Morgan
232b324c38 Port "Add support for no_proxy and case insensitive env variables" from mainline to dinsic (#93)
This PR is simply porting https://github.com/matrix-org/synapse/pull/9372 to dinsic.

I also had to bring in https://github.com/matrix-org/synapse/pull/8821 and https://github.com/matrix-org/synapse/pull/9084 for this code to work properly - a sign that we should merge mainline into dinsic again soon.
2021-03-22 17:48:42 +00:00
Andrew Morgan
0c897c7410 Merge commit '7c4344747' into anoa/dinsic_release_1_23_1 2020-12-31 17:08:54 +00:00
Andrew Morgan
1077d50c65 Merged commit 'deff8f628' into anoa/dinsic_release_1_23_1 2020-12-31 16:20:32 +00:00
Andrew Morgan
5c51806d1d Merge commit 'acfe3b306' into anoa/dinsic_release_1_23_1 2020-12-31 15:22:27 +00:00
Andrew Morgan
917d0f9250 Merge commit 'f12589547' into anoa/dinsic_release_1_23_1 2020-12-31 14:55:18 +00:00
Andrew Morgan
de6773523d Merge commit 'ef2d62701' into anoa/dinsic_release_1_23_1 2020-12-31 14:34:59 +00:00
Andrew Morgan
b90b34744a Merge commit 'c97da1e45' into anoa/dinsic_release_1_23_1 2020-12-31 13:40:57 +00:00
Andrew Morgan
468d7f6777 Merge commit '7a3adbd7a' into anoa/dinsic_release_1_23_1 2020-12-31 12:06:40 +00:00
Andrew Morgan
7e1a0bf068 Merge commit 'b19b63e6b' into anoa/dinsic_release_1_23_1 2020-12-31 11:51:30 +00:00
Andrew Morgan
9e11250574 Merge commit '9146a8a69' into dinsic 2020-12-31 11:44:21 +00:00
Andrew Morgan
b30484ab34 Merge commit '3ee17585c' into dinsic 2020-12-31 11:23:56 +00:00
Andrew Morgan
5fc0dd8126 Merge commit '74976a8e4' into dinsic 2020-12-31 11:23:24 +00:00
Erik Johnston
df4b1e9c74 Pass room_id to get_auth_chain_difference (#8879)
This is so that we can choose which algorithm to use based on the room ID.
2020-12-04 15:52:49 +00:00
Richard van der Hoff
90cf1eec44 Remove redundant mocking 2020-12-02 17:53:38 +00:00
Patrick Cloke
30fba62108 Apply an IP range blacklist to push and key revocation requests. (#8821)
Replaces the `federation_ip_range_blacklist` configuration setting with an
`ip_range_blacklist` setting with wider scope. It now applies to:

* Federation
* Identity servers
* Push notifications
* Checking key validitity for third-party invite events

The old `federation_ip_range_blacklist` setting is still honored if present, but
with reduced scope (it only applies to federation and identity servers).
2020-12-02 11:09:24 -05:00
Erik Johnston
c5b6abd53d Correctly handle unpersisted events when calculating auth chain difference. (#8827)
We do state res with unpersisted events when calculating the new current state of the room, so that should be the only thing impacted. I don't think this is tooooo big of a deal as:

1. the next time a state event happens in the room the current state should correct itself;
2. in the common case all the unpersisted events' auth events will be pulled in by other state, so will still return the correct result (or one which is sufficiently close to not affect the result); and
3. we mostly use the state at an event to do important operations, which isn't affected by this.
2020-12-02 15:22:37 +00:00
Dirk Klimpel
3f0ff53158 Remove deprecated /_matrix/client/*/admin endpoints (#8785)
These are now only available via `/_synapse/admin/v1`.
2020-11-25 16:26:11 -05:00
Richard van der Hoff
deff8f628d Merge pull request #8761 from matrix-org/rav/test_request_rendering
Make `make_request` actually render the request
2020-11-17 15:17:04 +00:00
Erik Johnston
f737368a26 Add admin API for logging in as a user (#8617) 2020-11-17 10:51:25 +00:00
Richard van der Hoff
be8fa65d0b Remove redundant calls to render() 2020-11-16 18:24:08 +00:00
Richard van der Hoff
c3e3552ec4 fixup test 2020-11-16 15:51:47 +00:00
Richard van der Hoff
ebc405446e Add a custom_headers param to make_request (#8760)
Some tests want to set some custom HTTP request headers, so provide a way to do
that before calling requestReceived().
2020-11-16 14:45:22 +00:00
Erik Johnston
f21e24ffc2 Add ability for access tokens to belong to one user but grant access to another user. (#8616)
We do it this way round so that only the "owner" can delete the access token (i.e. `/logout/all` by the "owner" also deletes that token, but `/logout/all` by the "target user" doesn't).

A future PR will add an API for creating such a token.

When the target user and authenticated entity are different the `Processed request` log line will be logged with a: `{@admin:server as @bob:server} ...`. I'm not convinced by that format (especially since it adds spaces in there, making it harder to use `cut -d ' '` to chop off the start of log lines). Suggestions welcome.
2020-10-29 15:58:44 +00:00
Dan Callahan
aff1eb7c67 Tell Black to format code for Python 3.5 (#8664)
This allows trailing commas in multi-line arg lists.

Minor, but we might as well keep our formatting current with regard to
our minimum supported Python version.

Signed-off-by: Dan Callahan <danc@element.io>
2020-10-27 23:26:36 +00:00
Will Hunt
9e0f5a0ac4 Fix get|set_type_stream_id_for_appservice store functions (#8648) 2020-10-26 15:00:56 +00:00
Will Hunt
e8dbbcb64c Fix get|set_type_stream_id_for_appservice store functions (#8648) 2020-10-26 10:51:33 -04:00
Erik Johnston
2ac908f377 Don't instansiate Requester directly (#8614) 2020-10-22 10:11:06 +01:00
Richard van der Hoff
7b71695388 Combine the two sets of tests for CacheDescriptor 2020-10-21 15:38:29 +01:00
Andrew Morgan
e15cca700b Merge commit 'f76194a02' into anoa/dinsic_release_1_21_x
* commit 'f76194a02':
  1.21.0
  Update change log
  1.21.0rc3
  Reduce serialization errors in MultiWriterIdGen (#8456)
  Add Ubuntu 20.10 (Groovy Gorilla) to build scripts. (#8475)
  move #8444 to 'feature'
  linkify changelog
2020-10-21 15:12:45 +01:00
Andrew Morgan
854764073c Merge commit 'cc40a59b4' into anoa/dinsic_release_1_21_x
* commit 'cc40a59b4':
  1.21.0
  Add prometheus metrics to track federation delays (#8430)
  Make token serializing/deserializing async (#8427)
  Allow additional SSO properties to be passed to the client (#8413)
  changelog
  Add an improved "forward extremities" metric
  Rewrite BucketCollector
  Fix _exposition.py to stop stripping samples
  Drop support for ancient prometheus_client (#8426)
  Various clean ups to room stream tokens. (#8423)
  changelog
  Report state res metrics to Prometheus and log
  Move Measure calls into `resolve_events_with_store`
  Expose a `get_resource_usage` method in `Measure`
  Move `resolve_events_with_store` into StateResolutionHandler
2020-10-21 14:53:23 +01:00
Andrew Morgan
e8ed9a6016 Merge commit '8238b55e0' into anoa/dinsic_release_1_21_x
* commit '8238b55e0':
  Update description of server_name config option (#8415)
  Discard an empty upload_name before persisting an uploaded file (#7905)
  Don't table scan events on worker startup (#8419)
  Mypy fixes for `synapse.handlers.federation` (#8422)
2020-10-21 14:53:16 +01:00
Andrew Morgan
ab26b5894a Merge commit 'f43c66d23' into anoa/dinsic_release_1_21_x
* commit 'f43c66d23':
  Add support for running Complement against the local checkout (#8317)
  Filter out appservices from mau count (#8404)
  Only assert valid next_link params when provided (#8417)
  Add metrics to track success/otherwise of replication requests (#8406)
  Fix handling of connection timeouts in outgoing http requests (#8400)
  Changelog
  Don't check whether a 3pid is allowed to register during password reset
  Add checks for postgres sequence consistency (#8402)
  Create a mechanism for marking tests "logcontext clean" (#8399)
  Add `ui_auth_sessions_ips` table to `synapse_port_db` ignore list (#8410)
  A pair of tiny cleanups in the federation request code. (#8401)
  typo
2020-10-21 14:53:14 +01:00
Andrew Morgan
5b0b103521 Merge commit '31acc5c30' into anoa/dinsic_release_1_21_x
* commit '31acc5c30':
  Escape the error description on the sso_error template. (#8405)
  Fix occasional "Re-starting finished log context" from keyring (#8398)
  Allow existing users to login via OpenID Connect. (#8345)
  Fix schema delta for servers that have not backfilled (#8396)
  Fix MultiWriteIdGenerator's handling of restarts. (#8374)
  s/URLs/variables in changelog
  s/accidentally/incorrectly in changelog
  Update changelog wording
  Add type annotations to SimpleHttpClient (#8372)
  Add new sequences to port DB script (#8387)
  Add EventStreamPosition type (#8388)
  Mark the shadow_banned column as boolean in synapse_port_db. (#8386)
2020-10-21 14:51:58 +01:00
Andrew Morgan
3729b15a64 Merge commit '2983049a7' into anoa/dinsic_release_1_21_x
* commit '2983049a7':
  Factor out `_send_dummy_event_for_room` (#8370)
  Improve logging of state resolution (#8371)
  Fix bug which caused failure on join with malformed membership events (#8385)
  Use `async with` for ID gens (#8383)
  Don't push if an user account has expired (#8353)
  Do not check lint/test dependencies at runtime. (#8377)
  Add note to reverse_proxy.md about disabling Apache's mod_security2 (#8375)
  Changelog
2020-10-21 14:15:38 +01:00
Andrew Morgan
e529fa7574 Merge commit '4325be1a5' into anoa/dinsic_release_1_21_x
* commit '4325be1a5':
  Fix missing null character check on guest_access room state
  Fixed a bug with reactivating users with the admin API (#8362)
  Admin API for reported events (#8217)
  Fix wording of deprecation notice in changelog
  Deprecation warning for synapse admin api being accessible under /_matrix
  Create function to check for long names in devices (#8364)
  Add a comment re #1691
  Fix a bad merge from release-v1.20.0. (#8354)
  Admin API for querying rooms where a user is a member (#8306)
  Catch-up after Federation Outage (bonus): Catch-up on Synapse Startup (#8322)
  Simplify super() calls to Python 3 syntax. (#8344)
  Allow appservice users to /login (#8320)
  Update test logging to be able to accept braces (#8335)
  Move lint dependencies to extras_require (#8330)
2020-10-21 14:14:36 +01:00
Andrew Morgan
82c379c20b Merge commit 'e45b83411' into anoa/dinsic_release_1_21_x
* commit 'e45b83411':
  Add types to async_helpers (#8260)
  Fix mypy error on develop (#8282)
  Include method in thumbnail media name (#7124)
  Add types to StreamToken and RoomStreamToken (#8279)
  Add a config option for validating 'next_link' parameters against a domain whitelist (#8275)
  Clean up types for PaginationConfig (#8250)
  Use the right constructor for log records (#8278)
  Fix `MultiWriterIdGenerator.current_position`. (#8257)
2020-10-20 19:12:51 +01:00
Andrew Morgan
061b9895eb Merge commit 'cca03dbec' into anoa/dinsic_release_1_21_x
* commit 'cca03dbec':
  fix typo
  s/fixes/fix/
  Directly import json from the standard library. (#8259)
  Allow for make_awaitable's return value to be re-used. (#8261)
  Rename 'populate_stats_process_rooms_2' background job back to 'populate_stats_process_rooms' again (#8243)
2020-10-20 18:29:12 +01:00
Andrew Morgan
47d53b32e3 Merge commit '17fa4c7ca' into anoa/dinsic_release_1_21_x
* commit '17fa4c7ca':
  Catch up after Federation Outage (split, 2): Track last successful stream ordering after transmission (#8247)
  Catch-up after Federation Outage (split, 1) (#8230)
  Fix type signature in simple_select_one_onecol and friends (#8241)
  Stop sub-classing object (#8249)
2020-10-20 18:09:59 +01:00
Andrew Morgan
d7fd0deaf1 Merge commit '112266eaf' into anoa/dinsic_release_1_21_x
* commit '112266eaf':
  Add StreamStore to mypy (#8232)
  Re-implement unread counts (again) (#8059)
2020-10-20 17:51:52 +01:00
Andrew Morgan
059e0fdedf Merge commit '5bf8e5f55' into anoa/dinsic_release_1_21_x
* commit '5bf8e5f55':
  Convert the well known resolver to async (#8214)
  Convert additional databases to async/await part 2 (#8200)
  Make MultiWriterIDGenerator work for streams that use negative stream IDs (#8203)
  Do not install setuptools 50.0. (#8212)
  Move and rename `get_devices_with_keys_by_user` (#8204)
  Rename `get_e2e_device_keys` to better reflect its purpose (#8205)
  Add a comment about _LimitedHostnameResolver
2020-10-20 17:42:29 +01:00
Andrew Morgan
ff8a9a6a28 Merge commit 'e00816ad9' into anoa/dinsic_release_1_21_x
* commit 'e00816ad9':
  Do not yield on awaitables in tests. (#8193)
2020-10-20 17:42:05 +01:00
Andrew Morgan
ed944cb079 Merge commit 'b49a5b930' into anoa/dinsic_release_1_21_x
* commit 'b49a5b930':
  Convert stats and related calls to async/await (#8192)
2020-10-20 17:42:00 +01:00
Andrew Morgan
6961a007dd Merge commit 'b71d4a094' into anoa/dinsic_release_1_21_x
* commit 'b71d4a094':
  Convert simple_delete to async/await. (#8191)
2020-10-20 17:41:55 +01:00
Andrew Morgan
ec50c99609 Merge commit '9b7ac03af' into anoa/dinsic_release_1_21_x
* commit '9b7ac03af':
  Convert calls of async database methods to async (#8166)
  simple_search_list_txn should return None, not 0. (#8187)
  Fix missing _add_persisted_position (#8179)
2020-10-20 17:41:11 +01:00
Andrew Morgan
118f41dba7 Merge commit '30426c706' into anoa/dinsic_release_1_21_x
* commit '30426c706':
  Convert additional database methods to async (select list, search, insert_many, delete_*) (#8168)
2020-10-20 17:33:58 +01:00
Andrew Morgan
a89a160c17 Merge commit '4a739c73b' into anoa/dinsic_release_1_21_x
* commit '4a739c73b':
  Convert simple_update* and simple_select* to async (#8173)
2020-10-20 17:33:47 +01:00
Andrew Morgan
df77da8b4d Merge commit 'a466b6797' into anoa/dinsic_release_1_21_x
* commit 'a466b6797':
  Reduce run-times of tests by advancing the reactor less (#7757)
  Update debian systemd service to use Type=notify (#8169)
  Remove remaining is_guest argument uses from get_room_data calls (#8181)
  Do not propagate typing notifications from shadow-banned users. (#8176)
  Remove unused parameter from, and add safeguard in, get_room_data (#8174)
  Add required Debian dependencies to allow docker builds on the arm platform (#8144)
  Allow running mypy directly. (#8175)
  Update the test federation client to handle streaming responses (#8130)
  Do not propagate profile changes of shadow-banned users into rooms. (#8157)
  Make SlavedIdTracker.advance have same interface as MultiWriterIDGenerator (#8171)
  Convert simple_select_one and simple_select_one_onecol to async (#8162)
2020-10-20 17:13:42 +01:00
Andrew Morgan
b7bb088b84 Merge commit '56efa9ec7' into anoa/dinsic_release_1_21_x
* commit '56efa9ec7': (22 commits)
  Fix rate limiting unit tests. (#8167)
  Add functions to `MultiWriterIdGen` used by events stream (#8164)
  Do not allow send_nonmember_event to be called with shadow-banned users. (#8158)
  Changelog fixes
  Make StreamIdGen `get_next` and `get_next_mult` async  (#8161)
  Wording fixes to 'name' user admin api filter (#8163)
  Fix missing double-backtick in RST document
  Search in columns 'name' and 'displayname' in the admin users endpoint (#7377)
  Add type hints for state. (#8140)
  Stop shadow-banned users from sending non-member events. (#8142)
  Allow capping a room's retention policy (#8104)
  Add healthcheck for default localhost 8008 port on /health endpoint. (#8147)
  Fix flaky shadow-ban tests. (#8152)
  Don't fail /submit_token requests on incorrect session ID if request_token_inhibit_3pid_errors is turned on (#7991)
  Do not apply ratelimiting on joins to appservices (#8139)
  Micro-optimisations to get_auth_chain_ids (#8132)
  Allow denying or shadow banning registrations via the spam checker (#8034)
  Stop shadow-banned users from sending invites. (#8095)
  Be more tolerant of membership events in unknown rooms (#8110)
  Improve the error code when trying to register using a name reserved for guests. (#8135)
  ...
2020-10-20 16:45:58 +01:00
Andrew Morgan
687d30b2ed Merge commit 'c9c544cda' into anoa/dinsic_release_1_21_x
* commit 'c9c544cda':
  Remove `ChainedIdGenerator`. (#8123)
  Switch the JSON byte producer from a pull to a push producer. (#8116)
  Updated docs: Added note about missing 308 redirect support. (#8120)
  Be stricter about JSON that is accepted by Synapse (#8106)
  Convert runWithConnection to async. (#8121)
  Remove the unused inlineCallbacks code-paths in the caching code (#8119)
  Separate `get_current_token` into two. (#8113)
  Convert events worker database to async/await. (#8071)
  Add a link to the matrix-synapse-rest-password-provider. (#8111)
2020-10-19 19:12:39 +01:00