1
0
Commit Graph

2973 Commits

Author SHA1 Message Date
David Robertson d102ad67fd annotate tests.server.FakeChannel (#13136) 2022-07-04 18:08:56 +01:00
Andrew Morgan 6180e1bc4b Merge tag 'v1.62.0rc3' into develop
Synapse 1.62.0rc3 (2022-07-04)
==============================

Bugfixes
--------

- Update the version of the [ldap3 plugin](https://github.com/matrix-org/matrix-synapse-ldap3/) included in the `matrixdotorg/synapse` DockerHub images and the Debian packages hosted on `packages.matrix.org` to 0.2.1. This fixes [a bug](https://github.com/matrix-org/matrix-synapse-ldap3/pull/163) with usernames containing uppercase characters. ([\#13156](https://github.com/matrix-org/synapse/issues/13156))
- Fix a bug introduced in Synapse 1.62.0rc1 affecting unread counts for users on small servers. ([\#13168](https://github.com/matrix-org/synapse/issues/13168))
2022-07-04 17:35:06 +01:00
Erik Johnston 723ce73d02 Fix stuck notification counts on small servers (#13168) 2022-07-04 16:02:21 +01:00
Andrew Morgan 9820665597 Remove tests/utils.py from mypy's exclude list (#13159) 2022-07-04 15:15:33 +01:00
Richard van der Hoff 8c2825276f Skip waiting for full state for incoming events (#13144)
When we receive an event over federation during a faster join, there is no need
to wait for full state, since we have a whole reconciliation process designed
to take the partial state into account.
2022-07-01 10:19:27 +01:00
Patrick Cloke b0366853ca Merge remote-tracking branch 'origin/release-v1.62' into develop 2022-06-30 13:27:24 -04:00
Erik Johnston dbce28b2f1 Fix unread counts on large servers (#13140) 2022-06-30 15:08:40 +01:00
Patrick Cloke 6ad012ef89 More type hints for synapse.logging (#13103)
Completes type hints for synapse.logging.scopecontextmanager and (partially)
for synapse.logging.opentracing.
2022-06-30 13:05:06 +00:00
Šimon Brandner 13e359aec8 Implement MSC3827: Filtering of /publicRooms by room type (#13031)
Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>
2022-06-29 17:12:45 +00:00
Erik Johnston 92a0c18ef0 Improve performance of getting unread counts in rooms (#13119) 2022-06-29 10:32:38 +00:00
Andrew Morgan 6cba6a51af Merge branch 'master' into develop 2022-06-28 15:19:48 +01:00
reivilibre fa13080618 Merge pull request from GHSA-22p3-qrh9-cx32
* Make _iterate_over_text easier to read by using simple data structures

* Prefer a set of tags to ignore

In my tests, it's 4x faster to check for containment in a set of this size

* Add a stack size limit to _iterate_over_text

* Continue accepting the case where there is no body element

* Use an early return instead for None

Co-authored-by: Richard van der Hoff <richard@matrix.org>
2022-06-28 14:29:08 +01:00
Erik Johnston 7469824d58 Fix serialization errors when rotating notifications (#13118) 2022-06-28 13:13:44 +01:00
David Robertson f1145563f6 Extra type annotations in test_server (#13124) 2022-06-28 12:12:17 +00:00
Robert Long 9b683ea80f Add Cross-Origin-Resource-Policy header to thumbnail and download media endpoints (#12944) 2022-06-27 14:44:05 +01:00
santhoshivan23 d54909956e validate room alias before interacting with the room directory (#13106) 2022-06-22 15:32:18 +01:00
Richard van der Hoff d4b1c0d800 Fix inconsistencies in event validation (#13088) 2022-06-17 16:30:59 +01:00
Richard van der Hoff e16ea87d0f Fix inconsistencies in event validation for m.room.create events (#13087)
* Extend the auth rule checks for `m.room.create` events

... and move them up to the top of the function. Since the no auth_events are
allowed for m.room.create events, we may as well get the m.room.create event
checks out of the way first.

* Add a test for create events with prev_events
2022-06-17 13:56:46 +00:00
Patrick Cloke d3d84685ce Add type hints to event push actions tests. (#13099) 2022-06-17 12:38:13 +00:00
Quentin Gliech 73af10f419 Simplify the alias deletion logic as an application service. (#13093) 2022-06-17 12:19:22 +01:00
Patrick Cloke 0fcc0ae37c Improve URL previews for sites with only Twitter card information. (#13056)
Pull out `twitter:` meta tags when generating a preview and
use it to augment any `og:` meta tags.

Prefers Open Graph information over Twitter card information.
2022-06-16 07:41:57 -04:00
Jacek Kuśnierz 0ef1307619 Add custom well-known (#13035)
Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
2022-06-16 11:48:18 +01:00
Richard van der Hoff 8ecf6be1e1 Move some event auth checks out to a different method (#13065)
* Add auth events to events used in tests

* Move some event auth checks out to a different method

Some of the event auth checks apply to an event's auth_events, rather than the
state at the event - which means they can play no part in state
resolution. Move them out to a separate method.

* Rename check_auth_rules_for_event

Now it only checks the state-dependent auth rules, it needs a better name.
2022-06-15 19:48:22 +01:00
Sean Quah 99d3931974 Add more tests for room upgrades (#13074)
Signed-off-by: Sean Quah <seanq@element.io>
2022-06-15 18:58:23 +01:00
Erik Johnston c95b04bb0e Change default sync_response_cache_duration (#13042) 2022-06-15 16:55:20 +00:00
Hannes Lerchl 7d99414edf Replace pyjwt with authlib in org.matrix.login.jwt (#13011) 2022-06-15 16:45:16 +00:00
Erik Johnston 0d1d3e0708 Speed up get_unread_event_push_actions_by_room (#13005)
Fixes #11887 hopefully.

The core change here is that `event_push_summary` now holds a summary of counts up until a much more recent point, meaning that the range of rows we need to count in `event_push_actions` is much smaller.

This needs two major changes:
1. When we get a receipt we need to recalculate `event_push_summary` rather than just delete it
2. The logic for deleting `event_push_actions` is now divorced from calculating `event_push_summary`.

In future it would be good to calculate `event_push_summary` while we persist a new event (it should just be a case of adding one to the relevant rows in `event_push_summary`), as that will further simplify the get counts logic and remove the need for us to periodically update `event_push_summary` in a background job.
2022-06-15 15:17:14 +00:00
reivilibre 0dbdc39940 Fix a long-standing bug which meant that rate limiting was not restrictive enough in some cases. (#13018) 2022-06-15 14:11:55 +00:00
David Robertson a4ae1406d1 Fix typechecks against twisted trunk (#13061) 2022-06-15 11:49:58 +01:00
David Robertson c99b511db9 Fix destination_is errors seen in sentry. (#13041)
* Rename test_fedclient to match its source file
* Require at least one destination to be truthy
* Explicitly validate user ID in profile endpoint GETs
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-06-14 18:28:26 +01:00
Quentin Gliech fe1daad672 Move the "email unsubscribe" resource, refactor the macaroon generator & simplify the access token verification logic. (#12986)
This simplifies the access token verification logic by removing the `rights`
parameter which was only ever used for the unsubscribe link in email
notifications. The latter has been moved under the `/_synapse` namespace,
since it is not a standard API.

This also makes the email verification link more secure, by embedding the
app_id and pushkey in the macaroon and verifying it. This prevents the user
from tampering the query parameters of that unsubscribe link.

Macaroon generation is refactored:

- Centralised all macaroon generation and verification logic to the
  `MacaroonGenerator`
- Moved to `synapse.utils`
- Changed the constructor to require only a `Clock`, hostname, and a secret key
  (instead of a full `Homeserver`).
- Added tests for all methods.
2022-06-14 09:12:08 -04:00
Quentin Gliech 92103cb2c8 Decouple synapse.api.auth_blocking.AuthBlocking from synapse.api.auth.Auth. (#13021) 2022-06-14 09:51:15 +01:00
David Teller a164a46038 Uniformize spam-checker API, part 4: port other spam-checker callbacks to return Union[Allow, Codes]. (#12857)
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-06-13 18:16:16 +00:00
Patrick Cloke 53b77b203a Replace noop background updates with DELETE. (#12954)
Removes the `register_noop_background_update` and deletes the background
updates directly in a delta file.
2022-06-13 14:06:27 -04:00
Richard van der Hoff f68b5e5773 Merge branch 'rav/simplify_event_auth_interface' into develop 2022-06-13 11:34:59 +01:00
Richard van der Hoff 0d9d36b15c Remove room_version param from check_auth_rules_for_event
Instead, use the `room_version` property of the event we're checking.

The `room_version` was originally added as a parameter somewhere around #4482,
but really it's been redundant since #6875 added a `room_version` field to `EventBase`.
2022-06-12 23:13:10 +01:00
Richard van der Hoff 2959184a42 EventAuthTestCase: build events for the right room version
In practice, when we run the auth rules, all of the events have the right room
version. Let's stop building Room V1 events for these tests and use the right
version.
2022-06-12 23:13:09 +01:00
Patrick Cloke 81608490e3 Stop depending on room_id to be returned for children state in the hierarchy response. (#12991)
The `room_id` field was removed from MSC2946 before
it was accepted. It was initially kept for backwards compatibility
and should be removed now that the stable form of the API
is used.

This change only stops Synapse from validating that it is returned,
a future PR will remove returning it as part of the response.
2022-06-10 07:15:51 -04:00
David Robertson 97053c9406 Type annotations for test_v2 (#12985) 2022-06-09 09:48:04 +01:00
Patrick Cloke dd2d66b0c9 Move the (unstable) dir parameter for /relations behind an experimental flag. (#12984)
MSC3715 defines this parameter, but the unstable version of it should be
behind an experimental flag.
2022-06-08 09:00:35 -04:00
Sean Quah 3c1c40d843 Clean up the test code for client disconnections (#12929)
* Reword failure message about `await_result=False`
* Use `reactor.advance()` instead of `reactor.pump()`
* Raise `AssertionError`s ourselves
* Un-instance method `_test_disconnect`
* Replace `ThreadedMemoryReactorClock` with `MemoryReactorClock`
2022-06-07 18:17:32 +01:00
Sean Quah a10cc5f824 Test cancellation at every await during request handling (#12674)
* Add tests for `/rooms/<room_id>/members` cancellation.
* Add tests for `/rooms/<room_id>/state` cancellation.

Signed-off-by: Sean Quah <seanq@element.io>
2022-06-07 17:14:47 +01:00
Patrick Cloke 9dc3293e0b Consolidate the logic of delete_device/delete_devices. (#12970)
By always using delete_devices and sometimes passing a list
with a single device ID.

Previously these methods had gotten out of sync with each
other and it seems there's little benefit to the single-device
variant.
2022-06-07 07:43:35 -04:00
Andrew Morgan a47636c570 Prevent local quarantined media from being claimed by media retention (#12972) 2022-06-07 10:53:47 +00:00
Patrick Cloke f7baffd8ec Remove remaining pieces of groups code. (#12966)
* Remove an unused stream ID generator.
* Remove the now unused remote profile cache.
2022-06-06 13:20:05 -04:00
Erik Johnston 44de53bb79 Reduce state pulled from DB due to sending typing and receipts over federation (#12964)
Reducing the amount of state we pull from the DB is useful as fetching state is expensive in terms of DB, CPU and memory.
2022-06-06 16:46:11 +01:00
Patrick Cloke 148fe58a24 Do not break URL previews if an image is unreachable. (#12950)
Avoid breaking a URL preview completely if the chosen image 404s
or is unreachable for some other reason (e.g. DNS).
2022-06-06 07:46:04 -04:00
Patrick Cloke 1acc897c31 Implement MSC3816, consider the root event for thread participation. (#12766)
As opposed to only considering a user to have "participated" if they
replied to the thread.
2022-06-06 07:18:04 -04:00
Erik Johnston e3163e2e11 Reduce the amount of state we pull from the DB (#12811) 2022-06-06 09:24:12 +01:00
Patrick Cloke 01df5bacac Improve URL previews for some pages (#12951)
* Skip `og` and `meta` tags where the value is empty.
* Fallback to the favicon if there are no other images.
* Ignore tags meant for navigation.
2022-06-03 12:09:12 -04:00