kaslo/synapse
1
0
Fork 0
Code Wiki Activity
21,535 Commits 1,048 Branches 805 Tags
7a3ded017cfb202452250a5eeb4569ebeff42533
Commit Graph

3108 Commits

Author SHA1 Message Date
Eric Eastwood
7a3ded017c Test running but no expected results yet 2022-09-27 02:05:43 -05:00
Eric Eastwood
31e2c10355 Non-working test 2022-09-27 01:48:43 -05:00
Eric Eastwood
85451b99c2 Merge branch 'madlittlemods/13356-messages-investigation-scratch-v1' into maddlittlemods/msc2716-many-batches-optimization 
Conflicts:
	synapse/handlers/federation.py
	synapse/storage/databases/main/cache.py
	synapse/storage/databases/main/event_federation.py
 2022-09-26 15:28:14 -05:00
Eric Eastwood
e4b9898dec Merge branch 'develop' into madlittlemods/11850-migrate-to-opentelemetry 2022-09-26 15:04:08 -05:00
Patrick Cloke
2fae1a3f78 Improve tests for get_unread_push_actions_for_user_in_range_*. (#13893) 
* Adds a docstring.
* Reduces a small amount of duplicated code.
* Improves tests.
 2022-09-26 18:28:12 +00:00
Mathieu Velten
41461fd4d6 typing: check origin server of typing event against room's servers (#13830) 
This is also using the partial state approximation if needed so we do
not block here during a fast join.

Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
 2022-09-26 17:33:32 +02:00
Eric Eastwood
ac1a31740b Only try to backfill event if we haven't tried before recently (#13635) 
Only try to backfill event if we haven't tried before recently (exponential backoff). No need to keep trying the same backfill point that fails over and over.

Fix https://github.com/matrix-org/synapse/issues/13622
Fix https://github.com/matrix-org/synapse/issues/8451

Follow-up to https://github.com/matrix-org/synapse/pull/13589

Part of https://github.com/matrix-org/synapse/issues/13356
 2022-09-23 14:01:29 -05:00
Patrick Cloke
efd108b45d Accept & store thread IDs for receipts (implement MSC3771). (#13782) 
Updates the `/receipts` endpoint and receipt EDU handler to parse a
`thread_id` from the body and insert it in the database.
 2022-09-23 14:33:28 +00:00
reivilibre
c06b2b7142 Faster Remote Room Joins: tell remote homeservers that we are unable to authorise them if they query a room which has partial state on our server. (#13823) 2022-09-23 11:47:16 +01:00
Eric Eastwood
44e97465e9 Merge branch 'madlittlemods/13856-fix-have-seen-events-not-being-invalidated' into maddlittlemods/msc2716-many-batches-optimization 
Conflicts:
	tests/storage/databases/main/test_events_worker.py
 2022-09-23 00:16:50 -05:00
Eric Eastwood
f8dc17b539 Add test to ensure the safety works 2022-09-22 23:05:51 -05:00
Eric Eastwood
4fa8f05344 Add test to make sure we can actually clear entries just by room_id 2022-09-22 22:28:56 -05:00
Eric Eastwood
5b9b645400 Add test description 2022-09-22 21:51:51 -05:00
Eric Eastwood
1054f91e92 Merge branch 'develop' into madlittlemods/13856-fix-have-seen-events-not-being-invalidated 2022-09-22 17:15:44 -05:00
Patrick Cloke
b7272b73aa Properly paginate forward in the /relations API. (#13840) 
This fixes a bug where the `/relations` API with `dir=f` would
skip the first item of each page (except the first page), causing
incomplete data to be returned to the client.
 2022-09-22 12:47:49 +00:00
Eric Eastwood
d07947d0d3 Fix have_seen_event cache not being invalidated 
Fix https://github.com/matrix-org/synapse/issues/13856

`_invalidate_caches_for_event` doesn't run in monolith mode which
means we never even tried to clear the `have_seen_event` and other
caches. And even in worker mode, it only runs on the workers, not
the master (AFAICT).

Additionally there is bug with the key being wrong so
`_invalidate_caches_for_event` never invalidates the
`have_seen_event` cache even when it does run.

Wrong:
```py
self.have_seen_event.invalidate((room_id, event_id))
```

Correct:
```py
self.have_seen_event.invalidate(((room_id, event_id),))
```
 2022-09-21 20:33:11 -05:00
Eric Eastwood
a847a35921 Fix have_seen_event cache not being invalidated 
Fix https://github.com/matrix-org/synapse/issues/13856

`_invalidate_caches_for_event` doesn't run in monolith mode which
means we never even tried to clear the `have_seen_event` and other
caches. And even in worker mode, it only runs on the workers, not
the master (AFAICT).

Additionally there is bug with the key being wrong so
`_invalidate_caches_for_event` never invalidates the
`have_seen_event` cache even when it does run.

Wrong:
```py
self.have_seen_event.invalidate((room_id, event_id))
```

Correct:
```py
self.have_seen_event.invalidate(((room_id, event_id),))
```
 2022-09-21 18:03:57 -05:00
Brendan Abolivier
ccca14140a Track device IDs for pushers (#13831) 
Second half of the MSC3881 implementation
 2022-09-21 15:31:53 +00:00
Hugh Nimmo-Smith
0fd2f2d460 Implementation of MSC3882 login token request (#13722) 2022-09-21 15:12:29 +00:00
Brendan Abolivier
8ae42ab8fa Support enabling/disabling pushers (from MSC3881) (#13799) 
Partial implementation of MSC3881
 2022-09-21 14:39:01 +00:00
Mathieu Velten
6bd8763804 Add cache invalidation across workers to module API (#13667) 
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
 2022-09-21 15:32:01 +02:00
Quentin Gliech
e0804ef898 Improve the synapse.api.auth.Auth mock used in unit tests. (#13809) 
To return the proper type (`Requester`) instead of a `dict`.
 2022-09-21 12:40:34 +00:00
Eric Eastwood
b86869feef Merge branch 'develop' into madlittlemods/11850-migrate-to-opentelemetry 
Conflicts:
	.github/workflows/tests.yml
	poetry.lock
	synapse/storage/schema/__init__.py
 2022-09-20 18:00:28 -05:00
Erik Johnston
42d261c32f Port the push rule classes to Rust. (#13768) 2022-09-20 12:10:31 +01:00
Quentin Gliech
74f60cec92 Add an admin API endpoint to find a user based on its external ID in an auth provider. (#13810) 2022-09-16 12:29:03 +00:00
Sean Quah
b73cbb8215 Avoid putting rejected events in room state (#13723) 
Signed-off-by: Sean Quah <seanq@matrix.org>
 2022-09-16 12:45:04 +01:00
David Robertson
742f9f9d78 A third batch of Pydantic validation for rest/client/account.py (#13736) 2022-09-15 18:36:02 +01:00
Andrew Morgan
918c74bfb5 Add a MXCUri class to make working with mxc uri's easier. (#13162) 2022-09-15 12:57:16 +00:00
Eric Eastwood
84f91e36f3 Merge branch 'develop' into madlittlemods/11850-migrate-to-opentelemetry 
Conflicts:
	synapse/storage/schema/__init__.py
 2022-09-14 15:36:57 -05:00
Eric Eastwood
957e3d74fc Keep track when we try and fail to process a pulled event (#13589) 
We can follow-up this PR with:

 1. Only try to backfill from an event if we haven't tried recently -> https://github.com/matrix-org/synapse/issues/13622
 1. When we decide to backfill that event again, process it in the background so it doesn't block and make `/messages` slow when we know it will probably fail again -> https://github.com/matrix-org/synapse/issues/13623
 1. Generally track failures everywhere we try and fail to pull an event over federation -> https://github.com/matrix-org/synapse/issues/13700

Fix https://github.com/matrix-org/synapse/issues/13621

Part of https://github.com/matrix-org/synapse/issues/13356

Mentioned in [internal doc](https://docs.google.com/document/d/1lvUoVfYUiy6UaHB6Rb4HicjaJAU40-APue9Q4vzuW3c/edit#bookmark=id.qv7cj51sv9i5)
 2022-09-14 13:57:50 -05:00
Patrick Cloke
666ae87729 Update event push action and receipt tables to support threads. (#13753) 
Adds a `thread_id` column to the `event_push_actions`, `event_push_actions_staging`,
and `event_push_summary` tables. This will notifications to be segmented by the thread
in a future pull request. The `thread_id` column stores the root event ID or the special
value `"main"`.

The `thread_id` column for `event_push_actions` and `event_push_summary` is
backfilled with `"main"` for all existing rows. New entries into `event_push_actions`
and `event_push_actions_staging` will get the proper thread ID.

`receipts_linearized` and `receipts_graph` also gain a `thread_id` column, which is similar,
except `NULL` is a special value meaning the receipt is "unthreaded".

See MSC3771 and MSC3773 for where this data will be useful.
 2022-09-14 17:11:16 +00:00
Sean Quah
c73774467e Fix bug in device list caching when remote users leave rooms (#13749) 
When a remote user leaves the last room shared with the homeserver, we
have to mark their device list as unsubscribed, otherwise we would hold
on to a stale device list in our cache. Crucially, the device list would
remain cached even after the remote user rejoined the room, which could
lead to E2EE failures until the next change to the remote user's device
list.

Fixes #13651.

Signed-off-by: Sean Quah <seanq@matrix.org>
 2022-09-14 10:42:57 +01:00
Eric Eastwood
50f0342594 Merge branch 'develop' into madlittlemods/11850-migrate-to-opentelemetry 
Conflicts:
	poetry.lock
	synapse/api/auth.py
	synapse/federation/federation_client.py
	synapse/logging/opentracing.py
	synapse/rest/client/keys.py
	synapse/rest/client/sendtodevice.py
	synapse/storage/schema/__init__.py
 2022-09-09 16:28:05 -05:00
David Robertson
f2d2481e56 Require SQLite >= 3.27.0 (#13760) 2022-09-09 11:14:10 +01:00
Dirk Klimpel
f799eac7ea Add timestamp to user's consent (#13741) 
Co-authored-by: reivilibre <olivier@librepush.net>
 2022-09-08 15:41:48 +00:00
Sean Quah
8ef0c8ff14 Fix error in is_mine_id when encountering a malformed ID (#13746) 
Previously, `is_mine_id` would raise an exception when passed an ID with
no colons. Return `False` instead.

Fixes #13040.

Signed-off-by: Sean Quah <seanq@matrix.org>
 2022-09-08 15:54:36 +01:00
reivilibre
d3d9ca156e Cancel the processing of key query requests when they time out. (#13680) 2022-09-07 12:03:32 +01:00
reivilibre
c2fe48a6ff Rename the EventFormatVersions enum values so that they line up with room version numbers. (#13706) 2022-09-07 11:08:20 +01:00
Connor Davis
bb5b47b62a Add Admin API to Fetch Messages Within a Particular Window (#13672) 
This adds two new admin APIs that allow us to fetch messages from a room within a particular time.
 2022-09-07 10:54:44 +01:00
Erik Johnston
c9b7e97355 Add a stub Rust crate (#12595) 2022-09-06 19:01:37 +01:00
Patrick Cloke
32fc3b7ba4 Remove configuration options for direct TCP replication. (#13647) 
Removes the ability to configure legacy direct TCP replication. Workers now require Redis to run.
 2022-09-06 07:50:02 +00:00
Brendan Abolivier
898fef2789 Share some metrics between the Prometheus exporter and the phone home stats (#13671) 2022-09-05 10:26:43 +00:00
Patrick Cloke
390b7ce946 Disable calculating unread counts unless the config flag is enabled. (#13694) 
This avoids doing work that will never be used (since the
resulting unread counts will never be sent in a /sync
response).

The negative of doing this is that unread counts will be
incorrect when the feature is initially enabled.
 2022-09-01 16:52:03 +00:00
Šimon Brandner
0e99f07952 Remove support for unstable private read receipts (#13653) 
Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>
 2022-09-01 13:31:54 +01:00
Jacek Kuśnierz
84ddcd7bbf Drop support for calling /_matrix/client/v3/rooms/{roomId}/invite without an id_access_token (#13241) 
Fixes #13206

Signed-off-by: Jacek Kusnierz jacek.kusnierz@tum.de
 2022-08-31 12:10:25 +00:00
reivilibre
7bc110a19e Generalise the @cancellable annotation so it can be used on functions other than just servlet methods. (#13662) 2022-08-31 11:16:05 +00:00
David Robertson
a160406d24 Fix admin List Room API return type on sqlite (#13509) 2022-08-31 10:38:16 +00:00
Dirk Klimpel
682dfcfc0d Fix that user cannot /forget rooms after the last member has left (#13546) 2022-08-30 09:58:38 +00:00
Eric Eastwood
1a209efdb2 Update get_users_in_room mis-use to get hosts with dedicated get_current_hosts_in_room (#13605) 
See https://github.com/matrix-org/synapse/pull/13575#discussion_r953023755
 2022-08-24 14:15:37 -05:00
Eric Eastwood
d58615c82c Directly lookup local membership instead of getting all members in a room first (get_users_in_room mis-use) (#13608) 
See https://github.com/matrix-org/synapse/pull/13575#discussion_r953023755
 2022-08-24 14:13:12 -05:00