1
0
Commit Graph

21538 Commits

Author SHA1 Message Date
Eric Eastwood
62f35ead9e Show extra unepexpected events 2022-09-28 18:33:17 -05:00
Eric Eastwood
20f4d1c4f9 Use event_id to compare 2022-09-27 03:10:37 -05:00
Eric Eastwood
6423938350 Better assertion message 2022-09-27 03:03:46 -05:00
Eric Eastwood
7a3ded017c Test running but no expected results yet 2022-09-27 02:05:43 -05:00
Eric Eastwood
31e2c10355 Non-working test 2022-09-27 01:48:43 -05:00
Eric Eastwood
85451b99c2 Merge branch 'madlittlemods/13356-messages-investigation-scratch-v1' into maddlittlemods/msc2716-many-batches-optimization
Conflicts:
	synapse/handlers/federation.py
	synapse/storage/databases/main/cache.py
	synapse/storage/databases/main/event_federation.py
2022-09-26 15:28:14 -05:00
Eric Eastwood
d8899e45a7 Merge branch 'madlittlemods/11850-migrate-to-opentelemetry' into madlittlemods/13356-messages-investigation-scratch-v1 2022-09-26 15:06:30 -05:00
Eric Eastwood
e4b9898dec Merge branch 'develop' into madlittlemods/11850-migrate-to-opentelemetry 2022-09-26 15:04:08 -05:00
Patrick Cloke
2fae1a3f78 Improve tests for get_unread_push_actions_for_user_in_range_*. (#13893)
* Adds a docstring.
* Reduces a small amount of duplicated code.
* Improves tests.
2022-09-26 18:28:12 +00:00
Andrew Morgan
58ab96747c Update the manpage documentation for the hash_password script (#13911) 2022-09-26 18:18:41 +00:00
David Robertson
0a38c7ec6d Snapshot schema 72 (#13873)
Including another batch of fixes to the schema dump script
2022-09-26 18:28:32 +01:00
Mathieu Velten
41461fd4d6 typing: check origin server of typing event against room's servers (#13830)
This is also using the partial state approximation if needed so we do
not block here during a fast join.

Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-09-26 17:33:32 +02:00
Nick Mills-Barrett
6b4593a80f Simplify cache invalidation after event persist txn (#13796)
This moves all the invalidations into a single place and de-duplicates
the code involved in invalidating caches for a given event by using
the base class method.
2022-09-26 16:26:35 +01:00
enterprisey
f34b0bc262 Update NixOS module URL (#13818)
* Update NixOS module URL

* Create 13818.doc
2022-09-26 15:26:59 +01:00
Erik Johnston
c5defa4cba Correctly add Cargo.lock to sdsit (#13909) 2022-09-26 13:58:10 +00:00
David Robertson
dcdd50e458 Fix mypy errors with latest canonicaljson (#13905)
* Lockfile: update canonicaljson 1.6.0 -> 1.6.3

* Fix mypy errors with latest canonicaljson

The change to `_encode_json_bytes` definition wasn't sufficient:

```
synapse/http/server.py:751: error: Incompatible types in assignment (expression has type "Callable[[Arg(object, 'json_object')], bytes]", variable has type "Callable[[Arg(object, 'data')], bytes]")  [assignment]
```

Which I think is mypy warning us that the two functions accept different
sets of kwargs. Fair enough!

* Changelog
2022-09-26 13:30:00 +01:00
Eric Eastwood
78b44340d6 More debugging 2022-09-24 04:16:56 -05:00
Eric Eastwood
33d12a516f Debugging 2022-09-24 01:29:11 -05:00
Eric Eastwood
ac1a31740b Only try to backfill event if we haven't tried before recently (#13635)
Only try to backfill event if we haven't tried before recently (exponential backoff). No need to keep trying the same backfill point that fails over and over.

Fix https://github.com/matrix-org/synapse/issues/13622
Fix https://github.com/matrix-org/synapse/issues/8451

Follow-up to https://github.com/matrix-org/synapse/pull/13589

Part of https://github.com/matrix-org/synapse/issues/13356
2022-09-23 14:01:29 -05:00
Sean Quah
f49f73c0da Faster room joins: Avoid blocking /keys/changes (#13888)
Part of the work for #12993.

Once #12993 is fully resolved, we expect `/keys/changes` to behave
sensibly when joined to a room with partial state.

Signed-off-by: Sean Quah <seanq@matrix.org>
2022-09-23 17:55:15 +01:00
Eric Eastwood
db868db594 Fix access token leak to logs from proxyagent (#13855)
This can happen specifically with an application service `/transactions/10722?access_token=leaked` request

Fix https://github.com/matrix-org/synapse/issues/13010

---

Saw an example leak in https://github.com/matrix-org/synapse/issues/13423#issuecomment-1205348482

```
2022-08-04 14:47:57,925 - synapse.http.client - 401 - DEBUG - as-sender-signal-1 - Sending request PUT http://localhost:29328/transactions/10722?access_token=<redacted>
2022-08-04 14:47:57,926 - synapse.http.proxyagent - 223 - DEBUG - as-sender-signal-1 - Requesting b'http://localhost:29328/transactions/10722?access_token=leaked' via <HostnameEndpoint localhost:29328>
```
2022-09-23 11:49:39 -05:00
Patrick Cloke
e3512a7719 Update Cargo.lock file. (#13889) 2022-09-23 11:59:39 -04:00
Patrick Cloke
efd108b45d Accept & store thread IDs for receipts (implement MSC3771). (#13782)
Updates the `/receipts` endpoint and receipt EDU handler to parse a
`thread_id` from the body and insert it in the database.
2022-09-23 14:33:28 +00:00
Sean Quah
03c2bfb7f8 Send device list updates out to servers in partially joined rooms (#13874)
Use the provided list of servers in the room from the `/send_join`
response, since we will not know which users are in the room.  This
isn't sufficient to ensure that all remote servers receive the right
device list updates, since the `/send_join` response may be inaccurate
or we may calculate the membership state of new users in the room
incorrectly.

Signed-off-by: Sean Quah <seanq@matrix.org>
2022-09-23 13:44:03 +01:00
reivilibre
c06b2b7142 Faster Remote Room Joins: tell remote homeservers that we are unable to authorise them if they query a room which has partial state on our server. (#13823) 2022-09-23 11:47:16 +01:00
reivilibre
ac7e5683d6 Add comments to the Prometheus recording rules to make it clear which set of rules you need for Grafana or Prometheus Console. (#13876) 2022-09-23 11:46:45 +01:00
Erik Johnston
c9316f9f76 Merge branch 'release-v1.68' into develop 2022-09-23 10:43:06 +01:00
Erik Johnston
682385917d Move warnings up v1.68.0rc2 2022-09-23 09:40:46 +01:00
Erik Johnston
11ebcc8a3a 1.68.0rc2 2022-09-23 09:40:16 +01:00
Eric Eastwood
44e97465e9 Merge branch 'madlittlemods/13856-fix-have-seen-events-not-being-invalidated' into maddlittlemods/msc2716-many-batches-optimization
Conflicts:
	tests/storage/databases/main/test_events_worker.py
2022-09-23 00:16:50 -05:00
Eric Eastwood
f8dc17b539 Add test to ensure the safety works 2022-09-22 23:05:51 -05:00
Eric Eastwood
b9be6c5b5d Raise exception so we don't run into this arg mismatch again 2022-09-22 22:48:58 -05:00
Eric Eastwood
4fa8f05344 Add test to make sure we can actually clear entries just by room_id 2022-09-22 22:28:56 -05:00
Eric Eastwood
9fb750dc2f Better changelog 2022-09-22 21:56:57 -05:00
Eric Eastwood
5b9b645400 Add test description 2022-09-22 21:51:51 -05:00
Eric Eastwood
0cdc7bf432 Fix @cachedList on _have_seen_events_dict
As mentioned by @erikjohnston,
https://github.com/matrix-org/synapse/issues/13865#issuecomment-1254751569
2022-09-22 17:36:54 -05:00
Eric Eastwood
2162ab5607 Invalidate cache like #13796
Copying what https://github.com/matrix-org/synapse/pull/13796
is doing
2022-09-22 17:18:34 -05:00
Eric Eastwood
1054f91e92 Merge branch 'develop' into madlittlemods/13856-fix-have-seen-events-not-being-invalidated 2022-09-22 17:15:44 -05:00
Eric Eastwood
5a5c3246f4 Scratch try different orders just to see how the tests pass differently 2022-09-22 14:26:58 -05:00
David Robertson
f7c89c44c5 Raise issue if complement fails with latest deps (#13859) 2022-09-22 17:33:37 +01:00
Patrick Cloke
8c3dcdf1b9 Fix the cross-link from register admin API to config docs. (#13870) 2022-09-22 09:11:50 -04:00
Patrick Cloke
b7272b73aa Properly paginate forward in the /relations API. (#13840)
This fixes a bug where the `/relations` API with `dir=f` would
skip the first item of each page (except the first page), causing
incomplete data to be returned to the client.
2022-09-22 12:47:49 +00:00
Erik Johnston
9615e23c29 Fix sdist to include rust build script (#13866)
* Fix sdist to include rust build script

Fixes #13851

* Newsfile
2022-09-22 11:59:23 +01:00
Eric Eastwood
a25821d160 Try sort backfill points by tie breaking on stream_ordering 2022-09-22 01:03:35 -05:00
Eric Eastwood
b23b3e4b29 Calculate the stream_ordering from newest -> oldest (in the correct order) and persist in the oldest -> newest to get the least missing prev_event fetch thrashing 2022-09-21 23:32:56 -05:00
Eric Eastwood
f2a5c70b88 Assert is not None 2022-09-21 21:29:35 -05:00
Eric Eastwood
d07947d0d3 Fix have_seen_event cache not being invalidated
Fix https://github.com/matrix-org/synapse/issues/13856

`_invalidate_caches_for_event` doesn't run in monolith mode which
means we never even tried to clear the `have_seen_event` and other
caches. And even in worker mode, it only runs on the workers, not
the master (AFAICT).

Additionally there is bug with the key being wrong so
`_invalidate_caches_for_event` never invalidates the
`have_seen_event` cache even when it does run.

Wrong:
```py
self.have_seen_event.invalidate((room_id, event_id))
```

Correct:
```py
self.have_seen_event.invalidate(((room_id, event_id),))
```
2022-09-21 20:33:11 -05:00
Eric Eastwood
f6393db0da Add changelog 2022-09-21 18:16:09 -05:00
Eric Eastwood
a847a35921 Fix have_seen_event cache not being invalidated
Fix https://github.com/matrix-org/synapse/issues/13856

`_invalidate_caches_for_event` doesn't run in monolith mode which
means we never even tried to clear the `have_seen_event` and other
caches. And even in worker mode, it only runs on the workers, not
the master (AFAICT).

Additionally there is bug with the key being wrong so
`_invalidate_caches_for_event` never invalidates the
`have_seen_event` cache even when it does run.

Wrong:
```py
self.have_seen_event.invalidate((room_id, event_id))
```

Correct:
```py
self.have_seen_event.invalidate(((room_id, event_id),))
```
2022-09-21 18:03:57 -05:00
David Robertson
1a1abdda42 Last batch of Pydantic for synapse/rest/client/account.py (#13832)
* Validation for `/add_threepid/msisdn/submit_token`

* Don't validate deprecated endpoint

* Changelog
2022-09-21 22:23:44 +01:00