1
0
Commit Graph

21402 Commits

Author SHA1 Message Date
Eric Eastwood
ed11237fe5 Remove linting from CI for now 2022-09-13 08:58:23 -05:00
Eric Eastwood
d730a46451 Update Twisted to lastest 2022-09-13 08:56:12 -05:00
Eric Eastwood
15e242eeaa OTEL install with DMR 2022-09-13 08:54:18 -05:00
Eric Eastwood
ad3e324118 Install otel deps from develop 2022-09-12 15:46:38 -05:00
Eric Eastwood
6c40dfafcf Merge branch 'develop' into madlittlemods/11850-migrate-to-opentelemetry 2022-09-12 15:34:02 -05:00
Mathieu Velten
fa2f3d8d0c Fix GHA skippable syntax (#13778)
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-09-12 17:31:23 +00:00
Brendan Abolivier
7571337445 Fix typo in ratelimiting documentation (#13727) 2022-09-12 14:11:18 +01:00
Erik Johnston
dd7484b562 Fix CI on non-PR builds (#13769)
Mark cargo-test as skippable since it only runs on Rust code change.
2022-09-12 13:26:33 +01:00
Nick Mills-Barrett
da41a7cd61 Remove check current state membership up to date (#13745)
* Remove checks for membership column in current_state_events
* Add schema script to force through the
  `current_state_events_membership` background job

Contributed by Nick @ Beeper (@fizzadar).
2022-09-12 12:58:33 +01:00
Erik Johnston
ebfeac7c5d Check if Rust lib needs rebuilding. (#13759)
This protects against the common mistake of failing to remember to rebuild Rust code after making changes.
2022-09-12 10:03:42 +00:00
Eric Eastwood
32b9d16d82 poetry update 2022-09-09 17:38:44 -05:00
Eric Eastwood
a15592d073 Poetry install again 2022-09-09 17:16:19 -05:00
Eric Eastwood
f73bc59d25 Try to resolve poetry deps 2022-09-09 17:02:40 -05:00
Eric Eastwood
50f0342594 Merge branch 'develop' into madlittlemods/11850-migrate-to-opentelemetry
Conflicts:
	poetry.lock
	synapse/api/auth.py
	synapse/federation/federation_client.py
	synapse/logging/opentracing.py
	synapse/rest/client/keys.py
	synapse/rest/client/sendtodevice.py
	synapse/storage/schema/__init__.py
2022-09-09 16:28:05 -05:00
Nick Mills-Barrett
4c4889cac0 Concurrently collect room unread counts for push badges (#13765)
Most of the time this function is heavily cached, but when that isn't
the case fetching the counts room by room slows down push delivery on
users with many (thousands) of rooms.

Signed off by Nick @ Beeper.
2022-09-09 19:00:21 +01:00
Eric Eastwood
a911ffb42c Tag trace with instance name (#13761)
We tag the Synapse instance name so that it's an easy jumping off point into the logs. Can also be used to filter for an instance that is under load.

As suggested by @clokep and @reivilibre in,

 - https://github.com/matrix-org/synapse/pull/13729#discussion_r964719258
 - https://github.com/matrix-org/synapse/pull/13729#discussion_r964733578
2022-09-09 11:31:37 -05:00
Eric Eastwood
f694bb71b7 Strip number suffix from instance name to consolidate services that traces are spread over (#13729)
The problem with many services is that it makes it hard to find which service has the trace you want, see https://github.com/jaegertracing/jaeger-ui/issues/985

Previously, we split traces out into services based on their instance name like `matrix.org client_reader-1`, etc but there are many worker instances of the same `client_reader` so there is a lot to click through.

With this PR, all of the traces are just collected under the worker type like `client_reader`, `event_persister` 😇

Note: A Synapse worker instance name is an opaque string with the number convention only being our own thing for the `matrix.org` deployment. But seems pretty sensible to group things this way.
2022-09-09 11:30:06 -05:00
Patrick Cloke
3d9f82efcb Use an upsert for receipts_graph. (#13752)
Instead of a delete, then insert.

This was previously done for `receipts_linearized` in
2dc430d36e (#7607).
2022-09-09 07:08:41 -04:00
Erik Johnston
c85c5ace52 Add rust to CI (#13763) 2022-09-09 11:29:04 +01:00
David Robertson
f2d2481e56 Require SQLite >= 3.27.0 (#13760) 2022-09-09 11:14:10 +01:00
Sean Quah
69fa29700e Re-type hint some collections in /sync code as read-only (#13754)
Signed-off-by: Sean Quah <seanq@matrix.org>
2022-09-08 20:13:39 +01:00
reivilibre
5261d2e2e8 Remove unused Prometheus recording rules from synapse-v2.rules and add comments describing where the rest are used. (#13756) 2022-09-08 17:50:15 +00:00
Dirk Klimpel
f799eac7ea Add timestamp to user's consent (#13741)
Co-authored-by: reivilibre <olivier@librepush.net>
2022-09-08 15:41:48 +00:00
Sean Quah
906cead9ca Update docstrings to explain the impact of partial state (#13750)
Update the docstrings for `get_users_in_room` and
`get_current_hosts_in_room` to explain the impact of partial state.

Signed-off-by: Sean Quah <seanq@matrix.org>
2022-09-08 15:55:29 +01:00
Sean Quah
89e8b98b65 Avoid raising errors due to malformed IDs in get_current_hosts_in_room (#13748)
Handle malformed user IDs with no colons in `get_current_hosts_in_room`.
It's not currently possible for a malformed user ID to join a room, so
this error would never be hit.

Signed-off-by: Sean Quah <seanq@matrix.org>
2022-09-08 15:55:03 +01:00
Sean Quah
8ef0c8ff14 Fix error in is_mine_id when encountering a malformed ID (#13746)
Previously, `is_mine_id` would raise an exception when passed an ID with
no colons. Return `False` instead.

Fixes #13040.

Signed-off-by: Sean Quah <seanq@matrix.org>
2022-09-08 15:54:36 +01:00
reivilibre
cf11919ddd Fix cache metrics not being updated when not using the legacy exposition module. (#13717) 2022-09-08 15:30:48 +01:00
reivilibre
526f84bc2e Fix Prometheus recording rules to not use legacy metric names. (#13718) 2022-09-08 15:01:42 +01:00
Erik Johnston
1cc729c177 Fix latest deps (#13743) 2022-09-08 13:58:31 +01:00
reivilibre
b7e4bfd005 Fix a bug where Synapse fails to start if a signing key file contains an empty line. (#13738) 2022-09-08 11:18:03 +01:00
Eric Eastwood
d4d3249ded Instrument get_metadata_for_events for tracing (#13730)
When backfilling, `_get_state_ids_after_missing_prev_event` calls [`get_metadata_for_events`](26bc26586b/synapse/handlers/federation_event.py (L1133)). For `#matrix:matrix.org`, it's called with 77k `state_events` which means 77 calls to the database and takes 28 seconds.
2022-09-07 11:41:52 -05:00
Erik Johnston
8d7fcf9b76 Fix latest deps CI (#13734) 2022-09-07 14:07:06 +00:00
Erik Johnston
dc0e896b68 Add some rust caching to CI (#13735) 2022-09-07 13:56:59 +00:00
David Robertson
c46fecd1f2 Correct out-of-date doc for event_cache_size (#13726) 2022-09-07 14:46:11 +01:00
David Robertson
77f3986451 Define SQLite compat policy (#13728) 2022-09-07 12:07:42 +00:00
David Robertson
b58386e37e A second batch of Pydantic models for rest/client/account.py (#13687) 2022-09-07 12:16:10 +01:00
reivilibre
d3d9ca156e Cancel the processing of key query requests when they time out. (#13680) 2022-09-07 12:03:32 +01:00
reivilibre
c2fe48a6ff Rename the EventFormatVersions enum values so that they line up with room version numbers. (#13706) 2022-09-07 11:08:20 +01:00
Connor Davis
bb5b47b62a Add Admin API to Fetch Messages Within a Particular Window (#13672)
This adds two new admin APIs that allow us to fetch messages from a room within a particular time.
2022-09-07 10:54:44 +01:00
reivilibre
26bc26586b Remove the unspecced room_id field in the /hierarchy response. (#13506)
This is a re-do of 57d334a13d (#13365),
which was backed out in 12abd72497 (#13501).

The `room_id` field represented the parent space for each room
and was made redundant by changes in the API shape where the
`children_state` is now nested underneath each `room`.

The room ID of each child is in the `state_key` field and is still
available.
2022-09-06 15:28:44 -04:00
Erik Johnston
c9b7e97355 Add a stub Rust crate (#12595) 2022-09-06 19:01:37 +01:00
Erik Johnston
3d20115115 Fix trial-olddeps (#13725) 2022-09-06 14:21:55 +00:00
David Robertson
a4ecb8e353 Actually fix typechecking with latest types-jsonschema (#13724) 2022-09-06 14:29:16 +01:00
Erik Johnston
b5effc7201 Update trial old deps CI to use poetry 1.2.0 (#13707) 2022-09-06 11:43:04 +00:00
reivilibre
b455c2a5ec Update Grafana dashboard to not use legacy metric names. (#13714) 2022-09-06 12:21:21 +01:00
Patrick Cloke
32fc3b7ba4 Remove configuration options for direct TCP replication. (#13647)
Removes the ability to configure legacy direct TCP replication. Workers now require Redis to run.
2022-09-06 07:50:02 +00:00
Erik Johnston
8edf3f66d5 Reduce number of CI jobs run on PRs (#13713)
* Reduce number of CI jobs run on PRs

* Newsfile

* Also limit sytest jobs

* Fix typo

* Fix up

* Fixup
2022-09-05 13:31:42 +01:00
reivilibre
c7b18d9d44 Extend the release script to wait for GitHub Actions to finish and to be usable as a guide for the whole process. (#13483) 2022-09-05 12:16:59 +01:00
David Robertson
8cb9261598 Fix typechecking with latest types-jsonschema (#13712) 2022-09-05 11:13:13 +00:00
Brendan Abolivier
898fef2789 Share some metrics between the Prometheus exporter and the phone home stats (#13671) 2022-09-05 10:26:43 +00:00