1
0
Commit Graph

19878 Commits

Author SHA1 Message Date
Andrew Morgan
2d514a695e possible perf boost to calculating device list update recipients 2021-12-08 18:28:59 +00:00
Andrew Morgan
3090000857 Add an experimental config option for sending device lists to AS's 2021-12-08 18:28:59 +00:00
Andrew Morgan
ba9143817f Fix calls to create_appservice_txn in tests 2021-12-07 10:45:30 +00:00
Andrew Morgan
c0b157dc9b Merge branch 'develop' of github.com:matrix-org/synapse into anoa/e2e_as_to_device 2021-12-07 10:36:37 +00:00
reivilibre
2f053f3f82 Stabilise support for MSC2918 refresh tokens as they have now been merged into the Matrix specification. (#11435) 2021-12-06 19:11:43 +00:00
Quentin Gliech
a15a893df8 Save the OIDC session ID (sid) with the device on login (#11482)
As a step towards allowing back-channel logout for OIDC.
2021-12-06 12:43:06 -05:00
Dirk Klimpel
8b4b153c9e Add admin API to get some information about federation status (#11407) 2021-12-06 16:59:50 +00:00
Patrick Cloke
494ebd7347 Include bundled aggregations in /sync and related fixes (#11478)
Due to updates to MSC2675 this includes a few fixes:

* Include bundled aggregations for /sync.
* Do not include bundled aggregations for /initialSync and /events.
* Do not bundle aggregations for state events.
* Clarifies comments and variable names.
2021-12-06 15:51:15 +00:00
Sean Quah
a77c369897 Move glob_to_regex and re_word_boundary to matrix-python-common (#11505) 2021-12-06 11:36:08 +00:00
Eric Eastwood
4eb77965cd Update backward extremity docs to make it clear that it does not indicate whether we have fetched an events' prev_events (#11469)
Spawning from https://github.com/matrix-org/synapse/pull/9445#discussion_r758958181

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-12-03 18:25:04 -06:00
Andrew Morgan
385b3bf056 Modify tests to handle new location of to-device messages in AS txns 2021-12-03 20:03:09 +00:00
Andrew Morgan
403490de8b Insert to-device messages into the new to-device part of AS txns 2021-12-03 20:03:09 +00:00
Andrew Morgan
275e1e0b3a Add to-device messages as their own special section in AS txns 2021-12-03 20:03:09 +00:00
Andrew Morgan
13b25cf51e Fix tests to mock _TransactionController.send of ApplicationServiceScheduler.enqueue*
With enqueue_for_appservice being called per-event per-appservice, it's
not a great target for mocking. So we use _TransactionController.send
instead, to track things that are actually sent out to AS's.

This also has the benefit of testing a wider bit of the AS txn pipeline
2021-12-03 20:03:09 +00:00
Andrew Morgan
6d68b8a4e8 Refactor and generalise the sending of arbitrary fields over AS transactions
Things were starting to get a little inflexible as we kept adding new
types of data to send to application services. It's better to just
have one method for adding data to AS transactions, than one for
each type of data.

Note that subsequent PRs will need to add device lists, one-time keys
and fallback keys to these transactions. Adding those are additional
arguments to a method is much nicer than a new method for each one.

Plus with this setup we can add multiple different types of data at
once without kicking off an AS transaction for each type. This will
be useful for OTK/fallback keys, as we plan to lazily attach those
when handling other event types.
2021-12-03 20:03:09 +00:00
reivilibre
637df95de6 Support configuring the lifetime of non-refreshable access tokens separately to refreshable access tokens. (#11445) 2021-12-03 16:42:44 +00:00
Dirk Klimpel
e5f426cd54 Add type hints to synapse/tests/rest/admin (#11501) 2021-12-03 13:57:13 +00:00
Olivier Wilkinson (reivilibre)
8cd68b8102 Revert accidental commits to develop. 2021-12-03 12:31:28 +00:00
Olivier Wilkinson (reivilibre)
6cae125e20 Newsfile
Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2021-12-03 12:25:37 +00:00
Olivier Wilkinson (reivilibre)
7be88fbf48 Give tests.server.setup_test_homeserver (nominally!) the same behaviour
by calling into `make_test_homeserver_synchronous`.

The function *could* have been inlined at this point but the function is big enough

and it felt fine to leave it as is.

At least there isn't a confusing name clash anymore!
2021-12-03 11:40:05 +00:00
Olivier Wilkinson (reivilibre)
b3fd99b74a Move tests.utils.setup_test_homeserver to tests.server
It had no users.

We have just taken the identity of a previous function but don't provide the same
behaviour, so we need to fix this in the next commit...
2021-12-03 11:38:14 +00:00
Olivier Wilkinson (reivilibre)
f7ec6e7d9e Convert one of the setup_test_homeservers to make_test_homeserver_synchronous
and pass in the homeserver rather than calling a same-named function to ask for one.

Later commits will jiggle things around to make this sensible.
2021-12-03 11:35:24 +00:00
Richard van der Hoff
5640992d17 Disambiguate queries on state_key (#11497)
We're going to add a `state_key` column to the `events` table, so we need to
add some disambiguation to queries which use it.
2021-12-02 22:42:58 +00:00
David Robertson
d26808dd85 Comments on the /sync tentacles (#11494)
This mainly consists of docstrings and inline comments. There are one or two type annotations and variable renames thrown in while I was here.

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-12-02 20:58:32 +00:00
reivilibre
f91624a595 Clean up tests.storage.test_appservice (#11492) 2021-12-02 18:43:33 +00:00
reivilibre
16d39a5490 Clean up tests.storage.test_main to remove use of legacy code. (#11493) 2021-12-02 18:13:43 +00:00
reivilibre
8a4c296987 Clean up tests.test_visibility to remove legacy code. (#11495) 2021-12-02 18:13:30 +00:00
Shay
49e1356ee3 Minor cleanup on recently ported doc pages (#11466)
* move wiki pages to synapse/docs and add a few titles where necessary

* update SUMMARY.md with added pages

* add changelog

* move incorrectly located newsfragment

* update changelog number

* snake case added files and update summary.md accordingly

* update issue/pr links

* update relative links to docs

* update changelog to indicate that we moved wiki pages to the docs and state reasoning

* requested changes to admin_faq.md

* requested changes to database_maintenance_tools.md

* requested changes to understanding_synapse_through_graphana_graphs.md

* add changelog

* fix leftover merge errata

* fix unwanted changes from merge

* use two spaces between entries

* outdent code blocks
2021-12-02 09:46:20 -08:00
Patrick Cloke
d2279f471b Add most of the missing type hints to synapse.federation. (#11483)
This skips a few methods which are difficult to type.
2021-12-02 16:18:10 +00:00
Sean Quah
b50e39df57 Avoid waiting for zombie processes in synctl stop (#11490) 2021-12-02 16:07:06 +00:00
Sean Quah
858d80bf0f Fix media repository failing when media store path contains symlinks (#11446) 2021-12-02 16:05:24 +00:00
reivilibre
435f044807 Add type annotations to tests.storage.test_appservice. (#11488) 2021-12-02 15:30:05 +00:00
Richard van der Hoff
f61462e1be scripts-dev/sign_json: support for signing events (#11486) 2021-12-02 15:18:40 +00:00
Eric Eastwood
a6f1a3abec Add MSC3030 experimental client and federation API endpoints to get the closest event to a given timestamp (#9445)
MSC3030: https://github.com/matrix-org/matrix-doc/pull/3030

Client API endpoint. This will also go and fetch from the federation API endpoint if unable to find an event locally or we found an extremity with possibly a closer event we don't know about.
```
GET /_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>
{
    "event_id": ...
    "origin_server_ts": ...
}
```

Federation API endpoint:
```
GET /_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>
{
    "event_id": ...
    "origin_server_ts": ...
}
```

Co-authored-by: Erik Johnston <erik@matrix.org>
2021-12-02 01:02:20 -06:00
Shay
84dc50e160 Port wiki pages to documentation website (#11402)
* move wiki pages to synapse/docs and add a few titles where necessary

* update SUMMARY.md with added pages

* add changelog

* move incorrectly located newsfragment

* update changelog number

* snake case added files and update summary.md accordingly

* update issue/pr links

* update relative links to docs

* update changelog to indicate that we moved wiki pages to the docs and state reasoning

* revert unintentional change to CHANGES.md

* add link

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

* Update CHANGES.md

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-12-01 10:12:19 -08:00
Patrick Cloke
ed635d3285 Add a license header and comment. (#11479) 2021-12-01 12:51:14 -05:00
Patrick Cloke
7b62791e00 Clean-up get_version_string (#11468) 2021-12-01 12:43:32 -05:00
Brendan Abolivier
153194c771 Link background update controller docs to summary (#11475) 2021-12-01 14:13:01 +00:00
Patrick Cloke
f44d729d4c Additional type hints for config module. (#11465)
This adds some misc. type hints to helper methods used
in the `synapse.config` module.
2021-12-01 07:28:23 -05:00
Patrick Cloke
a265fbd397 Register the login redirect endpoint for v3. (#11451)
As specified for Matrix v1.1.
2021-12-01 07:25:58 -05:00
Richard van der Hoff
b9fef1a7cd Update openid.md
fix header level
2021-12-01 10:57:55 +00:00
Etienne Dysli Metref
b0eb64ff7b Remove mention of OIDC certification from Dex (#11470)
Dex isn't yet [1,2] a certified OpenID Provider implementation. As of today, it's not on the list maintained by the OpenID Foundation. [3]

[1] https://github.com/dexidp/dex/issues/42
[2] https://github.com/dexidp/dex/issues/262
[3] https://openid.net/certification/
2021-12-01 09:40:51 +00:00
Shay
f1795463bf Add a note about huge pages to our Postgres doc (#11467)
* Add note to postgres doc about hugepages

* Newsfragment
2021-11-30 19:05:20 -08:00
Shay
70cbb1a5e3 Don't start Synapse master process if worker_app is set (#11416)
* Add check to catch syanpse master process starting when workers are configured

* add test to verify that starting master process with worker config raises error

* newsfragment

* specify config.worker.worker_app in check

* update test

* report specific config option that triggered the error

Co-authored-by: reivilibre <oliverw@matrix.org>

* clarify error message

Co-authored-by: reivilibre <oliverw@matrix.org>

Co-authored-by: reivilibre <oliverw@matrix.org>
2021-11-30 10:12:18 -08:00
Maximilian Bosch
42bf020463 Expose worker & homeserver as entrypoints in setup.py (#11449)
Co-authored-by: reivilibre <oliverw@matrix.org>
2021-11-30 16:52:45 +00:00
Patrick Cloke
379f2650cf Bundle relations of relations into the /relations result. (#11284)
Per updates to MSC2675 which now states that bundled
aggregations should be included from the `/relations`
endpoint.
2021-11-30 11:33:33 -05:00
Sean Quah
7ff22d6da4 Fix LruCache corruption bug with a size_callback that can return 0 (#11454)
When all entries in an `LruCache` have a size of 0 according to the
provided `size_callback`, and `drop_from_cache` is called on a cache
node, the node would be unlinked from the LRU linked list but remain in
the cache dictionary. An assertion would be later be tripped due to the
inconsistency.

Avoid unintentionally calling `__len__` and use a strict `is None`
check instead when unwrapping the weak reference.
2021-11-30 16:28:02 +00:00
Sean Quah
5a0b652d36 Eliminate a few Anys in LruCache type hints (#11453) 2021-11-30 15:39:07 +00:00
Dirk Klimpel
432a174bc1 Remove unnecessary json.dumps from tests.rest.admin (#11461)
The tests helpers automatically convert dictionaries to
JSON payloads, no need to do it manually for each
test.
2021-11-30 14:51:04 +00:00
Brendan Abolivier
b14f8a1baf Merge branch 'master' into develop 2021-11-30 14:28:30 +00:00