1
0
Commit Graph

19737 Commits

Author SHA1 Message Date
Andrew Morgan
401cb2bbda Deduplicate ephemeral events to send conditional
Test cases needed to be updated, as we now always call
submit_ephemeral_events_for_as, it may just be with an
empty events list.
2021-11-19 20:08:37 +00:00
Andrew Morgan
8f1183cf7b Broaden type hints; update comment 2021-11-19 18:44:25 +00:00
Andrew Morgan
ce020c30fc Move stream filter back into AppserviceHandler 2021-11-19 18:15:10 +00:00
Andrew Morgan
f65846b55b Make msc2409_to_device_messages_enabled private; remove unnecessary check
The second check for self._msc2409_to_device_messages_enabled was not necessary. It's
already checked in notify_interested_services_ephemeral earlier.
2021-11-19 15:09:46 +00:00
Andrew Morgan
2930fe6fea Changelog 2021-11-16 13:16:52 +00:00
Andrew Morgan
e914f1d734 Add tests
I decided to spin up another test class for this as the existing one is
1. quite old and 2. was mocking away too much of the infrastructure to
my liking. I've named the new class alluding to ephemeral messages, and
while we already have some ephemeral tests in AppServiceHandlerTestCase,
ideally I'd like to migrate those over.

There's two new tests here. One for testing that to-device messages for
a local user are received by any application services that have
registered interest in that user - and that those that haven't won't
receive those messages.

The next test is similar, but tests with a whole bunch of to-device
messages. Rather than actually registering tons of devices - which would
make for a very slow unit test - we just directly insert them into the
database.
2021-11-16 13:16:50 +00:00
Andrew Morgan
103f410bef Add a to_device_stream_id column to the application_services_state table
This is for tracking the stream id that each application service has
been sent up to. In other words, there shouldn't be any need to process
stream ids below the recorded one here as the AS should have already
received them.

Note that there is no reliability built-in here. Reliability of delivery
if intended for a separate PR.
2021-11-16 12:59:19 +00:00
Andrew Morgan
7899f823ae Add database method to fetch to-device messages by user_ids from db
This method is quite similar to the one below, except that it doesn't
support device ids, and supports querying with more than one user id,
both of which are relevant to application services.

The results are also formatted in a different data structure, so I'm
not sure how much we could really share here between the two methods.
2021-11-16 12:59:17 +00:00
Andrew Morgan
78bd5eaa4f Allow setting/getting stream id per appservice for to-device messages 2021-11-16 12:59:17 +00:00
Andrew Morgan
b7a44d4402 Add a new ephemeral AS handler for to_device message edus
Here we add the ability for the application service ephemeral message
processor to handle new events on the "to_device" stream.

We keep track of a stream id (token) per application service, and every
time a new to-device message comes in, for each appservice we pull the
messages between the last-recorded and current stream id and check
whether any of the messages are for a user in that appservice's user
namespace.

get_new_messages is implemented in the next commit.

since we rebased off latest develop.
2021-11-16 12:59:14 +00:00
Andrew Morgan
7fbfedb230 Add experimental config option to send to-device messages to AS's 2021-11-16 12:49:17 +00:00
Patrick Cloke
24b61f379a Add ability to un-shadow-ban via the admin API. (#11347) 2021-11-16 12:43:53 +00:00
David Robertson
0dda1a7968 Misc typing fixes for tests, part 2 of N (#11330) 2021-11-16 10:41:35 +00:00
Ashwin Nair
e72135b9d3 change 'Home Server' to one word 'homeserver' (#11320)
Signed-off-by: Ashwin S. Nair <58840757+Ashwin-exe@users.noreply.github.com>
2021-11-16 10:21:01 +00:00
David Robertson
e605e4b8f2 Database storage profile passes mypy (#11342)
It already seems to pass mypy. I wonder what changed, given that it was
on the exclusion list. So this commit consists of me ensuring
`--disallow-untyped-defs` passes and a minor fixup to a function that
returned either `True` or `None`.
2021-11-15 12:59:33 +00:00
David Robertson
5562ce6a53 Get directory db file to pass mypy (#11339) 2021-11-15 12:59:05 +00:00
Tulir Asokan
6f862c5c28 Add support for the stable version of MSC2778 (#11335)
* Add support for the stable version of MSC2778

Signed-off-by: Tulir Asokan <tulir@maunium.net>

* Expect m.login.application_service in login and password provider tests

Signed-off-by: Tulir Asokan <tulir@maunium.net>
2021-11-15 10:31:22 +00:00
Shay
605921bc6b Remove unused tables room_stats_historical and user_stats_historical (#11280)
* remove unused tables room_stats_historical and user_stats_historical

* update changelog number

* Bump schema compat version comment

* make linter happy

* Update comment to give more info

Co-authored-by: reivilibre <oliverw@matrix.org>

Co-authored-by: reivilibre <oliverw@matrix.org>
2021-11-12 16:47:56 -08:00
David Robertson
fe58672546 Annotations for state_deltas.py (#11316)
I was sad that I couldn't do better for
`_curr_state_delta_stream_cache`. At least it's explicitly called out in
a comment with #TODO.
2021-11-12 20:24:12 +00:00
Patrick Cloke
3fad4e3fe5 Rollback #11322 due to wrong syntax in mypy.ini. (#11332)
This was only checking the __init__ files in modules instead of
all files in a module, which don't pass yet.
2021-11-12 20:10:03 +00:00
David Robertson
bea815cec8 Test room alias deletion (#11327)
* Prefer `HTTPStatus` over plain `int`

This is an Opinion that no-one has seemed to object to yet.

* `--disallow-untyped-defs` for `tests.rest.client.test_directory`
* Improve synapse's annotations for deleting aliases
* Test case for deleting a room alias
* Changelog
2021-11-12 19:56:00 +00:00
Shay
0bcae8ad56 Change display names/avatar URLs to None if they contain null bytes before storing in DB (#11230)
* change display names/avatar URLS to None if they contain null bytes

* add changelog

* add POC test, requested changes

* add a saner test and remove old one

* update test to verify that display name has been changed to None

* make test less fragile
2021-11-12 10:38:24 -08:00
Patrick Cloke
9b90b9454b Add type hints to media repository storage module (#11311) 2021-11-12 11:05:26 -05:00
David Robertson
6f8f3d4bc5 Attempt to annotate events_forward_extremities (#11314)
* Make DataStore inherit from EventForwardExtremitiesStore before CacheInvalidationWorkerStore

the former implicitly inherits from the latter, so they should be
ordered like this when used.
2021-11-12 15:58:17 +00:00
David Robertson
4c96ce396e Misc typing fixes for tests, part 1 of N (#11323)
* Annotate HomeserverTestCase.servlets
* Correct annotation of federation_auth_origin
* Use AnyStr custom_headers instead of a Union

This allows (str, str) and (bytes, bytes).
This disallows (str, bytes) and (bytes, str)

* DomainSpecificString.SIGIL is a ClassVar
2021-11-12 15:50:54 +00:00
Patrick Cloke
95547e5300 Generalize the disallowed_untyped_defs in mypy.ini (#11322) 2021-11-12 14:27:45 +00:00
Patrick Cloke
b64b6d12d4 Add more type hints to synapse.util. (#11321) 2021-11-12 13:43:06 +00:00
reivilibre
2fffcb24d8 Suggest using /etc/matrix-synapse/conf.d/ for configuration with Debian packages (#11281) 2021-11-12 13:17:06 +00:00
Dirk Klimpel
8840a7b7f1 Convert delete room admin API to async endpoint (#11223)
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-11-12 12:35:31 +00:00
David Robertson
c99da2d079 Annotations for user_erasure_store (#11313)
I'm not sure why this was excluded---it seemed to be passing for me. But
it's easy enough to fixup.
2021-11-11 19:22:19 +00:00
David Robertson
6a605f4a77 Get db signatures file to pass mypy (#11312) 2021-11-11 17:04:44 +00:00
David Robertson
8dc666f785 Correct type hint for room_batch.py (#11310)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-11-11 16:49:28 +00:00
Dirk Klimpel
48278a0d09 Move sql file for remove_deleted_devices_from_device_inbox into v65 (#11303) 2021-11-11 15:01:13 +00:00
Patrick Cloke
64ef25391d Add type hints to some storage classes (#11307) 2021-11-11 08:47:31 -05:00
Neeeflix
6ce19b94e8 Fix error in thumbnail generation (#11288)
Signed-off-by: Jonas Zeunert <jonas@zeunert.org>
2021-11-10 20:49:43 +00:00
Patrick Cloke
5cace20bf1 Add missing type hints to synapse.app. (#11287) 2021-11-10 15:06:54 -05:00
Patrick Cloke
66c4b774fd Add type hints to synapse._scripts (#11297) 2021-11-10 17:55:32 +00:00
Andrew Morgan
5f277ffe89 Add documentation page stubs for Single Sign-On, SAML and CAS pages (#11298) 2021-11-10 17:54:56 +00:00
Richard van der Hoff
73cbb284b9 Remove redundant parameters on _check_event_auth (#11292)
as of #11012, these parameters are unused.
2021-11-10 14:16:06 +00:00
Olivier Wilkinson (reivilibre)
68c258a604 Merge tag 'v1.47.0rc2' into develop
Synapse 1.47.0rc2 (2021-11-10)
==============================

This fixes an issue with publishing the Debian packages for 1.47.0rc1.
It is otherwise identical to 1.47.0rc1.
2021-11-10 13:01:08 +00:00
Olivier Wilkinson (reivilibre)
595f28529c Changelog tweak from feedback v1.47.0rc2 2021-11-10 09:54:34 +00:00
Olivier Wilkinson (reivilibre)
ef7f9286d1 Move Debian changelog entries to rc2 since rc1 was not published 2021-11-10 09:48:50 +00:00
Olivier Wilkinson (reivilibre)
82e62b488a 1.47.0rc2 2021-11-10 09:44:38 +00:00
Olivier Wilkinson (reivilibre)
af6374905a Correct the Debian changelog 2021-11-10 09:37:48 +00:00
Stanislav Motylkov
b09d90cac9 Fix typos in the username_available admin API documentation. (#11286) 2021-11-09 21:11:05 +00:00
Eric Eastwood
f1d5c2f269 Split out federated PDU retrieval into a non-cached version (#11242)
Context: https://github.com/matrix-org/synapse/pull/11114/files#r741643968
2021-11-09 15:07:57 -06:00
Patrick Cloke
0ef69ddbdc Ignore missing imports for parameterized. (#11285)
This was due to a conflict between #11282, which changed
mypy configuration, and #11228, a normal change.
2021-11-09 19:04:53 +00:00
Dan Callahan
3b951445a7 Require mypy for synapse/ & tests/ unless excluded (#11282)
Signed-off-by: Dan Callahan <danc@element.io>
2021-11-09 16:22:47 +00:00
Andrew Morgan
a026695083 Clarifications and small fixes to to-device related code (#11247)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-11-09 14:31:15 +00:00
Olivier Wilkinson (reivilibre)
dc5f524974 Update __init__.py v1.47.0rc1 2021-11-09 13:51:08 +00:00