1
0
Commit Graph

17885 Commits

Author SHA1 Message Date
Andrew Morgan
f10ee502c7 Add some handy db methods for knocking
This just adds some database methods that we'll need to use when
implementing rescinding of knocks of clients. They are equivalent to
their invite-related counterparts.
2020-11-13 16:23:00 +00:00
Andrew Morgan
fa7f378bba Auto-add displaynames to knock events if they're missing
Tiny commit to just bring knocking up to feature parity.
2020-11-13 16:23:00 +00:00
Andrew Morgan
3dbe05d022 Extend sync to inform clients about the progress of their knocks
So we've got federation so that homeservers can communicate knocking
information between them - but how does that information actually get
down to the client? The client knows that it knocked successfully from a
200 in its original request, but what else does it need? This commit
adds a new "knock" section to /sync (in addition to "invite", "join",
and "leave") all help give the client the information it needs.

The new "knock" section is used for sending down the stripped state
events we collected earlier. The client will use these to display the
room and its metadata in a little "pending knocks" section or similar.

This is all this commit adds. If the user's knock has been accepted or
rejected, they will receive that information in the "join" or "leave"
sections of /sync.

Most of this code is just cribbing off the invite and join sync code yet
again, with some minor differences. For instance, we don't need to
exclude knock events from sync if the sender is in your ignore list, as
you are the only ones that can send knocks for yourself.

The structure of the "knock" dict in sync is modeled after "invite", as
clients also receive stripped state in that. The structure can be viewed
in the linked MSC.
2020-11-13 16:23:00 +00:00
Andrew Morgan
09e1942b04 Send stripped state events back to the knocking homeserver
Here we finally send the stripped state events back to the knocking
homeserver, which then ingests and stores those events.

A future commit will actually start sending those events down /sync to
the relevant user.
2020-11-13 16:23:00 +00:00
Andrew Morgan
a4dafd407d Federation: make_knock and send_knock implementations
Most of this is explained in the linked MSC (and don't miss the sequence
diagram in the MSC comments), but roughly knocking takes inspiration from
room joins and room invites. This commit is the room join stuff.

First the knocking homeserver POSTs to the make_knock endpoint on
another homeserver. The other homeserver will send back a knock event
that is valid for the knocking user and the room that they are knocking
on. The knocking homeserver will sign the event and send it back, before
the other homeserver takes that event and then sends it into the room on
the knocking homeserver's behalf.

It's worth noting that the accepting/rejecting knocks all happen over
existing room invite/leave flows. A homeserver rescinding its knock as
well is also just sending a leave.

Once the event has been inserted into the room, the homeserver that's in
the room will send back a 200 and an empty JSON dict to confirm
everything went well to the knocker. In a future commit, this dict will
instead be filled with some stripped state events from the room which
the knocking homeserver will pass back to the knocking user.

And yes, the logging statements in this commit are intentional. They're
consistent with the rest of the file :)
2020-11-13 16:22:56 +00:00
Andrew Morgan
fa1de1d858 Rename maybe_store_room_on_{invite,outlier_membership}
There's a handy function called maybe_store_room_on_invite which allows us to
create an entry in the rooms table for a room and its version for which we
aren't joined to yet, but we can reference when ingesting events about.

This is currently used for invites where we receive some stripped state about
the room and pass it down via /sync to the client, without us being in the
room yet.

There is a similar requirement for knocking, where we will eventually do the
same thing, and need an entry in the rooms table as well. Thus, reusing this
function works, however its name needs to be generalised a bit.

So that is what this commit does.
2020-11-11 17:19:13 +00:00
Andrew Morgan
63c767ac10 Add room_knock_state_types config option
This option serves the same purpose as the existing
room_invite_state_types option, which defines what state events are sent
over to a user that is invited to a room. This information is necessary
for the user - who isn't in the room yet - to get some metadata about
the room in order to display it in a pretty fashion in the user's
pending-invites list.

It includes information such as the room's name, avatar, topic,
canonical alias, room encryption state etc. as well as the invite
membership event which the invited user's homeserver can reference.

This new option is the exact same, but is sent by a homeserver in the
room to the knocker during the knock process. This option will actually
be utilised in a later commit.
2020-11-11 16:46:17 +00:00
Andrew Morgan
c4ff5ddd00 Add CS /_matrix/client/r0/knock/{roomIdOrAlias} endpoint
We're ditching the usual idea of having two endpoints for each
membership-related endpoint as per the MSC. Thus knocking only gets the
more powerful variant (the one that supports room aliases as well as
IDs. The reason is also optional.

The other small change is just to ensure displaynames get added to the
content of this particular membership event.
2020-11-11 16:40:41 +00:00
Andrew Morgan
6bd57f4bf3 Update the event auth rules for knocking
Hopefully most of these changes are explained through the added comments and error
messages. The changes are also described conceptually in the MSC:
https://github.com/Sorunome/matrix-doc/blob/soru/knock/proposals/2403-knock.md#join-rules
2020-11-11 16:40:38 +00:00
Andrew Morgan
f7d11de227 Add xyz.amorgan.knock /versions string 2020-11-10 19:04:49 +00:00
Andrew Morgan
8b79ea03e5 Add new experimental room version for knocking 2020-11-10 19:04:10 +00:00
Marcus Schopen
c059413001 Notes on SSO logins and media_repository worker (#8701)
If SSO login is used (e.g. SAML) in a multi worker setup, it should be mentioned that currently all SAML logins must run on the same worker, see https://github.com/matrix-org/synapse/issues/7530

Also, if you are using different ports (for example 443 and 8448) in a reverse proxy for client and federation, the path `/_matrix/media` on the client and federation port must point to the listener of the `media_repository` worker, otherwise you'll get a 404 on the federation port for the path `/_matrix/media`, if a remote server is trying to get the media object on federation port, see https://github.com/matrix-org/synapse/issues/8695
2020-11-06 14:33:07 +00:00
Andrew Morgan
2a6b685294 Add documentation about documentation to CONTRIBUTING.md (#8714)
This PR adds some documentation that:

* Describes who the audience for the `docs/`, `docs/dev/` and `docs/admin/` directories are, as well as Synapse's wiki page.
* Stresses that we'd like all documentation to be down in markdown.
2020-11-06 11:59:22 +00:00
Richard van der Hoff
fb56dfdccd Fix SIGHUP handler (#8697)
Fixes:

```
builtins.TypeError: _reload_logging_config() takes 1 positional argument but 2 were given
```
2020-11-06 11:42:07 +00:00
Dirk Klimpel
c3119d1536 Add an admin API for users' media statistics (#8700)
Add `GET /_synapse/admin/v1/statistics/users/media` to get statisics about local media usage by users.
Related to #6094
It is the first API for statistics.
Goal is to avoid/reduce usage of sql queries like [Wiki analyzing Synapse](https://github.com/matrix-org/synapse/wiki/SQL-for-analyzing-Synapse-PostgreSQL-database-stats)

Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-11-05 18:59:12 +00:00
Dirk Klimpel
e4676bd877 Add displayname to Shared-Secret Registration for admins (#8722)
Add `displayname` to Shared-Secret Registration for admins to `POST /_synapse/admin/v1/register`
2020-11-05 13:55:45 +00:00
Andrew Morgan
6abb1ad0be Consolidate purge table lists to prevent desyncronisation (#8713)
I idly noticed that these lists were out of sync with each other, causing us to miss a table in a test case (`local_invites`). Let's consolidate this list instead to prevent this from happening in the future.
2020-11-04 11:26:05 +00:00
Dirk Klimpel
4fda58ddd2 Remove the "draft" status of the Room Details Admin API (#8702)
Fixes #8550
2020-11-03 12:48:25 +00:00
Erik Johnston
243d427fbc Block clients from sending server ACLs that lock the local server out. (#8708)
Fixes #4042
2020-11-03 12:13:48 +00:00
Erik Johnston
4b09b7438e Document how to set up multiple event persisters (#8706) 2020-11-03 10:27:11 +00:00
Matthew Hodgson
d04c2d19b3 grammar 2020-11-02 21:22:36 +00:00
Andrew Morgan
e89bd3ea92 Improve error messages of non-str displayname/avatar_url (#8705)
This PR fixes two things:

* Corrects the copy/paste error of telling the client their displayname is wrong when they are submitting an `avatar_url`.
* Returns a `M_INVALID_PARAM` instead of `M_UNKNOWN` for non-str type parameters.

Reported by @t3chguy.
2020-11-02 18:01:09 +00:00
David Baker
59cc2472b3 Add base pushrule to notify for jitsi conferences (#8286)
This could be customised to trigger a different kind of notification in the future, but for now it's a normal non-highlight one.
2020-11-02 16:36:14 +00:00
Dan Callahan
ca39e67f3d Use Python 3.8 in Docker images by default (#8698)
This bumps us closer to current Python without going all the way to 3.9.

Fixes #8674

Signed-off-by: Dan Callahan <danc@element.io>
2020-11-02 16:33:06 +00:00
Erik Johnston
1eb9de90c0 Improve start time by adding index to e2e_cross_signing_keys (#8694)
We do a `SELECT MAX(stream_id) FROM e2e_cross_signing_keys` on startup.
2020-11-02 13:55:56 +00:00
Matthew Hodgson
11fd90a2b7 typo 2020-11-02 13:33:56 +00:00
Andrew Morgan
26b46796ea Fix typos in systemd-with-workers doc 2020-11-02 12:56:16 +00:00
Andrew Morgan
305545682d Fix typo in workers doc 2020-11-02 12:36:18 +00:00
Brendan Abolivier
7a0fd6f98d Fix error handling around when completing an AS transaction (#8693) 2020-10-30 16:50:48 +00:00
Erik Johnston
f27a789697 Merge branch 'master' into develop 2020-10-30 16:27:02 +00:00
Erik Johnston
b176f1036a Fix changelog v1.22.1 2020-10-30 15:33:33 +00:00
Erik Johnston
aef8514193 use correct version 2020-10-30 15:27:58 +00:00
Erik Johnston
b4289795ea 1.22.1 2020-10-30 15:25:44 +00:00
Erik Johnston
1b831f2bec Merge branch 'release-v1.22.1' into develop 2020-10-30 15:24:48 +00:00
Richard van der Hoff
90c900a8ff Merge pull request #8678 from matrix-org/rav/fix_frozen_events
Fix serialisation errors when using third-party event rules.
2020-10-30 15:24:10 +00:00
Will Hunt
b37aa1643b Tie together matches_user_in_member_list and get_users_in_room caches (#8676)
* Tie together matches_user_in_member_list and get_users_in_room

* changelog

* Remove type to fix mypy

* Add `on_invalidate` to the function signature in the hopes that may make things work well

* Remove **kwargs

* Update 8676.bugfix
2020-10-30 15:23:16 +00:00
Patrick Cloke
8f1aefa694 Improve the sample config for SSO (OIDC, SAML, and CAS). (#8635) 2020-10-30 10:01:59 -04:00
Richard van der Hoff
cbc82aa09f Implement and use an @lru_cache decorator (#8595)
We don't always need the full power of a DeferredCache.
2020-10-30 11:43:17 +00:00
Patrick Cloke
fd7c743445 Fail test cases if they fail to await all awaitables (#8690) 2020-10-30 07:15:07 -04:00
Erik Johnston
46f4be94b4 Fix race for concurrent downloads of remote media. (#8682)
Fixes #6755
2020-10-30 10:55:24 +00:00
Andrew Morgan
4504151546 Fix optional parameter in stripped state storage method (#8688)
Missed in #8671.
2020-10-30 00:22:31 +00:00
Erik Johnston
ef2d627015 Fix unit tests (#8689)
* Fix unit tests

* Newsfile
2020-10-29 18:21:49 +00:00
Will Hunt
70269fbd18 Tie together matches_user_in_member_list and get_users_in_room caches (#8676)
* Tie together matches_user_in_member_list and get_users_in_room

* changelog

* Remove type to fix mypy

* Add `on_invalidate` to the function signature in the hopes that may make things work well

* Remove **kwargs

* Update 8676.bugfix
2020-10-29 16:58:16 +00:00
Patrick Cloke
8b42a4eefd Gracefully handle a pending logging connection during shutdown. (#8685) 2020-10-29 12:53:57 -04:00
Erik Johnston
f21e24ffc2 Add ability for access tokens to belong to one user but grant access to another user. (#8616)
We do it this way round so that only the "owner" can delete the access token (i.e. `/logout/all` by the "owner" also deletes that token, but `/logout/all` by the "target user" doesn't).

A future PR will add an API for creating such a token.

When the target user and authenticated entity are different the `Processed request` log line will be logged with a: `{@admin:server as @bob:server} ...`. I'm not convinced by that format (especially since it adds spaces in there, making it harder to use `cut -d ' '` to chop off the start of log lines). Suggestions welcome.
2020-10-29 15:58:44 +00:00
Erik Johnston
22eeb6bc54 Fix cache call signature to accept on_invalidate. (#8684)
Cached functions accept an `on_invalidate` function, which we failed to add to the type signature. It's rarely used in the files that we have typed, which is why we haven't noticed it before.
2020-10-29 15:18:17 +00:00
Richard van der Hoff
0073fe914a Use %r rather than %s for stringifying events (#8679)
otherwise non-state events get written as `<FrozenEvent ... state_key='None'>`
which is indistinguishable from state events with the actual state_key `None`.
2020-10-29 12:16:49 +00:00
Richard van der Hoff
56f0ee78a9 Optimise createRoom with multiple invites (#8559)
By not dropping the membership lock between invites, we can stop joins from
grabbing the lock when we're half-done and slowing the whole thing down.
2020-10-29 11:48:39 +00:00
Patrick Cloke
00b24aa545 Support generating structured logs in addition to standard logs. (#8607)
This modifies the configuration of structured logging to be usable from
the standard Python logging configuration.

This also separates the formatting of logs from the transport allowing
JSON logs to files or standard logs to sockets.
2020-10-29 07:27:37 -04:00
Erik Johnston
9a7e0d2ea6 Don't require hiredis to run unit tests (#8680) 2020-10-29 11:17:35 +00:00