kaslo/synapse
1
0
Fork 0
Code Wiki Activity
19,932 Commits 1,047 Branches 805 Tags
ecd5a03e46d01893d90da06dac3137290bbc4e8e
Commit Graph

19932 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Olivier Wilkinson (reivilibre)
ecd5a03e46 Bypass caching from the old system 2021-12-20 16:45:43 +00:00
Olivier Wilkinson (reivilibre)
c26843c3bb Bleh logbugfix 2021-12-20 16:39:08 +00:00
Olivier Wilkinson (reivilibre)
4da1fdfd28 Change things around 2021-12-20 16:31:06 +00:00
Olivier Wilkinson (reivilibre)
7abb4bbe73 Log a little bit more 2021-12-20 16:21:33 +00:00
Olivier Wilkinson (reivilibre)
9858a2ca68 Add some logging and validation to help determine whether this all works or not :) 2021-12-20 13:21:30 +00:00
Olivier Wilkinson (reivilibre)
10bdbf1f5b Newsfile 
Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
 2021-12-20 13:05:45 +00:00
Olivier Wilkinson (reivilibre)
3ae955ea0b Iterate over in-flight requests in descending order 2021-12-20 12:57:47 +00:00
Olivier Wilkinson (reivilibre)
d9800c3f6f Use sorted dicts 
Note that they are sorted in ascending size order, so this needs further touch-ups.
 2021-12-20 12:55:44 +00:00
Olivier Wilkinson (reivilibre)
efe0059b98 Define a comparator for StateFilters 2021-12-20 12:54:25 +00:00
Olivier Wilkinson (reivilibre)
0142367bf2 Newsfile 
Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
 2021-12-20 12:26:34 +00:00
Olivier Wilkinson (reivilibre)
13a16f5d03 Shunt out with StateFilter.all() if many requests are made to the same group 2021-12-20 12:23:40 +00:00
Olivier Wilkinson (reivilibre)
232be1d25d Fix up misunderstanding (fixes tests) 2021-12-20 11:52:11 +00:00
Olivier Wilkinson (reivilibre)
8325ddd0bc Merge branch 'develop' into rei/gsfg_1 2021-12-20 11:42:10 +00:00
Dirk Klimpel
8428ef66c7 Add type hints to synapse/tests/rest/admin (#11590) 2021-12-16 14:59:56 -05:00
Dirk Klimpel
1847d027e6 Add type hints to synapse/storage/databases/main/transactions.py (#11589) 2021-12-16 14:59:35 -05:00
Eric Eastwood
43f5cc7adc Add MSC2716 and MSC3030 to /versions -> unstable_features (#11582) 
As suggested in https://github.com/matrix-org/matrix-react-sdk/pull/7372#discussion_r769523369
 2021-12-16 11:25:37 -06:00
Sean Quah
c7fe32edb4 Add type hints to synapse/storage/databases/main/room.py (#11575) 2021-12-15 18:00:48 +00:00
Patrick Cloke
f901f8b70e Require Collections as the parameters for simple_* methods. (#11580) 
Instead of Iterable since the generators are not allowed due
to the potential for their re-use.
 2021-12-15 17:00:50 +00:00
Patrick Cloke
323151b787 Convert EventStreamResult to attrs. (#11574) 2021-12-15 11:10:02 -05:00
reivilibre
17886d2603 Add experimental support for MSC3202: allowing application services to masquerade as specific devices. (#11538) 2021-12-15 10:40:52 +00:00
Sean Quah
ecfcd9bbbe Add type hints to synapse/storage/databases/main/e2e_room_keys.py (#11549) 2021-12-14 17:46:47 +00:00
Sean Quah
0147b3de20 Add missing type hints to synapse.logging.context (#11556) 2021-12-14 17:35:28 +00:00
Patrick Cloke
2519beaad2 Add missing type hints to synapse.appservice (#11360) 2021-12-14 17:02:46 +00:00
Olivier Wilkinson (reivilibre)
70ca05373b Merge branch 'master' into develop 2021-12-14 14:22:48 +00:00
Olivier Wilkinson (reivilibre)
a91698df90 Merge tag 'v1.49.0' 
Synapse 1.49.0 (2021-12-14)
===========================

No significant changes since version 1.49.0rc1.

Support for Ubuntu 21.04 ends next month on the 20th of January
---------------------------------------------------------------

For users of Ubuntu 21.04 (Hirsute Hippo), please be aware that [upstream support for this version of Ubuntu will end next month][Ubuntu2104EOL].
We will stop producing packages for Ubuntu 21.04 after upstream support ends.

[Ubuntu2104EOL]: https://lists.ubuntu.com/archives/ubuntu-announce/2021-December/000275.html

The wiki has been migrated to the documentation website
-------------------------------------------------------

We've decided to move the existing, somewhat stagnant pages from the GitHub wiki
to the [documentation website](https://matrix-org.github.io/synapse/latest/).

This was done for two reasons. The first was to ensure that changes are checked by
multiple authors before being committed (everyone makes mistakes!) and the second
was visibility of the documentation. Not everyone knows that Synapse has some very
useful information hidden away in its GitHub wiki pages. Bringing them to the
documentation website should help with visibility, as well as keep all Synapse documentation
in one, easily-searchable location.

Note that contributions to the documentation website happen through [GitHub pull
requests](https://github.com/matrix-org/synapse/pulls). Please visit [#synapse-dev:matrix.org](https://matrix.to/#/#synapse-dev:matrix.org)
if you need help with the process!

Synapse 1.49.0rc1 (2021-12-07)
==============================

Features
--------

- Add [MSC3030](https://github.com/matrix-org/matrix-doc/pull/3030) experimental client and federation API endpoints to get the closest event to a given timestamp. ([\#9445](https://github.com/matrix-org/synapse/issues/9445))
- Include bundled relation aggregations during a limited `/sync` request and `/relations` request, per [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675). ([\#11284](https://github.com/matrix-org/synapse/issues/11284), [\#11478](https://github.com/matrix-org/synapse/issues/11478))
- Add plugin support for controlling database background updates. ([\#11306](https://github.com/matrix-org/synapse/issues/11306), [\#11475](https://github.com/matrix-org/synapse/issues/11475), [\#11479](https://github.com/matrix-org/synapse/issues/11479))
- Support the stable API endpoints for [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946): the room `/hierarchy` endpoint. ([\#11329](https://github.com/matrix-org/synapse/issues/11329))
- Add admin API to get some information about federation status with remote servers. ([\#11407](https://github.com/matrix-org/synapse/issues/11407))
- Support expiry of refresh tokens and expiry of the overall session when refresh tokens are in use. ([\#11425](https://github.com/matrix-org/synapse/issues/11425))
- Stabilise support for [MSC2918](https://github.com/matrix-org/matrix-doc/blob/main/proposals/2918-refreshtokens.md#msc2918-refresh-tokens) refresh tokens as they have now been merged into the Matrix specification. ([\#11435](https://github.com/matrix-org/synapse/issues/11435), [\#11522](https://github.com/matrix-org/synapse/issues/11522))
- Update [MSC2918 refresh token](https://github.com/matrix-org/matrix-doc/blob/main/proposals/2918-refreshtokens.md#msc2918-refresh-tokens) support to confirm with the latest revision: accept the `refresh_tokens` parameter in the request body rather than in the URL parameters. ([\#11430](https://github.com/matrix-org/synapse/issues/11430))
- Support configuring the lifetime of non-refreshable access tokens separately to refreshable access tokens. ([\#11445](https://github.com/matrix-org/synapse/issues/11445))
- Expose `synapse_homeserver` and `synapse_worker` commands as entry points to run Synapse's main process and worker processes, respectively. Contributed by @Ma27. ([\#11449](https://github.com/matrix-org/synapse/issues/11449))
- `synctl stop` will now wait for Synapse to exit before returning. ([\#11459](https://github.com/matrix-org/synapse/issues/11459), [\#11490](https://github.com/matrix-org/synapse/issues/11490))
- Extend the "delete room" admin api to work correctly on rooms which have previously been partially deleted. ([\#11523](https://github.com/matrix-org/synapse/issues/11523))
- Add support for the `/_matrix/client/v3/login/sso/redirect/{idpId}` API from Matrix v1.1. This endpoint was overlooked when support for v3 endpoints was added in Synapse 1.48.0rc1. ([\#11451](https://github.com/matrix-org/synapse/issues/11451))

Bugfixes
--------

- Fix using [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) batch sending in combination with event persistence workers. Contributed by @tulir at Beeper. ([\#11220](https://github.com/matrix-org/synapse/issues/11220))
- Fix a long-standing bug where all requests that read events from the database could get stuck as a result of losing the database connection, properly this time. Also fix a race condition introduced in the previous insufficient fix in Synapse 1.47.0. ([\#11376](https://github.com/matrix-org/synapse/issues/11376))
- The `/send_join` response now includes the stable `event` field instead of the unstable field from [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083). ([\#11413](https://github.com/matrix-org/synapse/issues/11413))
- Fix a bug introduced in Synapse 1.47.0 where `send_join` could fail due to an outdated `ijson` version. ([\#11439](https://github.com/matrix-org/synapse/issues/11439), [\#11441](https://github.com/matrix-org/synapse/issues/11441), [\#11460](https://github.com/matrix-org/synapse/issues/11460))
- Fix a bug introduced in Synapse 1.36.0 which could cause problems fetching event-signing keys from trusted key servers. ([\#11440](https://github.com/matrix-org/synapse/issues/11440))
- Fix a bug introduced in Synapse 1.47.1 where the media repository would fail to work if the media store path contained any symbolic links. ([\#11446](https://github.com/matrix-org/synapse/issues/11446))
- Fix an `LruCache` corruption bug, introduced in Synapse 1.38.0, that would cause certain requests to fail until the next Synapse restart. ([\#11454](https://github.com/matrix-org/synapse/issues/11454))
- Fix a long-standing bug where invites from ignored users were included in incremental syncs. ([\#11511](https://github.com/matrix-org/synapse/issues/11511))
- Fix a regression in Synapse 1.48.0 where presence workers would not clear their presence updates over replication on shutdown. ([\#11518](https://github.com/matrix-org/synapse/issues/11518))
- Fix a regression in Synapse 1.48.0 where the module API's `looping_background_call` method would spam errors to the logs when given a non-async function. ([\#11524](https://github.com/matrix-org/synapse/issues/11524))

Updates to the Docker image
---------------------------

- Update `Dockerfile-workers` to healthcheck all workers in the container. ([\#11429](https://github.com/matrix-org/synapse/issues/11429))

Improved Documentation
----------------------

- Update the media repository documentation. ([\#11415](https://github.com/matrix-org/synapse/issues/11415))
- Update section about backward extremities in the room DAG concepts doc to correct the misconception about backward extremities indicating whether we have fetched an events' `prev_events`. ([\#11469](https://github.com/matrix-org/synapse/issues/11469))

Internal Changes
----------------

- Add `Final` annotation to string constants in `synapse.api.constants` so that they get typed as `Literal`s. ([\#11356](https://github.com/matrix-org/synapse/issues/11356))
- Add a check to ensure that users cannot start the Synapse master process when `worker_app` is set. ([\#11416](https://github.com/matrix-org/synapse/issues/11416))
- Add a note about postgres memory management and hugepages to postgres doc. ([\#11467](https://github.com/matrix-org/synapse/issues/11467))
- Add missing type hints to `synapse.config` module. ([\#11465](https://github.com/matrix-org/synapse/issues/11465))
- Add missing type hints to `synapse.federation`. ([\#11483](https://github.com/matrix-org/synapse/issues/11483))
- Add type annotations to `tests.storage.test_appservice`. ([\#11488](https://github.com/matrix-org/synapse/issues/11488), [\#11492](https://github.com/matrix-org/synapse/issues/11492))
- Add type annotations to some of the configuration surrounding refresh tokens. ([\#11428](https://github.com/matrix-org/synapse/issues/11428))
- Add type hints to `synapse/tests/rest/admin`. ([\#11501](https://github.com/matrix-org/synapse/issues/11501))
- Add type hints to storage classes. ([\#11411](https://github.com/matrix-org/synapse/issues/11411))
- Add wiki pages to documentation website. ([\#11402](https://github.com/matrix-org/synapse/issues/11402))
- Clean up `tests.storage.test_main` to remove use of legacy code. ([\#11493](https://github.com/matrix-org/synapse/issues/11493))
- Clean up `tests.test_visibility` to remove legacy code. ([\#11495](https://github.com/matrix-org/synapse/issues/11495))
- Convert status codes to `HTTPStatus` in `synapse.rest.admin`. ([\#11452](https://github.com/matrix-org/synapse/issues/11452), [\#11455](https://github.com/matrix-org/synapse/issues/11455))
- Extend the `scripts-dev/sign_json` script to support signing events. ([\#11486](https://github.com/matrix-org/synapse/issues/11486))
- Improve internal types in push code. ([\#11409](https://github.com/matrix-org/synapse/issues/11409))
- Improve type annotations in `synapse.module_api`. ([\#11029](https://github.com/matrix-org/synapse/issues/11029))
- Improve type hints for `LruCache`. ([\#11453](https://github.com/matrix-org/synapse/issues/11453))
- Preparation for database schema simplifications: disambiguate queries on `state_key`. ([\#11497](https://github.com/matrix-org/synapse/issues/11497))
- Refactor `backfilled` into specific behavior function arguments (`_persist_events_and_state_updates` and downstream calls). ([\#11417](https://github.com/matrix-org/synapse/issues/11417))
- Refactor `get_version_string` to fix-up types and duplicated code. ([\#11468](https://github.com/matrix-org/synapse/issues/11468))
- Refactor various parts of the `/sync` handler. ([\#11494](https://github.com/matrix-org/synapse/issues/11494), [\#11515](https://github.com/matrix-org/synapse/issues/11515))
- Remove unnecessary `json.dumps` from `tests.rest.admin`. ([\#11461](https://github.com/matrix-org/synapse/issues/11461))
- Save the OpenID Connect session ID on login. ([\#11482](https://github.com/matrix-org/synapse/issues/11482))
- Update and clean up recently ported documentation pages. ([\#11466](https://github.com/matrix-org/synapse/issues/11466))
 2021-12-14 14:22:16 +00:00
Olivier Wilkinson (reivilibre)
4dd9ea8f4f Revert "Revert accidental fast-forward merge from v1.49.0rc1" 
This reverts commit 158d73ebdd.
 2021-12-14 14:22:01 +00:00
Olivier Wilkinson (reivilibre)
92906e1b60 Restructure changelog v1.49.0 2021-12-14 13:00:46 +00:00
Olivier Wilkinson (reivilibre)
9f3c7e85a4 1.49.0 2021-12-14 12:56:14 +00:00
Sean Quah
a4dce5b53d Remove redundant COALESCE()s around COUNT()s in database queries (#11570) 
`COUNT()` never returns `NULL`. A `COUNT(*)` over 0 rows is 0 and a
`COUNT(NULL)` is also 0.
 2021-12-14 12:34:30 +00:00
Patrick Cloke
33abbc3278 Add missing type hints to synapse.http. (#11571) 2021-12-14 07:00:47 -05:00
Richard van der Hoff
ff6fd52160 checks for generators in database functions (#11564) 
A couple of safety-checks to hopefully stop people doing what I just did, and create a storage
function which only works the first time it is called (and not when it is re-run due to a database
concurrency error or similar).
 2021-12-13 19:01:27 +00:00
Patrick Cloke
eb39da6782 Move HTML parsing to a separate file for URL previews. (#11566) 
* Splits the logic for parsing HTML from the resource handling code.
* Fix a circular import in the oEmbed code (which uses the HTML parsing code).
* Renames some of the HTML parsing methods to:
  * Make it clear which methods are "internal" to the module.
  * Clarify what the methods do.
 2021-12-13 17:55:07 +00:00
Sean Quah
5305a5e881 Type hint the constructors of the data store classes (#11555) 2021-12-13 17:05:00 +00:00
Sean Quah
1abfb15f07 Add type hints to synapse/storage/databases/main/end_to_end_keys.py (#11551) 2021-12-13 16:28:26 +00:00
Sean Quah
6da8591f2e Add type hints to synapse/storage/databases/main/account_data.py (#11546) 2021-12-13 16:28:10 +00:00
reivilibre
e5cdb9e233 Make get_device return None if the device doesn't exist rather than raising an exception. (#11565) 
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
 2021-12-13 15:39:43 +00:00
Eric Eastwood
aa8708ebed Allow events to be created with no prev_events (MSC2716) (#11243) 
The event still needs to have `auth_events` defined to be valid.

Split out from https://github.com/matrix-org/synapse/pull/11114
 2021-12-10 23:08:51 -06:00
Eric Eastwood
8391bd6ab5 Test to ensure we share the same state_group across the whole historical batch (MSC2716) (#11487) 
Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716

We did some work on making sure the `state_groups` were shared in
https://github.com/matrix-org/synapse/pull/10975
 2021-12-10 20:59:20 -06:00
David Robertson
fd2dadb815 Adjust _get_rooms_changed comments (#11550) 
C.f. https://github.com/matrix-org/synapse/pull/11494#pullrequestreview-827780886
 2021-12-10 19:19:48 +00:00
Richard van der Hoff
f0562183e7 skip some dict munging in event persistence (#11560) 
Create a new dict helper method `simple_insert_many_values_txn`, which takes
raw row values, rather than {key=>value} dicts. This saves us a bunch of dict
munging, and makes it easier to use generators rather than creating
intermediate lists and dicts.
 2021-12-10 15:02:33 +00:00
Richard van der Hoff
86e7a6d16e Stop populating state_events.prev_state (#11558) 
this field is never read, so we may as well stop populating it.
 2021-12-10 14:13:23 +00:00
Patrick Cloke
9562f0c2f1 Ensure emails are canonicalized before fetching associated user. (#11547) 
This should fix pushers with an email in non-canonical form is used as
the pushkey.
 2021-12-10 07:17:28 -05:00
Patrick Cloke
3b8872299a Do not allow cross-room relations, per MSC2674. (#11516) 2021-12-09 13:16:01 -05:00
Patrick Cloke
0cc3bf97b4 Additional type hints for the config module, part 2. (#11480) 2021-12-09 16:15:46 +00:00
Dirk Klimpel
941ebe49ff Use HTTPStatus constants in place of literals in synapse.http (#11543) 2021-12-09 11:58:25 +00:00
Hubert Chathi
b47d10dc46 Support unprefixed versions of fallback key property names. (#11541) 2021-12-09 11:41:27 +00:00
Dirk Klimpel
b3bcacf3c1 Add missing errcode to parse_string and parse_boolean (#11542) 2021-12-09 11:23:34 +00:00
Robert Long
afa0a5e4fc Allow guests to send state events (#11378) 2021-12-09 11:02:05 +00:00
Patrick Cloke
d93362d87f Add a constant for receipt types (m.read). (#11531) 
And expand some type hints in the receipts storage module.
 2021-12-08 12:26:29 -05:00
Dirk Klimpel
7ecaa3b976 Clean up synapse.rest.admin (#11535) 2021-12-08 16:59:40 +00:00