kaslo/synapse
1
0
Fork 0
Code Wiki Activity
18,234 Commits 1,047 Branches 805 Tags
8965b6cfec8a1de847efe3d1be4b9babf4622e2e
Commit Graph

18234 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jason Robinson
8965b6cfec Merge branch 'develop' into jaywink/admin-forward-extremities 2021-01-23 21:41:35 +02:00
Jason Robinson
930ba00971 Add depth and received_ts to forward_extremities admin API response 
Also add a warning on the admin API documentation.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
 2021-01-23 21:34:32 +02:00
Patrick Cloke
a7882f9887 Return a 404 if no valid thumbnail is found. (#9163) 
If no thumbnail of the requested type exists, return a 404 instead
of erroring. This doesn't quite match the spec (which does not define
what happens if no thumbnail can be found), but is consistent with
what Synapse already does.
 2021-01-21 14:53:58 -05:00
Andrew Morgan
31c5382d7a Align the directories linted in CI with the defaults in scripts-dev/lint.sh (#9191) 
The lists of source directories to lint between `tox.ini` and `lint.sh` became out of sync. This PR tightens them up and adds some comments reminding any future readers to keep the list in sync.
 2021-01-21 18:26:52 +00:00
Erik Johnston
758ed5f1bc Speed up chain cover calculation (#9176) 2021-01-21 17:00:12 +00:00
Erik Johnston
12ec55bfaa Increase perf of handling concurrent use of StreamIDGenerators. (#9190) 
We have seen a failure mode here where if there are many in flight
unfinished IDs then marking an ID as finished takes a lot of CPU (as
calling deque.remove iterates over the list)
 2021-01-21 16:31:51 +00:00
Erik Johnston
939ef657ce Merge remote-tracking branch 'origin/release-v1.26.0' into develop 2021-01-21 16:05:13 +00:00
Erik Johnston
ccfafac882 Add schema update to fix existing DBs affected by #9193 (#9195) 2021-01-21 16:03:25 +00:00
Erik Johnston
b249f002b8 Merge remote-tracking branch 'origin/release-v1.26.0' into develop 2021-01-21 15:09:30 +00:00
Erik Johnston
2506074ef0 Fix receipts or account data not being sent down sync (#9193) 
Introduced in #9104 

This wasn't picked up by the tests as this is all fine the first time you run Synapse (after upgrading), but then when you restart the wrong value is pulled from `stream_positions`.
 2021-01-21 15:09:09 +00:00
Erik Johnston
7a43482f19 Use execute_batch in more places (#9188) 
* Use execute_batch in more places

* Newsfile
 2021-01-21 14:44:12 +00:00
Dirk Klimpel
c55e62548c Add tests for List Users Admin API (#9045) 2021-01-21 09:18:46 -05:00
Richard van der Hoff
42a8e81370 Add a check for duplicate IdP ids (#9184) 2021-01-21 13:20:58 +00:00
Richard van der Hoff
b5120f09f1 Merge remote-tracking branch 'origin/release-v1.26.0' into develop 2021-01-21 13:17:07 +00:00
Richard van der Hoff
7447f19702 Prefix idp_id with "oidc-" (#9189) 
... to avoid clashes with other SSO mechanisms
 2021-01-21 12:25:02 +00:00
Erik Johnston
eee6fcf5fa Use execute_batch instead of executemany in places (#9181) 
`execute_batch` does fewer round trips in postgres than `executemany`, but does not give a correct `txn.rowcount` result after.
 2021-01-21 10:22:53 +00:00
Patrick Cloke
1fa15b74e0 Specify that the long description is rST in the package metadata. (#9180) 
This avoids a warning when uploading packages to PyPI via twine.
 2021-01-20 16:00:59 -05:00
Patrick Cloke
937b849a2e Fix a typo in the release notes. 2021-01-20 11:34:34 -05:00
Patrick Cloke
818bf313bc Merge branch 'release-v1.26.0' into develop 2021-01-20 11:32:06 -05:00
Patrick Cloke
f81d02d75b Merge tag 'v1.26.0rc1' into develop 
Synapse 1.26.0rc1 (2021-01-20)
==============================

This release brings a new schema version for Synapse and rolling back to a previous
verious is not trivial.  Please review [UPGRADE.rst](UPGRADE.rst) for more details
on these changes  and for general upgrade guidance.

Features
--------

- Add support for multiple SSO Identity Providers. ([\#9015](https://github.com/matrix-org/synapse/issues/9015), [\#9017](https://github.com/matrix-org/synapse/issues/9017), [\#9036](https://github.com/matrix-org/synapse/issues/9036), [\#9067](https://github.com/matrix-org/synapse/issues/9067), [\#9081](https://github.com/matrix-org/synapse/issues/9081), [\#9082](https://github.com/matrix-org/synapse/issues/9082), [\#9105](https://github.com/matrix-org/synapse/issues/9105), [\#9107](https://github.com/matrix-org/synapse/issues/9107), [\#9109](https://github.com/matrix-org/synapse/issues/9109), [\#9110](https://github.com/matrix-org/synapse/issues/9110), [\#9127](https://github.com/matrix-org/synapse/issues/9127), [\#9153](https://github.com/matrix-org/synapse/issues/9153), [\#9154](https://github.com/matrix-org/synapse/issues/9154), [\#9177](https://github.com/matrix-org/synapse/issues/9177))
- During user-interactive authentication via single-sign-on, give a better error if the user uses the wrong account on the SSO IdP. ([\#9091](https://github.com/matrix-org/synapse/issues/9091))
- Give the `public_baseurl` a default value, if it is not explicitly set in the configuration file. ([\#9159](https://github.com/matrix-org/synapse/issues/9159))
- Improve performance when calculating ignored users in large rooms. ([\#9024](https://github.com/matrix-org/synapse/issues/9024))
- Implement [MSC2176](https://github.com/matrix-org/matrix-doc/pull/2176) in an experimental room version. ([\#8984](https://github.com/matrix-org/synapse/issues/8984))
- Add an admin API for protecting local media from quarantine. ([\#9086](https://github.com/matrix-org/synapse/issues/9086))
- Remove a user's avatar URL and display name when deactivated with the Admin API. ([\#8932](https://github.com/matrix-org/synapse/issues/8932))
- Update `/_synapse/admin/v1/users/<user_id>/joined_rooms` to work for both local and remote users. ([\#8948](https://github.com/matrix-org/synapse/issues/8948))
- Add experimental support for handling to-device messages on worker processes. ([\#9042](https://github.com/matrix-org/synapse/issues/9042), [\#9043](https://github.com/matrix-org/synapse/issues/9043), [\#9044](https://github.com/matrix-org/synapse/issues/9044), [\#9130](https://github.com/matrix-org/synapse/issues/9130))
- Add experimental support for handling `/keys/claim` and `/room_keys` APIs on worker processes. ([\#9068](https://github.com/matrix-org/synapse/issues/9068))
- Add experimental support for handling `/devices` API on worker processes. ([\#9092](https://github.com/matrix-org/synapse/issues/9092))
- Add experimental support for moving off receipts and account data persistence off master. ([\#9104](https://github.com/matrix-org/synapse/issues/9104), [\#9166](https://github.com/matrix-org/synapse/issues/9166))

Bugfixes
--------

- Fix a long-standing issue where an internal server error would occur when requesting a profile over federation that did not include a display name / avatar URL. ([\#9023](https://github.com/matrix-org/synapse/issues/9023))
- Fix a long-standing bug where some caches could grow larger than configured. ([\#9028](https://github.com/matrix-org/synapse/issues/9028))
- Fix error handling during insertion of client IPs into the database. ([\#9051](https://github.com/matrix-org/synapse/issues/9051))
- Fix bug where we didn't correctly record CPU time spent in `on_new_event` block. ([\#9053](https://github.com/matrix-org/synapse/issues/9053))
- Fix a minor bug which could cause confusing error messages from invalid configurations. ([\#9054](https://github.com/matrix-org/synapse/issues/9054))
- Fix incorrect exit code when there is an error at startup. ([\#9059](https://github.com/matrix-org/synapse/issues/9059))
- Fix `JSONDecodeError` spamming the logs when sending transactions to remote servers. ([\#9070](https://github.com/matrix-org/synapse/issues/9070))
- Fix "Failed to send request" errors when a client provides an invalid room alias. ([\#9071](https://github.com/matrix-org/synapse/issues/9071))
- Fix bugs in federation catchup logic that caused outbound federation to be delayed for large servers after start up. Introduced in v1.8.0 and v1.21.0. ([\#9114](https://github.com/matrix-org/synapse/issues/9114), [\#9116](https://github.com/matrix-org/synapse/issues/9116))
- Fix corruption of `pushers` data when a postgres bouncer is used. ([\#9117](https://github.com/matrix-org/synapse/issues/9117))
- Fix minor bugs in handling the `clientRedirectUrl` parameter for SSO login. ([\#9128](https://github.com/matrix-org/synapse/issues/9128))
- Fix "Unhandled error in Deferred: BodyExceededMaxSize" errors when .well-known files that are too large. ([\#9108](https://github.com/matrix-org/synapse/issues/9108))
- Fix "UnboundLocalError: local variable 'length' referenced before assignment" errors when the response body exceeds the expected size. This bug was introduced in v1.25.0. ([\#9145](https://github.com/matrix-org/synapse/issues/9145))
- Fix a long-standing bug "ValueError: invalid literal for int() with base 10" when `/publicRooms` is requested with an invalid `server` parameter. ([\#9161](https://github.com/matrix-org/synapse/issues/9161))

Improved Documentation
----------------------

- Add some extra docs for getting Synapse running on macOS. ([\#8997](https://github.com/matrix-org/synapse/issues/8997))
- Correct a typo in the `systemd-with-workers` documentation. ([\#9035](https://github.com/matrix-org/synapse/issues/9035))
- Correct a typo in `INSTALL.md`. ([\#9040](https://github.com/matrix-org/synapse/issues/9040))
- Add missing `user_mapping_provider` configuration to the Keycloak OIDC example. Contributed by @chris-ruecker. ([\#9057](https://github.com/matrix-org/synapse/issues/9057))
- Quote `pip install` packages when extras are used to avoid shells interpreting bracket characters. ([\#9151](https://github.com/matrix-org/synapse/issues/9151))

Deprecations and Removals
-------------------------

- Remove broken and unmaintained `demo/webserver.py` script. ([\#9039](https://github.com/matrix-org/synapse/issues/9039))

Internal Changes
----------------

- Improve efficiency of large state resolutions. ([\#8868](https://github.com/matrix-org/synapse/issues/8868), [\#9029](https://github.com/matrix-org/synapse/issues/9029), [\#9115](https://github.com/matrix-org/synapse/issues/9115), [\#9118](https://github.com/matrix-org/synapse/issues/9118), [\#9124](https://github.com/matrix-org/synapse/issues/9124))
- Various clean-ups to the structured logging and logging context code. ([\#8939](https://github.com/matrix-org/synapse/issues/8939))
- Ensure rejected events get added to some metadata tables. ([\#9016](https://github.com/matrix-org/synapse/issues/9016))
- Ignore date-rotated homeserver logs saved to disk. ([\#9018](https://github.com/matrix-org/synapse/issues/9018))
- Remove an unused column from `access_tokens` table. ([\#9025](https://github.com/matrix-org/synapse/issues/9025))
- Add a `-noextras` factor to `tox.ini`, to support running the tests with no optional dependencies. ([\#9030](https://github.com/matrix-org/synapse/issues/9030))
- Fix running unit tests when optional dependencies are not installed. ([\#9031](https://github.com/matrix-org/synapse/issues/9031))
- Allow bumping schema version when using split out state database. ([\#9033](https://github.com/matrix-org/synapse/issues/9033))
- Configure the linters to run on a consistent set of files. ([\#9038](https://github.com/matrix-org/synapse/issues/9038))
- Various cleanups to device inbox store. ([\#9041](https://github.com/matrix-org/synapse/issues/9041))
- Drop unused database tables. ([\#9055](https://github.com/matrix-org/synapse/issues/9055))
- Remove unused `SynapseService` class. ([\#9058](https://github.com/matrix-org/synapse/issues/9058))
- Remove unnecessary declarations in the tests for the admin API. ([\#9063](https://github.com/matrix-org/synapse/issues/9063))
- Remove `SynapseRequest.get_user_agent`. ([\#9069](https://github.com/matrix-org/synapse/issues/9069))
- Remove redundant `Homeserver.get_ip_from_request` method. ([\#9080](https://github.com/matrix-org/synapse/issues/9080))
- Add type hints to media repository. ([\#9093](https://github.com/matrix-org/synapse/issues/9093))
- Fix the wrong arguments being passed to `BlacklistingAgentWrapper` from `MatrixFederationAgent`. Contributed by Timothy Leung. ([\#9098](https://github.com/matrix-org/synapse/issues/9098))
- Reduce the scope of caught exceptions in `BlacklistingAgentWrapper`. ([\#9106](https://github.com/matrix-org/synapse/issues/9106))
- Improve `UsernamePickerTestCase`. ([\#9112](https://github.com/matrix-org/synapse/issues/9112))
- Remove dependency on `distutils`. ([\#9125](https://github.com/matrix-org/synapse/issues/9125))
- Enforce that replication HTTP clients are called with keyword arguments only. ([\#9144](https://github.com/matrix-org/synapse/issues/9144))
- Fix the Python 3.5 / old dependencies build in CI. ([\#9146](https://github.com/matrix-org/synapse/issues/9146))
- Replace the old `perspectives` option in the Synapse docker config file template with `trusted_key_servers`. ([\#9157](https://github.com/matrix-org/synapse/issues/9157))
 2021-01-20 11:27:39 -05:00
Patrick Cloke
4c37d2acd5 Fix reStructuredText formatting. 2021-01-20 11:10:00 -05:00
Patrick Cloke
adabf328ac Point people to the upgrade notes. v1.26.0rc1 2021-01-20 11:01:04 -05:00
Patrick Cloke
933f258967 Formatting. 2021-01-20 10:54:06 -05:00
Patrick Cloke
d5349959f4 SQL formatting in UPGRADE. 2021-01-20 10:53:39 -05:00
Patrick Cloke
1b37107cac Add upgrade notes about chain cover. 2021-01-20 10:52:55 -05:00
Patrick Cloke
c8e6e05842 Include upgrade notes. 2021-01-20 10:37:09 -05:00
Patrick Cloke
7e072d38b1 Re-run towncrier. 2021-01-20 08:56:25 -05:00
Richard van der Hoff
e51b2f3f91 Tighten the restrictions on idp_id (#9177) 2021-01-20 08:55:14 -05:00
Richard van der Hoff
0cd2938bc8 Support icons for Identity Providers (#9154) 2021-01-20 08:15:14 -05:00
Patrick Cloke
620ecf13b0 Various improvements to the federation client. (#9129) 
* Type hints for `FederationClient`.
* Using `async` functions instead of returning `Awaitable` instances.
 2021-01-20 07:59:18 -05:00
rht
a5b9c87ac6 docs: Add link to Matrix VoIP tester for turn-howto (#9135) 
Signed-off-by: rht <rhtbot@protonmail.com>
 2021-01-20 12:41:57 +00:00
Richard van der Hoff
6c0dfd2e8e Merge remote-tracking branch 'origin/develop' into release-v1.26.0 2021-01-20 12:33:05 +00:00
Richard van der Hoff
fa50e4bf4d Give public_baseurl a default value (#9159) 2021-01-20 12:30:41 +00:00
Patrick Cloke
5b8ee181b7 Reword confusing sentence in CHANGES. 2021-01-20 07:30:34 -05:00
Patrick Cloke
74ced7d070 Fix tenses in CHANGES. 2021-01-20 07:24:37 -05:00
Patrick Cloke
3005a2816c Re-arrange CHANGES.md. 2021-01-20 07:19:42 -05:00
Patrick Cloke
72822e60be 1.26.0rc1 2021-01-20 07:14:25 -05:00
Patrick Cloke
fa842a9866 Use the account handler in additional places. (#9166) 2021-01-20 10:44:52 +00:00
Patrick Cloke
47d48a5853 Validate the server name for the /publicRooms endpoint. (#9161) 
If a remote server name is provided, ensure it is something reasonable
before making remote connections to it.
 2021-01-19 14:21:59 -05:00
Andrew Morgan
94549771f7 Replace 'perspectives' config block with 'trusted_key_servers' in docker homeserver.yaml template (#9157) 2021-01-19 16:19:32 +00:00
Richard van der Hoff
73b03722f4 Fix error messages from OIDC config parsing (#9153) 
Make sure we report the correct config path for errors in the OIDC configs.
 2021-01-19 14:56:54 +00:00
Patrick Cloke
de45bf5b5b Quote pip install with brackets to avoid shell interpretation. (#9151) 2021-01-18 11:12:20 -05:00
Erik Johnston
6633a4015a Allow moving account data and receipts streams off master (#9104) 2021-01-18 15:47:59 +00:00
Erik Johnston
f08ef64926 Enforce all replication HTTP clients calls use kwargs (#9144) 2021-01-18 15:24:04 +00:00
Patrick Cloke
2b467d0b61 Properly raise an exception when the body exceeds the max size. (#9145) 
...instead of just creating the exception object and doing nothing with it.
 2021-01-18 10:21:42 -05:00
Richard van der Hoff
02070c69fa Fix bugs in handling clientRedirectUrl, and improve OIDC tests (#9127, #9128) 
* Factor out a common TestHtmlParser

Looks like I'm doing this in a few different places.

* Improve OIDC login test

Complete the OIDC login flow, rather than giving up halfway through.

* Ensure that OIDC login works with multiple OIDC providers

* Fix bugs in handling clientRedirectUrl

 - don't drop duplicate query-params, or params with no value
 - allow utf-8 in query-params
 2021-01-18 14:52:49 +00:00
Patrick Cloke
a8703819eb Fix the Python 3.5 old-deps build. (#9146) 
setuptools 51.0.0 dropped support for Python 3.5.
 2021-01-18 09:35:42 -05:00
Patrick Cloke
de1f8de319 Ensure the user ID is serialized in the payload instead of used as an instance name. (#9130) 2021-01-18 11:08:26 +00:00
Matthew Hodgson
883d4e6f2b link to the scalability blog post from workers.md 2021-01-18 00:27:27 +00:00
Richard van der Hoff
b5dea8702d Fix test failure due to bad merge 
0dd2649c1 (#9112) changed the signature of `auth_via_oidc`. Meanwhile,
26d10331e (#9091) introduced a new test which relied on the old signature of
`auth_via_oidc`. The two branches were never tested together until they landed
in develop.
 2021-01-15 18:03:33 +00:00