1
0

Compare commits

...

149 Commits

Author SHA1 Message Date
David Robertson 4d343db081 Get rid of my home dir, whoops 2021-11-16 16:37:43 +00:00
David Robertson a1367dcf8c Require networkx 2021-11-16 16:34:33 +00:00
David Robertson 9e361c8550 Changelog 2021-11-16 13:52:59 +00:00
David Robertson 51fec1a534 Commit hacky script to visualise store inheritance
Use e.g. with `scripts-dev/storage_inheritance.py DataStore --show`.
2021-11-16 13:51:50 +00:00
David Robertson e605e4b8f2 Database storage profile passes mypy (#11342)
It already seems to pass mypy. I wonder what changed, given that it was
on the exclusion list. So this commit consists of me ensuring
`--disallow-untyped-defs` passes and a minor fixup to a function that
returned either `True` or `None`.
2021-11-15 12:59:33 +00:00
David Robertson 5562ce6a53 Get directory db file to pass mypy (#11339) 2021-11-15 12:59:05 +00:00
Tulir Asokan 6f862c5c28 Add support for the stable version of MSC2778 (#11335)
* Add support for the stable version of MSC2778

Signed-off-by: Tulir Asokan <tulir@maunium.net>

* Expect m.login.application_service in login and password provider tests

Signed-off-by: Tulir Asokan <tulir@maunium.net>
2021-11-15 10:31:22 +00:00
Shay 605921bc6b Remove unused tables room_stats_historical and user_stats_historical (#11280)
* remove unused tables room_stats_historical and user_stats_historical

* update changelog number

* Bump schema compat version comment

* make linter happy

* Update comment to give more info

Co-authored-by: reivilibre <oliverw@matrix.org>

Co-authored-by: reivilibre <oliverw@matrix.org>
2021-11-12 16:47:56 -08:00
David Robertson fe58672546 Annotations for state_deltas.py (#11316)
I was sad that I couldn't do better for
`_curr_state_delta_stream_cache`. At least it's explicitly called out in
a comment with #TODO.
2021-11-12 20:24:12 +00:00
Patrick Cloke 3fad4e3fe5 Rollback #11322 due to wrong syntax in mypy.ini. (#11332)
This was only checking the __init__ files in modules instead of
all files in a module, which don't pass yet.
2021-11-12 20:10:03 +00:00
David Robertson bea815cec8 Test room alias deletion (#11327)
* Prefer `HTTPStatus` over plain `int`

This is an Opinion that no-one has seemed to object to yet.

* `--disallow-untyped-defs` for `tests.rest.client.test_directory`
* Improve synapse's annotations for deleting aliases
* Test case for deleting a room alias
* Changelog
2021-11-12 19:56:00 +00:00
Shay 0bcae8ad56 Change display names/avatar URLs to None if they contain null bytes before storing in DB (#11230)
* change display names/avatar URLS to None if they contain null bytes

* add changelog

* add POC test, requested changes

* add a saner test and remove old one

* update test to verify that display name has been changed to None

* make test less fragile
2021-11-12 10:38:24 -08:00
Patrick Cloke 9b90b9454b Add type hints to media repository storage module (#11311) 2021-11-12 11:05:26 -05:00
David Robertson 6f8f3d4bc5 Attempt to annotate events_forward_extremities (#11314)
* Make DataStore inherit from EventForwardExtremitiesStore before CacheInvalidationWorkerStore

the former implicitly inherits from the latter, so they should be
ordered like this when used.
2021-11-12 15:58:17 +00:00
David Robertson 4c96ce396e Misc typing fixes for tests, part 1 of N (#11323)
* Annotate HomeserverTestCase.servlets
* Correct annotation of federation_auth_origin
* Use AnyStr custom_headers instead of a Union

This allows (str, str) and (bytes, bytes).
This disallows (str, bytes) and (bytes, str)

* DomainSpecificString.SIGIL is a ClassVar
2021-11-12 15:50:54 +00:00
Patrick Cloke 95547e5300 Generalize the disallowed_untyped_defs in mypy.ini (#11322) 2021-11-12 14:27:45 +00:00
Patrick Cloke b64b6d12d4 Add more type hints to synapse.util. (#11321) 2021-11-12 13:43:06 +00:00
reivilibre 2fffcb24d8 Suggest using /etc/matrix-synapse/conf.d/ for configuration with Debian packages (#11281) 2021-11-12 13:17:06 +00:00
Dirk Klimpel 8840a7b7f1 Convert delete room admin API to async endpoint (#11223)
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-11-12 12:35:31 +00:00
David Robertson c99da2d079 Annotations for user_erasure_store (#11313)
I'm not sure why this was excluded---it seemed to be passing for me. But
it's easy enough to fixup.
2021-11-11 19:22:19 +00:00
David Robertson 6a605f4a77 Get db signatures file to pass mypy (#11312) 2021-11-11 17:04:44 +00:00
David Robertson 8dc666f785 Correct type hint for room_batch.py (#11310)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-11-11 16:49:28 +00:00
Dirk Klimpel 48278a0d09 Move sql file for remove_deleted_devices_from_device_inbox into v65 (#11303) 2021-11-11 15:01:13 +00:00
Patrick Cloke 64ef25391d Add type hints to some storage classes (#11307) 2021-11-11 08:47:31 -05:00
Neeeflix 6ce19b94e8 Fix error in thumbnail generation (#11288)
Signed-off-by: Jonas Zeunert <jonas@zeunert.org>
2021-11-10 20:49:43 +00:00
Patrick Cloke 5cace20bf1 Add missing type hints to synapse.app. (#11287) 2021-11-10 15:06:54 -05:00
Patrick Cloke 66c4b774fd Add type hints to synapse._scripts (#11297) 2021-11-10 17:55:32 +00:00
Andrew Morgan 5f277ffe89 Add documentation page stubs for Single Sign-On, SAML and CAS pages (#11298) 2021-11-10 17:54:56 +00:00
Richard van der Hoff 73cbb284b9 Remove redundant parameters on _check_event_auth (#11292)
as of #11012, these parameters are unused.
2021-11-10 14:16:06 +00:00
Olivier Wilkinson (reivilibre) 68c258a604 Merge tag 'v1.47.0rc2' into develop
Synapse 1.47.0rc2 (2021-11-10)
==============================

This fixes an issue with publishing the Debian packages for 1.47.0rc1.
It is otherwise identical to 1.47.0rc1.
2021-11-10 13:01:08 +00:00
Olivier Wilkinson (reivilibre) 595f28529c Changelog tweak from feedback 2021-11-10 09:54:34 +00:00
Olivier Wilkinson (reivilibre) ef7f9286d1 Move Debian changelog entries to rc2 since rc1 was not published 2021-11-10 09:48:50 +00:00
Olivier Wilkinson (reivilibre) 82e62b488a 1.47.0rc2 2021-11-10 09:44:38 +00:00
Olivier Wilkinson (reivilibre) af6374905a Correct the Debian changelog 2021-11-10 09:37:48 +00:00
Stanislav Motylkov b09d90cac9 Fix typos in the username_available admin API documentation. (#11286) 2021-11-09 21:11:05 +00:00
Eric Eastwood f1d5c2f269 Split out federated PDU retrieval into a non-cached version (#11242)
Context: https://github.com/matrix-org/synapse/pull/11114/files#r741643968
2021-11-09 15:07:57 -06:00
Patrick Cloke 0ef69ddbdc Ignore missing imports for parameterized. (#11285)
This was due to a conflict between #11282, which changed
mypy configuration, and #11228, a normal change.
2021-11-09 19:04:53 +00:00
Dan Callahan 3b951445a7 Require mypy for synapse/ & tests/ unless excluded (#11282)
Signed-off-by: Dan Callahan <danc@element.io>
2021-11-09 16:22:47 +00:00
Andrew Morgan a026695083 Clarifications and small fixes to to-device related code (#11247)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-11-09 14:31:15 +00:00
Olivier Wilkinson (reivilibre) dc5f524974 Update __init__.py 2021-11-09 13:51:08 +00:00
Olivier Wilkinson (reivilibre) a754510f28 Changelog tweaks from review 2021-11-09 13:22:36 +00:00
David Robertson b6f4d122ef Allow admins to proactively block rooms (#11228)
Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-11-09 13:11:47 +00:00
Patrick Cloke a19d01c3d9 Support filtering by relations per MSC3440 (#11236)
Adds experimental support for `relation_types` and `relation_senders`
fields for filters.
2021-11-09 08:10:58 -05:00
Olivier Wilkinson (reivilibre) b67a7c62a2 Make Deprecations and Removals more prominent 2021-11-09 12:32:05 +00:00
Olivier Wilkinson (reivilibre) 1a4f10045f Changelog tweaks 2021-11-09 12:30:15 +00:00
Olivier Wilkinson (reivilibre) 01f61da77f 1.47.0rc1 2021-11-09 12:17:35 +00:00
Andrew Morgan 4b3e30c276 Fix typo in RelationAggregationPaginationServlet error response (#11278) 2021-11-09 12:11:50 +00:00
Erik Johnston af784644c3 Include cross-signing signatures when syncing remote devices for the first time (#11234)
When fetching remote devices for the first time, we did not correctly include the cross signing keys in the returned results.

c.f. #11159
2021-11-09 11:45:36 +00:00
rogersheu 820337e6a4 Require body for read receipts with user-agent exceptions (#11157)
Co-authored-by: reivilibre <olivier@librepush.net>
2021-11-09 10:26:07 +00:00
Eric Eastwood 84f235aea4 Rename to more clear get_insertion_event_id_by_batch_id (MSC2716) (#11244)
`get_insertion_event_by_batch_id` -> `get_insertion_event_id_by_batch_id`

Split out from https://github.com/matrix-org/synapse/pull/11114
2021-11-08 21:21:10 -06:00
Erik Johnston 4ee71b9637 Add some background update admin APIs (#11263)
Fixes #11259
2021-11-08 16:08:02 +00:00
Patrick Cloke 0c82d4aabe Fix typo in comment from #11255. (#11276) 2021-11-08 14:36:49 +00:00
Richard van der Hoff 86a497efaa Default value for public_baseurl (#11210)
We might as well use a default value for `public_baseurl` based on
`server_name` - in many cases, it will be correct.
2021-11-08 14:13:10 +00:00
Dan Callahan 556a488209 Address review feedback from #11269 (#11273)
Signed-off-by: Dan Callahan <danc@element.io>
2021-11-08 11:57:37 +00:00
Erik Johnston a55e1ec9af Blacklist new sytest validation test (#11270) 2021-11-08 10:37:43 +00:00
Erik Johnston 98c8fc6ce8 Handle federation inbound instances being killed more gracefully (#11262)
* Make lock better handle process being killed

If the process gets killed and restarted (so that it didn't have a
chance to drop its locks gracefully) then there may still be locks in
the DB that are for the same instance that haven't yet timed out but are
safe to delete.

We handle this case by a) checking if the current instance already has
taken out the lock, and b) if not then ignoring locks that are for the
same instance.

* Periodically check for old staged events

This is to protect against other instances dying and their locks timing
out.
2021-11-08 09:54:47 +00:00
Dan Callahan 9799c569bb Minor cleanup to Debian packaging (#11269)
* Remove unused Vagrant scripts

* Change package Architecture to any

* Preinstall the wheel package when building venvs.

Addresses the following warnings during Debian builds:

    Using legacy 'setup.py install' for jaeger-client, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for matrix-synapse-ldap3, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for opentracing, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for psycopg2, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for systemd-python, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for pympler, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for threadloop, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for thrift, since package 'wheel' is not installed.

* Allow /etc/default/matrix-synapse to be missing

Per the systemd.exec manpage, prefixing an EnvironmentFile with "-":

> indicates that if the file does not exist, it will not be read and no
> error or warning message is logged.

Signed-off-by: Dan Callahan <danc@element.io>
2021-11-07 21:18:33 +00:00
Julian 09cb441a04 Add doc to integrate synapse with LemonLDAP OIDC (#11257)
Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
Co-authored-by: Julian Vanden Broeck <julian.vandenbroeck@dalibo.com>
2021-11-05 12:08:02 +00:00
Erik Johnston a37df1b091 Fix rolling back when using workers (#11255)
Fixes #11252
2021-11-05 11:12:10 +00:00
reivilibre 499c44d696 Make minor correction to type of auth_checkers callbacks (#11253) 2021-11-04 17:10:11 +00:00
Richard van der Hoff f36434590c Additional test for cachedList (#11246)
I was trying to understand how `cachedList` works, and ended up writing this
extra test. I figure we may as well keep it.
2021-11-04 14:45:34 +00:00
Sean Quah 8eec25a1d9 Track ongoing event fetches correctly in the presence of failure (#11240)
When an event fetcher aborts due to an exception, `_event_fetch_ongoing`
must be decremented, otherwise the event fetcher would never be
replaced. If enough event fetchers were to fail, no more events would be
fetched and requests would get stuck waiting for events.
2021-11-04 10:33:53 +00:00
Nick Barrett a271e233e9 Add a linearizer on (appservice, stream) when handling ephemeral events. (#11207)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-11-03 16:51:00 +00:00
Nick Barrett af54167516 Enable passing typing stream writers as a list. (#11237)
This makes the typing stream writer config match the other stream writers
that only currently support a single worker.
2021-11-03 14:25:47 +00:00
Patrick Cloke 2735b3e6f2 Remove a debug statement from tests. (#11239) 2021-11-03 13:11:16 +00:00
Erik Johnston bcc115c28d Add twine and towncrier as dev dependencies (#11233)
We don't pin them as we execute them as commands, rather than use them
as libs.
2021-11-03 11:10:25 +00:00
Andrew Morgan d688a6dee5 fix a small typo in the delete room api docs 2021-11-03 11:09:00 +00:00
Eric Eastwood da0040785e Support sending no state_events_at_start in the MSC2716 /batch_send endpoint (#11188)
As brought up by @tulir, https://matrix.to/#/!SBYNQlpqkwJzFIdzxI:nevarro.space/$Gwnb2ZvXHc3poYXuBhho0cmoYq4KJ11Jh3m5s8kjNOM?via=nevarro.space&via=beeper.com&via=matrix.org

This use case only works if the user is already joined in the current room state
at the given `?prev_event_id`
2021-11-03 03:13:51 -05:00
Erik Johnston 6250b95efe Add index to local_group_updates.stream_id (#11231)
This should speed up startup times and generally increase performance of
groups.
2021-11-02 15:46:48 +00:00
Erik Johnston 237f7eb87a Merge remote-tracking branch 'origin/master' into develop 2021-11-02 14:28:27 +00:00
Patrick Cloke c01bc5f43d Add remaining type hints to synapse.events. (#11098) 2021-11-02 09:55:52 -04:00
Erik Johnston 2d44ee6868 Update changelog 2021-11-02 13:25:42 +00:00
Erik Johnston df84ad602b 1.46.0 2021-11-02 13:23:01 +00:00
Dirk Klimpel 4535532526 Delete messages for hidden devices from device_inbox (#11199) 2021-11-02 13:18:30 +00:00
Andrew Morgan c9c3aea9b1 Fix providing a RoomStreamToken instance to _notify_app_services_ephemeral (#11137)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-11-02 10:39:02 +00:00
Dirk Klimpel 7537201840 Add search by room ID and room alias to List Room admin API (#11099)
Fixes: #10874
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-11-02 10:01:13 +00:00
Richard van der Hoff 46d0937447 ObservableDeferred: run observers in order (#11229) 2021-11-02 00:17:35 +00:00
Dirk Klimpel 93aa670642 Update outdated links in PULL_REQUEST_TEMPLATE.md (#11225) 2021-11-01 17:29:51 +00:00
Shay f5c6a80886 Handle missing Content-Type header when accessing remote media (#11200)
* add code to handle missing content-type header and a test to verify that it works

* add handling for missing content-type in the /upload endpoint as well

* slightly refactor test code to put private method in approriate place

* handle possible null value for content-type when pulling from the local db

* add changelog

* refactor test and add code to handle missing content-type in cached remote media

* requested changes

* Update changelog.d/11200.bugfix

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2021-11-01 10:26:02 -07:00
Shay e81fa92648 Add use_float=true to ijson calls in Synapse (#11217)
* add use_float=true to ijson calls

* lints

* add changelog

* Update changelog.d/11217.bugfix

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2021-11-01 09:28:04 -07:00
Dirk Klimpel caa706d825 Fix a bug in unit test test_block_room_and_not_purge (#11226) 2021-11-01 16:10:09 +00:00
reivilibre 69ab3dddbc Make check_event_allowed module API callback not fail open (accept events) when an exception is raised (#11033) 2021-11-01 15:45:56 +00:00
Dirk Klimpel 66bdca3e31 Remove deprecated delete room admin API (#11213)
Remove deprecated delete room admin API,
`POST /_synapse/admin/v1/rooms/<room_id>/delete`
2021-11-01 15:11:24 +00:00
Richard van der Hoff 71f9966f27 Support for serving server well-known files (#11211)
Fixes https://github.com/matrix-org/synapse/issues/8308
2021-11-01 15:10:16 +00:00
Brett Bethke 2014098d01 Add domain specific matching for haproxy config (#11128) 2021-11-01 14:16:02 +00:00
Richard van der Hoff 0b99d4c8d2 Docker: avoid changing userid unnecessarily (#11209)
* Docker image: avoid changing user during `generate`

The intention was always that the config files get written as the initial user
(normally root) - only the data directory needs to be writable by Synapse. This
got changed in https://github.com/matrix-org/synapse/pull/5970, but that seems
to have been a mistake.

* Avoid changing user if no explicit UID is given

* changelog
2021-11-01 13:55:30 +00:00
Aaron R 3ae1464efd Support Client-Server API r0.6.1 (#11097)
Fixes #11064

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2021-11-01 13:28:39 +00:00
Sumner Evans ece84f2c45 Improve code formatting and fix a few typos in docs (#11221)
* Labeled a lot more code blocks with the appropriate type
* Fixed a couple of minor typos (missing/extraneous commas)

Signed-off-by: Sumner Evans <me@sumnerevans.com>
2021-11-01 11:35:55 +00:00
Erik Johnston 82d2168a15 Add metrics to the threadpools (#11178) 2021-11-01 11:21:36 +00:00
Sean Quah 2451003f6f Test that ClientIpStore combines database and in-memory data correctly (#11179) 2021-11-01 11:20:54 +00:00
JohannesKleine 29ffd680bf Stop synapse from saving messages in device_inbox for hidden devices. (#10097)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-11-01 10:40:41 +00:00
Brendan Abolivier e320f5dba3 Deprecate user_may_create_room_with_invites (#11206) 2021-11-01 10:46:08 +01:00
Dirk Klimpel bfd7a9b65c Fix comments referencing v1.46.0 from PR #10969. (#11212)
#10969 was merged after 1.46.0rc1 was cut and will be included
in v1.47.0rc1 instead.
2021-10-29 13:43:51 -04:00
Brendan Abolivier ad4eab9862 Add a module API method to retrieve state from a room (#11204) 2021-10-29 16:28:29 +00:00
Sean Quah 3ed17ff651 Clarify lack of Windows support in documentation (#11198) 2021-10-29 14:03:58 +01:00
Patrick Cloke 56e281bf6c Additional type hints for relations database class. (#11205) 2021-10-28 14:35:12 -04:00
Rafael Gonçalves 0e16b418f6 Add knock information in admin exported data (#11171)
Signed-off-by: Rafael Goncalves <rafaelgoncalves@riseup.net>
2021-10-28 18:54:38 +01:00
Shay e002faee01 Fetch verify key locally rather than trying to do so over federation if origin and host are the same. (#11129)
* add tests for fetching key locally

* add logic to check if origin server is same as host and fetch verify key locally rather than over federation

* add changelog

* slight refactor, add docstring, change changelog entry

* Make changelog entry one line

* remove verify_json_locally and push locality check to process_request, add function process_request_locally

* remove leftover code reference

* refactor to add common call to 'verify_json and associated handling code

* add type hint to process_json

* add some docstrings + very slight refactor
2021-10-28 10:27:17 -07:00
Brendan Abolivier adc0d35b17 Add a ModuleApi method to update a user's membership in a room (#11147)
Co-authored-by: reivilibre <oliverw@matrix.org>
2021-10-28 16:45:53 +00:00
David Robertson 1bfd141205 Type hints for the remaining two files in synapse.http. (#11164)
* Teach MyPy that the sentinel context is False

This means that if `ctx: LoggingContextOrSentinel`
then `bool(ctx)` narrows us to `ctx:LoggingContext`, which is a really
neat find!

* Annotate RequestMetrics

- Raise errors for sentry if we use the sentinel context
- Ensure we don't raise an error and carry on, but not recording stats
- Include stack trace in the error case to lower Sean's blood pressure

* Make mypy pass for synapse.http.request_metrics

* Make synapse.http.connectproxyclient pass mypy

Co-authored-by: reivilibre <oliverw@matrix.org>
2021-10-28 14:14:42 +01:00
Skyler Mäntysaari a19bf32a03 docs/openid: Add Authentik documentation. (#11151) 2021-10-28 10:31:22 +00:00
Dan Callahan a1ba7a850a Update scripts to pass Shellcheck lints (#11166) 2021-10-27 21:36:18 +01:00
Dan Callahan 0dffa9d0e0 Merge remote-tracking branch 'origin/develop' into shellcheck
Fixes a merge conflict with debian/changelog

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-27 20:04:00 +01:00
reivilibre 75ca0a6168 Annotate log_function decorator (#10943)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-10-27 17:27:23 +01:00
Samuel Philipp 4e393af52f Fixed config parse bug in review_recent_signups (#11191) 2021-10-27 17:25:18 +01:00
Richard van der Hoff 576921c66a Force deb compression with xz. (#11197)
Fixes a problem where `impish` packages could not be processed by `reprepro`.
2021-10-27 17:06:32 +01:00
Patrick Cloke 19d5dc6931 Refactor Filter to handle fields according to data being filtered. (#11194)
This avoids filtering against fields which cannot exist on an
event source. E.g. presence updates don't have a room.
2021-10-27 11:26:30 -04:00
Dirk Klimpel 8d46fac98e Delete messages from device_inbox table when deleting device (#10969)
Fixes: #9346
2021-10-27 16:01:18 +01:00
Patrick Cloke b3e843be88 Fix URL preview errors when previewing XML documents. (#11196) 2021-10-27 14:48:02 +00:00
Patrick Cloke a930da3291 Include the stable identifier for MSC3288. (#11187)
Includes both the stable and unstable identifier to store-invite
calls to the identity server. In the future we should remove the
unstable identifier.
2021-10-27 14:19:19 +00:00
Erik Johnston 179dc8ae9e Merge remote-tracking branch 'origin/release-v1.46' into develop 2021-10-27 14:45:40 +01:00
Erik Johnston e0ef8fe58d Update release date 2021-10-27 14:44:45 +01:00
Erik Johnston b615fc35d6 Update changelog with new changes 2021-10-27 13:22:48 +01:00
Sean Quah f3a4be8700 Shut down the DNS threadpool (#11190)
The DNS threadpool must be explicitly stopped, otherwise Synapse will
hang indefinitely when asked to shut down.
2021-10-27 13:04:56 +01:00
Erik Johnston 72626b78ef Fix thread BG update to not seq scan event_json (#11192)
For some reason the query optimiser decided to seq scan both tables,
rather than index scanning `event_json`.
2021-10-27 11:33:21 +00:00
Sean Quah 2dbef6c10a Update CHANGES.md 2021-10-26 17:54:25 +01:00
Sean Quah 60ad9460c4 Move #10975 to bugfix section in changelog 2021-10-26 14:26:36 +01:00
Sean Quah 400f391f71 Update CHANGES.md 2021-10-26 14:21:09 +01:00
Brendan Abolivier c7a5e49664 Implement an on_new_event callback (#11126)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-10-26 15:17:36 +02:00
Sean Quah 34b0222c2b Update CHANGES.md 2021-10-26 14:08:52 +01:00
Sean Quah cc75a6b1b2 1.46.0rc1 2021-10-26 14:04:51 +01:00
Erik Johnston 7004f43da1 Move DNS lookups into separate thread pool (#11177)
This is to stop large bursts of lookups starving out other users of the
thread pools.

Fixes #11049.
2021-10-26 13:45:38 +01:00
Patrick Cloke d52c58dfa3 Add a background update for updating MSC3440 relation threads. (#11181) 2021-10-26 07:38:45 -04:00
Brendan Abolivier 8c8e36af0d Document the version each module API method was added to Synapse (#11183) 2021-10-26 11:09:10 +02:00
Jason Robinson 63cbdd8af0 Enable changing user type via users admin API (#11174)
Users admin API can now also modify user
type in addition to allowing it to be
set on user creation.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-10-26 09:01:06 +00:00
Brendan Abolivier c1510c97b5 Fix cyclic import in the module API (#11180)
Introduced in #10548

See https://github.com/matrix-org/synapse-email-account-validity/runs/3979337154?check_suite_focus=true for an example of a module's CI choking over this issue.
2021-10-25 16:45:19 +00:00
AndrewFerr 4387b791e0 Don't set new room alias before potential 403 (#10930)
Fixes: #10929 

Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net>
2021-10-25 15:24:49 +01:00
Richard van der Hoff da957a60e8 Ensure that we correctly auth events returned by send_join (#11012)
This is the final piece of the jigsaw for #9595. As with other changes before this one (eg #10771), we need to make sure that we auth the auth events in the right order, and actually check that their predecessors haven't been rejected.

To do this I've reused the existing code we use when persisting outliers elsewhere.

I've removed the code for attempting to fetch missing auth_events - the events should have been present in the send_join response, so the likely reason they are missing is that we couldn't verify them, so requesting them again is unlikely to help. Instead, we simply drop any state which relies on those auth events, as we do at a backwards-extremity. See also matrix-org/complement#216 for a test for this.
2021-10-25 15:21:09 +01:00
Sean Quah 85a09f8b8b Fix module API's get_user_ip_and_agents function when run on workers (#11112) 2021-10-25 13:01:04 +01:00
Dan Callahan 1afc6ecae1 Changelog
Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:21:40 +01:00
Dan Callahan d7141e0b8b Fix Shellcheck SC2006: Use $(...) notation
Use $(...) notation instead of legacy backticked `...`.

https://github.com/koalaman/shellcheck/wiki/SC2006

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:08:55 +01:00
Dan Callahan b5e910521b Fix Shellcheck SC2129: Consider using {..} >> file
Consider using { cmd1; cmd2; } >> file instead of individual redirects.

https://github.com/koalaman/shellcheck/wiki/SC2129

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:08:54 +01:00
Dan Callahan 13f084eb58 Fix Shellcheck SC2086: Quote to prevent splitting
Double quote to prevent globbing and word splitting.

https://github.com/koalaman/shellcheck/wiki/SC2086

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:08:54 +01:00
Dan Callahan 31096132c3 Fix Shellcheck SC2012: Use find instead of ls
Use find instead of ls to better handle non-alphanumeric filenames.

https://github.com/koalaman/shellcheck/wiki/SC2012

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:08:54 +01:00
Dan Callahan 9d0f9d51d5 Fix Shellcheck SC2016: Single quotes don't expand
Expressions don't expand in single quotes, use double quotes for that.

https://github.com/koalaman/shellcheck/wiki/SC2016

This specifically warned about the '$aregis...' part of the sed script.
Which is a relatively obscure use of sed.

Splitting this into two commands makes its intent more obvious and
avoids contravening Shellcheck's lints.

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:08:54 +01:00
Dan Callahan bab2bc844c Fix Shellcheck SC1091: Can't follow file
Not following: (error message here)

https://github.com/koalaman/shellcheck/wiki/SC1091

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:08:54 +01:00
Dan Callahan 7cf83c0aca Fix Shellcheck SC1001: Meaningless char escapes
This \o will be a regular 'o' in this context.

https://github.com/koalaman/shellcheck/wiki/SC1001

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:08:54 +01:00
Dan Callahan 99e698d6ed Fix Shellcheck SC2089 and SC2090: Quotes in vars
SC2089: Quotes/backslashes will be treated literally. Use an array.

https://github.com/koalaman/shellcheck/wiki/SC2089

SC2090: Quotes/backslashes in this variable will not be respected.

https://github.com/koalaman/shellcheck/wiki/SC2090

Putting literal JSON in a variable mistakenly triggers these warnings.
Instead of adding ignore directives, this can be avoided by inlining the
JSON data into the curl invocation.

Since the variable is only used in this one location, inlining is fine.

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:08:54 +01:00
Dan Callahan dfa6143133 Fix Shellcheck SC2155: Declare + export separately
Declare and assign separately to avoid masking return values.

https://github.com/koalaman/shellcheck/wiki/SC2155

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:08:54 +01:00
Dan Callahan 6a9d84a676 Fix Shellcheck SC2166: test -a is not well defined
Prefer [ p ] && [ q ] as [ p -a q ] is not well defined.

https://github.com/koalaman/shellcheck/wiki/SC2166

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:08:54 +01:00
Dan Callahan 6c736fa472 Fix Shellcheck SC2154: variable possibly undefined
var is referenced but not assigned.

https://github.com/koalaman/shellcheck/wiki/SC2154

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:08:54 +01:00
Dan Callahan 898e3be4c9 Fix Shellcheck SC2064: Use single quotes on traps
Use single quotes, otherwise this expands now rather than when signalled.

https://github.com/koalaman/shellcheck/wiki/SC2064

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:08:54 +01:00
Dan Callahan 5eb481cd5b Fix Shellcheck SC2115: Ensure never expands to /*
Use "${var:?}" to ensure this never expands to /* .

https://github.com/koalaman/shellcheck/wiki/SC2115

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:08:54 +01:00
Dan Callahan 64adbb7b54 Fix Shellcheck SC2046: Quote to prevent word split
Quote this to prevent word splitting

https://www.shellcheck.net/wiki/SC2046

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:08:53 +01:00
Dan Callahan 12d79ff1b6 Fix Shellcheck SC2164: exit in case cd fails.
Use `cd ... || exit` in case cd fails.

https://github.com/koalaman/shellcheck/wiki/SC2164

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:08:53 +01:00
Sean Quah 2b82ec425f Add type hints for most HomeServer parameters (#11095) 2021-10-22 18:15:41 +01:00
Jason Robinson b9ce53e878 Fix synapse.config module "read" command (#11145)
`synapse.config.__main__` has the possibility to read a config item. This can be used to conveniently also validate the config is valid before trying to start Synapse.

 The "read" command broke in https://github.com/matrix-org/synapse/pull/10916 as it now requires passing in "server.server_name" for example.

 Also made the read command optional so one can just call this with just the confirm file reference and get a "Config parses OK" if things are ok.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-10-22 12:00:52 +02:00
Dirk Klimpel b0f03aeb6a Add more information what happens when a user is deactivated (#11083) 2021-10-22 09:00:28 +01:00
Patrick Cloke ba00e20234 Add a thread relation type per MSC3440. (#11088)
Adds experimental support for MSC3440's `io.element.thread` relation
type (and the aggregation for it).
2021-10-21 14:39:16 -04:00
358 changed files with 8778 additions and 2434 deletions
+1 -1
View File
@@ -3,7 +3,7 @@
# Test for the export-data admin command against sqlite and postgres
set -xe
cd `dirname $0`/../..
cd "$(dirname "$0")/../.."
echo "--- Install dependencies"
+1 -1
View File
@@ -7,7 +7,7 @@
set -xe
cd `dirname $0`/../..
cd "$(dirname "$0")/../.."
echo "--- Install dependencies"
+5 -4
View File
@@ -1,12 +1,13 @@
### Pull Request Checklist
<!-- Please read CONTRIBUTING.md before submitting your pull request -->
<!-- Please read https://matrix-org.github.io/synapse/latest/development/contributing_guide.html before submitting your pull request -->
* [ ] Pull request is based on the develop branch
* [ ] Pull request includes a [changelog file](https://github.com/matrix-org/synapse/blob/master/CONTRIBUTING.md#changelog). The entry should:
* [ ] Pull request includes a [changelog file](https://matrix-org.github.io/synapse/latest/development/contributing_guide.html#changelog). The entry should:
- Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
* [ ] Pull request includes a [sign off](https://github.com/matrix-org/synapse/blob/master/CONTRIBUTING.md#sign-off)
* [ ] Code style is correct (run the [linters](https://github.com/matrix-org/synapse/blob/master/CONTRIBUTING.md#code-style))
* [ ] Pull request includes a [sign off](https://matrix-org.github.io/synapse/latest/development/contributing_guide.html#sign-off)
* [ ] [Code style](https://matrix-org.github.io/synapse/latest/code_style.html) is correct
(run the [linters](https://matrix-org.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
+171
View File
@@ -1,3 +1,174 @@
Synapse 1.47.0rc2 (2021-11-10)
==============================
This fixes an issue with publishing the Debian packages for 1.47.0rc1.
It is otherwise identical to 1.47.0rc1.
Synapse 1.47.0rc1 (2021-11-09)
==============================
Deprecations and Removals
-------------------------
- The `user_may_create_room_with_invites` module callback is now deprecated. Please refer to the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#upgrading-to-v1470) for more information. ([\#11206](https://github.com/matrix-org/synapse/issues/11206))
- Remove deprecated admin API to delete rooms (`POST /_synapse/admin/v1/rooms/<room_id>/delete`). ([\#11213](https://github.com/matrix-org/synapse/issues/11213))
Features
--------
- Advertise support for Client-Server API r0.6.1. ([\#11097](https://github.com/matrix-org/synapse/issues/11097))
- Add search by room ID and room alias to the List Room admin API. ([\#11099](https://github.com/matrix-org/synapse/issues/11099))
- Add an `on_new_event` third-party rules callback to allow Synapse modules to act after an event has been sent into a room. ([\#11126](https://github.com/matrix-org/synapse/issues/11126))
- Add a module API method to update a user's membership in a room. ([\#11147](https://github.com/matrix-org/synapse/issues/11147))
- Add metrics for thread pool usage. ([\#11178](https://github.com/matrix-org/synapse/issues/11178))
- Support the stable room type field for [MSC3288](https://github.com/matrix-org/matrix-doc/pull/3288). ([\#11187](https://github.com/matrix-org/synapse/issues/11187))
- Add a module API method to retrieve the current state of a room. ([\#11204](https://github.com/matrix-org/synapse/issues/11204))
- Calculate a default value for `public_baseurl` based on `server_name`. ([\#11210](https://github.com/matrix-org/synapse/issues/11210))
- Add support for serving `/.well-known/matrix/server` files, to redirect federation traffic to port 443. ([\#11211](https://github.com/matrix-org/synapse/issues/11211))
- Add admin APIs to pause, start and check the status of background updates. ([\#11263](https://github.com/matrix-org/synapse/issues/11263))
Bugfixes
--------
- Fix a long-standing bug which allowed hidden devices to receive to-device messages, resulting in unnecessary database bloat. ([\#10097](https://github.com/matrix-org/synapse/issues/10097))
- Fix a long-standing bug where messages in the `device_inbox` table for deleted devices would persist indefinitely. Contributed by @dklimpel and @JohannesKleine. ([\#10969](https://github.com/matrix-org/synapse/issues/10969), [\#11212](https://github.com/matrix-org/synapse/issues/11212))
- Do not accept events if a third-party rule `check_event_allowed` callback raises an exception. ([\#11033](https://github.com/matrix-org/synapse/issues/11033))
- Fix long-standing bug where verification requests could fail in certain cases if a federation whitelist was in place but did not include your own homeserver. ([\#11129](https://github.com/matrix-org/synapse/issues/11129))
- Allow an empty list of `state_events_at_start` to be sent when using the [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) `/batch_send` endpoint and the author of the historical messages is already part of the current room state at the given `?prev_event_id`. ([\#11188](https://github.com/matrix-org/synapse/issues/11188))
- Fix a bug introduced in Synapse 1.45.0 which prevented the `synapse_review_recent_signups` script from running. Contributed by @samuel-p. ([\#11191](https://github.com/matrix-org/synapse/issues/11191))
- Delete `to_device` messages for hidden devices that will never be read, reducing database size. ([\#11199](https://github.com/matrix-org/synapse/issues/11199))
- Fix a long-standing bug wherein a missing `Content-Type` header when downloading remote media would cause Synapse to throw an error. ([\#11200](https://github.com/matrix-org/synapse/issues/11200))
- Fix a long-standing bug which could result in serialization errors and potentially duplicate transaction data when sending ephemeral events to application services. Contributed by @Fizzadar at Beeper. ([\#11207](https://github.com/matrix-org/synapse/issues/11207))
- Fix a bug introduced in Synapse 1.35.0 which made it impossible to join rooms that return a `send_join` response containing floats. ([\#11217](https://github.com/matrix-org/synapse/issues/11217))
- Fix long-standing bug where cross signing keys were not included in the response to `/r0/keys/query` the first time a remote user was queried. ([\#11234](https://github.com/matrix-org/synapse/issues/11234))
- Fix a long-standing bug where all requests that read events from the database could get stuck as a result of losing the database connection. ([\#11240](https://github.com/matrix-org/synapse/issues/11240))
- Fix a bug preventing Synapse from being rolled back to an earlier version when using workers. ([\#11255](https://github.com/matrix-org/synapse/issues/11255), [\#11276](https://github.com/matrix-org/synapse/issues/11276))
- Fix a bug introduced in Synapse 1.37.1 which caused a remote event being processed by a worker to not get processed on restart if the worker was killed. ([\#11262](https://github.com/matrix-org/synapse/issues/11262))
- Only allow old Element/Riot Android clients to send read receipts without a request body. All other clients must include a request body as required by the specification. Contributed by @rogersheu. ([\#11157](https://github.com/matrix-org/synapse/issues/11157))
Updates to the Docker image
---------------------------
- Avoid changing user ID when started as a non-root user, and no explicit `UID` is set. ([\#11209](https://github.com/matrix-org/synapse/issues/11209))
Improved Documentation
----------------------
- Improve example HAProxy config in the docs to properly handle HTTP `Host` headers with port information. This is required for federation over port 443 to work correctly. ([\#11128](https://github.com/matrix-org/synapse/issues/11128))
- Add documentation for using Authentik as an OpenID Connect Identity Provider. Contributed by @samip5. ([\#11151](https://github.com/matrix-org/synapse/issues/11151))
- Clarify lack of support for Windows. ([\#11198](https://github.com/matrix-org/synapse/issues/11198))
- Improve code formatting and fix a few typos in docs. Contributed by @sumnerevans at Beeper. ([\#11221](https://github.com/matrix-org/synapse/issues/11221))
- Add documentation for using LemonLDAP as an OpenID Connect Identity Provider. Contributed by @l00ptr. ([\#11257](https://github.com/matrix-org/synapse/issues/11257))
Internal Changes
----------------
- Add type annotations for the `log_function` decorator. ([\#10943](https://github.com/matrix-org/synapse/issues/10943))
- Add type hints to `synapse.events`. ([\#11098](https://github.com/matrix-org/synapse/issues/11098))
- Remove and document unnecessary `RoomStreamToken` checks in application service ephemeral event code. ([\#11137](https://github.com/matrix-org/synapse/issues/11137))
- Add type hints so that `synapse.http` passes `mypy` checks. ([\#11164](https://github.com/matrix-org/synapse/issues/11164))
- Update scripts to pass Shellcheck lints. ([\#11166](https://github.com/matrix-org/synapse/issues/11166))
- Add knock information in admin export. Contributed by Rafael Gonçalves. ([\#11171](https://github.com/matrix-org/synapse/issues/11171))
- Add tests to check that `ClientIpStore.get_last_client_ip_by_device` and `get_user_ip_and_agents` combine database and in-memory data correctly. ([\#11179](https://github.com/matrix-org/synapse/issues/11179))
- Refactor `Filter` to check different fields depending on the data type. ([\#11194](https://github.com/matrix-org/synapse/issues/11194))
- Improve type hints for the relations datastore. ([\#11205](https://github.com/matrix-org/synapse/issues/11205))
- Replace outdated links in the pull request checklist with links to the rendered documentation. ([\#11225](https://github.com/matrix-org/synapse/issues/11225))
- Fix a bug in unit test `test_block_room_and_not_purge`. ([\#11226](https://github.com/matrix-org/synapse/issues/11226))
- In `ObservableDeferred`, run observers in the order they were registered. ([\#11229](https://github.com/matrix-org/synapse/issues/11229))
- Minor speed up to start up times and getting updates for groups by adding missing index to `local_group_updates.stream_id`. ([\#11231](https://github.com/matrix-org/synapse/issues/11231))
- Add `twine` and `towncrier` as dev dependencies, as they're used by the release script. ([\#11233](https://github.com/matrix-org/synapse/issues/11233))
- Allow `stream_writers.typing` config to be a list of one worker. ([\#11237](https://github.com/matrix-org/synapse/issues/11237))
- Remove debugging statement in tests. ([\#11239](https://github.com/matrix-org/synapse/issues/11239))
- Fix [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) historical messages backfilling in random order on remote homeservers. ([\#11244](https://github.com/matrix-org/synapse/issues/11244))
- Add an additional test for the `cachedList` method decorator. ([\#11246](https://github.com/matrix-org/synapse/issues/11246))
- Make minor correction to the type of `auth_checkers` callbacks. ([\#11253](https://github.com/matrix-org/synapse/issues/11253))
- Clean up trivial aspects of the Debian package build tooling. ([\#11269](https://github.com/matrix-org/synapse/issues/11269), [\#11273](https://github.com/matrix-org/synapse/issues/11273))
- Blacklist new SyTest that checks that key uploads are valid pending the validation being implemented in Synapse. ([\#11270](https://github.com/matrix-org/synapse/issues/11270))
Synapse 1.46.0 (2021-11-02)
===========================
The cause of the [performance regression affecting Synapse 1.44](https://github.com/matrix-org/synapse/issues/11049) has been identified and fixed. ([\#11177](https://github.com/matrix-org/synapse/issues/11177))
Bugfixes
--------
- Fix a bug introduced in v1.46.0rc1 where URL previews of some XML documents would fail. ([\#11196](https://github.com/matrix-org/synapse/issues/11196))
Synapse 1.46.0rc1 (2021-10-27)
==============================
Features
--------
- Add support for Ubuntu 21.10 "Impish Indri". ([\#11024](https://github.com/matrix-org/synapse/issues/11024))
- Port the Password Auth Providers module interface to the new generic interface. ([\#10548](https://github.com/matrix-org/synapse/issues/10548), [\#11180](https://github.com/matrix-org/synapse/issues/11180))
- Experimental support for the thread relation defined in [MSC3440](https://github.com/matrix-org/matrix-doc/pull/3440). ([\#11088](https://github.com/matrix-org/synapse/issues/11088), [\#11181](https://github.com/matrix-org/synapse/issues/11181), [\#11192](https://github.com/matrix-org/synapse/issues/11192))
- Users admin API can now also modify user type in addition to allowing it to be set on user creation. ([\#11174](https://github.com/matrix-org/synapse/issues/11174))
Bugfixes
--------
- Newly-created public rooms are now only assigned an alias if the room's creation has not been blocked by permission settings. Contributed by @AndrewFerr. ([\#10930](https://github.com/matrix-org/synapse/issues/10930))
- Fix a long-standing bug which meant that events received over federation were sometimes incorrectly accepted into the room state. ([\#11001](https://github.com/matrix-org/synapse/issues/11001), [\#11009](https://github.com/matrix-org/synapse/issues/11009), [\#11012](https://github.com/matrix-org/synapse/issues/11012))
- Fix 500 error on `/messages` when the server accumulates more than 5 backwards extremities at a given depth for a room. ([\#11027](https://github.com/matrix-org/synapse/issues/11027))
- Fix a bug where setting a user's `external_id` via the admin API returns 500 and deletes user's existing external mappings if that external ID is already mapped. ([\#11051](https://github.com/matrix-org/synapse/issues/11051))
- Fix a long-standing bug where users excluded from the user directory were added into the directory if they belonged to a room which became public or private. ([\#11075](https://github.com/matrix-org/synapse/issues/11075))
- Fix a long-standing bug when attempting to preview URLs which are in the `windows-1252` character encoding. ([\#11077](https://github.com/matrix-org/synapse/issues/11077), [\#11089](https://github.com/matrix-org/synapse/issues/11089))
- Fix broken export-data admin command and add test script checking the command to CI. ([\#11078](https://github.com/matrix-org/synapse/issues/11078))
- Show an error when timestamp in seconds is provided to the `/purge_media_cache` Admin API. ([\#11101](https://github.com/matrix-org/synapse/issues/11101))
- Fix local users who left all their rooms being removed from the user directory, even if the `search_all_users` config option was enabled. ([\#11103](https://github.com/matrix-org/synapse/issues/11103))
- Fix a bug which caused the module API's `get_user_ip_and_agents` function to always fail on workers. `get_user_ip_and_agents` was introduced in 1.44.0 and did not function correctly on worker processes at the time. ([\#11112](https://github.com/matrix-org/synapse/issues/11112))
- Identity server connection is no longer ignoring `ip_range_whitelist`. ([\#11120](https://github.com/matrix-org/synapse/issues/11120))
- Fix a bug introduced in Synapse 1.45.0 breaking the configuration file parsing script. ([\#11145](https://github.com/matrix-org/synapse/issues/11145))
- Fix a performance regression introduced in 1.44.0 which could cause client requests to time out when making large numbers of outbound requests. ([\#11177](https://github.com/matrix-org/synapse/issues/11177), [\#11190](https://github.com/matrix-org/synapse/issues/11190))
- Resolve and share `state_groups` for all [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) historical events in batch. ([\#10975](https://github.com/matrix-org/synapse/issues/10975))
Improved Documentation
----------------------
- Fix broken links relating to module API deprecation in the upgrade notes. ([\#11069](https://github.com/matrix-org/synapse/issues/11069))
- Add more information about what happens when a user is deactivated. ([\#11083](https://github.com/matrix-org/synapse/issues/11083))
- Clarify the the sample log config can be copied from the documentation without issue. ([\#11092](https://github.com/matrix-org/synapse/issues/11092))
- Update the admin API documentation with an updated list of the characters allowed in registration tokens. ([\#11093](https://github.com/matrix-org/synapse/issues/11093))
- Document Synapse's behaviour when dealing with multiple modules registering the same callbacks and/or handlers for the same HTTP endpoints. ([\#11096](https://github.com/matrix-org/synapse/issues/11096))
- Fix instances of `[example]{.title-ref}` in the upgrade documentation as a result of prior RST to Markdown conversion. ([\#11118](https://github.com/matrix-org/synapse/issues/11118))
- Document the version of Synapse each module callback was introduced in. ([\#11132](https://github.com/matrix-org/synapse/issues/11132))
- Document the version of Synapse that introduced each module API method. ([\#11183](https://github.com/matrix-org/synapse/issues/11183))
Internal Changes
----------------
- Fix spurious warnings about losing the logging context on the `ReplicationCommandHandler` when losing the replication connection. ([\#10984](https://github.com/matrix-org/synapse/issues/10984))
- Include rejected status when we log events. ([\#11008](https://github.com/matrix-org/synapse/issues/11008))
- Add some extra logging to the event persistence code. ([\#11014](https://github.com/matrix-org/synapse/issues/11014))
- Rearrange the internal workings of the incremental user directory updates. ([\#11035](https://github.com/matrix-org/synapse/issues/11035))
- Fix a long-standing bug where users excluded from the directory could still be added to the `users_who_share_private_rooms` table after a regular user joins a private room. ([\#11143](https://github.com/matrix-org/synapse/issues/11143))
- Add and improve type hints. ([\#10972](https://github.com/matrix-org/synapse/issues/10972), [\#11055](https://github.com/matrix-org/synapse/issues/11055), [\#11066](https://github.com/matrix-org/synapse/issues/11066), [\#11076](https://github.com/matrix-org/synapse/issues/11076), [\#11095](https://github.com/matrix-org/synapse/issues/11095), [\#11109](https://github.com/matrix-org/synapse/issues/11109), [\#11121](https://github.com/matrix-org/synapse/issues/11121), [\#11146](https://github.com/matrix-org/synapse/issues/11146))
- Mark the Synapse package as containing type annotations and fix export declarations so that Synapse pluggable modules may be type checked against Synapse. ([\#11054](https://github.com/matrix-org/synapse/issues/11054))
- Remove dead code from `MediaFilePaths`. ([\#11056](https://github.com/matrix-org/synapse/issues/11056))
- Be more lenient when parsing oEmbed response versions. ([\#11065](https://github.com/matrix-org/synapse/issues/11065))
- Create a separate module for the retention configuration. ([\#11070](https://github.com/matrix-org/synapse/issues/11070))
- Clean up some of the federation event authentication code for clarity. ([\#11115](https://github.com/matrix-org/synapse/issues/11115), [\#11116](https://github.com/matrix-org/synapse/issues/11116), [\#11122](https://github.com/matrix-org/synapse/issues/11122))
- Add docstrings and comments to the application service ephemeral event sending code. ([\#11138](https://github.com/matrix-org/synapse/issues/11138))
- Update the `sign_json` script to support inline configuration of the signing key. ([\#11139](https://github.com/matrix-org/synapse/issues/11139))
- Fix broken link in the docker image README. ([\#11144](https://github.com/matrix-org/synapse/issues/11144))
- Always dump logs from unit tests during CI runs. ([\#11068](https://github.com/matrix-org/synapse/issues/11068))
- Add tests for `MediaFilePaths` class. ([\#11057](https://github.com/matrix-org/synapse/issues/11057))
- Simplify the user admin API tests. ([\#11048](https://github.com/matrix-org/synapse/issues/11048))
- Add a test for the workaround introduced in [\#11042](https://github.com/matrix-org/synapse/pull/11042) concerning the behaviour of third-party rule modules and `SynapseError`s. ([\#11071](https://github.com/matrix-org/synapse/issues/11071))
Synapse 1.45.1 (2021-10-20)
===========================
-1
View File
@@ -1 +0,0 @@
Port the Password Auth Providers module interface to the new generic interface.
-1
View File
@@ -1 +0,0 @@
Add type hints to `synapse.storage.databases.main.client_ips`.
-1
View File
@@ -1 +0,0 @@
Resolve and share `state_groups` for all [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) historical events in batch.
-1
View File
@@ -1 +0,0 @@
Fix spurious warnings about losing the logging context on the `ReplicationCommandHandler` when losing the replication connection.
-1
View File
@@ -1 +0,0 @@
Fix a long-standing bug which meant that events received over federation were sometimes incorrectly accepted into the room state.
-1
View File
@@ -1 +0,0 @@
Include rejected status when we log events.
-1
View File
@@ -1 +0,0 @@
Fix a long-standing bug which meant that events received over federation were sometimes incorrectly accepted into the room state.
-1
View File
@@ -1 +0,0 @@
Add some extra logging to the event persistence code.
-1
View File
@@ -1 +0,0 @@
Add support for Ubuntu 21.10 "Impish Indri".
-1
View File
@@ -1 +0,0 @@
Fix 500 error on `/messages` when the server accumulates more than 5 backwards extremities at a given depth for a room.
-1
View File
@@ -1 +0,0 @@
Rearrange the internal workings of the incremental user directory updates.
-1
View File
@@ -1 +0,0 @@
Simplify the user admin API tests.
-1
View File
@@ -1 +0,0 @@
Fix a bug where setting a user's external_id via the admin API returns 500 and deletes users existing external mappings if that external ID is already mapped.
-1
View File
@@ -1 +0,0 @@
Mark the Synapse package as containing type annotations and fix export declarations so that Synapse pluggable modules may be type checked against Synapse.
-1
View File
@@ -1 +0,0 @@
Improve type hints for `_wrap_in_base_path` decorator used by `MediaFilePaths`.
-1
View File
@@ -1 +0,0 @@
Remove dead code from `MediaFilePaths`.
-1
View File
@@ -1 +0,0 @@
Add tests for `MediaFilePaths` class.
-1
View File
@@ -1 +0,0 @@
Be more lenient when parsing oEmbed response versions.
-1
View File
@@ -1 +0,0 @@
Add type hints to `synapse.events`.
-1
View File
@@ -1 +0,0 @@
Always dump logs from unit tests during CI runs.
-1
View File
@@ -1 +0,0 @@
Fix broken links relating to module API deprecation in the upgrade notes.
-1
View File
@@ -1 +0,0 @@
Create a separate module for the retention configuration.
-1
View File
@@ -1 +0,0 @@
Add a test for the workaround introduced in [\#11042](https://github.com/matrix-org/synapse/pull/11042) concerning the behaviour of third-party rule modules and `SynapseError`s.
-1
View File
@@ -1 +0,0 @@
Fix a long-standing bug where users excluded from the user directory were added into the directory if they belonged to a room which became public or private.
-1
View File
@@ -1 +0,0 @@
Fix type hints in the relations tests.
-1
View File
@@ -1 +0,0 @@
Fix a long-standing bug when attempting to preview URLs which are in the `windows-1252` character encoding.
-1
View File
@@ -1 +0,0 @@
Fix broken export-data admin command and add test script checking the command to CI.
-1
View File
@@ -1 +0,0 @@
Fix a long-standing bug when attempting to preview URLs which are in the `windows-1252` character encoding.
-1
View File
@@ -1 +0,0 @@
Clarify the the sample log config can be copied from the documentation without issue.
-1
View File
@@ -1 +0,0 @@
Update the admin API documentation with an updated list of the characters allowed in registration tokens.
-1
View File
@@ -1 +0,0 @@
Document Synapse's behaviour when dealing with multiple modules registering the same callbacks and/or handlers for the same HTTP endpoints.
-1
View File
@@ -1 +0,0 @@
Show an error when timestamp in seconds is provided to the `/purge_media_cache` Admin API.
-1
View File
@@ -1 +0,0 @@
Fix local users who left all their rooms being removed from the user directory, even if the "search_all_users" config option was enabled.
-1
View File
@@ -1 +0,0 @@
Add missing type hints to `synapse.api` module.
-1
View File
@@ -1 +0,0 @@
Clean up some of the federation event authentication code for clarity.
-1
View File
@@ -1 +0,0 @@
Clean up some of the federation event authentication code for clarity.
-1
View File
@@ -1 +0,0 @@
Fix instances of `[example]{.title-ref}` in the upgrade documentation as a result of prior RST to Markdown conversion.
-1
View File
@@ -1 +0,0 @@
Identity server connection is no longer ignoring `ip_range_whitelist`.
-1
View File
@@ -1 +0,0 @@
Add type hints for event fetching.
-1
View File
@@ -1 +0,0 @@
Clean up some of the federation event authentication code for clarity.
-1
View File
@@ -1 +0,0 @@
Document the version of Synapse each module callback was introduced in.
-1
View File
@@ -1 +0,0 @@
Add docstrings and comments to the application service ephemeral event sending code.
-1
View File
@@ -1 +0,0 @@
Update the `sign_json` script to support inline configuration of the signing key.
-1
View File
@@ -1 +0,0 @@
Fix a long-standing bug where users excluded from the directory could still be added to the `users_who_share_private_rooms` table after a regular user joins a private room.
-1
View File
@@ -1 +0,0 @@
Fix broken link in the docker image README.
-1
View File
@@ -1 +0,0 @@
Add missing type hints to `synapse.crypto`.
+1
View File
@@ -0,0 +1 @@
Add a new version of delete room admin API `DELETE /_synapse/admin/v2/rooms/<room_id>` to run it in background. Contributed by @dklimpel.
+1
View File
@@ -0,0 +1 @@
Allow the admin [Delete Room API](https://matrix-org.github.io/synapse/latest/admin_api/rooms.html#delete-room-api) to block a room without the need to join it.
+2
View File
@@ -0,0 +1,2 @@
Fix a long-standing bug wherein display names or avatar URLs containing null bytes cause an internal server error
when stored in the DB.
+1
View File
@@ -0,0 +1 @@
Support filtering by relation senders & types per [MSC3440](https://github.com/matrix-org/matrix-doc/pull/3440).
+1
View File
@@ -0,0 +1 @@
Split out federated PDU retrieval function into a non-cached version.
+1
View File
@@ -0,0 +1 @@
Clean up code relating to to-device messages and sending ephemeral events to application services.
+1
View File
@@ -0,0 +1 @@
Fix a small typo in the error response when a relation type other than 'm.annotation' is passed to `GET /rooms/{room_id}/aggregations/{event_id}`.
+1
View File
@@ -0,0 +1 @@
Drop unused db tables `room_stats_historical` and `user_stats_historical`.
+1
View File
@@ -0,0 +1 @@
Suggest users of the Debian packages add configuration to `/etc/matrix-synapse/conf.d/` to prevent, upon upgrade, being asked to choose between their configuration and the maintainer's.
+1
View File
@@ -0,0 +1 @@
Require all files in synapse/ and tests/ to pass mypy unless specifically excluded.
+1
View File
@@ -0,0 +1 @@
Require all files in synapse/ and tests/ to pass mypy unless specifically excluded.
+1
View File
@@ -0,0 +1 @@
Fix typo in the word `available` and fix HTTP method (should be `GET`) for the `username_available` admin API. Contributed by Stanislav Motylkov.
+1
View File
@@ -0,0 +1 @@
Add missing type hints to `synapse.app`.
+1
View File
@@ -0,0 +1 @@
Fix a long-standing bug where uploading extremely thin images (e.g. 1000x1) would fail. Contributed by @Neeeflix.
+1
View File
@@ -0,0 +1 @@
Remove unused parameters on `FederationEventHandler._check_event_auth`.
+1
View File
@@ -0,0 +1 @@
Add type hints to `synapse._scripts`.
+1
View File
@@ -0,0 +1 @@
Add Single Sign-On, SAML and CAS pages to the documentation.
+1
View File
@@ -0,0 +1 @@
Fix an issue which prevented the 'remove deleted devices from device_inbox column' background process from running when updating from a recent Synapse version.
+1
View File
@@ -0,0 +1 @@
Add type hints to storage classes.
+1
View File
@@ -0,0 +1 @@
Add type hints to storage classes.
+1
View File
@@ -0,0 +1 @@
Add type hints to storage classes.
+1
View File
@@ -0,0 +1 @@
Add type hints to storage classes.
+1
View File
@@ -0,0 +1 @@
Add type hints to storage classes.
+1
View File
@@ -0,0 +1 @@
Add type hints to storage classes.
+1
View File
@@ -0,0 +1 @@
Add type hints to storage classes.
+1
View File
@@ -0,0 +1 @@
Add type hints to `synapse.util`.
+1
View File
@@ -0,0 +1 @@
Add type hints to storage classes.
+1
View File
@@ -0,0 +1 @@
Improve type annotations in Synapse's test suite.
+1
View File
@@ -0,0 +1 @@
Test that room alias deletion works as intended.
+1
View File
@@ -0,0 +1 @@
Add type hints to storage classes.
+1
View File
@@ -0,0 +1 @@
Support the stable version of [MSC2778](https://github.com/matrix-org/matrix-doc/pull/2778): the `m.login.application_service` login type. Contributed by @tulir.
+1
View File
@@ -0,0 +1 @@
Add type hints to storage classes.
+1
View File
@@ -0,0 +1 @@
Add type hints to storage classes.
+1
View File
@@ -0,0 +1 @@
Add a development script for visualising the storage class inheritance hierarchy.
+4 -2
View File
@@ -84,7 +84,9 @@ AUTH="Authorization: Bearer $TOKEN"
###################################################################################################
# finally start pruning the room:
###################################################################################################
POSTDATA='{"delete_local_events":"true"}' # this will really delete local events, so the messages in the room really disappear unless they are restored by remote federation
# this will really delete local events, so the messages in the room really
# disappear unless they are restored by remote federation. This is because
# we pass {"delete_local_events":true} to the curl invocation below.
for ROOM in "${ROOMS_ARRAY[@]}"; do
echo "########################################### $(date) ################# "
@@ -104,7 +106,7 @@ for ROOM in "${ROOMS_ARRAY[@]}"; do
SLEEP=2
set -x
# call purge
OUT=$(curl --header "$AUTH" -s -d $POSTDATA POST "$API_URL/admin/purge_history/$ROOM/$EVENT_ID")
OUT=$(curl --header "$AUTH" -s -d '{"delete_local_events":true}' POST "$API_URL/admin/purge_history/$ROOM/$EVENT_ID")
PURGE_ID=$(echo "$OUT" |grep purge_id|cut -d'"' -f4 )
if [ "$PURGE_ID" == "" ]; then
# probably the history purge is already in progress for $ROOM
+6 -5
View File
@@ -15,7 +15,7 @@ export DH_VIRTUALENV_INSTALL_ROOT=/opt/venvs
# python won't look in the right directory. At least this way, the error will
# be a *bit* more obvious.
#
SNAKE=`readlink -e /usr/bin/python3`
SNAKE=$(readlink -e /usr/bin/python3)
# try to set the CFLAGS so any compiled C extensions are compiled with the most
# generic as possible x64 instructions, so that compiling it on a new Intel chip
@@ -24,7 +24,7 @@ SNAKE=`readlink -e /usr/bin/python3`
# TODO: add similar things for non-amd64, or figure out a more generic way to
# do this.
case `dpkg-architecture -q DEB_HOST_ARCH` in
case $(dpkg-architecture -q DEB_HOST_ARCH) in
amd64)
export CFLAGS=-march=x86-64
;;
@@ -40,6 +40,7 @@ dh_virtualenv \
--upgrade-pip \
--preinstall="lxml" \
--preinstall="mock" \
--preinstall="wheel" \
--extra-pip-arg="--no-cache-dir" \
--extra-pip-arg="--compile" \
--extras="all,systemd,test"
@@ -56,8 +57,8 @@ case "$DEB_BUILD_OPTIONS" in
*)
# Copy tests to a temporary directory so that we can put them on the
# PYTHONPATH without putting the uninstalled synapse on the pythonpath.
tmpdir=`mktemp -d`
trap "rm -r $tmpdir" EXIT
tmpdir=$(mktemp -d)
trap 'rm -r $tmpdir' EXIT
cp -r tests "$tmpdir"
@@ -98,7 +99,7 @@ esac
--output-file="${PACKAGE_BUILD_DIR}/etc/matrix-synapse/log.yaml"
# add a dependency on the right version of python to substvars.
PYPKG=`basename $SNAKE`
PYPKG=$(basename "$SNAKE")
echo "synapse:pydepends=$PYPKG" >> debian/matrix-synapse-py3.substvars
+30
View File
@@ -1,3 +1,33 @@
matrix-synapse-py3 (1.47.0~rc2) stable; urgency=medium
[ Dan Callahan ]
* Update scripts to pass Shellcheck lints.
* Remove unused Vagrant scripts from debian/ directory.
* Allow building Debian packages for any architecture, not just amd64.
* Preinstall the "wheel" package when building virtualenvs.
* Do not error if /etc/default/matrix-synapse is missing.
[ Synapse Packaging team ]
* New synapse release 1.47.0~rc2.
-- Synapse Packaging team <packages@matrix.org> Wed, 10 Nov 2021 09:41:01 +0000
matrix-synapse-py3 (1.46.0) stable; urgency=medium
[ Richard van der Hoff ]
* Compress debs with xz, to fix incompatibility of impish debs with reprepro.
[ Synapse Packaging team ]
* New synapse release 1.46.0.
-- Synapse Packaging team <packages@matrix.org> Tue, 02 Nov 2021 13:22:53 +0000
matrix-synapse-py3 (1.46.0~rc1) stable; urgency=medium
* New synapse release 1.46.0~rc1.
-- Synapse Packaging team <packages@matrix.org> Tue, 26 Oct 2021 14:04:04 +0100
matrix-synapse-py3 (1.45.1) stable; urgency=medium
* New synapse release 1.45.1.
+1 -1
View File
@@ -19,7 +19,7 @@ Standards-Version: 3.9.8
Homepage: https://github.com/matrix-org/synapse
Package: matrix-synapse-py3
Architecture: amd64
Architecture: any
Provides: matrix-synapse
Conflicts:
matrix-synapse (<< 0.34.0.1-0matrix2),
+1
View File
@@ -2,6 +2,7 @@
set -e
# shellcheck disable=SC1091
. /usr/share/debconf/confmodule
# try to update the debconf db according to whatever is in the config files
+1
View File
@@ -1,5 +1,6 @@
#!/bin/sh -e
# shellcheck disable=SC1091
. /usr/share/debconf/confmodule
CONFIGFILE_SERVERNAME="/etc/matrix-synapse/conf.d/server_name.yaml"
+1 -1
View File
@@ -5,7 +5,7 @@ Description=Synapse Matrix homeserver
Type=notify
User=matrix-synapse
WorkingDirectory=/var/lib/matrix-synapse
EnvironmentFile=/etc/default/matrix-synapse
EnvironmentFile=-/etc/default/matrix-synapse
ExecStartPre=/opt/venvs/matrix-synapse/bin/python -m synapse.app.homeserver --config-path=/etc/matrix-synapse/homeserver.yaml --config-path=/etc/matrix-synapse/conf.d/ --generate-keys
ExecStart=/opt/venvs/matrix-synapse/bin/python -m synapse.app.homeserver --config-path=/etc/matrix-synapse/homeserver.yaml --config-path=/etc/matrix-synapse/conf.d/
ExecReload=/bin/kill -HUP $MAINPID
+6
View File
@@ -51,5 +51,11 @@ override_dh_shlibdeps:
override_dh_virtualenv:
./debian/build_virtualenv
override_dh_builddeb:
# force the compression to xzip, to stop dpkg-deb on impish defaulting to zstd
# (which requires reprepro 5.3.0-1.3, which is currently only in 'experimental' in Debian:
# https://metadata.ftp-master.debian.org/changelogs/main/r/reprepro/reprepro_5.3.0-1.3_changelog)
dh_builddeb -- -Zxz
%:
dh $@ --with python-virtualenv
-2
View File
@@ -1,2 +0,0 @@
.vagrant
*.log
-23
View File
@@ -1,23 +0,0 @@
#!/bin/bash
#
# provisioning script for vagrant boxes for testing the matrix-synapse debs.
#
# Will install the most recent matrix-synapse-py3 deb for this platform from
# the /debs directory.
set -e
apt-get update
apt-get install -y lsb-release
deb=`ls /debs/matrix-synapse-py3_*+$(lsb_release -cs)*.deb | sort | tail -n1`
debconf-set-selections <<EOF
matrix-synapse matrix-synapse/report-stats boolean false
matrix-synapse matrix-synapse/server-name string localhost:18448
EOF
dpkg -i "$deb"
sed -i -e '/port: 8...$/{s/8448/18448/; s/8008/18008/}' -e '$aregistration_shared_secret: secret' /etc/matrix-synapse/homeserver.yaml
systemctl restart matrix-synapse
-13
View File
@@ -1,13 +0,0 @@
# -*- mode: ruby -*-
# vi: set ft=ruby :
ver = `cd ../../..; dpkg-parsechangelog -S Version`.strip()
Vagrant.configure("2") do |config|
config.vm.box = "debian/stretch64"
config.vm.synced_folder ".", "/vagrant", disabled: true
config.vm.synced_folder "../../../../debs", "/debs", type: "nfs"
config.vm.provision "shell", path: "../provision.sh"
end
-10
View File
@@ -1,10 +0,0 @@
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure("2") do |config|
config.vm.box = "ubuntu/xenial64"
config.vm.synced_folder ".", "/vagrant", disabled: true
config.vm.synced_folder "../../../../debs", "/debs"
config.vm.provision "shell", path: "../provision.sh"
end
+4 -4
View File
@@ -6,14 +6,14 @@ DIR="$( cd "$( dirname "$0" )" && pwd )"
PID_FILE="$DIR/servers.pid"
if [ -f $PID_FILE ]; then
if [ -f "$PID_FILE" ]; then
echo "servers.pid exists!"
exit 1
fi
for port in 8080 8081 8082; do
rm -rf $DIR/$port
rm -rf $DIR/media_store.$port
rm -rf "${DIR:?}/$port"
rm -rf "$DIR/media_store.$port"
done
rm -rf $DIR/etc
rm -rf "${DIR:?}/etc"
+77 -73
View File
@@ -4,21 +4,22 @@ DIR="$( cd "$( dirname "$0" )" && pwd )"
CWD=$(pwd)
cd "$DIR/.."
cd "$DIR/.." || exit
mkdir -p demo/etc
export PYTHONPATH=$(readlink -f $(pwd))
PYTHONPATH=$(readlink -f "$(pwd)")
export PYTHONPATH
echo $PYTHONPATH
echo "$PYTHONPATH"
for port in 8080 8081 8082; do
echo "Starting server on port $port... "
https_port=$((port + 400))
mkdir -p demo/$port
pushd demo/$port
pushd demo/$port || exit
#rm $DIR/etc/$port.config
python3 -m synapse.app.homeserver \
@@ -27,75 +28,78 @@ for port in 8080 8081 8082; do
--config-path "$DIR/etc/$port.config" \
--report-stats no
if ! grep -F "Customisation made by demo/start.sh" -q $DIR/etc/$port.config; then
printf '\n\n# Customisation made by demo/start.sh\n' >> $DIR/etc/$port.config
echo "public_baseurl: http://localhost:$port/" >> $DIR/etc/$port.config
echo 'enable_registration: true' >> $DIR/etc/$port.config
# Warning, this heredoc depends on the interaction of tabs and spaces. Please don't
# accidentaly bork me with your fancy settings.
listeners=$(cat <<-PORTLISTENERS
# Configure server to listen on both $https_port and $port
# This overides some of the default settings above
listeners:
- port: $https_port
type: http
tls: true
resources:
- names: [client, federation]
- port: $port
tls: false
bind_addresses: ['::1', '127.0.0.1']
type: http
x_forwarded: true
resources:
- names: [client, federation]
compress: false
PORTLISTENERS
)
echo "${listeners}" >> $DIR/etc/$port.config
# Disable tls for the servers
printf '\n\n# Disable tls on the servers.' >> $DIR/etc/$port.config
echo '# DO NOT USE IN PRODUCTION' >> $DIR/etc/$port.config
echo 'use_insecure_ssl_client_just_for_testing_do_not_use: true' >> $DIR/etc/$port.config
echo 'federation_verify_certificates: false' >> $DIR/etc/$port.config
# Set tls paths
echo "tls_certificate_path: \"$DIR/etc/localhost:$https_port.tls.crt\"" >> $DIR/etc/$port.config
echo "tls_private_key_path: \"$DIR/etc/localhost:$https_port.tls.key\"" >> $DIR/etc/$port.config
if ! grep -F "Customisation made by demo/start.sh" -q "$DIR/etc/$port.config"; then
# Generate tls keys
openssl req -x509 -newkey rsa:4096 -keyout $DIR/etc/localhost\:$https_port.tls.key -out $DIR/etc/localhost\:$https_port.tls.crt -days 365 -nodes -subj "/O=matrix"
openssl req -x509 -newkey rsa:4096 -keyout "$DIR/etc/localhost:$https_port.tls.key" -out "$DIR/etc/localhost:$https_port.tls.crt" -days 365 -nodes -subj "/O=matrix"
# Ignore keys from the trusted keys server
echo '# Ignore keys from the trusted keys server' >> $DIR/etc/$port.config
echo 'trusted_key_servers:' >> $DIR/etc/$port.config
echo ' - server_name: "matrix.org"' >> $DIR/etc/$port.config
echo ' accept_keys_insecurely: true' >> $DIR/etc/$port.config
# Regenerate configuration
{
printf '\n\n# Customisation made by demo/start.sh\n'
echo "public_baseurl: http://localhost:$port/"
echo 'enable_registration: true'
# Reduce the blacklist
blacklist=$(cat <<-BLACK
# Set the blacklist so that it doesn't include 127.0.0.1, ::1
federation_ip_range_blacklist:
- '10.0.0.0/8'
- '172.16.0.0/12'
- '192.168.0.0/16'
- '100.64.0.0/10'
- '169.254.0.0/16'
- 'fe80::/64'
- 'fc00::/7'
BLACK
)
echo "${blacklist}" >> $DIR/etc/$port.config
# Warning, this heredoc depends on the interaction of tabs and spaces.
# Please don't accidentaly bork me with your fancy settings.
listeners=$(cat <<-PORTLISTENERS
# Configure server to listen on both $https_port and $port
# This overides some of the default settings above
listeners:
- port: $https_port
type: http
tls: true
resources:
- names: [client, federation]
- port: $port
tls: false
bind_addresses: ['::1', '127.0.0.1']
type: http
x_forwarded: true
resources:
- names: [client, federation]
compress: false
PORTLISTENERS
)
echo "${listeners}"
# Disable tls for the servers
printf '\n\n# Disable tls on the servers.'
echo '# DO NOT USE IN PRODUCTION'
echo 'use_insecure_ssl_client_just_for_testing_do_not_use: true'
echo 'federation_verify_certificates: false'
# Set tls paths
echo "tls_certificate_path: \"$DIR/etc/localhost:$https_port.tls.crt\""
echo "tls_private_key_path: \"$DIR/etc/localhost:$https_port.tls.key\""
# Ignore keys from the trusted keys server
echo '# Ignore keys from the trusted keys server'
echo 'trusted_key_servers:'
echo ' - server_name: "matrix.org"'
echo ' accept_keys_insecurely: true'
# Reduce the blacklist
blacklist=$(cat <<-BLACK
# Set the blacklist so that it doesn't include 127.0.0.1, ::1
federation_ip_range_blacklist:
- '10.0.0.0/8'
- '172.16.0.0/12'
- '192.168.0.0/16'
- '100.64.0.0/10'
- '169.254.0.0/16'
- 'fe80::/64'
- 'fc00::/7'
BLACK
)
echo "${blacklist}"
} >> "$DIR/etc/$port.config"
fi
# Check script parameters
if [ $# -eq 1 ]; then
if [ $1 = "--no-rate-limit" ]; then
if [ "$1" = "--no-rate-limit" ]; then
# Disable any rate limiting
ratelimiting=$(cat <<-RC
@@ -137,22 +141,22 @@ for port in 8080 8081 8082; do
burst_count: 1000
RC
)
echo "${ratelimiting}" >> $DIR/etc/$port.config
echo "${ratelimiting}" >> "$DIR/etc/$port.config"
fi
fi
if ! grep -F "full_twisted_stacktraces" -q $DIR/etc/$port.config; then
echo "full_twisted_stacktraces: true" >> $DIR/etc/$port.config
if ! grep -F "full_twisted_stacktraces" -q "$DIR/etc/$port.config"; then
echo "full_twisted_stacktraces: true" >> "$DIR/etc/$port.config"
fi
if ! grep -F "report_stats" -q $DIR/etc/$port.config ; then
echo "report_stats: false" >> $DIR/etc/$port.config
if ! grep -F "report_stats" -q "$DIR/etc/$port.config" ; then
echo "report_stats: false" >> "$DIR/etc/$port.config"
fi
python3 -m synapse.app.homeserver \
--config-path "$DIR/etc/$port.config" \
-D \
popd
popd || exit
done
cd "$CWD"
cd "$CWD" || exit
+1 -1
View File
@@ -8,7 +8,7 @@ for pid_file in $FILES; do
pid=$(cat "$pid_file")
if [[ $pid ]]; then
echo "Killing $pid_file with $pid"
kill $pid
kill "$pid"
fi
done
+8 -5
View File
@@ -65,7 +65,8 @@ The following environment variables are supported in `generate` mode:
* `SYNAPSE_DATA_DIR`: where the generated config will put persistent data
such as the database and media store. Defaults to `/data`.
* `UID`, `GID`: the user id and group id to use for creating the data
directories. Defaults to `991`, `991`.
directories. If unset, and no user is set via `docker run --user`, defaults
to `991`, `991`.
## Running synapse
@@ -97,7 +98,9 @@ The following environment variables are supported in `run` mode:
`<SYNAPSE_CONFIG_DIR>/homeserver.yaml`.
* `SYNAPSE_WORKER`: module to execute, used when running synapse with workers.
Defaults to `synapse.app.homeserver`, which is suitable for non-worker mode.
* `UID`, `GID`: the user and group id to run Synapse as. Defaults to `991`, `991`.
* `UID`, `GID`: the user and group id to run Synapse as. If unset, and no user
is set via `docker run --user`, defaults to `991`, `991`. Note that this user
must have permission to read the config files, and write to the data directories.
* `TZ`: the [timezone](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) the container will run with. Defaults to `UTC`.
For more complex setups (e.g. for workers) you can also pass your args directly to synapse using `run` mode. For example like this:
@@ -186,7 +189,7 @@ point to another Dockerfile.
## Disabling the healthcheck
If you are using a non-standard port or tls inside docker you can disable the healthcheck
whilst running the above `docker run` commands.
whilst running the above `docker run` commands.
```
--no-healthcheck
@@ -212,7 +215,7 @@ If you wish to point the healthcheck at a different port with docker command, ad
## Setting the healthcheck in docker-compose file
You can add the following to set a custom healthcheck in a docker compose file.
You will need docker-compose version >2.1 for this to work.
You will need docker-compose version >2.1 for this to work.
```
healthcheck:
@@ -226,5 +229,5 @@ healthcheck:
## Using jemalloc
Jemalloc is embedded in the image and will be used instead of the default allocator.
You can read about jemalloc by reading the Synapse
You can read about jemalloc by reading the Synapse
[README](https://github.com/matrix-org/synapse/blob/HEAD/README.rst#help-synapse-is-slow-and-eats-all-my-ram-cpu).

Some files were not shown because too many files have changed in this diff Show More