* commit '837293c31':
Remove obsolete __future__ imports (#8337)
Use admin_patterns for all admin APIs. (#8331)
Fix a potential bug of UnboundLocalError (#8329)
Switch metaclass initialization to python 3-compatible syntax (#8326)
Catch-up after Federation Outage (split, 4): catch-up loop (#8272)
Use slots in attrs classes where possible (#8296)
Fix typos in comments.
Add the topic and avatar to the room details admin API (#8305)
Improve SAML error messages (#8248)
Add experimental support for sharding event persister. Again. (#8294)
Make `StreamToken.room_key` be a `RoomStreamToken` instance. (#8281)
Use TLSv1.2 for fake servers in tests (#8208)
Add /_synapse/client to the reverse proxy docs (#8227)
Clean up `Notifier.on_new_room_event` code path (#8288)
* commit '2832ef5bb':
1.20.0rc3
Fix the exception that is raised when invalid JSON is encountered. (#8291)
Remove shared rooms info from upgrade/workers doc as it's still experimental (#8290)
* commit 'a3a90ee03':
Show a confirmation page during user password reset (#8004)
Do not error when thumbnailing invalid files (#8236)
Remove some unused distributor signals (#8216)
Fixup pusher pool notifications (#8287)
Revert "Fixup pusher pool notifications"
Fixup pusher pool notifications
* commit '525efab61':
1.20.0rc1
Systemd docs: configure workers to start after main process. (#8276)
Only add rows to the push actions table if the event notifies or should be marked unread (#8274)
Avoid table-scanning users at startup (#8271)
Fix unread count failing on NULL values (#8270)
Add more logging to debug slow startup (#8264)
Refuse to upgrade database on worker processes (#8266)
* commit '0d4f614fd':
Refactor `_get_e2e_device_keys_for_federation_query_txn` (#8225)
Add experimental support for sharding event persister. (#8170)
Add /user/{user_id}/shared_rooms/ api (#7785)
Do not try to store invalid data in the stats table (#8226)
Convert the main methods run by the reactor to async. (#8213)
* commit 'abeab964d':
Make _get_e2e_device_keys_and_signatures_txn return an attrs (#8224)
Fix errors when updating the user directory with invalid data (#8223)
Explain better what GDPR-erased means (#8189)
Convert additional databases to async/await part 3 (#8201)
Convert appservice code to async/await. (#8207)
Rename `_get_e2e_device_keys_txn` (#8222)
* commit '56efa9ec7': (22 commits)
Fix rate limiting unit tests. (#8167)
Add functions to `MultiWriterIdGen` used by events stream (#8164)
Do not allow send_nonmember_event to be called with shadow-banned users. (#8158)
Changelog fixes
Make StreamIdGen `get_next` and `get_next_mult` async (#8161)
Wording fixes to 'name' user admin api filter (#8163)
Fix missing double-backtick in RST document
Search in columns 'name' and 'displayname' in the admin users endpoint (#7377)
Add type hints for state. (#8140)
Stop shadow-banned users from sending non-member events. (#8142)
Allow capping a room's retention policy (#8104)
Add healthcheck for default localhost 8008 port on /health endpoint. (#8147)
Fix flaky shadow-ban tests. (#8152)
Don't fail /submit_token requests on incorrect session ID if request_token_inhibit_3pid_errors is turned on (#7991)
Do not apply ratelimiting on joins to appservices (#8139)
Micro-optimisations to get_auth_chain_ids (#8132)
Allow denying or shadow banning registrations via the spam checker (#8034)
Stop shadow-banned users from sending invites. (#8095)
Be more tolerant of membership events in unknown rooms (#8110)
Improve the error code when trying to register using a name reserved for guests. (#8135)
...
* commit 'c9c544cda':
Remove `ChainedIdGenerator`. (#8123)
Switch the JSON byte producer from a pull to a push producer. (#8116)
Updated docs: Added note about missing 308 redirect support. (#8120)
Be stricter about JSON that is accepted by Synapse (#8106)
Convert runWithConnection to async. (#8121)
Remove the unused inlineCallbacks code-paths in the caching code (#8119)
Separate `get_current_token` into two. (#8113)
Convert events worker database to async/await. (#8071)
Add a link to the matrix-synapse-rest-password-provider. (#8111)
* commit 'e04e465b4':
Use the default templates when a custom template file cannot be found (#8037)
Changelog changes
Convert stream database to async/await. (#8074)
Add a shadow-banned flag to users. (#8092)
Convert pusher databases to async/await. (#8075)
Convert receipts and events databases to async/await. (#8076)
* commit 'db131b6b2':
Change the default log config to reduce disk I/O and storage (#8040)
Implement login blocking based on SAML attributes (#8052)
Add an assertion on prev_events in create_new_client_event (#8041)
Typo
Lint
why mypy why
Lint
Incorporate review
Incorporate review
Fix PUT /pushrules to use the right rule IDs
Back out the database hack and replace it with a temporary config setting
Fix cache name
Fix cache invalidation calls
Lint
Changelog
Implement new experimental push rules with a database hack to enable them
* commit '3950ae51e':
Ensure that remove_pusher is always async (#7981)
Ensure the msg property of HttpResponseException is a string. (#7979)
Remove from the event_relations table when purging historical events. (#7978)
Add additional logging for SAML sessions. (#7971)
Add MSC reference to changelog for #7736
Re-implement unread counts (#7736)
Various improvements to the docs (#7899)
Convert storage layer to async/await. (#7963)
Add an option to disable purge in delete room admin API (#7964)
Move some log lines from default logger to sql/transaction loggers (#7952)
Use the JSON module from the std library instead of simplejson. (#7936)
Fix exit code for `check_line_terminators.sh` (#7970)
Option to allow server admins to join complex rooms (#7902)
Fix typo in metrics docs (#7966)
Add script for finding files with unix line terminators (#7965)
Convert the remaining media repo code to async / await. (#7947)
Convert a synapse.events to async/await. (#7949)
Convert groups and visibility code to async / await. (#7951)
Convert push to async/await. (#7948)
Synapse 1.20.0rc3 (2020-09-11)
==============================
Bugfixes
--------
- Fix a bug introduced in v1.20.0rc1 where the wrong exception was raised when invalid JSON data is encountered. ([\#8291](https://github.com/matrix-org/synapse/issues/8291))
This PR adds a information about forwarding `/_synapse/client` endpoints through your reverse proxy. The first of these endpoints are introduced in https://github.com/matrix-org/synapse/pull/8004.
This PR adds a confirmation step to resetting your user password between clicking the link in your email and your password actually being reset.
This is to better align our password reset flow with the industry standard of requiring a confirmation from the user after email validation.
This is a config option ported over from DINUM's Sydent: https://github.com/matrix-org/sydent/pull/285
They've switched to validating 3PIDs via Synapse rather than Sydent, and would like to retain this functionality.
This original purpose for this change is phishing prevention. This solution could also potentially be replaced by a similar one to https://github.com/matrix-org/synapse/pull/8004, but across all `*/submit_token` endpoint.
This option may still be useful to enterprise even with that safeguard in place though, if they want to be absolutely sure that their employees don't follow links to other domains.
This is a config option ported over from DINUM's Sydent: https://github.com/matrix-org/sydent/pull/285
They've switched to validating 3PIDs via Synapse rather than Sydent, and would like to retain this functionality.
This original purpose for this change is phishing prevention. This solution could also potentially be replaced by a similar one to https://github.com/matrix-org/synapse/pull/8004, but across all `*/submit_token` endpoint.
This option may still be useful to enterprise even with that safeguard in place though, if they want to be absolutely sure that their employees don't follow links to other domains.
* Add shared_rooms api
* Add changelog
* Add .
* Wrap response in {"rooms": }
* linting
* Add unstable_features key
* Remove options from isort that aren't part of 5.x
`-y` and `-rc` are now default behaviour and no longer exist.
`dont-skip` is no longer required
https://timothycrosley.github.io/isort/CHANGELOG/#500-penny-july-4-2020
* Update imports to make isort happy
* Add changelog
* Update tox.ini file with correct invocation
* fix linting again for isort
* Vendor prefix unstable API
* Fix to match spec
* import Codes
* import Codes
* Use FORBIDDEN
* Update changelog.d/7785.feature
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Implement get_shared_rooms_for_users
* a comma
* trailing whitespace
* Handle the easy feedback
* Switch to using runInteraction
* Add tests
* Feedback
* Seperate unstable endpoint from v2
* Add upgrade node
* a line
* Fix style by adding a blank line at EOF.
* Update synapse/storage/databases/main/user_directory.py
Co-authored-by: Tulir Asokan <tulir@maunium.net>
* Update synapse/storage/databases/main/user_directory.py
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Update UPGRADE.rst
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Fix UPGRADE/CHANGELOG unstable paths
unstable unstable unstable
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Tulir Asokan <tulir@maunium.net>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Tulir Asokan <tulir@maunium.net>
We do this to prevent foot guns. The default config uses a MemoryFilter,
but users are free to change to logging to files directly. If they do
then they have to ensure to set the `filters: [context]` on the right
handler, otherwise records get written with the wrong context.
Instead we move the logic to happen when we generate a record, which is
when we *log* rather than *handle*.
(It's possible to add filters to loggers in the config, however they
don't apply to descendant loggers and so they have to be manually set on
*every* logger used in the code base)
* Change default log config to buffer by default.
This batches up writes to the filesystem, which is more efficient for
disk I/O. This means that it can take some time for logs to get written
to disk. Note that ERROR logs (and above) immediately flush the buffer.
This only effects new installs, as we only write the log config if
started with `--generate-config` (in the same way we do for generating
signing keys).
* Default to keeping last 4 days of logs.
This hopefully reduces the amount of logs kept for new servers. Keeping
the last 1GB of logs is likely overkill for new servers, but equally may
not be enough for busy ones.
Instead, we keep the last four days worth of logs, enough so that admins
can investigate any problems that happened over e.g. a long weekend.
* commit 'f88c48f3b':
1.18.0rc1
Fix error reporting when using `opentracing.trace` (#7961)
Fix typing replication not being handled on master (#7959)
Remove hacky error handling for inlineDeferreds. (#7950)
Convert tests/rest/admin/test_room.py to unix file endings (#7953)
Support oEmbed for media previews. (#7920)
Convert state resolution to async/await (#7942)
Fix up types and comments that refer to Deferreds. (#7945)
Do not convert async functions to Deferreds in the interactive_auth_handler (#7944)
Convert more of the media code to async/await (#7873)
Return an empty body for OPTIONS requests. (#7886)
Downgrade warning on client disconnect to INFO (#7928)
Convert presence handler helpers to async/await. (#7939)
Update the auth providers to be async. (#7935)
Put a cache on `/state_ids` (#7931)
* commit 'de119063f': (31 commits)
Convert room list handler to async/await. (#7912)
Element CSS and logo in email templates (#7919)
Lint the contrib/ directory in CI and linting scripts, add synctl to linting script (#7914)
Remove unused code from synapse.logging.utils. (#7897)
Fix a typo in the sample config. (#7890)
Fix deprecation warning: import ABC from collections.abc (#7892)
Change sample config's postgres user to synapse_user (#7889)
Fix deprecation warning due to invalid escape sequences (#7895)
Remove Ubuntu Eoan that is now EOL (#7888)
Fix the trace function for async functions. (#7872)
Add help for creating a user via docker (#7885)
Switch to Debian:Slim from Alpine for the docker image (#7839)
Stop using 'device_max_stream_id' (#7882)
Fix TypeError in synapse.notifier (#7880)
Add a default limit (of 100) to get/sync operations. (#7858)
Change "unknown room ver" logging to warning. (#7881)
Convert device handler to async/await (#7871)
Convert synapse.app to async/await. (#7868)
Convert _base, profile, and _receipts handlers to async/await (#7860)
Add admin endpoint to get members in a room. (#7842)
...
* commit 'a973bcb8a':
Add some tiny type annotations (#7870)
Remove obsolete comment.
Ensure that calls to `json.dumps` are compatible with the standard library json. (#7836)
Avoid brand new rooms in `delete_old_current_state_events` (#7854)
Allow accounts to be re-activated from the admin APIs. (#7847)
Fix tests
Fix typo
Newsfile
Use get_users_in_room rather than state handler in typing for speed
Fix client reader sharding tests (#7853)
Convert E2E key and room key handlers to async/await. (#7851)
Return the proper 403 Forbidden error during errors with JWT logins. (#7844)
remove `retry_on_integrity_error` wrapper for persist_events (#7848)
