* commit '3950ae51e':
Ensure that remove_pusher is always async (#7981)
Ensure the msg property of HttpResponseException is a string. (#7979)
Remove from the event_relations table when purging historical events. (#7978)
Add additional logging for SAML sessions. (#7971)
Add MSC reference to changelog for #7736
Re-implement unread counts (#7736)
Various improvements to the docs (#7899)
Convert storage layer to async/await. (#7963)
Add an option to disable purge in delete room admin API (#7964)
Move some log lines from default logger to sql/transaction loggers (#7952)
Use the JSON module from the std library instead of simplejson. (#7936)
Fix exit code for `check_line_terminators.sh` (#7970)
Option to allow server admins to join complex rooms (#7902)
Fix typo in metrics docs (#7966)
Add script for finding files with unix line terminators (#7965)
Convert the remaining media repo code to async / await. (#7947)
Convert a synapse.events to async/await. (#7949)
Convert groups and visibility code to async / await. (#7951)
Convert push to async/await. (#7948)
https://github.com/matrix-org/synapse-dinsic/pull/51 added an option that would automatically bind a user's threepid to a configured identity server after they had registered. Unfortunately, when you bind threepids, ideally you would store that mapping in the database so that later on you can remove those mappings when you deactivate an account.
We found that due the fact that we did not store these mappings, threepids were not unbound upon user account deactivation.
This PR fixes the issue by creating the mappings again, meaning they will again be removed upon account deactivation.
If the last admin of a room departs, and thus the room no longer has any admins within it, we "freeze" the room. Freezing a room means that the power level required to do anything in the room (sending messages, inviting others etc) will require power level 100.
At the moment, an admin can come back and unfreeze the room manually. The plan is to eventually make unfreezing of the room automatic on admin rejoin, though that will be in a separate PR.
This *could* work in mainline, however if the admin who leaves is on a homeserver without this functionality, then the room isn't frozen. I imagine this would probably be pretty confusing to people. Part of this feature was allowing Synapse modules to send events, which has been implemented in mainline at https://github.com/matrix-org/synapse/pull/8479, and cherry-picked to the `dinsic` fork in 62c7b10. The actual freezing logic has been implemented here in the RoomAccessRules module.
This PR allows Synapse modules making use of the `ModuleApi` to create and send non-membership events into a room. This can useful to have modules send messages, or change power levels in a room etc. Note that they must send event through a user that's already in the room.
The non-membership event limitation is currently arbitrary, as it's another chunk of work and not necessary at the moment.
This commit has been cherry-picked from mainline.
Swap these calls around, as the check for changing `restricted` to `unrestricted` will actually change `self.restricted_room` to an unrestricted room.
Do that last, instead of first. Additionally add a comment with a warning.
This PR switches several conditions regarding room access rules to check against the status of the room's inclusion in the public room list instead of its join rules.
The code includes a snapshot of https://github.com/matrix-org/synapse/pull/8292, which will likely change in time and need merging in again.
This PR modifies the `RoomAccessRules` module, an implementation of `ThirdPartyEventRules`, to both:
* Modify the default power levels when creating a room to set:
- `invite` to be minimum PL50
- `state_default` to be minimum PL100
* Enforce this when creating the room.
This is a config option ported over from DINUM's Sydent: https://github.com/matrix-org/sydent/pull/285
They've switched to validating 3PIDs via Synapse rather than Sydent, and would like to retain this functionality.
This original purpose for this change is phishing prevention. This solution could also potentially be replaced by a similar one to https://github.com/matrix-org/synapse/pull/8004, but across all `*/submit_token` endpoint.
This option may still be useful to enterprise even with that safeguard in place though, if they want to be absolutely sure that their employees don't follow links to other domains.
* commit 'f88c48f3b':
1.18.0rc1
Fix error reporting when using `opentracing.trace` (#7961)
Fix typing replication not being handled on master (#7959)
Remove hacky error handling for inlineDeferreds. (#7950)
Convert tests/rest/admin/test_room.py to unix file endings (#7953)
Support oEmbed for media previews. (#7920)
Convert state resolution to async/await (#7942)
Fix up types and comments that refer to Deferreds. (#7945)
Do not convert async functions to Deferreds in the interactive_auth_handler (#7944)
Convert more of the media code to async/await (#7873)
Return an empty body for OPTIONS requests. (#7886)
Downgrade warning on client disconnect to INFO (#7928)
Convert presence handler helpers to async/await. (#7939)
Update the auth providers to be async. (#7935)
Put a cache on `/state_ids` (#7931)
* commit 'de119063f': (31 commits)
Convert room list handler to async/await. (#7912)
Element CSS and logo in email templates (#7919)
Lint the contrib/ directory in CI and linting scripts, add synctl to linting script (#7914)
Remove unused code from synapse.logging.utils. (#7897)
Fix a typo in the sample config. (#7890)
Fix deprecation warning: import ABC from collections.abc (#7892)
Change sample config's postgres user to synapse_user (#7889)
Fix deprecation warning due to invalid escape sequences (#7895)
Remove Ubuntu Eoan that is now EOL (#7888)
Fix the trace function for async functions. (#7872)
Add help for creating a user via docker (#7885)
Switch to Debian:Slim from Alpine for the docker image (#7839)
Stop using 'device_max_stream_id' (#7882)
Fix TypeError in synapse.notifier (#7880)
Add a default limit (of 100) to get/sync operations. (#7858)
Change "unknown room ver" logging to warning. (#7881)
Convert device handler to async/await (#7871)
Convert synapse.app to async/await. (#7868)
Convert _base, profile, and _receipts handlers to async/await (#7860)
Add admin endpoint to get members in a room. (#7842)
...
* commit 'a973bcb8a':
Add some tiny type annotations (#7870)
Remove obsolete comment.
Ensure that calls to `json.dumps` are compatible with the standard library json. (#7836)
Avoid brand new rooms in `delete_old_current_state_events` (#7854)
Allow accounts to be re-activated from the admin APIs. (#7847)
Fix tests
Fix typo
Newsfile
Use get_users_in_room rather than state handler in typing for speed
Fix client reader sharding tests (#7853)
Convert E2E key and room key handlers to async/await. (#7851)
Return the proper 403 Forbidden error during errors with JWT logins. (#7844)
remove `retry_on_integrity_error` wrapper for persist_events (#7848)
* commit '43726783e': (22 commits)
1.17.0rc1
Fix some spelling mistakes / typos. (#7811)
`update_membership` declaration: now always returns an event id. (#7809)
Improve stacktraces from exceptions in background processes (#7808)
Fix `can only concatenate list (not "tuple") to list` exception (#7810)
Pass original request headers from workers to the main process. (#7797)
Generate real events when we reject invites (#7804)
Add `HomeServer.signing_key` property (#7805)
Revert "Update the installation docs on apt-transport-https (#7801)"
Do not use simplejson in Synapse. (#7800)
Stop passing bytes when dumping JSON (#7799)
Update the installation docs on apt-transport-https (#7801)
shuffle changelog slightly
Change Caddy links (old is deprecated) (#7789)
Stop populating unused table `local_invites`. (#7793)
Refactor getting replication updates from database v2. (#7740)
Add libwebp dependency to Dockerfile (#7791)
Add documentation for JWT login type and improve sample config. (#7776)
Convert the appservice handler to async/await. (#7775)
Don't ignore `set_tweak` actions with no explicit `value`. (#7766)
...
* commit '5cdca53aa':
Merge different Resource implementation classes (#7732)
Fix inconsistent handling of upper and lower cases of email addresses. (#7021)
Allow YAML config file to contain None (#7779)
Fix a typo.
Move 1.15.2 after 1.16.0rc2.
1.16.0rc2
Remove an extraneous space.
Add links to the fixes.
Fix tense in the release notes.
Hack to add push priority to push notifications (#7765)
Add early returns to `_check_for_soft_fail` (#7769)
Use symbolic names for replication stream names (#7768)
Type checking for `FederationHandler` (#7770)
Fix new metric where we used ms instead of seconds (#7771)
Fix incorrect error message when database CTYPE was set incorrectly. (#7760)
Pin link in CHANGES.md
Fixes to CHANGES.md
* commit 'dc80a0762':
1.16.0rc1
Back out MSC2625 implementation (#7761)
Additional configuration options for auto-join rooms (#7763)
Add some metrics for inbound and outbound federation processing times (#7755)
Explain the purpose of the "tests" conditional dependency requirement (#7751)
Add another yield point to state res v2 (#7746)
Move flake8 to end. Don't exit script on failure (#7738)
Make tox actions work on Debian 10 (#7703)
Yield during large v2 state res. (#7735)
add org.matrix.login.jwt so that m.login.jwt can be deprecated (#7675)
Set Content-Length for Metrics requests (#7730)
Sync ignored table names in synapse_port_db to current database schema (#7717)
Allow local media to be marked as safe from being quarantined. (#7718)
Convert directory handler to async/await (#7727)
Speed up state res v2 across large state differences. (#7725)
* commit '46613aaf7': (27 commits)
Incorporate review
Lint
Incorporate review bits
Pre-populate the unread_count column
Don't update the schema version
Use attr instead of a dict
Lint
Test that a mark_unread action updates the right counter when using a slave store
Remove debug logging
Test that a mark_unread action updates the right counter
Fix summary rotation
Log for invalid values of notif
Fix SQL
Fix schema update
Lint
Save the count of unread messages to event_push_summary
Actually act on mark_unread
Appease mypy
Lint
Use temporary prefixes as per the MSC
...
* commit '03619324f':
Create a ListenerConfig object (#7681)
Fix changelog wording
1.15.1
Wrap register_device coroutine in an ensureDeferred (#7684)
Ensure the body is a string before comparing push rules. (#7701)
Ensure etag is a string for GET room_keys/version response (#7691)
Update m.id.phone to use 'phone' instead of 'number' (#7687)
Fix "There was no active span when trying to log." error (#7698)
Enable 3PID add/bind/unbind endpoints on r0 routes
Discard RDATA from already seen positions. (#7648)
Replace iteritems/itervalues/iterkeys with native versions. (#7692)
Fix warnings about losing log context during UI auth. (#7688)
Fix a typo when comparing the URI & method during UI Auth. (#7689)
Remove "user_id" from GET /presence. (#7606)
Increase the default SAML session expirary time to 15 minutes. (#7664)
fix typo in sample_config.yaml (#7652)
Take out a lock before modifying _CACHES (#7663)
Add option to enable encryption by default for new rooms (#7639)
Clean-up the fallback login code. (#7657)
`StatsHandler` handles updates to the `current_state_delta_stream`, and updates room stats such as the amount of state events, joined users, etc.
However, it counts every new join membership as a new user entering a room (and that user being in another room), whereas it's possible for a user's membership status to go from join -> join, for instance when they change their per-room profile information.
This PR adds a check for join->join membership transitions, and bails out early, as none of the further checks are necessary at that point.
Due to this bug, membership stats in many rooms have ended up being wildly larger than their true values. I am not sure if we also want to include a migration step which recalculates these statistics (possibly using the `_populate_stats_process_rooms` bg update).
Bug introduced in the initial implementation https://github.com/matrix-org/synapse/pull/4338.
