* commit 'f88c48f3b':
1.18.0rc1
Fix error reporting when using `opentracing.trace` (#7961)
Fix typing replication not being handled on master (#7959)
Remove hacky error handling for inlineDeferreds. (#7950)
Convert tests/rest/admin/test_room.py to unix file endings (#7953)
Support oEmbed for media previews. (#7920)
Convert state resolution to async/await (#7942)
Fix up types and comments that refer to Deferreds. (#7945)
Do not convert async functions to Deferreds in the interactive_auth_handler (#7944)
Convert more of the media code to async/await (#7873)
Return an empty body for OPTIONS requests. (#7886)
Downgrade warning on client disconnect to INFO (#7928)
Convert presence handler helpers to async/await. (#7939)
Update the auth providers to be async. (#7935)
Put a cache on `/state_ids` (#7931)
* commit '43726783e': (22 commits)
1.17.0rc1
Fix some spelling mistakes / typos. (#7811)
`update_membership` declaration: now always returns an event id. (#7809)
Improve stacktraces from exceptions in background processes (#7808)
Fix `can only concatenate list (not "tuple") to list` exception (#7810)
Pass original request headers from workers to the main process. (#7797)
Generate real events when we reject invites (#7804)
Add `HomeServer.signing_key` property (#7805)
Revert "Update the installation docs on apt-transport-https (#7801)"
Do not use simplejson in Synapse. (#7800)
Stop passing bytes when dumping JSON (#7799)
Update the installation docs on apt-transport-https (#7801)
shuffle changelog slightly
Change Caddy links (old is deprecated) (#7789)
Stop populating unused table `local_invites`. (#7793)
Refactor getting replication updates from database v2. (#7740)
Add libwebp dependency to Dockerfile (#7791)
Add documentation for JWT login type and improve sample config. (#7776)
Convert the appservice handler to async/await. (#7775)
Don't ignore `set_tweak` actions with no explicit `value`. (#7766)
...
* commit 'dc80a0762':
1.16.0rc1
Back out MSC2625 implementation (#7761)
Additional configuration options for auto-join rooms (#7763)
Add some metrics for inbound and outbound federation processing times (#7755)
Explain the purpose of the "tests" conditional dependency requirement (#7751)
Add another yield point to state res v2 (#7746)
Move flake8 to end. Don't exit script on failure (#7738)
Make tox actions work on Debian 10 (#7703)
Yield during large v2 state res. (#7735)
add org.matrix.login.jwt so that m.login.jwt can be deprecated (#7675)
Set Content-Length for Metrics requests (#7730)
Sync ignored table names in synapse_port_db to current database schema (#7717)
Allow local media to be marked as safe from being quarantined. (#7718)
Convert directory handler to async/await (#7727)
Speed up state res v2 across large state differences. (#7725)
* commit '46613aaf7': (27 commits)
Incorporate review
Lint
Incorporate review bits
Pre-populate the unread_count column
Don't update the schema version
Use attr instead of a dict
Lint
Test that a mark_unread action updates the right counter when using a slave store
Remove debug logging
Test that a mark_unread action updates the right counter
Fix summary rotation
Log for invalid values of notif
Fix SQL
Fix schema update
Lint
Save the count of unread messages to event_push_summary
Actually act on mark_unread
Appease mypy
Lint
Use temporary prefixes as per the MSC
...
* 'release-v1.15.0' of github.com:matrix-org/synapse: (55 commits)
1.15.0
Fix some attributions
Update CHANGES.md
1.15.0rc1
Revert "1.15.0rc1"
1.15.0rc1
Fix bug in account data replication stream. (#7656)
Convert the registration handler to async/await. (#7649)
Accept device information at the login fallback endpoint. (#7629)
Convert user directory handler and related classes to async/await. (#7640)
Add an option to disable autojoin for guest accounts (#6637)
Clarifications to the admin api documentation (#7647)
Update to the stable SSO prefix for UI Auth. (#7630)
Fix type information on `assert_*_is_admin` methods (#7645)
Remove some unused constants. (#7644)
Typo fixes.
Allow new users to be registered via the admin API even if the monthly active user limit has been reached (#7263)
Add device management to admin API (#7481)
Attempt to fix PhoneHomeStatsTestCase.test_performance_100 being flaky. (#7634)
Support CS API v0.6.0 (#6585)
...
* 'release-v1.14.0' of github.com:matrix-org/synapse: (108 commits)
Fix typo in PR link
Update debian changelog
1.14.0
Improve changelog wording
1.14.0rc2
Fix sample config docs error (#7581)
Fix up comments
Fix specifying cache factors via env vars with * in name. (#7580)
Don't apply cache factor to event cache. (#7578)
Ensure ReplicationStreamer is always started when replication enabled. (#7579)
Remove the changes to the debian changelog
Not full release yet, this is rc1
Merge event persistence move changelog entries
More changelog fix
Changelog fixes
1.14.0
Replace device_27_unique_idx bg update with a fg one (#7562)
Fix incorrect exception handling in KeyUploadServlet.on_POST (#7563)
Fix recording of federation stream token (#7564)
Simplify reap_monthly_active_users (#7558)
...
* 'release-v1.13.0' of github.com:matrix-org/synapse: (257 commits)
Update changelog based on feedback.
Move warnings in the changelog and re-iterate changes to branches.
1.13.0
update dh-virtualenv (#7526)
1.13.0rc3
Hash passwords earlier in the registration process (#7523)
1.13.0rc2
1.13.0rc2
Stop `get_joined_users` corruption from custom statuses (#7376)
Do not validate that the client dict is stable during UI Auth. (#7483)
Fix new flake8 errors (#7489)
Don't UPGRADE database rows
RST indenting
Put rollback instructions in upgrade notes
Fix changelog typo
Oh yeah, RST
Absolute URL it is then
Fix upgrade notes link
Provide summary of upgrade issues in changelog. Fix )
Move next version notes from changelog to upgrade notes
...
Calls `self.get_success` on all deferred methods instead of abusing `self.pump()`. This has the benefit of working with coroutines, as well as checking that method execution completed successfully.
There are also a few small cleanups that I made in the process.
The idea here is that if an instance persists an event via the replication HTTP API it can return before we receive that event over replication, which can lead to races where code assumes that persisting an event immediately updates various caches (e.g. current state of the room).
Most of Synapse doesn't hit such races, so we don't do the waiting automagically, instead we do so where necessary to avoid unnecessary delays. We may decide to change our minds here if it turns out there are a lot of subtle races going on.
People probably want to look at this commit by commit.
Synapse 1.13.0rc2 (2020-05-14)
==============================
Bugfixes
--------
- Fix a long-standing bug which could cause messages not to be sent over federation, when state events with state keys matching user IDs (such as custom user statuses) were received. ([\#7376](https://github.com/matrix-org/synapse/issues/7376))
- Restore compatibility with non-compliant clients during the user interactive authentication process, fixing a problem introduced in v1.13.0rc1. ([\#7483](https://github.com/matrix-org/synapse/issues/7483))
Internal Changes
----------------
- Fix linting errors in new version of Flake8. ([\#7470](https://github.com/matrix-org/synapse/issues/7470))
Fix a bug where the `get_joined_users` cache could be corrupted by custom
status events (or other state events with a state_key matching the user ID).
The bug was introduced by #2229, but has largely gone unnoticed since then.
Fixes#7099, #7373.
The aim here is to get to a stage where we have a `PersistEventStore` that holds all the write methods used during event persistence, so that we can take that class out of the `DataStore` mixin and instansiate it separately. This will allow us to instansiate it on processes other than master, while also ensuring it is only available on processes that are configured to write to events stream.
This is a bit of an architectural change, where we end up with multiple classes per data store (rather than one per data store we have now). We end up having:
1. Storage classes that provide high level APIs that can talk to multiple data stores.
2. Data store modules that consist of classes that must point at the same database instance.
3. Classes in a data store that can be instantiated on processes depending on config.
* commit 'd20c34654':
port BackgroundUpdateTestCase to HomeserverTestCase (#6653)
changelog
Fix exceptions in log when rejected event is replicated
async/await for SyncReplicationHandler.process_and_notify
Clarify documentation on get_event* methods
* commit '1807db5e7':
changelog
Remove unused get_latest_event_ids_and_hashes_in_room
Remove unused get_prev_events_and_hashes_for_room
Remove unused hashes and depths from _update_membership params
Remove unused hashes and depths from create_event params
Remove unused hashes and depths from create_new_client_event params
replace get_prev_events_and_hashes_for_room with get_prev_events_for_room in create_new_client_event
rename get_prev_events_for_room to get_prev_events_and_hashes_for_room
* commit 'adfdd82b2':
Back out perf regression from get_cross_signing_keys_from_cache. (#6494)
Fix comment
Fix port db script
Newsfile
Fixup tests
Move start up DB checks to main data store.
Pass Database into the data store
Move are_all_users_on_domain checks to main data store.
Change DataStores to accept 'database' param.
* commit 'f3ea2f5a0':
Remove unused var
Fix DB scripts
Newsfile
Move background update handling out of store
Comments
Move DB pool and helper functions into dedicated Database class
* Add 'device_lists_outbound_pokes' as extra table.
This makes sure we check all the relevant tables to get the current max
stream ID.
Currently not doing so isn't problematic as the max stream ID in
`device_lists_outbound_pokes` is the same as in `device_lists_stream`,
however that will change.
* Change device lists stream to have one row per id.
This will make it possible to process the streams more incrementally,
avoiding having to process large chunks at once.
* Change device list replication to match new semantics.
Instead of sending down batches of user ID/host tuples, send down a row
per entity (user ID or host).
* Newsfile
* Remove handling of multiple rows per ID
* Fix worker handling
* Comments from review
