* commit '46613aaf7': (27 commits)
Incorporate review
Lint
Incorporate review bits
Pre-populate the unread_count column
Don't update the schema version
Use attr instead of a dict
Lint
Test that a mark_unread action updates the right counter when using a slave store
Remove debug logging
Test that a mark_unread action updates the right counter
Fix summary rotation
Log for invalid values of notif
Fix SQL
Fix schema update
Lint
Save the count of unread messages to event_push_summary
Actually act on mark_unread
Appease mypy
Lint
Use temporary prefixes as per the MSC
...
* 'release-v1.15.0' of github.com:matrix-org/synapse: (55 commits)
1.15.0
Fix some attributions
Update CHANGES.md
1.15.0rc1
Revert "1.15.0rc1"
1.15.0rc1
Fix bug in account data replication stream. (#7656)
Convert the registration handler to async/await. (#7649)
Accept device information at the login fallback endpoint. (#7629)
Convert user directory handler and related classes to async/await. (#7640)
Add an option to disable autojoin for guest accounts (#6637)
Clarifications to the admin api documentation (#7647)
Update to the stable SSO prefix for UI Auth. (#7630)
Fix type information on `assert_*_is_admin` methods (#7645)
Remove some unused constants. (#7644)
Typo fixes.
Allow new users to be registered via the admin API even if the monthly active user limit has been reached (#7263)
Add device management to admin API (#7481)
Attempt to fix PhoneHomeStatsTestCase.test_performance_100 being flaky. (#7634)
Support CS API v0.6.0 (#6585)
...
* 'release-v1.14.0' of github.com:matrix-org/synapse: (108 commits)
Fix typo in PR link
Update debian changelog
1.14.0
Improve changelog wording
1.14.0rc2
Fix sample config docs error (#7581)
Fix up comments
Fix specifying cache factors via env vars with * in name. (#7580)
Don't apply cache factor to event cache. (#7578)
Ensure ReplicationStreamer is always started when replication enabled. (#7579)
Remove the changes to the debian changelog
Not full release yet, this is rc1
Merge event persistence move changelog entries
More changelog fix
Changelog fixes
1.14.0
Replace device_27_unique_idx bg update with a fg one (#7562)
Fix incorrect exception handling in KeyUploadServlet.on_POST (#7563)
Fix recording of federation stream token (#7564)
Simplify reap_monthly_active_users (#7558)
...
* 'release-v1.13.0' of github.com:matrix-org/synapse: (257 commits)
Update changelog based on feedback.
Move warnings in the changelog and re-iterate changes to branches.
1.13.0
update dh-virtualenv (#7526)
1.13.0rc3
Hash passwords earlier in the registration process (#7523)
1.13.0rc2
1.13.0rc2
Stop `get_joined_users` corruption from custom statuses (#7376)
Do not validate that the client dict is stable during UI Auth. (#7483)
Fix new flake8 errors (#7489)
Don't UPGRADE database rows
RST indenting
Put rollback instructions in upgrade notes
Fix changelog typo
Oh yeah, RST
Absolute URL it is then
Fix upgrade notes link
Provide summary of upgrade issues in changelog. Fix )
Move next version notes from changelog to upgrade notes
...
Calls `self.get_success` on all deferred methods instead of abusing `self.pump()`. This has the benefit of working with coroutines, as well as checking that method execution completed successfully.
There are also a few small cleanups that I made in the process.
The idea here is that if an instance persists an event via the replication HTTP API it can return before we receive that event over replication, which can lead to races where code assumes that persisting an event immediately updates various caches (e.g. current state of the room).
Most of Synapse doesn't hit such races, so we don't do the waiting automagically, instead we do so where necessary to avoid unnecessary delays. We may decide to change our minds here if it turns out there are a lot of subtle races going on.
People probably want to look at this commit by commit.
Synapse 1.13.0rc2 (2020-05-14)
==============================
Bugfixes
--------
- Fix a long-standing bug which could cause messages not to be sent over federation, when state events with state keys matching user IDs (such as custom user statuses) were received. ([\#7376](https://github.com/matrix-org/synapse/issues/7376))
- Restore compatibility with non-compliant clients during the user interactive authentication process, fixing a problem introduced in v1.13.0rc1. ([\#7483](https://github.com/matrix-org/synapse/issues/7483))
Internal Changes
----------------
- Fix linting errors in new version of Flake8. ([\#7470](https://github.com/matrix-org/synapse/issues/7470))
Fix a bug where the `get_joined_users` cache could be corrupted by custom
status events (or other state events with a state_key matching the user ID).
The bug was introduced by #2229, but has largely gone unnoticed since then.
Fixes#7099, #7373.
The aim here is to get to a stage where we have a `PersistEventStore` that holds all the write methods used during event persistence, so that we can take that class out of the `DataStore` mixin and instansiate it separately. This will allow us to instansiate it on processes other than master, while also ensuring it is only available on processes that are configured to write to events stream.
This is a bit of an architectural change, where we end up with multiple classes per data store (rather than one per data store we have now). We end up having:
1. Storage classes that provide high level APIs that can talk to multiple data stores.
2. Data store modules that consist of classes that must point at the same database instance.
3. Classes in a data store that can be instantiated on processes depending on config.
* commit 'd20c34654':
port BackgroundUpdateTestCase to HomeserverTestCase (#6653)
changelog
Fix exceptions in log when rejected event is replicated
async/await for SyncReplicationHandler.process_and_notify
Clarify documentation on get_event* methods
* commit '1807db5e7':
changelog
Remove unused get_latest_event_ids_and_hashes_in_room
Remove unused get_prev_events_and_hashes_for_room
Remove unused hashes and depths from _update_membership params
Remove unused hashes and depths from create_event params
Remove unused hashes and depths from create_new_client_event params
replace get_prev_events_and_hashes_for_room with get_prev_events_for_room in create_new_client_event
rename get_prev_events_for_room to get_prev_events_and_hashes_for_room
* commit 'adfdd82b2':
Back out perf regression from get_cross_signing_keys_from_cache. (#6494)
Fix comment
Fix port db script
Newsfile
Fixup tests
Move start up DB checks to main data store.
Pass Database into the data store
Move are_all_users_on_domain checks to main data store.
Change DataStores to accept 'database' param.
* commit 'f3ea2f5a0':
Remove unused var
Fix DB scripts
Newsfile
Move background update handling out of store
Comments
Move DB pool and helper functions into dedicated Database class
* Add 'device_lists_outbound_pokes' as extra table.
This makes sure we check all the relevant tables to get the current max
stream ID.
Currently not doing so isn't problematic as the max stream ID in
`device_lists_outbound_pokes` is the same as in `device_lists_stream`,
however that will change.
* Change device lists stream to have one row per id.
This will make it possible to process the streams more incrementally,
avoiding having to process large chunks at once.
* Change device list replication to match new semantics.
Instead of sending down batches of user ID/host tuples, send down a row
per entity (user ID or host).
* Newsfile
* Remove handling of multiple rows per ID
* Fix worker handling
* Comments from review
It was originally implemented by pulling the full auth chain of all
state sets out of the database and doing set comparison. However, that
can take a lot work if the state and auth chains are large.
Instead, lets try and fetch the auth chains at the same time and
calculate the difference on the fly, allowing us to bail early if all
the auth chains converge. Assuming that the auth chains do converge more
often than not, this should improve performance. Hopefully.
* commit 'f713c01e2':
Move type annotation into docstring
Fix deleting state groups during room purge.
Use correct type annotation
Change to not require a state_groups.room_id index.
Fix up comment
Update log line to lie a little less
Add state_groups.room_id index
Docstrings
Fix purge room API
Newsfile
Split purge API into events vs state
* commit '53d7680e3':
Update synapse/storage/data_stores/main/devices.py
rename get_devices_by_remote to get_device_updates_by_remote
black
apply changes as a result of PR review
don't error if federation query doesn't have cross-signing keys
move get_e2e_cross_signing_key to EndToEndKeyWorkerStore so it works with workers
black
vendor-prefix the EDU name until MSC1756 is merged into the spec
fix unit test
add news file
update to work with newer code, and fix formatting
add missing param
make black happy
don't crash if the user doesn't have cross-signing keys
implement federation parts of cross-signing
