1
0

Compare commits

...

300 Commits

Author SHA1 Message Date
Neil Johnson a17421fb23 towncrier 2019-03-28 17:27:53 +00:00
Neil Johnson 4f1d7d29a6 remove log line for password 2019-03-28 17:16:50 +00:00
Andrew Morgan 7a91b9d81c Allow password providers to bind emails (#4947)
This PR allows password provider modules to bind email addresses when a user is registering and is motivated by matrix-org/matrix-synapse-ldap3#58
2019-03-28 15:48:07 +00:00
Erik Johnston 248014379e Merge pull request #4942 from matrix-org/erikj/fix_presence
Use event streams to calculate presence
2019-03-28 14:38:31 +00:00
Erik Johnston 4e5f0f7ca0 Use an assert 2019-03-28 14:05:05 +00:00
Erik Johnston 40e56997bc Review comments 2019-03-28 13:48:41 +00:00
Richard van der Hoff 4eeb2c2f07 Merge pull request #4953 from matrix-org/rav/refactor_replication_streams
Split up replication.tcp.streams into smaller files
2019-03-28 13:43:25 +00:00
Amber Brown 2e060774ad Run black on some storage modules that the stats branch touches (#4959) 2019-03-29 00:37:16 +11:00
Richard van der Hoff 91c3513668 changelog 2019-03-27 21:30:01 +00:00
Richard van der Hoff 71dcb275f1 move FederationStream out to its own file 2019-03-27 21:13:14 +00:00
Richard van der Hoff aa1e017864 move EventsStream out to its own file 2019-03-27 21:13:14 +00:00
Richard van der Hoff a5798de067 Move replication.tcp.streams into a package 2019-03-27 21:13:14 +00:00
Richard van der Hoff acaa18f7dd Fix/improve some docstrings in the replication code. (#4949) 2019-03-27 21:12:36 +00:00
Erik Johnston d5a5d1c632 Newsfile 2019-03-27 13:41:36 +00:00
Erik Johnston b7fa834c40 Add unit tests 2019-03-27 13:41:36 +00:00
Erik Johnston 197fae1639 Use event streams to calculate presence
Primarily this fixes a bug in the handling of remote users joining a
room where the server sent out the presence for all local users in the
room to all servers in the room.

We also change to using the state delta stream, rather than the
distributor, as it will make it easier to split processing out of the
master process (as well as being more flexible).

Finally, when sending presence states to newly joined servers we filter
out old presence states to reduce the number sent. Initially we filter
out states that are offline and have a last active more than a week ago,
though this can be changed down the line.

Fixes #3962
2019-03-27 13:41:36 +00:00
Andrew Morgan bbd244c7b2 Support 3PID login in password providers (#4931)
Adds a new method, check_3pid_auth, which gives password providers
the chance to allow authentication with third-party identifiers such
as email or msisdn.
2019-03-26 17:48:30 +00:00
Amber Brown 903f04c21f Use the state event amount for userdir import batching, not room count (#4944) 2019-03-27 02:49:28 +11:00
Andrew Morgan 4a125be138 Make federation endpoints more tolerant of trailing slashes v2 (#4935)
Redo of https://github.com/matrix-org/synapse/pull/4840
2019-03-26 11:35:29 +00:00
Richard van der Hoff 8cbbedaa2b Fix ClientReplicationStreamProtocol.__str__ (#4929)
`__str__` depended on `self.addr`, which was absent from
ClientReplicationStreamProtocol, so attempting to call str on such an object
would raise an exception.

We can calculate the peer addr from the transport, so there is no need for addr
anyway.
2019-03-25 16:41:51 +00:00
Richard van der Hoff 9bde730ef8 Fix bug where read-receipts lost their timestamps (#4927)
Make sure that they are sent correctly over the replication stream.

Fixes: #4898
2019-03-25 16:38:05 +00:00
Richard van der Hoff a54a44734f Use an explicit dbname for postgres connections in the tests. (#4928)
I don't have a database with the same name as my user, so leaving the database
name unset fails.

While we're at it, clear out some unused stuff in the test setup.
2019-03-25 16:36:56 +00:00
Richard van der Hoff 7105057cf2 Fix nginx example in ACME doc. (#4923) 2019-03-25 09:59:36 +00:00
Amber Brown ac396a0d32 Refactor out state delta handling into its own class (#4917) 2019-03-25 20:37:08 +11:00
Erik Johnston 5fee9d8067 Merge pull request #4869 from matrix-org/erikj/yaml_load
Fix yaml warnings by using safe_load
2019-03-22 11:58:13 +00:00
Erik Johnston d21a4d6be6 Newsfile 2019-03-22 10:24:58 +00:00
Erik Johnston 3677548a82 Use yaml safe_load 2019-03-22 10:20:17 +00:00
Richard van der Hoff 224783a73f Allow newsfragments to end with exclamation marks! (#4912) 2019-03-21 15:28:19 +00:00
Amber Brown a68e00fca8 Some more porting to HomeserverTestCase and remove old RESTHelper (#4913) 2019-03-22 02:10:21 +11:00
Andrew Morgan 7bef97dfb7 Remove trailing slashes from outbound federation requests and retry on 400 (#4840)
As per #3622, we remove trailing slashes from outbound federation requests. However, to ensure that we remain backwards compatible with previous versions of Synapse, if we receive a HTTP 400 with `M_UNRECOGNIZED`, then we are likely talking to an older version of Synapse in which case we retry with a trailing slash appended to the request path.
2019-03-21 15:07:28 +00:00
Andrew Morgan b41c2eaadc Clean up backoff_on_404 and metehod calls 2019-03-21 14:32:47 +00:00
Erik Johnston 01e6b405be Merge pull request #4908 from matrix-org/erikj/block_peek_on_blocked_rooms
Deny peeking into rooms that have been blocked
2019-03-21 14:07:17 +00:00
Richard van der Hoff 27813b4ca1 Update changelog.d/4908.bugfix
Co-Authored-By: erikjohnston <erikj@jki.re>
2019-03-21 14:05:59 +00:00
Colin W ab4e4c6c2f Update Apache Setup To Remove Location Syntax (#4870)
This one should close #4841. Many thanks to @dev4223 for bringing it up and finding a solution.

Signed-off-by: Colin White
2019-03-21 14:05:56 +00:00
Erik Johnston d3f640f0ac isort 2019-03-21 11:29:48 +00:00
Erik Johnston 017ed9d423 Newsfile 2019-03-21 11:26:47 +00:00
Erik Johnston 3959858eaa Merge pull request #4904 from matrix-org/erikj/fix_shutdown
Fixup shutdown room API
2019-03-21 11:24:42 +00:00
Erik Johnston cd80cbffea Fix typo and add description 2019-03-21 11:24:04 +00:00
Erik Johnston 536a266520 Deny peeking into rooms that have been blocked 2019-03-21 11:20:13 +00:00
Erik Johnston 4a8a1ac962 Rejig testcase to make it more extensible 2019-03-21 11:02:11 +00:00
Erik Johnston 9c9e618b93 Remove debug 2019-03-21 10:58:56 +00:00
Erik Johnston 5c6f61f81c Add tests 2019-03-21 10:51:21 +00:00
Erik Johnston 3ecec5ede2 Fix upsert 2019-03-21 10:21:15 +00:00
Erik Johnston 09f991a63d Merge pull request #4896 from matrix-org/erikj/disable_room_directory
Add option to disable search room lists
2019-03-21 10:16:54 +00:00
Richard van der Hoff a6f2d3053d Log requests which are simulated by the unit tests. (#4905)
Rather than stubbing out the access_log, make it actually log the requests,
which makes it a lot more obvious what is going on during tests.
2019-03-20 18:00:02 +00:00
Erik Johnston cd62981a6a Revert spurious delete 2019-03-20 17:51:27 +00:00
Erik Johnston 8d8834d3e7 comment block_room 2019-03-20 17:49:56 +00:00
Erik Johnston aa959a6c07 Use flags 2019-03-20 17:40:29 +00:00
Erik Johnston 7d47cc1305 Move requester check into assert_accepted_privacy_policy 2019-03-20 17:08:36 +00:00
Erik Johnston 30e69ff9b6 Newsfile 2019-03-20 16:56:55 +00:00
Erik Johnston 72a14860ab Gracefully handle failing to kick user 2019-03-20 16:54:21 +00:00
Erik Johnston 6b28890543 Log new room ID 2019-03-20 16:52:28 +00:00
Erik Johnston 74c46d81fa Only require consent for events with an associated request
There are a number of instances where a server or admin may puppet a
user to join/leave rooms, which we don't want to fail if the user has
not consented to the privacy policy. We fix this by adding a check to
test if the requester has an associated access_token, which is used as a
proxy to answer the question of whether the action is being done on
behalf of a real request from the user.
2019-03-20 16:50:23 +00:00
Erik Johnston 67d618e111 Allow blocking a room multiple times 2019-03-20 16:50:05 +00:00
Erik Johnston 263f2c9ce1 Merge pull request #4895 from matrix-org/erikj/disable_user_search
Add option to disable searching in the user dir
2019-03-20 16:47:15 +00:00
Amber Brown 4d53017432 Batching in the user directory import (#4900) 2019-03-21 03:06:36 +11:00
Richard van der Hoff cdb8036161 Add a config option for torture-testing worker replication. (#4902)
Setting this to 50 or so makes a bunch of sytests fail in worker mode.
2019-03-20 16:04:35 +00:00
Richard van der Hoff a902d13180 Batch up outgoing read-receipts to reduce federation traffic. (#4890)
Rate-limit outgoing read-receipts as per #4730.
2019-03-20 16:02:25 +00:00
Erik Johnston 3660d24ebe Add test 2019-03-20 15:16:36 +00:00
Erik Johnston cc09685830 Add test 2019-03-20 14:53:44 +00:00
Erik Johnston cd8c5b91ad Fix up sample config 2019-03-20 14:35:41 +00:00
Richard van der Hoff ab20f85c59 Update synapse/config/user_directory.py
Co-Authored-By: erikjohnston <erikj@jki.re>
2019-03-20 14:33:11 +00:00
Erik Johnston cc197a61a1 Disable publishing to room list when its disabled 2019-03-20 14:30:36 +00:00
Erik Johnston 2c90422146 Pull out config option 2019-03-20 14:25:58 +00:00
Erik Johnston 7529038e66 Return before we log 2019-03-20 14:25:28 +00:00
Erik Johnston 926f29ea6d Fix up config comments 2019-03-20 14:24:53 +00:00
Andrew Morgan 2150151abe kwargs doesn't like commas on calling funcs either. TIL 2019-03-20 14:13:32 +00:00
Andrew Morgan bb52a2e653 lint 2019-03-20 14:08:57 +00:00
Andrew Morgan cd36a1283b New test, fix issues 2019-03-20 14:00:39 +00:00
Andrew Morgan c69df5d5d3 Fix comments. v0.99.2 -> v0.99.3 2019-03-20 11:27:18 +00:00
Andrew Morgan 551ea11559 Just return if not doing any trailing slash shennanigans 2019-03-20 11:07:36 +00:00
Andrew Morgan 94cb7939e4 Federation test fixed! 2019-03-20 10:50:44 +00:00
Erik Johnston bf5876990f Newsfile 2019-03-19 17:10:54 +00:00
Erik Johnston 213c98c00a Add option to disable search room lists
This disables both local and remote room list searching.
2019-03-19 17:10:52 +00:00
Erik Johnston 855bf4658d Update sample config 2019-03-19 16:47:04 +00:00
Erik Johnston 0891202629 Newsfile 2019-03-19 16:41:57 +00:00
Erik Johnston 320667a479 Add option to disable searching in the user dir
We still populate it, as it can still be accessed via the admin API.
2019-03-19 16:40:19 +00:00
Erik Johnston 11f2125885 Merge pull request #4894 from matrix-org/erikj/postgres_tuning
Add note on tuning postgres
2019-03-19 16:21:24 +00:00
Erik Johnston 38ae23d5c2 Newsfile 2019-03-19 16:07:07 +00:00
Erik Johnston b616a8717b Add note on tuning postgres 2019-03-19 16:05:32 +00:00
Erik Johnston 4aa0b707d2 Merge pull request #4879 from matrix-org/erikj/test_old_deps
Add py27-old test case to buildkite
2019-03-19 13:05:39 +00:00
Erik Johnston b0e767f7bb Add comment back in 2019-03-19 12:51:32 +00:00
Richard van der Hoff 5cf00c9f60 Merge pull request #4889 from matrix-org/rav/test_real_config
Use a regular HomeServerConfig object for unit tests
2019-03-19 12:27:07 +00:00
Richard van der Hoff 7872638c31 Merge pull request #4888 from matrix-org/rav/fix_disabled_hs
Enforce hs_disabled_message correctly
2019-03-19 12:22:13 +00:00
Richard van der Hoff b5d48560c7 Fix RegistrationTestCase
turns out this relies on there being a `user_consent_version` set.
2019-03-19 12:05:05 +00:00
Richard van der Hoff 07f057ac80 changelog 2019-03-19 11:44:43 +00:00
Richard van der Hoff 13bc1e0746 Use a regular HomeServerConfig object for unit tests
Rather than using a Mock for the homeserver config, use a genuine
HomeServerConfig object. This makes for a more realistic test, and means that
we don't have to keep remembering to add things to the mock config every time
we add a new config setting.
2019-03-19 11:44:43 +00:00
Richard van der Hoff 053c50bcb3 Fix resource limits tests
Make sure that we have a `server_notices_mxid` set, given that we are relying
on it.
2019-03-19 11:44:43 +00:00
Richard van der Hoff 45bb54a6c6 Fix registration test
* Set allow_guest_access = True, since we rely on it
* config doesn't have a `hostname` attribute; it is `server_name`
2019-03-19 11:44:43 +00:00
Richard van der Hoff 8c1774e821 Fix email test
The Mailer expects the config object to have `email_smtp_pass` and
`email_riot_base_url` attributes (and it won't by default, because the default
config impl doesn't set any of the attributes unless email_enable_notifs is
set).
2019-03-19 11:44:43 +00:00
Richard van der Hoff 45c4e19c74 Merge remote-tracking branch 'origin/develop' into HEAD 2019-03-19 11:43:46 +00:00
Neil Johnson 88f0675967 fix test_auto_create_auto_join_where_no_consent (#4886) 2019-03-19 11:38:59 +00:00
Richard van der Hoff 0dbfae03f9 Enforce hs_disabled_message correctly
Fixes a bug where hs_disabled_message was not enforced for 3pid-based requests
if there was no server_notices_mxid configured.
2019-03-19 11:30:54 +00:00
Erik Johnston e9eeca1314 Fix user directory background update (#4887) 2019-03-19 22:13:53 +11:00
Erik Johnston cfc5a442ac Update newsfile 2019-03-19 10:56:11 +00:00
Richard van der Hoff d2a537ea60 Merge remote-tracking branch 'origin/master' into develop 2019-03-19 10:37:50 +00:00
Michael Kaye 9482a84c0a Repoint docs for federation (#4881) 2019-03-19 10:37:18 +00:00
Erik Johnston 4d25624ff6 Revert changes 2019-03-19 10:28:00 +00:00
Richard van der Hoff fd463b4f5d Comment out most options in the generated config. (#4863)
Make it so that most options in the config are optional, and commented out in
the generated config.

The reasons this is a good thing are as follows:

* If we decide that we should change the default for an option, we can do so,
  and only those admins that have deliberately chosen to override that option
  will be stuck on the old setting.

* It moves us towards a point where we can get rid of the super-surprising
  feature of synapse where the default settings for the config come from the
  generated yaml.

* It makes setting up a test config for unit testing an order of magnitude
  easier (see forthcoming PR).

* It makes the generated config more consistent, and hopefully easier for users
  to understand.
2019-03-19 10:06:40 +00:00
Amber Brown 282c97327f Migrate the user directory initial population to a background task (#4864) 2019-03-19 04:50:24 +11:00
Andrew Morgan a8ad39eec7 lint 2019-03-18 17:47:39 +00:00
Andrew Morgan 621e7f37f1 Better exception handling 2019-03-18 17:45:54 +00:00
Erik Johnston 00d97668bf Bring py27-old into line with other test envs 2019-03-18 17:45:49 +00:00
Erik Johnston b6ac5e40a0 Add coverage to py27-old 2019-03-18 17:38:22 +00:00
Erik Johnston 9ef1107b33 Newsfile 2019-03-18 17:19:49 +00:00
Erik Johnston 45f97de657 Add py27-old test case to buildkite 2019-03-18 17:15:46 +00:00
Brendan Abolivier 651ad8bc96 Add ratelimiting on failed login attempts (#4865) 2019-03-18 12:57:20 +00:00
Brendan Abolivier 899e523d6d Add ratelimiting on login (#4821)
Add two ratelimiters on login (per-IP address and per-userID).
2019-03-15 17:46:16 +00:00
Richard van der Hoff 3b7ceb2c69 Merge pull request #4855 from matrix-org/rav/refactor_transaction_queue
Split TransactionQueue up
2019-03-15 12:32:11 +00:00
Richard van der Hoff 2dee441bdb Merge pull request #4852 from matrix-org/rav/move_rr_sending_to_worker
Move client receipt processing to federation sender worker.
2019-03-15 12:30:30 +00:00
Erik Johnston b0fa3f6ff3 Merge pull request #4853 from matrix-org/erikj/worker_docker_ci
Allow passing --daemonize to workers
2019-03-15 10:35:38 +00:00
Luca Corbatto a6d84190eb Add systemd setup that supports workers (#4662)
This setup is a way to manage workers with systemd. It does however not
require workers. You can use this setup without workers. You just have
to make sure that the homeserver is forking and writes its PID file
to the location the service is looking in.

The currently distributed setup in the debian package does not work in
conjunction with workers.

* Adds changelog

* Lets systemd handle the forking

Sets all services to `type=simple` and disables daemonizing on the
synapse side.

* Formats readme to 80 columns per line

* Allows for full restart of all workers

* Changes README to reflect the new setup

* Adds dot to end of changelog file

* Removes surplus word

Co-Authored-By: targodan <targodan@users.noreply.github.com>

* Adds missing word

Co-Authored-By: targodan <targodan@users.noreply.github.com>

* Fixes linebreak

Co-Authored-By: targodan <targodan@users.noreply.github.com>

* Fixes unit type
2019-03-15 09:51:46 +00:00
Aaron Raimist 2fb4ff8c89 Add some stuff back to the .gitignore (#4843)
* Add some stuff back to the .gitignore

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Reorder and remove old items from .gitignore

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-03-15 09:22:29 +00:00
Erik Johnston 6861ce3fb1 Merge pull request #4859 from matrix-org/revert-4793-anoa/trailing_slashes
Revert "Make federation endpoints more tolerant of trailing slashes for some endpoints"
2019-03-14 14:31:40 +00:00
Erik Johnston 271cb1998b Revert "Make federation endpoints more tolerant of trailing slashes for some endpoints (#4793)"
This reverts commit 290552fd83.
2019-03-14 14:30:54 +00:00
Erik Johnston 9073cfc8bd Merge pull request #4846 from matrix-org/hawkowl/userdir-search
Improve searching in the userdir
2019-03-14 13:54:15 +00:00
Erik Johnston 9ad448c1e5 Correctly handle all command line options 2019-03-14 13:32:14 +00:00
Richard van der Hoff ec3a59de50 changelog 2019-03-13 22:08:37 +00:00
Andrew Morgan ecea5af491 Correct var name 2019-03-13 21:21:03 +00:00
Andrew Morgan b2df0e8e2c receiving a 400 caused an exception. handle it 2019-03-13 21:08:10 +00:00
Andrew Morgan 9a2e22fd41 is this what purgatory feels like 2019-03-13 20:29:38 +00:00
Andrew Morgan 86c60bda15 i should have given up x3 2019-03-13 20:19:07 +00:00
Andrew Morgan 45524f2f5e i should have given up x2 2019-03-13 20:17:39 +00:00
Andrew Morgan 8d16ffaf7a i should have given up 2019-03-13 20:03:10 +00:00
Richard van der Hoff 02e23b36bc Rename and move the classes 2019-03-13 20:02:56 +00:00
Andrew Morgan 26f8e2d099 there comes a time when you should give up. but you dont 2019-03-13 19:49:08 +00:00
Andrew Morgan 5ca857ad84 as above 2019-03-13 19:35:23 +00:00
Andrew Morgan 7c0295f13c no kwargs today 2019-03-13 19:27:10 +00:00
Andrew Morgan 66cdb840a6 Or perhaps I was the one who was drunk 2019-03-13 19:18:25 +00:00
Andrew Morgan bec313818c go home python, you're drunk 2019-03-13 19:10:56 +00:00
Andrew Morgan c991e7aec7 Syntax checker is bork 2019-03-13 19:08:08 +00:00
Andrew Morgan c2d848b80d Destructure again 2019-03-13 19:04:43 +00:00
Andrew Morgan ee8ba397e8 Are you happy now 2019-03-13 18:48:31 +00:00
Andrew Morgan 9dd0e34679 Syntax test 2019-03-13 18:45:17 +00:00
Andrew Morgan 220607a618 Remove testing code 2019-03-13 18:43:40 +00:00
Andrew Morgan 660b77f362 Add missing docstring detail 2019-03-13 18:38:16 +00:00
Andrew Morgan 5526b054aa Fix syntax issues 2019-03-13 18:35:21 +00:00
Andrew Morgan 09626bfd39 Switch to wrapper function around _send_request 2019-03-13 18:26:06 +00:00
Richard van der Hoff 5d89a526f1 Factor per-destination stuff out of TransactionQueue
This is easier than having to have a million fields keyed on destination.
2019-03-13 17:48:29 +00:00
Erik Johnston f5d57d4848 Newsfile 2019-03-13 17:38:58 +00:00
Erik Johnston 72bfaf746d Allow passing --daemonize to workers 2019-03-13 17:33:54 +00:00
Richard van der Hoff 6accbd25bc changelog 2019-03-13 17:24:10 +00:00
Richard van der Hoff fdcad8eabd Move client receipt processing to federation sender worker.
This is mostly a prerequisite for #4730, but also fits with the general theme
of "move everything off the master that we possibly can".
2019-03-13 17:21:19 +00:00
Richard van der Hoff eed7271b3b declare a ReadReceipt class
I'm going to use this in queues and things, so it'll be useful to give it more
of a structure.
2019-03-13 17:20:55 +00:00
Andrew Morgan 7998ca3a66 Document using a certificate with a full chain (#4849) 2019-03-13 15:26:29 +00:00
Andrew Morgan 7d053cfe10 Retry on 400:M_UNRECOGNIZED 2019-03-13 12:10:33 +00:00
Andrew Morgan 7e75d9644b Fix paranthesis indent 2019-03-13 11:15:23 +00:00
Andrew Morgan cf301e37d8 Add workaround note 2019-03-13 11:14:43 +00:00
Neil Johnson 332b60ec68 Merge branch 'master' of github.com:matrix-org/synapse into develop 2019-03-12 17:15:21 +00:00
Andrew Morgan d42c81d724 Transfer local user's push rules on room upgrade (#4838)
Transfer push rules (notifications) on room upgrade
2019-03-12 14:42:53 +00:00
Amber Brown c0332d095f fixup 2019-03-13 01:30:54 +11:00
Andrew Morgan 97653ef1f4 Correct argument name 2019-03-12 14:30:26 +00:00
Amber Brown 797b6a63fc fixup 2019-03-13 01:17:51 +11:00
Andrew Morgan 0ea8582f8b Cleaner way of implementing trailing slashes 2019-03-12 14:11:11 +00:00
Amber Brown d306bd1b26 fixup 2019-03-12 22:38:01 +11:00
Amber Brown 81d9d1bee6 fixup 2019-03-12 22:28:48 +11:00
Amber Brown 6f5890b2fa fixup 2019-03-12 22:27:56 +11:00
Amber Brown 10480c4348 fixup 2019-03-12 21:47:14 +11:00
Amber Brown 8b618041ef fixup 2019-03-12 18:06:28 +11:00
Amber Brown c980c7e31f use the old method 2019-03-12 17:51:14 +11:00
Andrew Morgan 4868b12029 and again 2019-03-11 18:22:26 +00:00
Aaron Raimist 8ea1b41a0e Clarify what registration_shared_secret allows for (#2885) (#4844)
* Clarify what registration_shared_secret allows for (#2885)

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-03-11 18:21:52 +00:00
Andrew Morgan 802cb5dcf0 Fix syntax error 2019-03-11 18:08:28 +00:00
Andrew Morgan 66f205e93d We're calling different functions now 2019-03-11 18:01:58 +00:00
Andrew Morgan f18dca26da Merge branch 'develop' into anoa/trailing_slashes_client 2019-03-11 17:44:29 +00:00
Andrew Morgan 290552fd83 Make federation endpoints more tolerant of trailing slashes for some endpoints (#4793)
Server side of a solution towards #3622.
2019-03-11 17:44:03 +00:00
Erik Johnston 9c0e6e8e7c Merge pull request #4847 from matrix-org/erikj/fix_stacktrace_keyring
Correctly log expected errors when fetching server keys
2019-03-11 17:12:20 +00:00
Andrew Morgan a8a028dfce Merge branch 'develop' into anoa/trailing_slashes_client 2019-03-11 17:07:31 +00:00
Erik Johnston 33dfd9a288 Newsfile 2019-03-11 14:13:58 +00:00
Erik Johnston 78c563b77c Correctly log expected errors when fetching server keys 2019-03-11 14:11:10 +00:00
Amber Brown 78a6b950b3 fix 2019-03-12 00:50:28 +11:00
Amber Brown 1b77bd69fb pep8 2019-03-12 00:39:12 +11:00
Amber Brown 41a5ba1682 changelog 2019-03-12 00:38:17 +11:00
Amber Brown 5ba8ceab4c fixes 2019-03-12 00:35:31 +11:00
Amber Brown 26eefca3b7 setup master 2019-03-11 21:16:10 +11:00
Amber Brown 8da22e2b53 master startup 2019-03-11 21:13:35 +11:00
Amber Brown 30a8deeb68 Merge remote-tracking branch 'origin/develop' into hawkowl/userdir-search 2019-03-11 21:12:30 +11:00
Amber Brown 4abc988c6a initial 2019-03-11 21:11:36 +11:00
Andrew Morgan b61ac9660a Merge pull request #4839 from matrix-org/anoa/no_captcha_tests
Disable captcha registration by default in tests
2019-03-11 09:58:43 +00:00
Matthew Hodgson 2326e00bc4 fix incorrect encoding of filenames with spaces in (#2090)
fixes https://github.com/vector-im/riot-web/issues/3155
2019-03-11 09:53:45 +00:00
Erik Johnston ca7f7d84f4 Merge pull request #4837 from matrix-org/erikj/optional_prev_state
Make `prev_state` field optional
2019-03-11 09:22:33 +00:00
Andrew Morgan a5dd335cd8 lint 2019-03-08 18:25:59 +00:00
Andrew Morgan f8740d57de Add changelog 2019-03-08 18:23:54 +00:00
Andrew Morgan 64ff11019e Retry certain federation requests on 404 2019-03-08 18:22:47 +00:00
Andrew Morgan 525dd02bbe Remove trailing slashes from outbound federation requests 2019-03-08 16:55:52 +00:00
Andrew Morgan 50924ee34a Add changelog 2019-03-08 16:49:16 +00:00
Erik Johnston fe6c12e6cd Add comment to schema 2019-03-08 16:38:23 +00:00
Andrew Morgan d6e0be92fe Disable captcha registration by default in tests 2019-03-08 15:49:38 +00:00
Erik Johnston cac4723afe Newsfile 2019-03-08 15:08:34 +00:00
Erik Johnston 5536ddba75 Make prev_state field optional
The `prev_state` field on events is not specced and so synapse shouldn't
explode if an event is missing the field.

Fixes #4787
2019-03-08 15:05:32 +00:00
Erik Johnston 39e57f9728 Merge pull request #4814 from matrix-org/erikj/soft_fail_impl
Implement soft fail
2019-03-08 14:51:06 +00:00
Erik Johnston 8c4896668f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/soft_fail_impl 2019-03-08 11:44:20 +00:00
Erik Johnston 0ff8163eae Factor out soft fail checks 2019-03-08 11:26:33 +00:00
Erik Johnston 469b66c102 Merge pull request #4829 from matrix-org/erikj/device_list_seen_updates
When re-syncing device lists reset the state
2019-03-08 09:54:54 +00:00
Erik Johnston 436b1c8be1 Merge pull request #4828 from matrix-org/erikj/debug_device_lists
Add some debug logging for device list handling
2019-03-07 17:02:58 +00:00
Erik Johnston cef80da903 Newsfile 2019-03-07 16:18:02 +00:00
Erik Johnston d42b41544a When re-syncing device lists reset the state
We keep track of what stream IDs we've seen so that we know what updates
we've handled or missed. If we re-sync we don't know if the updates
we've seen are included in the re-sync (there may be a race), so we
should reset the seen updates.
2019-03-07 16:04:24 +00:00
Erik Johnston 7404fb3cdb Newsfile 2019-03-07 16:00:09 +00:00
Erik Johnston c633fc02d7 Add some debug logging for device list handling 2019-03-07 15:58:03 +00:00
Richard van der Hoff fde26e47d6 Merge pull request #4824 from matrix-org/rav/docker_docs
Document the `generate` option for the docker image.
2019-03-07 14:12:44 +00:00
Richard van der Hoff ba7a6807c8 Debian package: fix warning during preconfiguration. (#4823) 2019-03-07 14:12:27 +00:00
Richard van der Hoff 32471d63b7 Update example_log_config.yaml (#4820) 2019-03-07 14:12:10 +00:00
Richard van der Hoff 685704536f Fix check-newsfragment for debian-only changes. (#4825) 2019-03-07 14:03:05 +00:00
Amber Brown f6135d06cf Rewrite userdir to be faster (#4537) 2019-03-07 01:22:53 -08:00
Richard van der Hoff 5580616235 Document the generate option for the docker image. 2019-03-07 07:35:42 +00:00
Richard van der Hoff 6e4931aa19 Debian package: fix warning during preconfiguration. 2019-03-07 07:18:06 +00:00
Matthew Hodgson 8f4b9f5210 Reword the sample config header to be less scary (#4801) 2019-03-07 07:09:01 +00:00
Travis Ralston 06cbf79fe3 Merge pull request #4779 from matrix-org/travis/icons
Use static locations for Riot icons
2019-03-06 15:11:41 -07:00
Richard van der Hoff 898378c9b5 Update changelog.d/4779.misc
Co-Authored-By: turt2live <travpc@gmail.com>
2019-03-06 14:29:56 -07:00
Erik Johnston 1f85c2c0ce Merge pull request #4818 from matrix-org/erikj/prefill_client_ips
Prefill client IPs cache on workers
2019-03-06 19:17:39 +00:00
Erik Johnston 366877c579 Update changelog 2019-03-06 19:04:52 +00:00
Erik Johnston 0a6e716600 Merge pull request #4815 from matrix-org/erikj/docstrings
Add docstrings from matrix-org-hotfixes
2019-03-06 18:59:28 +00:00
Erik Johnston c665b637de Merge pull request #4816 from matrix-org/erikj/4422_debug
Port #4422 debug logging from hotfixes
2019-03-06 18:59:12 +00:00
Erik Johnston ff7bd29ea9 Merge pull request #4817 from matrix-org/erikj/shutdown_room_message
Send message after room has been shutdown
2019-03-06 18:59:05 +00:00
Erik Johnston 7791c5194e Newsfile 2019-03-06 17:40:51 +00:00
Erik Johnston face0c5b3c Prefill client IPs cache on workers 2019-03-06 17:39:32 +00:00
Erik Johnston 03dce32019 Newsfile 2019-03-06 17:38:19 +00:00
Erik Johnston b879870b2d Send message after room has been shutdown
Currently the explanation message is sent to the abuse room before any
users are forced joined, which means it tends to get lost in the backlog
of joins.

So instead we send the message *after* we've forced joined everyone.
2019-03-06 17:35:11 +00:00
Erik Johnston 4238f63545 Newsfile 2019-03-06 17:32:48 +00:00
Erik Johnston 8b7790e68f Port #4422 debug logging from hotfixes 2019-03-06 17:29:15 +00:00
Erik Johnston 9c50074c21 Newsfile 2019-03-06 17:24:53 +00:00
Erik Johnston 6d13bdec91 Add docstrings from matrix-org-hotfixes 2019-03-06 17:24:26 +00:00
Erik Johnston 4c473ba088 Newsfile 2019-03-06 16:24:03 +00:00
Erik Johnston a9de04be72 Implement soft fail 2019-03-06 16:22:16 +00:00
Andrew Morgan 7b8a157b79 Merge pull request #4792 from matrix-org/anoa/replication_tokens
Support batch updates in the worker sender
2019-03-06 15:48:29 +00:00
Brendan Abolivier 20dd3403ee Merge pull request #4804 from matrix-org/babolivier/ratelimit_registration_improvements
Improve ratelimit on registration
2019-03-06 11:40:10 +00:00
Brendan Abolivier 6f3cde8b25 Make registration ratelimiter separate from the main events one 2019-03-06 11:02:42 +00:00
Brendan Abolivier f4195f4118 Revert "Split ratelimiters in two (one for events, one for registration)"
This reverts commit d7dbad3526.
2019-03-06 10:55:22 +00:00
Brendan Abolivier 6fcecb4859 Add changelog 2019-03-05 18:55:29 +00:00
Brendan Abolivier d7dbad3526 Split ratelimiters in two (one for events, one for registration) 2019-03-05 18:41:27 +00:00
Brendan Abolivier c23e8c3333 Update sample config 2019-03-05 18:03:48 +00:00
Brendan Abolivier 067ce795c0 Move settings from registration to ratelimiting in config file 2019-03-05 18:03:14 +00:00
Erik Johnston 16c8b4ecbd Merge pull request #4772 from jbweston/jbweston/server-version-api
Add 'server_version' endpoint to admin API
2019-03-05 16:31:00 +00:00
Brendan Abolivier a4c3a361b7 Add rate-limiting on registration (#4735)
* Rate-limiting for registration

* Add unit test for registration rate limiting

* Add config parameters for rate limiting on auth endpoints

* Doc

* Fix doc of rate limiting function

Co-Authored-By: babolivier <contact@brendanabolivier.com>

* Incorporate review

* Fix config parsing

* Fix linting errors

* Set default config for auth rate limiting

* Fix tests

* Add changelog

* Advance reactor instead of mocked clock

* Move parameters to registration specific config and give them more sensible default values

* Remove unused config options

* Don't mock the rate limiter un MAU tests

* Rename _register_with_store into register_with_store

* Make CI happy

* Remove unused import

* Update sample config

* Fix ratelimiting test for py2

* Add non-guest test
2019-03-05 14:25:33 +00:00
Andrew Morgan b9f6163092 Simplify token replication logic 2019-03-05 13:58:30 +00:00
Andrew Morgan 3887e0cd80 Merge pull request #4795 from matrix-org/anoa/configinatoractoring
Remove reference to that no longer exists in README
2019-03-05 09:47:14 +00:00
Andrew Morgan ae90531036 Merge pull request #4794 from matrix-org/anoa/erroneous_dollahs
Remove unnecessary dollar signs
2019-03-05 09:46:59 +00:00
Erik Johnston b050a10871 Merge pull request #4699 from matrix-org/erikj/stop_fed_not_in_room
Stop backpaginating when events not visible
2019-03-05 09:32:33 +00:00
Erik Johnston 9e8bca5667 Merge pull request #4799 from matrix-org/rav/clean_up_replication_code
Clean ups in replication notifier
2019-03-05 09:19:48 +00:00
Erik Johnston aa06d26ae0 clarify comments 2019-03-05 09:16:35 +00:00
Erik Johnston c3c542bb4a Merge pull request #4796 from matrix-org/erikj/factor_out_e2e_keys
Allow /keys/{changes,query} API to run on worker
2019-03-05 09:06:25 +00:00
Richard van der Hoff 48583cef7e Merge pull request #4798 from matrix-org/rav/rr_debug
Add some debug about processing read receipts.
2019-03-04 19:04:05 +00:00
Richard van der Hoff cd7110c869 Merge pull request #4797 from matrix-org/rav/inline_rr_send
Clean up read-receipt handling.
2019-03-04 19:03:40 +00:00
Richard van der Hoff eaa9f43603 changelog 2019-03-04 19:01:19 +00:00
Richard van der Hoff c7325776a7 Remove redundant PreserveLoggingContext
Both (!) things that register as replication listeners do the right thing wrt
logcontexts, so this is redundant.
2019-03-04 18:31:18 +00:00
Erik Johnston 00b0e8b7df Newsfile 2019-03-04 18:30:07 +00:00
Erik Johnston bfa7d46a10 Allow /keys/{changes,query} API to run on worker 2019-03-04 18:30:01 +00:00
Erik Johnston 157e5a8f27 Split DeviceHandler into master and worker 2019-03-04 18:29:26 +00:00
Richard van der Hoff daa10e3e66 Remove unused wait_for_replication method
I guess this was used once? It's not now, anyway.
2019-03-04 18:27:32 +00:00
Richard van der Hoff 2db49ea476 Add some debug about processing read receipts.
I'm hoping to establish which rooms are having lots of RRs sent for them, and
how old the events are when they are sent.
2019-03-04 18:19:40 +00:00
Richard van der Hoff b29693a30b Clean up read-receipt handling.
Remove a call to run_as_background_process: there is no need to run this as a
background process, because build_and_send_edu does not block.

We may as well inline the whole of _push_remotes.
2019-03-04 18:16:43 +00:00
Erik Johnston a84b8d56c2 Fixup slave stores 2019-03-04 18:04:57 +00:00
Richard van der Hoff 8e28bc5eee Include a default configuration file in the 'docs' directory. (#4791) 2019-03-04 17:14:58 +00:00
Erik Johnston 0d2d046709 Fix missing null guard 2019-03-04 16:04:04 +00:00
Andrew Morgan be18073692 Add changelog 2019-03-04 16:01:17 +00:00
Andrew Morgan d3f270f06a Remove reference to that no longer exists in README 2019-03-04 15:59:43 +00:00
Andrew Morgan 3a438c24a6 Add changelog 2019-03-04 15:28:22 +00:00
Andrew Morgan 336de1d45b Remove unnecessary dollar signs
A dollar sign is already appended to the end of each PATH, so there's
no need to add one in the PATH declaration as well.
2019-03-04 15:25:12 +00:00
Andrew Morgan fe7bd23a85 Clean up logic and add comments 2019-03-04 15:08:15 +00:00
Erik Johnston d1523aed6b Only check history visibility when filtering
When filtering events to send to server we check more than just history
visibility. However when deciding whether to backfill or not we only
care about the history visibility.
2019-03-04 14:43:42 +00:00
Andrew Morgan 9f7cdf3da1 Clearer branching, fix missing list clear 2019-03-04 14:36:52 +00:00
Andrew Morgan 0bc50fb60a Add changelog 2019-03-04 14:05:16 +00:00
Andrew Morgan 5f0c449dd5 Prevent replication wedging 2019-03-04 14:03:18 +00:00
Seebi aba5eeabd5 Fix v4v6 option in HAProxy example config (#4790)
The v4v6 option only has a usage one ipv6 socket: https://serverfault.com/q/747895

Signed-off-by: Flakebi <flakebi@t-online.de>
2019-03-04 13:19:41 +00:00
Richard van der Hoff 856c83f5f8 Avoid rebuilding Edu objects in worker mode (#4770)
In worker mode, on the federation sender, when we receive an edu for sending
over the replication socket, it is parsed into an Edu object. There is no point
extracting the contents of it so that we can then immediately build another Edu.
2019-03-04 12:57:44 +00:00
Erik Johnston 8b63fe4c26 s/get_forward_events/get_successor_events/ 2019-03-04 11:56:03 +00:00
Erik Johnston fbc047f2a5 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/stop_fed_not_in_room 2019-03-04 11:54:58 +00:00
Richard van der Hoff 2c3548d9d8 Update test_typing to use HomeserverTestCase. (#4771) 2019-03-04 10:05:39 +00:00
Joseph Weston 144cbfd650 add API documentation
Signed-off-by: Joseph Weston <joseph@weston.cloud>
2019-03-02 03:07:04 +01:00
Travis Ralston 13c18853b9 Changelog 2019-03-01 15:15:00 -07:00
Travis Ralston 4dfbae18fe Use static locations for Riot icons
See https://github.com/vector-im/riot-web/issues/9009
2019-03-01 15:06:21 -07:00
Richard van der Hoff 3064952939 Fix incorrect log about not persisting duplicate state event. (#4776)
We were logging this when it was not true.
2019-03-01 16:47:12 +00:00
Richard van der Hoff 1beebe916f Merge branch 'master' into develop 2019-03-01 10:58:39 +00:00
Joseph Weston d3dcb64501 Add changelog and AUTHORS file entry
Signed-off-by: Joseph Weston <joseph@weston.cloud>
2019-03-01 10:44:40 +01:00
Joseph Weston 1e8388b311 Add 'server_version' endpoint to admin API
This is required because the 'Server' HTTP header is not always
passed through proxies.
2019-03-01 09:56:58 +01:00
Joseph Weston b136ee10df Import 'admin' module rather than 'register_servlets' directly
We will later need also to import 'register_servlets' from the
'login' module, so we un-pollute the namespace now to keep the
logical changes separate.
2019-03-01 09:46:25 +01:00
Andrew Morgan ac61b45a75 Minor docstring fixes for MatrixFederationAgent (#4765) 2019-02-28 16:24:01 +00:00
Amber Brown b131cc77df Make 'event_id' a required parameter in federated state requests (#4741)
* make 'event_id' a required parameter in federated state requests

As per the spec: https://matrix.org/docs/spec/server_server/r0.1.1.html#id40

Signed-off-by: Joseph Weston <joseph@weston.cloud>

* add changelog entry for bugfix

Signed-off-by: Joseph Weston <joseph@weston.cloud>

* Update server.py
2019-02-27 14:35:47 -08:00
Richard van der Hoff 68f47d6744 Fix parsing of Content-Disposition headers (#4763)
* Fix parsing of Content-Disposition headers

TIL: filenames in content-dispostion headers can contain semicolons, and aren't
%-encoded.

* fix python2 incompatibility

* Fix docstrings
2019-02-27 14:29:10 -08:00
Amber Brown f2a753ea38 Move from TravisCI to BuildKite (#4752) 2019-02-27 13:03:14 -08:00
Erik Johnston 76550c58d2 Merge pull request #4759 from matrix-org/erikj/3pid_client_reader
Move /account/3pid to client_reader
2019-02-27 16:11:21 +00:00
Erik Johnston 8267034a63 Merge pull request #4758 from matrix-org/erikj/use_presence_replication
When presence is disabled don't send over replication
2019-02-27 15:46:26 +00:00
Richard van der Hoff 3134964054 Update changelog.d/4759.feature
Co-Authored-By: erikjohnston <erikj@jki.re>
2019-02-27 15:32:21 +00:00
Erik Johnston 46b0151524 Merge pull request #4757 from matrix-org/erikj/key_api_fed_readae
Move server key queries to federation reader
2019-02-27 15:30:40 +00:00
Erik Johnston 95840d84d4 Newsfile 2019-02-27 14:28:52 +00:00
Erik Johnston 54f9ce11a7 Move /account/3pid to client_reader 2019-02-27 14:26:08 +00:00
Erik Johnston d4dc527a1a Fix unit tests 2019-02-27 14:24:45 +00:00
Erik Johnston 1b2940b3bd Newsfile 2019-02-27 13:54:45 +00:00
Erik Johnston 1e315017d3 When presence is enabled don't send over replication 2019-02-27 13:53:46 +00:00
Erik Johnston b5c13df0c4 Newsfile 2019-02-27 13:46:29 +00:00
Erik Johnston 4cff9376f7 Move server key queries to federation reader 2019-02-27 13:43:53 +00:00
Erik Johnston 71ef5fc411 Update newsfile to have a full stop 2019-02-27 13:22:23 +00:00
Erik Johnston b183fef9ac Update comments 2019-02-27 13:06:10 +00:00
Erik Johnston 7590e9fa28 Merge pull request #4749 from matrix-org/erikj/replication_connection_backoff
Fix tightloop over connecting to replication server
2019-02-27 11:00:59 +00:00
Erik Johnston 6870fc496f Move connecting logic into ClientReplicationStreamProtocol 2019-02-27 10:23:51 +00:00
Erik Johnston 09fc34c935 Newsfile 2019-02-26 15:13:55 +00:00
Erik Johnston 25814921f1 Increase the max delay between retry attempts
Otherwise if you have many workers they can easily take out master with
their connection attempts
2019-02-26 15:12:33 +00:00
Erik Johnston 313987187e Fix tightloop over connecting to replication server
If the client failed to process incoming commands during the initial set
up of the replication connection it would immediately disconnect and
reconnect, resulting in a tightloop.

This can happen, for example, when subscribing to a stream that has a
row that is too long in the backlog.

The fix here is to not consider the connection successfully set up until
the client has succesfully subscribed and caught up with the streams.
This ensures that the retry logic timers aren't reset until then,
meaning that if an error does happen during start up the client will
continue backing off before retrying again.
2019-02-26 15:05:41 +00:00
Erik Johnston 56f4ece778 Newsfile 2019-02-20 18:15:13 +00:00
Erik Johnston 71b625d808 Stop backpaginating when events not visible 2019-02-20 18:14:12 +00:00
264 changed files with 8730 additions and 3961 deletions
+13
View File
@@ -0,0 +1,13 @@
CI
BUILDKITE
BUILDKITE_BUILD_NUMBER
BUILDKITE_BRANCH
BUILDKITE_BUILD_NUMBER
BUILDKITE_JOB_ID
BUILDKITE_BUILD_URL
BUILDKITE_PROJECT_SLUG
BUILDKITE_COMMIT
BUILDKITE_PULL_REQUEST
BUILDKITE_TAG
CODECOV_TOKEN
TRIAL_FLAGS
+21
View File
@@ -0,0 +1,21 @@
version: '3.1'
services:
postgres:
image: postgres:9.4
environment:
POSTGRES_PASSWORD: postgres
testenv:
image: python:2.7
depends_on:
- postgres
env_file: .env
environment:
SYNAPSE_POSTGRES_HOST: postgres
SYNAPSE_POSTGRES_USER: postgres
SYNAPSE_POSTGRES_PASSWORD: postgres
working_dir: /app
volumes:
- ..:/app
+21
View File
@@ -0,0 +1,21 @@
version: '3.1'
services:
postgres:
image: postgres:9.5
environment:
POSTGRES_PASSWORD: postgres
testenv:
image: python:2.7
depends_on:
- postgres
env_file: .env
environment:
SYNAPSE_POSTGRES_HOST: postgres
SYNAPSE_POSTGRES_USER: postgres
SYNAPSE_POSTGRES_PASSWORD: postgres
working_dir: /app
volumes:
- ..:/app
+21
View File
@@ -0,0 +1,21 @@
version: '3.1'
services:
postgres:
image: postgres:9.4
environment:
POSTGRES_PASSWORD: postgres
testenv:
image: python:3.5
depends_on:
- postgres
env_file: .env
environment:
SYNAPSE_POSTGRES_HOST: postgres
SYNAPSE_POSTGRES_USER: postgres
SYNAPSE_POSTGRES_PASSWORD: postgres
working_dir: /app
volumes:
- ..:/app
+21
View File
@@ -0,0 +1,21 @@
version: '3.1'
services:
postgres:
image: postgres:9.5
environment:
POSTGRES_PASSWORD: postgres
testenv:
image: python:3.5
depends_on:
- postgres
env_file: .env
environment:
SYNAPSE_POSTGRES_HOST: postgres
SYNAPSE_POSTGRES_USER: postgres
SYNAPSE_POSTGRES_PASSWORD: postgres
working_dir: /app
volumes:
- ..:/app
+21
View File
@@ -0,0 +1,21 @@
version: '3.1'
services:
postgres:
image: postgres:11
environment:
POSTGRES_PASSWORD: postgres
testenv:
image: python:3.7
depends_on:
- postgres
env_file: .env
environment:
SYNAPSE_POSTGRES_HOST: postgres
SYNAPSE_POSTGRES_USER: postgres
SYNAPSE_POSTGRES_PASSWORD: postgres
working_dir: /app
volumes:
- ..:/app
+21
View File
@@ -0,0 +1,21 @@
version: '3.1'
services:
postgres:
image: postgres:9.5
environment:
POSTGRES_PASSWORD: postgres
testenv:
image: python:3.7
depends_on:
- postgres
env_file: .env
environment:
SYNAPSE_POSTGRES_HOST: postgres
SYNAPSE_POSTGRES_USER: postgres
SYNAPSE_POSTGRES_PASSWORD: postgres
working_dir: /app
volumes:
- ..:/app
+168
View File
@@ -0,0 +1,168 @@
env:
CODECOV_TOKEN: "2dd7eb9b-0eda-45fe-a47c-9b5ac040045f"
steps:
- command:
- "python -m pip install tox"
- "tox -e pep8"
label: "\U0001F9F9 PEP-8"
plugins:
- docker#v3.0.1:
image: "python:3.6"
- command:
- "python -m pip install tox"
- "tox -e packaging"
label: "\U0001F9F9 packaging"
plugins:
- docker#v3.0.1:
image: "python:3.6"
- command:
- "python -m pip install tox"
- "tox -e check_isort"
label: "\U0001F9F9 isort"
plugins:
- docker#v3.0.1:
image: "python:3.6"
- command:
- "python -m pip install tox"
- "scripts-dev/check-newsfragment"
label: ":newspaper: Newsfile"
branches: "!master !develop !release-*"
plugins:
- docker#v3.0.1:
image: "python:3.6"
propagate-environment: true
- wait
- command:
- "python -m pip install tox"
- "tox -e check-sampleconfig"
label: "\U0001F9F9 check-sample-config"
plugins:
- docker#v3.0.1:
image: "python:3.6"
- command:
- "python -m pip install tox"
- "tox -e py27,codecov"
label: ":python: 2.7 / SQLite"
env:
TRIAL_FLAGS: "-j 2"
plugins:
- docker#v3.0.1:
image: "python:2.7"
propagate-environment: true
- command:
- "python -m pip install tox"
- "tox -e py35,codecov"
label: ":python: 3.5 / SQLite"
env:
TRIAL_FLAGS: "-j 2"
plugins:
- docker#v3.0.1:
image: "python:3.5"
propagate-environment: true
- command:
- "python -m pip install tox"
- "tox -e py36,codecov"
label: ":python: 3.6 / SQLite"
env:
TRIAL_FLAGS: "-j 2"
plugins:
- docker#v3.0.1:
image: "python:3.6"
propagate-environment: true
- command:
- "python -m pip install tox"
- "tox -e py37,codecov"
label: ":python: 3.7 / SQLite"
env:
TRIAL_FLAGS: "-j 2"
plugins:
- docker#v3.0.1:
image: "python:3.7"
propagate-environment: true
- command:
- "python -m pip install tox"
- "tox -e py27-old,codecov"
label: ":python: 2.7 / SQLite / Old Deps"
env:
TRIAL_FLAGS: "-j 2"
plugins:
- docker#v3.0.1:
image: "python:2.7"
propagate-environment: true
- label: ":python: 2.7 / :postgres: 9.4"
env:
TRIAL_FLAGS: "-j 4"
command:
- "bash -c 'python -m pip install tox && python -m tox -e py27-postgres,codecov'"
plugins:
- docker-compose#v2.1.0:
run: testenv
config:
- .buildkite/docker-compose.py27.pg94.yaml
- label: ":python: 2.7 / :postgres: 9.5"
env:
TRIAL_FLAGS: "-j 4"
command:
- "bash -c 'python -m pip install tox && python -m tox -e py27-postgres,codecov'"
plugins:
- docker-compose#v2.1.0:
run: testenv
config:
- .buildkite/docker-compose.py27.pg95.yaml
- label: ":python: 3.5 / :postgres: 9.4"
env:
TRIAL_FLAGS: "-j 4"
command:
- "bash -c 'python -m pip install tox && python -m tox -e py35-postgres,codecov'"
plugins:
- docker-compose#v2.1.0:
run: testenv
config:
- .buildkite/docker-compose.py35.pg94.yaml
- label: ":python: 3.5 / :postgres: 9.5"
env:
TRIAL_FLAGS: "-j 4"
command:
- "bash -c 'python -m pip install tox && python -m tox -e py35-postgres,codecov'"
plugins:
- docker-compose#v2.1.0:
run: testenv
config:
- .buildkite/docker-compose.py35.pg95.yaml
- label: ":python: 3.7 / :postgres: 9.5"
env:
TRIAL_FLAGS: "-j 4"
command:
- "bash -c 'python -m pip install tox && python -m tox -e py37-postgres,codecov'"
plugins:
- docker-compose#v2.1.0:
run: testenv
config:
- .buildkite/docker-compose.py37.pg95.yaml
- label: ":python: 3.7 / :postgres: 11"
env:
TRIAL_FLAGS: "-j 4"
command:
- "bash -c 'python -m pip install tox && python -m tox -e py37-postgres,codecov'"
plugins:
- docker-compose#v2.1.0:
run: testenv
config:
- .buildkite/docker-compose.py37.pg11.yaml
+7 -3
View File
@@ -12,11 +12,15 @@ _trial_temp/
_trial_temp*/
# stuff that is likely to exist when you run a server locally
/*.db
/*.log
/*.log.config
/*.pid
/*.signing.key
/*.tls.crt
/*.tls.key
/uploads
/env/
/homeserver*.yaml
/media_store/
/uploads
# IDEs
/.idea/
-97
View File
@@ -1,97 +0,0 @@
dist: xenial
language: python
cache:
directories:
# we only bother to cache the wheels; parts of the http cache get
# invalidated every build (because they get served with a max-age of 600
# seconds), which means that we end up re-uploading the whole cache for
# every build, which is time-consuming In any case, it's not obvious that
# downloading the cache from S3 would be much faster than downloading the
# originals from pypi.
#
- $HOME/.cache/pip/wheels
# don't clone the whole repo history, one commit will do
git:
depth: 1
# only build branches we care about (PRs are built seperately)
branches:
only:
- master
- develop
- /^release-v/
- rav/pg95
# When running the tox environments that call Twisted Trial, we can pass the -j
# flag to run the tests concurrently. We set this to 2 for CPU bound tests
# (SQLite) and 4 for I/O bound tests (PostgreSQL).
matrix:
fast_finish: true
include:
- name: "pep8"
python: 3.6
env: TOX_ENV="pep8,check_isort,packaging"
- name: "py2.7 / sqlite"
python: 2.7
env: TOX_ENV=py27,codecov TRIAL_FLAGS="-j 2"
- name: "py2.7 / sqlite / olddeps"
python: 2.7
env: TOX_ENV=py27-old TRIAL_FLAGS="-j 2"
- name: "py2.7 / postgres9.5"
python: 2.7
addons:
postgresql: "9.5"
env: TOX_ENV=py27-postgres,codecov TRIAL_FLAGS="-j 4"
services:
- postgresql
- name: "py3.5 / sqlite"
python: 3.5
env: TOX_ENV=py35,codecov TRIAL_FLAGS="-j 2"
- name: "py3.7 / sqlite"
python: 3.7
env: TOX_ENV=py37,codecov TRIAL_FLAGS="-j 2"
- name: "py3.7 / postgres9.4"
python: 3.7
addons:
postgresql: "9.4"
env: TOX_ENV=py37-postgres TRIAL_FLAGS="-j 4"
services:
- postgresql
- name: "py3.7 / postgres9.5"
python: 3.7
addons:
postgresql: "9.5"
env: TOX_ENV=py37-postgres,codecov TRIAL_FLAGS="-j 4"
services:
- postgresql
- # we only need to check for the newsfragment if it's a PR build
if: type = pull_request
name: "check-newsfragment"
python: 3.6
script: scripts-dev/check-newsfragment
install:
# this just logs the postgres version we will be testing against (if any)
- psql -At -U postgres -c 'select version();' || true
- pip install tox
# if we don't have python3.6 in this environment, travis unhelpfully gives us
# a `python3.6` on our path which does nothing but spit out a warning. Tox
# tries to run it (even if we're not running a py36 env), so the build logs
# then have warnings which look like errors. To reduce the noise, remove the
# non-functional python3.6.
- ( ! command -v python3.6 || python3.6 --version ) &>/dev/null || rm -f $(command -v python3.6)
script:
- tox -e $TOX_ENV
+3
View File
@@ -69,3 +69,6 @@ Serban Constantin <serban.constantin at gmail dot com>
Jason Robinson <jasonr at matrix.org>
* Minor fixes
Joseph Weston <joseph at weston.cloud>
+ Add admin API for querying HS version
+8 -4
View File
@@ -375,9 +375,13 @@ To configure Synapse to expose an HTTPS port, you will need to edit
* You will also need to uncomment the `tls_certificate_path` and
`tls_private_key_path` lines under the `TLS` section. You can either
point these settings at an existing certificate and key, or you can
enable Synapse's built-in ACME (Let's Encrypt) support. Instructions
enable Synapse's built-in ACME (Let's Encrypt) support. Instructions
for having Synapse automatically provision and renew federation
certificates through ACME can be found at [ACME.md](docs/ACME.md).
certificates through ACME can be found at [ACME.md](docs/ACME.md). If you
are using your own certificate, be sure to use a `.pem` file that includes
the full certificate chain including any intermediate certificates (for
instance, if using certbot, use `fullchain.pem` as your certificate, not
`cert.pem`).
For those of you upgrading your TLS certificate in readiness for Synapse 1.0,
please take a look at `our guide <docs/MSC1711_certificates_FAQ.md#configuring-certificates-for-compatibility-with-synapse-100>`_.
@@ -406,8 +410,8 @@ This process uses a setting `registration_shared_secret` in
`homeserver.yaml`, which is shared between Synapse itself and the
`register_new_matrix_user` script. It doesn't matter what it is (a random
value is generated by `--generate-config`), but it should be kept secret, as
anyone with knowledge of it can register users on your server even if
`enable_registration` is `false`.
anyone with knowledge of it can register users, including admin accounts,
on your server even if `enable_registration` is `false`.
## Setting up a TURN server
+1
View File
@@ -39,6 +39,7 @@ prune .circleci
prune .coveragerc
prune debian
prune .codecov.yml
prune .buildkite
exclude jenkins*
recursive-exclude jenkins *.sh
+3 -3
View File
@@ -120,9 +120,9 @@ recommended to also set up CAPTCHA - see `<docs/CAPTCHA_SETUP.rst>`_.)
Once ``enable_registration`` is set to ``true``, it is possible to register a
user via `riot.im <https://riot.im/app/#/register>`_ or other Matrix clients.
Your new user name will be formed partly from the ``server_name`` (see
`Configuring synapse`_), and partly from a localpart you specify when you
create the account. Your name will take the form of::
Your new user name will be formed partly from the ``server_name``, and partly
from a localpart you specify when you create the account. Your name will take
the form of::
@localpart:my.domain.name
+1
View File
@@ -0,0 +1 @@
Fix a bug where media with spaces in the name would get a corrupted name.
+1
View File
@@ -0,0 +1 @@
The user directory has been rewritten to make it faster, with less chance of falling behind on a large server.
+1
View File
@@ -0,0 +1 @@
Add a systemd setup that supports synapse workers. Contributed by Luca Corbatto.
+1
View File
@@ -0,0 +1 @@
Fix attempting to paginate in rooms where server cannot see any events, to avoid unnecessarily pulling in lots of redacted events.
+1
View File
@@ -0,0 +1 @@
Add configurable rate limiting to the /register endpoint.
+1
View File
@@ -0,0 +1 @@
'event_id' is now a required parameter in federated state requests, as per the matrix spec.
+1
View File
@@ -0,0 +1 @@
Fix tightloop over connecting to replication server.
+1
View File
@@ -0,0 +1 @@
Change from TravisCI to Buildkite for CI.
+1
View File
@@ -0,0 +1 @@
Move server key queries to federation reader.
+1
View File
@@ -0,0 +1 @@
When presence is disabled don't send over replication.
+1
View File
@@ -0,0 +1 @@
Add support for /account/3pid REST endpoint to client_reader worker.
+1
View File
@@ -0,0 +1 @@
Fix parsing of Content-Disposition headers on remote media requests and URL previews.
+1
View File
@@ -0,0 +1 @@
Minor docstring fixes for MatrixFederationAgent.
+1
View File
@@ -0,0 +1 @@
Optimise EDU transmission for the federation_sender worker.
+1
View File
@@ -0,0 +1 @@
Update test_typing to use HomeserverTestCase.
+1
View File
@@ -0,0 +1 @@
Add an endpoint to the admin API for querying the server version. Contributed by Joseph Weston.
+1
View File
@@ -0,0 +1 @@
Fix incorrect log about not persisting duplicate state event.
+1
View File
@@ -0,0 +1 @@
Update URLs for riot.im icons and logos in the default notification templates.
+1
View File
@@ -0,0 +1 @@
Fix v4v6 option in HAProxy example config. Contributed by Flakebi.
+1
View File
@@ -0,0 +1 @@
Include a default configuration file in the 'docs' directory.
+1
View File
@@ -0,0 +1 @@
Handle batch updates in worker replication protocol.
+1
View File
@@ -0,0 +1 @@
Synapse is now permissive about trailing slashes on some of its federation endpoints, allowing zero or more to be present.
+1
View File
@@ -0,0 +1 @@
Removed unnecessary $ from some federation endpoint path regexes.
+1
View File
@@ -0,0 +1 @@
Remove link to deleted title in README.
+1
View File
@@ -0,0 +1 @@
Add support for /keys/query and /keys/changes REST endpoints to client_reader worker.
+1
View File
@@ -0,0 +1 @@
Clean up read-receipt handling.
+1
View File
@@ -0,0 +1 @@
Add some debug about processing read receipts.
+1
View File
@@ -0,0 +1 @@
Clean up some replication code.
+1
View File
@@ -0,0 +1 @@
Include a default configuration file in the 'docs' directory.
+1
View File
@@ -0,0 +1 @@
Add configurable rate limiting to the /register endpoint.
+1
View File
@@ -0,0 +1 @@
Add checks to incoming events over federation for events evading auth (aka "soft fail").
+1
View File
@@ -0,0 +1 @@
Add some docstrings.
+1
View File
@@ -0,0 +1 @@
Add debug logger to try and track down #4422.
+1
View File
@@ -0,0 +1 @@
Make shutdown API send explanation message to room after users have been forced joined.
+1
View File
@@ -0,0 +1 @@
Fix bug where we didn't correctly throttle sending of USER_IP commands over replication.
+1
View File
@@ -0,0 +1 @@
Update example_log_config.yaml.
+1
View File
@@ -0,0 +1 @@
Add configurable rate limiting to the /login endpoint.
+1
View File
@@ -0,0 +1 @@
Document the `generate` option for the docker image.
+1
View File
@@ -0,0 +1 @@
Fix check-newsfragment for debian-only changes.
+1
View File
@@ -0,0 +1 @@
Add some debug logging for device list updates to help with #4828.
+1
View File
@@ -0,0 +1 @@
Fix potential race in handling missing updates in device list updates.
+1
View File
@@ -0,0 +1 @@
Fix bug where synapse expected an un-specced `prev_state` field on state events.
+1
View File
@@ -0,0 +1 @@
Transfer a user's notification settings (push rules) on room upgrade.
+1
View File
@@ -0,0 +1 @@
Disable captcha registration by default in unit tests.
+1
View File
@@ -0,0 +1 @@
Remove trailing slashes from certain outbound federation requests. Retry if receiving a 404. Context: #3622.
+1
View File
@@ -0,0 +1 @@
Add stuff back to the .gitignore.
+1
View File
@@ -0,0 +1 @@
Clarify what registration_shared_secret allows for.
+1
View File
@@ -0,0 +1 @@
The user directory has been rewritten to make it faster, with less chance of falling behind on a large server.
+1
View File
@@ -0,0 +1 @@
Correctly log expected errors when fetching server keys.
+1
View File
@@ -0,0 +1 @@
Update install docs to explicitly state a full-chain (not just the top-level) TLS certificate must be provided to Synapse. This caused some people's Synapse ports to appear correct in a browser but still (rightfully so) upset the federation tester.
+1
View File
@@ -0,0 +1 @@
Move client read-receipt processing to federation sender worker.
+1
View File
@@ -0,0 +1 @@
Allow passing --daemonize flags to workers in the same way as with master.
+1
View File
@@ -0,0 +1 @@
Refactor federation TransactionQueue.
+1
View File
@@ -0,0 +1 @@
Comment out most options in the generated config.
+1
View File
@@ -0,0 +1 @@
The user directory has been rewritten to make it faster, with less chance of falling behind on a large server.
+1
View File
@@ -0,0 +1 @@
Add configurable rate limiting to the /login endpoint.
+1
View File
@@ -0,0 +1 @@
Fix yaml library warnings by using safe_load.
+1
View File
@@ -0,0 +1 @@
Update Apache setup to remove location syntax. Thanks to @cwmke!
+1
View File
@@ -0,0 +1 @@
Reinstate test case that runs unit tests against oldest supported dependencies.
+1
View File
@@ -0,0 +1 @@
Update link to federation docs.
+1
View File
@@ -0,0 +1 @@
fix test_auto_create_auto_join_where_no_consent.
+1
View File
@@ -0,0 +1 @@
fix test_auto_create_auto_join_where_no_consent.
+1
View File
@@ -0,0 +1 @@
The user directory has been rewritten to make it faster, with less chance of falling behind on a large server.
+2
View File
@@ -0,0 +1,2 @@
Fix a bug where hs_disabled_message was sometimes not correctly enforced.
+1
View File
@@ -0,0 +1 @@
Use a regular HomeServerConfig object for unit tests rater than a Mock.
+1
View File
@@ -0,0 +1 @@
Batch up outgoing read-receipts to reduce federation traffic.
+1
View File
@@ -0,0 +1 @@
Add option to disable searching the user directory.
+1
View File
@@ -0,0 +1 @@
Add some notes about tuning postgres for larger deployments.
+1
View File
@@ -0,0 +1 @@
Add option to disable searching of local and remote public room lists.
+1
View File
@@ -0,0 +1 @@
The user directory has been rewritten to make it faster, with less chance of falling behind on a large server.
+1
View File
@@ -0,0 +1 @@
Add a config option for torture-testing worker replication.
+1
View File
@@ -0,0 +1 @@
Fix bug in shutdown room admin API where it would fail if a user in the room hadn't consented to the privacy policy.
+1
View File
@@ -0,0 +1 @@
Log requests which are simulated by the unit tests.
+1
View File
@@ -0,0 +1 @@
Fix bug where blocked world-readable rooms were still peekable.
+1
View File
@@ -0,0 +1 @@
Allow newsfragments to end with exclamation marks. Exciting!
+1
View File
@@ -0,0 +1 @@
Refactor some more tests to use HomeserverTestCase.
+1
View File
@@ -0,0 +1 @@
Refactor out the state deltas portion of the user directory store and handler.
+1
View File
@@ -0,0 +1 @@
Fix nginx example in ACME doc.
+1
View File
@@ -0,0 +1 @@
Batch up outgoing read-receipts to reduce federation traffic.
+1
View File
@@ -0,0 +1 @@
Use an explicit dbname for postgres connections in the tests.
+1
View File
@@ -0,0 +1 @@
Fix `ClientReplicationStreamProtocol.__str__()`.
+1
View File
@@ -0,0 +1 @@
Add ability for password providers to login/register a user via 3PID (email, phone).
+1
View File
@@ -0,0 +1 @@
Fix bug where presence updates were sent to all servers in a room when a new server joined, rather than to just the new server.
+1
View File
@@ -0,0 +1 @@
The user directory has been rewritten to make it faster, with less chance of falling behind on a large server.
+1
View File
@@ -0,0 +1 @@
Add ability for password provider modules to bind email addresses to users upon registration.
+1
View File
@@ -0,0 +1 @@
Fix/improve some docstrings in the replication code.

Some files were not shown because too many files have changed in this diff Show More