### Changes proposed in this PR
- Add support for the `no_proxy` and `NO_PROXY` environment variables
- Internally rely on urllib's [`proxy_bypass_environment`](bdb941be42/Lib/urllib/request.py (L2519))
- Extract env variables using urllib's `getproxies`/[`getproxies_environment`](bdb941be42/Lib/urllib/request.py (L2488)) which supports lowercase + uppercase, preferring lowercase, except for `HTTP_PROXY` in a CGI environment
This does contain behaviour changes for consumers so making sure these are called out:
- `no_proxy`/`NO_PROXY` is now respected
- lowercase `https_proxy` is now allowed and taken over `HTTPS_PROXY`
Related to #9306 which also uses `ProxyAgent`
Signed-off-by: Timothy Leung tim95@hotmail.co.uk
This reduces the memory usage of previewing media files which
end up larger than the `max_spider_size` by avoiding buffering
content internally in treq.
It also checks the `Content-Length` header in additional places
instead of streaming the content to check the body length.
- Update black version to the latest
- Run black auto formatting over the codebase
- Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
- Update `code_style.md` docs around installing black to use the correct version
The two are equivalent, but really we want to check the HTTP result that got
returned to the channel, not the code that the Request object *intended* to
return to the channel.
Replaces the `federation_ip_range_blacklist` configuration setting with an
`ip_range_blacklist` setting with wider scope. It now applies to:
* Federation
* Identity servers
* Push notifications
* Checking key validitity for third-party invite events
The old `federation_ip_range_blacklist` setting is still honored if present, but
with reduced scope (it only applies to federation and identity servers).
Where we want to render a request against a specific Resource, call the global
make_request() function rather than the one in HomeserverTestCase, allowing us
to pass in an appropriate `Site`.
* commit 'f43c66d23':
Add support for running Complement against the local checkout (#8317)
Filter out appservices from mau count (#8404)
Only assert valid next_link params when provided (#8417)
Add metrics to track success/otherwise of replication requests (#8406)
Fix handling of connection timeouts in outgoing http requests (#8400)
Changelog
Don't check whether a 3pid is allowed to register during password reset
Add checks for postgres sequence consistency (#8402)
Create a mechanism for marking tests "logcontext clean" (#8399)
Add `ui_auth_sessions_ips` table to `synapse_port_db` ignore list (#8410)
A pair of tiny cleanups in the federation request code. (#8401)
typo
* commit '837293c31':
Remove obsolete __future__ imports (#8337)
Use admin_patterns for all admin APIs. (#8331)
Fix a potential bug of UnboundLocalError (#8329)
Switch metaclass initialization to python 3-compatible syntax (#8326)
Catch-up after Federation Outage (split, 4): catch-up loop (#8272)
Use slots in attrs classes where possible (#8296)
Fix typos in comments.
Add the topic and avatar to the room details admin API (#8305)
Improve SAML error messages (#8248)
Add experimental support for sharding event persister. Again. (#8294)
Make `StreamToken.room_key` be a `RoomStreamToken` instance. (#8281)
Use TLSv1.2 for fake servers in tests (#8208)
Add /_synapse/client to the reverse proxy docs (#8227)
Clean up `Notifier.on_new_room_event` code path (#8288)
* commit '2832ef5bb':
1.20.0rc3
Fix the exception that is raised when invalid JSON is encountered. (#8291)
Remove shared rooms info from upgrade/workers doc as it's still experimental (#8290)
* commit '17fa4c7ca':
Catch up after Federation Outage (split, 2): Track last successful stream ordering after transmission (#8247)
Catch-up after Federation Outage (split, 1) (#8230)
Fix type signature in simple_select_one_onecol and friends (#8241)
Stop sub-classing object (#8249)
* commit '5bf8e5f55':
Convert the well known resolver to async (#8214)
Convert additional databases to async/await part 2 (#8200)
Make MultiWriterIDGenerator work for streams that use negative stream IDs (#8203)
Do not install setuptools 50.0. (#8212)
Move and rename `get_devices_with_keys_by_user` (#8204)
Rename `get_e2e_device_keys` to better reflect its purpose (#8205)
Add a comment about _LimitedHostnameResolver
* commit 'a466b6797':
Reduce run-times of tests by advancing the reactor less (#7757)
Update debian systemd service to use Type=notify (#8169)
Remove remaining is_guest argument uses from get_room_data calls (#8181)
Do not propagate typing notifications from shadow-banned users. (#8176)
Remove unused parameter from, and add safeguard in, get_room_data (#8174)
Add required Debian dependencies to allow docker builds on the arm platform (#8144)
Allow running mypy directly. (#8175)
Update the test federation client to handle streaming responses (#8130)
Do not propagate profile changes of shadow-banned users into rooms. (#8157)
Make SlavedIdTracker.advance have same interface as MultiWriterIDGenerator (#8171)
Convert simple_select_one and simple_select_one_onecol to async (#8162)
* Remove `on_timeout_cancel` from `timeout_deferred`
The `on_timeout_cancel` param to `timeout_deferred` wasn't always called on a
timeout (in particular if the canceller raised an exception), so it was
unreliable. It was also only used in one place, and to be honest it's easier to
do what it does a different way.
* Fix handling of connection timeouts in outgoing http requests
Turns out that if we get a timeout during connection, then a different
exception is raised, which wasn't always handled correctly.
To fix it, catch the exception in SimpleHttpClient and turn it into a
RequestTimedOutError (which is already a documented exception).
Also add a description to RequestTimedOutError so that we can see which stage
it failed at.
* Fix incorrect handling of timeouts reading federation responses
This was trapping the wrong sort of TimeoutError, so was never being hit.
The effect was relatively minor, but we should fix this so that it does the
expected thing.
* Fix inconsistent handling of `timeout` param between methods
`get_json`, `put_json` and `delete_json` were applying a different timeout to
the response body to `post_json`; bring them in line and test.
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Erik Johnston <erik@matrix.org>
Synapse 1.20.0rc3 (2020-09-11)
==============================
Bugfixes
--------
- Fix a bug introduced in v1.20.0rc1 where the wrong exception was raised when invalid JSON data is encountered. ([\#8291](https://github.com/matrix-org/synapse/issues/8291))
Some Linux distros have begun disabling TLSv1.0 and TLSv1.1 by default
for security reasons, for example in Fedora 33 onwards:
https://fedoraproject.org/wiki/Changes/StrongCryptoSettings2
Use TLSv1.2 for the fake TLS servers created in the test suite, to avoid
failures due to OpenSSL disallowing TLSv1.0:
<twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines',
'ssl_choose_client_version', 'unsupported protocol')]>
Signed-off-by: Dan Callaghan <djc@djc.id.au>
* commit '5cdca53aa':
Merge different Resource implementation classes (#7732)
Fix inconsistent handling of upper and lower cases of email addresses. (#7021)
Allow YAML config file to contain None (#7779)
Fix a typo.
Move 1.15.2 after 1.16.0rc2.
1.16.0rc2
Remove an extraneous space.
Add links to the fixes.
Fix tense in the release notes.
Hack to add push priority to push notifications (#7765)
Add early returns to `_check_for_soft_fail` (#7769)
Use symbolic names for replication stream names (#7768)
Type checking for `FederationHandler` (#7770)
Fix new metric where we used ms instead of seconds (#7771)
Fix incorrect error message when database CTYPE was set incorrectly. (#7760)
Pin link in CHANGES.md
Fixes to CHANGES.md
