Respond with useful error codes when Content-Length header/s are invalid (#19212)
Related to https://github.com/element-hq/synapse/issues/17035, when Synapse receives a request that is larger than the maximum size allowed, it aborts the connection without ever sending back a HTTP response. I dug into our usage of twisted and how best to try and report such an error and this is what I came up with. It would be ideal to be able to report the status from within `handleContentChunk` but that is called too early on in the twisted http handling code, before things have been setup enough to be able to properly write a response. I tested this change out locally (both with C-S and S-S apis) and they do receive a 413 response now in addition to the connection being closed. Hopefully this will aid in being able to quickly detect when https://github.com/element-hq/synapse/issues/17035 is occurring as the current situation makes it very hard to narrow things down to that specific issue without making a lot of assumptions. This PR also responds with more meaningful error codes now in the case of: - multiple `Content-Length` headers - invalid `Content-Length` header value - request content size being larger than the `Content-Length` value ### Pull Request Checklist <!-- Please read https://element-hq.github.io/synapse/latest/development/contributing_guide.html before submitting your pull request --> * [X] Pull request is based on the develop branch * [X] Pull request includes a [changelog file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog). The entry should: - Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from `EventStore` to `EventWorkerStore`.". - Use markdown where necessary, mostly for `code blocks`. - End with either a period (.) or an exclamation mark (!). - Start with a capital letter. - Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry. * [X] [Code style](https://element-hq.github.io/synapse/latest/code_style.html) is correct (run the [linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters)) --------- Co-authored-by: Eric Eastwood <erice@element.io>
This commit is contained in:
@@ -81,6 +81,7 @@ from twisted.web.http_headers import Headers
|
||||
from twisted.web.resource import IResource
|
||||
from twisted.web.server import Request, Site
|
||||
|
||||
from synapse.api.constants import MAX_REQUEST_SIZE
|
||||
from synapse.config.database import DatabaseConnectionConfig
|
||||
from synapse.config.homeserver import HomeServerConfig
|
||||
from synapse.events.auto_accept_invites import InviteAutoAccepter
|
||||
@@ -241,7 +242,6 @@ class FakeChannel:
|
||||
|
||||
def loseConnection(self) -> None:
|
||||
self.unregisterProducer()
|
||||
self.transport.loseConnection()
|
||||
|
||||
# Type ignore: mypy doesn't like the fact that producer isn't an IProducer.
|
||||
def registerProducer(self, producer: IProducer, streaming: bool) -> None:
|
||||
@@ -428,18 +428,29 @@ def make_request(
|
||||
|
||||
channel = FakeChannel(site, reactor, ip=client_ip)
|
||||
|
||||
req = request(channel, site, our_server_name="test_server")
|
||||
req = request(
|
||||
channel,
|
||||
site,
|
||||
our_server_name="test_server",
|
||||
max_request_body_size=MAX_REQUEST_SIZE,
|
||||
)
|
||||
channel.request = req
|
||||
|
||||
req.content = BytesIO(content)
|
||||
# Twisted expects to be at the end of the content when parsing the request.
|
||||
req.content.seek(0, SEEK_END)
|
||||
|
||||
# Old version of Twisted (<20.3.0) have issues with parsing x-www-form-urlencoded
|
||||
# bodies if the Content-Length header is missing
|
||||
req.requestHeaders.addRawHeader(
|
||||
b"Content-Length", str(len(content)).encode("ascii")
|
||||
)
|
||||
# If `Content-Length` was passed in as a custom header, don't automatically add it
|
||||
# here.
|
||||
if custom_headers is None or not any(
|
||||
(k if isinstance(k, bytes) else k.encode("ascii")) == b"Content-Length"
|
||||
for k, _ in custom_headers
|
||||
):
|
||||
# Old version of Twisted (<20.3.0) have issues with parsing x-www-form-urlencoded
|
||||
# bodies if the Content-Length header is missing
|
||||
req.requestHeaders.addRawHeader(
|
||||
b"Content-Length", str(len(content)).encode("ascii")
|
||||
)
|
||||
|
||||
if access_token:
|
||||
req.requestHeaders.addRawHeader(
|
||||
|
||||
Reference in New Issue
Block a user