Wait for response

Spawning from https://github.com/element-hq/synapse/pull/18804#discussion_r2272826978

CHANGES.md

@@ -1,3 +1,40 @@
+# Synapse 1.137.0rc1 (2025-08-19)
+
+### Bugfixes
+
+- Fix a bug which could corrupt auth chains making it impossible to perform state resolution. ([\#18746](https://github.com/element-hq/synapse/issues/18746))
+- Fix error message in `register_new_matrix_user` utility script for empty `registration_shared_secret`. ([\#18780](https://github.com/element-hq/synapse/issues/18780))
+- Allow enabling [MSC4108](https://github.com/matrix-org/matrix-spec-proposals/pull/4108) when the stable Matrix Authentication Service integration is enabled. ([\#18832](https://github.com/element-hq/synapse/issues/18832))
+
+### Improved Documentation
+
+- Include IPv6 networks in `denied-peer-ips` of coturn setup. Contributed by @litetex. ([\#18781](https://github.com/element-hq/synapse/issues/18781))
+
+### Internal Changes
+
+- Update tests to ensure all database tables are emptied when purging a room. ([\#18794](https://github.com/element-hq/synapse/issues/18794))
+- Instrument the `encode_response` part of Sliding Sync requests for more complete traces in Jaeger. ([\#18815](https://github.com/element-hq/synapse/issues/18815))
+- Tag Sliding Sync traces when we `wait_for_events`. ([\#18816](https://github.com/element-hq/synapse/issues/18816))
+- Fix `portdb` CI by hardcoding the new `pg_dump` restrict key that was added due to [CVE-2025-8714](https://nvd.nist.gov/vuln/detail/cve-2025-8714). ([\#18824](https://github.com/element-hq/synapse/issues/18824))
+
+
+
+### Updates to locked dependencies
+
+* Bump actions/add-to-project from 5b1a254a3546aef88e0a7724a77a623fa2e47c36 to 0c37450c4be3b6a7582b2fb013c9ebfd9c8e9300. ([\#18557](https://github.com/element-hq/synapse/issues/18557))
+* Bump actions/cache from 4.2.3 to 4.2.4. ([\#18799](https://github.com/element-hq/synapse/issues/18799))
+* Bump actions/checkout from 4.2.2 to 4.3.0. ([\#18800](https://github.com/element-hq/synapse/issues/18800))
+* Bump actions/download-artifact from 4.3.0 to 5.0.0. ([\#18801](https://github.com/element-hq/synapse/issues/18801))
+* Bump docker/metadata-action from 5.7.0 to 5.8.0. ([\#18773](https://github.com/element-hq/synapse/issues/18773))
+* Bump mypy from 1.16.1 to 1.17.1. ([\#18775](https://github.com/element-hq/synapse/issues/18775))
+* Bump phonenumbers from 9.0.10 to 9.0.11. ([\#18797](https://github.com/element-hq/synapse/issues/18797))
+* Bump pygithub from 2.6.1 to 2.7.0. ([\#18779](https://github.com/element-hq/synapse/issues/18779))
+* Bump serde_json from 1.0.141 to 1.0.142. ([\#18776](https://github.com/element-hq/synapse/issues/18776))
+* Bump slab from 0.4.10 to 0.4.11. ([\#18809](https://github.com/element-hq/synapse/issues/18809))
+* Bump tokio from 1.47.0 to 1.47.1. ([\#18774](https://github.com/element-hq/synapse/issues/18774))
+* Bump types-pyyaml from 6.0.12.20250516 to 6.0.12.20250809. ([\#18798](https://github.com/element-hq/synapse/issues/18798))
+* Bump types-setuptools from 80.9.0.20250529 to 80.9.0.20250809. ([\#18796](https://github.com/element-hq/synapse/issues/18796))
+
 # Synapse 1.136.0 (2025-08-12)
 
 Note: This release includes the security fixes from `1.135.2` and `1.136.0rc2`, detailed below.

changelog.d/18746.bugfix

@@ -1 +0,0 @@
-Fix a bug which could corrupt auth chains making it impossible to perform state resolution.

changelog.d/18780.bugfix

@@ -1 +0,0 @@
-Fix error message in `register_new_matrix_user` utility script for empty `registration_shared_secret`.

changelog.d/18781.doc

@@ -1 +0,0 @@
-Include IPv6 networks in denied-peer-ips of coturn setup. Contributed by @litetex.

changelog.d/18794.misc

@@ -1 +0,0 @@
-Update tests to ensure all database tables are emptied when purging a room.

changelog.d/18815.misc

@@ -1 +0,0 @@
-Instrument the `encode_response` part of Sliding Sync requests for more complete traces in Jaeger.

changelog.d/18816.misc

@@ -1 +0,0 @@
-Tag Sliding Sync traces when we `wait_for_events`.

changelog.d/18824.misc

@@ -1 +0,0 @@
-Fix portdb CI by hardcoding the new pg_dump restrict key that was added due to CVE-2025-8714.

changelog.d/18832.bugfix

@@ -1 +0,0 @@
-Allow enabling MSC4108 when the stable Matrix Authentication Service integration is enabled.

debian/changelog

@@ -1,3 +1,9 @@
+matrix-synapse-py3 (1.137.0~rc1) stable; urgency=medium
+
+  * New Synapse release 1.137.0rc1.
+
+ -- Synapse Packaging team <packages@matrix.org>  Tue, 19 Aug 2025 10:55:22 +0100
+
 matrix-synapse-py3 (1.136.0) stable; urgency=medium
 
   * New Synapse release 1.136.0.

pyproject.toml

@@ -101,7 +101,7 @@ module-name = "synapse.synapse_rust"
 
 [tool.poetry]
 name = "matrix-synapse"
-version = "1.136.0"
+version = "1.137.0rc1"
 description = "Homeserver for the Matrix decentralised comms protocol"
 authors = ["Matrix.org Team and Contributors <packages@matrix.org>"]
 license = "AGPL-3.0-or-later"

schema/synapse-config.schema.yaml

@@ -1,5 +1,5 @@
 $schema: https://element-hq.github.io/synapse/latest/schema/v1/meta.schema.json
-$id: https://element-hq.github.io/synapse/schema/synapse/v1.136/synapse-config.schema.json
+$id: https://element-hq.github.io/synapse/schema/synapse/v1.137/synapse-config.schema.json
 type: object
 properties:
   modules:

synapse/http/proxy.py

@@ -173,7 +173,7 @@ class ProxyResource(_AsyncResource):
 
         return response.code, response
 
-    def _send_response(
+    async def _send_response(
         self,
         request: "SynapseRequest",
         code: int,
@@ -205,7 +205,7 @@ class ProxyResource(_AsyncResource):
 
         response.deliverBody(_ProxyResponseBody(request))
 
-    def _send_error_response(
+    async def _send_error_response(
         self,
         f: failure.Failure,
         request: "SynapseRequest",

synapse/http/server.py

@@ -75,7 +75,7 @@ from synapse.api.errors import (
     UnrecognizedRequestError,
 )
 from synapse.config.homeserver import HomeServerConfig
-from synapse.logging.context import defer_to_thread, preserve_fn, run_in_background
+from synapse.logging.context import defer_to_thread, preserve_fn
 from synapse.logging.opentracing import active_span, start_active_span, trace_servlet
 from synapse.util import Clock, json_encoder
 from synapse.util.caches import intern_dict
@@ -111,7 +111,7 @@ HTML_ERROR_TEMPLATE = """<!DOCTYPE html>
 HTTP_STATUS_REQUEST_CANCELLED = 499
 
 
-def return_json_error(
+async def return_json_error(
     f: failure.Failure, request: "SynapseRequest", config: Optional[HomeServerConfig]
 ) -> None:
     """Sends a JSON error response to clients."""
@@ -163,7 +163,7 @@ def return_json_error(
                 # abortConnection throws if the connection is already closed
                 pass
     else:
-        respond_with_json(
+        await respond_with_json(
             request,
             error_code,
             error_dict,
@@ -342,13 +342,13 @@ class _AsyncResource(resource.Resource, metaclass=abc.ABCMeta):
 
                 if callback_return is not None:
                     code, response = callback_return
-                    self._send_response(request, code, response)
+                    await self._send_response(request, code, response)
         except Exception:
             # failure.Failure() fishes the original Failure out
             # of our stack, and thus gives us a sensible stack
             # trace.
             f = failure.Failure()
-            self._send_error_response(f, request)
+            await self._send_error_response(f, request)
 
     async def _async_render(
         self, request: "SynapseRequest"
@@ -380,7 +380,7 @@ class _AsyncResource(resource.Resource, metaclass=abc.ABCMeta):
         raise UnrecognizedRequestError(code=405)
 
     @abc.abstractmethod
-    def _send_response(
+    async def _send_response(
         self,
         request: "SynapseRequest",
         code: int,
@@ -389,7 +389,7 @@ class _AsyncResource(resource.Resource, metaclass=abc.ABCMeta):
         raise NotImplementedError()
 
     @abc.abstractmethod
-    def _send_error_response(
+    async def _send_error_response(
         self,
         f: failure.Failure,
         request: "SynapseRequest",
@@ -415,7 +415,7 @@ class DirectServeJsonResource(_AsyncResource):
         super().__init__(clock, extract_context)
         self.canonical_json = canonical_json
 
-    def _send_response(
+    async def _send_response(
         self,
         request: "SynapseRequest",
         code: int,
@@ -423,7 +423,7 @@ class DirectServeJsonResource(_AsyncResource):
     ) -> None:
         """Implements _AsyncResource._send_response"""
         # TODO: Only enable CORS for the requests that need it.
-        respond_with_json(
+        await respond_with_json(
             request,
             code,
             response_object,
@@ -431,13 +431,13 @@ class DirectServeJsonResource(_AsyncResource):
             canonical_json=self.canonical_json,
         )
 
-    def _send_error_response(
+    async def _send_error_response(
         self,
         f: failure.Failure,
         request: "SynapseRequest",
     ) -> None:
         """Implements _AsyncResource._send_error_response"""
-        return_json_error(f, request, None)
+        await return_json_error(f, request, None)
 
 
 @attr.s(slots=True, frozen=True, auto_attribs=True)
@@ -565,13 +565,13 @@ class JsonResource(DirectServeJsonResource):
 
         return callback_return
 
-    def _send_error_response(
+    async def _send_error_response(
         self,
         f: failure.Failure,
         request: "SynapseRequest",
     ) -> None:
         """Implements _AsyncResource._send_error_response"""
-        return_json_error(f, request, self.hs.config)
+        await return_json_error(f, request, self.hs.config)
 
 
 class DirectServeHtmlResource(_AsyncResource):
@@ -593,7 +593,7 @@ class DirectServeHtmlResource(_AsyncResource):
 
         super().__init__(clock, extract_context)
 
-    def _send_response(
+    async def _send_response(
         self,
         request: "SynapseRequest",
         code: int,
@@ -606,7 +606,7 @@ class DirectServeHtmlResource(_AsyncResource):
 
         respond_with_html_bytes(request, code, html_bytes)
 
-    def _send_error_response(
+    async def _send_error_response(
         self,
         f: failure.Failure,
         request: "SynapseRequest",
@@ -780,7 +780,7 @@ def _encode_json_bytes(json_object: object) -> bytes:
     return json_encoder.encode(json_object).encode("utf-8")
 
 
-def respond_with_json(
+async def respond_with_json(
     request: "SynapseRequest",
     code: int,
     json_object: Any,
@@ -824,9 +824,7 @@ def respond_with_json(
     if send_cors:
         set_cors_headers(request)
 
-    run_in_background(
-        _async_write_json_to_request_in_thread, request, encoder, json_object
-    )
+    await _async_write_json_to_request_in_thread(request, encoder, json_object)
     return NOT_DONE_YET
 
 
@@ -882,6 +880,8 @@ async def _async_write_json_to_request_in_thread(
     Note: We don't use JsonEncoder.iterencode here as that falls back to the
     Python implementation (rather than the C backend), which is *much* more
     expensive.
+
+    The actual writing of bytes is not finished when this returns.
     """
 
     def encode(opentracing_span: "Optional[opentracing.Span]") -> bytes:

synapse/media/_base.py

@@ -122,9 +122,9 @@ MAXIMUM_ALLOWED_MAX_TIMEOUT_MS = 60_000
 _IMMUTABLE_ETAG = "1"
 
 
-def respond_404(request: SynapseRequest) -> None:
+async def respond_404(request: SynapseRequest) -> None:
     assert request.path is not None
-    respond_with_json(
+    await respond_with_json(
         request,
         404,
         cs_error("Not found '%s'" % (request.path.decode(),), code=Codes.NOT_FOUND),
@@ -154,7 +154,7 @@ async def respond_with_file(
 
         finish_request(request)
     else:
-        respond_404(request)
+        await respond_404(request)
 
 
 def add_file_headers(

synapse/media/media_repository.py

@@ -396,8 +396,8 @@ class MediaRepository:
 
         return MXCUri(self.server_name, media_id)
 
-    def respond_not_yet_uploaded(self, request: SynapseRequest) -> None:
-        respond_with_json(
+    async def respond_not_yet_uploaded(self, request: SynapseRequest) -> None:
+        await respond_with_json(
             request,
             504,
             cs_error("Media has not been uploaded yet", code=Codes.NOT_YET_UPLOADED),
@@ -455,7 +455,7 @@ class MediaRepository:
             await self.clock.sleep(0.5)
 
         logger.info("Media %s has not yet been uploaded", media_id)
-        self.respond_not_yet_uploaded(request)
+        await self.respond_not_yet_uploaded(request)
         return None
 
     async def get_local_media(

synapse/media/thumbnailer.py

@@ -699,7 +699,7 @@ class ThumbnailProvider:
             logger.info("Failed to find any generated thumbnails")
 
             assert request.path is not None
-            respond_with_json(
+            await respond_with_json(
                 request,
                 400,
                 cs_error(

synapse/rest/client/media.py

@@ -112,7 +112,7 @@ class MediaConfigResource(RestServlet):
             )
         )
         response = user_specific_config if user_specific_config else self.limits_dict
-        respond_with_json(request, 200, response, send_cors=True)
+        await respond_with_json(request, 200, response, send_cors=True)
 
 
 class ThumbnailResource(RestServlet):

synapse/rest/media/config_resource.py

@@ -50,4 +50,4 @@ class MediaConfigResource(RestServlet):
             )
         )
         response = user_specific_config if user_specific_config else self.limits_dict
-        respond_with_json(request, 200, response, send_cors=True)
+        await respond_with_json(request, 200, response, send_cors=True)

synapse/rest/media/create_resource.py

@@ -79,7 +79,7 @@ class CreateResource(RestServlet):
             content_uri,
             unused_expires_at,
         )
-        respond_with_json(
+        await respond_with_json(
             request,
             200,
             {

synapse/rest/media/upload_resource.py

@@ -130,7 +130,7 @@ class UploadServlet(BaseUploadServlet):
 
         logger.info("Uploaded content with URI '%s'", content_uri)
 
-        respond_with_json(
+        await respond_with_json(
             request, 200, {"content_uri": str(content_uri)}, send_cors=True
         )
 
@@ -184,4 +184,4 @@ class AsyncUploadServlet(BaseUploadServlet):
                 raise SynapseError(400, "Bad content")
 
             logger.info("Uploaded content for media ID %r", media_id)
-            respond_with_json(request, 200, {}, send_cors=True)
+            await respond_with_json(request, 200, {}, send_cors=True)

tests/http/test_additional_resource.py

@@ -36,7 +36,7 @@ class _AsyncTestCustomEndpoint:
 
     async def handle_request(self, request: Request) -> None:
         assert isinstance(request, SynapseRequest)
-        respond_with_json(request, 200, {"some_key": "some_value_async"})
+        await respond_with_json(request, 200, {"some_key": "some_value_async"})
 
 
 class _SyncTestCustomEndpoint:
@@ -45,7 +45,7 @@ class _SyncTestCustomEndpoint:
 
     async def handle_request(self, request: Request) -> None:
         assert isinstance(request, SynapseRequest)
-        respond_with_json(request, 200, {"some_key": "some_value_sync"})
+        await respond_with_json(request, 200, {"some_key": "some_value_sync"})
 
 
 class AdditionalResourceTests(HomeserverTestCase):