Fix changelog number

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

CHANGES.md

@@ -1,48 +1,11 @@
-Synapse 1.32.1 (2021-04-21)
-===========================
-
-This release fixes [a regression](https://github.com/matrix-org/synapse/issues/9853)
-in Synapse 1.32.0 that caused connected Prometheus instances to become unstable. If you
-ran Synapse 1.32.0 with Prometheus metrics, first upgrade to Synapse 1.32.1 and follow
-[these instructions](https://github.com/matrix-org/synapse/pull/9854#issuecomment-823472183)
-to clean up any excess writeahead logs.
-
-Bugfixes
---------
-
-- Fix a regression in Synapse 1.32.0 which caused Synapse to report large numbers of Prometheus time series, potentially overwhelming Prometheus instances. ([\#9854](https://github.com/matrix-org/synapse/issues/9854))
-
-
-Synapse 1.32.0 (2021-04-20)
-===========================
-
-**Note:** This release introduces [a regression](https://github.com/matrix-org/synapse/issues/9853)
-that can overwhelm connected Prometheus instances. This issue was not present in
-1.32.0rc1, and is fixed in 1.32.1. See the changelog for 1.32.1 above for more information.
+Synapse 1.32.0rc1 (2021-04-13)
+==============================
 
 **Note:** This release requires Python 3.6+ and Postgres 9.6+ or SQLite 3.22+.
 
 This release removes the deprecated `GET /_synapse/admin/v1/users/<user_id>` admin API. Please use the [v2 API](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/user_admin_api.rst#query-user-account) instead, which has improved capabilities.
 
-This release requires Application Services to use type `m.login.application_service` when registering users via the `/_matrix/client/r0/register` endpoint to comply with the spec. Please ensure your Application Services are up to date.
-
-If you are using the `packages.matrix.org` Debian repository for Synapse packages, 
-note that we have recently updated the expiry date on the gpg signing key. If you see an 
-error similar to `The following signatures were invalid: EXPKEYSIG F473DD4473365DE1`, you
-will need to get a fresh copy of the keys. You can do so with:
-
-```sh
-sudo wget -O /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg
-```
-
-Bugfixes
---------
-
-- Fix the log lines of nested logging contexts. Broke in 1.32.0rc1. ([\#9829](https://github.com/matrix-org/synapse/issues/9829))
-
-
-Synapse 1.32.0rc1 (2021-04-13)
-==============================
+This release requires Application Services to use type `m.login.application_services` when registering users via the `/_matrix/client/r0/register` endpoint to comply with the spec. Please ensure your Application Services are up to date.
 
 Features
 --------

UPGRADE.rst

@@ -88,26 +88,6 @@ for example:
 Upgrading to v1.32.0
 ====================
 
-Regression causing connected Prometheus instances to become overwhelmed
------------------------------------------------------------------------
-
-This release introduces `a regression <https://github.com/matrix-org/synapse/issues/9853>`_
-that can overwhelm connected Prometheus instances. This issue is not present in
-Synapse v1.32.0rc1, and is fixed in Synapse v1.32.1.
-
-If you have been affected, please first upgrade to a more recent Synapse version.
-You then may need to remove excess writeahead logs in order for Prometheus to recover.
-Instructions for doing so are provided
-`here <https://github.com/matrix-org/synapse/pull/9854#issuecomment-823472183>`_.
-
-Dropping support for old Python, Postgres and SQLite versions
--------------------------------------------------------------
-
-In line with our `deprecation policy <https://github.com/matrix-org/synapse/blob/release-v1.32.0/docs/deprecation_policy.md>`_,
-we've dropped support for Python 3.5 and PostgreSQL 9.5, as they are no longer supported upstream.
-
-This release of Synapse requires Python 3.6+ and PostgresSQL 9.6+ or SQLite 3.22+.
-
 Removal of old List Accounts Admin API
 --------------------------------------
 
@@ -118,16 +98,6 @@ has been available since Synapse 1.7.0 (2019-12-13), and is accessible under ``G
 
 The deprecation of the old endpoint was announced with Synapse 1.28.0 (released on 2021-02-25).
 
-Application Services must use type ``m.login.application_service`` when registering users
------------------------------------------------------------------------------------------
-
-In compliance with the
-`Application Service spec <https://matrix.org/docs/spec/application_service/r0.1.2#server-admin-style-permissions>`_,
-Application Services are now required to use the ``m.login.application_service`` type when registering users via the
-``/_matrix/client/r0/register`` endpoint. This behaviour was deprecated in Synapse v1.30.0.
-
-Please ensure your Application Services are up to date.
-
 Upgrading to v1.29.0
 ====================
 

changelog.d/9803.doc

@@ -0,0 +1 @@
+Add hardened systemd files as proposed in [#9760](https://github.com/matrix-org/synapse/issues/9760) and added them to `contrib/`. Change the docs to reflect the presence of these files.

contrib/systemd/override-hardened.conf

@@ -0,0 +1,71 @@
+[Service]
+# The following directives give the synapse service R/W access to:
+# - /run/matrix-synapse
+# - /var/lib/matrix-synapse
+# - /var/log/matrix-synapse
+
+RuntimeDirectory=matrix-synapse
+StateDirectory=matrix-synapse
+LogsDirectory=matrix-synapse
+
+######################
+## Security Sandbox ##
+######################
+
+# Make sure that the service has its own unshared tmpfs at /tmp and that it
+# cannot see or change any real devices
+PrivateTmp=true
+PrivateDevices=true
+
+# We give no capabilities to a service by default
+CapabilityBoundingSet=
+AmbientCapabilities=
+
+# Protect the following from modification:
+# - The entire filesystem
+# - sysctl settings and loaded kernel modules
+# - No modifications allowed to Control Groups
+# - Hostname
+# - System Clock
+ProtectSystem=strict
+ProtectKernelTunables=true
+ProtectKernelModules=true
+ProtectControlGroups=true
+ProtectClock=true
+ProtectHostname=true
+
+# Prevent access to the following:
+# - /home directory
+# - Kernel logs
+ProtectHome=tmpfs
+ProtectKernelLogs=true
+
+# Make sure that the process can only see PIDs and process details of itself,
+# and the second option disables seeing details of things like system load and
+# I/O etc
+ProtectProc=invisible
+ProcSubset=pid
+
+# While not needed, we set these options explicitly
+# - This process has been given access to the host network
+# - It can also communicate with any IP Address
+PrivateNetwork=false
+RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
+IPAddressAllow=any
+
+# Restrict system calls to a sane bunch
+SystemCallArchitectures=native
+SystemCallFilter=@system-service
+SystemCallFilter=~@privileged @resources @obsolete
+
+# Misc restrictions
+# - Since the process is a python process it needs to be able to write and
+#   execute memory regions, so we set MemoryDenyWriteExecute to false
+RestrictSUIDSGID=true
+RemoveIPC=true
+NoNewPrivileges=true
+RestrictRealtime=true
+RestrictNamespaces=true
+LockPersonality=true
+PrivateUsers=true
+MemoryDenyWriteExecute=false

debian/changelog

@@ -1,18 +1,8 @@
-matrix-synapse-py3 (1.32.1) stable; urgency=medium
+matrix-synapse-py3 (1.31.0+nmu1) UNRELEASED; urgency=medium
 
-  * New synapse release 1.32.1.
-
- -- Synapse Packaging team <packages@matrix.org>  Wed, 21 Apr 2021 14:00:55 +0100
-
-matrix-synapse-py3 (1.32.0) stable; urgency=medium
-
-  [ Dan Callahan ]
   * Skip tests when DEB_BUILD_OPTIONS contains "nocheck".
 
-  [ Synapse Packaging team ]
-  * New synapse release 1.32.0.
-
- -- Synapse Packaging team <packages@matrix.org>  Tue, 20 Apr 2021 14:28:39 +0100
+ -- Dan Callahan <danc@element.io>  Mon, 12 Apr 2021 13:07:36 +0000
 
 matrix-synapse-py3 (1.31.0) stable; urgency=medium
 

docs/systemd-with-workers/README.md

@@ -65,3 +65,33 @@ systemctl restart matrix-synapse-worker@federation_reader.service
 systemctl enable matrix-synapse-worker@federation_writer.service
 systemctl restart matrix-synapse.target
 ```
+
+## Hardening
+
+**Optional:** If further hardening is desired, the file
+`override-hardened.conf` may be copied from
+`contrib/systemd/override-hardened.conf` in this repository to the location
+`/etc/systemd/system/matrix-synapse.service.d/override-hardened.conf` (the
+directory may have to be created). It enables certain sandboxing features in
+systemd to further secure the synapse service. You may read the comments to
+understand what the override file is doing. The same file will need to be copied
+to
+`/etc/systemd/system/matrix-synapse-worker@.service.d/override-hardened-worker.conf`
+(this directory may also have to be created) in order to apply the same
+hardening options to any worker processes.
+
+Once these files have been copied to their appropriate locations, simply reload
+systemd's manager config files and restart all Synapse services to apply the hardening options. They will automatically
+be applied at every restart as long as the override files are present at the
+specified locations.
+
+```sh
+systemctl daemon-reload
+
+# Restart services
+systemctl restart matrix-synapse.target
+```
+
+In order to see their effect, you may run `systemd-analyze security
+matrix-synapse.service` before and after applying the hardening options to see
+the changes being applied at a glance.

synapse/__init__.py

@@ -48,7 +48,7 @@ try:
 except ImportError:
     pass
 
-__version__ = "1.32.1"
+__version__ = "1.32.0rc1"
 
 if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
     # We import here so that we don't have to install a bunch of deps when

synapse/logging/context.py

@@ -277,7 +277,7 @@ class LoggingContext:
 
     def __init__(
         self,
-        name: str,
+        name: Optional[str] = None,
         parent_context: "Optional[LoggingContext]" = None,
         request: Optional[ContextRequest] = None,
     ) -> None:
@@ -315,7 +315,9 @@ class LoggingContext:
             self.request = request
 
     def __str__(self) -> str:
-        return self.name
+        if self.request:
+            return self.request.request_id
+        return "%s@%x" % (self.name, id(self))
 
     @classmethod
     def current_context(cls) -> LoggingContextOrSentinel:
@@ -692,13 +694,17 @@ def nested_logging_context(suffix: str) -> LoggingContext:
             "Starting nested logging context from sentinel context: metrics will be lost"
         )
         parent_context = None
+        prefix = ""
+        request = None
     else:
         assert isinstance(curr_context, LoggingContext)
         parent_context = curr_context
-    prefix = str(curr_context)
+        prefix = str(parent_context.name)
+        request = parent_context.request
     return LoggingContext(
         prefix + "-" + suffix,
         parent_context=parent_context,
+        request=request,
     )
 
 
@@ -889,7 +895,7 @@ def defer_to_threadpool(reactor, threadpool, f, *args, **kwargs):
         parent_context = curr_context
 
     def g():
-        with LoggingContext(str(curr_context), parent_context=parent_context):
+        with LoggingContext(parent_context=parent_context):
             return f(*args, **kwargs)
 
     return make_deferred_yieldable(threads.deferToThreadPool(reactor, threadpool, g))

synapse/metrics/background_process_metrics.py

@@ -242,24 +242,19 @@ class BackgroundProcessLoggingContext(LoggingContext):
     processes.
     """
 
-    __slots__ = ["_proc"]
+    __slots__ = ["_id", "_proc"]
 
-    def __init__(self, name: str, instance_id: Optional[Union[int, str]] = None):
-        """
+    def __init__(self, name: str, id: Optional[Union[int, str]] = None):
+        super().__init__(name)
+        self._id = id
 
-        Args:
-            name: The name of the background process. Each distinct `name` gets a
-                separate prometheus time series.
-
-            instance_id: an identifer to add to `name` to distinguish this instance of
-                the named background process in the logs. If this is `None`, one is
-                made up based on id(self).
-        """
-        if instance_id is None:
-            instance_id = id(self)
-        super().__init__("%s-%s" % (name, instance_id))
         self._proc = _BackgroundProcess(name, self)
 
+    def __str__(self) -> str:
+        if self._id is not None:
+            return "%s-%s" % (self.name, self._id)
+        return "%s@%x" % (self.name, id(self))
+
     def start(self, rusage: "Optional[resource._RUsage]"):
         """Log context has started running (again)."""
 

synapse/util/metrics.py

@@ -105,13 +105,7 @@ class Measure:
         "start",
     ]
 
-    def __init__(self, clock, name: str):
-        """
-        Args:
-            clock: A n object with a "time()" method, which returns the current
-                time in seconds.
-            name: The name of the metric to report.
-        """
+    def __init__(self, clock, name):
         self.clock = clock
         self.name = name
         curr_context = current_context()
@@ -124,8 +118,10 @@ class Measure:
         else:
             assert isinstance(curr_context, LoggingContext)
             parent_context = curr_context
-        self._logging_context = LoggingContext(str(curr_context), parent_context)
-        self.start = None  # type: Optional[int]
+        self._logging_context = LoggingContext(
+            "Measure[%s]" % (self.name,), parent_context
+        )
+        self.start = None
 
     def __enter__(self) -> "Measure":
         if self.start is not None:

tests/logging/test_terse_json.py

@@ -138,7 +138,7 @@ class TerseJsonTestCase(LoggerCleanupMixin, TestCase):
         ]
         self.assertCountEqual(log.keys(), expected_log_keys)
         self.assertEqual(log["log"], "Hello there, wally!")
-        self.assertEqual(log["request"], "name")
+        self.assertTrue(log["request"].startswith("name@"))
 
     def test_with_request_context(self):
         """
@@ -165,9 +165,7 @@ class TerseJsonTestCase(LoggerCleanupMixin, TestCase):
         # Also set the requester to ensure the processing works.
         request.requester = "@foo:test"
 
-        with LoggingContext(
-            request.get_request_id(), parent_context=request.logcontext
-        ):
+        with LoggingContext(parent_context=request.logcontext):
             logger.info("Hello there, %s!", "wally")
 
         log = self.get_log_line()

tests/test_federation.py

@@ -134,7 +134,7 @@ class MessageAcceptTests(unittest.HomeserverTestCase):
             }
         )
 
-        with LoggingContext("test-context"):
+        with LoggingContext():
             failure = self.get_failure(
                 self.handler.on_receive_pdu(
                     "test.serv", lying_event, sent_to_us_directly=True

tests/util/caches/test_descriptors.py

@@ -231,7 +231,8 @@ class DescriptorTestCase(unittest.TestCase):
 
         @defer.inlineCallbacks
         def do_lookup():
-            with LoggingContext("c1") as c1:
+            with LoggingContext() as c1:
+                c1.name = "c1"
                 r = yield obj.fn(1)
                 self.assertEqual(current_context(), c1)
             return r
@@ -273,7 +274,8 @@ class DescriptorTestCase(unittest.TestCase):
 
         @defer.inlineCallbacks
         def do_lookup():
-            with LoggingContext("c1") as c1:
+            with LoggingContext() as c1:
+                c1.name = "c1"
                 try:
                     d = obj.fn(1)
                     self.assertEqual(