Version picker added for v1.49 docs

postgres 10 _+_

CHANGES.md

@@ -1,5 +1,46 @@
-Synapse 1.49.0rc1 (2021-12-07)
-==============================
+Synapse 1.49.2 (2021-12-21)
+===========================
+
+This release fixes a regression introduced in Synapse 1.49.0 which could cause `/sync` requests to take significantly longer. This would particularly affect "initial" syncs for users participating in a large number of rooms, and in extreme cases, could make it impossible for such users to log in on a new client.
+
+**Note:** in line with our [deprecation policy](https://matrix-org.github.io/synapse/latest/deprecation_policy.html) for platform dependencies, this will be the last release to support Python 3.6 and PostgreSQL 9.6, both of which have now reached upstream end-of-life. Synapse will require Python 3.7+ and PostgreSQL 10+.
+
+**Note:** We will also stop producing packages for Ubuntu 18.04 (Bionic Beaver) after this release, as it uses Python 3.6.
+
+Bugfixes
+--------
+
+- Fix a performance regression in `/sync` handling, introduced in 1.49.0. ([\#11583](https://github.com/matrix-org/synapse/issues/11583))
+
+Internal Changes
+----------------
+
+- Work around a build problem on Debian Buster. ([\#11625](https://github.com/matrix-org/synapse/issues/11625))
+
+
+Synapse 1.49.1 (2021-12-21)
+===========================
+
+Not released due to problems building the debian packages.
+
+
+Synapse 1.49.0 (2021-12-14)
+===========================
+
+No significant changes since version 1.49.0rc1.
+
+
+Support for Ubuntu 21.04 ends next month on the 20th of January
+---------------------------------------------------------------
+
+For users of Ubuntu 21.04 (Hirsute Hippo), please be aware that [upstream support for this version of Ubuntu will end next month][Ubuntu2104EOL].
+We will stop producing packages for Ubuntu 21.04 after upstream support ends.
+
+[Ubuntu2104EOL]: https://lists.ubuntu.com/archives/ubuntu-announce/2021-December/000275.html
+
+
+The wiki has been migrated to the documentation website
+-------------------------------------------------------
 
 We've decided to move the existing, somewhat stagnant pages from the GitHub wiki
 to the [documentation website](https://matrix-org.github.io/synapse/latest/).
@@ -16,6 +57,9 @@ requests](https://github.com/matrix-org/synapse/pulls). Please visit [#synapse-d
 if you need help with the process!
 
 
+Synapse 1.49.0rc1 (2021-12-07)
+==============================
+
 Features
 --------
 

book.toml

@@ -34,6 +34,14 @@ additional-css = [
     "docs/website_files/table-of-contents.css",
     "docs/website_files/remove-nav-buttons.css",
     "docs/website_files/indent-section-headers.css",
+    "docs/website_files/version-picker.css",
 ]
-additional-js = ["docs/website_files/table-of-contents.js"]
-theme = "docs/website_files/theme"
+additional-js = [
+    "docs/website_files/table-of-contents.js",
+    "docs/website_files/version-picker.js",
+    "docs/website_files/version.js",
+]
+theme = "docs/website_files/theme"
+
+[preprocessor.schema_versions]
+command = "./scripts-dev/schema_versions.py"

changelog.d/10520.misc

@@ -1 +0,0 @@
-Send and handle cross-signing messages using the stable prefix.

changelog.d/11331.misc

@@ -1 +0,0 @@
-A test helper (`wait_for_background_updates`) no longer depends on classes defining a `store` property.

changelog.d/11358.feature

@@ -1 +0,0 @@
-Add an admin API endpoint to force a local user to leave all non-public rooms in a space.

debian/changelog

@@ -1,3 +1,21 @@
+matrix-synapse-py3 (1.49.2) stable; urgency=medium
+
+  * New synapse release 1.49.2.
+
+ -- Synapse Packaging team <packages@matrix.org>  Tue, 21 Dec 2021 17:31:03 +0000
+
+matrix-synapse-py3 (1.49.1) stable; urgency=medium
+
+  * New synapse release 1.49.1.
+
+ -- Synapse Packaging team <packages@matrix.org>  Tue, 21 Dec 2021 11:07:30 +0000
+
+matrix-synapse-py3 (1.49.0) stable; urgency=medium
+
+  * New synapse release 1.49.0.
+
+ -- Synapse Packaging team <packages@matrix.org>  Tue, 14 Dec 2021 12:39:46 +0000
+
 matrix-synapse-py3 (1.49.0~rc1) stable; urgency=medium
 
   * New synapse release 1.49.0~rc1.

docs/SUMMARY.md

@@ -61,7 +61,6 @@
       - [Registration Tokens](usage/administration/admin_api/registration_tokens.md)
       - [Manipulate Room Membership](admin_api/room_membership.md)
       - [Rooms](admin_api/rooms.md)
-      - [Spaces](usage/administration/admin_api/spaces.md)
       - [Server Notices](admin_api/server_notices.md)
       - [Statistics](admin_api/statistics.md)
       - [Users](admin_api/user_admin_api.md)

docs/deprecation_policy.md

@@ -14,8 +14,8 @@ i.e. when a version reaches End of Life Synapse will withdraw support for that
 version in future releases.
 
 Details on the upstream support life cycles for Python and PostgreSQL are
-documented at https://endoflife.date/python and
-https://endoflife.date/postgresql.
+documented at [https://endoflife.date/python](https://endoflife.date/python) and
+[https://endoflife.date/postgresql](https://endoflife.date/postgresql).
 
 
 Context

docs/usage/administration/admin_api/spaces.md

@@ -1,57 +0,0 @@
-# Spaces API
-
-This API allows a server administrator to manage spaces.
-
-## Remove local user
-
-This API forces a local user to leave all non-public rooms in a space.
-
-The space itself is always left, regardless of whether it is public.
-
-May succeed partially if the user fails to leave some rooms.
-
-The API is:
-
-```
-DELETE /_synapse/admin/v1/rooms/<room_id>/hierarchy/members/<user_id>
-```
-
-with an optional body of:
-
-```json
-{
-  "include_remote_spaces": true,
-}
-```
-
-`include_remote_spaces` controls whether to process subspaces that the
-local homeserver is not participating in. The listings of such subspaces
-have to be retrieved over federation and their accuracy cannot be
-guaranteed.
-
-Returning:
-
-```json
-{
-    "left_rooms": ["!room1:example.net", "!room2:example.net", ...],
-    "inaccessible_rooms": ["!subspace1:example.net", ...],
-    "failed_rooms": {
-        "!room4:example.net": "Failed to leave room.",
-        ...
-    }
-}
-```
-
-`left_rooms`: A list of rooms that the user has been made to leave.
-
-`inaccessible_rooms`: A list of rooms and spaces that the local
-homeserver is not in, and may have not been fully processed. Rooms may
-appear here if:
-  * The room is a space that the local homeserver is not in, and so its
-    full list of child rooms could not be determined.
-  * The room is inaccessible to the local homeserver, and it is not
-    known whether the room is a subspace containing further rooms.
-
-`failed_rooms`: A dictionary of errors encountered when leaving rooms.
-The keys of the dictionary are room IDs and the values of the dictionary
-are error messages.

docs/website_files/README.md

@@ -24,6 +24,11 @@ Finally, we also stylise the chapter titles in the left sidebar by indenting the
 slightly so that they are more visually distinguishable from the section headers
 (the bold titles). This is done through the `indent-section-headers.css` file.
 
+In addition to these modifications, we have added a version picker to the documentation.
+Users can switch between documentations for different versions of Synapse.
+This functionality was implemented through the `version-picker.js` and
+`version-picker.css` files.
+
 More information can be found in mdbook's official documentation for
 [injecting page JS/CSS](https://rust-lang.github.io/mdBook/format/config.html)
 and

docs/website_files/theme/index.hbs

@@ -131,6 +131,18 @@
                             <i class="fa fa-search"></i>
                         </button>
                         {{/if}}
+                        <div class="version-picker">
+                            <div class="dropdown">
+                                <div class="select">
+                                    <span></span>
+                                    <i class="fa fa-chevron-down"></i>
+                                </div>
+                                <input type="hidden" name="version">
+                                <ul class="dropdown-menu">
+                                    <!-- Versions will be added dynamically in version-picker.js -->
+                                </ul>
+                            </div>
+                        </div>      
                     </div>
 
                     <h1 class="menu-title">{{ book_title }}</h1>
@@ -309,4 +321,4 @@
         {{/if}}
 
     </body>
-</html>
+</html>

docs/website_files/version-picker.css

@@ -0,0 +1,78 @@
+.version-picker {
+    display: flex;
+    align-items: center;
+}
+
+.version-picker .dropdown {
+    width: 130px;
+    max-height: 29px;
+    margin-left: 10px;
+    display: inline-block;
+    border-radius: 4px;
+    border: 1px solid var(--theme-popup-border);
+    position: relative;
+    font-size: 13px;
+    color: var(--fg);
+    height: 100%;
+    text-align: left;
+}
+.version-picker .dropdown .select {
+    cursor: pointer;
+    display: block;
+    padding: 5px 2px 5px 15px;
+}
+.version-picker .dropdown .select > i {
+    font-size: 10px;
+    color: var(--fg);
+    cursor: pointer;
+    float: right;
+    line-height: 20px !important;
+}
+.version-picker .dropdown:hover {
+    border: 1px solid var(--theme-popup-border);
+}
+.version-picker .dropdown:active {
+    background-color: var(--theme-popup-bg);
+}
+.version-picker .dropdown.active:hover,
+.version-picker .dropdown.active {
+    border: 1px solid var(--theme-popup-border);
+    border-radius: 2px 2px 0 0;
+    background-color: var(--theme-popup-bg);
+}
+.version-picker .dropdown.active .select > i {
+    transform: rotate(-180deg);
+}
+.version-picker .dropdown .dropdown-menu {
+    position: absolute;
+    background-color: var(--theme-popup-bg);
+    width: 100%;
+    left: -1px;
+    right: 1px;
+    margin-top: 1px;
+    border: 1px solid var(--theme-popup-border);
+    border-radius: 0 0 4px 4px;
+    overflow: hidden;
+    display: none;
+    max-height: 300px;
+    overflow-y: auto;
+    z-index: 9;
+}
+.version-picker .dropdown .dropdown-menu li {
+    font-size: 12px;
+    padding: 6px 20px;
+    cursor: pointer;
+} 
+.version-picker .dropdown .dropdown-menu {
+    padding: 0;
+    list-style: none;
+}
+.version-picker .dropdown .dropdown-menu li:hover {
+    background-color: var(--theme-hover);
+}
+.version-picker .dropdown .dropdown-menu li.active::before {
+    display: inline-block;
+    content: "✓";
+    margin-inline-start: -14px;
+    width: 14px;
+}

docs/website_files/version-picker.js

@@ -0,0 +1,127 @@
+
+const dropdown = document.querySelector('.version-picker .dropdown');
+const dropdownMenu = dropdown.querySelector('.dropdown-menu');
+
+fetchVersions(dropdown, dropdownMenu).then(() => {
+    initializeVersionDropdown(dropdown, dropdownMenu);
+});
+
+/**
+ * Initialize the dropdown functionality for version selection.
+ * 
+ * @param {Element} dropdown - The dropdown element.
+ * @param {Element} dropdownMenu - The dropdown menu element.
+ */
+function initializeVersionDropdown(dropdown, dropdownMenu) {
+    // Toggle the dropdown menu on click
+    dropdown.addEventListener('click', function () {
+        this.setAttribute('tabindex', 1);
+        this.classList.toggle('active');
+        dropdownMenu.style.display = (dropdownMenu.style.display === 'block') ? 'none' : 'block';
+    });
+  
+    // Remove the 'active' class and hide the dropdown menu on focusout
+    dropdown.addEventListener('focusout', function () {
+        this.classList.remove('active');
+        dropdownMenu.style.display = 'none';
+    });
+  
+    // Handle item selection within the dropdown menu
+    const dropdownMenuItems = dropdownMenu.querySelectorAll('li');    
+    dropdownMenuItems.forEach(function (item) {
+        item.addEventListener('click', function () {
+            dropdownMenuItems.forEach(function (item) {
+                item.classList.remove('active');
+            });
+            this.classList.add('active');
+            dropdown.querySelector('span').textContent = this.textContent;
+            dropdown.querySelector('input').value = this.getAttribute('id');
+
+            window.location.href = changeVersion(window.location.href, this.textContent);
+        });
+    });
+};
+
+/**
+ * This function fetches the available versions from a GitHub repository
+ * and inserts them into the version picker.
+ * 
+ * @param {Element} dropdown - The dropdown element.
+ * @param {Element} dropdownMenu - The dropdown menu element.
+ * @returns {Promise<Array<string>>} A promise that resolves with an array of available versions.
+ */
+function fetchVersions(dropdown, dropdownMenu) {
+    return new Promise((resolve, reject) => {
+        window.addEventListener("load", () => {
+
+            fetch("https://api.github.com/repos/matrix-org/synapse/git/trees/gh-pages", {
+                cache: "force-cache",
+            }).then(res => 
+                res.json()
+            ).then(resObject => {
+                const excluded = ['dev-docs', 'v1.91.0', 'v1.80.0', 'v1.69.0'];
+                const tree = resObject.tree.filter(item => item.type === "tree" && !excluded.includes(item.path));
+                const versions = tree.map(item => item.path).sort(sortVersions);
+
+                // Create a list of <li> items for versions
+                versions.forEach((version) => {
+                    const li = document.createElement("li");
+                    li.textContent = version;
+                    li.id = version;
+    
+                    if (window.SYNAPSE_VERSION === version) {
+                        li.classList.add('active');
+                        dropdown.querySelector('span').textContent = version;
+                        dropdown.querySelector('input').value = version;
+                    }
+    
+                    dropdownMenu.appendChild(li);
+                });
+
+                resolve(versions);
+
+            }).catch(ex => {
+                console.error("Failed to fetch version data", ex);
+                reject(ex);
+            })
+        });
+    });
+}
+
+/**
+ * Custom sorting function to sort an array of version strings.
+ *
+ * @param {string} a - The first version string to compare.
+ * @param {string} b - The second version string to compare.
+ * @returns {number} - A negative number if a should come before b, a positive number if b should come before a, or 0 if they are equal.
+ */
+function sortVersions(a, b) {
+    // Put 'develop' and 'latest' at the top
+    if (a === 'develop' || a === 'latest') return -1;
+    if (b === 'develop' || b === 'latest') return 1;
+
+    const versionA = (a.match(/v\d+(\.\d+)+/) || [])[0];
+    const versionB = (b.match(/v\d+(\.\d+)+/) || [])[0];
+
+    return versionB.localeCompare(versionA);
+}
+
+/**
+ * Change the version in a URL path.
+ *
+ * @param {string} url - The original URL to be modified.
+ * @param {string} newVersion - The new version to replace the existing version in the URL.
+ * @returns {string} The updated URL with the new version.
+ */
+function changeVersion(url, newVersion) {
+    const parsedURL = new URL(url);
+    const pathSegments = parsedURL.pathname.split('/');
+  
+    // Modify the version
+    pathSegments[2] = newVersion;
+
+    // Reconstruct the URL
+    parsedURL.pathname = pathSegments.join('/');
+  
+    return parsedURL.href;
+}

docs/website_files/version.js

@@ -0,0 +1 @@
+window.SYNAPSE_VERSION = 'v1.49';

synapse/__init__.py

@@ -47,7 +47,7 @@ try:
 except ImportError:
     pass
 
-__version__ = "1.49.0rc1"
+__version__ = "1.49.2"
 
 if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
     # We import here so that we don't have to install a bunch of deps when

synapse/handlers/e2e_keys.py

@@ -65,12 +65,8 @@ class E2eKeysHandler:
         else:
             # Only register this edu handler on master as it requires writing
             # device updates to the db
-            federation_registry.register_edu_handler(
-                "m.signing_key_update",
-                self._edu_updater.incoming_signing_key_update,
-            )
-            # also handle the unstable version
-            # FIXME: remove this when enough servers have upgraded
+            #
+            # FIXME: switch to m.signing_key_update when MSC1756 is merged into the spec
             federation_registry.register_edu_handler(
                 "org.matrix.signing_key_update",
                 self._edu_updater.incoming_signing_key_update,

synapse/handlers/room_summary.py

@@ -1045,7 +1045,7 @@ class RoomSummaryHandler:
 
         # filter out any events without a "via" (which implies it has been redacted),
         # and order to ensure we return stable results.
-        return sorted(filter(has_valid_via, events), key=child_events_comparison_key)
+        return sorted(filter(_has_valid_via, events), key=_child_events_comparison_key)
 
     async def get_room_summary(
         self,
@@ -1139,7 +1139,7 @@ class _RoomEntry:
         return result
 
 
-def has_valid_via(e: EventBase) -> bool:
+def _has_valid_via(e: EventBase) -> bool:
     via = e.content.get("via")
     if not via or not isinstance(via, Sequence):
         return False
@@ -1162,7 +1162,7 @@ def _is_suggested_child_event(edge_event: EventBase) -> bool:
 _INVALID_ORDER_CHARS_RE = re.compile(r"[^\x20-\x7E]")
 
 
-def child_events_comparison_key(
+def _child_events_comparison_key(
     child: EventBase,
 ) -> Tuple[bool, Optional[str], int, str]:
     """

synapse/handlers/space_hierarchy.py

@@ -1,294 +0,0 @@
-# Copyright 2021 The Matrix.org Foundation C.I.C.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import logging
-from typing import (
-    TYPE_CHECKING,
-    Dict,
-    Iterable,
-    List,
-    Mapping,
-    Optional,
-    Sequence,
-    Tuple,
-)
-
-from synapse.api.constants import EventContentFields, EventTypes, RoomTypes
-from synapse.api.errors import SynapseError
-from synapse.handlers.room_summary import child_events_comparison_key, has_valid_via
-from synapse.storage.state import StateFilter
-from synapse.types import JsonDict
-
-if TYPE_CHECKING:
-    from synapse.server import HomeServer
-
-
-logger = logging.getLogger(__name__)
-
-
-class SpaceHierarchyHandler:
-    """Provides methods for walking over space hierarchies.
-
-    Also see `RoomSummaryHandler`, which has similar functionality.
-    """
-
-    def __init__(self, hs: "HomeServer"):
-        self._store = hs.get_datastore()
-        self._federation_client = hs.get_federation_client()
-
-        self._server_name = hs.hostname
-
-    async def get_space_descendants(
-        self,
-        space_id: str,
-        via: Optional[Iterable[str]] = None,
-        enable_federation: Optional[bool] = True,
-    ) -> Tuple[Sequence[Tuple[str, Iterable[str]]], Sequence[str]]:
-        """Gets the children of a space, recursively.
-
-        Args:
-            space_id: The room ID of the space.
-            via: A list of servers which may know about the space.
-            enable_federation: A boolean controlling whether children of unknown rooms
-                should be fetched over federation. Defaults to `True`.
-
-        Returns:
-            A tuple containing:
-             * A list of (room ID, via) tuples, representing the descendants of the
-               space. `space_id` is included in the list.
-             * A list of room IDs whose children could not be fully listed.
-               Rooms in this list are either spaces not known locally, and thus require
-               listing over federation, or are unknown rooms or subspaces completely
-               inaccessible to the local homeserver which may contain further rooms.
-               Subspaces requiring listing over federation are always included here,
-               regardless of the value of the `enable_federation` flag.
-
-               This list is a subset of the previous list, except it may include
-               `space_id`.
-        """
-        via = via or []
-
-        # (room ID, via, federation room chunks)
-        todo: List[Tuple[str, Iterable[str], Mapping[str, Optional[JsonDict]]]] = [
-            (space_id, via, {})
-        ]
-        # [(room ID, via)]
-        descendants: List[Tuple[str, Iterable[str]]] = []
-
-        seen = {space_id}
-
-        inaccessible_room_ids: List[str] = []
-
-        while todo:
-            space_id, via, federation_room_chunks = todo.pop()
-            descendants.append((space_id, via))
-            try:
-                (
-                    is_in_room,
-                    children,
-                    federation_room_chunks,
-                ) = await self._get_space_children(
-                    space_id,
-                    via,
-                    federation_room_chunks,
-                    enable_federation=enable_federation,
-                )
-            except SynapseError:
-                # Could not list children over federation
-                inaccessible_room_ids.append(space_id)
-                continue
-
-            # Children were retrieved over federation, which is not guaranteed to be
-            # the full list.
-            if not is_in_room:
-                inaccessible_room_ids.append(space_id)
-
-            for child_room_id, child_via in reversed(children):
-                if child_room_id in seen:
-                    continue
-
-                seen.add(child_room_id)
-
-                # Queue up the child for processing.
-                # The child may not actually be a space, but that's checked by
-                # `_get_space_children`.
-                todo.append((child_room_id, child_via, federation_room_chunks))
-
-        return descendants, inaccessible_room_ids
-
-    async def _get_space_children(
-        self,
-        space_id: str,
-        via: Optional[Iterable[str]] = None,
-        federation_room_chunks: Optional[Mapping[str, Optional[JsonDict]]] = None,
-        enable_federation: Optional[bool] = True,
-    ) -> Tuple[
-        bool, Sequence[Tuple[str, Iterable[str]]], Mapping[str, Optional[JsonDict]]
-    ]:
-        """Gets the direct children of a space.
-
-        Args:
-            space_id: The room ID of the space.
-            via: A list of servers which may know about the space.
-            federation_room_chunks: A cache of room chunks previously returned by
-               `_get_space_children` that may be used to skip federation requests for
-               inaccessible or non-space rooms.
-
-        Returns:
-            A tuple containing:
-             * A boolean indicating whether `space_id` is known to the local homeserver.
-             * A list of (room ID, via) tuples, representing the children of the space,
-               if `space_id` refers to a space; an empty list otherwise.
-             * A dictionary of child room ID: `PublicRoomsChunk`s returned over
-               federation:
-               https://spec.matrix.org/latest/client-server-api/#get_matrixclientv3publicrooms
-               These are supposed to include extra `room_type` and `allowed_room_ids`
-               fields, as described in MSC2946.
-
-               Contains `None` for rooms to which the remote homeserver thinks we do not
-               have access.
-
-               Local information about rooms should be trusted over data in this
-               dictionary.
-
-        Raises:
-            SynapseError: if `space_id` is not known locally and its children could not
-                be retrieved over federation or `enable_federation` is `False`.
-        """
-        via = via or []
-        federation_room_chunks = federation_room_chunks or {}
-
-        is_in_room = await self._store.is_host_joined(space_id, self._server_name)
-        if is_in_room:
-            children = await self._get_space_children_local(space_id)
-            return True, children, {}
-        else:
-            # Check the room chunks previously returned over federation to see if we
-            # should really make a request.
-            # `federation_room_chunks` is intentionally not used earlier since we want
-            # to trust local data over data from federation.
-            if space_id in federation_room_chunks:
-                room_chunk = federation_room_chunks[space_id]
-                if room_chunk is None:
-                    # `space_id` is inaccessible to the local homeserver according to
-                    # federation.
-                    raise SynapseError(
-                        502, f"{space_id} is not accessible to the local homeserver"
-                    )
-                elif room_chunk.get("room_type") != RoomTypes.SPACE:
-                    # `space_id` is not a space according to federation.
-                    return False, [], {}
-
-            if not enable_federation:
-                raise SynapseError(
-                    502, f"{space_id} is not accessible to the local homeserver"
-                )
-
-            children, room_chunks = await self._get_space_children_remote(space_id, via)
-            return False, children, room_chunks
-
-    async def _get_space_children_local(
-        self, space_id: str
-    ) -> Sequence[Tuple[str, Iterable[str]]]:
-        """Gets the direct children of a space that the local homeserver is in.
-
-        Args:
-            space_id: The room ID of the space.
-
-        Returns:
-            A list of (room ID, via) tuples, representing the children of the space,
-            if `space_id` refers to a space; an empty list otherwise.
-
-        Raises:
-            ValueError: if `space_id` is not known locally.
-        """
-        # Fetch the `m.room.create` and `m.space.child` events for `space_id`
-        state_filter = StateFilter.from_types(
-            [(EventTypes.Create, ""), (EventTypes.SpaceChild, None)]
-        )
-        current_state_ids = await self._store.get_filtered_current_state_ids(
-            space_id, state_filter
-        )
-        state_events = await self._store.get_events_as_list(current_state_ids.values())
-        assert len(state_events) == len(current_state_ids)
-
-        create_event_id = current_state_ids.get((EventTypes.Create, ""))
-        if create_event_id is None:
-            # The local homeserver is not in this room
-            raise ValueError(f"{space_id} is not a room known locally.")
-
-        create_event = next(
-            event for event in state_events if event.event_id == create_event_id
-        )
-        if create_event.content.get(EventContentFields.ROOM_TYPE) != RoomTypes.SPACE:
-            # `space_id` is a regular room and not a space.
-            # Ignore any `m.space.child` events.
-            return []
-
-        child_events = [
-            event
-            for event in state_events
-            # Ignore events with a missing or non-array `via`, as per MSC1772
-            if event.event_id != create_event_id and has_valid_via(event)
-        ]
-        child_events.sort(key=child_events_comparison_key)
-        return [(event.state_key, event.content["via"]) for event in child_events]
-
-    async def _get_space_children_remote(
-        self, space_id: str, via: Iterable[str]
-    ) -> Tuple[Sequence[Tuple[str, Iterable[str]]], Mapping[str, Optional[JsonDict]]]:
-        """Gets the direct children of a space over federation.
-
-        Args:
-            space_id: The room ID of the space.
-            via: A list of servers which may know about the space.
-
-        Returns:
-            A tuple containing:
-             * A list of (room ID, via) tuples, representing the children of the space,
-               if `space_id` refers to a space; an empty list otherwise.
-             * A dictionary of child room ID: `PublicRoomsChunk`s returned over
-               federation:
-               https://spec.matrix.org/latest/client-server-api/#get_matrixclientv3publicrooms
-               These are supposed to include extra `room_type` and `allowed_room_ids`
-               fields, as described in MSC2946.
-
-               Contains `None` for rooms to which the remote homeserver thinks we do not
-               have access.
-
-        Raises:
-            SynapseError: if none of the remote servers provided us with the space's
-                children.
-        """
-        (
-            room,
-            children_chunks,
-            inaccessible_children,
-        ) = await self._federation_client.get_room_hierarchy(
-            via, space_id, suggested_only=False
-        )
-
-        child_events: List[JsonDict] = room["children_state"]
-        children = [
-            (child_event["room_id"], child_event["content"]["via"])
-            for child_event in child_events
-        ]
-
-        room_chunks: Dict[str, Optional[JsonDict]] = {}
-        room_chunks.update((room_id, None) for room_id in inaccessible_children)
-        room_chunks.update(
-            (room_chunk["room_id"], room_chunk) for room_chunk in children_chunks
-        )
-
-        return children, room_chunks

synapse/python_dependencies.py

@@ -50,7 +50,8 @@ logger = logging.getLogger(__name__)
 REQUIREMENTS = [
     # we use the TYPE_CHECKER.redefine method added in jsonschema 3.0.0
     "jsonschema>=3.0.0",
-    "frozendict>=1",
+    # frozendict 2.1.2 is broken on Debian 10: https://github.com/Marco-Sulla/python-frozendict/issues/41
+    "frozendict>=1,<2.1.2",
     "unpaddedbase64>=1.1.0",
     "canonicaljson>=1.4.0",
     # we use the type definitions added in signedjson 1.1.

synapse/rest/admin/__init__.py

@@ -66,7 +66,6 @@ from synapse.rest.admin.rooms import (
     RoomStateRestServlet,
 )
 from synapse.rest.admin.server_notice_servlet import SendServerNoticeServlet
-from synapse.rest.admin.space import RemoveSpaceMemberRestServlet
 from synapse.rest.admin.statistics import UserMediaStatisticsRestServlet
 from synapse.rest.admin.username_available import UsernameAvailableRestServlet
 from synapse.rest.admin.users import (
@@ -268,7 +267,6 @@ def register_servlets(hs: "HomeServer", http_server: HttpServer) -> None:
     RegistrationTokenRestServlet(hs).register(http_server)
     DestinationsRestServlet(hs).register(http_server)
     ListDestinationsRestServlet(hs).register(http_server)
-    RemoveSpaceMemberRestServlet(hs).register(http_server)
 
     # Some servlets only get registered for the main process.
     if hs.config.worker.worker_app is None:

synapse/rest/admin/space.py

@@ -1,169 +0,0 @@
-# Copyright 2021 The Matrix.org Foundation C.I.C.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-import logging
-from http import HTTPStatus
-from typing import TYPE_CHECKING, Dict, List, Tuple
-
-from synapse.api.constants import EventTypes, JoinRules, Membership
-from synapse.api.errors import SynapseError
-from synapse.http.servlet import (
-    ResolveRoomIdMixin,
-    RestServlet,
-    parse_json_object_from_request,
-)
-from synapse.http.site import SynapseRequest
-from synapse.rest.admin._base import admin_patterns, assert_user_is_admin
-from synapse.storage.state import StateFilter
-from synapse.types import JsonDict, UserID, create_requester
-
-if TYPE_CHECKING:
-    from synapse.server import HomeServer
-
-logger = logging.getLogger(__name__)
-
-
-class RemoveSpaceMemberRestServlet(ResolveRoomIdMixin, RestServlet):
-    """
-    Puppets a local user to remove them from all rooms in a space.
-    """
-
-    PATTERNS = admin_patterns(
-        "/rooms/(?P<space_id>[^/]+)/hierarchy/members/(?P<user_id>[^/]+)$"
-    )
-
-    def __init__(self, hs: "HomeServer"):
-        super().__init__(hs)
-        self._hs = hs
-        self._auth = hs.get_auth()
-        self._store = hs.get_datastore()
-        self._room_member_handler = hs.get_room_member_handler()
-        self._space_hierarchy_handler = hs.get_space_hierarchy_handler()
-
-    async def on_DELETE(
-        self, request: SynapseRequest, space_id: str, user_id: str
-    ) -> Tuple[int, JsonDict]:
-        """Forces a local user to leave all non-public rooms in a space.
-
-        The space itself is always left, regardless of whether it is public.
-
-        May succeed partially if the user fails to leave some rooms.
-
-        Returns:
-            A tuple containing the HTTP status code and a JSON dictionary containing:
-             * `left_rooms`: A list of rooms that the user has been made to leave.
-             * `inaccessible_rooms`: A list of rooms and spaces that the local
-                homeserver is not in, and may have not been fully processed. Rooms may
-                appear here if:
-                 * The room is a space that the local homeserver is not in, and so its
-                   full list of child rooms could not be determined.
-                 * The room is inaccessible to the local homeserver, and it is not known
-                   whether the room is a subspace containing further rooms.
-             * `failed_rooms`: A dictionary of errors encountered when leaving rooms.
-                The keys of the dictionary are room IDs and the values of the dictionary
-                are error messages.
-        """
-        requester = await self._auth.get_user_by_req(request)
-        await assert_user_is_admin(self._auth, requester.user)
-
-        content = parse_json_object_from_request(request, allow_empty_body=True)
-        include_remote_spaces = content.get("include_remote_spaces", True)
-        if not isinstance(include_remote_spaces, bool):
-            raise SynapseError(
-                HTTPStatus.BAD_REQUEST,
-                "'include_remote_spaces' parameter must be a boolean",
-            )
-
-        space_id, _ = await self.resolve_room_id(space_id)
-
-        target_user = UserID.from_string(user_id)
-
-        if not self._hs.is_mine(target_user):
-            raise SynapseError(
-                HTTPStatus.BAD_REQUEST,
-                "This endpoint can only be used with local users",
-            )
-
-        # Fetch the list of rooms the target user is currently in
-        user_rooms = await self._store.get_rooms_for_local_user_where_membership_is(
-            user_id, [Membership.INVITE, Membership.JOIN, Membership.KNOCK]
-        )
-        user_room_ids = {room.room_id for room in user_rooms}
-
-        # Fetch the list of rooms in the space hierarchy
-        (
-            descendants,
-            inaccessible_room_ids,
-        ) = await self._space_hierarchy_handler.get_space_descendants(
-            space_id, enable_federation=include_remote_spaces
-        )
-
-        # Determine which rooms to leave by checking join rules
-        rooms_to_leave: List[str] = []
-        state_filter = StateFilter.from_types([(EventTypes.JoinRules, "")])
-        for room_id, _via in descendants:
-            if room_id not in user_room_ids:
-                # The user is not in this room. There is nothing to do here.
-                continue
-
-            current_state_ids = await self._store.get_filtered_current_state_ids(
-                room_id, state_filter
-            )
-            join_rules_event_id = current_state_ids.get((EventTypes.JoinRules, ""))
-            if join_rules_event_id is not None:
-                join_rules_event = await self._store.get_event(join_rules_event_id)
-                join_rules = join_rules_event.content.get("join_rule")
-            else:
-                # The user is invited to or has knocked on a room that is not known
-                # locally. Assume that such rooms are not public and should be left.
-                # If it turns out that the room is actually public, then we've not
-                # actually prevented the user from joining it.
-                join_rules = None
-
-            # Leave the room if it is not public, or it is the root space.
-            if join_rules != JoinRules.PUBLIC or room_id == space_id:
-                rooms_to_leave.append(room_id)
-
-        # Now start leaving rooms
-        failures: Dict[str, str] = {}
-        left_rooms: List[str] = []
-
-        fake_requester = create_requester(
-            target_user, authenticated_entity=requester.user.to_string()
-        )
-
-        for room_id in rooms_to_leave:
-            # There is a race condition here where the user may have left or been kicked
-            # from a room since their list of memberships was fetched.
-            # `update_membership` will raise if the user is no longer in the room,
-            # but it's tricky to distinguish from other failure modes.
-
-            try:
-                await self._room_member_handler.update_membership(
-                    requester=fake_requester,
-                    target=target_user,
-                    room_id=room_id,
-                    action=Membership.LEAVE,
-                    content={},
-                    ratelimit=False,
-                    require_consent=False,
-                )
-                left_rooms.append(room_id)
-            except Exception as e:
-                failures[room_id] = str(e)
-
-        return 200, {
-            "left_rooms": left_rooms,
-            "inaccessible_rooms": inaccessible_room_ids,
-            "failed_rooms": failures,
-        }

synapse/rest/client/sync.py

@@ -522,7 +522,15 @@ class SyncRestServlet(RestServlet):
                 time_now=time_now,
                 # Don't bother to bundle aggregations if the timeline is unlimited,
                 # as clients will have all the necessary information.
-                bundle_aggregations=room.timeline.limited,
+                # bundle_aggregations=room.timeline.limited,
+                #
+                # richvdh 2021-12-15: disable this temporarily as it has too high an
+                # overhead for initialsyncs. We need to figure out a way that the
+                # bundling can be done *before* the events are stored in the
+                # SyncResponseCache so that this part can be synchronous.
+                #
+                # Ensure to re-enable the test at tests/rest/client/test_relations.py::RelationsTestCase.test_bundled_aggregations.
+                bundle_aggregations=False,
                 token_id=token_id,
                 event_format=event_formatter,
                 only_event_fields=only_fields,

synapse/server.py

@@ -107,7 +107,6 @@ from synapse.handlers.room_summary import RoomSummaryHandler
 from synapse.handlers.search import SearchHandler
 from synapse.handlers.send_email import SendEmailHandler
 from synapse.handlers.set_password import SetPasswordHandler
-from synapse.handlers.space_hierarchy import SpaceHierarchyHandler
 from synapse.handlers.sso import SsoHandler
 from synapse.handlers.stats import StatsHandler
 from synapse.handlers.sync import SyncHandler
@@ -796,10 +795,6 @@ class HomeServer(metaclass=abc.ABCMeta):
     def get_account_data_handler(self) -> AccountDataHandler:
         return AccountDataHandler(self)
 
-    @cache_in_self
-    def get_space_hierarchy_handler(self) -> SpaceHierarchyHandler:
-        return SpaceHierarchyHandler(self)
-
     @cache_in_self
     def get_room_summary_handler(self) -> RoomSummaryHandler:
         return RoomSummaryHandler(self)

synapse/storage/databases/main/devices.py

@@ -274,9 +274,7 @@ class DeviceWorkerStore(SQLBaseStore):
         # add the updated cross-signing keys to the results list
         for user_id, result in cross_signing_keys_by_user.items():
             result["user_id"] = user_id
-            results.append(("m.signing_key_update", result))
-            # also send the unstable version
-            # FIXME: remove this when enough servers have upgraded
+            # FIXME: switch to m.signing_key_update when MSC1756 is merged into the spec
             results.append(("org.matrix.signing_key_update", result))
 
         return now_stream_id, results

tests/federation/test_federation_sender.py

@@ -266,8 +266,7 @@ class FederationSenderDevicesTestCases(HomeserverTestCase):
         )
 
         # expect signing key update edu
-        self.assertEqual(len(self.edus), 2)
-        self.assertEqual(self.edus.pop(0)["edu_type"], "m.signing_key_update")
+        self.assertEqual(len(self.edus), 1)
         self.assertEqual(self.edus.pop(0)["edu_type"], "org.matrix.signing_key_update")
 
         # sign the devices
@@ -492,7 +491,7 @@ class FederationSenderDevicesTestCases(HomeserverTestCase):
     ) -> None:
         """Check that the txn has an EDU with a signing key update."""
         edus = txn["edus"]
-        self.assertEqual(len(edus), 2)
+        self.assertEqual(len(edus), 1)
 
     def generate_and_upload_device_signing_key(
         self, user_id: str, device_id: str

tests/handlers/test_room_summary.py

@@ -28,7 +28,7 @@ from synapse.api.constants import (
 from synapse.api.errors import AuthError, NotFoundError, SynapseError
 from synapse.api.room_versions import RoomVersions
 from synapse.events import make_event_from_dict
-from synapse.handlers.room_summary import _RoomEntry, child_events_comparison_key
+from synapse.handlers.room_summary import _child_events_comparison_key, _RoomEntry
 from synapse.rest import admin
 from synapse.rest.client import login, room
 from synapse.server import HomeServer
@@ -48,7 +48,7 @@ def _create_event(room_id: str, order: Optional[Any] = None, origin_server_ts: i
 
 
 def _order(*events):
-    return sorted(events, key=child_events_comparison_key)
+    return sorted(events, key=_child_events_comparison_key)
 
 
 class TestSpaceSummarySort(unittest.TestCase):

tests/handlers/test_space_hierarchy.py

@@ -1,239 +0,0 @@
-# Copyright 2021 The Matrix.org Foundation C.I.C.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from typing import Dict, Iterable, Mapping, NoReturn, Optional, Sequence, Tuple
-from unittest import mock
-
-from twisted.test.proto_helpers import MemoryReactor
-
-from synapse.api.constants import EventContentFields, EventTypes, RoomTypes
-from synapse.handlers.space_hierarchy import SpaceHierarchyHandler
-from synapse.rest import admin
-from synapse.rest.client import login, room
-from synapse.server import HomeServer
-from synapse.types import JsonDict
-from synapse.util import Clock
-
-from tests import unittest
-
-
-class SpaceDescendantsTestCase(unittest.HomeserverTestCase):
-    """Tests iteration over the descendants of a space."""
-
-    servlets = [
-        admin.register_servlets_for_client_rest_resource,
-        login.register_servlets,
-        room.register_servlets,
-    ]
-
-    def prepare(self, reactor: MemoryReactor, clock: Clock, hs: HomeServer):
-        self.hs = hs
-        self.handler = self.hs.get_space_hierarchy_handler()
-
-        # Create a user.
-        self.user = self.register_user("user", "pass")
-        self.token = self.login("user", "pass")
-
-        # Create a space and a child room.
-        self.space = self.helper.create_room_as(
-            self.user,
-            tok=self.token,
-            extra_content={
-                "creation_content": {EventContentFields.ROOM_TYPE: RoomTypes.SPACE}
-            },
-        )
-        self.room = self.helper.create_room_as(self.user, tok=self.token)
-        self._add_child(self.space, self.room)
-
-    def _add_child(
-        self, space_id: str, room_id: str, order: Optional[str] = None
-    ) -> None:
-        """Adds a room to a space."""
-        content: JsonDict = {"via": [self.hs.hostname]}
-        if order is not None:
-            content["order"] = order
-        self.helper.send_state(
-            space_id,
-            event_type=EventTypes.SpaceChild,
-            body=content,
-            tok=self.token,
-            state_key=room_id,
-        )
-
-    def _create_space(self) -> str:
-        """Creates a space."""
-        return self._create_room(
-            extra_content={
-                "creation_content": {EventContentFields.ROOM_TYPE: RoomTypes.SPACE}
-            },
-        )
-
-    def _create_room(self, extra_content: Optional[Dict] = None) -> str:
-        """Creates a room."""
-        return self.helper.create_room_as(
-            self.user,
-            tok=self.token,
-            extra_content=extra_content,
-        )
-
-    def test_empty_space(self):
-        """Tests iteration over an empty space."""
-        space_id = self._create_space()
-
-        descendants, inaccessible_room_ids = self.get_success(
-            self.handler.get_space_descendants(space_id)
-        )
-
-        self.assertEqual(descendants, [(space_id, [])])
-        self.assertEqual(inaccessible_room_ids, [])
-
-    def test_invalid_space(self):
-        """Tests iteration over an inaccessible space."""
-        space_id = f"!invalid:{self.hs.hostname}"
-
-        descendants, inaccessible_room_ids = self.get_success(
-            self.handler.get_space_descendants(space_id)
-        )
-
-        self.assertEqual(descendants, [(space_id, [])])
-        self.assertEqual(inaccessible_room_ids, [space_id])
-
-    def test_invalid_room(self):
-        """Tests iteration over a space containing an inaccessible room."""
-        space_id = self._create_space()
-        room_id = f"!invalid:{self.hs.hostname}"
-        self._add_child(space_id, room_id)
-
-        descendants, inaccessible_room_ids = self.get_success(
-            self.handler.get_space_descendants(space_id)
-        )
-
-        self.assertEqual(descendants, [(space_id, []), (room_id, [self.hs.hostname])])
-        self.assertEqual(inaccessible_room_ids, [room_id])
-
-    def test_remote_space_with_federation_enabled(self):
-        """Tests iteration over a remote space with federation enabled."""
-        space_id = "!space:remote"
-        room_id = "!room:remote"
-
-        async def _get_space_children_remote(
-            _self: SpaceHierarchyHandler, space_id: str, via: Iterable[str]
-        ) -> Tuple[
-            Sequence[Tuple[str, Iterable[str]]], Mapping[str, Optional[JsonDict]]
-        ]:
-            if space_id == "!space:remote":
-                self.assertEqual(via, ["remote"])
-                return [("!room:remote", ["remote"])], {}
-            elif space_id == "!room:remote":
-                self.assertEqual(via, ["remote"])
-                return [], {}
-            else:
-                self.fail(
-                    f"Unexpected _get_space_children_remote({space_id!r}, {via!r}) call"
-                )
-                raise  # `fail` is missing type hints
-
-        with mock.patch(
-            "synapse.handlers.space_hierarchy.SpaceHierarchyHandler._get_space_children_remote",
-            new=_get_space_children_remote,
-        ):
-            descendants, inaccessible_room_ids = self.get_success(
-                self.handler.get_space_descendants(
-                    space_id, via=["remote"], enable_federation=True
-                )
-            )
-
-        self.assertEqual(descendants, [(space_id, ["remote"]), (room_id, ["remote"])])
-        self.assertEqual(inaccessible_room_ids, [space_id, room_id])
-
-    def test_remote_space_with_federation_disabled(self):
-        """Tests iteration over a remote space with federation disabled."""
-        space_id = "!space:remote"
-
-        async def _get_space_children_remote(
-            _self: SpaceHierarchyHandler, space_id: str, via: Iterable[str]
-        ) -> NoReturn:
-            self.fail(
-                f"Unexpected _get_space_children_remote({space_id!r}, {via!r}) call"
-            )
-            raise  # `fail` is missing type hints
-
-        with mock.patch(
-            "synapse.handlers.space_hierarchy.SpaceHierarchyHandler._get_space_children_remote",
-            new=_get_space_children_remote,
-        ):
-            descendants, inaccessible_room_ids = self.get_success(
-                self.handler.get_space_descendants(
-                    space_id, via=["remote"], enable_federation=False
-                )
-            )
-
-        self.assertEqual(descendants, [(space_id, ["remote"])])
-        self.assertEqual(inaccessible_room_ids, [space_id])
-
-    def test_cycle(self):
-        """Tests iteration over a cyclic space."""
-        # space_id
-        #  - subspace_id
-        #    - space_id
-        space_id = self._create_space()
-        subspace_id = self._create_space()
-        self._add_child(space_id, subspace_id)
-        self._add_child(subspace_id, space_id)
-
-        descendants, inaccessible_room_ids = self.get_success(
-            self.handler.get_space_descendants(space_id)
-        )
-
-        self.assertEqual(
-            descendants, [(space_id, []), (subspace_id, [self.hs.hostname])]
-        )
-        self.assertEqual(inaccessible_room_ids, [])
-
-    def test_duplicates(self):
-        """Tests iteration over a space with repeated rooms."""
-        # space_id
-        #  - subspace_id
-        #     - duplicate_room_1_id
-        #     - duplicate_room_2_id
-        #     - room_id
-        #  - duplicate_room_1_id
-        #  - duplicate_room_2_id
-        space_id = self._create_space()
-        subspace_id = self._create_space()
-        room_id = self._create_room()
-        duplicate_room_1_id = self._create_room()
-        duplicate_room_2_id = self._create_room()
-        self._add_child(space_id, subspace_id, order="1")
-        self._add_child(space_id, duplicate_room_1_id, order="2")
-        self._add_child(space_id, duplicate_room_2_id, order="3")
-        self._add_child(subspace_id, duplicate_room_1_id, order="1")
-        self._add_child(subspace_id, duplicate_room_2_id, order="2")
-        self._add_child(subspace_id, room_id, order="3")
-
-        descendants, inaccessible_room_ids = self.get_success(
-            self.handler.get_space_descendants(space_id)
-        )
-
-        self.assertEqual(
-            descendants,
-            [
-                (space_id, []),
-                (subspace_id, [self.hs.hostname]),
-                (room_id, [self.hs.hostname]),
-                (duplicate_room_1_id, [self.hs.hostname]),
-                (duplicate_room_2_id, [self.hs.hostname]),
-            ],
-        )
-        self.assertEqual(inaccessible_room_ids, [])

tests/rest/admin/test_space.py

@@ -1,399 +0,0 @@
-# Copyright 2021 The Matrix.org Foundation C.I.C.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from typing import (
-    Dict,
-    Iterable,
-    List,
-    Mapping,
-    NoReturn,
-    Optional,
-    Sequence,
-    Tuple,
-    Union,
-)
-from unittest import mock
-
-from typing_extensions import Literal
-
-from twisted.test.proto_helpers import MemoryReactor
-
-import synapse.rest.admin
-from synapse.api.constants import (
-    EventContentFields,
-    EventTypes,
-    JoinRules,
-    Membership,
-    RestrictedJoinRuleTypes,
-    RoomTypes,
-)
-from synapse.api.room_versions import RoomVersions
-from synapse.handlers.space_hierarchy import SpaceHierarchyHandler
-from synapse.rest.client import login, room
-from synapse.server import HomeServer
-from synapse.types import JsonDict
-from synapse.util import Clock
-
-from tests import unittest
-
-
-class RemoveSpaceMemberTestCase(unittest.HomeserverTestCase):
-    """Tests removal of a user from a space."""
-
-    servlets = [
-        synapse.rest.admin.register_servlets,
-        login.register_servlets,
-        room.register_servlets,
-    ]
-
-    def prepare(self, reactor: MemoryReactor, clock: Clock, hs: HomeServer):
-        self.store = hs.get_datastore()
-
-        # Create users
-        self.admin_user = self.register_user("admin", "pass", admin=True)
-        self.admin_user_tok = self.login("admin", "pass")
-        self.space_owner_user = self.register_user("space_owner", "pass")
-        self.space_owner_user_tok = self.login("space_owner", "pass")
-        self.target_user = self.register_user("user", "pass")
-        self.target_user_tok = self.login("user", "pass")
-
-        # Create a space hierarchy for testing:
-        #   space, invite-only
-        #    * subspace, restricted
-        self.space_id = self._create_space(JoinRules.INVITE)
-
-        # Make the target user a member of the space
-        self.helper.invite(
-            self.space_id,
-            src=self.space_owner_user,
-            targ=self.target_user,
-            tok=self.space_owner_user_tok,
-        )
-        self.helper.join(self.space_id, self.target_user, tok=self.target_user_tok)
-
-        self.subspace_id = self._create_space((JoinRules.RESTRICTED, self.space_id))
-        self._add_child(self.space_id, self.subspace_id)
-
-    def _add_child(
-        self, space_id: str, room_id: str, via: Optional[List[str]] = None
-    ) -> None:
-        """Adds a room to a space."""
-        if via is None:
-            via = [self.hs.hostname]
-
-        self.helper.send_state(
-            space_id,
-            event_type=EventTypes.SpaceChild,
-            body={"via": via},
-            tok=self.space_owner_user_tok,
-            state_key=room_id,
-        )
-
-    def _create_space(
-        self,
-        join_rules: Union[
-            Literal["public", "invite", "knock"],
-            Tuple[Literal["restricted"], str],
-        ],
-    ) -> str:
-        """Creates a space."""
-        return self._create_room(
-            join_rules,
-            extra_content={
-                "creation_content": {EventContentFields.ROOM_TYPE: RoomTypes.SPACE}
-            },
-        )
-
-    def _create_room(
-        self,
-        join_rules: Union[
-            Literal["public", "invite", "knock"],
-            Tuple[Literal["restricted"], str],
-        ],
-        extra_content: Optional[Dict] = None,
-    ) -> str:
-        """Creates a room."""
-        room_id = self.helper.create_room_as(
-            self.space_owner_user,
-            room_version=RoomVersions.V8.identifier,
-            tok=self.space_owner_user_tok,
-            extra_content=extra_content,
-        )
-
-        if isinstance(join_rules, str):
-            self.helper.send_state(
-                room_id,
-                event_type=EventTypes.JoinRules,
-                body={"join_rule": join_rules},
-                tok=self.space_owner_user_tok,
-            )
-        else:
-            _, space_id = join_rules
-            self.helper.send_state(
-                room_id,
-                event_type=EventTypes.JoinRules,
-                body={
-                    "join_rule": JoinRules.RESTRICTED,
-                    "allow": [
-                        {
-                            "type": RestrictedJoinRuleTypes.ROOM_MEMBERSHIP,
-                            "room_id": space_id,
-                            "via": [self.hs.hostname],
-                        }
-                    ],
-                },
-                tok=self.space_owner_user_tok,
-            )
-
-        return room_id
-
-    def _remove_from_space(
-        self,
-        user_id: str,
-        space_id: Optional[str] = None,
-        include_remote_spaces: Optional[bool] = None,
-    ) -> JsonDict:
-        """Removes the given user from the test space."""
-        if space_id is None:
-            space_id = self.space_id
-
-        content: Union[bytes, JsonDict] = b""
-        if include_remote_spaces is not None:
-            content = {"include_remote_spaces": include_remote_spaces}
-
-        url = f"/_synapse/admin/v1/rooms/{self.space_id}/hierarchy/members/{user_id}"
-        channel = self.make_request(
-            "DELETE",
-            url.encode("ascii"),
-            access_token=self.admin_user_tok,
-            content=content,
-        )
-
-        self.assertEqual(200, channel.code, channel.json_body)
-
-        return channel.json_body
-
-    def test_public_space(self) -> None:
-        """Tests that the user is removed from the space, even if public."""
-        self.helper.send_state(
-            self.space_id,
-            event_type=EventTypes.JoinRules,
-            body={"join_rule": JoinRules.PUBLIC},
-            tok=self.space_owner_user_tok,
-        )
-
-        response = self._remove_from_space(self.target_user)
-        self.assertEqual(
-            response,
-            {
-                "left_rooms": [self.space_id],
-                "inaccessible_rooms": [],
-                "failed_rooms": {},
-            },
-        )
-
-        membership, _ = self.get_success(
-            self.store.get_local_current_membership_for_user_in_room(
-                self.target_user, self.space_id
-            )
-        )
-        self.assertEqual(membership, Membership.LEAVE)
-
-    def test_public_room(self) -> None:
-        """Tests that the user is not removed from public rooms."""
-        public_room_id = self._create_room(JoinRules.PUBLIC)
-        self._add_child(self.subspace_id, public_room_id)
-
-        self.helper.join(public_room_id, self.target_user, tok=self.target_user_tok)
-
-        response = self._remove_from_space(self.target_user)
-        self.assertEqual(
-            response,
-            {
-                "left_rooms": [self.space_id],
-                "inaccessible_rooms": [],
-                "failed_rooms": {},
-            },
-        )
-
-        membership, _ = self.get_success(
-            self.store.get_local_current_membership_for_user_in_room(
-                self.target_user, public_room_id
-            )
-        )
-        self.assertEqual(membership, Membership.JOIN)
-
-    def test_invited(self) -> None:
-        """Tests that the user is made to decline invites to rooms in the space."""
-        invite_only_room_id = self._create_room(JoinRules.INVITE)
-        self._add_child(self.subspace_id, invite_only_room_id)
-
-        self.helper.invite(
-            invite_only_room_id,
-            src=self.space_owner_user,
-            targ=self.target_user,
-            tok=self.space_owner_user_tok,
-        )
-
-        response = self._remove_from_space(self.target_user)
-        self.assertEqual(
-            response,
-            {
-                "left_rooms": [self.space_id, invite_only_room_id],
-                "inaccessible_rooms": [],
-                "failed_rooms": {},
-            },
-        )
-
-        membership, _ = self.get_success(
-            self.store.get_local_current_membership_for_user_in_room(
-                self.target_user, invite_only_room_id
-            )
-        )
-        self.assertEqual(membership, Membership.LEAVE)
-
-    def test_invite_only_room(self) -> None:
-        """Tests that the user is made to leave invite-only rooms."""
-        invite_only_room_id = self._create_room(JoinRules.INVITE)
-        self._add_child(self.subspace_id, invite_only_room_id)
-
-        self.helper.invite(
-            invite_only_room_id,
-            src=self.space_owner_user,
-            targ=self.target_user,
-            tok=self.space_owner_user_tok,
-        )
-        self.helper.join(
-            invite_only_room_id, self.target_user, tok=self.target_user_tok
-        )
-
-        response = self._remove_from_space(self.target_user)
-        self.assertEqual(
-            response,
-            {
-                "left_rooms": [self.space_id, invite_only_room_id],
-                "inaccessible_rooms": [],
-                "failed_rooms": {},
-            },
-        )
-
-        membership, _ = self.get_success(
-            self.store.get_local_current_membership_for_user_in_room(
-                self.target_user, invite_only_room_id
-            )
-        )
-        self.assertEqual(membership, Membership.LEAVE)
-
-    def test_restricted_room(self) -> None:
-        """Tests that the user is made to leave restricted rooms."""
-        restricted_room_id = self._create_room((JoinRules.RESTRICTED, self.space_id))
-        self._add_child(self.subspace_id, restricted_room_id)
-        self.helper.join(restricted_room_id, self.target_user, tok=self.target_user_tok)
-
-        response = self._remove_from_space(self.target_user)
-        self.assertEqual(
-            response,
-            {
-                "left_rooms": [self.space_id, restricted_room_id],
-                "inaccessible_rooms": [],
-                "failed_rooms": {},
-            },
-        )
-
-        membership, _ = self.get_success(
-            self.store.get_local_current_membership_for_user_in_room(
-                self.target_user, restricted_room_id
-            )
-        )
-        self.assertEqual(membership, Membership.LEAVE)
-
-    def test_remote_space(self) -> None:
-        """Tests that the user is made to leave rooms in a remote space."""
-        remote_space_id = "!space:remote"
-        self._add_child(self.subspace_id, remote_space_id, via=["remote"])
-
-        restricted_room_id = self._create_room((JoinRules.RESTRICTED, self.space_id))
-        self.helper.join(restricted_room_id, self.target_user, tok=self.target_user_tok)
-
-        async def _get_space_children_remote(
-            _self: SpaceHierarchyHandler, space_id: str, via: Iterable[str]
-        ) -> Tuple[
-            Sequence[Tuple[str, Iterable[str]]], Mapping[str, Optional[JsonDict]]
-        ]:
-            self.assertEqual(space_id, remote_space_id)
-            self.assertEqual(via, ["remote"])
-
-            return [(restricted_room_id, [self.hs.hostname])], {}
-
-        with mock.patch(
-            "synapse.handlers.space_hierarchy.SpaceHierarchyHandler._get_space_children_remote",
-            new=_get_space_children_remote,
-        ):
-            response = self._remove_from_space(
-                self.target_user, space_id="!space:remote", include_remote_spaces=True
-            )
-        self.assertEqual(
-            response,
-            {
-                "left_rooms": [self.space_id, restricted_room_id],
-                "inaccessible_rooms": [remote_space_id],
-                "failed_rooms": {},
-            },
-        )
-
-        membership, _ = self.get_success(
-            self.store.get_local_current_membership_for_user_in_room(
-                self.target_user, restricted_room_id
-            )
-        )
-        self.assertEqual(membership, Membership.LEAVE)
-
-    def test_remote_spaces_excluded(self) -> None:
-        """Tests the exclusion of remote spaces."""
-        remote_space_id = "!space:remote"
-        self._add_child(self.subspace_id, remote_space_id, via=["remote"])
-
-        restricted_room_id = self._create_room((JoinRules.RESTRICTED, self.space_id))
-        self.helper.join(restricted_room_id, self.target_user, tok=self.target_user_tok)
-
-        async def _get_space_children_remote(
-            _self: SpaceHierarchyHandler, space_id: str, via: Iterable[str]
-        ) -> NoReturn:
-            self.fail(
-                f"Unexpected _get_space_children_remote({space_id!r}, {via!r}) call"
-            )
-            raise  # `fail` is missing type hints
-
-        with mock.patch(
-            "synapse.handlers.space_hierarchy.SpaceHierarchyHandler._get_space_children_remote",
-            new=_get_space_children_remote,
-        ):
-            response = self._remove_from_space(
-                self.target_user, space_id="!space:remote", include_remote_spaces=False
-            )
-        self.assertEqual(
-            response,
-            {
-                "left_rooms": [self.space_id],
-                "inaccessible_rooms": [remote_space_id],
-                "failed_rooms": {},
-            },
-        )
-
-        membership, _ = self.get_success(
-            self.store.get_local_current_membership_for_user_in_room(
-                self.target_user, restricted_room_id
-            )
-        )
-        self.assertEqual(membership, Membership.JOIN)

tests/rest/client/test_relations.py

@@ -574,11 +574,11 @@ class RelationsTestCase(unittest.HomeserverTestCase):
         assert_bundle(channel.json_body["event"]["unsigned"].get("m.relations"))
 
         # Request sync.
-        channel = self.make_request("GET", "/sync", access_token=self.user_token)
-        self.assertEquals(200, channel.code, channel.json_body)
-        room_timeline = channel.json_body["rooms"]["join"][self.room]["timeline"]
-        self.assertTrue(room_timeline["limited"])
-        _find_and_assert_event(room_timeline["events"])
+        # channel = self.make_request("GET", "/sync", access_token=self.user_token)
+        # self.assertEquals(200, channel.code, channel.json_body)
+        # room_timeline = channel.json_body["rooms"]["join"][self.room]["timeline"]
+        # self.assertTrue(room_timeline["limited"])
+        # _find_and_assert_event(room_timeline["events"])
 
         # Note that /relations is tested separately in test_aggregation_get_event_for_thread
         # since it needs different data configured.

tests/unittest.py

@@ -331,13 +331,16 @@ class HomeserverTestCase(TestCase):
             time.sleep(0.01)
 
     def wait_for_background_updates(self) -> None:
-        """Block until all background database updates have completed."""
-        store = self.hs.get_datastore()
+        """Block until all background database updates have completed.
+
+        Note that callers must ensure there's a store property created on the
+        testcase.
+        """
         while not self.get_success(
-            store.db_pool.updates.has_completed_background_updates()
+            self.store.db_pool.updates.has_completed_background_updates()
         ):
             self.get_success(
-                store.db_pool.updates.do_next_background_update(False), by=0.1
+                self.store.db_pool.updates.do_next_background_update(False), by=0.1
             )
 
     def make_homeserver(self, reactor, clock):