1
0

Add in-repo Complement tests (#19406)

This is useful so we can test Synapse
specific behaviors like our admin API.

(see docs in PR, `complement/README.md`)
```
COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh --in-repo
```

Complement calls these
["out-of-repo"](78c255edce/OUT-OF-REPO-TESTS.md)
tests but it's a bit of a misnomer once they're in your project. (just
depends on the perspective)

There has been [previous
desire](https://github.com/element-hq/synapse/pull/19021#discussion_r2453442191)
for this kind of thing but this is spawning from wanting to have some
tests for our purge history admin API
(https://github.com/element-hq/synapse-rust-apps/issues/430). There are
some Sytest tests ([`matrix-org/sytest` ->
`tests/48admin.pl#L91-L618`](1be04cce46/tests/48admin.pl (L91-L618)))
for this already but I'd much rather work in Complement instead of
Sytest. I'm wanting these tests to ensure that our new `event-cache`
rust app for Synapse Pro doesn't break these kind of erasure features
(https://github.com/element-hq/synapse-rust-apps/issues/366 and
https://github.com/element-hq/synapse-rust-apps/issues/153).

Interestingly, there is already [`matrix-org/complement` ->
`tests/csapi/admin_test.go`](78c255edce/tests/csapi/admin_test.go)
(added in https://github.com/matrix-org/complement/pull/322) in the
Complement repo iteslf that tests the
`/_synapse/admin/v1/send_server_notice` endpoint but it's a bit of an
interesting case as [Dendrite also supports this
endpoint](https://github.com/matrix-org/dendrite/pull/2180). I don't
think it's good practice to continually shove in more and more
Synapse-specific behavior into the Complement repo itself.

We already have success with other out-of-repo tests for projects like
the
[SBG](b76b05b53e/complement_tests),
[TI-Messenger
Proxy](c8fa87fecc/complement),
and our [Synapse Pro for small
hosts](c2ea7eabf3/complement).
This commit is contained in:
Eric Eastwood
2026-02-05 17:11:55 -06:00
committed by GitHub
parent f6105b73f0
commit 46d235cd52
12 changed files with 1144 additions and 21 deletions

View File

@@ -47,6 +47,9 @@ usage() {
cat >&2 <<EOF
Usage: $0 [-f] <go test arguments>...
Run the complement test suite on Synapse.
--in-repo
Whether to run the in-repo suite of Complement tests (see `./complement` in this project)
vs the Complement tests from the Complement repo.
-f, --fast
Skip rebuilding the docker images, and just use the most recent
@@ -82,6 +85,7 @@ main() {
# parse our arguments
skip_docker_build=""
skip_complement_run=""
use_in_repo_tests=""
while [ $# -ge 1 ]; do
arg=$1
case "$arg" in
@@ -89,6 +93,9 @@ main() {
usage
return 1
;;
"--in-repo")
use_in_repo_tests=1
;;
"-f"|"--fast")
skip_docker_build=1
;;
@@ -216,7 +223,10 @@ main() {
echo "Skipping Docker image build as requested."
fi
test_packages=(
# Default set of Complement tests to run from the Complement repo
#
# We pick and choose the specific MSC's that Synapse supports.
default_complement_test_packages=(
./tests/csapi
./tests
./tests/msc3874
@@ -233,7 +243,15 @@ main() {
# Export the list of test packages as a space-separated environment variable, so other
# scripts can use it.
export SYNAPSE_SUPPORTED_COMPLEMENT_TEST_PACKAGES="${test_packages[@]}"
export SYNAPSE_SUPPORTED_COMPLEMENT_TEST_PACKAGES="${default_complement_test_packages[@]}"
# Default set of Complement tests to run when using the in-repo test suite. Most
# likely, this should be all tests.
#
# Relative to the `./complement` repo in this project
default_in_repo_complement_test_packages=(
./tests/...
)
export COMPLEMENT_BASE_IMAGE=complement-synapse
if [ -n "$use_editable_synapse" ]; then
@@ -316,11 +334,26 @@ main() {
echo "Skipping Complement run as requested."
return 0
fi
# Print out the executed commands so it's more obvious what's happening at the end here.
# Things are slightly ambiguous with the in-repo vs Complement tests.
set -x
# Run the tests!
echo "Running Complement with ${test_args[@]} $@ ${test_packages[@]}"
cd "$COMPLEMENT_DIR"
go test "${test_args[@]}" "$@" "${test_packages[@]}"
if [ -n "$use_in_repo_tests" ]; then
# Run the suite of Complement tests in the `./complement` directory in this repo
cd "./complement"
go test "${test_args[@]}" "$@" "${default_in_repo_complement_test_packages[@]}"
else
# Run the tests (from the Complement repo)!
cd "$COMPLEMENT_DIR"
go test "${test_args[@]}" "$@" "${default_complement_test_packages[@]}"
fi
# We don't need to print out executed commands anymore
#
# This is just `set +x` without printing `+ set +x` to the console (via
# https://stackoverflow.com/questions/13195655/bash-set-x-without-it-being-printed/19226038#19226038)
{ set +x; } 2>/dev/null
}
main "$@"