Add in-repo Complement tests (#19406)
This is useful so we can test Synapse specific behaviors like our admin API. (see docs in PR, `complement/README.md`) ``` COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh --in-repo ``` Complement calls these ["out-of-repo"](78c255edce/OUT-OF-REPO-TESTS.md) tests but it's a bit of a misnomer once they're in your project. (just depends on the perspective) There has been [previous desire](https://github.com/element-hq/synapse/pull/19021#discussion_r2453442191) for this kind of thing but this is spawning from wanting to have some tests for our purge history admin API (https://github.com/element-hq/synapse-rust-apps/issues/430). There are some Sytest tests ([`matrix-org/sytest` -> `tests/48admin.pl#L91-L618`](1be04cce46/tests/48admin.pl (L91-L618))) for this already but I'd much rather work in Complement instead of Sytest. I'm wanting these tests to ensure that our new `event-cache` rust app for Synapse Pro doesn't break these kind of erasure features (https://github.com/element-hq/synapse-rust-apps/issues/366 and https://github.com/element-hq/synapse-rust-apps/issues/153). Interestingly, there is already [`matrix-org/complement` -> `tests/csapi/admin_test.go`](78c255edce/tests/csapi/admin_test.go) (added in https://github.com/matrix-org/complement/pull/322) in the Complement repo iteslf that tests the `/_synapse/admin/v1/send_server_notice` endpoint but it's a bit of an interesting case as [Dendrite also supports this endpoint](https://github.com/matrix-org/dendrite/pull/2180). I don't think it's good practice to continually shove in more and more Synapse-specific behavior into the Complement repo itself. We already have success with other out-of-repo tests for projects like the [SBG](b76b05b53e/complement_tests), [TI-Messenger Proxy](c8fa87fecc/complement), and our [Synapse Pro for small hosts](c2ea7eabf3/complement).
This commit is contained in:
@@ -47,6 +47,9 @@ usage() {
|
||||
cat >&2 <<EOF
|
||||
Usage: $0 [-f] <go test arguments>...
|
||||
Run the complement test suite on Synapse.
|
||||
--in-repo
|
||||
Whether to run the in-repo suite of Complement tests (see `./complement` in this project)
|
||||
vs the Complement tests from the Complement repo.
|
||||
|
||||
-f, --fast
|
||||
Skip rebuilding the docker images, and just use the most recent
|
||||
@@ -82,6 +85,7 @@ main() {
|
||||
# parse our arguments
|
||||
skip_docker_build=""
|
||||
skip_complement_run=""
|
||||
use_in_repo_tests=""
|
||||
while [ $# -ge 1 ]; do
|
||||
arg=$1
|
||||
case "$arg" in
|
||||
@@ -89,6 +93,9 @@ main() {
|
||||
usage
|
||||
return 1
|
||||
;;
|
||||
"--in-repo")
|
||||
use_in_repo_tests=1
|
||||
;;
|
||||
"-f"|"--fast")
|
||||
skip_docker_build=1
|
||||
;;
|
||||
@@ -216,7 +223,10 @@ main() {
|
||||
echo "Skipping Docker image build as requested."
|
||||
fi
|
||||
|
||||
test_packages=(
|
||||
# Default set of Complement tests to run from the Complement repo
|
||||
#
|
||||
# We pick and choose the specific MSC's that Synapse supports.
|
||||
default_complement_test_packages=(
|
||||
./tests/csapi
|
||||
./tests
|
||||
./tests/msc3874
|
||||
@@ -233,7 +243,15 @@ main() {
|
||||
|
||||
# Export the list of test packages as a space-separated environment variable, so other
|
||||
# scripts can use it.
|
||||
export SYNAPSE_SUPPORTED_COMPLEMENT_TEST_PACKAGES="${test_packages[@]}"
|
||||
export SYNAPSE_SUPPORTED_COMPLEMENT_TEST_PACKAGES="${default_complement_test_packages[@]}"
|
||||
|
||||
# Default set of Complement tests to run when using the in-repo test suite. Most
|
||||
# likely, this should be all tests.
|
||||
#
|
||||
# Relative to the `./complement` repo in this project
|
||||
default_in_repo_complement_test_packages=(
|
||||
./tests/...
|
||||
)
|
||||
|
||||
export COMPLEMENT_BASE_IMAGE=complement-synapse
|
||||
if [ -n "$use_editable_synapse" ]; then
|
||||
@@ -316,11 +334,26 @@ main() {
|
||||
echo "Skipping Complement run as requested."
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Print out the executed commands so it's more obvious what's happening at the end here.
|
||||
# Things are slightly ambiguous with the in-repo vs Complement tests.
|
||||
set -x
|
||||
|
||||
# Run the tests!
|
||||
echo "Running Complement with ${test_args[@]} $@ ${test_packages[@]}"
|
||||
cd "$COMPLEMENT_DIR"
|
||||
go test "${test_args[@]}" "$@" "${test_packages[@]}"
|
||||
if [ -n "$use_in_repo_tests" ]; then
|
||||
# Run the suite of Complement tests in the `./complement` directory in this repo
|
||||
cd "./complement"
|
||||
go test "${test_args[@]}" "$@" "${default_in_repo_complement_test_packages[@]}"
|
||||
else
|
||||
# Run the tests (from the Complement repo)!
|
||||
cd "$COMPLEMENT_DIR"
|
||||
go test "${test_args[@]}" "$@" "${default_complement_test_packages[@]}"
|
||||
fi
|
||||
|
||||
# We don't need to print out executed commands anymore
|
||||
#
|
||||
# This is just `set +x` without printing `+ set +x` to the console (via
|
||||
# https://stackoverflow.com/questions/13195655/bash-set-x-without-it-being-printed/19226038#19226038)
|
||||
{ set +x; } 2>/dev/null
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
Reference in New Issue
Block a user