This takes down the CI time to build wheels from 50 minutes to <10
minutes.
**It also fixes macOS ARM builds, and includes more ARM builds in
general** (we were ignoring pypy and musl before). This doesn't cost
much for us to do this, reasons for not doing this is 1. space on PyPI
and 2. keeping them 'officially' supported?
This is the list of wheels this built (`+` are the ones added):
```diff
matrix_synapse-1.133.0-cp39-abi3-macosx_10_9_x86_64.whl
+ matrix_synapse-1.133.0-cp39-abi3-macosx_11_0_arm64.whl
matrix_synapse-1.133.0-cp39-abi3-manylinux_2_28_aarch64.whl
matrix_synapse-1.133.0-cp39-abi3-manylinux_2_28_x86_64.whl
+ matrix_synapse-1.133.0-cp39-abi3-musllinux_1_2_aarch64.whl
matrix_synapse-1.133.0-cp39-abi3-musllinux_1_2_x86_64.whl
matrix_synapse-1.133.0-pp310-pypy310_pp73-macosx_10_15_x86_64.whl
+ matrix_synapse-1.133.0-pp310-pypy310_pp73-macosx_11_0_arm64.whl
+ matrix_synapse-1.133.0-pp310-pypy310_pp73-manylinux_2_28_aarch64.whl
matrix_synapse-1.133.0-pp310-pypy310_pp73-manylinux_2_28_x86_64.whl
matrix_synapse-1.133.0-pp311-pypy311_pp73-macosx_10_15_x86_64.whl
+ matrix_synapse-1.133.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl
+ matrix_synapse-1.133.0-pp311-pypy311_pp73-manylinux_2_28_aarch64.whl
matrix_synapse-1.133.0-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl
```
And the numbers aaaaare 🥁
-
[before](https://github.com/element-hq/synapse/actions/runs/16072488018):
54 minutes
-
[after](https://github.com/element-hq/synapse/actions/runs/16004034949?pr=18618):
10 minutes
**Revert
[e43b0f9](e43b0f9bd1)
before merging**
This splits the building of docker images in 2 jobs, one for each
platform, using the native ARM runners for arm64.
The tricky part here is to get back a nice multi-arch manifest.
Previously, you'd do that by pushing each platform image in two distinct
tags, then referencing them in a multi-arch manifest. Nowadays, it's
possible to push images by their digest only, then creating the manifest
for those pushed digests separately
This is inspired by the Docker docs on how to distribute multi-platform
image builds:
https://docs.docker.com/build/ci/github-actions/multi-platform/#distribute-build-across-multiple-runners
`ghcr.io/element-hq/synapse:sha-c733dd6` is an example image that got
built by this workflow (there is a temporary sha-* tag on
workflow_dispatch runs to help trying out the workflow)
I also had to make sure we sign the manifests correctly:
```
$ cosign verify --certificate-oidc-issuer https://token.actions.githubusercontent.com --certificate-identity-regexp 'https://github.com/element-hq/synapse/.github/workflows/docker.yml@.*' ghcr.io/element-hq/synapse:sha-c733dd6
Verification for ghcr.io/element-hq/synapse:sha-c733dd6 --
The following checks were performed on each of these signatures:
- The cosign claims were validated
- Existence of the claims in the transparency log was verified offline
- The code-signing certificate was verified using trusted certificate authority certificates
```
And the numbers aaaaare 🥁
-
[before](https://github.com/element-hq/synapse/actions/runs/16118229296/job/45477093703):
30 minutes
-
[after](https://github.com/element-hq/synapse/actions/runs/16021743575):
4 minutes
---------
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Fixes https://github.com/element-hq/synapse/issues/18614
This upgrade CIBW to 3.0, which now builds using the manylinux_2_28
image, as the previous image is EOL and not supported by some of our
dependencies anymore.
This also updates the job to use the `ubuntu-24.04` base image instead
of `ubuntu-22.04`
Bumps
[actions/upload-artifact](https://github.com/actions/upload-artifact)
from 4.6.1 to 4.6.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/upload-artifact/releases">actions/upload-artifact's
releases</a>.</em></p>
<blockquote>
<h2>v4.6.2</h2>
<h2>What's Changed</h2>
<ul>
<li>Update to use artifact 2.3.2 package & prepare for new
upload-artifact release by <a
href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/upload-artifact/pull/685">actions/upload-artifact#685</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/upload-artifact/pull/685">actions/upload-artifact#685</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/upload-artifact/compare/v4...v4.6.2">https://github.com/actions/upload-artifact/compare/v4...v4.6.2</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="ea165f8d65"><code>ea165f8</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/upload-artifact/issues/685">#685</a>
from salmanmkc/salmanmkc/3-new-upload-artifacts-release</li>
<li><a
href="08396203c1"><code>0839620</code></a>
Prepare for new release of actions/upload-artifact with new toolkit
cache ver...</li>
<li>See full diff in <a
href="4cec3d8aa0...ea165f8d65">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps
[actions/download-artifact](https://github.com/actions/download-artifact)
from 4.2.0 to 4.2.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/download-artifact/releases">actions/download-artifact's
releases</a>.</em></p>
<blockquote>
<h2>v4.2.1</h2>
<h2>What's Changed</h2>
<ul>
<li>Add unit tests by <a
href="https://github.com/GhadimiR"><code>@GhadimiR</code></a> in <a
href="https://redirect.github.com/actions/download-artifact/pull/392">actions/download-artifact#392</a></li>
<li>Fix bug introduced in 4.2.0 by <a
href="https://github.com/GhadimiR"><code>@GhadimiR</code></a> in <a
href="https://redirect.github.com/actions/download-artifact/pull/391">actions/download-artifact#391</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/download-artifact/compare/v4.2.0...v4.2.1">https://github.com/actions/download-artifact/compare/v4.2.0...v4.2.1</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="95815c38cf"><code>95815c3</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/download-artifact/issues/391">#391</a>
from GhadimiR/main</li>
<li><a
href="278fca438a"><code>278fca4</code></a>
Move log statements</li>
<li><a
href="68909842a1"><code>6890984</code></a>
Merge branch 'main' into main</li>
<li><a
href="f9415c0ec3"><code>f9415c0</code></a>
Run unit tests in CI</li>
<li><a
href="76a6eb5cbc"><code>76a6eb5</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/download-artifact/issues/392">#392</a>
from GhadimiR/add_unit_tests</li>
<li><a
href="a2426d7c45"><code>a2426d7</code></a>
Merge branch 'main' into add_unit_tests</li>
<li><a
href="3ffa694f6f"><code>3ffa694</code></a>
lint</li>
<li><a
href="53f6aa5f93"><code>53f6aa5</code></a>
Add extra assertion to download single artifact test</li>
<li><a
href="b456700053"><code>b456700</code></a>
lint</li>
<li><a
href="9eab798a98"><code>9eab798</code></a>
Configure tsconfig</li>
<li>Additional commits viewable in <a
href="b14cf4c926...95815c38cf">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/cache](https://github.com/actions/cache) from 4.2.2 to
4.2.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/cache/releases">actions/cache's
releases</a>.</em></p>
<blockquote>
<h2>v4.2.3</h2>
<h2>What's Changed</h2>
<ul>
<li>Update to use <code>@actions/cache</code> 4.0.3 package &
prepare for new release by <a
href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1577">actions/cache#1577</a>
(SAS tokens for cache entries are now masked in debug logs)</li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/cache/pull/1577">actions/cache#1577</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/cache/compare/v4.2.2...v4.2.3">https://github.com/actions/cache/compare/v4.2.2...v4.2.3</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/actions/cache/blob/main/RELEASES.md">actions/cache's
changelog</a>.</em></p>
<blockquote>
<h1>Releases</h1>
<h3>4.2.3</h3>
<ul>
<li>Bump <code>@actions/cache</code> to v4.0.3 (obfuscates SAS token in
debug logs for cache entries)</li>
</ul>
<h3>4.2.2</h3>
<ul>
<li>Bump <code>@actions/cache</code> to v4.0.2</li>
</ul>
<h3>4.2.1</h3>
<ul>
<li>Bump <code>@actions/cache</code> to v4.0.1</li>
</ul>
<h3>4.2.0</h3>
<p>TLDR; The cache backend service has been rewritten from the ground up
for improved performance and reliability. <a
href="https://github.com/actions/cache">actions/cache</a> now integrates
with the new cache service (v2) APIs.</p>
<p>The new service will gradually roll out as of <strong>February 1st,
2025</strong>. The legacy service will also be sunset on the same date.
Changes in these release are <strong>fully backward
compatible</strong>.</p>
<p><strong>We are deprecating some versions of this action</strong>. We
recommend upgrading to version <code>v4</code> or <code>v3</code> as
soon as possible before <strong>February 1st, 2025.</strong> (Upgrade
instructions below).</p>
<p>If you are using pinned SHAs, please use the SHAs of versions
<code>v4.2.0</code> or <code>v3.4.0</code></p>
<p>If you do not upgrade, all workflow runs using any of the deprecated
<a href="https://github.com/actions/cache">actions/cache</a> will
fail.</p>
<p>Upgrading to the recommended versions will not break your
workflows.</p>
<h3>4.1.2</h3>
<ul>
<li>Add GitHub Enterprise Cloud instances hostname filters to inform API
endpoint choices - <a
href="https://redirect.github.com/actions/cache/pull/1474">#1474</a></li>
<li>Security fix: Bump braces from 3.0.2 to 3.0.3 - <a
href="https://redirect.github.com/actions/cache/pull/1475">#1475</a></li>
</ul>
<h3>4.1.1</h3>
<ul>
<li>Restore original behavior of <code>cache-hit</code> output - <a
href="https://redirect.github.com/actions/cache/pull/1467">#1467</a></li>
</ul>
<h3>4.1.0</h3>
<ul>
<li>Ensure <code>cache-hit</code> output is set when a cache is missed -
<a
href="https://redirect.github.com/actions/cache/pull/1404">#1404</a></li>
<li>Deprecate <code>save-always</code> input - <a
href="https://redirect.github.com/actions/cache/pull/1452">#1452</a></li>
</ul>
<h3>4.0.2</h3>
<ul>
<li>Fixed restore <code>fail-on-cache-miss</code> not working.</li>
</ul>
<h3>4.0.1</h3>
<ul>
<li>Updated <code>isGhes</code> check</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="5a3ec84eff"><code>5a3ec84</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/cache/issues/1577">#1577</a>
from salmanmkc/salmanmkc/4-test</li>
<li><a
href="7de21022a7"><code>7de2102</code></a>
Update releases.md</li>
<li><a
href="76d40dd347"><code>76d40dd</code></a>
Update to use the latest version of the cache package to obfuscate the
SAS</li>
<li><a
href="76dd5eb692"><code>76dd5eb</code></a>
update cache with main</li>
<li><a
href="8c80c27c5e"><code>8c80c27</code></a>
new package</li>
<li><a
href="45cfd0e7ff"><code>45cfd0e</code></a>
updates</li>
<li><a
href="edd449b9cf"><code>edd449b</code></a>
updated cache with latest changes</li>
<li><a
href="0576707e37"><code>0576707</code></a>
latest test before pr</li>
<li><a
href="3105dc9754"><code>3105dc9</code></a>
update</li>
<li><a
href="9450d42d15"><code>9450d42</code></a>
mask</li>
<li>Additional commits viewable in <a
href="d4323d4df1...5a3ec84eff">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
After the [recent supply chain attack](https://www.wiz.io/blog/new-github-action-supply-chain-attack-reviewdog-action-setup)
in `tj-actions/changed-files` and actions based on it, it's become clear
that relying on git tags to pin our dependencies is not enough (as tags
can simply be replaced). Therefore we need to switch to hashes.
Dependabot should continue to update these dependencies for us.
Best reviewed commit-by-commit. Though if CI passes, we're *probably*
fine.
We were pinned to an old version that had deprecation warnings.
In new versions of the action leaving off properties (i.e. `draft` and
`prerelease`) tells the action to not modify those properties of the
release.
This changes the release artefacts workflow to use `macos-12` runners
instead of `macos-11`, as the latter will be fully deprecated in a few
days.
This also updates `cibuildwheel` to a newer version, as it would not
'repair' the macOS wheels correctly
The difference is that now instead of outputting a macOS 11+ compatible
wheel, we output a macOS 12+ compatible one. This is fine, as macOS 11
is considered EOL since September 2023.
We can also expect that macOS 12 will be considered EOL in September
2024, as Apple usually supports the last 3 macOS version, and macOS 15
is scheduled to be released around that time.
Partially reverts #16796
This is causing errors of the form:
```
Error: Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run
```
for the debs and wheels stages.
There were breaking changes that weren't included in the dependabot
changelog (:/):
https://github.com/actions/upload-artifact#breaking-changes
<!--
Fixes: # <!-- -->
<!--
Supersedes: # <!-- -->
<!--
Follows: # <!-- -->
<!--
Part of: # <!-- -->
Base: `release-v1.100` <!-- git-stack-base-branch:release-v1.100 -->
<!--
This pull request is commit-by-commit review friendly. <!-- -->
<!--
This pull request is intended for commit-by-commit review. <!-- -->
Original commit schedule, with full messages:
<ol>
<li>
Downgrade the `upload-artifact` and `download-artifact` actions to v3
</li>
</ol>
---------
Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
pip was using a vendored setuptools that was incompatible with
Python 3.12. Upgrading cibuildwheels to a version with a newer
version of pip (and thus a newer version of setuptools) fixes
the issue.
