Run 5156 failed copying python:3.12-slim / golang:1.25 into Gitea: `PUT .../manifests/<tag>-staging: 404`, leaving a stray single-manifest <tag>-staging version. Reproduced on real Gitea with scratch packages: buildx <= 0.20.x (runner distro buildx is 0.15.1) pushes the children of a copied index to the TAG path; Gitea only records manifests PUT by digest, so the index PUT is rejected (MANIFEST_BLOB_UNKNOWN -> 404). buildx >= 0.24 pushes children by digest and works; Gitea then preserves the index digest, so the digest-equality assertion stays. - workflow: install a sha256-verified pinned buildx (v0.28.0) instead of setup-buildx-action (imagetools needs no builder) and pass it as $BUILDX - mirror-index-copy.sh: BUILDX/MIN_BUILDX guard refuses buildx < 0.24 before any registry traffic; digest computed as sha256 of the raw index bytes (buildx 0.28 ignores `inspect --format` here) - test: attestation-children index and old-buildx guard cases Merge with [skip ci] in the merge title: any push to main triggers the HMB image build. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0185ukXjXyosSkunDsdWz3mb
110 lines
5.6 KiB
YAML
110 lines
5.6 KiB
YAML
name: Mirror base images
|
|
run-name: weekly base-image mirror
|
|
|
|
# Copies each declared base image (the whole multi-arch INDEX, not one
|
|
# platform) from upstream to the in-house registry, so any of our images that FROM these don't depend on docker.io's
|
|
# Cloudflare R2 blob storage being reachable. The 2026-05-12 Cloudflare
|
|
# incident motivated this for python:3.12-slim and again for golang:1.25
|
|
# when the coraza-spoa build hit the same blob-fetch failure.
|
|
#
|
|
# Adding a new mirror = add one entry to the matrix below. The destination
|
|
# tag is always cloud-hosting-platform/<image>:<tag>, matching upstream.
|
|
|
|
on:
|
|
schedule:
|
|
# Mondays 06:00 UTC — outside customer peak hours and well before the
|
|
# typical Tuesday/Thursday push cycles. workflow_dispatch lets us trigger
|
|
# manually from the Gitea UI when upstream publishes patches.
|
|
- cron: '0 6 * * 1'
|
|
workflow_dispatch:
|
|
|
|
jobs:
|
|
Mirror-Base:
|
|
runs-on: ubuntu-latest
|
|
strategy:
|
|
# fail-fast=false so one image's upstream being down doesn't block the
|
|
# others from refreshing.
|
|
fail-fast: false
|
|
matrix:
|
|
# hub: the Docker Hub IMAGE:TAG whose digest is resolved DIRECTLY at
|
|
# registry-1.docker.io (resolve-base-image.sh expects no docker.io/
|
|
# prefix; official images resolve as library/<image>).
|
|
image:
|
|
- { hub: 'python:3.12-slim', dst_path: 'cloud-hosting-platform/python', tag: '3.12-slim' }
|
|
- { hub: 'golang:1.25', dst_path: 'cloud-hosting-platform/golang', tag: '1.25' }
|
|
|
|
steps:
|
|
# Only for .gitea/scripts/resolve-base-image.sh.
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
|
|
# Supply chain: ask Docker Hub itself (over HTTPS, never via a mirror) which
|
|
# digest the tag points at right now, then pull exactly that digest. What
|
|
# lands in the in-house registry is therefore content-addressed to the
|
|
# digest Docker Hub reported - not whatever a `docker pull TAG` on the
|
|
# runner happened to resolve through any configured registry mirror.
|
|
# If the lookup fails the job FAILS and last week's mirror stays in place.
|
|
- name: Resolve ${{ matrix.image.hub }} digest (Docker Hub, direct)
|
|
id: base
|
|
env:
|
|
HUB_TAG: ${{ matrix.image.hub }}
|
|
run: |
|
|
set -euo pipefail
|
|
ref=$(.gitea/scripts/resolve-base-image.sh "$HUB_TAG")
|
|
[[ "$ref" =~ ^[a-z0-9][a-z0-9._/-]*@sha256:[0-9a-f]{64}$ ]] || { echo "::error::bad digest reference for ${HUB_TAG}"; exit 1; }
|
|
echo "base-image: ${HUB_TAG} -> ${ref} (digest from registry-1.docker.io)"
|
|
echo "ref=${ref}" >> "$GITHUB_OUTPUT"
|
|
|
|
# imagetools needs a NEW buildx. The runner's distro buildx is 0.15.1, which
|
|
# pushes the children of a copied index to the TAG path: Gitea answers the
|
|
# index PUT with 404 and leaves a stray single-manifest <tag>-staging
|
|
# version (run 5156). buildx >= 0.24 pushes children by digest and works
|
|
# (measured on repo.anhonesthost.net). Pin one, verified by sha256
|
|
# (linux-amd64 binary of the v0.28.0 release); mirror-index-copy.sh also
|
|
# refuses to run on a buildx older than 0.24.
|
|
- name: Install pinned buildx
|
|
env:
|
|
BUILDX_VERSION: v0.28.0
|
|
BUILDX_SHA256: 696bc104bac3bb708eff1af3f8bbc09fda0fd88f5757c1f9b404a35117889224
|
|
run: |
|
|
set -euo pipefail
|
|
dir="${RUNNER_TEMP:-/tmp}/pinned-buildx"; mkdir -p "$dir"
|
|
curl -fsSL --retry 3 --max-time 120 -o "$dir/docker-buildx" \
|
|
"https://github.com/docker/buildx/releases/download/${BUILDX_VERSION}/buildx-${BUILDX_VERSION}.linux-amd64"
|
|
echo "${BUILDX_SHA256} $dir/docker-buildx" | sha256sum -c -
|
|
chmod +x "$dir/docker-buildx"
|
|
"$dir/docker-buildx" version
|
|
echo "BUILDX=$dir/docker-buildx" >> "$GITHUB_ENV"
|
|
|
|
# The registry token is written ONLY by this trusted x86 runner step
|
|
# (unchanged from before; no new secret exposure).
|
|
- name: Login to in-house registry
|
|
uses: docker/login-action@v3
|
|
with:
|
|
registry: repo.anhonesthost.net
|
|
username: ${{ secrets.CI_USER }}
|
|
password: ${{ secrets.CI_TOKEN }}
|
|
|
|
# Registry-to-registry copy of the WHOLE upstream index (all platforms,
|
|
# incl. linux/arm64; $BUILDX = the pinned buildx from the step above). The old pull/tag/push re-published only the runner's
|
|
# amd64 manifest. mirror-index-copy.sh stages to <tag>-staging, asserts
|
|
# amd64+arm64 present and index digest == the Docker-Hub-reported digest,
|
|
# and only then moves the real tag, so a bad copy never replaces a good
|
|
# multi-arch mirror. Logic + tests: .gitea/scripts/mirror-index-copy.sh,
|
|
# .gitea/scripts/test-mirror-index-copy.sh.
|
|
- name: Copy index by digest, assert platforms + digest ${{ matrix.image.hub }}
|
|
# Values reach the script via env, never via ${{ }} inside run:.
|
|
env:
|
|
REF: ${{ steps.base.outputs.ref }}
|
|
DST_PATH: ${{ matrix.image.dst_path }}
|
|
DST_TAG: ${{ matrix.image.tag }}
|
|
run: |
|
|
set -euo pipefail
|
|
[[ "$REF" =~ ^[a-z0-9][a-z0-9._/-]*@sha256:[0-9a-f]{64}$ ]] || { echo "::error::bad digest reference '${REF}'"; exit 1; }
|
|
[[ "$DST_PATH" =~ ^[a-z0-9][a-z0-9._/-]*$ && "$DST_TAG" =~ ^[A-Za-z0-9_][A-Za-z0-9_.-]{0,127}$ ]] || { echo "::error::bad destination"; exit 1; }
|
|
name="${REF%@*}"; digest="${REF#*@}"
|
|
[[ "$name" == */* ]] || name="library/${name}" # official images
|
|
.gitea/scripts/mirror-index-copy.sh \
|
|
"docker.io/${name}@${digest}" \
|
|
"repo.anhonesthost.net/${DST_PATH}:${DST_TAG}"
|