ci: pin base images by digest resolved directly from Docker Hub

With the pkg-cache BuildKit mirror on, `FROM almalinux/10-base` was resolved
by tag through the cache, so a compromised cache could substitute the image.

Each job using the mirror now looks up the base tag's digest DIRECTLY at
registry-1.docker.io (.gitea/scripts/resolve-base-image.sh, HTTPS, HEAD on
the manifest, regex-validated) and builds FROM image@sha256:... via the
BASE_IMAGE build-arg, so the mirror can only serve content-addressed blobs.
If the lookup fails, the mirror is NOT used for that job and the tag is pulled
straight from Docker Hub. The digest is resolved every run (no manual bumps);
local builds default to the tag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

.gitea/scripts/resolve-base-image.sh

@@ -0,0 +1,35 @@
+#!/bin/bash
+## resolve-base-image.sh IMAGE:TAG — CI only (not copied into images).
+## Prints IMAGE@sha256:<digest> for a Docker Hub tag, looked up DIRECTLY at
+## registry-1.docker.io over HTTPS (never via the pkg-cache mirror), so the
+## build can say `FROM IMAGE@sha256:...` and the mirror can only serve
+## content-addressed blobs that BuildKit verifies. Exits non-zero (prints
+## nothing on stdout) if the lookup fails; the workflow then skips the mirror
+## and pulls the tag directly from Docker Hub. HEAD requests do not count
+## against Docker Hub pull rate limits.
+## Canonical copy: cloud-apache-container/.gitea/scripts/resolve-base-image.sh
+## (docs/pkg-cache.md there); other repos carry identical copies.
+set -euo pipefail
+ref="${1:?usage: resolve-base-image.sh IMAGE:TAG}"
+name_re='^[a-z0-9]+([._-][a-z0-9]+)*(/[a-z0-9]+([._-][a-z0-9]+)*)?$'
+tag_re='^[A-Za-z0-9_][A-Za-z0-9_.-]{0,127}$'
+name="${ref%:*}"; tag="${ref##*:}"
+[ "$name" != "$ref" ] || { echo "resolve-base-image: '$ref' has no :tag" >&2; exit 2; }
+[[ "$name" =~ $name_re ]] || { echo "resolve-base-image: bad image name '$name'" >&2; exit 2; }
+[[ "$tag" =~ $tag_re ]] || { echo "resolve-base-image: bad tag '$tag'" >&2; exit 2; }
+repo="$name"; [[ "$repo" == */* ]] || repo="library/$repo"   # official images
+
+curl_opts=(-fsS --proto '=https' --retry 3 --retry-all-errors --max-time 20)
+token=$(curl "${curl_opts[@]}" "https://auth.docker.io/token?service=registry.docker.io&scope=repository:${repo}:pull" |
+        python3 -c 'import json,sys; print(json.load(sys.stdin)["token"])')
+[ -n "$token" ] || { echo "resolve-base-image: no Docker Hub token for $repo" >&2; exit 1; }
+hdrs=$(curl "${curl_opts[@]}" -I \
+    -H "Authorization: Bearer ${token}" \
+    -H 'Accept: application/vnd.oci.image.index.v1+json' \
+    -H 'Accept: application/vnd.docker.distribution.manifest.list.v2+json' \
+    -H 'Accept: application/vnd.oci.image.manifest.v1+json' \
+    -H 'Accept: application/vnd.docker.distribution.manifest.v2+json' \
+    "https://registry-1.docker.io/v2/${repo}/manifests/${tag}")
+digest=$(printf '%s\n' "$hdrs" | tr -d '\r' | awk 'tolower($1)=="docker-content-digest:" {print $2}' | tail -n1)
+[[ "$digest" =~ ^sha256:[0-9a-f]{64}$ ]] || { echo "resolve-base-image: no valid digest for $ref (got '${digest}')" >&2; exit 1; }
+printf '%s@%s\n' "$name" "$digest"

.gitea/workflows/build-push.yaml

@@ -57,7 +57,8 @@ jobs:
             scripts/lsphp-abi-gate.sh \
             scripts/tune-mpm.sh \
             scripts/tests/lsphp-info-probe.test.sh \
-            scripts/tests/lsphp-abi-gate.test.sh
+            scripts/tests/lsphp-abi-gate.test.sh \
+            .gitea/scripts/resolve-base-image.sh
 
       - name: lsphp probe regression test
         run: ./scripts/tests/lsphp-info-probe.test.sh
@@ -85,6 +86,26 @@ jobs:
       - name: Checkout
         uses: actions/checkout@v4
 
+      # Base image pinned by DIGEST, looked up directly at Docker Hub (never via
+      # the pkg-cache mirror), so the mirror can only serve content-addressed
+      # blobs that BuildKit verifies. If the lookup fails this job does NOT use
+      # the mirror and pulls the tag straight from Docker Hub. docs/pkg-cache.md.
+      - name: Resolve base image digest (Docker Hub, direct)
+        id: base
+        env:
+          BASE_TAG: almalinux/10-base:latest
+        run: |
+          set -uo pipefail
+          if ref=$(.gitea/scripts/resolve-base-image.sh "$BASE_TAG") && [[ "$ref" =~ ^[a-z0-9][a-z0-9._/-]*@sha256:[0-9a-f]{64}$ ]]; then
+            echo "base-image: ${BASE_TAG} -> ${ref} (digest from registry-1.docker.io)"
+            echo "image=${ref}" >> "$GITHUB_OUTPUT"
+            echo "pinned=true" >> "$GITHUB_OUTPUT"
+          else
+            echo "::warning::base image ${BASE_TAG}: Docker Hub digest lookup failed - pulling the tag directly from Docker Hub, pkg-cache registry mirror NOT used"
+            echo "image=${BASE_TAG}" >> "$GITHUB_OUTPUT"
+            echo "pinned=false" >> "$GITHUB_OUTPUT"
+          fi
+
       # Internal package cache (CT130 pkg-cache, docs/pkg-cache.md). Used ONLY if
       # it answers over TLS with our pinned CA; otherwise (down, unreachable,
       # bad/expired cert) this job builds from the upstream repos and Docker Hub
@@ -92,11 +113,14 @@ jobs:
       # separately so either can fail back on its own.
       - name: Probe package cache
         id: pkgcache
+        # The registry mirror is only used for a digest-pinned base image.
+        env:
+          BASE_PINNED: ${{ steps.base.outputs.pinned }}
         run: |
           url=""; mirror=""
           if curl -fsS --max-time 5 --cacert keys/pkg-cache/ca.crt -o /dev/null https://192.168.1.40/; then url="https://192.168.1.40"; fi
-          if curl -fsS --max-time 5 --cacert keys/pkg-cache/ca.crt -o /dev/null https://192.168.1.40:5000/v2/; then mirror="192.168.1.40:5000"; fi
-          echo "pkg-cache: repos=${url:-<unreachable: upstream>} registry-mirror=${mirror:-<unreachable: Docker Hub>}"
+          if [ "$BASE_PINNED" = true ] && curl -fsS --max-time 5 --cacert keys/pkg-cache/ca.crt -o /dev/null https://192.168.1.40:5000/v2/; then mirror="192.168.1.40:5000"; fi
+          echo "pkg-cache: repos=${url:-<unreachable: upstream>} registry-mirror=${mirror:-<not used: Docker Hub direct>}"
           [ -n "$url" ] || echo "::warning::pkg-cache unreachable - building from upstream repositories"
           echo "url=${url}" >> "$GITHUB_OUTPUT"
           echo "mirror=${mirror}" >> "$GITHUB_OUTPUT"
@@ -165,6 +189,7 @@ jobs:
           build-args: |
             PHPVER=${{ matrix.phpver }}
             CACHE_BUST=${{ steps.bust.outputs.value }}
+            BASE_IMAGE=${{ steps.base.outputs.image }}
             PKG_CACHE=${{ steps.pkgcache.outputs.url }}
             EXPECT_PHP_MIN=${{ steps.phpmin.outputs.min }}
           tags: |
@@ -183,6 +208,26 @@ jobs:
       - name: Checkout
         uses: actions/checkout@v4
 
+      # Base image pinned by DIGEST, looked up directly at Docker Hub (never via
+      # the pkg-cache mirror), so the mirror can only serve content-addressed
+      # blobs that BuildKit verifies. If the lookup fails this job does NOT use
+      # the mirror and pulls the tag straight from Docker Hub. docs/pkg-cache.md.
+      - name: Resolve base image digest (Docker Hub, direct)
+        id: base
+        env:
+          BASE_TAG: almalinux/10-base:latest
+        run: |
+          set -uo pipefail
+          if ref=$(.gitea/scripts/resolve-base-image.sh "$BASE_TAG") && [[ "$ref" =~ ^[a-z0-9][a-z0-9._/-]*@sha256:[0-9a-f]{64}$ ]]; then
+            echo "base-image: ${BASE_TAG} -> ${ref} (digest from registry-1.docker.io)"
+            echo "image=${ref}" >> "$GITHUB_OUTPUT"
+            echo "pinned=true" >> "$GITHUB_OUTPUT"
+          else
+            echo "::warning::base image ${BASE_TAG}: Docker Hub digest lookup failed - pulling the tag directly from Docker Hub, pkg-cache registry mirror NOT used"
+            echo "image=${BASE_TAG}" >> "$GITHUB_OUTPUT"
+            echo "pinned=false" >> "$GITHUB_OUTPUT"
+          fi
+
       # Internal package cache (CT130 pkg-cache, docs/pkg-cache.md). Used ONLY if
       # it answers over TLS with our pinned CA; otherwise (down, unreachable,
       # bad/expired cert) this job builds from the upstream repos and Docker Hub
@@ -190,11 +235,14 @@ jobs:
       # separately so either can fail back on its own.
       - name: Probe package cache
         id: pkgcache
+        # The registry mirror is only used for a digest-pinned base image.
+        env:
+          BASE_PINNED: ${{ steps.base.outputs.pinned }}
         run: |
           url=""; mirror=""
           if curl -fsS --max-time 5 --cacert keys/pkg-cache/ca.crt -o /dev/null https://192.168.1.40/; then url="https://192.168.1.40"; fi
-          if curl -fsS --max-time 5 --cacert keys/pkg-cache/ca.crt -o /dev/null https://192.168.1.40:5000/v2/; then mirror="192.168.1.40:5000"; fi
-          echo "pkg-cache: repos=${url:-<unreachable: upstream>} registry-mirror=${mirror:-<unreachable: Docker Hub>}"
+          if [ "$BASE_PINNED" = true ] && curl -fsS --max-time 5 --cacert keys/pkg-cache/ca.crt -o /dev/null https://192.168.1.40:5000/v2/; then mirror="192.168.1.40:5000"; fi
+          echo "pkg-cache: repos=${url:-<unreachable: upstream>} registry-mirror=${mirror:-<not used: Docker Hub direct>}"
           [ -n "$url" ] || echo "::warning::pkg-cache unreachable - building from upstream repositories"
           echo "url=${url}" >> "$GITHUB_OUTPUT"
           echo "mirror=${mirror}" >> "$GITHUB_OUTPUT"
@@ -264,6 +312,7 @@ jobs:
           build-args: |
             PHPVER=${{ matrix.phpver }}
             CACHE_BUST=${{ steps.bust.outputs.value }}
+            BASE_IMAGE=${{ steps.base.outputs.image }}
             PKG_CACHE=${{ steps.pkgcache.outputs.url }}
             EXPECT_PHP_MIN=${{ steps.phpmin.outputs.min }}
           tags: |
@@ -478,6 +527,26 @@ jobs:
       - name: Checkout
         uses: actions/checkout@v4
 
+      # Base image pinned by DIGEST, looked up directly at Docker Hub (never via
+      # the pkg-cache mirror), so the mirror can only serve content-addressed
+      # blobs that BuildKit verifies. If the lookup fails this job does NOT use
+      # the mirror and pulls the tag straight from Docker Hub. docs/pkg-cache.md.
+      - name: Resolve base image digest (Docker Hub, direct)
+        id: base
+        env:
+          BASE_TAG: almalinux/10-base:latest
+        run: |
+          set -uo pipefail
+          if ref=$(.gitea/scripts/resolve-base-image.sh "$BASE_TAG") && [[ "$ref" =~ ^[a-z0-9][a-z0-9._/-]*@sha256:[0-9a-f]{64}$ ]]; then
+            echo "base-image: ${BASE_TAG} -> ${ref} (digest from registry-1.docker.io)"
+            echo "image=${ref}" >> "$GITHUB_OUTPUT"
+            echo "pinned=true" >> "$GITHUB_OUTPUT"
+          else
+            echo "::warning::base image ${BASE_TAG}: Docker Hub digest lookup failed - pulling the tag directly from Docker Hub, pkg-cache registry mirror NOT used"
+            echo "image=${BASE_TAG}" >> "$GITHUB_OUTPUT"
+            echo "pinned=false" >> "$GITHUB_OUTPUT"
+          fi
+
       # Internal package cache (CT130 pkg-cache, docs/pkg-cache.md). Used ONLY if
       # it answers over TLS with our pinned CA; otherwise (down, unreachable,
       # bad/expired cert) this job builds from the upstream repos and Docker Hub
@@ -485,11 +554,14 @@ jobs:
       # separately so either can fail back on its own.
       - name: Probe package cache
         id: pkgcache
+        # The registry mirror is only used for a digest-pinned base image.
+        env:
+          BASE_PINNED: ${{ steps.base.outputs.pinned }}
         run: |
           url=""; mirror=""
           if curl -fsS --max-time 5 --cacert keys/pkg-cache/ca.crt -o /dev/null https://192.168.1.40/; then url="https://192.168.1.40"; fi
-          if curl -fsS --max-time 5 --cacert keys/pkg-cache/ca.crt -o /dev/null https://192.168.1.40:5000/v2/; then mirror="192.168.1.40:5000"; fi
-          echo "pkg-cache: repos=${url:-<unreachable: upstream>} registry-mirror=${mirror:-<unreachable: Docker Hub>}"
+          if [ "$BASE_PINNED" = true ] && curl -fsS --max-time 5 --cacert keys/pkg-cache/ca.crt -o /dev/null https://192.168.1.40:5000/v2/; then mirror="192.168.1.40:5000"; fi
+          echo "pkg-cache: repos=${url:-<unreachable: upstream>} registry-mirror=${mirror:-<not used: Docker Hub direct>}"
           [ -n "$url" ] || echo "::warning::pkg-cache unreachable - building from upstream repositories"
           echo "url=${url}" >> "$GITHUB_OUTPUT"
           echo "mirror=${mirror}" >> "$GITHUB_OUTPUT"
@@ -528,16 +600,41 @@ jobs:
           platforms: linux/amd64
           push: true
           build-args: |
+            BASE_IMAGE=${{ steps.base.outputs.image }}
             PKG_CACHE=${{ steps.pkgcache.outputs.url }}
           tags: |
             repo.anhonesthost.net/cloud-hosting-platform/shared-httpd:latest
 
   Build-Shared-OLS:
     runs-on: ubuntu-latest
+    # One place for the OLS base tag: digest lookup and build-args both read it.
+    env:
+      OLS_VERSION: "1.8.4"
+      PHPVER: "83"
     steps:
       - name: Checkout
         uses: actions/checkout@v4
 
+      # Base image pinned by DIGEST, looked up directly at Docker Hub (never via
+      # the pkg-cache mirror), so the mirror can only serve content-addressed
+      # blobs that BuildKit verifies. If the lookup fails this job does NOT use
+      # the mirror and pulls the tag straight from Docker Hub. docs/pkg-cache.md.
+      - name: Resolve base image digest (Docker Hub, direct)
+        id: base
+        env:
+          BASE_TAG: litespeedtech/openlitespeed:${{ env.OLS_VERSION }}-lsphp${{ env.PHPVER }}
+        run: |
+          set -uo pipefail
+          if ref=$(.gitea/scripts/resolve-base-image.sh "$BASE_TAG") && [[ "$ref" =~ ^[a-z0-9][a-z0-9._/-]*@sha256:[0-9a-f]{64}$ ]]; then
+            echo "base-image: ${BASE_TAG} -> ${ref} (digest from registry-1.docker.io)"
+            echo "image=${ref}" >> "$GITHUB_OUTPUT"
+            echo "pinned=true" >> "$GITHUB_OUTPUT"
+          else
+            echo "::warning::base image ${BASE_TAG}: Docker Hub digest lookup failed - pulling the tag directly from Docker Hub, pkg-cache registry mirror NOT used"
+            echo "image=${BASE_TAG}" >> "$GITHUB_OUTPUT"
+            echo "pinned=false" >> "$GITHUB_OUTPUT"
+          fi
+
       # Internal package cache (CT130 pkg-cache, docs/pkg-cache.md). Used ONLY if
       # it answers over TLS with our pinned CA; otherwise (down, unreachable,
       # bad/expired cert) this job builds from the upstream repos and Docker Hub
@@ -545,11 +642,14 @@ jobs:
       # separately so either can fail back on its own.
       - name: Probe package cache
         id: pkgcache
+        # The registry mirror is only used for a digest-pinned base image.
+        env:
+          BASE_PINNED: ${{ steps.base.outputs.pinned }}
         run: |
           url=""; mirror=""
           if curl -fsS --max-time 5 --cacert keys/pkg-cache/ca.crt -o /dev/null https://192.168.1.40/; then url="https://192.168.1.40"; fi
-          if curl -fsS --max-time 5 --cacert keys/pkg-cache/ca.crt -o /dev/null https://192.168.1.40:5000/v2/; then mirror="192.168.1.40:5000"; fi
-          echo "pkg-cache: repos=${url:-<unreachable: upstream>} registry-mirror=${mirror:-<unreachable: Docker Hub>}"
+          if [ "$BASE_PINNED" = true ] && curl -fsS --max-time 5 --cacert keys/pkg-cache/ca.crt -o /dev/null https://192.168.1.40:5000/v2/; then mirror="192.168.1.40:5000"; fi
+          echo "pkg-cache: repos=${url:-<unreachable: upstream>} registry-mirror=${mirror:-<not used: Docker Hub direct>}"
           [ -n "$url" ] || echo "::warning::pkg-cache unreachable - building from upstream repositories"
           echo "url=${url}" >> "$GITHUB_OUTPUT"
           echo "mirror=${mirror}" >> "$GITHUB_OUTPUT"
@@ -590,8 +690,9 @@ jobs:
           # Single image (runs no PHP). PHPVER just selects the OLS base tag;
           # pinned to 83 / OLS 1.8.4 to match the rest of the litespeed family.
           build-args: |
-            PHPVER=83
-            OLS_VERSION=1.8.4
+            PHPVER=${{ env.PHPVER }}
+            OLS_VERSION=${{ env.OLS_VERSION }}
+            BASE_IMAGE=${{ steps.base.outputs.image }}
             PKG_CACHE=${{ steps.pkgcache.outputs.url }}
           tags: |
             repo.anhonesthost.net/cloud-hosting-platform/shared-ols:latest

Dockerfile

@@ -1,4 +1,9 @@
-FROM almalinux/10-base
+# Base image. CI passes BASE_IMAGE=almalinux/10-base@sha256:<digest>, resolved
+# DIRECTLY from Docker Hub in the same run (.gitea/scripts/resolve-base-image.sh),
+# so the pkg-cache registry mirror can only serve content-addressed blobs.
+# Local builds use the tag. docs/pkg-cache.md "Base-image digest pinning".
+ARG BASE_IMAGE=almalinux/10-base
+FROM ${BASE_IMAGE}
 ARG PHPVER=83
 # Per-CI-run value: the first RUN below (dnf update) and everything after it can
 # never be a layer-cache hit, so every build gets the repos' current packages.

Dockerfile.fpm

@@ -1,4 +1,9 @@
-FROM almalinux/10-base
+# Base image. CI passes BASE_IMAGE=almalinux/10-base@sha256:<digest>, resolved
+# DIRECTLY from Docker Hub in the same run (.gitea/scripts/resolve-base-image.sh),
+# so the pkg-cache registry mirror can only serve content-addressed blobs.
+# Local builds use the tag. docs/pkg-cache.md "Base-image digest pinning".
+ARG BASE_IMAGE=almalinux/10-base
+FROM ${BASE_IMAGE}
 ARG PHPVER=83
 # Per-CI-run value: the first RUN below (dnf update) and everything after it can
 # never be a layer-cache hit, so every build gets the repos' current packages.

Dockerfile.shared-httpd

@@ -1,4 +1,9 @@
-FROM almalinux/10-base
+# Base image. CI passes BASE_IMAGE=almalinux/10-base@sha256:<digest>, resolved
+# DIRECTLY from Docker Hub in the same run (.gitea/scripts/resolve-base-image.sh),
+# so the pkg-cache registry mirror can only serve content-addressed blobs.
+# Local builds use the tag. docs/pkg-cache.md "Base-image digest pinning".
+ARG BASE_IMAGE=almalinux/10-base
+FROM ${BASE_IMAGE}
 
 # OPTIONAL internal package cache, e.g. --build-arg PKG_CACHE=https://192.168.1.40
 # (CT130 pkg-cache, https only). Unset = upstream repositories. Release RPMs are

Dockerfile.shared-ols

@@ -17,7 +17,12 @@
 
 ARG OLS_VERSION=1.8.4
 ARG PHPVER=83
-FROM litespeedtech/openlitespeed:${OLS_VERSION}-lsphp${PHPVER}
+## CI passes BASE_IMAGE=litespeedtech/openlitespeed@sha256:<digest> of the
+## ${OLS_VERSION}-lsphp${PHPVER} tag, resolved DIRECTLY from Docker Hub in the same
+## run (.gitea/scripts/resolve-base-image.sh), so the pkg-cache registry mirror can
+## only serve content-addressed blobs. Local builds use the tag.
+ARG BASE_IMAGE=litespeedtech/openlitespeed:${OLS_VERSION}-lsphp${PHPVER}
+FROM ${BASE_IMAGE}
 
 ## OPTIONAL internal package cache, e.g. --build-arg PKG_CACHE=https://192.168.1.40
 ## (CT130 pkg-cache, https only). Unset = upstream repositories. Release RPMs are

docs/pkg-cache.md

@@ -149,6 +149,36 @@ Remaining risks:
    # /etc/docker/daemon.json:  { "registry-mirrors": ["https://192.168.1.40:5000"] }   then restart docker
    ```
 
+## Base-image digest pinning (2026-10-22)
+
+Gap closed (security review): with the BuildKit mirror on, `FROM almalinux/10-base`
+was resolved **by tag through the cache**, so a compromised cache could hand out a
+different image for that tag.
+
+Now every job that uses the mirror first runs **"Resolve base image digest (Docker
+Hub, direct)"**:
+- `.gitea/scripts/resolve-base-image.sh IMAGE:TAG` gets an anonymous pull token from
+  `auth.docker.io` and sends a `HEAD /v2/<repo>/manifests/<tag>` to
+  `registry-1.docker.io` over HTTPS (never the mirror; HEADs don't count against the
+  pull rate limit). It prints `IMAGE@sha256:<index digest>`. Input and output are
+  regex-validated; values reach `run:` only via `env:`.
+- The job passes `BASE_IMAGE=<that>` as a build arg; the Dockerfiles say
+  `ARG BASE_IMAGE=<tag>` / `FROM ${BASE_IMAGE}`. The mirror can then only serve
+  content-addressed manifests/blobs, which BuildKit verifies against the digest.
+- **Lookup fails:** `BASE_IMAGE` falls back to the tag, the probe step does **not**
+  enable the registry mirror (`registry-mirror=<not used: Docker Hub direct>`), and
+  the tag is pulled straight from Docker Hub, as before the cache existed. The
+  package-repo cache (`PKG_CACHE`) is unaffected, since its content is GPG-checked.
+- Freshness is automatic: the digest is looked up on every run, nothing is
+  hard-coded. Local builds without the arg use the tag.
+- Log lines to look for: `base-image: almalinux/10-base:latest -> almalinux/10-base@sha256:...`
+  and `pkg-cache: repos=https://192.168.1.40 registry-mirror=192.168.1.40:5000`.
+
+Applied to: Build-and-Push, Build-FPM-Images, Build-Shared-httpd, Build-Shared-OLS
+(`litespeedtech/openlitespeed:${OLS_VERSION}-lsphp${PHPVER}`) here, and cloud-node-container.
+The script is canonical here (`.gitea/scripts/resolve-base-image.sh`); other repos
+carry byte-identical copies.
+
 ## WHP hosts (`scripts/host/pkgcache-host-repos.sh`)
 
 `pkgcache-host-repos.sh apply https://192.168.1.40` does the following: