fix(pkg-cache): HTTPS-only cache, GPG-verified release RPMs, strict restore
Security review follow-up for the opt-in PKG_CACHE path: - PKG_CACHE must be https:// (http is refused). The cache's certificate comes from a name-constrained internal CA (keys/pkg-cache/ca.crt, valid only for 192.168.1.40 / pkg-cache.*). The CA is trusted only by the rewritten repos (dnf sslcacert bundle, apt per-host CaInfo) and only for the duration of the RUN. - epel-release / remi-release are no longer installed straight from a URL. release-install downloads them over HTTPS, checks the package name, imports the signing keys committed in keys/pkg-cache/ and requires `rpm -K` "signatures OK", then installs with localpkg_gpgcheck=1. This applies with and without the cache. - Rewritten repos get gpgcheck=1. Remi and AlmaLinux also get repo_gpgcheck=1, since both sign repomd.xml. EPEL does not publish a repomd signature, so it relies on end-to-end HTTPS. - restore also deletes files it created and any new repo/apt file that still references the cache, then fails the build if a reference remains under /etc/yum.repos.d, /etc/dnf or /etc/apt. - docs: trust model, replay risk, proposed openssl/httpd/ImageMagick minimum-version gates, TLS registry mirror (BuildKit ca= / dockerd certs.d, no insecure-registries). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
shadowdaoandClaude Opus 5.5 committed
1 parent
b2bec5b3ee
commit
18f65c6d84
10 files changed
+411
-134
No files matched your search
+141
-54
@@ -1,63 +1,140 @@
|
||||
#!/bin/bash
|
||||
## pkg-cache-repos.sh — BUILD-TIME ONLY, OPT-IN. Points dnf/apt at the internal
|
||||
## package cache (nginx proxy_cache, CT130 "pkg-cache" on the Proxmox cluster)
|
||||
## when PKG_CACHE is set; a no-op otherwise, so builds behave exactly as before
|
||||
## without it.
|
||||
## when PKG_CACHE is set; without it the repo files are left alone and builds
|
||||
## use the upstream repositories as before.
|
||||
##
|
||||
## PKG_CACHE=http://192.168.1.40 pkg-cache-repos.sh rpm # AlmaLinux/EPEL/Remi
|
||||
## PKG_CACHE=http://192.168.1.40 pkg-cache-repos.sh apt # Ubuntu + LiteSpeed
|
||||
## pkg-cache-repos.sh release-urls # prints EPEL+Remi release RPM URLs
|
||||
## pkg-cache-repos.sh restore # put the original repo files back
|
||||
## PKG_CACHE=https://192.168.1.40 pkg-cache-repos.sh rpm # AlmaLinux/EPEL/Remi
|
||||
## PKG_CACHE=https://192.168.1.40 pkg-cache-repos.sh apt # Ubuntu + LiteSpeed
|
||||
## pkg-cache-repos.sh release-install epel [remi] # GPG-verified release RPMs
|
||||
## pkg-cache-repos.sh restore # undo everything above
|
||||
##
|
||||
## Every RUN that rewrites must `restore` before it ends, so no shipped layer
|
||||
## carries the cache's LAN address in /etc/yum.repos.d or /etc/apt (images run on
|
||||
## hosts that cannot reach it). restore is a no-op when nothing was rewritten.
|
||||
## Every RUN that rewrites must `restore` before it ends: it puts rewritten files
|
||||
## back, deletes files this script created, deletes any NEW repo/apt file that
|
||||
## still references the cache, then FAILS if any reference is left — so no shipped
|
||||
## layer carries the cache's LAN address (images run on hosts that can't reach it).
|
||||
##
|
||||
## Why: on 2026-10-21 (PHP security release) builds failed on Remi mirrors lagging
|
||||
## hours behind origin, Remi's origin timing out over HTTPS, and an EPEL mirror
|
||||
## with a broken certificate. The cache fetches from the ORIGINS over HTTPS (with
|
||||
## origin/mirror fallbacks and stale-on-error), keeps metadata only ~3 minutes,
|
||||
## and keeps immutable .rpm/.deb files long-term.
|
||||
## origin/mirror fallbacks and stale-on-error), keeps metadata ~3 minutes and
|
||||
## immutable .rpm/.deb files long-term.
|
||||
##
|
||||
## Integrity is unchanged: dnf/apt still verify package GPG signatures (and apt
|
||||
## the signed InRelease/Release.gpg); the build's php.net version gates
|
||||
## (EXPECT_PHP_MIN / EXPECT_PHP_VERSION) still fail a build that gets old PHP.
|
||||
## Integrity (security review 2026-10-21):
|
||||
## * PKG_CACHE must be https://. The cache's certificate is issued by the
|
||||
## name-constrained "pkg-cache internal CA" (keys/pkg-cache/ca.crt, valid only
|
||||
## for 192.168.1.40 / pkg-cache.*). That CA is trusted ONLY by the rewritten
|
||||
## repos (dnf sslcacert= bundle, apt per-host CaInfo) and only during the RUN —
|
||||
## never added to the system trust store.
|
||||
## * gpgcheck=1 is forced on every rewritten repo; repo_gpgcheck=1 on Remi and
|
||||
## AlmaLinux, which sign repomd.xml (measured: repomd.xml.asc 200). EPEL
|
||||
## publishes no repomd signature (404), so its metadata relies on HTTPS end
|
||||
## to end (client->cache->dl.fedoraproject.org/mirrors, all verified).
|
||||
## * release-install downloads epel-release / remi-release (from the cache, or
|
||||
## from the HTTPS origin without PKG_CACHE), checks the package name, imports
|
||||
## the signing keys committed in keys/pkg-cache/ and requires
|
||||
## `rpm -K` "signatures OK" before installing with localpkg_gpgcheck=1.
|
||||
## * The php.net version gates (EXPECT_PHP_MIN / EXPECT_PHP_VERSION) still fail
|
||||
## a build that gets old PHP (replay of old-but-signed metadata).
|
||||
##
|
||||
## rpm: each rewritten baseurl keeps the upstream origin as a SECOND url, so if
|
||||
## the cache dies mid-build dnf fails over to the origin by itself.
|
||||
## apt: apt has no per-source failover; CI only sets PKG_CACHE after probing the
|
||||
## cache, and a build without PKG_CACHE uses the upstream sources.
|
||||
## cache (with the CA), and a build without PKG_CACHE uses upstream sources.
|
||||
set -euo pipefail
|
||||
|
||||
MODE="${1:?usage: pkg-cache-repos.sh rpm|apt|release-urls|restore}"
|
||||
STATE=/var/lib/pkg-cache-repos # pristine copies of every file we rewrite
|
||||
|
||||
if [ "$MODE" = restore ]; then
|
||||
[ -d "$STATE" ] || exit 0
|
||||
(cd "$STATE" && find . -type f) | while read -r rel; do
|
||||
cat "$STATE/$rel" > "/${rel#./}"
|
||||
done
|
||||
rm -rf "$STATE"
|
||||
echo "pkg-cache-repos: original repository files restored"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Keep the first-seen (pre-rewrite) copy of a file; later calls in the same RUN
|
||||
# never overwrite it.
|
||||
save() {
|
||||
[ -e "$STATE$1" ] && return 0
|
||||
mkdir -p "$STATE$(dirname "$1")"
|
||||
cp -p "$1" "$STATE$1"
|
||||
}
|
||||
MODE="${1:?usage: pkg-cache-repos.sh rpm|apt|release-install NAME...|restore}"
|
||||
shift || true
|
||||
STATE=/var/lib/pkg-cache-repos # saved originals + manifest + created files
|
||||
SHARE="${PKG_CACHE_SHARE:-/usr/local/share/pkg-cache}" # ca.crt + RPM-GPG-KEY-*
|
||||
C="${PKG_CACHE:-}"
|
||||
C="${C%/}"
|
||||
|
||||
if [ "$MODE" = release-urls ]; then
|
||||
die() { echo "FATAL(pkg-cache-repos): $*" >&2; exit 1; }
|
||||
|
||||
if [ -n "$C" ]; then
|
||||
case "$C" in
|
||||
https://*) ;;
|
||||
*) die "PKG_CACHE must be an https:// URL (plain http is refused), got '$C'" ;;
|
||||
esac
|
||||
HOSTPORT="${C#https://}"; HOSTPORT="${HOSTPORT%%/*}"
|
||||
[ -s "$SHARE/ca.crt" ] || die "PKG_CACHE set but $SHARE/ca.crt (pkg-cache CA) is missing"
|
||||
fi
|
||||
|
||||
# Files that may hold repo configuration; restore re-checks all of them.
|
||||
repo_files() {
|
||||
find /etc/yum.repos.d /etc/dnf /etc/apt -type f 2>/dev/null | sort || true
|
||||
}
|
||||
|
||||
start_session() {
|
||||
[ -d "$STATE" ] && return 0
|
||||
mkdir -p "$STATE/orig"
|
||||
repo_files > "$STATE/manifest" # what existed before this RUN touched anything
|
||||
: > "$STATE/created"
|
||||
printf '%s\n' "$HOSTPORT" > "$STATE/host"
|
||||
}
|
||||
save() { # keep the first-seen (pre-rewrite) copy
|
||||
[ -e "$STATE/orig$1" ] && return 0
|
||||
mkdir -p "$STATE/orig$(dirname "$1")"
|
||||
cp -p "$1" "$STATE/orig$1"
|
||||
}
|
||||
created() { printf '%s\n' "$1" >> "$STATE/created"; }
|
||||
|
||||
if [ "$MODE" = restore ]; then
|
||||
[ -d "$STATE" ] || exit 0
|
||||
host=$(cat "$STATE/host")
|
||||
(cd "$STATE/orig" && find . -type f) | while read -r rel; do
|
||||
cat "$STATE/orig/$rel" > "/${rel#./}"
|
||||
done
|
||||
while read -r f; do [ -n "$f" ] && rm -f -- "$f"; done < "$STATE/created"
|
||||
# A repo/apt file that appeared during this RUN and still names the cache
|
||||
# (e.g. written by a package from the rewritten repos) is removed too.
|
||||
repo_files | while read -r f; do
|
||||
if ! grep -qxF -- "$f" "$STATE/manifest" && grep -qF -- "$host" "$f"; then
|
||||
echo "pkg-cache-repos: removing new file that references the cache: $f"
|
||||
rm -f -- "$f"
|
||||
fi
|
||||
done
|
||||
left=$(repo_files | xargs -r grep -lF -- "$host" 2>/dev/null || true)
|
||||
rm -rf "$STATE"
|
||||
[ -z "$left" ] || die "cache address $host still present after restore in: $left"
|
||||
echo "pkg-cache-repos: original repository files restored (no reference to $host left)"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ "$MODE" = release-install ]; then
|
||||
[ $# -gt 0 ] || die "release-install needs: epel and/or remi"
|
||||
tmp=$(mktemp -d)
|
||||
curl_opts=(-fsS --proto '=https' --retry 5 --retry-all-errors --max-time 120)
|
||||
if [ -n "$C" ]; then
|
||||
echo "$C/epel/epel-release-latest-10.noarch.rpm $C/remi/enterprise/remi-release-10.rpm"
|
||||
else
|
||||
echo "https://dl.fedoraproject.org/pub/epel/epel-release-latest-10.noarch.rpm https://rpms.remirepo.net/enterprise/remi-release-10.rpm"
|
||||
start_session
|
||||
cat /etc/pki/tls/certs/ca-bundle.crt "$SHARE/ca.crt" > "$STATE/ca-bundle.pem"
|
||||
curl_opts+=(--cacert "$STATE/ca-bundle.pem")
|
||||
fi
|
||||
files=()
|
||||
for name in "$@"; do
|
||||
case "$name" in
|
||||
epel) pkg=epel-release key=RPM-GPG-KEY-EPEL-10
|
||||
path=/epel/epel-release-latest-10.noarch.rpm
|
||||
origin=https://dl.fedoraproject.org/pub/epel/epel-release-latest-10.noarch.rpm ;;
|
||||
remi) pkg=remi-release key=RPM-GPG-KEY-remi2024
|
||||
path=/remi/enterprise/remi-release-10.rpm
|
||||
origin=https://rpms.remirepo.net/enterprise/remi-release-10.rpm ;;
|
||||
*) die "unknown release package '$name'" ;;
|
||||
esac
|
||||
url="$origin"; [ -n "$C" ] && url="$C$path"
|
||||
f="$tmp/$pkg.rpm"
|
||||
curl "${curl_opts[@]}" -o "$f" "$url" || die "download failed: $url"
|
||||
rpm --import "$SHARE/$key" || die "cannot import $SHARE/$key"
|
||||
got=$(rpm -qp --nosignature --qf '%{NAME}' "$f" 2>/dev/null || true)
|
||||
[ "$got" = "$pkg" ] || die "$url is '$got', expected $pkg"
|
||||
chk=$(rpm -K "$f" 2>&1 || true)
|
||||
[[ "$chk" == *": digests signatures OK" ]] || die "signature check failed for $url: $chk"
|
||||
echo "pkg-cache-repos: $pkg verified ($chk) from $url"
|
||||
files+=("$f")
|
||||
done
|
||||
dnf install -y --setopt=localpkg_gpgcheck=1 "${files[@]}"
|
||||
rm -rf "$tmp"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
@@ -65,31 +142,34 @@ if [ -z "$C" ]; then
|
||||
echo "pkg-cache-repos($MODE): PKG_CACHE not set - using upstream repositories"
|
||||
exit 0
|
||||
fi
|
||||
case "$C" in
|
||||
http://*|https://*) ;;
|
||||
*) echo "FATAL(pkg-cache-repos): PKG_CACHE must be an http(s) URL, got '$C'" >&2; exit 1 ;;
|
||||
esac
|
||||
|
||||
rewrite_rpm() {
|
||||
local f="$1"
|
||||
grep -q -E '^#? ?baseurl=https?://(repo\.almalinux\.org|download\.example|rpms\.remirepo\.net)/' "$f" || return 0
|
||||
save "$f"
|
||||
# Per [section]: a (commented or live) baseurl on a known origin becomes
|
||||
# "baseurl=<cache>/<prefix>/<path> <origin>/<path>", and that section's
|
||||
# mirrorlist/metalink is commented out (dnf prefers them over baseurl).
|
||||
# Sections without a known origin (vault debug/source, ...) are untouched.
|
||||
awk -v C="$C" '
|
||||
# "baseurl=<cache>/<prefix>/<path> <origin>/<path>"; that section's
|
||||
# mirrorlist/metalink is commented out (dnf prefers them over baseurl) and it
|
||||
# gets sslcacert=<system bundle + pkg-cache CA>, gpgcheck=1 and, for repos
|
||||
# that sign repomd.xml, repo_gpgcheck=1. Other sections are untouched.
|
||||
awk -v C="$C" -v BUNDLE="$STATE/ca-bundle.pem" '
|
||||
function flush( i) {
|
||||
for (i = 1; i <= n; i++) {
|
||||
if (hit && buf[i] ~ /^(mirrorlist|metalink)=/) buf[i] = "#" buf[i]
|
||||
if (hit && buf[i] ~ /^(sslcacert|gpgcheck|repo_gpgcheck)[ \t]*=/) continue
|
||||
print buf[i]
|
||||
}
|
||||
n = 0; hit = 0
|
||||
if (hit) {
|
||||
print "sslcacert=" BUNDLE
|
||||
print "gpgcheck=1"
|
||||
if (signed) print "repo_gpgcheck=1"
|
||||
}
|
||||
n = 0; hit = 0; signed = 0
|
||||
}
|
||||
function rw(line, re, prefix, origin, p) {
|
||||
function rw(line, re, prefix, origin, sig, p) {
|
||||
if (match(line, re)) {
|
||||
p = substr(line, RSTART + RLENGTH); sub(/[ \t].*$/, "", p)
|
||||
hit = 1
|
||||
hit = 1; if (sig) signed = 1
|
||||
return "baseurl=" C "/" prefix "/" p " " origin "/" p
|
||||
}
|
||||
return line
|
||||
@@ -98,9 +178,9 @@ rewrite_rpm() {
|
||||
{
|
||||
l = $0
|
||||
if (l ~ /^#? ?baseurl=/) {
|
||||
l = rw(l, "^#? ?baseurl=https?://repo\\.almalinux\\.org/almalinux/", "almalinux", "https://repo.almalinux.org/almalinux")
|
||||
l = rw(l, "^#? ?baseurl=https?://download\\.example/pub/epel/", "epel", "https://dl.fedoraproject.org/pub/epel")
|
||||
l = rw(l, "^#? ?baseurl=https?://rpms\\.remirepo\\.net/", "remi", "https://rpms.remirepo.net")
|
||||
l = rw(l, "^#? ?baseurl=https?://repo\\.almalinux\\.org/almalinux/", "almalinux", "https://repo.almalinux.org/almalinux", 1)
|
||||
l = rw(l, "^#? ?baseurl=https?://download\\.example/pub/epel/", "epel", "https://dl.fedoraproject.org/pub/epel", 0)
|
||||
l = rw(l, "^#? ?baseurl=https?://rpms\\.remirepo\\.net/", "remi", "https://rpms.remirepo.net", 1)
|
||||
}
|
||||
buf[++n] = l
|
||||
}
|
||||
@@ -121,16 +201,23 @@ rewrite_apt() {
|
||||
|
||||
case "$MODE" in
|
||||
rpm)
|
||||
start_session
|
||||
cat /etc/pki/tls/certs/ca-bundle.crt "$SHARE/ca.crt" > "$STATE/ca-bundle.pem"
|
||||
for f in /etc/yum.repos.d/*.repo; do [ -f "$f" ] && rewrite_rpm "$f"; done
|
||||
echo "pkg-cache-repos(rpm): repos now via $C:"
|
||||
grep -h '^baseurl=' /etc/yum.repos.d/*.repo | grep -F "$C/" | sort -u | sed 's/^/ /'
|
||||
;;
|
||||
apt)
|
||||
start_session
|
||||
conf=/etc/apt/apt.conf.d/99pkg-cache
|
||||
# Trust the pkg-cache CA for the cache host ONLY (apt per-host option).
|
||||
printf 'Acquire::https::%s::CaInfo "%s";\n' "${HOSTPORT%%:*}" "$SHARE/ca.crt" > "$conf"
|
||||
created "$conf"
|
||||
for f in /etc/apt/sources.list /etc/apt/sources.list.d/*.list /etc/apt/sources.list.d/*.sources; do
|
||||
[ -f "$f" ] && rewrite_apt "$f"
|
||||
done
|
||||
echo "pkg-cache-repos(apt): sources now via $C:"
|
||||
grep -h -E '^(deb|URIs:)' /etc/apt/sources.list /etc/apt/sources.list.d/* 2>/dev/null | grep -F "$C/" | sort -u | sed 's/^/ /'
|
||||
;;
|
||||
*) echo "FATAL(pkg-cache-repos): unknown mode '$MODE'" >&2; exit 1 ;;
|
||||
*) die "unknown mode '$MODE'" ;;
|
||||
esac
|
||||
Reference in new issue
Block a user