Compare commits

...
Author SHA1 Message Date
shadowdaoandClaude Sonnet 5.5 3eee8e6023 wp-admin gate: state the real residual risks of exempting the install screens
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0185ukXi4YX5FxJGqhi6qab6
2026-11-19 16:47:16 -08:00
shadowdaoandClaude Sonnet 5.5 c91187aa85 fix(wp-admin gate): exempt the WordPress install screens (setup-config.php, install.php)
HAProxy Manager Build and Push / Build-and-Push (push) Successful in 16m35s
A fresh WordPress with no wp-config.php redirects every request to
/wp-admin/setup-config.php. The edge wp-admin gate bounced that cookieless
request to wp-login.php?redirect_to=/wp-admin/setup-config.php, which then
tripped CRS 930120 ("config.php" in ARGS:redirect_to) and showed the WAF
block page: manual installers never saw WordPress's setup screen.

New ACL wp_admin_install (path_end, exact /wp-admin/setup-config.php and
/wp-admin/install.php on the normalised path, case-sensitive) added as a
negated exclusion on the redirect rule. upgrade.php stays gated (only
logged-in users are sent there). Variants (.php.php, .phpx, case, path info,
inserted directory, dotdot/encoded escapes) stay gated; encoded separators
still deny.

Tests: the structural suite now pins the new ACL; new
test-wpadmin-gate-behaviour.py runs the real gate rules in the real haproxy
and sends raw requests (run by the image build). VERSION 2026.11.7.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0185ukXi4YX5FxJGqhi6qab6
2026-11-19 16:34:52 -08:00
jknapp 193d39e109 Merge pull request 'HAProxy 3.2 LTS edge image (default for all new deployments)' (#20) from feat/haproxy-upgrade into main
HAProxy Manager Build and Push / Build-and-Push (push) Successful in 9m40s
2026-11-17 21:07:33 +00:00
shadowdaoandClaude Sonnet 5.5 aebb68fe9d ci: make tag verification independent of buildx version
buildx v0.28.0 ignores `imagetools inspect --format '{{.Manifest.Digest}}'`
and prints the whole inspect text, so the verify step rejected every ref as a
"bad digest" (CI run 5381). Move the gate into scripts/verify-published-tags.sh:
the index digest is the sha256 of `inspect --raw`, cross-checked against the
anchored `Digest:` line and a bare --format output when present; platforms are
read from the raw JSON. All other checks (every tag in every registry, platform
match, one digest across registries, any miss fails) are unchanged. Adds an
offline test with a stub emulating 0.28 and 0.38 output.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0185ukXi4YX5FxJGqhi6qab6
2026-11-16 22:30:21 -08:00
shadowdaoandClaude Sonnet 5.5 ba8722b0b7 chore: bump VERSION to 2026.11.6 (rc with gunicorn access-log fix + hardened CI)
HAProxy Manager Build and Push / Build-and-Push (push) Failing after 13m10s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0185ukXi4YX5FxJGqhi6qab6
2026-11-16 22:13:46 -08:00
shadowdaoandClaude Sonnet 5.5 fa8fec0b4f fix(logging): keep gunicorn access logs off container stdout
Both gunicorns ran with --access-logfile -, so every :8080 default-page /
blocked-IP / suspended-site hit and every :8000 API call became a docker
stdout line. With the journald log driver (WHP PR #260) that is one journal
entry per request: journald's default rate limit (10000 msgs / 30 s per
service) could suppress the relay's crash-dump lines, and Graylog would be
flooded (the edge access log must stay out of it).

* :8080 default-page gunicorn: no access log.
* :8000 API gunicorn: access log to /var/log/haproxy/api-access.log (mode
  0640, haproxy-logs volume), bounded by scripts/gunicorn-access-rotate.py
  (copy-truncate, 20 MiB x 3, 30 s poll). 401/403 are in the file only.
* gunicorn ERROR logging stays on stderr; supervisor/relay/startup lines
  unchanged on stdout/stderr.
* New build-gate test scripts/test-gunicorn-access-logging.py; docs/logging.md.

VERSION unchanged (2026.11.5): a NEW rc image must be built from this branch
for the change to ship.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0185ukXi4YX5FxJGqhi6qab6
2026-11-16 22:12:53 -08:00
shadowdaoandClaude Sonnet 5.5 68976142f5 ci: harden assemble step (pinned buildx, plain progress, timeout) and verify every tag in every registry
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0185ukXi4YX5FxJGqhi6qab6
2026-11-16 21:10:51 -08:00
14 changed files with 949 additions and 13 deletions

No files matched your search

+58 -1
View File
@@ -133,22 +133,79 @@ jobs:
PKG_CACHE=${{ steps.pkgcache.outputs.url }}
outputs: type=image,"name=repo.anhonesthost.net/cloud-hosting-platform/haproxy-manager-base,ghcr.io/shadowdao/haproxy-manager-base",push-by-digest=true,name-canonical=true,push=true
# imagetools needs a NEW buildx. Same pinned, sha256-verified binary as
# mirror-base-image.yaml (kept identical on purpose). The assemble step
# previously ran on whatever buildx the (possibly ephemeral) runner had
# (0.13.1 on the runner that hung in run 5375, 0.15.1 on the trusted one).
- name: Install pinned buildx
env:
BUILDX_VERSION: v0.28.0
BUILDX_SHA256: 696bc104bac3bb708eff1af3f8bbc09fda0fd88f5757c1f9b404a35117889224
run: |
set -euo pipefail
dir="${RUNNER_TEMP:-/tmp}/pinned-buildx"; mkdir -p "$dir"
curl -fsSL --retry 3 --max-time 120 -o "$dir/docker-buildx" \
"https://github.com/docker/buildx/releases/download/${BUILDX_VERSION}/buildx-${BUILDX_VERSION}.linux-amd64"
echo "${BUILDX_SHA256} $dir/docker-buildx" | sha256sum -c -
chmod +x "$dir/docker-buildx"
"$dir/docker-buildx" version
echo "BUILDX=$dir/docker-buildx" >> "$GITHUB_ENV"
# main: three tags per registry: :latest (moving), :<version> (human-readable
# release), :<sha> (immutable, guaranteed-unique rollback target).
# release/* or rc-*: :<version>-rc and :<sha> only (see "Resolve tags").
# Hardened after run 5375 (assemble hung after pushing, TTY progress, old
# buildx): plain progress, pinned buildx, hard 10 minute step timeout.
- name: Assemble tag indexes
id: assemble
timeout-minutes: 10
env:
BUILDKIT_PROGRESS: plain
AMD64_DIGEST: ${{ steps.amd64.outputs.digest }}
ARM64_DIGEST: ${{ steps.arm64.outputs.digest }}
ARM64_OUTCOME: ${{ steps.arm64.outcome }}
VERSION: ${{ steps.ver.outputs.version }}
TAGS: ${{ steps.tags.outputs.tags }}
run: |
set -euo pipefail
tag_args=()
for t in $TAGS; do tag_args+=(--tag "$t"); done
scripts/assemble-multiarch-index.sh \
# Use the pinned buildx for create AND the script's platform inspects.
export IMAGETOOLS="${BUILDX} imagetools"
out=$(scripts/assemble-multiarch-index.sh \
--amd64 "$AMD64_DIGEST" \
--arm64 "$ARM64_DIGEST" --arm64-outcome "$ARM64_OUTCOME" \
--repo repo.anhonesthost.net/cloud-hosting-platform/haproxy-manager-base \
--repo ghcr.io/shadowdao/haproxy-manager-base \
"${tag_args[@]}" | tee /dev/stderr)
platforms=$(printf '%s\n' "$out" | sed -n 's/^platforms=//p' | tail -n1)
[ -n "$platforms" ] || { echo "::error::assemble script reported no platforms line"; exit 1; }
if [ "$platforms" = "linux/amd64" ]; then
echo "::warning::SINGLE-ARCH publish (amd64 only) on purpose: arm64 outcome was '${ARM64_OUTCOME}'. Tags carry no arm64 manifest."
else
echo "multi-arch publish: ${platforms}"
fi
echo "platforms=${platforms}" >> "$GITHUB_OUTPUT"
# Final gate: every tag in every registry must resolve, carry the platforms
# the assemble step reported, and have ONE digest across all tags and both
# registries (they are built from the same sources). Any miss fails the job
# instead of trusting that the create calls "looked" fine.
- name: Verify published tags in every registry
timeout-minutes: 10
env:
BUILDKIT_PROGRESS: plain
TAGS: ${{ steps.tags.outputs.tags }}
PLATFORMS: ${{ steps.assemble.outputs.platforms }}
run: |
set -euo pipefail
# Version-independent digest logic lives in the script (tested by
# scripts/test-verify-published-tags.py); buildx 0.28 ignores --format.
export IMAGETOOLS="${BUILDX} imagetools"
tag_args=()
for t in $TAGS; do tag_args+=(--tag "$t"); done
scripts/verify-published-tags.sh \
--platforms "$PLATFORMS" \
--repo repo.anhonesthost.net/cloud-hosting-platform/haproxy-manager-base \
--repo ghcr.io/shadowdao/haproxy-manager-base \
"${tag_args[@]}"
+2
View File
@@ -14,7 +14,9 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
- **WAF body inspection**: `python3 scripts/test-waf-body-inspection.py` - offline (runs in the image build); pins `option http-buffer-request`, the `X-WHP-WAF-Body` complete/partial stamp (strip -> stamp -> SPOE -> strip), `tune.bufsize` against the 100 ms SPOE fail-open budget and the 65535-byte SPOP frame cap, and Coraza's body limits in `coraza-spoa/config.yaml`. Run it after touching the Coraza block, `tune.bufsize`, or the SPOE engine.
- **Superseded-cert retirement**: covered by the same file (`TestRetirementRules`, `TestSanOverlapRetirement`). A new bundle retires any OTHER pem/lineage sharing ANY name (CN or SAN) with it - the pem moved to `cert-retired/`, the lineage's live+archive+renewal moved TOGETHER to `/etc/letsencrypt/retired/<stamp>/` (leaving archive/ behind makes certbot refuse the name forever), never deleted; the bundle's own `--cert-name` lineage is `live/<primary>` exactly and is never retired - unless that cert is still the only valid one for a name this edge serves (`retirement_blockers()`). `GET /api/capabilities` advertises this to WHP, which must not use whole-site re-bundling without it. Expired pems already in the crt dir are moved to `cert-retired/` by `cert_retire_expired_pems` (never the last pem, never `default_self_signed_cert.pem`). Expired lineages are never published (`cert_is_expired` in `cert-publish-lib.sh`, `leaf_is_expired()` in Python). Fixtures: `scripts/test-fixtures/san-certs/`.
- **Multi-arch index assembly (coraza-spoa)**: `python3 scripts/test-assemble-multiarch-index.py` - offline (NOT run in the image build; it also reads `.gitea/workflows/build-push-coraza.yaml`); `scripts/assemble-multiarch-index.sh` turns the by-digest amd64 + arm64 pushes into the tag(s): arm64 is attached only if THIS run's arm64 step outcome is `success` (failure => amd64-only tag, never a stale arm64), a bad amd64 digest publishes nothing. `scripts/e2e-coraza-multiarch.sh` rehearses the whole thing against throwaway local registries (see coraza-spoa/README.md "Multi-arch").
- **Publish verify gate**: `python3 scripts/test-verify-published-tags.py` - offline (needs `jq`; NOT run in the image build); `scripts/verify-published-tags.sh` is the last build-push.yaml step: every tag in every registry must resolve, carry the assembled platforms, and share ONE index digest. The digest is the sha256 of `imagetools inspect --raw` cross-checked against the text `Digest:` line / bare `--format` output (buildx 0.28 ignores `--format` and prints full text; CI run 5381).
- **Default splash**: `python3 scripts/test-default-splash.py` - offline (runs in the image build); unrouted hosts (server hostname, bare IP, unknown names) are answered by `http-request return ... file /haproxy/errors/default-splash.html` BEFORE the Coraza SPOE call, never for a routed host (`routed_hosts.list`/`routed_wildcards.list`, written by `generate_config()` from the routing rows and in the rollback set), never for ACME/blocked/suspended. Run it after touching the splash block, the routing ACL templates, or the domain loops in `generate_config()`. See README "Default Splash".
- **gunicorn access logging**: `python3 scripts/test-gunicorn-access-logging.py` - real gunicorn; :8080 has no access log, :8000 access goes to `/var/log/haproxy/api-access.log` (rotated by `scripts/gunicorn-access-rotate.py`), nothing per-request on stdout (journald rate-limit/Graylog flood), gunicorn errors stay on stderr. See docs/logging.md. Run after touching start-up.sh's gunicorn lines.
- **Access log receiver**: `python3 scripts/test-access-log-receiver.py` - offline (runs in the image build); one datagram = one line (PRI stripped, control chars neutralised), size cap + gzip generations, write errors drop without raising, real UNIX-socket path. Run it after touching `haproxy-log-receiver.py` or the `log` line in `hap_header.tpl`.
- **Manual Testing**: Run `curl` commands against `http://localhost:8000` endpoints as shown in README.md
+2
View File
@@ -147,6 +147,7 @@ RUN pip install -r requirements.txt
# The unit suites run here too. They had never run anywhere automated either,
# and they cost a few seconds.
RUN python3 /haproxy/scripts/test-wpadmin-gate.py \
&& python3 /haproxy/scripts/test-wpadmin-gate-behaviour.py \
&& python3 /haproxy/scripts/test-trusted-proxy-gate.py \
&& python3 /haproxy/scripts/test-xmlrpc-rate-limit.py \
&& python3 /haproxy/scripts/test-default-splash.py \
@@ -158,6 +159,7 @@ RUN python3 /haproxy/scripts/test-wpadmin-gate.py \
&& python3 /haproxy/scripts/test-cert-scripts.py \
&& python3 /haproxy/scripts/test-waf-body-inspection.py \
&& python3 /haproxy/scripts/test-gunicorn-socket-timeouts.py \
&& python3 /haproxy/scripts/test-gunicorn-access-logging.py \
&& python3 /haproxy/scripts/validate-rendered-config.py
# Create log directories
RUN mkdir -p /var/log/haproxy && touch /var/log/haproxy-manager.log /var/log/haproxy-manager-errors.log
+2
View File
@@ -451,6 +451,8 @@ Covered by `scripts/test-default-splash.py` (template order/conditions, the
page, list generation) and the `splash-off` scenario of
`scripts/validate-rendered-config.py`.
> gunicorn access logs (:8080 none, :8000 bounded file, not stdout): see docs/logging.md.
## Access Log (in-container, since 2026.11.2)
HAProxy logs (`log /log.sock ... format rfc5424`, `hap_header.tpl`) to a UNIX
+1 -1
View File
@@ -1 +1 @@
2026.11.5
2026.11.7
+43
View File
@@ -0,0 +1,43 @@
# Container logging policy (what reaches `docker logs`)
`docker logs haproxy-manager` (and the docker logging driver: json-file today,
journald -> Graylog under WHP PR #260) must carry only low-volume, high-value
lines. Per-request logs stay out of it.
| Source | Destination | Volume |
|---|---|---|
| HAProxy edge access log (1.5M lines/day) | `/var/log/haproxy/access.log` via `haproxy-log-receiver.py` (20 MiB cap, gzip generations) | NOT in `docker logs` |
| HAProxy master stderr (ALERT, crash dumps) | `haproxy-stderr.log` -> `haproxy-stderr-relay.py` -> stdout (benign, rate limited) / stderr (crash evidence) | rare |
| Supervisor / relay / init lines | stdout / stderr | rare |
| gunicorn :8080 default-page / blocked-IP / suspended-site | NO access log | none |
| gunicorn :8000 management API access log | `/var/log/haproxy/api-access.log` (haproxy-logs volume, mode 0640), rotated by `scripts/gunicorn-access-rotate.py` (copy-truncate, 20 MiB x 3) | NOT in `docker logs` |
| gunicorn error log (tracebacks, worker timeouts) | stderr (`--error-logfile -`) | rare |
## Why
Before this change both gunicorns ran with `--access-logfile -`, so each :8080
hit and each :8000 API call was one docker stdout line. With the journald
driver that is one journal entry per request: journald's default rate limit
(10000 messages / 30 s per sending service) can then suppress the very
crash-dump lines the relay emits, and Graylog would be flooded.
## Knobs
`HAPROXY_MGR_API_ACCESS_LOG` (path), `HAPROXY_API_ACCESS_LOG_MAX_BYTES`
(20971520), `HAPROXY_API_ACCESS_LOG_BACKUPS` (3), `HAPROXY_API_ACCESS_LOG_POLL`
(30 s; the live file can exceed the cap by at most one interval of API traffic).
## Not done
Failed-auth (401/403) lines are NOT mirrored to stdout; they are in the file
(grep the status column). Add a rate-limited mirror later if wanted.
## Tests
`python3 scripts/test-gunicorn-access-logging.py` (runs in the image build):
real gunicorn, a burst of default-page requests produces zero stdout lines,
API requests land in the file not stdout, handler exceptions still reach
stderr, and rotation is bounded.
Deploying this needs a NEW rc image built from `feat/haproxy-upgrade`
(VERSION stays 2026.11.5 until then).
+76
View File
@@ -0,0 +1,76 @@
#!/usr/bin/env python3
"""Bound the management-API (:8000) gunicorn access log file.
Why
---
Gunicorn's access log used to go to stdout (`--access-logfile -`), i.e. every
:8080 default-page / blocked-IP / suspended-site hit and every :8000 API call
became a `docker logs` line. With the journald docker log driver that is one
journal entry per request (journald's default rate limit, 10000 msgs / 30 s per
sending service, can then SUPPRESS the crash-dump lines the relay emits) and it
floods Graylog. Now:
* :8080 (default page) has NO access log at all;
* :8000 (API) writes its access log to API_ACCESS_LOG (default
/var/log/haproxy/api-access.log, in the haproxy-logs volume), NOT stdout;
* gunicorn ERROR logging (tracebacks, worker timeouts) stays on stderr.
This process keeps that file bounded: copy-truncate rotation at
HAPROXY_API_ACCESS_LOG_MAX_BYTES (default 20 MiB), HAPROXY_API_ACCESS_LOG_BACKUPS
generations (default 3), rotated copies mode 0640 (same policy as
haproxy-stderr.log). Truncating in place is safe because gunicorn opens the
file in append mode. Several gunicorn processes share the file, so renaming it
is not an option.
Bound: size is checked every HAPROXY_API_ACCESS_LOG_POLL seconds (default 30),
so the live file can exceed the limit only by what the API writes in one
interval (a few hundred bytes per call).
"""
import os
import shutil
import sys
import time
def _env_num(name, default, cast=int):
try:
v = cast(os.environ.get(name, default))
return v if v > 0 else default
except (TypeError, ValueError):
return default
def rotate_if_needed(path, max_bytes, backups):
"""Copy-truncate rotate. Returns True if it rotated."""
try:
if os.path.getsize(path) <= max_bytes:
return False
for i in range(backups, 1, -1):
src = '%s.%d' % (path, i - 1)
if os.path.exists(src):
os.replace(src, '%s.%d' % (path, i))
shutil.copyfile(path, path + '.1')
os.chmod(path + '.1', 0o640)
with open(path, 'r+b') as fh:
fh.truncate(0)
return True
except FileNotFoundError:
return False
except OSError as exc:
sys.stderr.write('gunicorn-access-rotate: rotate failed: %s\n' % exc)
return False
def main():
path = os.environ.get('HAPROXY_MGR_API_ACCESS_LOG', '/var/log/haproxy/api-access.log')
max_bytes = _env_num('HAPROXY_API_ACCESS_LOG_MAX_BYTES', 20 * 1024 * 1024)
backups = _env_num('HAPROXY_API_ACCESS_LOG_BACKUPS', 3)
poll = _env_num('HAPROXY_API_ACCESS_LOG_POLL', 30.0, float)
while True:
try:
rotate_if_needed(path, max_bytes, backups)
except Exception as exc: # never die; the supervisor loop restarts us anyway
sys.stderr.write('gunicorn-access-rotate: %r\n' % exc)
time.sleep(poll)
if __name__ == '__main__':
main()
+24 -2
View File
@@ -89,6 +89,28 @@ cron &
done
) &
# gunicorn access logs (incident-prevention for the journald log driver).
# gunicorn used to log every request to stdout (`--access-logfile -`), so every
# :8080 default-page / blocked-IP hit and every :8000 API call became a docker
# log line -> journal entry -> Graylog line, and could trip journald's rate
# limit and suppress the relay's crash-dump lines. Now:
# :8080 no access log at all;
# :8000 access log to a size-bounded FILE in the haproxy-logs volume (mode
# 0640), rotated by scripts/gunicorn-access-rotate.py (copy-truncate,
# 20 MiB x 3). Set HAPROXY_MGR_API_ACCESS_LOG to relocate it.
# Gunicorn ERROR logging (--error-logfile -) stays on stderr -> `docker logs`.
API_ACCESS_LOG="${HAPROXY_MGR_API_ACCESS_LOG:-/var/log/haproxy/api-access.log}"
export HAPROXY_MGR_API_ACCESS_LOG="${API_ACCESS_LOG}"
mkdir -p "$(dirname "${API_ACCESS_LOG}")"
[ -e "${API_ACCESS_LOG}" ] || : > "${API_ACCESS_LOG}"
chmod 0640 "${API_ACCESS_LOG}" 2>/dev/null || true
(
while true; do
python /haproxy/scripts/gunicorn-access-rotate.py || true
sleep 2
done
) &
# Phase 1: container init
python /haproxy/scripts/init.py
@@ -135,7 +157,7 @@ gunicorn \
--workers 1 --threads 4 --worker-class "${WORKER_CLASS}" \
--max-requests "${MAX_REQ}" --max-requests-jitter "${MAX_REQ_JITTER}" \
--timeout 30 \
--access-logfile - --error-logfile - --log-level info \
--error-logfile - --log-level info \
--pythonpath /haproxy \
'haproxy_manager:default_app' &
@@ -147,6 +169,6 @@ exec gunicorn \
--workers "${API_WORKERS}" --threads 4 --worker-class "${WORKER_CLASS}" \
--max-requests "${MAX_REQ}" --max-requests-jitter "${MAX_REQ_JITTER}" \
--timeout "${API_TIMEOUT}" \
--access-logfile - --error-logfile - --log-level info \
--access-logfile "${API_ACCESS_LOG}" --error-logfile - --log-level info \
--pythonpath /haproxy \
'haproxy_manager:app'
+177
View File
@@ -0,0 +1,177 @@
#!/usr/bin/env python3
"""Tests: gunicorn access logging must not flood the container's stdout.
Context: with the journald docker log driver every stdout line is a journal
entry (default rate limit 10000 msgs/30 s per service could suppress crash-dump
lines) and, via Graylog, a stored message. start-up.sh therefore gives :8080 no
access log and sends :8000's to a bounded FILE; gunicorn errors stay on stderr.
Pins (static, from start-up.sh):
* no gunicorn invocation uses `--access-logfile -`;
* the :8080 invocation has no --access-logfile at all;
* the :8000 invocation logs access to ${API_ACCESS_LOG}, errors to `-`.
Pins (live, REAL gunicorn, same flag shapes):
* a burst of default-page requests (no access log) writes ZERO lines to
stdout and none to stderr;
* a burst of API requests writes ZERO stdout lines, the access file receives
them, and a handler exception still reaches STDERR (error logging intact);
* scripts/gunicorn-access-rotate.py keeps the file bounded (copy-truncate,
backups capped, rotated copy mode 0640).
Runs in the Docker build gate.
"""
import importlib.util
import os
import re
import socket
import stat
import subprocess
import sys
import tempfile
import textwrap
import time
import unittest
import urllib.request
HERE = os.path.dirname(os.path.abspath(__file__))
try:
import gunicorn # noqa: F401
except ImportError:
sys.stderr.write('SKIP: gunicorn not installed\n')
raise SystemExit(77)
spec = importlib.util.spec_from_file_location('rot', os.path.join(HERE, 'gunicorn-access-rotate.py'))
rot = importlib.util.module_from_spec(spec)
spec.loader.exec_module(rot)
APP = textwrap.dedent('''\
def app(environ, start_response):
if environ['PATH_INFO'] == '/boom':
raise RuntimeError('handler-exploded')
start_response('200 OK', [('Content-Type', 'text/plain'), ('Content-Length', '2')])
return [b'ok']
''')
def free_port():
s = socket.socket()
s.bind(('127.0.0.1', 0))
p = s.getsockname()[1]
s.close()
return p
def start_script():
with open(os.path.join(HERE, 'start-up.sh')) as fh:
return fh.read()
class StaticFlags(unittest.TestCase):
def setUp(self):
self.src = start_script()
# Each invocation = from "gunicorn \" to its app target line.
self.inv = re.findall(r'^(?:exec )?gunicorn \\\n(?:.*\\\n)*?.*haproxy_manager:\w+.*$', self.src, re.M)
def test_two_invocations(self):
self.assertEqual(len(self.inv), 2, self.inv)
def test_no_stdout_access_log(self):
for i in self.inv:
self.assertNotRegex(i, r'--access-logfile\s+-(\s|$)')
def test_default_page_has_no_access_log(self):
d = [i for i in self.inv if '0.0.0.0:8080' in i]
self.assertEqual(len(d), 1)
self.assertNotIn('--access-logfile', d[0])
self.assertIn('--error-logfile -', d[0])
def test_api_access_log_is_a_file(self):
a = [i for i in self.inv if '0.0.0.0:8000' in i]
self.assertEqual(len(a), 1)
self.assertIn('--access-logfile "${API_ACCESS_LOG}"', a[0])
self.assertIn('--error-logfile -', a[0])
def test_rotator_started(self):
self.assertIn('gunicorn-access-rotate.py', self.src)
class Live(unittest.TestCase):
N_DEFAULT = 300
N_API = 100
def setUp(self):
self.dir = tempfile.mkdtemp(prefix='gunicorn-access-')
with open(os.path.join(self.dir, 'tapp.py'), 'w') as fh:
fh.write(APP)
def run_gunicorn(self, extra, n, boom=False):
port = free_port()
out = open(os.path.join(self.dir, 'out-%d' % port), 'w+')
err = open(os.path.join(self.dir, 'err-%d' % port), 'w+')
p = subprocess.Popen(
[sys.executable, '-m', 'gunicorn', '--bind', '127.0.0.1:%d' % port,
'--workers', '1', '--threads', '4', '--error-logfile', '-', '--log-level', 'info',
'--pythonpath', self.dir] + extra + ['tapp:app'],
stdout=out, stderr=err)
try:
for _ in range(100):
try:
urllib.request.urlopen('http://127.0.0.1:%d/' % port, timeout=2).read()
break
except Exception:
time.sleep(0.2)
for _ in range(n):
urllib.request.urlopen('http://127.0.0.1:%d/x' % port, timeout=5).read()
if boom:
try:
urllib.request.urlopen('http://127.0.0.1:%d/boom' % port, timeout=5).read()
except Exception:
pass
time.sleep(0.5)
finally:
p.terminate()
p.wait(10)
out.seek(0)
err.seek(0)
o, e = out.read(), err.read()
out.close()
err.close()
return o, e
def test_default_page_burst_is_silent(self):
out, err = self.run_gunicorn([], self.N_DEFAULT)
self.assertEqual(out, '', 'access logging leaked to stdout: %r' % out[:200])
self.assertNotIn('GET /x', err)
def test_api_burst_goes_to_file_not_stdout_and_errors_stay_on_stderr(self):
path = os.path.join(self.dir, 'api-access.log')
open(path, 'w').close()
out, err = self.run_gunicorn(['--access-logfile', path], self.N_API, boom=True)
self.assertEqual(out, '', 'access logging leaked to stdout: %r' % out[:200])
self.assertNotIn('GET /x', err)
with open(path) as fh:
lines = fh.read().splitlines()
self.assertGreaterEqual(len([l for l in lines if 'GET /x' in l]), self.N_API)
self.assertIn('handler-exploded', err) # error logging intact
class Rotation(unittest.TestCase):
def test_bounded_and_mode(self):
d = tempfile.mkdtemp(prefix='gunicorn-rot-')
path = os.path.join(d, 'api-access.log')
for _ in range(12):
with open(path, 'ab') as fh:
fh.write(b'x' * 1000 + b'\n')
rot.rotate_if_needed(path, 4000, 3)
files = sorted(os.listdir(d))
self.assertLessEqual(len(files), 4, files) # live + 3 backups
self.assertLessEqual(os.path.getsize(path), 4000 + 1001)
self.assertEqual(stat.S_IMODE(os.stat(path + '.1').st_mode), 0o640)
self.assertFalse(os.path.exists(path + '.4'))
def test_missing_file_is_not_fatal(self):
self.assertFalse(rot.rotate_if_needed('/nonexistent/api-access.log', 10, 3))
if __name__ == '__main__':
unittest.main()
+193
View File
@@ -0,0 +1,193 @@
#!/usr/bin/env python3
"""Offline tests for scripts/verify-published-tags.sh (final publish gate).
CI run 5381 failed because buildx v0.28.0 ignores `--format '{{.Manifest.Digest}}'`
and prints the whole inspect text, while buildx 0.38 prints a bare digest. A stub
`imagetools` emulates BOTH output styles (plus `--raw`), so no registry or Docker
is needed. Held invariants:
- the 0.28-style (full text) and 0.38-style (bare digest) outputs both verify
- a reported digest that disagrees with the sha256 of the raw manifest fails
- a missing tag, a platform mismatch, or different digests across registries /
tags fail the job
- build-push.yaml's verify step uses the script, and release/* + rc-* tags
resolve to ONLY <version>-rc and <sha> (never latest)
Running: python3 scripts/test-verify-published-tags.py (needs jq)
"""
import hashlib
import json
import os
import re
import stat
import subprocess
import tempfile
import unittest
HERE = os.path.dirname(os.path.abspath(__file__))
ROOT = os.path.dirname(HERE)
SCRIPT = os.path.join(HERE, 'verify-published-tags.sh')
WORKFLOW = os.path.join(ROOT, '.gitea', 'workflows', 'build-push.yaml')
STUB = r'''#!/usr/bin/env python3
import hashlib, json, os, sys
a = sys.argv[1:]
mode = os.environ['STUB_MODE'] # v028 | v038
table = json.load(open(os.environ['STUB_TABLE'])) # ref -> {"raw": str, "lie": optional digest}
assert a[0] == 'inspect'
ref = [x for x in a[1:] if not x.startswith('-') and not x.startswith('{{')][0]
ent = table.get(ref)
if ent is None:
sys.stderr.write('ERROR: %s: not found\n' % ref); sys.exit(1)
raw = ent['raw'].encode() + (b'\n' if ent.get('nl') else b'')
digest = ent.get('lie') or 'sha256:' + hashlib.sha256(ent['raw'].encode()).hexdigest()
if '--raw' in a:
sys.stdout.buffer.write(raw); sys.exit(0)
if '--format' in a:
if mode == 'v038':
print(digest)
else: # 0.28: --format ignored, whole text printed
print('Name: ' + ref); print('MediaType: application/vnd.oci.image.index.v1+json')
print('Digest: ' + digest)
sys.exit(0)
print('Name: ' + ref)
print('MediaType: application/vnd.oci.image.index.v1+json')
print('Digest: ' + digest)
print(' ')
print('Manifests: ')
for m in json.loads(ent['raw'])['manifests']:
print(' Name: %s@%s' % (ref, m['digest']))
print(' Digest: sha256:' + 'f' * 64) # indented: must never be mistaken for the index digest
print(' Platform: %s/%s' % (m['platform']['os'], m['platform']['architecture']))
'''
def index(archs, salt=''):
ms = [{'mediaType': 'application/vnd.oci.image.manifest.v1+json',
'digest': 'sha256:' + hashlib.sha256((x + salt).encode()).hexdigest(), 'size': 1,
'platform': {'architecture': x, 'os': 'linux'}} for x in archs]
ms.append({'mediaType': 'application/vnd.oci.image.manifest.v1+json',
'digest': 'sha256:' + '7' * 64, 'size': 1,
'platform': {'architecture': 'unknown', 'os': 'unknown'},
'annotations': {'vnd.docker.reference.type': 'attestation-manifest'}})
return json.dumps({'schemaVersion': 2, 'mediaType': 'application/vnd.oci.image.index.v1+json',
'manifests': ms}, separators=(',', ':'))
REPOS = ['reg.one/a/img', 'reg.two/b/img']
TAGS = ['1.2-rc', 'abc123']
class VerifyTest(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.stub = os.path.join(self.tmp.name, 'stub')
open(self.stub, 'w').write(STUB)
os.chmod(self.stub, os.stat(self.stub).st_mode | stat.S_IEXEC)
self.table = os.path.join(self.tmp.name, 'table.json')
def tearDown(self):
self.tmp.cleanup()
def table_for(self, raw, **over):
t = {'%s:%s' % (r, g): {'raw': raw} for r in REPOS for g in TAGS}
for k, v in over.items():
ref = k.replace('__', ':').replace('_', '/', 0)
t[ref] = v
return t
def run_script(self, table, mode, platforms='linux/amd64,linux/arm64', tags=TAGS):
json.dump(table, open(self.table, 'w'))
args = [SCRIPT, '--platforms', platforms]
for r in REPOS:
args += ['--repo', r]
for t in tags:
args += ['--tag', t]
env = dict(os.environ, IMAGETOOLS=self.stub, STUB_MODE=mode, STUB_TABLE=self.table)
return subprocess.run(args, env=env, capture_output=True, text=True, timeout=60)
def test_buildx_028_full_text_output_verifies(self):
r = self.run_script(self.table_for(index(['amd64', 'arm64'])), 'v028')
self.assertEqual(r.returncode, 0, r.stdout + r.stderr)
self.assertIn('verified:', r.stdout)
self.assertNotIn('bad digest', r.stdout)
def test_buildx_038_bare_digest_output_verifies(self):
r = self.run_script(self.table_for(index(['amd64', 'arm64'])), 'v038')
self.assertEqual(r.returncode, 0, r.stdout + r.stderr)
def test_trailing_newline_on_raw_output_tolerated(self):
raw = index(['amd64', 'arm64'])
t = {'%s:%s' % (r, g): {'raw': raw, 'nl': True} for r in REPOS for g in TAGS}
self.assertEqual(self.run_script(t, 'v028').returncode, 0)
def test_amd64_only_publish_matches_platforms(self):
raw = index(['amd64'])
self.assertEqual(self.run_script(self.table_for(raw), 'v028', 'linux/amd64').returncode, 0)
r = self.run_script(self.table_for(raw), 'v028', 'linux/amd64,linux/arm64')
self.assertEqual(r.returncode, 1)
self.assertIn('platform mismatch', r.stdout)
def test_missing_tag_fails(self):
t = self.table_for(index(['amd64', 'arm64']))
del t['reg.two/b/img:abc123']
r = self.run_script(t, 'v038')
self.assertEqual(r.returncode, 1)
self.assertIn('MISSING', r.stdout)
def test_different_digest_across_registries_fails(self):
t = self.table_for(index(['amd64', 'arm64']))
t['reg.two/b/img:1.2-rc'] = {'raw': index(['amd64', 'arm64'], salt='other')}
for mode in ('v028', 'v038'):
r = self.run_script(t, mode)
self.assertEqual(r.returncode, 1, mode)
self.assertIn('digest mismatch', r.stdout)
def test_reported_digest_disagreeing_with_raw_manifest_fails(self):
t = self.table_for(index(['amd64', 'arm64']))
t['reg.one/a/img:1.2-rc']['lie'] = 'sha256:' + '0' * 64
for mode in ('v028', 'v038'):
r = self.run_script(t, mode)
self.assertEqual(r.returncode, 1, mode)
self.assertIn('cross-check FAILED', r.stdout)
def test_garbage_manifest_fails(self):
t = self.table_for('not json')
self.assertEqual(self.run_script(t, 'v038').returncode, 1)
class WorkflowTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
import yaml
cls.src = open(WORKFLOW).read()
cls.doc = yaml.safe_load(cls.src)
cls.steps = cls.doc['jobs']['Build-and-Push']['steps']
def step(self, name):
return [s for s in self.steps if s.get('name') == name][0]
def test_verify_step_uses_script_not_format_digest(self):
run = self.step('Verify published tags in every registry')['run']
self.assertIn('scripts/verify-published-tags.sh', run)
self.assertNotIn('{{.Manifest.Digest}}', run)
def test_run_blocks_are_valid_bash(self):
for s in self.steps:
if 'run' in s:
p = subprocess.run(['bash', '-n'], input=s['run'], text=True, capture_output=True)
self.assertEqual(p.returncode, 0, '%s: %s' % (s.get('name'), p.stderr))
def test_release_and_rc_never_publish_latest(self):
run = self.step('Resolve tags')['run']
m = re.search(r'refs/heads/release/\*\|refs/tags/rc-\*\)\s*\n\s*tags="([^"]*)"', run)
self.assertIsNotNone(m)
self.assertEqual(m.group(1), '${VERSION}-rc ${SHA}')
main = re.search(r'refs/heads/main\)\s*\n\s*tags="([^"]*)"', run)
self.assertIsNotNone(main)
self.assertEqual(main.group(1), 'latest ${VERSION} ${SHA}')
# latest appears in no other tag assignment
self.assertEqual(len(re.findall(r'tags="[^"]*latest', run)), 1)
if __name__ == '__main__':
unittest.main()
+226
View File
@@ -0,0 +1,226 @@
#!/usr/bin/env python3
"""BEHAVIOURAL test of the wp-admin edge gate against the REAL haproxy binary.
test-wpadmin-gate.py asserts on the TEXT of the rendered config. This file runs
the actual gate rules (extracted from the rendered hap_listener.tpl, not
re-typed) in a real haproxy and sends real requests, so the install-screen
exemption and its path-normalisation edge cases are measured, not inferred.
Only the exclusion ACLs that depend on the client address (is_local,
is_trusted_ip, is_whitelisted) are stubbed to never match: the test client is
127.0.0.1, which the real is_local would exempt wholesale. Everything else
(normalize-uri chain, wp_admin_* ACLs, deny rules, set-var, redirect) is the
production text.
A request that PASSES the gate gets 200 "ok" from a stub `http-request return`;
a gated one gets the edge 302 (Location: .../wp-login.php?redirect_to=...) or
the 403 deny.
Skips (exit 0, loud message) when no `haproxy` binary is on PATH. The Dockerfile
build has one, so the image build always runs it.
python3 scripts/test-wpadmin-gate-behaviour.py
"""
import os
import re
import shutil
import socket
import subprocess
import sys
import tempfile
import time
import unittest
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, HERE)
os.environ.setdefault('HAPROXY_MANAGER_DIR', os.path.abspath(os.path.join(HERE, '..')))
import importlib.util # noqa: E402
_spec = importlib.util.spec_from_file_location(
'wpadmin_gate_tests', os.path.join(HERE, 'test-wpadmin-gate.py'))
gate = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(gate)
HAPROXY = shutil.which('haproxy')
# Code lines (comment-stripped) that make up the gate, in template order.
_KEEP = re.compile(
r'^(?:'
r'http-request normalize-uri '
r'|acl (?:has_wp_logged_in|wp_gate_exempt|wp_admin_\w+|path_has_encoded_sep) '
r'|http-request deny deny_status 403 if wp_admin'
r'|http-request set-var\(txn\.wp_login_url\) '
r'|http-request redirect code 302 location %\[var\(txn\.wp_login_url\)\]'
r')')
# (description, raw request path, extra header, expected status, location-prefix)
PASS, REDIR, DENY = 200, 302, 403
LOGIN = '/wp-login.php?redirect_to='
CASES = [
# --- the install/setup screens: exempt ------------------------------
('setup-config.php', '/wp-admin/setup-config.php', None, PASS, None),
('setup-config.php language step', '/wp-admin/setup-config.php?step=-1', None, PASS, None),
('install.php', '/wp-admin/install.php', None, PASS, None),
('install.php step 2', '/wp-admin/install.php?step=2', None, PASS, None),
('subdirectory install', '/blog/wp-admin/install.php', None, PASS, None),
('subdirectory setup-config', '/blog/wp-admin/setup-config.php', None, PASS, None),
# spellings that NORMALISE to the exempt path are the same request to the backend
('dotdot normalises to the exempt page', '/wp-admin/css/../setup-config.php', None, PASS, None),
('encoded dot normalises to the exempt page', '/wp-admin/setup-config%2Ephp', None, PASS, None),
('double slash merges to the exempt page', '//wp-admin/install.php', None, PASS, None),
# --- everything near them stays gated -------------------------------
('dashboard', '/wp-admin/', None, REDIR, LOGIN + '%2Fwp-admin%2F'),
('index.php', '/wp-admin/index.php', None, REDIR, LOGIN),
('upgrade.php is NOT exempt', '/wp-admin/upgrade.php', None, REDIR, LOGIN),
('plugins.php', '/wp-admin/plugins.php', None, REDIR, LOGIN),
('setup-config.php.php', '/wp-admin/setup-config.php.php', None, REDIR, LOGIN),
('setup-config.phpx', '/wp-admin/setup-config.phpx', None, REDIR, LOGIN),
('install.php.bak', '/wp-admin/install.php.bak', None, REDIR, LOGIN),
('case variant Setup-Config.php', '/wp-admin/Setup-Config.php', None, REDIR, LOGIN),
('case variant Install.php', '/wp-admin/Install.php', None, REDIR, LOGIN),
('upper-case dir WP-ADMIN', '/WP-ADMIN/install.php', None, REDIR, LOGIN),
('path info after the script', '/wp-admin/install.php/x', None, REDIR, LOGIN),
('trailing slash', '/wp-admin/install.php/', None, REDIR, LOGIN),
('inserted directory', '/wp-admin/evil/install.php', None, REDIR, LOGIN),
('inserted directory (setup-config)', '/wp-admin/x/setup-config.php', None, REDIR, LOGIN),
('dotdot climbs off the exempt page', '/wp-admin/install.php/../plugins.php', None, REDIR, LOGIN),
('asset dir dotdot to plugins.php', '/wp-admin/css/../plugins.php', None, REDIR, LOGIN),
('encoded slash separator', '/wp-admin%2Fsetup-config.php', None, DENY, None),
('encoded slash before wp-admin', '/blog%2Fwp-admin/install.php', None, DENY, None),
('backslash prefix (open redirect guard)', '/%5Cevil.example/wp-admin/plugins.php', None, DENY, None),
# --- identity still passes ------------------------------------------
('logged-in cookie on plugins.php', '/wp-admin/plugins.php',
'Cookie: wordpress_logged_in_abc=1', PASS, None),
]
def _free_port():
s = socket.socket()
s.bind(('127.0.0.1', 0))
port = s.getsockname()[1]
s.close()
return port
def _raw_get(port, path, header=None):
req = 'GET %s HTTP/1.1\r\nHost: site.example.com\r\nConnection: close\r\n' % path
if header:
req += header + '\r\n'
req += '\r\n'
with socket.create_connection(('127.0.0.1', port), timeout=5) as s:
s.sendall(req.encode('latin-1'))
buf = b''
while True:
chunk = s.recv(4096)
if not chunk:
break
buf += chunk
head = buf.split(b'\r\n\r\n', 1)[0].decode('latin-1')
lines = head.split('\r\n')
status = int(lines[0].split()[1])
loc = None
for ln in lines[1:]:
if ln.lower().startswith('location:'):
loc = ln.split(':', 1)[1].strip()
return status, loc
@unittest.skipUnless(HAPROXY, 'SKIP: no haproxy binary on PATH (the image build has one)')
class WpAdminGateBehaviour(unittest.TestCase):
@classmethod
def setUpClass(cls):
cfg = gate.render_listener()
lines = []
for raw in cfg.split('\n'):
stripped = raw.strip()
if not stripped or stripped.startswith('#'):
continue
code = stripped.split(' #', 1)[0].rstrip()
if _KEEP.match(code):
lines.append(code)
joined = '\n'.join(lines)
# Guard against a silently empty extraction: every moving part must be there.
for needle in ('normalize-uri path-strip-dotdot full', 'acl wp_admin_install',
'acl wp_admin_allowed', 'acl wp_admin_asset', 'acl wp_gate_exempt',
'acl has_wp_logged_in', 'acl path_has_encoded_sep',
'set-var(txn.wp_login_url)', 'http-request redirect code 302'):
assert needle in joined, 'extraction lost %r' % needle
cls.tmp = tempfile.mkdtemp(prefix='wpgate-behaviour-')
exempt = os.path.join(cls.tmp, 'exempt.list')
open(exempt, 'w').close()
joined = joined.replace('/etc/haproxy/wpadmin_gate_exempt.list', exempt)
cls.port = _free_port()
conf = '\n'.join([
'global',
' expose-experimental-directives',
' maxconn 100',
'defaults',
' mode http',
' timeout connect 2s',
' timeout client 5s',
' timeout server 5s',
'frontend gate',
' bind 127.0.0.1:%d' % cls.port,
# client-address exclusions stubbed: the real is_local would match 127.0.0.1
' acl is_local src 203.0.113.1',
' acl is_trusted_ip src 203.0.113.2',
' acl is_whitelisted src 203.0.113.3',
' ' + joined.replace('\n', '\n '),
' http-request return status 200 content-type text/plain string ok',
'',
])
cls.conf_path = os.path.join(cls.tmp, 'gate.cfg')
with open(cls.conf_path, 'w') as f:
f.write(conf)
chk = subprocess.run([HAPROXY, '-c', '-f', cls.conf_path],
capture_output=True, text=True)
assert chk.returncode == 0, 'haproxy -c rejected the gate:\n' + chk.stdout + chk.stderr
cls.proc = subprocess.Popen([HAPROXY, '-db', '-f', cls.conf_path],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
for _ in range(50):
try:
socket.create_connection(('127.0.0.1', cls.port), timeout=0.2).close()
break
except OSError:
time.sleep(0.1)
else:
raise AssertionError('haproxy did not start listening')
@classmethod
def tearDownClass(cls):
proc = getattr(cls, 'proc', None)
if proc:
proc.terminate()
try:
proc.wait(timeout=5)
except subprocess.TimeoutExpired:
proc.kill()
shutil.rmtree(getattr(cls, 'tmp', ''), ignore_errors=True)
def test_cases(self):
for desc, path, header, want, loc_prefix in CASES:
with self.subTest(desc=desc, path=path):
status, loc = _raw_get(self.port, path, header)
self.assertEqual(status, want, '%s: %s -> %s %s' % (desc, path, status, loc))
if loc_prefix is not None:
self.assertIsNotNone(loc, 'no Location header for ' + path)
self.assertTrue(loc.endswith(loc_prefix) or loc_prefix in loc,
'%s: Location %r' % (path, loc))
# redirect must stay same-origin (open-redirect guard)
self.assertTrue(loc.startswith('/') and not loc.startswith('//'), loc)
def test_nul_byte_after_exempt_name_is_not_exempt(self):
"""%00 is not an unreserved escape, so it is not decoded and the path no
longer ends in the exempt name. Anything but a pass-through is fine
(gated 302, or haproxy refusing the request)."""
status, _ = _raw_get(self.port, '/wp-admin/setup-config.php%00')
self.assertNotEqual(status, PASS)
if __name__ == '__main__':
unittest.main(verbosity=2)
+16 -3
View File
@@ -59,7 +59,7 @@ import haproxy_manager # noqa: E402
ALLOWLIST = ('/admin-ajax.php', '/admin-post.php',
'/load-styles.php', '/load-scripts.php')
EXCLUSIONS = ('!wp_admin_allowed', '!wp_admin_asset', '!has_wp_logged_in',
EXCLUSIONS = ('!wp_admin_allowed', '!wp_admin_install', '!wp_admin_asset', '!has_wp_logged_in',
'!wp_gate_exempt', '!is_local', '!is_trusted_ip', '!is_whitelisted')
# The normalizer set, in the order it MUST render. Decoding has to precede the
@@ -220,10 +220,23 @@ class WpAdminGate(unittest.TestCase):
line = require_rule(self.cfg, 'acl wp_admin_asset', 'wp_admin_asset ACL')
self.assertRegex(line, r'path_reg.*\(css\|js\|images\)')
def test_install_php_is_NOT_allowlisted(self):
"""install.php is deliberately gated -- a takeover vector on abandoned installs."""
def test_install_php_is_not_in_the_ajax_allowlist(self):
"""install.php / setup-config.php live in their OWN ACL (wp_admin_install),
not in wp_admin_allowed, so the admin-ajax/asset allowlist stays minimal."""
line = require_rule(self.cfg, 'acl wp_admin_allowed', 'wp_admin_allowed ACL')
self.assertNotIn('install.php', line)
self.assertNotIn('setup-config.php', line)
def test_install_screens_exempt_exactly_two_anchored_paths(self):
"""A fresh WordPress redirects every request to /wp-admin/setup-config.php
and a manual installer has no cookie. Exactly these two pages are exempt,
each anchored to /wp-admin/<file> (path_end, no regex, no wildcard).
upgrade.php is deliberately NOT exempt (logged-in only)."""
line = require_rule(self.cfg, 'acl wp_admin_install', 'wp_admin_install ACL')
self.assertEqual(line.split(None, 3)[2], 'path_end')
entries = line.split()[3:]
self.assertEqual(sorted(entries),
['/wp-admin/install.php', '/wp-admin/setup-config.php'])
def test_redirect_rule_has_all_exclusions(self):
rule = require_rule_by_predicate(
+97
View File
@@ -0,0 +1,97 @@
#!/usr/bin/env bash
# Final publish gate: every <repo>:<tag> must resolve, carry exactly the
# platforms the assemble step reported, and have ONE index digest across all
# tags and all registries. Any miss or mismatch exits 1.
#
# verify-published-tags.sh --platforms linux/amd64,linux/arm64 \
# --repo <registry/path> [--repo ...] --tag <tag> [--tag ...]
#
# The index digest is computed in a buildx-VERSION-INDEPENDENT way. CI run 5381
# showed buildx v0.28.0 ignores `--format '{{.Manifest.Digest}}'` and prints the
# whole inspect text, while 0.38 prints a bare digest. So the digest comes from:
# 1. sha256 of the raw manifest bytes (`inspect --raw`) - always, authoritative
# (a content digest IS the sha256 of the manifest bytes);
# 2. the strictly anchored `Digest:` line of the default text output, if present;
# 3. the `--format '{{.Manifest.Digest}}'` output, ONLY if it is a bare digest
# (0.28's full-text output is ignored here, not an error).
# Every source that is available must agree, else the ref fails.
# Platforms are read from the raw JSON (also version independent).
#
# Env: IMAGETOOLS overrides the command (default "docker buildx imagetools");
# the CI step sets it to "$BUILDX imagetools"; the test suite uses a stub.
set -uo pipefail
IMAGETOOLS=${IMAGETOOLS:-docker buildx imagetools}
platforms=""; repos=(); tags=()
while [ $# -gt 0 ]; do
case "$1" in
--platforms) platforms=${2-}; shift 2 ;;
--repo) repos+=("${2-}"); shift 2 ;;
--tag) tags+=("${2-}"); shift 2 ;;
*) echo "verify-published-tags: unknown argument: $1" >&2; exit 2 ;;
esac
done
[ -n "$platforms" ] && [ ${#repos[@]} -gt 0 ] && [ ${#tags[@]} -gt 0 ] \
|| { echo "::error::verify-published-tags: --platforms, --repo and --tag are required"; exit 2; }
command -v jq >/dev/null || { echo "::error::verify-published-tags: jq is required"; exit 2; }
want=$(printf '%s' "$platforms" | tr ',' '\n' | sort | tr '\n' ' ')
work=$(mktemp -d); trap 'rm -rf "$work"' EXIT
digest_re='^sha256:[0-9a-f]{64}$'
ref_digest=""; ref_name=""; fail=0
for r in "${repos[@]}"; do
for t in "${tags[@]}"; do
ref="${r}:${t}"
raw="$work/raw"
if ! timeout 90 $IMAGETOOLS inspect --raw "$ref" >"$raw" 2>"$work/err" || [ ! -s "$raw" ]; then
echo "::error::MISSING or unreadable: ${ref}: $(head -c 300 "$work/err")"; fail=1; continue
fi
# Raw bytes as served, and with trailing newlines stripped (some buildx
# versions append one when printing); the content digest is one of the two.
h1="sha256:$(sha256sum <"$raw" | cut -d' ' -f1)"
h2="sha256:$(printf '%s' "$(cat "$raw")" | sha256sum | cut -d' ' -f1)"
# Source 2: strictly anchored top-level "Digest:" line of the text output.
textd=""
if timeout 90 $IMAGETOOLS inspect "$ref" >"$work/text" 2>/dev/null; then
mapfile -t lines < <(grep -E '^Digest:[[:space:]]+sha256:[0-9a-f]{64}[[:space:]]*$' "$work/text" | sed -E 's/^Digest:[[:space:]]+//; s/[[:space:]]+$//')
[ ${#lines[@]} -eq 1 ] && textd=${lines[0]}
fi
# Source 3: bare --format output, only when it really is a bare digest.
fmtd=""
if out=$(timeout 90 $IMAGETOOLS inspect "$ref" --format '{{.Manifest.Digest}}' 2>/dev/null); then
out=$(printf '%s' "$out" | tr -d '[:space:]')
[[ "$out" =~ $digest_re ]] && fmtd=$out
fi
d=""
for cand in "$textd" "$fmtd"; do
[ -n "$cand" ] || continue
if [ "$cand" != "$h1" ] && [ "$cand" != "$h2" ]; then
echo "::error::digest cross-check FAILED for ${ref}: reported ${cand}, raw manifest hashes to ${h1}"; fail=1; d="!"; break
fi
d=$cand
done
[ "$d" = "!" ] && continue
[ -n "$d" ] || d=$h2
[[ "$d" =~ $digest_re ]] || { echo "::error::bad digest for ${ref}: ${d}"; fail=1; continue; }
if [ -z "$textd$fmtd" ]; then
echo "::warning::no reported digest for ${ref}; using sha256 of the raw manifest"
fi
if ! got=$(jq -r '.manifests[]? | select(.platform.os != "unknown") | "\(.platform.os)/\(.platform.architecture)"' "$raw" 2>/dev/null | sort | tr '\n' ' '); then
echo "::error::unparseable manifest for ${ref}"; fail=1; continue
fi
if [ "$got" != "$want" ]; then
echo "::error::platform mismatch for ${ref}: have [${got}] want [${want}]"; fail=1
fi
echo "ok ${ref} ${d} [${got}]"
if [ -z "$ref_digest" ]; then ref_digest="$d"; ref_name="$ref"
elif [ "$d" != "$ref_digest" ]; then
echo "::error::digest mismatch: ${ref}=${d} vs ${ref_name}=${ref_digest}"; fail=1
fi
done
done
[ "$fail" = 0 ] || { echo "::error::tag verification FAILED"; exit 1; }
echo "verified: ${tags[*]} in ${#repos[@]} registries, digest ${ref_digest}, platforms ${platforms}"
+32 -6
View File
@@ -431,11 +431,36 @@ frontend web
# suffix matching (/blog/wp-admin/admin-ajax.php ends with
# /wp-admin/admin-ajax.php) while rejecting an inserted directory.
#
# install.php is DELIBERATELY NOT allowlisted. It is legitimately
# reachable without a cookie during a fresh install, but it is also a
# standing scanner target and a real takeover vector on a site that was
# half-installed and then abandoned. Anyone genuinely installing uses the
# per-site exempt-list opt-out below instead.
# The WordPress install/setup screens ARE exempt (wp_admin_install below):
# exactly /wp-admin/setup-config.php and /wp-admin/install.php. A fresh
# WordPress with no wp-config.php redirects EVERY request to
# /wp-admin/setup-config.php, and a person doing a manual install has no
# cookie by definition -- gating those two pages bounced the installer to
# wp-login.php?redirect_to=/wp-admin/setup-config.php (a login form that
# can never succeed on a site with no database) and, worse, that URL then
# tripped CRS 930120 ("config.php" in ARGS:redirect_to) so the visitor saw
# the WAF block page instead of WordPress's own setup screen. WordPress
# itself refuses both once the site is installed (setup-config.php exits
# "wp-config.php already exists"; install.php shows "Already Installed"
# when the options table has siteurl), so the exemption exposes nothing on
# a finished site. RESIDUAL RISK (known tradeoff of exempting these two pages,
# requested 2026-11-19): (1) WordPress files present but NO wp-config.php
# (half upload, deleted by mistake, mid-restore/transfer): any visitor can
# run setup-config.php with their own database and then install.php, i.e.
# take the site over (admin -> plugin upload -> code execution). Scanners
# probe for exactly this. (2) A half-installed site (wp-config.php present,
# no tables, or a $table_prefix pointing at no tables) lets any visitor
# finish install.php?step=2 and choose the admin account. The old gate
# blocked cookieless scanners from both on edge-fronted sites (not a real
# defence: has_wp_logged_in only checks the cookie NAME, so a targeted
# attacker could always send one); WordPress itself never did.
# Matching is exact (path_end on the normalised path,
# case-sensitive): /wp-admin/setup-config.php.php, /wp-admin/Install.php,
# /wp-admin/install.php/x, /wp-admin/evil/install.php are all still gated.
# upgrade.php is NOT exempt: WordPress only sends logged-in users there
# (admin_init), and a logged-in user carries the cookie, so it needs no
# exemption. The install screens' CSS/JS come from the css|js|images dirs
# and load-styles.php / load-scripts.php, already allowed above.
#
# Honors the same whitelist as every other rule in this frontend
# (RFC1918 / trusted_ips.list / trusted_ips.map), and a per-site opt-out
@@ -515,6 +540,7 @@ frontend web
# the intended "reject a literal backslash" semantics.
acl wp_admin_safe_path path_reg ^/[^/\\\\]
acl wp_admin_allowed path_end /wp-admin/admin-ajax.php /wp-admin/admin-post.php /wp-admin/load-styles.php /wp-admin/load-scripts.php
acl wp_admin_install path_end /wp-admin/setup-config.php /wp-admin/install.php
# The (?!.*\.php) lookahead is DEFENCE IN DEPTH, not the primary fix. This
# ACL grants an un-gated bypass to everything under wp-admin/css|js|images,
# and it used to anchor its prefix but not its suffix, so
@@ -589,7 +615,7 @@ frontend web
http-request deny deny_status 403 if wp_admin_word path_has_encoded_sep !has_wp_logged_in !wp_gate_exempt !is_local !is_trusted_ip !is_whitelisted
http-request deny deny_status 403 if wp_admin_path !wp_admin_safe_path !has_wp_logged_in !wp_gate_exempt !is_local !is_trusted_ip !is_whitelisted
http-request set-var(txn.wp_login_url) path,regsub(/wp-admin/.*,/wp-login.php,i) if wp_admin_path
http-request redirect code 302 location %[var(txn.wp_login_url)]?redirect_to=%[path,url_enc] if wp_admin_path wp_admin_safe_path !wp_admin_allowed !wp_admin_asset !has_wp_logged_in !wp_gate_exempt !is_local !is_trusted_ip !is_whitelisted
http-request redirect code 302 location %[var(txn.wp_login_url)]?redirect_to=%[path,url_enc] if wp_admin_path wp_admin_safe_path !wp_admin_allowed !wp_admin_install !wp_admin_asset !has_wp_logged_in !wp_gate_exempt !is_local !is_trusted_ip !is_whitelisted
# IP blocking using map file (manual blocks only)
# Map file format: /etc/haproxy/blocked_ips.map contains "<ip_or_cidr> 1" per line