Default splash for unrouted hosts (404+noindex, answered before the WAF) and in-container access log (2026.11.2) (#15)

CLAUDE.md

@@ -13,6 +13,8 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 - **API thread-pool safety**: `python3 scripts/test-gunicorn-socket-timeouts.py` - runs REAL gunicorn (~60s); asserts a silent/half-written client cannot pin a gthread pool thread (per-connection socket timeout, `HAPROXY_MGR_CLIENT_TIMEOUT`), 2050 never-sending connections are reaped (`HAPROXY_MGR_FIRST_BYTE_TIMEOUT`), slowloris header drips are cut (`HAPROXY_MGR_HEADER_DEADLINE`, counted from enqueue), 40 reconnecting drippers cannot starve legit requests, the :8000 peer filter (`HAPROXY_MGR_API_ALLOWED_PEERS`) refuses non-allowed peers, slow legit requests are not cut off, the wedge watchdog (`HAPROXY_MGR_WEDGE_SECONDS`) replaces a fully stuck worker, and an in-process call to a stalled upstream times out. Run it after touching `haproxy_gunicorn_worker.py`, start-up.sh's gunicorn lines, or upgrading gunicorn (the worker overrides gthread internals: `enqueue_req`, `accept`, `on_client_socket_readable`, `murder_keepalived`, `handle`, `handle_request`).
 - **WAF body inspection**: `python3 scripts/test-waf-body-inspection.py` - offline (runs in the image build); pins `option http-buffer-request`, the `X-WHP-WAF-Body` complete/partial stamp (strip -> stamp -> SPOE -> strip), `tune.bufsize` against the 100 ms SPOE fail-open budget and the 65535-byte SPOP frame cap, and Coraza's body limits in `coraza-spoa/config.yaml`. Run it after touching the Coraza block, `tune.bufsize`, or the SPOE engine.
 - **Superseded-cert retirement**: covered by the same file (`TestRetirementRules`, `TestSanOverlapRetirement`). A new bundle retires any OTHER pem/lineage sharing ANY name (CN or SAN) with it - the pem moved to `cert-retired/`, the lineage's live+archive+renewal moved TOGETHER to `/etc/letsencrypt/retired/<stamp>/` (leaving archive/ behind makes certbot refuse the name forever), never deleted; the bundle's own `--cert-name` lineage is `live/<primary>` exactly and is never retired - unless that cert is still the only valid one for a name this edge serves (`retirement_blockers()`). `GET /api/capabilities` advertises this to WHP, which must not use whole-site re-bundling without it. Expired pems already in the crt dir are moved to `cert-retired/` by `cert_retire_expired_pems` (never the last pem, never `default_self_signed_cert.pem`). Expired lineages are never published (`cert_is_expired` in `cert-publish-lib.sh`, `leaf_is_expired()` in Python). Fixtures: `scripts/test-fixtures/san-certs/`.
+- **Default splash**: `python3 scripts/test-default-splash.py` - offline (runs in the image build); unrouted hosts (server hostname, bare IP, unknown names) are answered by `http-request return ... file /haproxy/errors/default-splash.html` BEFORE the Coraza SPOE call, never for a routed host (`routed_hosts.list`/`routed_wildcards.list`, written by `generate_config()` from the routing rows and in the rollback set), never for ACME/blocked/suspended. Run it after touching the splash block, the routing ACL templates, or the domain loops in `generate_config()`. See README "Default Splash".
+- **Access log receiver**: `python3 scripts/test-access-log-receiver.py` - offline (runs in the image build); one datagram = one line (PRI stripped, control chars neutralised), size cap + gzip generations, write errors drop without raising, real UNIX-socket path. Run it after touching `haproxy-log-receiver.py` or the `log` line in `hap_header.tpl`.
 - **Manual Testing**: Run `curl` commands against `http://localhost:8000` endpoints as shown in README.md
 
 ### Reading stick tables (and why it is easy to get silently wrong)
@@ -28,8 +30,10 @@ columns; the first token is an allocation pointer (`0x...:`), not the key. Parse
 by NAME, and treat a missing field as an ERROR — never default it to `0`. The
 `web` table stores only `conn_cur`, `conn_rate`, `http_req_rate`,
 `http_err_rate`; it holds **no history and no counter of past blocks**. What was
-actually denied/tarpitted is in the edge access log on the **host** at
+actually denied/tarpitted is in the edge access log, since 2026.11.2 INSIDE the
-`/var/log/haproxy.log` (shipped 2026.08.8), not in any stick table.
+container at `/var/log/haproxy/access.log` (in-container receiver, volume
+`haproxy-logs`; README "Access Log"), not in any stick table. (2026.08.8-2026.11.1
+sent it to host rsyslog on 172.18.0.1:514, which had no listener on the fleet.)
 
 This is written down because `/api/security/stats` and `show-tarpit-ips.sh`
 reported "Scan Count"/"BLOCKED" figures parsed from `gpc0`/`gpc1` — fields no

Dockerfile

@@ -105,6 +105,8 @@ RUN pip install -r requirements.txt
 RUN python3 /haproxy/scripts/test-wpadmin-gate.py \
  && python3 /haproxy/scripts/test-trusted-proxy-gate.py \
  && python3 /haproxy/scripts/test-xmlrpc-rate-limit.py \
+ && python3 /haproxy/scripts/test-default-splash.py \
+ && python3 /haproxy/scripts/test-access-log-receiver.py \
  && python3 /haproxy/scripts/test-config-rollback.py \
  && python3 /haproxy/scripts/test-cert-write-safety.py \
  && python3 /haproxy/scripts/test-cert-scripts.py \
@@ -112,7 +114,7 @@ RUN python3 /haproxy/scripts/test-wpadmin-gate.py \
  && python3 /haproxy/scripts/test-gunicorn-socket-timeouts.py \
  && python3 /haproxy/scripts/validate-rendered-config.py
 # Create log directories
-RUN mkdir -p /var/log && touch /var/log/haproxy-manager.log /var/log/haproxy-manager-errors.log
+RUN mkdir -p /var/log/haproxy && touch /var/log/haproxy-manager.log /var/log/haproxy-manager-errors.log
 RUN chmod 755 /var/log/haproxy-manager.log /var/log/haproxy-manager-errors.log
 # Set up cron for certificate renewal with proper permissions and environment
 RUN mkdir -p /var/spool/cron/crontabs && \

README.md

@@ -379,9 +379,123 @@ tail -f /var/log/haproxy-manager-errors.log
 | `HAPROXY_DEFAULT_PAGE_TITLE` | Title for the default page | Site Not Configured |
 | `HAPROXY_DEFAULT_MAIN_MESSAGE` | Main message on the default page | This domain has not been configured yet. Please contact your system administrator to set up this website. |
 | `HAPROXY_DEFAULT_SECONDARY_MESSAGE` | Secondary message on the default page | If you believe this is an error, please check the domain name and try again. |
+| `HAPROXY_DEFAULT_SPLASH` | `off` (or `0`/`false`/`no`) disables the default splash for unrouted hosts and restores the default-backend page below | on |
+
+## Default Splash (unrouted hosts)
+
+Since 2026.11.1 (404-everywhere + robots since 2026.11.2), any request whose Host is **not routed to a site** -- the
+server's own hostname (registered cert-only, no backend), a bare IP, a stale
+or unknown name, a request with no Host at all -- is answered by HAProxy
+itself with the static page `errors/default-splash.html` (`http-request
+return ... file`): `404` on EVERY path (including `/`) so search engines never
+index the server's IP or hostname as a site, with `cache-control: no-store`,
+`x-robots-tag: noindex, nofollow, noarchive` and `x-content-type-options:
+nosniff`; `/robots.txt` is `200 text/plain` `User-agent: *` / `Disallow: /`
+(`errors/robots-unrouted.txt`). Monitors pointed at the bare server hostname
+on 80/443 must accept 404 (WHP's own probes -- healthcheck.sh, the watchdog --
+accept any status). The page
+names no version, echoes nothing from the request (it is loaded with `file`,
+not `lf-file`), and links nowhere -- in particular not to the control panel.
+
+**Why.** `use_backend default-backend` does not short-circuit HAProxy's
+`http-request` rules, so the old catch-all was inspected by Coraza with the
+full server-wide rule set like customer traffic. A browser on
+`https://<server-hostname>/` also sends the control panel's cookies (cookies
+are not port-scoped, so `:8443` cookies go to `:443`), and anything Coraza
+matched there showed the WAF block page for a host with no site behind it.
+The splash rules sit before the SPOE call, so unrouted requests never reach
+the WAF; there is nothing behind them to protect.
+
+**What still applies / is kept out:** rate limits, the IP block (blocked IPs
+still get the blocked-ip page), the wp-* gates, and the multipart guard run
+first; `/.well-known/acme-challenge/` still reaches certbot (the server
+hostname's own certificate is issued through it); hosts in
+`suspended_domains.list` still get the suspended page. Configured sites,
+wildcards and their WAF inspection are unchanged. Plain HTTP on `:80` gets
+the splash too, not a redirect: an unknown name usually has no valid
+certificate, so a redirect would trade a clean page for a TLS warning.
+Over TLS the presented certificate is whatever SNI selects -- the
+hostname's own cert if one is issued, otherwise the default bundle.
+
+**How "routed" is decided.** `generate_config()` writes
+`/etc/haproxy/routed_hosts.list` (exact names) and
+`/etc/haproxy/routed_wildcards.list` (`.base` suffixes) from the same domain
+rows, in the same call, as the routing ACLs, and both are in the rollback set
+next to `haproxy.cfg`. The splash ACLs compare the Host port-stripped and
+lowercased, a superset of the routing ACLs (which compare the raw Host), so a
+host that routes is never splashed. Never hand-edit the lists; they are
+rewritten on every regeneration (also when the splash is off, so turning it
+on never meets a missing `-f` file).
+
+**Deploying to an existing host.** The page ships at
+`/haproxy/errors/default-splash.html` and the template under
+`/haproxy/templates/` -- both in the image, OUTSIDE the `/etc/haproxy` named
+volume, so the volume cannot shadow them. The lists live in the volume but are
+generated, not copied. So an image bump plus a recreate of the container is
+the whole deployment: on start `generate_config()` renders the new listener
+and writes both lists, then HAProxy loads them. Nothing to copy into the
+volume, no WHP-side change. Verify after the recreate:
+
+```bash
+docker exec haproxy-manager grep -c default-splash.html /etc/haproxy/haproxy.cfg   # 2
+docker exec haproxy-manager cat /etc/haproxy/routed_hosts.list                     # every site
+curl -skI https://<server-hostname>/                       # 404 + x-robots-tag noindex
+curl -sk https://<server-hostname>/robots.txt              # User-agent: * / Disallow: /
+curl -sk -o /dev/null -w '%{http_code}\n' https://<a-customer-site>/                 # unchanged
+```
+
+Rollback: recreate on the previous image tag, or set
+`HAPROXY_DEFAULT_SPLASH=off` and regenerate.
+
+Covered by `scripts/test-default-splash.py` (template order/conditions, the
+page, list generation) and the `splash-off` scenario of
+`scripts/validate-rendered-config.py`.
+
+## Access Log (in-container, since 2026.11.2)
+
+HAProxy logs (`log /log.sock ... format rfc5424`, `hap_header.tpl`) to a UNIX
+datagram socket inside its chroot (`/var/lib/haproxy/log.sock`).
+`scripts/haproxy-log-receiver.py`, started and supervised by `start-up.sh` (a
+restart loop: back within ~2 s), writes one line per request to
+`/var/log/haproxy/access.log`, rotated at 40 MB with 4 gzip generations
+(worst case 200 MB; `HAPROXY_ACCESS_LOG_MAX_BYTES`, `HAPROXY_ACCESS_LOG_BACKUPS`).
+No network listener anywhere. Fields: client/real IP (`cip=`), Host, status,
+termination state, request id, UA, SNI, HTTP version, WAF body stamp and the
+Coraza `wrule=/wrules=/wscore=` fields (unchanged `log-format`).
+
+* **Never blocks HAProxy.** Sends are non-blocking: a dead or stalled receiver,
+  or a full disk, only drops log lines (counted as `DroppedLogs` in
+  `echo "@1 show info" | socat stdio /tmp/haproxy-cli`). Lab: with the
+  receiver SIGSTOPped, 2500/2500 requests still answered 200 in ~1 s;
+  1025 lines queued, 1475 dropped and counted.
+* **Not in `docker logs`.** The receiver prints no log lines (WHP's AI log
+  monitor reads `docker logs haproxy-manager`; a per-request firehose there
+  would storm it). Only its start line and rate-limited errors go to stderr.
+* **Persistence.** Mount a named volume at `/var/log/haproxy` (WHP:
+  `-v haproxy-logs:/var/log/haproxy`) or the log resets on every recreate.
+  Give the container `--sysctl net.unix.max_dgram_qlen=1024` (a netns starts
+  at 10) for burst headroom.
+* `HAPROXY_SYSLOG_TARGET=<ip>:<port>` still overrides the target for a remote
+  collector. The old default `172.18.0.1:514` (host rsyslog) is gone.
+
+Reading it:
+
+```bash
+docker exec haproxy-manager tail -f /var/log/haproxy/access.log
+docker exec haproxy-manager grep ' 403 ' /var/log/haproxy/access.log | tail
+docker exec haproxy-manager sh -c 'zcat /var/log/haproxy/access.log.*.gz' | grep <request-id>
+docker cp haproxy-manager:/var/log/haproxy/ ./edge-logs/    # or the volume's _data dir on the host
+```
+
+Rollout: image bump + recreate with the volume and sysctl (WHP's
+`lib-edge-container.sh` adds both). The receiver, its script and the template
+are in the image, outside the `/etc/haproxy` named volume, so nothing can be
+shadowed; `haproxy.cfg` is regenerated on start with the new `log` line.
 
 ## Default Backend Configuration
 
+With the default splash on (above), unrouted hosts no longer reach this page; it still serves the blocked-IP and suspended pages, and serves unrouted hosts again when `HAPROXY_DEFAULT_SPLASH=off`.
+
 When a domain is accessed that hasn't been configured in HAProxy, the system will serve a default page instead of showing an error. This default page:
 
 - Informs visitors that the site is not configured

VERSION

@@ -1 +1 @@
-2026.10.7
+2026.11.2

errors/default-splash.html

@@ -0,0 +1,34 @@
+<!DOCTYPE html>
+<!--
+  Default splash: served by HAProxy itself (`http-request return ... file`,
+  hap_listener.tpl) for any request whose Host is not routed to a site --
+  the server's own hostname, a bare IP, an unconfigured name. Static on
+  purpose: no version, no hostname echo, no customer data, no links (in
+  particular none into the control panel). Loaded with `file`, not
+  `lf-file`, so `%` is literal here and nothing from the request is
+  expanded into it. Must stay well under tune.bufsize (16 KB) or HAProxy
+  rejects the config.
+-->
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<meta name="robots" content="noindex, nofollow">
+<title>Web Hosting Platform</title>
+<style>
+:root{color-scheme:light dark;--bg:#f4f5f7;--card:#fff;--fg:#1d2330;--muted:#5b6475;--line:#dfe3ea}
+@media (prefers-color-scheme:dark){:root{--bg:#0f1217;--card:#181c23;--fg:#e6e9ef;--muted:#9aa3b2;--line:#2a303a}}
+html,body{margin:0;height:100%}
+body{display:flex;align-items:center;justify-content:center;background:var(--bg);color:var(--fg);font:16px/1.5 system-ui,-apple-system,"Segoe UI",Roboto,sans-serif;padding:16px;box-sizing:border-box}
+main{background:var(--card);border:1px solid var(--line);border-radius:10px;padding:32px 28px;max-width:480px;width:100%;text-align:center}
+h1{font-size:1.25rem;margin:0 0 .5rem}
+p{margin:0;color:var(--muted)}
+</style>
+</head>
+<body>
+<main>
+<h1>Web Hosting Platform</h1>
+<p>This server is operated by a web hosting platform. There is no website configured at this address.</p>
+</main>
+</body>
+</html>

errors/robots-unrouted.txt

@@ -0,0 +1,2 @@
+User-agent: *
+Disallow: /

haproxy_manager.py

@@ -135,6 +135,16 @@ BLOCKED_IPS_MAP_BACKUP_PATH = '/etc/haproxy/blocked_ips.map.backup'
 # while leaving a broken coraza-spoe.cfg behind still fails validation.
 CORAZA_SPOE_CONFIG_PATH = '/etc/haproxy/coraza-spoe.cfg'
 CORAZA_SPOE_BACKUP_PATH = '/etc/haproxy/coraza-spoe.cfg.backup'
+# Hosts the frontend routes to a site (default splash, hap_listener.tpl). The
+# paths are hardcoded in the template's `-f` ACLs, like blocked_ips.map, and
+# written by generate_config() from the same rows as the routing ACLs.
+# HAProxy's access-log target: the receiver's socket, CHROOT-RELATIVE
+# (hap_header.tpl `chroot /var/lib/haproxy`), i.e. /var/lib/haproxy/log.sock.
+DEFAULT_LOG_TARGET = '/log.sock'
+ROUTED_HOSTS_PATH = '/etc/haproxy/routed_hosts.list'
+ROUTED_HOSTS_BACKUP_PATH = '/etc/haproxy/routed_hosts.list.backup'
+ROUTED_WILDCARDS_PATH = '/etc/haproxy/routed_wildcards.list'
+ROUTED_WILDCARDS_BACKUP_PATH = '/etc/haproxy/routed_wildcards.list.backup'
 HAPROXY_SOCKET_PATH = '/var/run/haproxy.sock'
 # HAProxy loads this path as a DIRECTORY (`bind ... ssl crt /etc/haproxy/certs`),
 # which means it tries to load EVERY file it finds in here. Nothing but final,
@@ -3138,15 +3148,22 @@ def generate_config():
                 except Exception as e:
                     logger.error(f"Failed to create {suspended_list_path}: {e}")
 
-        # Access-log destination for the `log` line in the global section.
+        # Access-log destination for the `log` line in the global section:
-        # Default 172.18.0.1:514 is the docker bridge gateway for WHP's
+        # the in-container receiver's UNIX socket (scripts/haproxy-log-receiver.py),
-        # `client-net`, i.e. the host, where rsyslog's imudp listener is bound
+        # given relative to haproxy's chroot (/var/lib/haproxy) -- see
-        # by setup-haproxy-logrotate.sh. Overridable so this image stays usable
+        # hap_header.tpl. HAPROXY_SYSLOG_TARGET overrides it (`<ip>:<port>` for
-        # on standalone/home deployments with a different bridge subnet or a
+        # a remote collector). The old default, 172.18.0.1:514 (host rsyslog on
-        # remote log collector -- set HAPROXY_SYSLOG_TARGET to `<ip>:<port>`.
+        # the docker bridge), had no listener on the fleet.
-        # UDP, so an absent listener drops log lines and never affects request
+        syslog_target = (os.environ.get('HAPROXY_SYSLOG_TARGET', '').strip()
-        # handling.
+                         or DEFAULT_LOG_TARGET)
-        syslog_target = os.environ.get('HAPROXY_SYSLOG_TARGET', '172.18.0.1:514').strip()
+
+        # Default splash for unrouted hosts (server hostname, bare IP, unknown
+        # names) answered by HAProxy before the WAF -- see hap_listener.tpl.
+        # ON by default: it depends on no sidecar and only changes what an
+        # unrouted request sees. HAPROXY_DEFAULT_SPLASH=off restores the old
+        # default-backend page (WAF-inspected).
+        splash_raw = os.environ.get('HAPROXY_DEFAULT_SPLASH', '').strip().lower()
+        default_splash = splash_raw not in ('0', 'false', 'no', 'off')
 
         # Add Haproxy Default Headers
         default_headers = template_env.get_template('hap_header.tpl').render(
@@ -3168,6 +3185,7 @@ def generate_config():
             crt_path = SSL_CERTS_DIR,
             coraza_spoe_backend = coraza_spoe_backend,
             suspension_enabled = suspension_enabled,
+            default_splash = default_splash,
         )
         config_parts.append(listener_block)
 
@@ -3176,6 +3194,9 @@ def generate_config():
         config_parts.append(letsencrypt_acl)
         config_acls = []
         config_backends = []
+        # Every name a routing ACL is rendered for (default splash lists).
+        routed_hosts = []
+        routed_wildcards = []
 
         # Add default backend rule (will be used when no domain matches)
         default_rule = "    # Default backend for unmatched domains\n    default_backend default-backend\n"
@@ -3231,6 +3252,7 @@ def generate_config():
                     name=domain['backend_name']
                 )
                 config_acls.append(domain_acl)
+                routed_hosts.append(domain['domain'])
                 logger.info(f"Added ACL for domain: {domain['domain']}")
             except Exception as e:
                 logger.error(f"Error generating domain ACL for {domain['domain']}: {e}")
@@ -3257,6 +3279,7 @@ def generate_config():
                     base_domain=base_domain
                 )
                 config_acls.append(domain_acl)
+                routed_wildcards.append('.' + base_domain)
                 logger.info(f"Added wildcard ACL for domain: {domain['domain']}")
             except Exception as e:
                 logger.error(f"Error generating wildcard ACL for {domain['domain']}: {e}")
@@ -3318,6 +3341,14 @@ backend default-backend
                         f"{CORAZA_SPOE_CONFIG_PATH} "
                         f"(SPOA target: {coraza_spoe_backend})")
 
+        # Routed-host lists for the default splash. Written on every render
+        # (also with the splash off, so turning it on later never meets a
+        # missing `-f` file, which is FATAL). Same render, same rows as the
+        # routing ACLs above, and in the rollback set, so the loaded
+        # haproxy.cfg and these lists always come from one generation.
+        write_config_atomically(ROUTED_HOSTS_PATH, render_routed_list(routed_hosts))
+        write_config_atomically(ROUTED_WILDCARDS_PATH, render_routed_list(routed_wildcards))
+
         config_content = '\n'.join(config_parts)
         logger.debug("Generated HAProxy configuration")
 
@@ -3401,6 +3432,28 @@ def _config_set_matches_backup():
     return True
 
 
+def render_routed_list(names):
+    """Pattern-file body for the default-splash routed-host ACLs.
+
+    One lowercased name per line, sorted and de-duplicated so an unchanged
+    fleet renders byte-identical files (create_backup() skips re-validation
+    when the live set equals the backup). A name containing whitespace cannot
+    be one pattern line (a newline would split it in two) and could not be a
+    working `hdr(host) -i` routing ACL either; it is dropped and logged at
+    ERROR, because a missing entry means the splash answers that host.
+    """
+    out = set()
+    for name in names:
+        clean = (name or '').strip().lower()
+        if not clean or any(c.isspace() for c in clean):
+            logger.error(f"default splash: cannot list routed host {name!r}; "
+                         "it will be answered by the splash page")
+            continue
+        out.add(clean)
+    body = ''.join(f'{n}\n' for n in sorted(out))
+    return '# Generated by haproxy_manager.generate_config() - do not edit\n' + body
+
+
 def _config_backup_pairs():
     """(live, backup) pairs forming one restorable config set.
 
@@ -3411,6 +3464,8 @@ def _config_backup_pairs():
         (HAPROXY_CONFIG_PATH, HAPROXY_BACKUP_PATH),
         (BLOCKED_IPS_MAP_PATH, BLOCKED_IPS_MAP_BACKUP_PATH),
         (CORAZA_SPOE_CONFIG_PATH, CORAZA_SPOE_BACKUP_PATH),
+        (ROUTED_HOSTS_PATH, ROUTED_HOSTS_BACKUP_PATH),
+        (ROUTED_WILDCARDS_PATH, ROUTED_WILDCARDS_BACKUP_PATH),
     )
 
 

scripts/haproxy-log-receiver.py

@@ -0,0 +1,264 @@
+#!/usr/bin/env python3
+"""In-container receiver for the HAProxy access log.
+
+HAProxy sends its access log (`log <socket> ... format rfc5424`, hap_header.tpl)
+as datagrams to a UNIX socket inside this container. This process is the only
+reader: it writes one line per datagram to a size-capped, rotating file in the
+haproxy-logs volume (default /var/log/haproxy/access.log).
+
+Why it looks the way it does
+----------------------------
+* UNIX DATAGRAM socket, no network listener. Nothing is exposed on the host
+  or on client-net (the old target was the host's rsyslog on the docker
+  bridge, 172.18.0.1:514, which had no listener on the fleet).
+* HAProxy never waits for us. Its log sends are non-blocking: a dead receiver
+  (ECONNREFUSED/ENOENT) or a full socket buffer (EAGAIN) drops that line and
+  HAProxy carries on. So a crash, a stall or a full disk here can only lose
+  log lines, never slow or fail a request. That is the intended direction.
+* Bounded on disk: the live file is rotated at MAX_BYTES and BACKUPS gzip
+  generations are kept, so the worst case (incompressible data) is
+  (1 + BACKUPS) * MAX_BYTES -- 5 * 40 MB = 200 MB by default. Compression runs
+  in a thread so a rotation does not stall reading.
+* A full disk (ENOSPC) or any write error drops lines and is reported at most
+  once a minute; the file is reopened on the next attempt.
+* Lines go to the FILE ONLY, never stdout: `docker logs haproxy-manager` is a
+  source of WHP's AI log monitor, and a per-request firehose there would storm
+  it (see the WHP note on audit/firehose sources). Our own status messages are
+  rare and go to stderr.
+* Control characters inside a datagram are replaced so one datagram is always
+  exactly one line (a crafted header cannot forge extra log lines).
+
+Supervised by start-up.sh (restart loop), started before HAProxy.
+
+Environment
+-----------
+  HAPROXY_LOG_SOCKET       socket path   (default /var/lib/haproxy/log.sock)
+  HAPROXY_ACCESS_LOG       file path     (default /var/log/haproxy/access.log)
+  HAPROXY_ACCESS_LOG_MAX_BYTES  rotate size (default 41943040 = 40 MiB)
+  HAPROXY_ACCESS_LOG_BACKUPS    gzip generations kept (default 4)
+"""
+
+import gzip
+import os
+import shutil
+import socket
+import sys
+import threading
+import time
+
+# HAProxy sends at most `len` (2048) bytes per line; leave headroom.
+MAX_DATAGRAM = 65535
+FLUSH_INTERVAL = 1.0
+ERROR_REPORT_INTERVAL = 60.0
+# Kernel receive buffer to absorb bursts while a write is slow.
+RCVBUF = 4 * 1024 * 1024
+
+_CTRL = {c: ' ' for c in range(0x20) if c != 0x09}
+_CTRL[0x7f] = ' '
+
+
+def _env_int(name, default, minimum):
+    try:
+        value = int(os.environ.get(name, default))
+    except ValueError:
+        value = default
+    return max(minimum, value)
+
+
+def clean_line(datagram):
+    """One datagram -> one log line (str, no trailing newline).
+
+    Drops the RFC 5424 `<PRI>` prefix (constant for our single facility) and
+    replaces any control character so the datagram cannot span lines.
+    """
+    text = datagram.decode('utf-8', 'replace').rstrip('\r\n\x00')
+    if text.startswith('<'):
+        end = text.find('>', 1, 6)
+        if end > 0 and text[1:end].isdigit():
+            text = text[end + 1:]
+    return text.translate(_CTRL)
+
+
+class RotatingWriter:
+    """Size-capped writer: live file + `backups` gzip generations."""
+
+    def __init__(self, path, max_bytes, backups, report=None):
+        self.path = path
+        self.max_bytes = max_bytes
+        self.backups = backups
+        self.report = report or (lambda msg: None)
+        self.fh = None
+        self.size = 0
+        self.dirty = False
+        self.dropped = 0
+        self._compressor = None
+        self._last_error = 0.0
+
+    # -- file handling --------------------------------------------------
+    def _open(self):
+        os.makedirs(os.path.dirname(self.path) or '.', exist_ok=True)
+        self.fh = open(self.path, 'a', encoding='utf-8', buffering=65536)
+        os.chmod(self.path, 0o640)
+        self.size = self.fh.tell()
+
+    def _error(self, what, exc):
+        self.dropped += 1
+        try:
+            if self.fh:
+                self.fh.close()
+        except OSError:
+            pass
+        self.fh = None
+        now = time.monotonic()
+        if now - self._last_error >= ERROR_REPORT_INTERVAL:
+            self._last_error = now
+            self.report(f'{what} failed ({exc}); dropping access-log lines '
+                        f'(dropped so far: {self.dropped})')
+
+    def _gen(self, n):
+        return f'{self.path}.{n}.gz'
+
+    def _compress(self, src, dst):
+        tmp = dst + '.tmp'
+        try:
+            with open(src, 'rb') as fin, gzip.open(tmp, 'wb', compresslevel=6) as fout:
+                shutil.copyfileobj(fin, fout, 1024 * 1024)
+            os.replace(tmp, dst)
+            os.chmod(dst, 0o640)
+        except OSError as exc:
+            self.report(f'compressing {src} failed ({exc}); removing it to keep the size cap')
+            for leftover in (tmp,):
+                try:
+                    os.unlink(leftover)
+                except OSError:
+                    pass
+        finally:
+            try:
+                os.unlink(src)
+            except OSError:
+                pass
+
+    def rotate(self):
+        if self._compressor is not None:
+            self._compressor.join()
+            self._compressor = None
+        if self.fh:
+            try:
+                self.fh.close()
+            except OSError:
+                pass
+            self.fh = None
+        if self.backups <= 0:
+            try:
+                os.unlink(self.path)
+            except OSError:
+                pass
+        else:
+            try:
+                os.unlink(self._gen(self.backups))
+            except OSError:
+                pass
+            for n in range(self.backups - 1, 0, -1):
+                if os.path.exists(self._gen(n)):
+                    os.replace(self._gen(n), self._gen(n + 1))
+            staged = f'{self.path}.1'
+            if os.path.exists(self.path):
+                os.replace(self.path, staged)
+                self._compressor = threading.Thread(
+                    target=self._compress, args=(staged, self._gen(1)), daemon=True)
+                self._compressor.start()
+        self.size = 0
+
+    # -- public ---------------------------------------------------------
+    def write(self, line):
+        data = line + '\n'
+        try:
+            if self.fh is None:
+                self._open()
+            if self.size and self.size + len(data) > self.max_bytes:
+                self.rotate()
+                self._open()
+            self.fh.write(data)
+            self.size += len(data.encode('utf-8'))
+            self.dirty = True
+        except OSError as exc:
+            self._error('writing ' + self.path, exc)
+
+    def flush(self):
+        if self.fh and self.dirty:
+            try:
+                self.fh.flush()
+                self.dirty = False
+            except OSError as exc:
+                self._error('flushing ' + self.path, exc)
+
+    def close(self):
+        self.flush()
+        if self._compressor is not None:
+            self._compressor.join()
+        if self.fh:
+            try:
+                self.fh.close()
+            except OSError:
+                pass
+
+
+def bind_socket(path):
+    sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+    try:
+        sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, RCVBUF)
+    except OSError:
+        pass
+    os.makedirs(os.path.dirname(path) or '.', exist_ok=True)
+    try:
+        os.unlink(path)  # stale socket from a previous (crashed) receiver
+    except FileNotFoundError:
+        pass
+    sock.bind(path)
+    # HAProxy's workers run as user haproxy (hap_header.tpl `user haproxy`).
+    os.chmod(path, 0o666)
+    sock.settimeout(FLUSH_INTERVAL)
+    return sock
+
+
+def serve(sock, writer, stop=None):
+    last_flush = time.monotonic()
+    while stop is None or not stop.is_set():
+        try:
+            datagram = sock.recv(MAX_DATAGRAM)
+        except socket.timeout:
+            datagram = None
+        except OSError:
+            if stop is not None and stop.is_set():
+                break
+            raise
+        if datagram:
+            writer.write(clean_line(datagram))
+        now = time.monotonic()
+        if now - last_flush >= FLUSH_INTERVAL:
+            writer.flush()
+            last_flush = now
+
+
+def main():
+    sock_path = os.environ.get('HAPROXY_LOG_SOCKET', '/var/lib/haproxy/log.sock')
+    log_path = os.environ.get('HAPROXY_ACCESS_LOG', '/var/log/haproxy/access.log')
+    max_bytes = _env_int('HAPROXY_ACCESS_LOG_MAX_BYTES', 40 * 1024 * 1024, 64 * 1024)
+    backups = _env_int('HAPROXY_ACCESS_LOG_BACKUPS', 4, 0)
+
+    def report(msg):
+        sys.stderr.write(f'haproxy-log-receiver: {msg}\n')
+        sys.stderr.flush()
+
+    writer = RotatingWriter(log_path, max_bytes, backups, report)
+    sock = bind_socket(sock_path)
+    report(f'listening on {sock_path} -> {log_path} '
+           f'(rotate at {max_bytes} bytes, keep {backups} gz)')
+    try:
+        serve(sock, writer)
+    finally:
+        writer.close()
+
+
+if __name__ == '__main__':
+    main()

scripts/monitor-attacks.sh

@@ -20,13 +20,13 @@
 #   against.
 #
 # Usage: monitor-attacks.sh [live]
-# Env:   LOG_FILE=<path>  access log to read (default /var/log/haproxy.log)
+# Env:   LOG_FILE=<path>  access log to read (default /var/log/haproxy/access.log)
 #        LOG_LINES=<n>    how many trailing log lines to scan (default 500)
 
 set -uo pipefail
 
 SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
-LOG_FILE="${LOG_FILE:-/var/log/haproxy.log}"
+LOG_FILE="${LOG_FILE:-/var/log/haproxy/access.log}"
 LOG_LINES="${LOG_LINES:-500}"
 
 # --- Section 1: current rates (real stick-table data) -----------------------
@@ -47,12 +47,12 @@ show_recent_blocks() {
   syslog on the DOCKER HOST, and the file lives on the host, not in here.
   Read it from the host instead:
 
-    grep -aE ' (PT|PR)--' /var/log/haproxy.log | tail -20   # tarpit / deny
+    grep -aE ' (PT|PR)--' /var/log/haproxy/access.log | tail -20   # tarpit / deny
-    grep -aE ' (429|403) ' /var/log/haproxy.log | tail -20  # rate-limited / blocked
+    grep -aE ' (429|403) ' /var/log/haproxy/access.log | tail -20  # rate-limited / blocked
-    grep -a 'cip=<IP>'  /var/log/haproxy.log | tail -50     # one client IP
+    grep -a 'cip=<IP>'  /var/log/haproxy/access.log | tail -50     # one client IP
-    grep -a 'id=<uuid>' /var/log/haproxy.log                # one request reference
+    grep -a 'id=<uuid>' /var/log/haproxy/access.log                # one request reference
                                                             # (the UUID on the block page)
-    tail -f /var/log/haproxy.log | grep -aE ' (PT|PR)--'    # live
+    tail -f /var/log/haproxy/access.log | grep -aE ' (PT|PR)--'    # live
 
   Or point this script at a copy:  LOG_FILE=/path/to/haproxy.log $0
 MSG

scripts/show-tarpit-ips.sh

@@ -12,9 +12,9 @@
 #
 # The stick table also keeps NO history, so nothing in it can identify who was
 # tarpitted. Real enforcement events live in the access log ON THE DOCKER HOST
-# at /var/log/haproxy.log (it does not exist inside this container):
+# at /var/log/haproxy/access.log (inside this container since 2026.11.2):
-#   grep -aE ' (PT|PR)--' /var/log/haproxy.log | tail -20   # tarpit / deny
+#   grep -aE ' (PT|PR)--' /var/log/haproxy/access.log | tail -20   # tarpit / deny
-#   grep -aE ' (429|403) ' /var/log/haproxy.log | tail -20  # rate-limit / block
+#   grep -aE ' (429|403) ' /var/log/haproxy/access.log | tail -20  # rate-limit / block
 #
 # What IS knowable from the stick table — the current per-IP rates — is printed
 # by show-edge-ip-rates.sh, which this shim now runs.
@@ -25,8 +25,8 @@ cat >&2 <<'NOTE'
 NOTE: show-tarpit-ips.sh is deprecated and cannot report tarpits.
       The HAProxy stick table stores no history and no gpc0/gpc1 counters, so
       the old "Scan Count"/"BLOCKED" columns were fabricated numbers.
-      Actual tarpit/deny events are in /var/log/haproxy.log ON THE HOST:
+      Actual tarpit/deny events are in /var/log/haproxy/access.log (in this container since 2026.11.2):
-        grep -aE ' (PT|PR)--' /var/log/haproxy.log | tail -20
+        grep -aE ' (PT|PR)--' /var/log/haproxy/access.log | tail -20
       Running show-edge-ip-rates.sh instead (current rates, real values):
 
 NOTE

scripts/start-up.sh

@@ -55,6 +55,24 @@ if [ ! -f /etc/haproxy/wpadmin_gate_exempt.list ]; then
     fi
 fi
 
+# HAProxy access log receiver (see scripts/haproxy-log-receiver.py and the
+# `log` line in hap_header.tpl). HAProxy sends datagrams to a UNIX socket
+# inside its chroot; this process writes them to a size-capped, rotating
+# /var/log/haproxy/access.log (WHP mounts the haproxy-logs volume there).
+# Started BEFORE init.py so haproxy's first lines land. Supervised by a
+# restart loop: if it dies it is back within 2s, and while it is down HAProxy
+# just drops log lines (non-blocking sends), it never blocks on them.
+# stdout is discarded on purpose -- it never prints log lines, and
+# `docker logs` must not carry a per-request firehose (WHP's AI monitor reads
+# it); its rare status/error lines go to stderr.
+mkdir -p /var/log/haproxy /var/lib/haproxy
+(
+    while true; do
+        python /haproxy/scripts/haproxy-log-receiver.py >/dev/null || true
+        sleep 2
+    done
+) &
+
 cron &
 
 # Phase 1: container init

scripts/test-access-log-receiver.py

@@ -0,0 +1,178 @@
+#!/usr/bin/env python3
+"""Tests for scripts/haproxy-log-receiver.py (in-container access log).
+
+Pins: one datagram == one line (PRI stripped, control chars neutralised),
+the size cap (live file + N gzip generations, oldest dropped), a write error
+(full disk / unwritable dir) drops lines without raising, and an end-to-end
+datagram path through a real UNIX socket.
+
+Running
+-------
+    python3 scripts/test-access-log-receiver.py
+"""
+
+import gzip
+import importlib.util
+import os
+import shutil
+import socket
+import tempfile
+import threading
+import time
+import unittest
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+_spec = importlib.util.spec_from_file_location(
+    'haproxy_log_receiver', os.path.join(HERE, 'haproxy-log-receiver.py'))
+rx = importlib.util.module_from_spec(_spec)
+_spec.loader.exec_module(rx)
+
+
+class CleanLine(unittest.TestCase):
+
+    def test_strips_pri_and_trailing_newline(self):
+        self.assertEqual(rx.clean_line(b'<150>1 2026-11-08T00:00:00Z h haproxy 7 - - GET /\n'),
+                         '1 2026-11-08T00:00:00Z h haproxy 7 - - GET /')
+
+    def test_embedded_newlines_cannot_forge_lines(self):
+        out = rx.clean_line(b'<150>1 x ua=evil\r\nFAKE 200 line\x1b[2J')
+        self.assertNotIn('\n', out)
+        self.assertNotIn('\r', out)
+        self.assertNotIn('\x1b', out)
+        self.assertIn('FAKE 200 line', out)
+
+    def test_invalid_utf8_does_not_raise(self):
+        self.assertIn('�', rx.clean_line(b'<150>1 \xff\xfe'))
+
+    def test_non_pri_angle_bracket_kept(self):
+        self.assertEqual(rx.clean_line(b'<html> x'), '<html> x')
+
+
+class Rotation(unittest.TestCase):
+
+    def setUp(self):
+        self.dir = tempfile.mkdtemp(prefix='hlr-test-')
+        self.path = os.path.join(self.dir, 'access.log')
+
+    def tearDown(self):
+        shutil.rmtree(self.dir, ignore_errors=True)
+
+    def _files(self):
+        return sorted(os.listdir(self.dir))
+
+    def test_size_cap_and_generations(self):
+        max_bytes, backups = 4096, 3
+        w = rx.RotatingWriter(self.path, max_bytes, backups)
+        line = 'x' * 99  # 100 bytes with newline
+        for i in range(1000):  # ~100 KB -> ~24 rotations
+            w.write(f'{i:05d}' + line[5:])
+        w.close()
+        files = self._files()
+        self.assertEqual(files, ['access.log', 'access.log.1.gz',
+                                 'access.log.2.gz', 'access.log.3.gz'], files)
+        self.assertLessEqual(os.path.getsize(self.path), max_bytes)
+        total = sum(os.path.getsize(os.path.join(self.dir, f)) for f in files)
+        self.assertLessEqual(total, (1 + backups) * max_bytes)
+        # newest generation holds the lines written just before the live file
+        with gzip.open(os.path.join(self.dir, 'access.log.1.gz'), 'rt') as fh:
+            gen1 = fh.read().splitlines()
+        with open(self.path) as fh:
+            live = fh.read().splitlines()
+        self.assertEqual(int(gen1[-1][:5]) + 1, int(live[0][:5]))
+        self.assertEqual(live[-1][:5], '00999')
+        for l in gen1:
+            self.assertEqual(len(l), 99)
+
+    def test_zero_backups_keeps_only_live_file(self):
+        w = rx.RotatingWriter(self.path, 1024, 0)
+        for _ in range(100):
+            w.write('y' * 63)
+        w.close()
+        self.assertEqual(self._files(), ['access.log'])
+        self.assertLessEqual(os.path.getsize(self.path), 1024)
+
+    def test_reopens_existing_file_and_counts_its_size(self):
+        with open(self.path, 'w') as fh:
+            fh.write('z' * 1000 + '\n')
+        w = rx.RotatingWriter(self.path, 1500, 2)
+        w.write('a' * 600)  # 1001 + 601 > 1500 -> rotate first
+        w.close()
+        self.assertIn('access.log.1.gz', self._files())
+        with open(self.path) as fh:
+            self.assertEqual(fh.read(), 'a' * 600 + '\n')
+
+    def test_write_error_drops_and_never_raises(self):
+        blocker = os.path.join(self.dir, 'not-a-dir')
+        with open(blocker, 'w') as fh:
+            fh.write('')
+        msgs = []
+        w = rx.RotatingWriter(os.path.join(blocker, 'access.log'), 4096, 2, msgs.append)
+        for _ in range(50):
+            w.write('line')
+        w.flush()
+        w.close()
+        self.assertEqual(w.dropped, 50)
+        self.assertEqual(len(msgs), 1, msgs)  # rate-limited report
+
+
+class EndToEnd(unittest.TestCase):
+
+    def test_datagrams_become_lines(self):
+        d = tempfile.mkdtemp(prefix='hlr-e2e-')
+        try:
+            sock_path = os.path.join(d, 'log.sock')
+            log_path = os.path.join(d, 'logs', 'access.log')
+            sock = rx.bind_socket(sock_path)
+            self.assertEqual(os.stat(sock_path).st_mode & 0o777, 0o666)
+            w = rx.RotatingWriter(log_path, 1 << 20, 2)
+            stop = threading.Event()
+            t = threading.Thread(target=rx.serve, args=(sock, w, stop), daemon=True)
+            t.start()
+            cl = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+            cl.setblocking(False)
+            for i in range(20):
+                msg = f'<150>1 t h haproxy 1 - - req {i} host=a.example cip=192.0.2.1'.encode()
+                # The default net.unix.max_dgram_qlen is 10 per netns: a full
+                # queue is EAGAIN (HAProxy drops the line); here, retry.
+                for _ in range(100):
+                    try:
+                        cl.sendto(msg, sock_path)
+                        break
+                    except BlockingIOError:
+                        time.sleep(0.01)
+            deadline = time.time() + 5
+            lines = []
+            while time.time() < deadline:
+                time.sleep(0.2)
+                w.flush()
+                if os.path.exists(log_path):
+                    with open(log_path) as fh:
+                        lines = fh.read().splitlines()
+                if len(lines) == 20:
+                    break
+            stop.set()
+            t.join(3)
+            sock.close()
+            w.close()
+            self.assertEqual(len(lines), 20, lines)
+            self.assertTrue(lines[0].startswith('1 t h haproxy 1 - - req 0 host=a.example'))
+        finally:
+            shutil.rmtree(d, ignore_errors=True)
+
+    def test_sender_to_dead_receiver_fails_fast_nonblocking(self):
+        """What HAProxy sees when the receiver is down: an immediate error,
+        never a block (HAProxy then just counts a dropped log line)."""
+        d = tempfile.mkdtemp(prefix='hlr-dead-')
+        try:
+            cl = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+            cl.setblocking(False)
+            start = time.monotonic()
+            with self.assertRaises(OSError):
+                cl.sendto(b'x', os.path.join(d, 'log.sock'))
+            self.assertLess(time.monotonic() - start, 0.5)
+        finally:
+            shutil.rmtree(d, ignore_errors=True)
+
+
+if __name__ == '__main__':
+    unittest.main(verbosity=2)

scripts/test-default-splash.py

@@ -0,0 +1,304 @@
+#!/usr/bin/env python3
+"""Regression tests for the default splash (unrouted hosts answered pre-WAF).
+
+Why this file exists
+--------------------
+Visiting a WHP server's own hostname (https://whpNN.<domain>/ on 80/443, not
+the panel port), its bare IP, or any name with no site behind it used to fall
+through to `default_backend default-backend` -- but only AFTER every
+http-request rule had run, Coraza's send-spoe-group included. `use_backend`
+does not short-circuit http-request processing. So the catch-all was WAF
+inspected with the server-wide rule set, and a browser on the server hostname
+also carries the control panel's cookies (cookies are not port-scoped), which
+produced the WAF block page for a host that has nothing behind it to protect.
+
+The splash answers those requests with `http-request return` BEFORE the SPOE
+call. Its safety property is that it must NEVER answer a host that routes to
+a site. That is held by:
+  - routed_hosts.list / routed_wildcards.list being written from the same
+    rows, in the same generate_config() call, as the routing ACLs
+  - the lists being port-stripped + lowercased (a superset of the raw-Host
+    routing ACLs)
+  - the lists being part of the rollback set
+and it must keep ACME http-01, the blocked-IP page and the suspended page.
+
+Running
+-------
+    python3 scripts/test-default-splash.py
+"""
+
+import logging
+import os
+import re
+import sqlite3
+import sys
+import tempfile
+import unittest
+
+MODULE_DIR = os.path.abspath(
+    os.environ.get('HAPROXY_MANAGER_DIR',
+                   os.path.join(os.path.dirname(os.path.abspath(__file__)), '..'))
+)
+os.chdir(MODULE_DIR)
+sys.path.insert(0, MODULE_DIR)
+
+_LOG_DIR = tempfile.mkdtemp(prefix='haproxy-mgr-test-logs-')
+_real_file_handler = logging.FileHandler
+logging.FileHandler = (
+    lambda filename, *a, **kw: _real_file_handler(
+        os.path.join(_LOG_DIR, os.path.basename(filename)), *a, **kw)
+)
+try:
+    import haproxy_manager as hm  # noqa: E402
+finally:
+    logging.FileHandler = _real_file_handler
+logging.getLogger('haproxy_manager').setLevel(logging.CRITICAL)
+
+SPLASH_FILE = os.path.join(MODULE_DIR, 'errors', 'default-splash.html')
+
+
+def render_listener(**kw):
+    args = dict(crt_path='/etc/haproxy/certs', suspension_enabled=False,
+                coraza_spoe_backend='127.0.0.1:9000', default_splash=True)
+    args.update(kw)
+    return hm.template_env.get_template('hap_listener.tpl').render(**args)
+
+
+def return_rules(cfg):
+    return [l.strip() for l in cfg.splitlines()
+            if l.strip().startswith('http-request return')
+            and ('default-splash.html' in l or 'robots-unrouted.txt' in l)]
+
+
+class Template(unittest.TestCase):
+
+    def test_off_renders_nothing(self):
+        cfg = render_listener(default_splash=False)
+        self.assertNotIn('default-splash', cfg)
+        self.assertNotIn('splash_routed_host', cfg)
+
+    def test_404_everywhere_robots_txt_200(self):
+        """Josh 2026-11-08: unrouted hosts are 404 on EVERY path (incl. "/")
+        so search engines never index the server IP/hostname; /robots.txt is
+        the one 200 and disallows everything."""
+        rules = return_rules(render_listener())
+        self.assertEqual(len(rules), 2, rules)
+        robots, splash = rules
+        self.assertTrue(robots.startswith('http-request return status 200 content-type "text/plain'))
+        self.assertIn(' file /haproxy/errors/robots-unrouted.txt ', robots)
+        self.assertTrue(robots.endswith(' splash_robots'))
+        self.assertTrue(splash.startswith('http-request return status 404 content-type "text/html'))
+        self.assertTrue(splash.endswith(' !splash_robots'))
+        self.assertNotIn('splash_root', render_listener())
+        self.assertRegex(render_listener(), r'acl splash_robots path /robots\.txt\n')
+
+    def test_noindex_and_no_store_headers_on_both(self):
+        for rule in return_rules(render_listener()):
+            self.assertIn('hdr x-robots-tag "noindex, nofollow, noarchive"', rule)
+            self.assertIn('hdr cache-control "no-store"', rule)
+            self.assertIn('hdr x-content-type-options "nosniff"', rule)
+
+    def test_static_file_not_lf_file(self):
+        """`file` (verbatim), never `lf-file`: nothing from the request may be
+        expanded into the page (Host echo, unique-id, ...)."""
+        for rule in return_rules(render_listener()):
+            self.assertRegex(rule, r' file /haproxy/errors/(default-splash\.html|robots-unrouted\.txt) ')
+            self.assertNotIn('lf-file', rule)
+
+    def test_conditions_keep_acme_blocked_and_suspended_out(self):
+        for susp in (False, True):
+            for rule in return_rules(render_listener(suspension_enabled=susp)):
+                cond = rule.split(' if ', 1)[1].split()
+                self.assertIn('!splash_routed_host', cond)
+                self.assertIn('!splash_acme', cond)
+                self.assertIn('!is_blocked_ip', cond)
+                if susp:
+                    self.assertIn('!is_suspended_domain', cond)
+                else:
+                    self.assertNotIn('!is_suspended_domain', cond)
+
+    def test_acme_acl_is_the_challenge_prefix(self):
+        self.assertRegex(render_listener(),
+                         r'acl splash_acme path_beg /\.well-known/acme-challenge/\n')
+
+    def test_routed_acls_strip_port_and_lowercase(self):
+        cfg = render_listener()
+        self.assertIn('acl splash_routed_host req.hdr(host),field(1,:),lower '
+                      '-f /etc/haproxy/routed_hosts.list', cfg)
+        self.assertIn('acl splash_routed_host req.hdr(host),field(1,:),lower '
+                      '-m end -f /etc/haproxy/routed_wildcards.list', cfg)
+
+    def test_splash_runs_before_waf_and_after_ip_block_and_suspension(self):
+        for susp in (False, True):
+            cfg = render_listener(suspension_enabled=susp)
+            first_return = cfg.index('file /haproxy/errors/robots-unrouted.txt')
+            self.assertLess(first_return, cfg.index('http-request send-spoe-group coraza'))
+            self.assertLess(first_return, cfg.index('filter spoe engine coraza'))
+            self.assertGreater(first_return, cfg.index('acl is_blocked_ip'))
+            self.assertGreater(first_return, cfg.index('http-request track-sc0'))
+            if susp:
+                self.assertGreater(first_return, cfg.index('acl is_suspended_domain'))
+
+
+class Page(unittest.TestCase):
+
+    def setUp(self):
+        with open(SPLASH_FILE, 'rb') as fh:
+            self.raw = fh.read()
+        self.text = self.raw.decode('utf-8')
+
+    def test_small_enough_for_one_buffer(self):
+        # http-request return payload must fit tune.bufsize (16384) with headers.
+        self.assertLess(len(self.raw), 8192)
+
+    def test_no_links_scripts_or_external_resources(self):
+        low = self.text.lower()
+        for needle in ('href', 'src=', '<script', '<form', '<a ', 'http://', 'https://',
+                       '8443', '@import', 'url('):
+            self.assertNotIn(needle, low, needle)
+
+    def test_no_version_or_template_expansions(self):
+        self.assertNotRegex(self.text, r'20\d\d\.\d+\.\d+')
+        self.assertNotIn('{{', self.text)
+        self.assertNotIn('%[', self.text)
+
+
+class Robots(unittest.TestCase):
+
+    def test_robots_disallows_everything(self):
+        with open(os.path.join(MODULE_DIR, 'errors', 'robots-unrouted.txt')) as fh:
+            self.assertEqual(fh.read(), 'User-agent: *\nDisallow: /\n')
+
+
+class RoutedLists(unittest.TestCase):
+
+    def test_render_sorted_dedup_lower(self):
+        body = hm.render_routed_list(['B.example.com', 'a.example.com', 'b.example.com ', ''])
+        lines = [l for l in body.splitlines() if not l.startswith('#')]
+        self.assertEqual(lines, ['a.example.com', 'b.example.com'])
+
+    def test_name_with_whitespace_is_dropped_not_split(self):
+        body = hm.render_routed_list(['ok.example.com', 'bad\nevil.example.com', 'a b'])
+        lines = [l for l in body.splitlines() if not l.startswith('#')]
+        self.assertEqual(lines, ['ok.example.com'])
+
+    def test_lists_are_in_the_rollback_set(self):
+        pairs = dict(hm._config_backup_pairs())
+        self.assertIn(hm.ROUTED_HOSTS_PATH, pairs)
+        self.assertIn(hm.ROUTED_WILDCARDS_PATH, pairs)
+
+
+class GenerateConfigWritesLists(unittest.TestCase):
+    """Run the real generate_config() with paths redirected and reload stubbed."""
+
+    def setUp(self):
+        self.tmp = tempfile.mkdtemp(prefix='splash-test-')
+        self.saved = {k: getattr(hm, k) for k in (
+            'DB_FILE', 'HAPROXY_CONFIG_PATH', 'HAPROXY_BACKUP_PATH',
+            'CLUSTER_SECRET_PATH', 'HAPROXY_SOCKET_PATH', 'SSL_CERTS_DIR',
+            'BLOCKED_IPS_MAP_PATH', 'BLOCKED_IPS_MAP_BACKUP_PATH',
+            'CORAZA_SPOE_CONFIG_PATH', 'CORAZA_SPOE_BACKUP_PATH',
+            'ROUTED_HOSTS_PATH', 'ROUTED_HOSTS_BACKUP_PATH',
+            'ROUTED_WILDCARDS_PATH', 'ROUTED_WILDCARDS_BACKUP_PATH',
+            'reload_haproxy_safely', 'create_backup')}
+        j = lambda n: os.path.join(self.tmp, n)  # noqa: E731
+        hm.DB_FILE = j('db.sqlite')
+        hm.HAPROXY_CONFIG_PATH = j('haproxy.cfg')
+        hm.HAPROXY_BACKUP_PATH = j('haproxy.cfg.backup')
+        hm.CLUSTER_SECRET_PATH = j('cluster-secret')
+        hm.HAPROXY_SOCKET_PATH = j('haproxy.sock')
+        hm.SSL_CERTS_DIR = j('certs')
+        hm.BLOCKED_IPS_MAP_PATH = j('blocked_ips.map')
+        hm.BLOCKED_IPS_MAP_BACKUP_PATH = j('blocked_ips.map.backup')
+        hm.CORAZA_SPOE_CONFIG_PATH = j('coraza-spoe.cfg')
+        hm.CORAZA_SPOE_BACKUP_PATH = j('coraza-spoe.cfg.backup')
+        hm.ROUTED_HOSTS_PATH = j('routed_hosts.list')
+        hm.ROUTED_HOSTS_BACKUP_PATH = j('routed_hosts.list.backup')
+        hm.ROUTED_WILDCARDS_PATH = j('routed_wildcards.list')
+        hm.ROUTED_WILDCARDS_BACKUP_PATH = j('routed_wildcards.list.backup')
+        hm.reload_haproxy_safely = lambda *a, **kw: (True, 'stubbed')
+        hm.create_backup = lambda *a, **kw: (True, 'unavailable')
+        self.env = {k: os.environ.pop(k, None) for k in (
+            'HAPROXY_DEFAULT_SPLASH', 'HAPROXY_SUSPENSION_ENABLED',
+            'HAPROXY_CORAZA_SPOE_BACKEND')}
+        hm.init_db()
+        with sqlite3.connect(hm.DB_FILE) as conn:
+            cur = conn.cursor()
+
+            def add(domain, backend, wildcard=0):
+                cur.execute('INSERT INTO domains (domain, ssl_enabled, ssl_cert_path, '
+                            'template_override, is_wildcard) VALUES (?, 1, NULL, NULL, ?)',
+                            (domain, wildcard))
+                did = cur.lastrowid
+                if backend:
+                    cur.execute('INSERT INTO backends (name, domain_id, settings) VALUES (?, ?, NULL)',
+                                (backend, did))
+                    cur.execute('INSERT INTO backend_servers (backend_id, server_name, '
+                                'server_address, server_port, server_options) VALUES (?, ?, ?, ?, ?)',
+                                (cur.lastrowid, 'web1', '10.0.0.10', 80, 'check'))
+            add('Site.Example.com', 'site')
+            add('*.wild.example.com', 'wild', wildcard=1)
+            add('whp01.example.net', None)   # server hostname: cert-only
+            conn.commit()
+
+    def tearDown(self):
+        for k, v in self.saved.items():
+            setattr(hm, k, v)
+        for k, v in self.env.items():
+            os.environ.pop(k, None)
+            if v is not None:
+                os.environ[k] = v
+        import shutil
+        shutil.rmtree(self.tmp, ignore_errors=True)
+
+    def _lines(self, path):
+        with open(path) as fh:
+            return [l for l in fh.read().splitlines() if l and not l.startswith('#')]
+
+    def test_lists_written_from_routed_rows_only(self):
+        hm.generate_config()
+        self.assertEqual(self._lines(hm.ROUTED_HOSTS_PATH), ['site.example.com'])
+        self.assertEqual(self._lines(hm.ROUTED_WILDCARDS_PATH), ['.wild.example.com'])
+        with open(hm.HAPROXY_CONFIG_PATH) as fh:
+            cfg = fh.read()
+        # every exact routing ACL name is in the list (superset property)
+        for name in re.findall(r'acl \S+-acl hdr\(host\) -i (\S+)', cfg):
+            self.assertIn(name.lower(), self._lines(hm.ROUTED_HOSTS_PATH))
+        for base in re.findall(r'acl \S+-acl hdr_end\(host\) -i (\S+)', cfg):
+            self.assertIn(base.lower(), self._lines(hm.ROUTED_WILDCARDS_PATH))
+        self.assertIn('default-splash.html', cfg)
+
+    def test_default_log_target_is_the_in_container_socket(self):
+        """Access log goes to the receiver's UNIX socket (chroot-relative),
+        never the old host bridge target 172.18.0.1:514."""
+        os.environ.pop('HAPROXY_SYSLOG_TARGET', None)
+        hm.generate_config()
+        with open(hm.HAPROXY_CONFIG_PATH) as fh:
+            cfg = fh.read()
+        self.assertRegex(cfg, r'\n    log         /log\.sock len 2048 format rfc5424 local2 info\n')
+        self.assertIn('chroot      /var/lib/haproxy', cfg)
+        log_lines = [l for l in cfg.splitlines() if l.strip().startswith('log ')
+                     and 'global' not in l]
+        self.assertEqual(len(log_lines), 1, log_lines)
+        self.assertNotIn('172.18.0.1', log_lines[0])
+
+    def test_log_target_env_override(self):
+        os.environ['HAPROXY_SYSLOG_TARGET'] = '192.0.2.5:514'
+        try:
+            hm.generate_config()
+        finally:
+            os.environ.pop('HAPROXY_SYSLOG_TARGET', None)
+        with open(hm.HAPROXY_CONFIG_PATH) as fh:
+            self.assertIn('log         192.0.2.5:514 len 2048', fh.read())
+
+    def test_lists_written_even_with_splash_off(self):
+        os.environ['HAPROXY_DEFAULT_SPLASH'] = 'off'
+        hm.generate_config()
+        self.assertTrue(os.path.exists(hm.ROUTED_HOSTS_PATH))
+        self.assertTrue(os.path.exists(hm.ROUTED_WILDCARDS_PATH))
+        with open(hm.HAPROXY_CONFIG_PATH) as fh:
+            self.assertNotIn('default-splash.html', fh.read())
+
+
+if __name__ == '__main__':
+    unittest.main(verbosity=2)

scripts/validate-rendered-config.py

@@ -128,6 +128,9 @@ STUB_FILES = {
     # `lf-file` on the Coraza deny rule; loaded at parse time. Present in the
     # image (COPY errors /haproxy/errors), stubbed for anything else.
     '/haproxy/errors/403-waf.html': '<html><body>blocked %[unique-id]</body></html>\n',
+    # `file` on the default-splash return rules; loaded at parse time too.
+    '/haproxy/errors/default-splash.html': '<html><body>splash</body></html>\n',
+    '/haproxy/errors/robots-unrouted.txt': 'User-agent: *\nDisallow: /\n',
 }
 
 # (suspension_enabled, coraza_spoe_backend) combinations to render + validate.
@@ -137,6 +140,12 @@ SCENARIOS = (
         'HAPROXY_SUSPENSION_ENABLED': 'true',
         'HAPROXY_CORAZA_SPOE_BACKEND': '127.0.0.1:9000',
     }),
+    # The default splash is ON unless switched off; render the off shape too.
+    ('splash-off', {
+        'HAPROXY_SUSPENSION_ENABLED': 'true',
+        'HAPROXY_CORAZA_SPOE_BACKEND': '127.0.0.1:9000',
+        'HAPROXY_DEFAULT_SPLASH': 'off',
+    }),
 )
 
 
@@ -298,7 +307,8 @@ def render(scenario_name, env_overrides, workdir):
     seed_database(os.path.join(scenario_dir, 'haproxy_config.db'))
 
     saved_env = {}
-    for key in ('HAPROXY_SUSPENSION_ENABLED', 'HAPROXY_CORAZA_SPOE_BACKEND'):
+    for key in ('HAPROXY_SUSPENSION_ENABLED', 'HAPROXY_CORAZA_SPOE_BACKEND',
+                'HAPROXY_DEFAULT_SPLASH'):
         saved_env[key] = os.environ.pop(key, None)
     os.environ.update(env_overrides)
 
@@ -330,6 +340,8 @@ def assert_scenario_branches(scenario_name, config_path, env_overrides):
                        'HAPROXY_SUSPENSION_ENABLED' in env_overrides),
         'coraza': ('filter spoe engine coraza',
                    'HAPROXY_CORAZA_SPOE_BACKEND' in env_overrides),
+        'splash': ('file /haproxy/errors/default-splash.html',
+                   env_overrides.get('HAPROXY_DEFAULT_SPLASH') != 'off'),
     }
     for label, (needle, should_be_present) in expected.items():
         present = needle in text

templates/hap_header.tpl

@@ -2,42 +2,33 @@
 # Global settings
 #---------------------------------------------------------------------
 global
-    # ACCESS LOG DESTINATION.
+    # ACCESS LOG DESTINATION: a UNIX datagram socket INSIDE this container.
     #
-    # This used to be `log 127.0.0.1 local2`, which was a silent black hole:
+    # scripts/haproxy-log-receiver.py (supervised by start-up.sh) reads it and
-    # 127.0.0.1 is the CONTAINER's own loopback, nothing has ever listened on
+    # writes a size-capped, rotating /var/log/haproxy/access.log in the
-    # udp/514 in the container netns, and there is no /dev/log in the image.
+    # haproxy-logs volume. No network listener anywhere.
-    # Every access log line -- ~1.5M/day across the whole edge -- was written
-    # to a socket with no receiver and dropped. Nothing errored, nothing
-    # warned, and `haproxy -c` was perfectly happy. The cost only shows up
-    # during an incident: per-IP 429s, tarpits, wp-admin gate redirects,
-    # WAF 403s and `silent-drop`s left no record anywhere, so the edge could
-    # not be asked what it had rejected. Only aggregate stick-table counters
-    # survived.
     #
-    # Now points at the DOCKER BRIDGE GATEWAY, where the host's rsyslog has an
+    # History: `log 127.0.0.1 local2` was a black hole (nothing listened in
-    # imudp listener bound (installed idempotently by WHP's
+    # the container netns); its successor, the docker bridge gateway
-    # setup-haproxy-logrotate.sh, which also writes the logrotate stanza).
+    # 172.18.0.1:514 with a host rsyslog imudp listener, had no listener on
-    # The host writes local2 to /var/log/haproxy.log and stops it there, so it
+    # the fleet either (probable cause, not measured: rsyslog binds that
-    # does not also flood /var/log/messages or the Graylog forwarder.
+    # address once at boot, before docker has created the bridge, and does
+    # not retry). Either way per-IP
+    # 429s, tarpits, gate redirects and WAF 403s left no record.
     #
-    # WHY NOT `log stdout format raw local0`: it is INCOMPATIBLE with the
+    # WHY NOT `log stdout`: silently emits NOTHING with `daemon` (below) on
-    # `daemon` keyword below, and incompatible SILENTLY. Verified on the
+    # the pinned 3.0.11 (`haproxy -c` still passes), and dropping `daemon`
-    # pinned 3.0.11 binary: with `daemon` set, a `log stdout` config serves
+    # breaks the synchronous `haproxy -W` launch sites in haproxy_manager.py.
-    # traffic normally and emits ZERO log lines, and `haproxy -c` returns 0
+    # Also, `docker logs haproxy-manager` feeds WHP's AI log monitor, which
-    # with no error and no warning -- so the CI config gate
+    # must not get a per-request firehose.
-    # (scripts/validate-rendered-config.py) cannot catch it either. Making it
-    # work means dropping `daemon` / adding -db so haproxy stays in the
-    # foreground, which in turn breaks the three synchronous
-    # `subprocess.run(['haproxy', '-W', ...], check=True)` launch sites in
-    # haproxy_manager.py (they would block until the 180s timeout and then be
-    # killed). That is a change to the exact code path whose failure mode is
-    # "container Up, ports 80/443 never bound, every site down, /health still
-    # 200". Not worth it for a logging change.
     #
-    # UDP means a dead listener degrades to dropped log lines, never to a
+    # The path is relative to the `chroot` below: HAProxy resolves it after
-    # stalled or failing request path -- the correct failure direction for an
+    # chrooting, so `/log.sock` here is /var/lib/haproxy/log.sock on disk,
-    # edge fronting ~60 customer sites.
+    # which is where the receiver binds (HAPROXY_LOG_SOCKET).
+    #
+    # Sends are non-blocking: a dead receiver or a full socket buffer drops
+    # log lines, never stalls a request. HAPROXY_SYSLOG_TARGET still
+    # overrides the target (`<ip>:<port>` for a remote collector).
     #
     # len 2048 accommodates the enriched log-format in hap_listener.tpl
     # (URL + User-Agent + UUID); the default 1024 would truncate long ones.

templates/hap_listener.tpl

@@ -616,6 +616,47 @@ frontend web
     http-request set-path /suspended if is_suspended_domain
     use_backend default-backend if is_suspended_domain
 {%- endif %}
+{%- if default_splash %}
+
+    # Default splash for traffic that is NOT routed to any site: the server's
+    # own hostname (registered cert-only, no backend), a bare IP, a stale or
+    # mistyped name. HAProxy answers it ITSELF with a static page and the
+    # request goes no further -- in particular it never reaches the Coraza
+    # SPOE call below.
+    #
+    # Why: `use_backend default-backend` does not short-circuit anything --
+    # HAProxy runs EVERY http-request rule (Coraza included) before it picks
+    # a backend. So the catch-all used to be WAF-inspected with the full
+    # server-wide rule set like customer traffic, and a browser visiting the
+    # server hostname also carries the control panel's cookies (cookies are
+    # not port-scoped: whp01:8443 cookies are sent to whp01:443). Anything
+    # Coraza matched there showed the WAF block page for a host that has no
+    # site behind it to protect. A fixed static reply has nothing to protect.
+    #
+    # "Routed" is decided from two lists generate_config() writes from the
+    # SAME domain rows it renders the routing ACLs from (exact names, and
+    # ".base" suffixes for wildcards), port stripped and lowercased. They
+    # are a SUPERSET of what routes -- the routing ACLs below compare the raw
+    # Host (port included) -- so a host that routes is never splashed; the
+    # worst a mismatch can do is send an unrouted request through the WAF to
+    # default-backend, which is the pre-splash behaviour. Both lists are part
+    # of the rollback set (_config_backup_pairs) so they cannot drift from
+    # the haproxy.cfg that is actually loaded.
+    #
+    # Kept OUT of the splash: ACME http-01 (the server hostname's own cert is
+    # issued through it), blocked IPs (blocked-ip page) and suspended hosts
+    # (suspended page). Rate limits, the IP block and the wp-* gates above
+    # still apply. EVERY path is 404 with the same page, plus
+    # X-Robots-Tag noindex and /robots.txt "Disallow: /", so search engines
+    # never index the server's IP or hostname as a site.
+    # Disable with HAPROXY_DEFAULT_SPLASH=off (back to default-backend).
+    acl splash_routed_host req.hdr(host),field(1,:),lower -f /etc/haproxy/routed_hosts.list
+    acl splash_routed_host req.hdr(host),field(1,:),lower -m end -f /etc/haproxy/routed_wildcards.list
+    acl splash_acme path_beg /.well-known/acme-challenge/
+    acl splash_robots path /robots.txt
+    http-request return status 200 content-type "text/plain; charset=utf-8" hdr cache-control "no-store" hdr x-robots-tag "noindex, nofollow, noarchive" hdr x-content-type-options "nosniff" file /haproxy/errors/robots-unrouted.txt if !splash_routed_host !splash_acme !is_blocked_ip{% if suspension_enabled %} !is_suspended_domain{% endif %} splash_robots
+    http-request return status 404 content-type "text/html; charset=utf-8" hdr cache-control "no-store" hdr x-robots-tag "noindex, nofollow, noarchive" hdr x-content-type-options "nosniff" file /haproxy/errors/default-splash.html if !splash_routed_host !splash_acme !is_blocked_ip{% if suspension_enabled %} !is_suspended_domain{% endif %} !splash_robots
+{%- endif %}
 
     # CVE-2026-109618 (GHSA-w8hw-w9fx-j79w, PHP "RCE via OOM corruption"),
     # vector (a): a multipart/form-data body with no declared length is not