fix(logging): capture the full User-Agent, drop per-request SPOE log noise
HAProxy Manager Build and Push / Build-and-Push (push) Successful in 1m19s

Two defects caught by watching the real production access log on whp01 in the
minutes after 2026.08.7 made access logging work for the first time.

1. User-Agent was being truncated to its tail.

   `http-request capture req.hdr(User-Agent)` treats the header as a
   comma-separated list and returns only the LAST element. Real User-Agent
   strings contain commas, so

     Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
     (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36

   logged as

     ua=like Gecko) Chrome/131.0.0.0 Safari/537.36

   losing the platform half -- exactly the half needed to tell a spoofed
   crawler from a real browser, which is one of the main reasons the field was
   added. Switched to req.fhdr(), which returns the full unsplit header value.

2. SPOE was writing one log line per inspected request.

   `log global` inside the spoe-agent block emitted

     SPOE: [coraza] <GROUP:coraza-req> sid=537 st=0 0/0/0/0/0 32/32 0/0 0/467

   for every single request. Measured on whp01: 618 SPOE lines against 669 real
   access lines -- ~48% of the log volume, roughly doubling the edge's log
   footprint (~400 MB/day extra) to record `st=0` over and over.

   It carries nothing incident response needs. The WAF verdict is already in
   the access line (status 403 plus the id= UUID, which joins to
   /var/log/coraza/audit.log for the rule_id), and per-transaction WAF detail
   is written by the SPOA itself to /var/log/coraza/spoa.log. Agent-level
   failures still surface through `option set-on-error error` ->
   var(txn.coraza.error) and the fail-open path in hap_listener.tpl.

Verified: scripts/validate-rendered-config.py passes `haproxy -c` on both the
"default" and "full" scenarios against the real 3.0.11 binary; wp-admin gate,
trusted-proxy gate and xmlrpc rate-limit suites all still pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude
2026-08-22 10:17:07 -07:00
parent 711c670319
commit b2f835a88c
3 changed files with 26 additions and 3 deletions
+16 -1
View File
@@ -37,7 +37,22 @@ spoe-agent coraza
timeout processing 100ms
use-backend coraza-spoa-backend
log global
# NO `log global` here, deliberately.
#
# `log global` in a spoe-agent emits one line PER INSPECTED REQUEST, e.g.
# SPOE: [coraza] <GROUP:coraza-req> sid=537 st=0 0/0/0/0/0 32/32 0/0 0/467
# Measured on whp01 immediately after access logging started working:
# 618 SPOE lines vs 669 real access lines -- it was ~48% of the log volume,
# i.e. it would roughly DOUBLE the edge's log footprint (~400 MB/day extra)
# to record `st=0` over and over.
#
# It carries nothing incident response needs: the WAF's verdict is already
# visible in the access log line (status 403 + the `id=` UUID, which joins
# to /var/log/coraza/audit.log for the rule_id), and per-transaction WAF
# detail is written by the SPOA itself to /var/log/coraza/spoa.log.
# Agent-level failures still surface via `option set-on-error error` ->
# var(txn.coraza.error) and the fail-open path in hap_listener.tpl.
# Per-request inspection message. No `event` directive — fires only when
# explicitly invoked from haproxy.cfg via `http-request send-spoe-group`.
+9 -1
View File
@@ -29,7 +29,15 @@ frontend web
# Any new capture MUST be appended AFTER this line, never inserted
# above it, or the slot indices in the log-format silently shift and
# the access log starts attributing the wrong string to the wrong field.
http-request capture req.hdr(User-Agent) len 200
# req.fhdr(), NOT req.hdr(): req.hdr() treats the header as a comma-
# separated list and returns only the LAST element. Real User-Agent strings
# contain commas -- "Mozilla/5.0 (Windows NT 10.0; Win64; x64)
# AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
# captured with req.hdr() logs as just "like Gecko) Chrome/131.0.0.0
# Safari/537.36", silently losing the platform half -- which is exactly the
# half you need to tell a spoofed crawler from a real browser.
# Observed in production on whp01 before this was corrected.
http-request capture req.fhdr(User-Agent) len 200
# --- Access logging -----------------------------------------------------
# Scoped to THIS frontend on purpose: it references capture slots and