fix(logging): capture the full User-Agent, drop per-request SPOE log noise
HAProxy Manager Build and Push / Build-and-Push (push) Successful in 1m19s
HAProxy Manager Build and Push / Build-and-Push (push) Successful in 1m19s
Two defects caught by watching the real production access log on whp01 in the
minutes after 2026.08.7 made access logging work for the first time.
1. User-Agent was being truncated to its tail.
`http-request capture req.hdr(User-Agent)` treats the header as a
comma-separated list and returns only the LAST element. Real User-Agent
strings contain commas, so
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36
logged as
ua=like Gecko) Chrome/131.0.0.0 Safari/537.36
losing the platform half -- exactly the half needed to tell a spoofed
crawler from a real browser, which is one of the main reasons the field was
added. Switched to req.fhdr(), which returns the full unsplit header value.
2. SPOE was writing one log line per inspected request.
`log global` inside the spoe-agent block emitted
SPOE: [coraza] <GROUP:coraza-req> sid=537 st=0 0/0/0/0/0 32/32 0/0 0/467
for every single request. Measured on whp01: 618 SPOE lines against 669 real
access lines -- ~48% of the log volume, roughly doubling the edge's log
footprint (~400 MB/day extra) to record `st=0` over and over.
It carries nothing incident response needs. The WAF verdict is already in
the access line (status 403 plus the id= UUID, which joins to
/var/log/coraza/audit.log for the rule_id), and per-transaction WAF detail
is written by the SPOA itself to /var/log/coraza/spoa.log. Agent-level
failures still surface through `option set-on-error error` ->
var(txn.coraza.error) and the fail-open path in hap_listener.tpl.
Verified: scripts/validate-rendered-config.py passes `haproxy -c` on both the
"default" and "full" scenarios against the real 3.0.11 binary; wp-admin gate,
trusted-proxy gate and xmlrpc rate-limit suites all still pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -37,7 +37,22 @@ spoe-agent coraza
|
|||||||
timeout processing 100ms
|
timeout processing 100ms
|
||||||
|
|
||||||
use-backend coraza-spoa-backend
|
use-backend coraza-spoa-backend
|
||||||
log global
|
|
||||||
|
# NO `log global` here, deliberately.
|
||||||
|
#
|
||||||
|
# `log global` in a spoe-agent emits one line PER INSPECTED REQUEST, e.g.
|
||||||
|
# SPOE: [coraza] <GROUP:coraza-req> sid=537 st=0 0/0/0/0/0 32/32 0/0 0/467
|
||||||
|
# Measured on whp01 immediately after access logging started working:
|
||||||
|
# 618 SPOE lines vs 669 real access lines -- it was ~48% of the log volume,
|
||||||
|
# i.e. it would roughly DOUBLE the edge's log footprint (~400 MB/day extra)
|
||||||
|
# to record `st=0` over and over.
|
||||||
|
#
|
||||||
|
# It carries nothing incident response needs: the WAF's verdict is already
|
||||||
|
# visible in the access log line (status 403 + the `id=` UUID, which joins
|
||||||
|
# to /var/log/coraza/audit.log for the rule_id), and per-transaction WAF
|
||||||
|
# detail is written by the SPOA itself to /var/log/coraza/spoa.log.
|
||||||
|
# Agent-level failures still surface via `option set-on-error error` ->
|
||||||
|
# var(txn.coraza.error) and the fail-open path in hap_listener.tpl.
|
||||||
|
|
||||||
# Per-request inspection message. No `event` directive — fires only when
|
# Per-request inspection message. No `event` directive — fires only when
|
||||||
# explicitly invoked from haproxy.cfg via `http-request send-spoe-group`.
|
# explicitly invoked from haproxy.cfg via `http-request send-spoe-group`.
|
||||||
|
|||||||
@@ -29,7 +29,15 @@ frontend web
|
|||||||
# Any new capture MUST be appended AFTER this line, never inserted
|
# Any new capture MUST be appended AFTER this line, never inserted
|
||||||
# above it, or the slot indices in the log-format silently shift and
|
# above it, or the slot indices in the log-format silently shift and
|
||||||
# the access log starts attributing the wrong string to the wrong field.
|
# the access log starts attributing the wrong string to the wrong field.
|
||||||
http-request capture req.hdr(User-Agent) len 200
|
# req.fhdr(), NOT req.hdr(): req.hdr() treats the header as a comma-
|
||||||
|
# separated list and returns only the LAST element. Real User-Agent strings
|
||||||
|
# contain commas -- "Mozilla/5.0 (Windows NT 10.0; Win64; x64)
|
||||||
|
# AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
|
||||||
|
# captured with req.hdr() logs as just "like Gecko) Chrome/131.0.0.0
|
||||||
|
# Safari/537.36", silently losing the platform half -- which is exactly the
|
||||||
|
# half you need to tell a spoofed crawler from a real browser.
|
||||||
|
# Observed in production on whp01 before this was corrected.
|
||||||
|
http-request capture req.fhdr(User-Agent) len 200
|
||||||
|
|
||||||
# --- Access logging -----------------------------------------------------
|
# --- Access logging -----------------------------------------------------
|
||||||
# Scoped to THIS frontend on purpose: it references capture slots and
|
# Scoped to THIS frontend on purpose: it references capture slots and
|
||||||
|
|||||||
Reference in New Issue
Block a user