From 4f21bb2f764866740b7015a3fb634c23bcf2340d Mon Sep 17 00:00:00 2001 From: Josh Knapp Date: Fri, 9 Oct 2026 10:49:27 -0700 Subject: [PATCH] feat(waf): make Coraza see request bodies; stamp complete/partial; 2026.10.2 Measured on haproxy 3.0.11 + coraza-spoa v0.7.1 + CRS 4.25.0 (local haproxy-manager + coraza-spoa + echo backend): - send-spoe-group fired at header time, so req.body held only what arrived with the headers. A 933180 payload whose body came in a second send(), 2s late, or after Expect: 100-continue passed uninspected; 8 parallel 14 KB payload-first POSTs: 6/40 blocked before, 40/40 after. - Add `option http-buffer-request` (Coraza block only). Uploads are not held: one buffer is inspected, the rest streams (50 MB upload: sha-ok, same timing before/after). - Stamp X-WHP-WAF-Body: complete|partial for Coraza (client copies stripped first, stripped again before the backend). Content-Length: exact compare; chunked/no-CL: body+headers < 12 KB => complete. WHP's generated rules use it to deny body-parse errors (200002/200003) only on complete bodies. - Pin tune.bufsize 16384 (= default) when Coraza is on. Coraza costs ~3.3 ms/KB of body and the 100 ms SPOE timeout fails open: at 32 KB a payload at byte 0 of a 30 KB base64 body bypassed. SPOP frames > 65535 are refused by the agent library. - coraza-spoa/config.yaml: SecRequestBodyLimit 131072 / InMemory 131072 / ProcessPartial (Reject 413s even under DetectionOnly). - Access log gains wafb= (complete|partial) and wafe= (SPOE error = request passed uninspected). - overrides.conf: document (comment only) that the 920440 promotion turns the engine On for every URL with a file extension. - scripts/test-waf-body-inspection.py (16 tests), wired into the build. Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 1 + Dockerfile | 1 + VERSION | 2 +- coraza-spoa/config.yaml | 18 ++ coraza-spoa/overrides.conf | 10 ++ haproxy_manager.py | 1 + scripts/test-waf-body-inspection.py | 253 ++++++++++++++++++++++++++++ templates/hap_header.tpl | 35 ++++ templates/hap_listener.tpl | 61 ++++++- 9 files changed, 380 insertions(+), 2 deletions(-) create mode 100644 scripts/test-waf-body-inspection.py diff --git a/CLAUDE.md b/CLAUDE.md index f4c43cf..f820e63 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -10,6 +10,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co - **Stick-table contract**: `python3 scripts/test-stick-table-contract.py` - offline; holds the templates' `store` clauses, `STICK_TABLE_FIELD_CONTRACT`, and every consumer to each other. Run it after touching any `stick-table` line. - **Runtime-map contract**: `python3 scripts/test-runtime-map-contract.py` - offline; asserts the runtime map commands are `@1`-prefixed, reference the map by FILE PATH (never `#`), carry the value `1`, and that every captured rejection is classified as a failure. Run it after touching any `add map`/`del map`/`clear map` path. - **Certificate destruction safety**: `python3 scripts/test-cert-write-safety.py` - offline; asserts a live `.pem` is never truncated, removed, or its certbot lineage deleted while any configured domain still references it (one bundle serves many names, so `ssl_cert_path` is routinely shared). Run it after touching any `os.remove`/`certbot delete`/PEM-write path. +- **WAF body inspection**: `python3 scripts/test-waf-body-inspection.py` - offline (runs in the image build); pins `option http-buffer-request`, the `X-WHP-WAF-Body` complete/partial stamp (strip -> stamp -> SPOE -> strip), `tune.bufsize` against the 100 ms SPOE fail-open budget and the 65535-byte SPOP frame cap, and Coraza's body limits in `coraza-spoa/config.yaml`. Run it after touching the Coraza block, `tune.bufsize`, or the SPOE engine. - **Manual Testing**: Run `curl` commands against `http://localhost:8000` endpoints as shown in README.md ### Reading stick tables (and why it is easy to get silently wrong) diff --git a/Dockerfile b/Dockerfile index f66654f..3a2a23a 100644 --- a/Dockerfile +++ b/Dockerfile @@ -92,6 +92,7 @@ RUN python3 /haproxy/scripts/test-wpadmin-gate.py \ && python3 /haproxy/scripts/test-config-rollback.py \ && python3 /haproxy/scripts/test-cert-write-safety.py \ && python3 /haproxy/scripts/test-cert-scripts.py \ + && python3 /haproxy/scripts/test-waf-body-inspection.py \ && python3 /haproxy/scripts/validate-rendered-config.py # Create log directories RUN mkdir -p /var/log && touch /var/log/haproxy-manager.log /var/log/haproxy-manager-errors.log diff --git a/VERSION b/VERSION index 3715deb..6e34cd3 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -2026.09.1 +2026.10.2 diff --git a/coraza-spoa/config.yaml b/coraza-spoa/config.yaml index 53a281e..cd4a82e 100644 --- a/coraza-spoa/config.yaml +++ b/coraza-spoa/config.yaml @@ -26,6 +26,24 @@ applications: # CRS-bundled defaults: recommended Coraza settings + CRS setup + # the rule pack itself (~16 MB of rules embedded in the binary). Include @coraza.conf-recommended + + # Request-body limits, set explicitly instead of inheriting the + # recommended file's 12.5 MB / Reject. HAProxy only ever hands us one + # buffer of body (tune.bufsize 16384 in haproxy-manager's + # hap_header.tpl), and the SPOP library refuses frames over 65535 + # bytes, so no body we receive can exceed 64 KB. The invariant is: + # tune.bufsize (16 KB) < SPOP frame cap (65535) < SecRequestBodyLimit + # Breaking it is NOT harmless: SecRequestBodyLimitAction Reject + # interrupts with 413 even under DetectionOnly (Coraza + # setAndReturnBodyLimitInterruption ignores the engine mode), which the + # edge turns into a 403 on every large upload. ProcessPartial is the + # belt to that brace. InMemoryLimit == BodyLimit keeps the body buffer + # off disk (this image is distroless). Pinned by + # haproxy-manager-base/scripts/test-waf-body-inspection.py. + SecRequestBodyLimit 131072 + SecRequestBodyInMemoryLimit 131072 + SecRequestBodyLimitAction ProcessPartial + Include @crs-setup.conf.example # Runtime-managed PRE-CRS exclusions written by WHP UI. Empty by default. diff --git a/coraza-spoa/overrides.conf b/coraza-spoa/overrides.conf index 57bfcd0..d8a7383 100644 --- a/coraza-spoa/overrides.conf +++ b/coraza-spoa/overrides.conf @@ -59,6 +59,16 @@ SecRuleUpdateActionById 944100-944300 "ctl:ruleEngine=On" # single GCP-hosted scanner. Zero false positives observed; standard # WP/WooCommerce/Divi/HPR URLs do not end in these extensions. # --------------------------------------------------------------------------- +# WARNING (measured 2026-10-09, Coraza v3.7.0 + CRS 4.25.0, in-process AND +# live behind haproxy): 920440 is a CHAIN whose first link is +# `REQUEST_BASENAME "@rx \.([^.]+)$"` -- it matches EVERY URL with a file +# extension. Coraza runs the appended ctl:ruleEngine=On when that first link +# matches, even though the chain then fails. So on any *.php / *.css / ... +# request the engine is On for the rest of the transaction: 949110 enforces +# the whole CRS anomaly score (a 942100 SQLi GET is 403 on /x.php, 200 on /x) +# and 200002 denies body-parse errors -- in detect_only mode too, because +# SecRuleEngine DetectionOnly cannot override a runtime ctl. Not changed here +# (it would loosen live blocking fleet-wide); see waf-body-hardening-report.md. SecRuleUpdateActionById 920440 "ctl:ruleEngine=On" # --------------------------------------------------------------------------- diff --git a/haproxy_manager.py b/haproxy_manager.py index 2e023bf..c7bd2ec 100644 --- a/haproxy_manager.py +++ b/haproxy_manager.py @@ -2763,6 +2763,7 @@ def generate_config(): default_headers = template_env.get_template('hap_header.tpl').render( cluster_secret = get_or_create_cluster_secret(), syslog_target = syslog_target, + coraza_spoe_backend = coraza_spoe_backend, ) config_parts.append(default_headers) diff --git a/scripts/test-waf-body-inspection.py b/scripts/test-waf-body-inspection.py new file mode 100644 index 0000000..8386d8d --- /dev/null +++ b/scripts/test-waf-body-inspection.py @@ -0,0 +1,253 @@ +#!/usr/bin/env python3 +"""Regression tests for WAF request-body inspection (Coraza via SPOE). + +Why this file exists +-------------------- +Measured 2026-10-09 on haproxy 3.0.11 + coraza-spoa v0.7.1 + CRS 4.25.0 +(report: waf-body-hardening-report.md): + + * Without `option http-buffer-request`, send-spoe-group fires at header + time and `req.body` holds only what happened to arrive with the headers. + A 14-byte `content=$a($b)` POST (CRS 933180, an ENFORCED rule) passed + uninspected when its body was written in a second send(); a body sent 2s + late, or after `Expect: 100-continue`, was never inspected at all. + * Coraza+CRS costs ~3.3 ms per KB of body. The SPOE `timeout processing` + is 100 ms and FAILS OPEN, so raising tune.bufsize to 32 KB turned a + payload at byte 0 of a 30 KB base64 body into a bypass. bufsize is a + security limit, not a performance knob. + * coraza-spoa's SPOP library refuses frames > 65535 bytes, and Coraza's + SecRequestBodyLimitAction Reject 413s even under DetectionOnly, so the + three limits must stay ordered. + * The X-WHP-WAF-Body stamp drives enforcement of body-parse errors + (200002/200003) in WHP's generated Coraza rules. A client must never be + able to set it, and a backend must never see it. + +These tests pin the rendered config. That HAProxy ACCEPTS it is +validate-rendered-config.py's job; that it BEHAVES is the live measurement. + +Running +------- + python3 scripts/test-waf-body-inspection.py +""" + +import logging +import os +import re +import sys +import tempfile +import unittest + +MODULE_DIR = os.path.abspath( + os.environ.get('HAPROXY_MANAGER_DIR', + os.path.join(os.path.dirname(os.path.abspath(__file__)), '..')) +) +os.chdir(MODULE_DIR) +sys.path.insert(0, MODULE_DIR) + +_LOG_DIR = tempfile.mkdtemp(prefix='haproxy-mgr-test-logs-') +_real_file_handler = logging.FileHandler +logging.FileHandler = ( + lambda filename, *a, **kw: _real_file_handler( + os.path.join(_LOG_DIR, os.path.basename(filename)), *a, **kw) +) + +import haproxy_manager # noqa: E402 + +SPOA_FRAME_CAP = 65535 # dropmorepackets/haproxy-go v0.0.8 maxFrameSize +CORAZA_CONFIG = os.path.join(MODULE_DIR, 'coraza-spoa', 'config.yaml') + + +def render(template, coraza): + kw = dict(crt_path='/etc/haproxy/certs', suspension_enabled=False, + coraza_spoe_backend='coraza-spoa:9000' if coraza else None, + cluster_secret=None, syslog_target='172.18.0.1:514') + return haproxy_manager.template_env.get_template(template).render(**kw) + + +def active_lines(text): + """Config lines with comments and blanks stripped (comments mention the + directives by name, so a raw substring check would pass on prose).""" + out = [] + for line in text.splitlines(): + s = line.strip() + if s and not s.startswith('#'): + out.append(s) + return out + + +def index_of(lines, pattern): + for i, line in enumerate(lines): + if re.search(pattern, line): + return i + return -1 + + +class ListenerWithCoraza(unittest.TestCase): + + def setUp(self): + self.lines = active_lines(render('hap_listener.tpl', coraza=True)) + + def test_body_is_buffered_before_inspection(self): + self.assertIn('option http-buffer-request', self.lines) + + def test_stamp_order_strip_set_inspect_strip(self): + strip1 = index_of(self.lines, r'^http-request del-header X-WHP-WAF-Body$') + size = index_of(self.lines, r'^http-request set-var\(txn\.waf_body_size\) req\.body_size if \{ req\.hdr_cnt\(content-length\) eq 1 \}$') + complete = index_of(self.lines, r'^http-request set-var\(txn\.waf_body\) str\(complete\) if \{ var\(txn\.waf_body_size\) -m found \} \{ req\.body_len,sub\(txn\.waf_body_size\) eq 0 \}$') + partial = index_of(self.lines, r'^http-request set-var\(txn\.waf_body\) str\(partial\) if \{ var\(txn\.waf_body_size\) -m found \} !\{ req\.body_len,sub\(txn\.waf_body_size\) eq 0 \}$') + hdrs_len = index_of(self.lines, r'^http-request set-var\(txn\.waf_hdrs_len\) req\.hdrs,length if !\{ var\(txn\.waf_body_size\) -m found \} \{ req\.body_len gt 0 \}$') + nocl_complete = index_of(self.lines, r'^http-request set-var\(txn\.waf_body\) str\(complete\) if \{ var\(txn\.waf_hdrs_len\) -m found \} \{ req\.body_len,add\(txn\.waf_hdrs_len\) lt (\d+) \}$') + nocl_partial = index_of(self.lines, r'^http-request set-var\(txn\.waf_body\) str\(partial\) if !\{ var\(txn\.waf_body_size\) -m found \} \{ req\.body_len gt 0 \} !\{ var\(txn\.waf_body\) -m found \}$') + stamp = index_of(self.lines, r'^http-request set-header X-WHP-WAF-Body %\[var\(txn\.waf_body\)\] if \{ var\(txn\.waf_body\) -m found \}$') + spoe = index_of(self.lines, r'^http-request send-spoe-group coraza coraza-req$') + deny = index_of(self.lines, r'^http-request return status 403 .*var\(txn\.coraza\.action\) -m str deny') + for name, idx in (('client-copy strip', strip1), ('size var', size), ('complete', complete), + ('partial', partial), ('no-CL header length', hdrs_len), + ('no-CL complete', nocl_complete), + ('no-CL partial', nocl_partial), ('stamp', stamp), + ('send-spoe-group', spoe), ('deny', deny)): + self.assertGreaterEqual(idx, 0, f'{name} rule missing from the rendered listener') + # Client copies stripped BEFORE we stamp, stamp BEFORE the SPOE call. + self.assertLess(strip1, size) + self.assertLess(size, complete) + # The no-CL `partial` fallback relies on "-m found" of an earlier + # `complete`: order is load-bearing. + self.assertLess(size, hdrs_len) + self.assertLess(hdrs_len, nocl_complete) + self.assertLess(nocl_complete, nocl_partial) + self.assertLess(max(complete, partial, nocl_partial), stamp) + self.assertLess(stamp, spoe) + # Stripped again AFTER the SPOE call so no backend ever sees it. + strip2 = [i for i, l in enumerate(self.lines) + if l == 'http-request del-header X-WHP-WAF-Body' and i > spoe] + self.assertTrue(strip2, 'X-WHP-WAF-Body must be deleted again after send-spoe-group') + self.assertLess(strip2[0], deny) + + def test_no_content_length_complete_cannot_have_filled_the_buffer(self): + """Without a Content-Length, `complete` is inferred from "the buffer + cannot have been full". body + headers thresholds must sit well below + bufsize - maxrewrite (1024), or a truncated chunked body would be + stamped complete and a parse error on it would 403.""" + line = [l for l in self.lines if 'str(complete) if { var(txn.waf_hdrs_len)' in l][0] + total_lt = int(re.search(r'add\(txn\.waf_hdrs_len\) lt (\d+)', line).group(1)) + header = active_lines(render('hap_header.tpl', coraza=True)) + bufsize = int([l for l in header if l.startswith('tune.bufsize')][0].split()[1]) + self.assertLessEqual(total_lt, (bufsize - 1024) * 0.85) + + def test_complete_and_partial_are_mutually_exclusive_conditions(self): + """With a Content-Length, `complete` and `partial` must be the same + condition, negated -- an inverted or loosened `complete` would enforce + parse errors on truncated bodies (403 on large legitimate posts).""" + c = [l for l in self.lines if 'str(complete) if { var(txn.waf_body_size)' in l] + p = [l for l in self.lines if 'str(partial) if { var(txn.waf_body_size)' in l] + self.assertEqual((len(c), len(p)), (1, 1)) + c, p = c[0], p[0] + self.assertEqual(c.split(' if ', 1)[1].replace('!{', '{'), + p.split(' if ', 1)[1].replace('!{', '{')) + self.assertIn('!{ req.body_len', p) + self.assertNotIn('!{', c) + + def test_stamp_never_set_from_client_input(self): + """The only set-header for the stamp is from our own var.""" + sets = [l for l in self.lines if re.search(r'set-header\s+X-WHP-WAF-Body', l, re.I)] + self.assertEqual(len(sets), 1) + self.assertIn('%[var(txn.waf_body)]', sets[0]) + self.assertFalse([l for l in self.lines if re.search(r'add-header\s+X-WHP-WAF-Body', l, re.I)]) + + def test_access_log_carries_coverage_fields(self): + lf = [l for l in self.lines if l.startswith('log-format ')] + self.assertEqual(len(lf), 1) + self.assertTrue(lf[0].endswith('wafb=%[var(txn.waf_body)] wafe=%[var(txn.coraza.error)]"')) + + def test_fail_open_preserved(self): + """Nothing may deny on var(txn.coraza.error): an SPOA outage must not + 503 customer sites.""" + self.assertFalse([l for l in self.lines if 'txn.coraza.error' in l and + re.search(r'\b(deny|reject|return|silent-drop|tarpit)\b', l)]) + + +class ListenerWithoutCoraza(unittest.TestCase): + + def test_waf_off_config_has_no_body_buffering(self): + lines = active_lines(render('hap_listener.tpl', coraza=False)) + self.assertNotIn('option http-buffer-request', lines) + self.assertFalse([l for l in lines if 'X-WHP-WAF-Body' in l]) + self.assertFalse([l for l in lines if 'send-spoe-group' in l]) + + def test_waf_off_header_keeps_default_bufsize(self): + lines = active_lines(render('hap_header.tpl', coraza=False)) + self.assertFalse([l for l in lines if l.startswith('tune.bufsize')]) + + +class BufsizeCeilings(unittest.TestCase): + + def setUp(self): + lines = active_lines(render('hap_header.tpl', coraza=True)) + found = [l for l in lines if l.startswith('tune.bufsize')] + self.assertEqual(len(found), 1, 'tune.bufsize must be pinned when Coraza is on') + self.bufsize = int(found[0].split()[1]) + spoe = active_lines(haproxy_manager.template_env.get_template( + 'hap_coraza_spoe_engine.tpl').render()) + m = [re.match(r'timeout\s+processing\s+(\d+)ms$', l) for l in spoe] + m = [x for x in m if x] + self.assertEqual(len(m), 1) + self.processing_ms = int(m[0].group(1)) + + def test_bufsize_fits_the_spoe_time_budget(self): + """~3.3 ms/KB measured; 31 KB took 104 ms against a 100 ms timeout and + FAILED OPEN. Keep the worst case at about half the budget.""" + worst_ms = (self.bufsize / 1024.0) * 3.3 + self.assertLessEqual(worst_ms, self.processing_ms * 0.6, + f'tune.bufsize {self.bufsize} => ~{worst_ms:.0f} ms of Coraza time ' + f'against a {self.processing_ms} ms fail-open timeout') + + def test_bufsize_below_spop_frame_cap(self): + self.assertLess(self.bufsize, SPOA_FRAME_CAP) + + def test_body_arg_is_the_full_buffered_body(self): + """The message must keep sending req.body unsliced: the `complete` + stamp compares req.body_len to Content-Length, so a slice would make + a truncated body look complete to Coraza.""" + spoe = haproxy_manager.template_env.get_template('hap_coraza_spoe_engine.tpl').render() + args = [l.strip() for l in spoe.splitlines() if l.strip().startswith('args ')] + self.assertEqual(len(args), 1) + self.assertTrue(args[0].endswith(' body=req.body')) + + +@unittest.skipUnless(os.path.exists(CORAZA_CONFIG), + 'coraza-spoa/config.yaml is not shipped in this image; ' + 'this class runs from a repo checkout') +class CorazaBodyLimits(unittest.TestCase): + + def setUp(self): + with open(CORAZA_CONFIG) as fh: + self.cfg = active_lines(fh.read()) + header = active_lines(render('hap_header.tpl', coraza=True)) + self.bufsize = int([l for l in header if l.startswith('tune.bufsize')][0].split()[1]) + + def directive(self, name): + vals = [l.split(None, 1)[1] for l in self.cfg if l.split()[0] == name] + self.assertEqual(len(vals), 1, f'{name} must be set exactly once in config.yaml') + return vals[0] + + def test_limit_strictly_above_anything_haproxy_can_send(self): + limit = int(self.directive('SecRequestBodyLimit')) + self.assertGreater(limit, SPOA_FRAME_CAP) + self.assertGreater(limit, self.bufsize) + + def test_over_limit_never_rejects(self): + """Reject interrupts even under DetectionOnly -> 403 on uploads.""" + self.assertEqual(self.directive('SecRequestBodyLimitAction'), 'ProcessPartial') + + def test_body_buffer_stays_in_memory(self): + self.assertGreaterEqual(int(self.directive('SecRequestBodyInMemoryLimit')), + int(self.directive('SecRequestBodyLimit'))) + + def test_limits_are_set_after_the_recommended_file(self): + inc = self.cfg.index('Include @coraza.conf-recommended') + lim = [i for i, l in enumerate(self.cfg) if l.startswith('SecRequestBodyLimit ')][0] + self.assertLess(inc, lim, 'the recommended file would override our limits') + + +if __name__ == '__main__': + unittest.main(verbosity=2) diff --git a/templates/hap_header.tpl b/templates/hap_header.tpl index de59f98..31513ae 100644 --- a/templates/hap_header.tpl +++ b/templates/hap_header.tpl @@ -106,6 +106,41 @@ global # Stats persistence for zero-downtime reloads stats-file /var/lib/haproxy/stats.dat +{%- if coraza_spoe_backend %} + + # WAF BODY INSPECTION LIMIT. Only emitted when Coraza is wired in, so a + # WAF-off / standalone config is unchanged byte-for-byte. 16384 is also + # HAProxy's default: this line changes no behaviour or memory, it pins + # the value so nobody raises it without reading this. + # + # The request body Coraza sees is whatever `req.body` holds when + # send-spoe-group fires, and with `option http-buffer-request` + # (hap_listener.tpl) that is "the whole body, or as much as fits in ONE + # buffer". So bufsize IS the inspection limit: a body up to roughly + # bufsize - maxrewrite (1024) - the request headers (~15 KB) is inspected + # whole and stamped `complete`; anything larger is forwarded intact, its + # first ~15 KB inspected, and stamped `partial`. Large uploads are never + # rejected or held for it. + # + # DO NOT RAISE THIS WITHOUT RE-MEASURING -- three ceilings, all measured: + # * TIME. Coraza+CRS 4.25 costs ~3.3 ms per KB of body (linear; 31 KB + # base64 = 104 ms, 15 KB = ~50 ms on a 32-core box). The SPOE + # `timeout processing` is 100 ms (hap_coraza_spoe_engine.tpl) and a + # timeout FAILS OPEN: with tune.bufsize 32768 a payload at byte 0 of a + # 30 KB base64 multipart body passed 933180 uninspected. A larger + # buffer is a bypass, not more coverage, unless the timeout (and CPU + # budget) grows with it. + # * FRAME. coraza-spoa's SPOP library (dropmorepackets/haproxy-go + # v0.0.8) refuses a HELLO whose max-frame-size exceeds 65535; HAProxy + # derives its offer from bufsize. bufsize >= 64 KB = every SPOE + # handshake fails = WAF silently off for all traffic. + # * MEMORY. maxconn 4000 x 2 buffers x bufsize, inside the 1 GB edge + # container. + # Coraza's SecRequestBodyLimit (coraza-spoa/config.yaml, 131072) must stay + # above anything this can deliver. Pinned by + # scripts/test-waf-body-inspection.py. + tune.bufsize 16384 +{%- endif %} #--------------------------------------------------------------------- # DNS resolver for Docker container name resolution diff --git a/templates/hap_listener.tpl b/templates/hap_listener.tpl index 2225e69..c6f8229 100644 --- a/templates/hap_listener.tpl +++ b/templates/hap_listener.tpl @@ -63,7 +63,12 @@ frontend web # host=/ua= identify the vhost and client; %ST/%B/%tsc give status, # bytes and the termination state that distinguishes a rate-limit # deny (PR--) from a tarpit (PT--) from a normal close. - log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r cip=%[var(txn.real_ip)] id=%ID host=%[capture.req.hdr(0)] ua=%[capture.req.hdr(1)] sni=%[ssl_fc_sni] hv=%[fc_http_major]" + # wafb=/wafe= WAF body coverage: wafb is complete|partial|- (did Coraza + # get the whole request body, see the Coraza block below); + # wafe is var(txn.coraza.error), set when the SPOE call failed or + # timed out and the request went through UNINSPECTED (fail-open). + # Both are "-" when the WAF is off or the request had no body. + log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r cip=%[var(txn.real_ip)] id=%ID host=%[capture.req.hdr(0)] ua=%[capture.req.hdr(1)] sni=%[ssl_fc_sni] hv=%[fc_http_major] wafb=%[var(txn.waf_body)] wafe=%[var(txn.coraza.error)]" # --- URI normalisation (MUST be the first path-touching block here) --- # Every path-based control in this frontend (the ACME health-check bypass, @@ -611,7 +616,61 @@ frontend web # guards (no point asking the WAF about requests we're already dropping) # and AFTER the real-client-IP resolution (so Coraza sees the right src). filter spoe engine coraza config /etc/haproxy/coraza-spoe.cfg + + # --- Make Coraza actually see the request body ------------------------- + # send-spoe-group sends `req.body` = whatever body bytes are in the buffer + # AT THAT MOMENT. Without buffering that is usually NOTHING: measured on + # haproxy 3.0.11 + coraza-spoa v0.7.1, a 14-byte `content=$a($b)` POST + # whose body was written in a second send() right after the headers + # passed 933180 (an enforced rule) uninspected, while the identical bytes + # in one send() were blocked. A body sent 2s late, or after an + # `Expect: 100-continue`, was never inspected at all. + # + # `option http-buffer-request` makes HAProxy wait, before evaluating the + # http-request rules, until the WHOLE body is in or the buffer is full, + # bounded by `timeout http-request` (30s, hap_header.tpl). It answers + # `Expect: 100-continue` itself so the client actually sends the body. + # It does NOT hold uploads: once one buffer (tune.bufsize, 16 KB) is full + # the request is forwarded and the rest streams through as before -- a + # 50 MB upload is not stored at the edge. + option http-buffer-request + + # Tell Coraza whether it got the whole body. Coraza cannot tell on its + # own (REQUEST_BODY_LENGTH is only filled by the urlencoded/raw + # processors), and the difference matters: a TRUNCATED JSON body, or a + # multipart body cut inside a boundary/part header, fails to parse by + # construction, so the body-parse-error rules (200002/200003) may only + # deny a COMPLETE body. WHP's generated Coraza rules + # (regenerate-coraza-overrides.sh) key on this header: + # complete -> 200002/200003 deny on multipart/JSON (enforce mode) + # partial -> 200002/200003 removed for this request (every mode) + # + # With a Content-Length: complete iff req.body_len (buffered) equals + # req.body_size (= Content-Length). + # Without one (HTTP/1.1 chunked, HTTP/2-3 without content-length) HAProxy + # 3.0 has no "end of message" fetch and req.body_size is just what is + # buffered. But http-buffer-request only releases the request early when + # the body ENDED or the buffer FILLED (a stalled body gets a 408 instead), + # and body + headers well below one buffer (12 KB of the ~15 KB usable) + # cannot have filled it. So that is complete, anything else partial. + # Without this branch an attacker only had to add + # `Transfer-Encoding: chunked` to escape enforcement (measured). + # + # Client-supplied copies are deleted FIRST so it cannot be forged, and it + # is deleted again after the SPOE call so backends never see it. + # txn.waf_body is also logged (`wafb=` in the log-format above) so the + # complete/partial mix is measurable per host. + http-request del-header X-WHP-WAF-Body + http-request set-var(txn.waf_body_size) req.body_size if { req.hdr_cnt(content-length) eq 1 } + http-request set-var(txn.waf_body) str(complete) if { var(txn.waf_body_size) -m found } { req.body_len,sub(txn.waf_body_size) eq 0 } + http-request set-var(txn.waf_body) str(partial) if { var(txn.waf_body_size) -m found } !{ req.body_len,sub(txn.waf_body_size) eq 0 } + http-request set-var(txn.waf_hdrs_len) req.hdrs,length if !{ var(txn.waf_body_size) -m found } { req.body_len gt 0 } + http-request set-var(txn.waf_body) str(complete) if { var(txn.waf_hdrs_len) -m found } { req.body_len,add(txn.waf_hdrs_len) lt 12288 } + http-request set-var(txn.waf_body) str(partial) if !{ var(txn.waf_body_size) -m found } { req.body_len gt 0 } !{ var(txn.waf_body) -m found } + http-request set-header X-WHP-WAF-Body %[var(txn.waf_body)] if { var(txn.waf_body) -m found } + http-request send-spoe-group coraza coraza-req + http-request del-header X-WHP-WAF-Body # Enforce Coraza's verdict. The SPOA sets var(txn.coraza.action) to # "deny" / "drop" / "redirect" when a rule with the corresponding