Merge pull request 'fix(haproxy): set X-CLIENT-IP instead of adding it' (#11) from fix/x-client-ip-set-not-add into main

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

CLAUDE.md

@@ -7,8 +7,55 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 ### Testing
 - **API Testing**: `./scripts/test-api.sh` - Tests all API endpoints with optional authentication
 - **Certificate Request Testing**: `./scripts/test-certificate-request.sh` - Tests certificate generation endpoints
+- **Stick-table contract**: `python3 scripts/test-stick-table-contract.py` - offline; holds the templates' `store` clauses, `STICK_TABLE_FIELD_CONTRACT`, and every consumer to each other. Run it after touching any `stick-table` line.
+- **Runtime-map contract**: `python3 scripts/test-runtime-map-contract.py` - offline; asserts the runtime map commands are `@1`-prefixed, reference the map by FILE PATH (never `#<id>`), carry the value `1`, and that every captured rejection is classified as a failure. Run it after touching any `add map`/`del map`/`clear map` path.
+- **Certificate destruction safety**: `python3 scripts/test-cert-write-safety.py` - offline; asserts a live `.pem` is never truncated, removed, or its certbot lineage deleted while any configured domain still references it (one bundle serves many names, so `ssl_cert_path` is routinely shared). Run it after touching any `os.remove`/`certbot delete`/PEM-write path.
 - **Manual Testing**: Run `curl` commands against `http://localhost:8000` endpoints as shown in README.md
 
+### Reading stick tables (and why it is easy to get silently wrong)
+
+`/tmp/haproxy-cli` is HAProxy's **master** CLI socket. Worker commands
+(`show table`, `show map`, `add map`, ...) need an `@1` prefix. Without it
+HAProxy answers `Unknown command: 'show', ...` **and socat still exits 0** — so
+an exit-status check passes and the help text gets parsed as data. Always use
+`haproxy_cli(cmd, worker=True)` in Python, which inspects the response body.
+
+Stick-table entries are `name=value` / `name(window_ms)=value` pairs, not fixed
+columns; the first token is an allocation pointer (`0x...:`), not the key. Parse
+by NAME, and treat a missing field as an ERROR — never default it to `0`. The
+`web` table stores only `conn_cur`, `conn_rate`, `http_req_rate`,
+`http_err_rate`; it holds **no history and no counter of past blocks**. What was
+actually denied/tarpitted is in the edge access log on the **host** at
+`/var/log/haproxy.log` (shipped 2026.08.8), not in any stick table.
+
+This is written down because `/api/security/stats` and `show-tarpit-ips.sh`
+reported "Scan Count"/"BLOCKED" figures parsed from `gpc0`/`gpc1` — fields no
+stick table has ever stored — for their entire existence. See the header of
+`haproxy_tarpit_config.txt` and the contract test.
+
+### Changing a runtime map (`add map` / `del map`)
+
+Same socket, two more ways to fail silently — and both were live in
+`add_ip_to_runtime_map()`/`remove_ip_from_runtime_map()` for their whole
+existence:
+
+* **Reference the map by FILE PATH, never `#<id>`.** Ids are assigned at
+  config-parse time and move on every config regeneration (on whp01
+  `blocked_ips.map` is 37, `trusted_ips.map` is 10 — there is no id 0). Use
+  `add map /etc/haproxy/blocked_ips.map <ip> 1`.
+* **A mutation answers NOTHING on success**, so an empty body is the only
+  success — any output at all is a rejection. Worse, `@1 add map #0 <ip> 1`
+  *also* answers nothing and adds nothing, so the body cannot prove an add
+  worked. **Read it back** with `@1 get map <path> <key>`.
+* Entries must carry the value `1`; haproxy.cfg matches with
+  `map_ip(...,0) -m int gt 0`, so a valueless entry does not block.
+
+In Python use `haproxy_cli(cmd, worker=True, expect_empty=True)` for mutations
+and `runtime_map_lookup()` / `runtime_map_keys()` to verify. The runtime map is
+only a fast path: `/etc/haproxy/blocked_ips.map` is authoritative and HAProxy
+re-reads it on reload, so a failed runtime command must degrade to
+"enforced on reload" and be reported, never swallowed.
+
 ### Running the Application
 - **Docker Build**: `docker build -t haproxy-manager .`
 - **Local Development**: `python haproxy_manager.py` (requires HAProxy, certbot, and dependencies installed)

IP_BLOCKING_API.md

@@ -508,20 +508,45 @@ curl -X POST http://localhost:8000/api/blocked-ips/sync \
 
 For advanced users, you can interact directly with HAProxy's runtime API:
 
+Three things about these commands are easy to get wrong, and each one fails
+**silently** (socat exits 0 either way — the rejection, if any, is only in the
+response body):
+
+* `/tmp/haproxy-cli` is HAProxy's **master** CLI socket. Map commands are
+  worker commands and need the `@1` prefix. Without it the reply is
+  `Unknown command: 'add', ...`.
+* Reference the map by its **file path**, never by `#<id>`. Ids are assigned at
+  config-parse time and move on every config regeneration (on a live edge,
+  `blocked_ips.map` is id 37, `trusted_ips.map` is 10 — there is no id 0).
+  Worse, `@1 add map #0 <ip> 1` returns an **empty** reply and adds nothing.
+* Entries must carry the value `1`. `haproxy.cfg` matches with
+  `map_ip(...,0) -m int gt 0`, so a valueless entry does not block. (`add map`
+  with no value is rejected: `'add map' expects three parameters ...`.)
+
 ```bash
+MAP=/etc/haproxy/blocked_ips.map
+
 # Add IP to runtime (immediate effect)
-echo "add map #0 192.168.1.100" | socat stdio /var/run/haproxy.sock
+echo "@1 add map $MAP 192.168.1.100 1" | socat stdio /tmp/haproxy-cli
 
 # Remove IP from runtime
-echo "del map #0 192.168.1.100" | socat stdio /var/run/haproxy.sock
+echo "@1 del map $MAP 192.168.1.100" | socat stdio /tmp/haproxy-cli
+
+# Confirm what actually happened (do not trust the exit status)
+echo "@1 get map $MAP 192.168.1.100" | socat stdio /tmp/haproxy-cli
 
 # Clear all blocked IPs from runtime
-echo "clear map #0" | socat stdio /var/run/haproxy.sock
+echo "@1 clear map $MAP" | socat stdio /tmp/haproxy-cli
 
-# Show all runtime map entries
+# Show all runtime map entries, and the map ids currently in use
-echo "show map #0" | socat stdio /var/run/haproxy.sock
+echo "@1 show map $MAP" | socat stdio /tmp/haproxy-cli
+echo "@1 show map" | socat stdio /tmp/haproxy-cli
 ```
 
+The runtime map is a **fast path only**. `/etc/haproxy/blocked_ips.map` is
+authoritative: HAProxy re-reads it on reload, so a failed runtime command
+delays a block until the next reload rather than losing it.
+
 ## Migration from ACL Method
 
 If you're upgrading from the old ACL-based method:

MIGRATION_GUIDE.md

@@ -50,14 +50,22 @@ http-request deny status 403 if { src -f /etc/haproxy/blocked_ips.map }
 - **Graceful error handling**
 
 ### 2. Runtime IP Management
+Map commands go to a **worker** (`@1`), reference the map by **file path**
+(ids move between config regenerations, and `#0` silently adds nothing), and
+carry the value `1` that `map_ip(...,0) -m int gt 0` matches on:
+
 ```bash
 # Add IP without reload (immediate effect)
-echo "add map #0 192.168.1.100" | socat stdio /var/run/haproxy.sock
+echo "@1 add map /etc/haproxy/blocked_ips.map 192.168.1.100 1" | socat stdio /tmp/haproxy-cli
 
 # Remove IP without reload
-echo "del map #0 192.168.1.100" | socat stdio /var/run/haproxy.sock
+echo "@1 del map /etc/haproxy/blocked_ips.map 192.168.1.100" | socat stdio /tmp/haproxy-cli
 ```
 
+socat exits 0 even when HAProxy rejects the command, so read the response body
+(or read the entry back with `@1 get map ...`) rather than the exit status.
+See IP_BLOCKING_API.md for the full set.
+
 ### 3. New API Endpoints
 
 #### Safe Config Reload

VERSION

@@ -1 +1 @@
-2026.08.8
+2026.09.1

haproxy_manager.py

@@ -1624,6 +1624,61 @@ def request_certificates():
     else:
         return jsonify(response), 500  # All failed
 
+def lineage_name_for_cert_path(cert_path):
+    """The certbot lineage name implied by a published bundle path.
+
+    Every writer in this module publishes to ``{SSL_CERTS_DIR}/<name>.pem`` and
+    issues that lineage with ``--cert-name <name>`` (see request_ssl() and
+    request_ssl_bundle()); _quarantine_superseded_certs() derives the lineage
+    from the filename the same way. So the basename minus ``.pem`` IS the
+    lineage, and it is NOT necessarily the domain being removed: a bundle
+    issued as ``--cert-name example.com`` also serves www.example.com and any
+    other SAN, all of whose DB rows point at ``/etc/haproxy/certs/example.com.pem``.
+    """
+    if not cert_path:
+        return None
+    base = os.path.basename(cert_path)
+    return base[:-len('.pem')] if base.endswith('.pem') else base
+
+
+def domains_referencing_cert_path(cursor, cert_path):
+    """Domains (still in the table) whose ssl_cert_path is exactly `cert_path`.
+
+    Call this AFTER the row being removed has been deleted, so the answer is
+    "who else still needs this file".
+
+    ssl_enabled is deliberately NOT filtered on. HAProxy binds the whole crt
+    directory, so a file is load-bearing for any row that names it; and the
+    asymmetry of the two mistakes is total - keeping a stale PEM costs nothing,
+    unlinking a live one is HTTPS down for every name it serves.
+    """
+    if not cert_path:
+        return []
+    cursor.execute(
+        'SELECT domain FROM domains WHERE ssl_cert_path = ? ORDER BY domain',
+        (cert_path,))
+    return [row[0] for row in cursor.fetchall()]
+
+
+def domains_referencing_lineage(cursor, lineage):
+    """Domains (still in the table) served by certbot lineage `lineage`.
+
+    Same shape as domains_referencing_cert_path(), one level further back:
+    `certbot delete --cert-name X` destroys the archive, live symlinks and
+    renewal config for X. If any remaining domain is served by a bundle
+    published from that lineage, deleting it means the next renewal silently
+    stops happening for all of them and the material cannot be recovered
+    without a fresh, rate-limited ACME order.
+    """
+    if not lineage:
+        return []
+    cursor.execute(
+        "SELECT domain, ssl_cert_path FROM domains "
+        "WHERE ssl_cert_path IS NOT NULL AND ssl_cert_path != ''")
+    return sorted({domain for domain, path in cursor.fetchall()
+                   if lineage_name_for_cert_path(path) == lineage})
+
+
 @app.route('/api/domain', methods=['DELETE'])
 @require_api_key
 def remove_domain():
@@ -1662,8 +1717,33 @@ def remove_domain():
             # Delete domain
             cursor.execute('DELETE FROM domains WHERE id = ?', (domain_id,))
 
+            # Refcount the certificate BEFORE anything is unlinked or deleted,
+            # with the row above already gone so the query answers "who else
+            # still needs this". One .pem serves many names: request_ssl_bundle()
+            # issues a single SAN cert and points EVERY included domain's row at
+            # the same /etc/haproxy/certs/<primary>.pem. Removing one of those
+            # names used to os.remove() that file and `certbot delete` its
+            # lineage unconditionally - taking HTTPS down for every other name
+            # in the bundle, and destroying the only recoverable copy with it.
+            cert_path_users = domains_referencing_cert_path(cursor, ssl_cert_path)
+            lineage = lineage_name_for_cert_path(ssl_cert_path) if ssl_cert_path else None
+            # The lineage to delete is the one this domain's bundle came from,
+            # not the domain's own name - for a SAN member those differ.
+            lineage_users = domains_referencing_lineage(cursor, lineage)
+
+        cert_retained_for = []
+        lineage_retained_for = []
+
         # Delete SSL certificate from HAProxy certs directory
         if ssl_enabled and ssl_cert_path:
+            if cert_path_users:
+                cert_retained_for = cert_path_users
+                logger.info(
+                    "Kept HAProxy certificate file %s while removing %s: still "
+                    "referenced by %d other domain(s): %s",
+                    ssl_cert_path, domain, len(cert_path_users),
+                    ', '.join(cert_path_users))
+            else:
                 try:
                     os.remove(ssl_cert_path)
                     logger.info(f"Removed HAProxy certificate file: {ssl_cert_path}")
@@ -1672,23 +1752,39 @@ def remove_domain():
 
         # Remove certificate from certbot
         if ssl_enabled:
+            if not lineage:
+                logger.info(
+                    "Skipping certbot delete for %s: no certificate path on the "
+                    "removed row, so no lineage can be attributed to it", domain)
+            elif lineage_users:
+                lineage_retained_for = lineage_users
+                logger.info(
+                    "Kept Let's Encrypt lineage %s while removing %s: still "
+                    "serving %d other domain(s): %s",
+                    lineage, domain, len(lineage_users), ', '.join(lineage_users))
+            else:
                 try:
                     result = subprocess.run(
-                    ['certbot', 'delete', '--cert-name', domain, '--non-interactive'],
+                        ['certbot', 'delete', '--cert-name', lineage, '--non-interactive'],
                         capture_output=True, text=True
                     )
                     if result.returncode == 0:
-                    logger.info(f"Removed Let's Encrypt certificate for {domain}")
+                        logger.info(f"Removed Let's Encrypt certificate {lineage} for {domain}")
                     else:
-                    logger.warning(f"Failed to remove Let's Encrypt certificate for {domain}: {result.stderr}")
+                        logger.warning(f"Failed to remove Let's Encrypt certificate {lineage} for {domain}: {result.stderr}")
                 except Exception as e:
-                logger.warning(f"Error removing Let's Encrypt certificate for {domain}: {e}")
+                    logger.warning(f"Error removing Let's Encrypt certificate {lineage} for {domain}: {e}")
 
         # Regenerate HAProxy config
         generate_config()
 
         log_operation('remove_domain', True, f'Domain {domain} removed successfully')
-        return jsonify({'status': 'success', 'message': 'Domain configuration removed'})
+        return jsonify({
+            'status': 'success',
+            'message': 'Domain configuration removed',
+            'certificate_retained_for': cert_retained_for,
+            'lineage_retained_for': lineage_retained_for,
+        })
 
     except Exception as e:
         log_operation('remove_domain', False, str(e))
@@ -1735,8 +1831,10 @@ def add_blocked_ip():
             log_operation('add_blocked_ip', False, f'Failed to update map file for {ip_address}')
             return jsonify({'status': 'error', 'message': 'Failed to update blocked IPs map file'}), 500
         
-        # Add to runtime map for immediate effect
+        # Add to runtime map for immediate effect. The map FILE above is what
-        add_ip_to_runtime_map(ip_address)
+        # actually enforces the block once HAProxy re-reads it; this is only
+        # the fast path, so a False here is reported, not fatal.
+        runtime_ok = add_ip_to_runtime_map(ip_address)
         
         # Reload HAProxy to ensure consistency
         try:
@@ -1753,8 +1851,18 @@ def add_blocked_ip():
         except Exception as e:
             logger.warning(f"Error reloading HAProxy after blocking IP {ip_address}: {e}")
         
-        log_operation('add_blocked_ip', True, f'IP {ip_address} blocked successfully')
+        log_operation('add_blocked_ip', True,
-        return jsonify({'status': 'success', 'blocked_ip_id': blocked_ip_id, 'message': f'IP {ip_address} has been blocked'})
+                      f'IP {ip_address} blocked successfully '
+                      f'(runtime map fast path: {"ok" if runtime_ok else "FAILED, enforced on reload"})')
+        return jsonify({
+            'status': 'success',
+            'blocked_ip_id': blocked_ip_id,
+            'message': f'IP {ip_address} has been blocked',
+            # False = the block is enforced from the map file at reload rather
+            # than instantly. The caller can tell the difference; it used to be
+            # reported as instant unconditionally.
+            'runtime_map_updated': runtime_ok
+        })
     except sqlite3.IntegrityError:
         log_operation('add_blocked_ip', False, f'IP {ip_address} is already blocked')
         return jsonify({'status': 'error', 'message': 'IP address is already blocked'}), 409
@@ -1790,8 +1898,9 @@ def remove_blocked_ip():
             log_operation('remove_blocked_ip', False, f'Failed to update map file for {ip_address}')
             return jsonify({'status': 'error', 'message': 'Failed to update blocked IPs map file'}), 500
         
-        # Remove from runtime map for immediate effect
+        # Remove from runtime map for immediate effect. As with blocking, the
-        remove_ip_from_runtime_map(ip_address)
+        # map file is authoritative and the reload below picks it up.
+        runtime_ok = remove_ip_from_runtime_map(ip_address)
         
         # Reload HAProxy to ensure consistency
         try:
@@ -1808,8 +1917,14 @@ def remove_blocked_ip():
         except Exception as e:
             logger.warning(f"Error reloading HAProxy after unblocking IP {ip_address}: {e}")
         
-        log_operation('remove_blocked_ip', True, f'IP {ip_address} unblocked successfully')
+        log_operation('remove_blocked_ip', True,
-        return jsonify({'status': 'success', 'message': f'IP {ip_address} has been unblocked'})
+                      f'IP {ip_address} unblocked successfully '
+                      f'(runtime map fast path: {"ok" if runtime_ok else "FAILED, applied on reload"})')
+        return jsonify({
+            'status': 'success',
+            'message': f'IP {ip_address} has been unblocked',
+            'runtime_map_updated': runtime_ok
+        })
     except Exception as e:
         log_operation('remove_blocked_ip', False, str(e))
         return jsonify({'status': 'error', 'message': str(e)}), 500
@@ -1843,103 +1958,375 @@ def sync_blocked_ips():
             cursor.execute('SELECT ip_address FROM blocked_ips ORDER BY ip_address')
             blocked_ips = [row[0] for row in cursor.fetchall()]
         
-        # Try to clear all entries from runtime map (might fail if empty, that's ok)
+        # Clear the runtime map and re-add every blocked IP. The map is
+        # referenced by FILE PATH: `clear map #0` (what this used to send, with
+        # no `@1` either) was answered by the master socket with "Unknown
+        # command: 'clear'" and socat exited 0, so this whole block was a no-op
+        # that reported a full sync.
         try:
-            if os.path.exists(HAPROXY_SOCKET_PATH):
+            haproxy_cli('clear map %s' % BLOCKED_IPS_MAP_PATH,
-                socket_path = HAPROXY_SOCKET_PATH
+                        worker=True, expect_empty=True)
-            else:
+        except HaproxyCliError as e:
-                socket_path = '/tmp/haproxy-cli'
+            log_operation('sync_blocked_ips', False, f'Failed to clear runtime map: {e}')
-            
+            logger.warning("Failed to clear runtime map: %s. The map file is "
-            subprocess.run(f'echo "clear map #0" | socat stdio {socket_path}', 
+                           "still authoritative and correct.", e)
-                         shell=True, capture_output=True)
-        except:
-            pass  # Clear might fail if map is empty
-        
-        # Add all IPs to runtime map
-        success_count = 0
-        for ip in blocked_ips:
-            if add_ip_to_runtime_map(ip):
-                success_count += 1
-        
-        log_operation('sync_blocked_ips', True, f'Synced {success_count}/{len(blocked_ips)} IPs to runtime map')
             return jsonify({
-            'status': 'success', 
+                'status': 'error',
-            'message': f'Synced {success_count}/{len(blocked_ips)} IPs to runtime map',
+                'message': f'Failed to clear runtime map: {e}',
+                'map_file_updated': True,
+                'runtime_map_synced': False,
+                'total_ips': len(blocked_ips)
+            }), 500
+
+        # verify=False here: one `show map` read-back below costs a single
+        # round trip instead of one per IP, and answers the same question for
+        # the whole set.
+        accepted = 0
+        for ip in blocked_ips:
+            if add_ip_to_runtime_map(ip, verify=False):
+                accepted += 1
+
+        # Ground truth, not a count of commands that did not visibly complain.
+        try:
+            present = runtime_map_keys(BLOCKED_IPS_MAP_PATH)
+        except HaproxyCliError as e:
+            log_operation('sync_blocked_ips', False, f'Could not read back runtime map: {e}')
+            return jsonify({
+                'status': 'error',
+                'message': f'Could not read back the runtime map to verify the sync: {e}',
+                'map_file_updated': True,
+                'runtime_map_synced': False,
+                'total_ips': len(blocked_ips)
+            }), 500
+
+        missing = [ip for ip in blocked_ips if ip not in present]
+        synced = len(blocked_ips) - len(missing)
+        ok = not missing
+        if missing:
+            logger.warning(
+                "Runtime map sync incomplete: %d/%d IPs are not in %s (first "
+                "few: %s). They remain blocked via the map file on reload.",
+                len(missing), len(blocked_ips), BLOCKED_IPS_MAP_PATH, missing[:5])
+
+        log_operation('sync_blocked_ips', ok,
+                      f'Verified {synced}/{len(blocked_ips)} IPs present in the runtime map')
+        return jsonify({
+            'status': 'success' if ok else 'partial',
+            'message': f'Verified {synced}/{len(blocked_ips)} IPs present in the runtime map',
             'total_ips': len(blocked_ips),
-            'synced_ips': success_count
+            'synced_ips': synced,
-        })
+            'accepted_commands': accepted,
+            'missing_ips': missing[:50],
+            'runtime_map_synced': ok
+        }), (200 if ok else 207)
     except Exception as e:
         log_operation('sync_blocked_ips', False, str(e))
         return jsonify({'status': 'error', 'message': str(e)}), 500
 
+# ---------------------------------------------------------------------------
+# HAProxy runtime API (stick tables)
+#
+# WHY THIS SECTION IS SO DEFENSIVE
+# --------------------------------
+# The previous /api/security/stats read `gpc0` and `gpc1` out of `show table
+# web` and reported them as "scan count" / "offense count" / "blocked". No
+# stick table in this repo has EVER stored a general-purpose counter — see
+# STICK_TABLE_FIELD_CONTRACT below and the `store` clauses in
+# templates/hap_listener.tpl and templates/hap_security_tables.tpl. Three
+# separate silences let that survive:
+#
+#   1. `int(parts[3])` on a positional split raised ValueError on `exp=368842`
+#      and the loop just `continue`d, so every row was skipped and the endpoint
+#      always answered `active_threats: 0` with an empty list. An operator
+#      reading that saw "no threats" and could not tell it apart from "the
+#      parser is broken".
+#   2. The command was sent WITHOUT a worker prefix. /tmp/haproxy-cli is the
+#      MASTER socket; `show table web` there is answered with "Unknown command:
+#      'show', but maybe one of the following ones is a better match: ..." --
+#      and socat still exits 0, so `result.returncode != 0` never fired. The
+#      reported `total_tracked_ips` was literally the number of lines in that
+#      help text minus one (8), while the real table held 388 entries.
+#   3. The shell consumers defaulted every missing field to 0 (`${gpc0:-0}`),
+#      so a field that does not exist rendered as a confident zero.
+#
+# Rules for anything added here, all three aimed at the same failure mode:
+#   * Read the CONTRACT, not positions. Stick-table output is `name=value` /
+#     `name(window)=value` pairs whose order and presence follow the template's
+#     `store` clause. Positional indexing silently reads the wrong column the
+#     moment that clause changes.
+#   * NEVER default a missing field to a number. A field the table does not
+#     store must surface as an error naming the field, not as 0.
+#   * NEVER trust socat's exit status. HAProxy reports command errors in the
+#     response BODY and the socket still closes cleanly. Use haproxy_cli().
+#
+# scripts/test-stick-table-contract.py holds STICK_TABLE_FIELD_CONTRACT, the
+# rendered templates and the shell consumers to each other, and fails if any
+# one of them drifts.
+# ---------------------------------------------------------------------------
+
+# What each stick table ACTUALLY stores, per its `store` clause. The single
+# source of truth for every consumer in this repo. Keep in sync with the
+# templates -- the contract test enforces that, in both directions.
+STICK_TABLE_FIELD_CONTRACT = {
+    'web': ('conn_cur', 'conn_rate', 'http_req_rate', 'http_err_rate'),
+    'wp_bruteforce': ('http_req_rate',),
+    'xmlrpc_bruteforce': ('http_req_rate',),
+}
+
+# Metadata every stick-table entry carries regardless of the `store` clause.
+STICK_TABLE_ENTRY_META = ('key', 'use', 'exp', 'shard')
+
+# HAProxy answers a rejected runtime command in the response body and the
+# socket still closes 0. These are the prefixes it uses.
+_HAPROXY_CLI_ERROR_MARKERS = (
+    'Unknown command',
+    'No such table',
+    # `add map #0 ...` / `get map /etc/haproxy/nope.map ...` -- captured
+    # verbatim from HAProxy 3.0.11 on the live edge.
+    'Unknown map identifier',
+    'Key not found',
+    'Permission denied',
+    "Can't find the specified process",
+    'unknown process',
+    'Missing ',
+)
+
+_STICK_TABLE_TOKEN_RE = re.compile(r'^([a-z_][a-z0-9_]*)(?:\(([^)]*)\))?=(.*)$')
+_STICK_TABLE_HEADER_RE = re.compile(
+    r'^#\s*table:\s*(?P<name>[^,]+),\s*type:\s*(?P<type>[^,]+),\s*'
+    r'size:\s*(?P<size>\d+),\s*used:\s*(?P<used>\d+)')
+
+
+class HaproxyCliError(RuntimeError):
+    """A runtime-API command was rejected, timed out, or answered nothing.
+
+    Exists so a rejected command cannot be mistaken for an empty result. That
+    distinction is the whole point of this module's stick-table code.
+
+    `.responses` holds the raw, stripped response body of every attempt, so a
+    caller can tell one rejection apart from another (`del map` answering
+    "Key not found." is a no-op, not a failure) without regex-matching the
+    formatted message.
+    """
+
+    responses = ()
+
+
+def _haproxy_socket_path():
+    return HAPROXY_SOCKET_PATH if os.path.exists(HAPROXY_SOCKET_PATH) else '/tmp/haproxy-cli'
+
+
+def _cli_response_is_error(text):
+    if text is None:
+        return True
+    head = text.lstrip()
+    return any(head.startswith(marker) for marker in _HAPROXY_CLI_ERROR_MARKERS)
+
+
+def _cli_send(command, socket_path, timeout):
+    proc = subprocess.run(
+        ['socat', 'stdio', socket_path],
+        input=command + '\n', capture_output=True, text=True, timeout=timeout)
+    if proc.returncode != 0:
+        raise HaproxyCliError(
+            'socat failed talking to %s (exit %d): %s'
+            % (socket_path, proc.returncode, (proc.stderr or '').strip()))
+    return proc.stdout
+
+
+def haproxy_cli(command, worker=False, timeout=None, expect_empty=False):
+    """Send one runtime-API command and return its response, or raise.
+
+    `worker=True` marks a command that only the WORKER answers (show table,
+    show map, add map, ...). On this deployment the socket is HAProxy's MASTER
+    CLI, where those need an `@1` prefix; on a plain stats socket they must NOT
+    have one. Rather than guessing from configuration that can change under us,
+    try the prefixed form and fall back -- and raise if BOTH are rejected,
+    instead of returning HAProxy's help text as if it were data.
+
+    `expect_empty=True` is for MUTATING commands (add/del/clear map, set map).
+    HAProxy answers those with nothing at all on success, so the rule inverts:
+    an empty body is the success, and ANY non-empty body is a rejection. That
+    is deliberately stricter than matching _HAPROXY_CLI_ERROR_MARKERS -- the
+    marker list can only ever recognise the rejections someone has already
+    seen, and a mutation that prints anything has not done what was asked. Two
+    real examples this catches that the marker list did not:
+    `'add map' expects three parameters ...` and `Unknown map identifier.`
+    """
+    socket_path = _haproxy_socket_path()
+    timeout = timeout if timeout is not None else DEFAULT_SUBPROCESS_TIMEOUT
+    attempts = (['@1 ' + command, command] if worker else [command])
+    failures = []
+    bodies = []
+    for attempt in attempts:
+        try:
+            out = _cli_send(attempt, socket_path, timeout)
+        except subprocess.TimeoutExpired:
+            raise HaproxyCliError('timed out after %ss running %r on %s'
+                                  % (timeout, attempt, socket_path))
+        body = out.strip()
+        if expect_empty:
+            if not body:
+                return out
+        elif body and not _cli_response_is_error(out):
+            return out
+        bodies.append(body)
+        failures.append('%r -> %r' % (attempt, body[:200] or '<empty response>'))
+    error = HaproxyCliError(
+        'HAProxy rejected %r on %s (socat exited 0 -- the rejection is in the '
+        'response body, which is exactly why this is checked): %s'
+        % (command, socket_path, '; '.join(failures)))
+    error.responses = tuple(bodies)
+    raise error
+
+
+def parse_stick_table_entry(line):
+    """{field: {'value': str, 'window_ms': int|None}} for one `show table` row.
+
+    Parses `name=value` / `name(window)=value` pairs by NAME. The leading
+    `0x...:` allocation pointer is skipped -- reading it as the key is how the
+    old code came to report memory addresses as IP addresses.
+    """
+    fields = {}
+    for token in line.split():
+        m = _STICK_TABLE_TOKEN_RE.match(token)
+        if not m:
+            continue  # the 0x...: pointer, or anything else unnamed
+        name, window, value = m.group(1), m.group(2), m.group(3)
+        fields[name] = {
+            'value': value,
+            'window_ms': int(window) if window and window.isdigit() else None,
+        }
+    return fields
+
+
+def read_stick_table(table):
+    """(header dict, [(raw line, parsed fields)]) for a stick table.
+
+    Raises HaproxyCliError if the response is not a stick-table dump, or if any
+    row is missing a field the contract says the table stores. A field the
+    table does not carry is an ERROR here, never a zero.
+    """
+    expected = STICK_TABLE_FIELD_CONTRACT.get(table)
+    if expected is None:
+        raise HaproxyCliError(
+            'no field contract for stick table %r; add it to '
+            'STICK_TABLE_FIELD_CONTRACT (and to the templates) first' % table)
+
+    raw = haproxy_cli('show table %s' % table, worker=True)
+    lines = raw.strip().split('\n')
+    header = _STICK_TABLE_HEADER_RE.match(lines[0]) if lines else None
+    if not header:
+        raise HaproxyCliError(
+            'response to `show table %s` is not a stick-table dump; first line '
+            'was %r' % (table, lines[0][:200] if lines else ''))
+
+    entries = []
+    for line in lines[1:]:
+        if not line.strip() or line.lstrip().startswith('#'):
+            continue
+        fields = parse_stick_table_entry(line)
+        if 'key' not in fields:
+            raise HaproxyCliError(
+                'stick-table row for %r has no key= field: %r' % (table, line[:200]))
+        missing = [f for f in expected if f not in fields]
+        if missing:
+            raise HaproxyCliError(
+                'stick table %r no longer stores %s -- STICK_TABLE_FIELD_CONTRACT '
+                'and the `store` clause in templates/hap_listener.tpl have drifted '
+                'apart. Present: %s. Offending row: %r'
+                % (table, ', '.join(missing),
+                   ', '.join(sorted(fields)), line[:200]))
+        entries.append((line, fields))
+
+    return {
+        'name': header.group('name'),
+        'type': header.group('type'),
+        'size': int(header.group('size')),
+        'used': int(header.group('used')),
+    }, entries
+
+
 @app.route('/api/security/stats', methods=['GET'])
 @require_api_key
 def get_security_stats():
-    """Get current security statistics from HAProxy stick table"""
+    """Per-source connection and request rates from the `web` stick table.
+
+    Reports ONLY what the table stores: conn_cur, conn_rate, http_req_rate and
+    http_err_rate, each with the window HAProxy is actually counting over. It
+    deliberately does NOT classify a "threat level" or report a "blocked" flag:
+    the thresholds live in templates/hap_listener.tpl and a copy here would be
+    a second source of truth free to drift, which is the class of bug this
+    endpoint used to be. Sources at or over a limit are visible from the rates
+    themselves, and the enforcement that actually happened -- 429s, tarpits
+    (termination state PT), WAF denials -- is in the edge access log on the
+    HOST at /var/log/haproxy.log, which records per-request outcomes the stick
+    table never held.
+
+    Query params:
+      limit         max sources returned (default 50)
+      min_req_rate  only sources at or above this http_req_rate (default 1,
+                    i.e. sources with current activity; pass 0 for all)
+    """
     try:
-        if os.path.exists(HAPROXY_SOCKET_PATH):
+        limit = max(1, min(int(request.args.get('limit', 50)), 1000))
-            socket_path = HAPROXY_SOCKET_PATH
+        min_req_rate = max(0, int(request.args.get('min_req_rate', 1)))
-        else:
+    except ValueError:
-            socket_path = '/tmp/haproxy-cli'
+        return jsonify({'status': 'error',
+                        'message': 'limit and min_req_rate must be integers'}), 400
 
-        # Get stick table data
-        cmd = f'echo "show table web" | socat stdio {socket_path}'
-        result = subprocess.run(cmd, shell=True, capture_output=True, text=True)
-
-        if result.returncode != 0:
-            return jsonify({'status': 'error', 'message': 'Failed to get stick table data'}), 500
-
-        # Parse stick table output
-        lines = result.stdout.strip().split('\n')
-        threats = []
-
-        for line in lines[1:]:  # Skip header
-            parts = line.split()
-            if len(parts) >= 8:
-                ip = parts[0]
     try:
-                    gpc0 = int(parts[3]) if len(parts) > 3 else 0
+        header, entries = read_stick_table('web')
-                    gpc1 = int(parts[4]) if len(parts) > 4 else 0
+    except HaproxyCliError as e:
-                    req_rate = int(parts[5]) if len(parts) > 5 else 0
+        log_operation('get_security_stats', False, str(e))
-                    err_rate = int(parts[6]) if len(parts) > 6 else 0
+        return jsonify({'status': 'error', 'message': str(e)}), 502
-                    conn_rate = int(parts[7]) if len(parts) > 7 else 0
-
-                    # Only include IPs with significant activity
-                    if gpc0 > 0 or gpc1 > 0 or req_rate > 30 or err_rate > 5 or conn_rate > 10:
-                        threat_level = 'low'
-                        if gpc1 > 2:
-                            threat_level = 'critical'
-                        elif gpc0 > 0 or err_rate > 10:
-                            threat_level = 'high'
-                        elif req_rate > 40 or conn_rate > 15:
-                            threat_level = 'medium'
-
-                        threats.append({
-                            'ip': ip,
-                            'blocked': gpc0 > 0,
-                            'repeat_offender': gpc1 > 2,
-                            'offense_count': gpc1,
-                            'request_rate': req_rate,
-                            'error_rate': err_rate,
-                            'connection_rate': conn_rate,
-                            'threat_level': threat_level
-                        })
-                except (ValueError, IndexError):
-                    continue
-
-        # Sort by threat level
-        threats.sort(key=lambda x: (x['offense_count'], x['error_rate'], x['request_rate']), reverse=True)
-
-        return jsonify({
-            'status': 'success',
-            'total_tracked_ips': len(lines) - 1,
-            'active_threats': len(threats),
-            'threats': threats[:50]  # Limit to top 50
-        })
     except Exception as e:
         log_operation('get_security_stats', False, str(e))
         return jsonify({'status': 'error', 'message': str(e)}), 500
 
+    counters = STICK_TABLE_FIELD_CONTRACT['web']
+    windows = {}
+    sources = []
+    active = 0
+    for line, fields in entries:
+        row = {'ip': fields['key']['value']}
+        for name in counters:
+            # A non-numeric counter means HAProxy's output format changed under
+            # us. Say so; do not coerce it to 0 and report it as a measurement.
+            try:
+                row[name] = int(fields[name]['value'])
+            except ValueError:
+                msg = ('stick table web reported a non-numeric %s=%r; the '
+                       '`show table` output format has changed. Row: %r'
+                       % (name, fields[name]['value'], line[:200]))
+                log_operation('get_security_stats', False, msg)
+                return jsonify({'status': 'error', 'message': msg}), 502
+            if fields[name]['window_ms'] is not None:
+                windows.setdefault(name, fields[name]['window_ms'])
+        if any(row[name] > 0 for name in counters):
+            active += 1
+        if row['http_req_rate'] >= min_req_rate:
+            sources.append(row)
+
+    sources.sort(key=lambda r: (r['http_req_rate'], r['http_err_rate'],
+                                r['conn_rate'], r['conn_cur']), reverse=True)
+
+    return jsonify({
+        'status': 'success',
+        'table': header['name'],
+        'table_size': header['size'],
+        'total_tracked_ips': header['used'],
+        'sources_with_activity': active,
+        'counters': list(counters),
+        'counter_windows_ms': windows,
+        'returned': len(sources[:limit]),
+        'min_req_rate': min_req_rate,
+        'sources': sources[:limit],
+        'note': ('Current per-source rates only. The stick table stores no '
+                 'history and no counter of past blocks; enforcement events '
+                 'are in the edge access log on the host at /var/log/haproxy.log.'),
+    })
+
 @app.route('/api/security/temporary-block', methods=['POST'])
 @require_api_key
 def temporary_block():
@@ -1977,13 +2364,16 @@ def temporary_block():
         if not update_blocked_ips_map():
             return jsonify({'status': 'error', 'message': 'Failed to update map file'}), 500
 
-        add_ip_to_runtime_map(ip_address)
+        runtime_ok = add_ip_to_runtime_map(ip_address)
 
-        log_operation('temporary_block', True, f'Temporarily blocked {ip_address} for {duration_minutes} minutes')
+        log_operation('temporary_block', True,
+                      f'Temporarily blocked {ip_address} for {duration_minutes} minutes '
+                      f'(runtime map fast path: {"ok" if runtime_ok else "FAILED, enforced on reload"})')
         return jsonify({
             'status': 'success',
             'message': f'IP {ip_address} temporarily blocked for {duration_minutes} minutes',
-            'expires_at': expiry_time.isoformat()
+            'expires_at': expiry_time.isoformat(),
+            'runtime_map_updated': runtime_ok
         })
     except Exception as e:
         log_operation('temporary_block', False, str(e))
@@ -1996,6 +2386,10 @@ def clear_expired_blocks():
     try:
         current_time = datetime.now()
         expired_ips = []
+        # IPs the runtime fast path could not drop. They still come out of the
+        # map file below, so they unblock on the next reload -- but silently
+        # reporting them as cleared is what this whole change is about.
+        runtime_failures = []
 
         with sqlite3.connect(DB_FILE) as conn:
             cursor = conn.cursor()
@@ -2016,17 +2410,21 @@ def clear_expired_blocks():
                 # Remove expired IPs
                 for ip in expired_ips:
                     cursor.execute('DELETE FROM blocked_ips WHERE ip_address = ?', (ip,))
-                    remove_ip_from_runtime_map(ip)
+                    if not remove_ip_from_runtime_map(ip):
+                        runtime_failures.append(ip)
 
         # Update map file if any IPs were removed
         if expired_ips:
             update_blocked_ips_map()
 
-        log_operation('clear_expired_blocks', True, f'Cleared {len(expired_ips)} expired IP blocks')
+        log_operation('clear_expired_blocks', not runtime_failures,
+                      f'Cleared {len(expired_ips)} expired IP blocks '
+                      f'({len(runtime_failures)} not removed from the runtime map)')
         return jsonify({
-            'status': 'success',
+            'status': 'success' if not runtime_failures else 'partial',
             'message': f'Cleared {len(expired_ips)} expired IP blocks',
-            'cleared_ips': expired_ips
+            'cleared_ips': expired_ips,
+            'runtime_map_failures': runtime_failures
         })
     except Exception as e:
         log_operation('clear_expired_blocks', False, str(e))
@@ -3065,50 +3463,167 @@ def update_blocked_ips_map(promote_backup=True):
         logger.error(f"Failed to update blocked IPs map: {e}")
         return False
 
-def add_ip_to_runtime_map(ip_address):
+# ---------------------------------------------------------------------------
-    """Add IP to HAProxy runtime map without reload"""
+# Runtime map fast path (blocked IPs)
+#
+# WHAT WAS WRONG, AND WHY NOBODY NOTICED FOR ITS ENTIRE EXISTENCE
+# ---------------------------------------------------------------
+# add_ip_to_runtime_map()/remove_ip_from_runtime_map() sent
+# `add map #0 <ip> 1` / `del map #0 <ip>` to /tmp/haproxy-cli and returned True
+# whenever socat exited 0. Two independent defects, three silences:
+#
+#   1. NO `@1` PREFIX. /tmp/haproxy-cli is HAProxy's MASTER CLI socket. Map
+#      commands are worker commands. The master answers
+#      `Unknown command: 'add', but maybe one of the following ones is a better
+#      match: ...` -- and socat still exits 0, so `result.returncode == 0` was
+#      true and the function logged "Added IP x to runtime map".
+#   2. `#0` IS NOT A VALID MAP ID. Ids are assigned at config-parse time and
+#      move whenever the config is regenerated; on whp01 blocked_ips.map is
+#      id 37 and trusted_ips.map is 10. There is no id 0. Hardcoding ANY number
+#      is wrong -- reference the map by its FILE PATH, which is stable because
+#      it is what haproxy.cfg names in `map_ip(/etc/haproxy/blocked_ips.map,0)`.
+#   3. `add map #0 ...` fails SILENTLY EVEN WITH `@1`. Captured on HAProxy
+#      3.0.11: `@1 add map #0 192.0.2.88 1` returns an EMPTY body, exit 0, and
+#      adds nothing to any map -- while `@1 del map #0 <ip>` and
+#      `@1 show map #0` both answer `Unknown map identifier.`. So a
+#      response-body check alone cannot catch defect 2 on the add path. That is
+#      why every mutation here is READ BACK with `get map` instead of trusting
+#      either the exit status or the (empty) reply.
+#
+# The blocking itself never depended on this: update_blocked_ips_map() rewrites
+# /etc/haproxy/blocked_ips.map and the callers reload HAProxy, which re-reads
+# the file. The FILE IS AUTHORITATIVE; this is only the no-reload fast path.
+# Every function below therefore returns a bool the caller can report, and
+# never raises into a request handler -- a runtime-map failure must degrade to
+# "enforced on reload", not to "not blocked" and not to a 500.
+#
+# scripts/test-runtime-map-contract.py holds the command strings and the
+# classification of every captured response to these rules.
+# ---------------------------------------------------------------------------
+
+# The value every blocked_ips.map entry must carry. haproxy.cfg matches with
+# `map_ip(/etc/haproxy/blocked_ips.map,0) -m int gt 0`, so a keyed entry with
+# no value evaluates to 0 and is NOT blocked. Runtime map and file must agree.
+BLOCKED_IPS_MAP_VALUE = '1'
+
+_GET_MAP_FOUND_RE = re.compile(r'\bfound=(yes|no)\b')
+_GET_MAP_VALUE_RE = re.compile(r'\bvalue="([^"]*)"')
+
+
+def runtime_map_lookup(map_path, key):
+    """(found, value) for one key in a runtime map, or raise HaproxyCliError.
+
+    Reads back what `add map`/`del map` actually did. `get map` answers
+    `type=ip, case=sensitive, found=yes, idx=tree, key="1.2.3.4", value="1",
+    type="str"` or `type=ip, case=sensitive, found=no`; an unusable map
+    reference answers `Unknown map identifier.`, which haproxy_cli() rejects.
+    """
+    out = haproxy_cli('get map %s %s' % (map_path, key), worker=True)
+    match = _GET_MAP_FOUND_RE.search(out)
+    if not match:
+        raise HaproxyCliError(
+            'unparseable `get map %s %s` response (no found=yes/no): %r'
+            % (map_path, key, out.strip()[:200]))
+    if match.group(1) == 'no':
+        return (False, None)
+    value = _GET_MAP_VALUE_RE.search(out)
+    return (True, value.group(1) if value else None)
+
+
+def runtime_map_keys(map_path):
+    """The set of keys currently in a runtime map, or raise HaproxyCliError.
+
+    `show map <file>` emits `<0x-pointer> <key> <value>` per line. An empty map
+    legitimately emits nothing, which is why this does not go through the
+    non-empty check.
+    """
     try:
-        if os.path.exists(HAPROXY_SOCKET_PATH):
+        out = haproxy_cli('show map %s' % map_path, worker=True)
-            socket_path = HAPROXY_SOCKET_PATH
+    except HaproxyCliError as e:
-        else:
+        # An empty map is a real, distinguishable state -- not a rejection.
-            socket_path = '/tmp/haproxy-cli'
+        if e.responses and all(body == '' for body in e.responses):
+            return set()
+        raise
+    keys = set()
+    for line in out.splitlines():
+        parts = line.split()
+        if len(parts) >= 2 and parts[0].startswith('0x'):
+            keys.add(parts[1])
+    return keys
 
-        # Add to runtime map (map file ID 0 for blocked IPs)
-        # Format: add map #<id> <key> <value>
-        # For IP blocking, value is always "1"
-        cmd = f'echo "add map #0 {ip_address} 1" | socat stdio {socket_path}'
-        result = subprocess.run(cmd, shell=True, capture_output=True, text=True)
 
-        if result.returncode == 0:
+def add_ip_to_runtime_map(ip_address, verify=True):
-            logger.info(f"Added IP {ip_address} to runtime map")
+    """Add IP to the running HAProxy's blocked map without a reload.
+
+    Returns True only when the entry is verifiably present with the value the
+    config matches on. False means the FILE + reload path is what will enforce
+    this block -- which it does regardless; see the section comment above.
+    """
+    try:
+        haproxy_cli(
+            'add map %s %s %s' % (BLOCKED_IPS_MAP_PATH, ip_address, BLOCKED_IPS_MAP_VALUE),
+            worker=True, expect_empty=True)
+        if verify:
+            found, value = runtime_map_lookup(BLOCKED_IPS_MAP_PATH, ip_address)
+            if not found:
+                raise HaproxyCliError(
+                    '`add map` was accepted but %s is not in %s afterwards -- '
+                    'the command did nothing (this is exactly how `#0` failed)'
+                    % (ip_address, BLOCKED_IPS_MAP_PATH))
+            if value != BLOCKED_IPS_MAP_VALUE:
+                raise HaproxyCliError(
+                    '%s is in %s with value %r, not %r -- haproxy.cfg matches '
+                    'with `-m int gt 0`, so this entry does NOT block'
+                    % (ip_address, BLOCKED_IPS_MAP_PATH, value, BLOCKED_IPS_MAP_VALUE))
+        logger.info(f"Added IP {ip_address} to runtime map (verified={verify})")
         return True
-        else:
+    except HaproxyCliError as e:
-            logger.warning(f"Failed to add IP to runtime map: {result.stderr}")
+        logger.warning(
+            "Runtime map fast path FAILED for %s: %s. The block is NOT lost -- "
+            "%s was rewritten and HAProxy re-reads it on reload -- but it does "
+            "not take effect until that reload completes.",
+            ip_address, e, BLOCKED_IPS_MAP_PATH)
         return False
     except Exception as e:
-        logger.error(f"Error adding IP to runtime map: {e}")
+        logger.error(f"Error adding IP {ip_address} to runtime map: {e}")
         return False
 
-def remove_ip_from_runtime_map(ip_address):
+
-    """Remove IP from HAProxy runtime map without reload"""
+def remove_ip_from_runtime_map(ip_address, verify=True):
+    """Remove IP from the running HAProxy's blocked map without a reload.
+
+    Returns True only when the key is verifiably gone. `Key not found.` means
+    the runtime map never had it, which is the requested end state, so that is
+    a success -- but it is logged, because it also means the runtime map and
+    the file had drifted apart.
+    """
     try:
-        if os.path.exists(HAPROXY_SOCKET_PATH):
+        try:
-            socket_path = HAPROXY_SOCKET_PATH
+            haproxy_cli('del map %s %s' % (BLOCKED_IPS_MAP_PATH, ip_address),
-        else:
+                        worker=True, expect_empty=True)
-            socket_path = '/tmp/haproxy-cli'
+        except HaproxyCliError as e:
-        
+            if not any(body.startswith('Key not found') for body in e.responses):
-        # Remove from runtime map (map file ID 0 for blocked IPs)
+                raise
-        cmd = f'echo "del map #0 {ip_address}" | socat stdio {socket_path}'
+            logger.info(
-        result = subprocess.run(cmd, shell=True, capture_output=True, text=True)
+                "Runtime map had no entry for %s to remove (`Key not found.`); "
-        
+                "the file and the runtime map had drifted", ip_address)
-        if result.returncode == 0:
+        if verify:
-            logger.info(f"Removed IP {ip_address} from runtime map")
+            found, _ = runtime_map_lookup(BLOCKED_IPS_MAP_PATH, ip_address)
+            if found:
+                raise HaproxyCliError(
+                    '`del map` was accepted but %s is STILL in %s'
+                    % (ip_address, BLOCKED_IPS_MAP_PATH))
+        logger.info(f"Removed IP {ip_address} from runtime map (verified={verify})")
         return True
-        else:
+    except HaproxyCliError as e:
-            logger.warning(f"Failed to remove IP from runtime map: {result.stderr}")
+        logger.warning(
+            "Runtime map fast path FAILED for %s: %s. The unblock is NOT lost -- "
+            "%s was rewritten and HAProxy re-reads it on reload -- but the IP "
+            "stays blocked until that reload completes.",
+            ip_address, e, BLOCKED_IPS_MAP_PATH)
         return False
     except Exception as e:
-        logger.error(f"Error removing IP from runtime map: {e}")
+        logger.error(f"Error removing IP {ip_address} from runtime map: {e}")
         return False
 
 def start_haproxy():

haproxy_tarpit_config.txt

@@ -1,3 +1,37 @@
+# =============================================================================
+# NOT IMPLEMENTED. THIS FILE IS A DESIGN SKETCH THAT WAS NEVER SHIPPED.
+# =============================================================================
+#
+# Nothing here is deployed, has ever been deployed, or is rendered into
+# haproxy.cfg. The real edge config is generated from templates/*.tpl. Compare:
+#
+#   THIS FILE proposes:  store gpc0,gpc1,gpc2,http_err_rate(30s),...
+#                        plus sc-inc-gpc0/1/2 scan-escalation rules
+#   templates/hap_listener.tpl ACTUALLY has:
+#                        store conn_cur,conn_rate(10s),http_req_rate(10s),http_err_rate(30s)
+#
+# There is no gpc0, no gpc1, no gpc2, and no scan-escalation state anywhere on
+# the edge, and never has been.
+#
+# WHY THE BANNER. This file was mistaken for the shipped config. Two consumers
+# -- /api/security/stats in haproxy_manager.py and scripts/show-tarpit-ips.sh --
+# were written to parse gpc0/gpc1 out of `show table web`, defaulting missing
+# fields to 0. The result was a "Scan Count" column and BLOCKED/TARPITTED
+# statuses that were pure fabrication, presented to an operator as fact, for
+# the entire life of both tools. Fixed 2026-08-22; scripts/test-stick-table-
+# contract.py now fails if any consumer's field expectations and the templates'
+# `store` clauses ever drift apart again.
+#
+# IF YOU WANT TO REVIVE ANY OF THIS: the counters must be added to a template
+# `store` clause and to STICK_TABLE_FIELD_CONTRACT in haproxy_manager.py first.
+# Adding them to a consumer alone produces confident zeros, not data. Note also
+# that sc0/sc1/sc2 are all in use and HAProxy's tune.stick-counters defaults to
+# 3, and that since 2026.08.8 the edge has real per-request access logging on
+# the host at /var/log/haproxy.log -- which records what was actually denied,
+# tarpitted and rate-limited, with request references. That log is a better
+# source for most of what this sketch was reaching for.
+# =============================================================================
+
 global
     daemon
     log stdout local0 info

scripts/monitor-attacks.sh

@@ -1,136 +1,148 @@
-#!/bin/bash
+#!/usr/bin/env bash
+#
+# monitor-attacks.sh — HAProxy edge activity monitor.
+#
+# Two sections, both fed from real data:
+#   1. Current per-IP rates, from the `web` stick table (delegated to
+#      show-edge-ip-rates.sh — there is exactly one stick-table parser).
+#   2. Recent enforcement events, from the HAProxy access log.
+#
+# HISTORY / WHY THIS IS SHORTER THAN IT USED TO BE
+#   The previous version printed a "Threat Intelligence Dashboard" with
+#   fourteen categories (auth_fail, authz_fail, scanner, sql_inj, traversal,
+#   wp_brute, admin_scan, shell_att, repeat_off, manual_bl, auto_bl,
+#   glitch_rate, ...) and a composite "threat score", all parsed out of
+#   gpc(0), gpc(1), gpc(3), gpc(12), gpc(13) and glitch_rate(300s). NONE of
+#   those fields exist: the `web` table stores only conn_cur, conn_rate,
+#   http_req_rate and http_err_rate. Every category was permanently 0 and the
+#   whole dashboard printed nothing while implying it was watching. All of it
+#   has been deleted rather than "fixed" — there was no data source to fix it
+#   against.
+#
+# Usage: monitor-attacks.sh [live]
+# Env:   LOG_FILE=<path>  access log to read (default /var/log/haproxy.log)
+#        LOG_LINES=<n>    how many trailing log lines to scan (default 500)
 
-# Real-time attack monitoring for HAProxy
+set -uo pipefail
-# Shows blocked requests and suspicious activity
 
-LOG_FILE="/var/log/haproxy.log"
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
-SOCKET="/tmp/haproxy-cli"
+LOG_FILE="${LOG_FILE:-/var/log/haproxy.log}"
+LOG_LINES="${LOG_LINES:-500}"
 
-echo "==================================================="
+# --- Section 1: current rates (real stick-table data) -----------------------
-echo "HAProxy Security Monitor - Real-time Attack Detection"
+show_rates() {
-echo "==================================================="
+    "$SCRIPT_DIR/show-edge-ip-rates.sh" "$@"
-echo ""
-
-# Function to show current threats with HAProxy 3.0.11 metrics
-show_threats() {
-    echo "HAProxy 3.0.11 Threat Intelligence Dashboard:"
-    echo "show table web" | socat stdio "$SOCKET" 2>/dev/null | \
-        awk 'NR>1 {
-            # Parse the stick table output for array-based GPC values
-            ip = $1
-            # Look for GPC array values in the data
-            auth_fail = 0
-            authz_fail = 0
-            rate_viol = 0
-            scanner = 0
-            sql_inj = 0
-            traversal = 0
-            wp_brute = 0
-            admin_scan = 0
-            shell_att = 0
-            repeat_off = 0
-            manual_bl = 0
-            auto_bl = 0
-            glitch_rate = 0
-            threat_score = 0
-
-            # Extract relevant metrics (simplified parsing)
-            if ($0 ~ /gpc\(0\)=([0-9]+)/) {
-                match($0, /gpc\(0\)=([0-9]+)/, arr); auth_fail = arr[1]
-            }
-            if ($0 ~ /gpc\(1\)=([0-9]+)/) {
-                match($0, /gpc\(1\)=([0-9]+)/, arr); authz_fail = arr[1]
-            }
-            if ($0 ~ /gpc\(3\)=([0-9]+)/) {
-                match($0, /gpc\(3\)=([0-9]+)/, arr); scanner = arr[1]
-            }
-            if ($0 ~ /gpc\(12\)=([0-9]+)/) {
-                match($0, /gpc\(12\)=([0-9]+)/, arr); repeat_off = arr[1]
-            }
-            if ($0 ~ /gpc\(13\)=([0-9]+)/) {
-                match($0, /gpc\(13\)=([0-9]+)/, arr); manual_bl = arr[1]
-            }
-            if ($0 ~ /glitch_rate\(300s\)=([0-9]+)/) {
-                match($0, /glitch_rate\(300s\)=([0-9]+)/, arr); glitch_rate = arr[1]
 }
 
-            # Calculate composite threat score (simplified)
+# --- Section 2: recent enforcement events (real access-log data) ------------
-            threat_score = auth_fail*10 + authz_fail*8 + scanner*12 + repeat_off*25 + manual_bl*100
-
-            # Only show IPs with significant threat indicators
-            if (auth_fail > 0 || authz_fail > 0 || scanner > 0 || repeat_off > 0 || manual_bl > 0 || glitch_rate > 0) {
-                threat_level = "LOW"
-                if (threat_score >= 100) threat_level = "CRITICAL"
-                else if (threat_score >= 50) threat_level = "HIGH"
-                else if (threat_score >= 20) threat_level = "MEDIUM"
-
-                printf "%-15s [%8s] Score:%-3d Auth:%-2d Authz:%-2d Scanner:%-1d Repeat:%-1d Glitch:%-2d\n",
-                       ip, threat_level, threat_score, auth_fail, authz_fail, scanner, repeat_off, glitch_rate
-            }
-        }' | head -15
-
-    echo ""
-    echo "Top HTTP/2 Protocol Violators:"
-    echo "show table web" | socat stdio "$SOCKET" 2>/dev/null | \
-        awk 'NR>1 && $0 ~ /glitch/ {
-            if ($0 ~ /glitch_rate\(300s\)=([0-9]+)/) {
-                match($0, /glitch_rate\(300s\)=([0-9]+)/, arr)
-                if (arr[1] > 2) {
-                    printf "%-15s glitch_rate:%-3s\n", $1, arr[1]
-                }
-            }
-        }' | head -5
-    echo "---------------------------------------------------"
-}
-
-# Function to show recent blocks
 show_recent_blocks() {
-    echo "Recent Blocked Requests:"
+    echo "Recent enforcement events (last $LOG_LINES log lines):"
-    tail -100 "$LOG_FILE" 2>/dev/null | \
+    echo
-        grep -E "(bot_scanner|scan_admin|scan_shells|sql_injection|directory_traversal|rate_abuse|tarpit|denied|403)" | \
+
-        tail -10 | \
+    if [ ! -f "$LOG_FILE" ] || [ ! -r "$LOG_FILE" ]; then
-        awk '{
+        cat <<MSG
-            if (match($0, /[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+:[0-9]+/)) {
+  Access log not readable at: $LOG_FILE
-                ip = substr($0, RSTART, RLENGTH)
+
-                gsub(/:.*/, "", ip)
+  This is expected INSIDE the haproxy-manager container: HAProxy logs to
-                reason = ""
+  syslog on the DOCKER HOST, and the file lives on the host, not in here.
-                if ($0 ~ /bot_scanner/) reason = "BOT_SCANNER"
+  Read it from the host instead:
-                else if ($0 ~ /scan_admin/) reason = "ADMIN_SCAN"
+
-                else if ($0 ~ /scan_shells/) reason = "SHELL_SCAN"
+    grep -aE ' (PT|PR)--' /var/log/haproxy.log | tail -20   # tarpit / deny
-                else if ($0 ~ /sql_injection/) reason = "SQL_INJECTION"
+    grep -aE ' (429|403) ' /var/log/haproxy.log | tail -20  # rate-limited / blocked
-                else if ($0 ~ /directory_traversal/) reason = "DIR_TRAVERSAL"
+    grep -a 'cip=<IP>'  /var/log/haproxy.log | tail -50     # one client IP
-                else if ($0 ~ /rate_abuse/) reason = "RATE_ABUSE"
+    grep -a 'id=<uuid>' /var/log/haproxy.log                # one request reference
-                else if ($0 ~ /tarpit/) reason = "TARPIT"
+                                                            # (the UUID on the block page)
-                else if ($0 ~ /denied/) reason = "DENIED"
+    tail -f /var/log/haproxy.log | grep -aE ' (PT|PR)--'    # live
-                else if ($0 ~ /403/) reason = "BLOCKED"
+
-                printf "[%s] %-15s %s\n", strftime("%H:%M:%S"), ip, reason
+  Or point this script at a copy:  LOG_FILE=/path/to/haproxy.log $0
-            }
+MSG
-        }'
+        echo
-    echo ""
+        return 0
+    fi
+
+    printf "%-15s %-16s %-4s %-5s %-28s %s\n" "TIME" "CLIENT IP" "CODE" "TERM" "HOST" "REQUEST / REQUEST-ID"
+    printf "%s\n" "-----------------------------------------------------------------------------------------------------"
+
+    local found
+    found=$(tail -n "$LOG_LINES" "$LOG_FILE" 2>/dev/null | awk '
+    {
+        status = ""; term = ""; cip = ""; host = ""; id = ""; ts = ""; req = ""
+
+        # %tr is bracketed: [22/Aug/2026:10:11:12.345] -> keep HH:MM:SS
+        # No {n} interval expressions here: not every awk in a slim Debian
+        # image supports them. Spelled out instead.
+        if (match($0, /\[[0-9][0-9]\/[A-Za-z][A-Za-z][A-Za-z]\/[0-9][0-9][0-9][0-9]:[0-9][0-9]:[0-9][0-9]:[0-9][0-9]/)) {
+            ts = substr($0, RSTART + 13, 8)
         }
 
-# Monitor mode selection
+        # Anchor on the %TR/%Tw/%Tc/%Tr/%Ta timers block: %ST follows it, and
-if [ "$1" == "live" ]; then
+        # the termination state (%tsc) is 4 fields further on (%B %CC %CS %tsc).
+        for (i = 1; i <= NF; i++) {
+            if ($i ~ /^[+-]?[0-9]+\/[+-]?[0-9]+\/[+-]?[0-9]+\/[+-]?[0-9]+\/[+-]?[0-9]+$/) {
+                status = $(i + 1)
+                term = $(i + 5)
+                break
+            }
+        }
+
+        # Only enforcement outcomes: tarpit (PT--), deny (PR--), 403, 429.
+        if (!(term ~ /^PT/ || term ~ /^PR/ || status == "403" || status == "429")) next
+
+        for (i = 1; i <= NF; i++) {
+            if (substr($i, 1, 4) == "cip=")  cip  = substr($i, 5)
+            if (substr($i, 1, 5) == "host=") host = substr($i, 6)
+            if (substr($i, 1, 3) == "id=")   id   = substr($i, 4)
+        }
+
+        if (match($0, /"[A-Z]+ [^"]*"/)) {
+            req = substr($0, RSTART + 1, RLENGTH - 2)
+            if (length(req) > 42) req = substr(req, 1, 41) "..."
+        }
+
+        if (cip == "") cip = "-"
+        if (host == "") host = "-"
+        if (term == "") term = "-"
+        if (status == "") status = "-"
+        printf "%-15s %-16s %-4s %-5s %-28s %s\n", ts, cip, status, term, host, req
+        if (id != "" && id != "-") printf "%-15s %s\n", "", "  id=" id
+        n++
+    }
+    END { if (n == 0) print "(no tarpit/deny/403/429 events in the scanned window)" }
+    ')
+    printf '%s\n' "$found"
+    echo
+    echo "TERM = HAProxy termination state: PT-- tarpit, PR-- deny (incl. WAF/rate limit)."
+    echo "id=  = request reference; it is printed on the block page and is how a"
+    echo "       customer support ticket correlates to an exact request here."
+}
+
+banner() {
+    echo "==================================================="
+    echo "HAProxy Edge Monitor - $(date '+%Y-%m-%d %H:%M:%S')"
+    echo "==================================================="
+    echo
+}
+
+if [ "${1:-}" = "live" ]; then
     echo "Live monitoring mode - Press Ctrl+C to exit"
-    echo ""
-
     while true; do
         clear
-        echo "==================================================="
+        banner
-        echo "HAProxy Security Monitor - $(date '+%Y-%m-%d %H:%M:%S')"
+        show_rates || true
-        echo "==================================================="
+        echo
-        echo ""
-        show_threats
-        echo ""
         show_recent_blocks
         sleep 5
     done
 else
-    # Single run mode
+    banner
-    show_threats
+    rc=0
-    echo ""
+    show_rates || rc=$?
+    echo
     show_recent_blocks
-    echo ""
+    echo
-    echo "Tip: Run with 'live' parameter for continuous monitoring"
+    echo "Tip: run with 'live' for a refreshing view."
     echo "Usage: $0 [live]"
+    # Propagate a stick-table read failure: if the rates section could not be
+    # produced, this run did NOT report what it claims to report.
+    exit "$rc"
 fi

scripts/show-edge-ip-rates.sh

@@ -0,0 +1,314 @@
+#!/usr/bin/env bash
+#
+# show-edge-ip-rates.sh — real, current per-IP rate counters from the HAProxy
+# `web` stick table.
+#
+# WHAT THIS CAN TELL YOU
+#   The `web` stick table (templates/hap_listener.tpl) stores exactly four
+#   counters per client IP:
+#       conn_cur, conn_rate(10s), http_req_rate(10s), http_err_rate(30s)
+#   Those are INSTANTANEOUS values — the current concurrency and the current
+#   sliding-window rates. This script prints them, and nothing else.
+#
+# WHAT THIS CANNOT TELL YOU
+#   * Who has been tarpitted, denied, or rate-limited. The stick table stores
+#     NO history and NO counter of past enforcement actions. It has no gpc0 /
+#     gpc1 / gpc(N) / gpc_rate / glitch_rate columns at all — any tool that
+#     claims to read them from this table is fabricating numbers.
+#   * Anything about an IP that has gone quiet: entries expire after 10m.
+#
+#   Real enforcement events live in the HAProxy ACCESS LOG, which is on the
+#   DOCKER HOST at /var/log/haproxy.log (it does NOT exist inside this
+#   container). The log-format carries the HAProxy termination state plus
+#   cip= (real client IP), host=, ua= and id= (the request UUID shown on the
+#   block page, which correlates with customer support tickets).
+#
+#   Ready to run ON THE HOST:
+#       # last 20 tarpitted (PT--) or denied (PR--) requests
+#       grep -aE ' (PT|PR)--' /var/log/haproxy.log | tail -20
+#       # everything HAProxy answered 429/403 to, newest last
+#       grep -aE ' (429|403) ' /var/log/haproxy.log | tail -20
+#       # everything for one client IP
+#       grep -a 'cip=203.0.113.7' /var/log/haproxy.log | tail -50
+#       # look up one request reference from a support ticket
+#       grep -a 'id=<uuid-from-the-block-page>' /var/log/haproxy.log
+#
+# USAGE
+#   show-edge-ip-rates.sh [-a|--all]
+#     -a, --all   also show rows whose counters are all zero (off by default:
+#                 a table with hundreds of idle entries is pure noise)
+#
+# ENVIRONMENT
+#   SHOW_ALL=1               same as --all
+#   HAPROXY_SOCKET=<path>    override the CLI socket (default /tmp/haproxy-cli)
+#   HAPROXY_TABLE_DUMP=<file>
+#                            parse a previously captured `show table web` dump
+#                            from a file instead of talking to the socket.
+#                            Supported seam for offline analysis of a captured
+#                            support bundle, and for testing this parser.
+#
+# NOTE ON THE SOCKET
+#   /tmp/haproxy-cli is HAProxy's MASTER CLI socket, so worker commands need an
+#   `@1` prefix. Without it HAProxy answers "Unknown command: 'show' ..." AND
+#   socat still exits 0 — so exit status is worthless here and this script
+#   inspects the RESPONSE BODY instead.
+
+set -euo pipefail
+
+SOCKET="${HAPROXY_SOCKET:-/tmp/haproxy-cli}"
+TABLE="web"
+
+# Fields this script expects the `web` stick table to store. Keep on ONE line
+# in this exact NAME=(a b c d) shape — the contract test greps for it, and the
+# parser below is driven entirely by it.
+EXPECTED_FIELDS=(conn_cur conn_rate http_req_rate http_err_rate)
+
+# Which of EXPECTED_FIELDS to sort on (descending). Falls back to the first
+# field if this name is not in the list.
+SORT_FIELD="http_req_rate"
+
+SHOW_ALL="${SHOW_ALL:-0}"
+while [ $# -gt 0 ]; do
+    case "$1" in
+        -a|--all) SHOW_ALL=1 ;;
+        -h|--help) sed -n '2,60p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
+        *) echo "Unknown argument: $1" >&2; echo "Usage: $0 [-a|--all]" >&2; exit 2 ;;
+    esac
+    shift
+done
+
+die() { echo "ERROR: $*" >&2; exit 1; }
+
+# First non-blank line of a blob.
+#
+# Deliberately NOT `printf ... | sed -n '/./{p;q;}'`. sed quits after the first
+# match and closes the pipe; on a real 550-entry table dump printf is still
+# writing and takes SIGPIPE, so under `set -o pipefail` the whole command
+# substitution returns 141 and `set -e` kills the script -- silently, with no
+# output at all. That is the same class of failure this script exists to stop
+# hiding, so it does not get to happen here. A plain read loop has no pipeline
+# and no early close.
+first_nonblank() {
+    local line
+    while IFS= read -r line || [ -n "$line" ]; do
+        case "$line" in
+            *[![:space:]]*) printf '%s\n' "$line"; return 0 ;;
+        esac
+    done <<EOF
+$1
+EOF
+    return 0
+}
+
+# Return 0 if the CLI response body is a rejection rather than table data.
+# Checked on the body because socat's exit status is 0 either way.
+body_is_rejected() {
+    local first
+    first=$(first_nonblank "$1")
+    case "$first" in
+        "Unknown command"*|"No such table"*|"Permission denied"*) return 0 ;;
+        *) return 1 ;;
+    esac
+}
+
+send_cmd() {
+    printf '%s\n' "$1" | socat stdio "$SOCKET" 2>/dev/null
+}
+
+# ---------------------------------------------------------------- fetch data
+BODY=""
+SOURCE=""
+if [ -n "${HAPROXY_TABLE_DUMP:-}" ]; then
+    [ -r "$HAPROXY_TABLE_DUMP" ] || die "HAPROXY_TABLE_DUMP is set but '$HAPROXY_TABLE_DUMP' is not readable."
+    BODY=$(cat "$HAPROXY_TABLE_DUMP")
+    SOURCE="file $HAPROXY_TABLE_DUMP"
+else
+    [ -S "$SOCKET" ] || die "HAProxy CLI socket not found at $SOCKET (is HAProxy running, and are you inside the haproxy-manager container?)"
+    command -v socat >/dev/null 2>&1 || die "socat is not installed; cannot talk to $SOCKET"
+
+    # Master socket form first, then the plain stats-socket form.
+    BODY=$(send_cmd "@1 show table $TABLE" || true)
+    SOURCE="socket $SOCKET (@1 show table $TABLE)"
+    if [ -z "${BODY//[[:space:]]/}" ] || body_is_rejected "$BODY"; then
+        FALLBACK=$(send_cmd "show table $TABLE" || true)
+        if [ -n "${FALLBACK//[[:space:]]/}" ] && ! body_is_rejected "$FALLBACK"; then
+            BODY="$FALLBACK"
+            SOURCE="socket $SOCKET (show table $TABLE)"
+        else
+            echo "ERROR: HAProxy rejected BOTH '@1 show table $TABLE' and 'show table $TABLE'." >&2
+            echo "  @1 response : $(first_nonblank "$BODY")" >&2
+            echo "  bare response: $(first_nonblank "$FALLBACK")" >&2
+            echo "  Check the socket is HAProxy's CLI and that the table '$TABLE' exists" >&2
+            echo "  (a config reload without the frontend would drop it)." >&2
+            exit 1
+        fi
+    fi
+fi
+
+# ------------------------------------------------------------- header checks
+HEADER=$(first_nonblank "$BODY")
+case "$HEADER" in
+    "# table: $TABLE,"*) : ;;
+    *)
+        echo "ERROR: unexpected first line from '$SOURCE'." >&2
+        echo "  expected it to start with: # table: $TABLE," >&2
+        echo "  got                      : $HEADER" >&2
+        exit 1
+        ;;
+esac
+
+TBL_SIZE=$(printf '%s\n' "$HEADER" | sed -n 's/.*size:\([0-9]*\).*/\1/p')
+TBL_USED=$(printf '%s\n' "$HEADER" | sed -n 's/.*used:\([0-9]*\).*/\1/p')
+[ -n "$TBL_SIZE" ] || TBL_SIZE="?"
+[ -n "$TBL_USED" ] || TBL_USED="?"
+
+# ------------------------------------------------------------------- parsing
+# awk emits:
+#   W \t <field>:<window-seconds-or-dash> ...       (one line, from first row)
+#   R \t <sortkey> \t <ip> \t <value per EXPECTED_FIELDS in order>
+# and exits 1 after reporting any row missing an expected field.
+PARSED=""
+if ! PARSED=$(printf '%s\n' "$BODY" | awk -v fieldlist="${EXPECTED_FIELDS[*]}" -v sortfield="$SORT_FIELD" '
+BEGIN {
+    nf = split(fieldlist, F, " ")
+    sortidx = 1
+    for (i = 1; i <= nf; i++) if (F[i] == sortfield) sortidx = i
+    wprinted = 0
+}
+/^#/ { next }
+!/key=/ { next }
+{
+    split("", val, " "); split("", win, " ")
+    for (i = 1; i <= NF; i++) {
+        tok = $i
+        p = index(tok, "=")
+        if (p == 0) continue
+        lhs = substr(tok, 1, p - 1)
+        rhs = substr(tok, p + 1)
+        b = index(lhs, "(")
+        if (b > 0) {
+            nm = substr(lhs, 1, b - 1)
+            win[nm] = substr(lhs, b + 1, length(lhs) - b - 1)
+        } else {
+            nm = lhs
+            win[nm] = ""
+        }
+        val[nm] = rhs
+    }
+
+    missing = ""
+    for (i = 1; i <= nf; i++) if (!(F[i] in val)) missing = missing (missing == "" ? "" : ", ") F[i]
+    if (missing != "") {
+        printf "ERROR: stick table row is missing expected field(s): %s\n", missing > "/dev/stderr"
+        printf "  offending row: %s\n", $0 > "/dev/stderr"
+        printf "  this script expects the web table to store: %s\n", fieldlist > "/dev/stderr"
+        print  "  Those expectations and the templates/hap_listener.tpl `store` clause have DRIFTED." > "/dev/stderr"
+        print  "  Fix one or the other; refusing to print 0 for a counter HAProxy never reported." > "/dev/stderr"
+        exit 1
+    }
+    if (!("key" in val)) {
+        printf "ERROR: stick table row has no key= field: %s\n", $0 > "/dev/stderr"
+        exit 1
+    }
+
+    if (!wprinted) {
+        line = "W"
+        for (i = 1; i <= nf; i++) {
+            w = win[F[i]]
+            if (w ~ /^[0-9]+$/) w = sprintf("%g", w / 1000); else w = "-"
+            line = line "\t" F[i] ":" w
+        }
+        print line
+        wprinted = 1
+    }
+
+    nonzero = 0
+    row = ""
+    for (i = 1; i <= nf; i++) {
+        v = val[F[i]]
+        if (v + 0 != 0) nonzero = 1
+        row = row "\t" v
+    }
+    printf "R\t%s\t%s\t%d%s\n", val[F[sortidx]] + 0, val["key"], nonzero, row
+}
+'); then
+    exit 1
+fi
+
+# ------------------------------------------------------------------ printing
+WINSPEC=$(printf '%s\n' "$PARSED" | sed -n 's/^W\t//p' || true)
+
+echo "==================================================================="
+echo "  HAProxy edge IP rates — table '$TABLE' (current values only)"
+echo "==================================================================="
+echo "Source     : $SOURCE"
+echo "Tracked    : ${TBL_USED} of ${TBL_SIZE} slots in use"
+if [ "$SHOW_ALL" = "1" ]; then
+    echo "Filter     : showing ALL tracked IPs"
+else
+    echo "Filter     : showing only IPs with a non-zero counter (use --all for every row)"
+fi
+echo
+
+# Column headers, with each counter's window rendered in SECONDS (HAProxy
+# reports the window in milliseconds, e.g. conn_rate(10000) = 10s).
+HDR=$(printf "%-18s" "IP Address")
+i=0
+for f in "${EXPECTED_FIELDS[@]}"; do
+    w=$(printf '%s\n' "$WINSPEC" | tr '\t' '\n' | sed -n "s/^${f}://p")
+    if [ -n "$w" ] && [ "$w" != "-" ]; then
+        label="${f}/${w}s"
+    else
+        label="$f"
+    fi
+    HDR="$HDR $(printf '%18s' "$label")"
+    i=$((i + 1))
+done
+echo "$HDR"
+printf '%s\n' "$HDR" | sed 's/./-/g'
+
+ROWS=$(printf '%s\n' "$PARSED" | sed -n 's/^R\t//p' || true)
+shown=0
+if [ -n "$ROWS" ]; then
+    while IFS=$'\t' read -r sortkey ip nonzero rest; do
+        [ -n "${ip:-}" ] || continue
+        if [ "$SHOW_ALL" != "1" ] && [ "$nonzero" = "0" ]; then
+            continue
+        fi
+        line=$(printf "%-18s" "$ip")
+        oldifs="$IFS"; IFS=$'\t'
+        # shellcheck disable=SC2086
+        set -- $rest
+        IFS="$oldifs"
+        for v in "$@"; do
+            line="$line $(printf '%18s' "$v")"
+        done
+        echo "$line"
+        shown=$((shown + 1))
+    done < <(printf '%s\n' "$ROWS" | sort -t"$(printf '\t')" -k1,1nr)
+fi
+
+if [ "$shown" -eq 0 ]; then
+    if [ "$SHOW_ALL" = "1" ]; then
+        echo "(no IPs currently tracked)"
+    else
+        echo "(no IP currently has a non-zero counter — re-run with --all to list idle entries)"
+    fi
+fi
+
+echo
+echo "==================================================================="
+echo "These are CURRENT values. The table keeps no history and no record of"
+echo "past tarpits/denials. For actual enforcement events, read the access"
+echo "log ON THE DOCKER HOST (it does not exist in this container):"
+echo "  grep -aE ' (PT|PR)--' /var/log/haproxy.log | tail -20     # tarpit / deny"
+echo "  grep -aE ' (429|403) ' /var/log/haproxy.log | tail -20    # rate-limit / block"
+echo "  grep -a 'cip=<IP>' /var/log/haproxy.log | tail -50        # one client"
+echo "  grep -a 'id=<uuid>' /var/log/haproxy.log                  # one request reference"
+echo
+echo "Operator actions (via the MASTER CLI socket — the @1 prefix is required):"
+echo "  printf '@1 show table $TABLE key <IP>\\n'  | socat stdio $SOCKET"
+echo "  printf '@1 set table $TABLE key <IP> data.http_req_rate 0\\n' | socat stdio $SOCKET"
+echo "  printf '@1 clear table $TABLE key <IP>\\n' | socat stdio $SOCKET   # drop one entry"
+echo "  printf '@1 clear table $TABLE\\n'          | socat stdio $SOCKET   # drop ALL entries"
+echo "==================================================================="

scripts/show-tarpit-ips.sh

@@ -1,123 +1,35 @@
-#!/bin/bash
+#!/usr/bin/env bash
-
-# Script to display IPs that have been tarpitted by HAProxy 3.0
-# Uses HAProxy stats socket to query stick-table data
 #
-# Usage in Docker container:
+# DEPRECATED SHIM — kept so existing docs/runbooks/muscle memory keep working.
-#   docker exec -it haproxy-manager /haproxy/scripts/show-tarpit-ips.sh
+#
+# This script used to print a "Tarpitted IPs Report" with a "Scan Count" and a
+# BLOCKED / SILENT-DROP / TARPIT status per IP, all derived from gpc0 and gpc1
+# stick-table columns. Those columns DO NOT EXIST: the `web` table
+# (templates/hap_listener.tpl) stores only conn_cur, conn_rate, http_req_rate
+# and http_err_rate. The old parser defaulted every missing field to 0, so the
+# whole report was fabricated — every IP showed "Scan Count 0 / Normal"
+# regardless of what it was actually doing.
+#
+# The stick table also keeps NO history, so nothing in it can identify who was
+# tarpitted. Real enforcement events live in the access log ON THE DOCKER HOST
+# at /var/log/haproxy.log (it does not exist inside this container):
+#   grep -aE ' (PT|PR)--' /var/log/haproxy.log | tail -20   # tarpit / deny
+#   grep -aE ' (429|403) ' /var/log/haproxy.log | tail -20  # rate-limit / block
+#
+# What IS knowable from the stick table — the current per-IP rates — is printed
+# by show-edge-ip-rates.sh, which this shim now runs.
 
-SOCKET="/tmp/haproxy-cli"
+set -euo pipefail
 
-# Check if socket exists
+cat >&2 <<'NOTE'
-if [ ! -S "$SOCKET" ]; then
+NOTE: show-tarpit-ips.sh is deprecated and cannot report tarpits.
-    echo "Error: HAProxy socket not found at $SOCKET"
+      The HAProxy stick table stores no history and no gpc0/gpc1 counters, so
-    echo "Make sure HAProxy is running with stats socket enabled"
+      the old "Scan Count"/"BLOCKED" columns were fabricated numbers.
-    exit 1
+      Actual tarpit/deny events are in /var/log/haproxy.log ON THE HOST:
-fi
+        grep -aE ' (PT|PR)--' /var/log/haproxy.log | tail -20
+      Running show-edge-ip-rates.sh instead (current rates, real values):
 
-echo "==================================================================="
+NOTE
-echo "                    HAProxy Tarpitted IPs Report                   "
-echo "==================================================================="
-echo
-echo "Showing IPs tracked in the stick-table with scan detection counters:"
-echo "(gpc0 = total scan attempts, gpc1 = escalation level)"
-echo
 
-# In HAProxy 3.0, we need to use the proper process prefix
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
-# The web frontend table is in the worker process, not master
+exec "$SCRIPT_DIR/show-edge-ip-rates.sh" "$@"
-# First check which process has the table
-# Note: grep for actual worker line, not the header
-PROCESS_ID=$(echo "show proc" | socat stdio "$SOCKET" 2>/dev/null | grep -E '^[0-9]+.*worker' | awk '{print $1}' | head -1)
-
-if [ -z "$PROCESS_ID" ]; then
-    echo "Error: Could not find HAProxy worker process"
-    echo "Try: echo 'show proc' | socat stdio $SOCKET"
-    exit 1
-fi
-
-# Show stick-table entries from the web frontend using the worker process
-# Use printf to avoid bash history expansion issues with !
-printf "@!%s show table web\n" "${PROCESS_ID}" | socat stdio "$SOCKET" 2>/dev/null | {
-    # Skip the header line
-    read header
-    
-    # Check if we got an error or empty response
-    if echo "$header" | grep -q "No such table"; then
-        echo "Error: Table 'web' not found. HAProxy may need to be reloaded."
-        exit 1
-    fi
-    
-    has_data=false
-    echo "IP Address           | Scan Count | Level | HTTP Err Rate | Status"
-    echo "---------------------|------------|-------|---------------|------------------"
-    
-    # Process each line
-    while IFS= read -r line; do
-        # Skip empty lines and comments
-        if [ -z "$line" ] || echo "$line" | grep -q "^#"; then
-            continue
-        fi
-        
-        # HAProxy 3.0 format: 0x... key=<ip> use=... exp=... gpc0=... gpc1=... http_err_rate(10s)=...
-        if echo "$line" | grep -q "key="; then
-            has_data=true
-            
-            # Extract IP and counters
-            ip=$(echo "$line" | grep -o 'key=[^ ]*' | cut -d'=' -f2)
-            gpc0=$(echo "$line" | grep -o 'gpc0=[0-9]*' | cut -d'=' -f2)
-            gpc1=$(echo "$line" | grep -o 'gpc1=[0-9]*' | cut -d'=' -f2)
-            err_rate=$(echo "$line" | grep -o 'http_err_rate([^)]*=[0-9]*' | grep -o '[0-9]*$')
-            
-            # Set defaults if values are empty
-            gpc0=${gpc0:-0}
-            gpc1=${gpc1:-0}
-            err_rate=${err_rate:-0}
-            
-            # Determine status based on scan count and escalation
-            status=""
-            if [ "$gpc0" -ge 100 ]; then
-                status="BLOCKED (429)"
-            elif [ "$gpc0" -ge 60 ]; then
-                status="SILENT-DROP"
-            elif [ "$gpc0" -ge 40 ]; then
-                if [ "$gpc1" -ge 2 ]; then
-                    status="SILENT-DROP (repeat)"
-                else
-                    status="TARPIT 10s"
-                fi
-            elif [ "$gpc0" -ge 25 ]; then
-                status="TARPIT 10s"
-            else
-                status="Normal"
-            fi
-            
-            # Format output
-            printf "%-20s | %10s | %5s | %13s | %s\n" "$ip" "$gpc0" "$gpc1" "$err_rate/10s" "$status"
-        fi
-    done
-    
-    if [ "$has_data" = false ]; then
-        echo "(No IPs currently tracked - table is empty)"
-    fi
-}
-
-echo
-echo "==================================================================="
-echo "Legend:"
-echo "  - Scan Count 25-39: Low scanner → TARPIT 10s delay"
-echo "  - Scan Count 40-59: Medium scanner → TARPIT 10s (1st), SILENT-DROP (repeat)"
-echo "  - Scan Count 60-99: High scanner → SILENT-DROP (immediate disconnect)"
-echo "  - Scan Count 100+:  Critical scanner → BLOCKED (429 response)"
-echo "  - Burst (5+ in 10s): → TARPIT 10s (1st), SILENT-DROP (repeat)"
-echo "==================================================================="
-echo "Note: Only counts suspicious scripts/configs, NOT missing images/fonts/CSS"
-echo "Note: IPs are tracked for 1 hour since last activity"
-echo
-echo "To clear a specific IP from the table:"
-echo "  printf '@!${PROCESS_ID} del table web key <IP>\\n' | socat stdio $SOCKET"
-echo
-echo "To clear all entries:"
-echo "  printf '@!${PROCESS_ID} clear table web\\n' | socat stdio $SOCKET"
-echo
-echo "Debug: Worker PID is ${PROCESS_ID}"
-echo

scripts/test-cert-write-safety.py

@@ -27,6 +27,8 @@ These tests pin the invariants:
   * nothing is published that is not a complete, validated cert+key pair;
   * no old certificate file is removed and no lineage deleted until the
     replacement is validated, in place, and actually loaded by HAProxy;
+  * removing ONE domain never unlinks a .pem, or deletes a certbot lineage,
+    that other still-configured domains are being served from;
   * only final .pem files ever exist in the crt directory.
 
 Running
@@ -894,6 +896,184 @@ class TestBundleValidation(CertPublishTestCase):
                          'a private key must never be group/world writable')
 
 
+class TestSharedCertificateSurvivesDomainRemoval(CertPublishTestCase):
+    """Bug 5: DELETE /api/domain unlinked a PEM other live sites were served from.
+
+    `domains.domain` is UNIQUE; `domains.ssl_cert_path` is not, and nothing
+    ever made it so. request_ssl_bundle() issues one SAN certificate and points
+    every included name's row at the same /etc/haproxy/certs/<primary>.pem, so
+    sharing is not an edge case - it is the normal shape of the table. Measured
+    on production the day this was written: 39 of 71 distinct cert paths on one
+    host were referenced by more than one domain row (118 of 150 SSL-enabled
+    rows), and one shared path was
+    /etc/haproxy/certs/threeworldsoneheart.org.pem, referenced by the live
+    apex, its www, and a mail.* alias.
+
+    remove_domain() did, unconditionally:
+
+        os.remove(ssl_cert_path)
+        certbot delete --cert-name <domain>
+
+    Removing the mail.* alias would therefore have deleted the PEM the apex was
+    serving on, and (had the alias been the bundle primary) the lineage behind
+    it - HTTPS down for every other name in the bundle, with no local copy and
+    only a fresh, rate-limited ACME order to recover from.
+
+    These tests assert file-system and certbot-invocation outcomes, not which
+    branch was taken.
+    """
+
+    BUNDLE = '/etc/haproxy/certs'  # documentation only; SSL_CERTS_DIR is stubbed
+
+    def _cert_for(self, primary):
+        """Publish a bundle for `primary` and return its path."""
+        return self.publish_live_bundle(primary)
+
+    def remove(self, domain):
+        return self.client.delete('/api/domain', json={'domain': domain})
+
+    # -- last reference: the cleanup must still happen --------------------
+
+    def test_last_reference_removal_unlinks_the_pem(self):
+        cert = self._cert_for('example.com')
+        self.add_domain('example.com', 'be_example', ssl_cert_path=cert)
+
+        resp = self.remove('example.com')
+
+        self.assertEqual(200, resp.status_code, resp.data)
+        self.assertFalse(
+            os.path.exists(cert),
+            'nothing else referenced this bundle - it must be cleaned up, or '
+            'the crt directory accumulates certs for domains that are gone')
+
+    def test_last_reference_removal_deletes_the_lineage(self):
+        cert = self._cert_for('example.com')
+        self.add_domain('example.com', 'be_example', ssl_cert_path=cert)
+
+        self.remove('example.com')
+
+        self.assertEqual(
+            ['delete --cert-name example.com --non-interactive'],
+            self.certbot_deletes(),
+            'the last name on a lineage went away - the lineage should go too')
+
+    def test_lineage_deleted_is_the_bundles_not_the_domains_own_name(self):
+        """Removing a SAN member must target the lineage that issued the file."""
+        cert = self._cert_for('example.com')
+        self.add_domain('www.example.com', 'be_www', ssl_cert_path=cert)
+
+        self.remove('www.example.com')
+
+        self.assertEqual(
+            ['delete --cert-name example.com --non-interactive'],
+            self.certbot_deletes(),
+            'the lineage is named after the bundle primary (--cert-name), not '
+            'after whichever SAN happened to be removed last')
+
+    # -- shared reference: nothing may be destroyed -----------------------
+
+    def test_shared_pem_survives_removal_of_one_name(self):
+        cert = self._cert_for('example.com')
+        before = self.read(cert)
+        self.add_domain('example.com', 'be_apex', ssl_cert_path=cert)
+        self.add_domain('www.example.com', 'be_www', ssl_cert_path=cert)
+
+        resp = self.remove('www.example.com')
+
+        self.assertEqual(200, resp.status_code, resp.data)
+        self.assertTrue(
+            os.path.exists(cert),
+            'example.com is still configured and still served from this file')
+        self.assertEqual(before, self.read(cert),
+                         'the surviving bundle must be byte-for-byte intact')
+        self.assertTrue(self.edge_would_start(),
+                        'the edge must still load the crt directory')
+
+    def test_shared_pem_survives_removal_of_the_bundle_primary(self):
+        """The worst shape: the name being removed IS the lineage/file name."""
+        cert = self._cert_for('example.com')
+        before = self.read(cert)
+        self.add_domain('example.com', 'be_apex', ssl_cert_path=cert)
+        self.add_domain('www.example.com', 'be_www', ssl_cert_path=cert)
+
+        self.remove('example.com')
+
+        self.assertTrue(
+            os.path.exists(cert),
+            'www.example.com is still configured and is served from this exact '
+            'file - removing the apex must not unlink it')
+        self.assertEqual(before, self.read(cert))
+        self.assertTrue(self.edge_would_start())
+
+    def test_shared_lineage_is_not_certbot_deleted(self):
+        cert = self._cert_for('example.com')
+        self.add_domain('example.com', 'be_apex', ssl_cert_path=cert)
+        self.add_domain('www.example.com', 'be_www', ssl_cert_path=cert)
+
+        self.remove('example.com')
+
+        self.assertEqual(
+            [], self.certbot_deletes(),
+            'certbot delete destroys archive, live and renewal config for a '
+            'lineage that is still renewing the certificate www.example.com '
+            'is served with')
+
+    def test_production_shape_mail_alias_removal(self):
+        """The exact row that stopped a production cleanup.
+
+        mail.threeworldsoneheart.org carried
+        ssl_cert_path=/etc/haproxy/certs/threeworldsoneheart.org.pem - the live
+        PEM of a different, serving site.
+        """
+        cert = self._cert_for('threeworldsoneheart.org')
+        before = self.read(cert)
+        self.add_domain('threeworldsoneheart.org', 'be_apex', ssl_cert_path=cert)
+        self.add_domain('www.threeworldsoneheart.org', 'be_www', ssl_cert_path=cert)
+        self.add_domain('mail.threeworldsoneheart.org', 'be_mail', ssl_cert_path=cert)
+
+        self.remove('mail.threeworldsoneheart.org')
+
+        self.assertTrue(os.path.exists(cert),
+                        'two sites are still served from this bundle')
+        self.assertEqual(before, self.read(cert))
+        self.assertEqual([], self.certbot_deletes())
+        self.assertTrue(self.edge_would_start())
+
+    def test_removing_every_name_eventually_cleans_up(self):
+        """The guard defers cleanup, it does not cancel it."""
+        cert = self._cert_for('example.com')
+        self.add_domain('example.com', 'be_apex', ssl_cert_path=cert)
+        self.add_domain('www.example.com', 'be_www', ssl_cert_path=cert)
+
+        self.remove('www.example.com')
+        self.assertTrue(os.path.exists(cert), 'apex still needs it')
+
+        self.remove('example.com')
+        self.assertFalse(os.path.exists(cert),
+                         'the last reference is gone - now it may be removed')
+        self.assertEqual(
+            ['delete --cert-name example.com --non-interactive'],
+            self.certbot_deletes())
+
+    def test_unrelated_domains_certificate_is_untouched(self):
+        """Sharing is by exact path; different bundles stay independent."""
+        mine = self._cert_for('example.com')
+        theirs = self._cert_for('other.example')
+        theirs_before = self.read(theirs)
+        self.add_domain('example.com', 'be_mine', ssl_cert_path=mine)
+        self.add_domain('other.example', 'be_theirs', ssl_cert_path=theirs)
+
+        self.remove('example.com')
+
+        self.assertFalse(os.path.exists(mine))
+        self.assertTrue(os.path.exists(theirs),
+                        'a different bundle must not be collateral damage')
+        self.assertEqual(theirs_before, self.read(theirs))
+        self.assertEqual(
+            ['delete --cert-name example.com --non-interactive'],
+            self.certbot_deletes())
+
+
 @unittest.skipIf(TESTING_FOREIGN_TREE,
                  'HAPROXY_MANAGER_DIR points at another tree')
 class TestPublisherApiIsPresent(unittest.TestCase):

scripts/test-runtime-map-contract.py

@@ -0,0 +1,397 @@
+#!/usr/bin/env python3
+"""Contract test: the runtime-map fast path (blocked IPs).
+
+Why this file exists
+--------------------
+`add_ip_to_runtime_map()` and `remove_ip_from_runtime_map()` spent their whole
+existence sending
+
+    add map #0 <ip> 1
+    del map #0 <ip>
+
+to `/tmp/haproxy-cli` and returning True whenever socat exited 0. Neither
+command has ever worked. Two independent defects:
+
+  * **No `@1` prefix.** `/tmp/haproxy-cli` is HAProxy's MASTER CLI socket; map
+    commands are worker commands. The master answers `Unknown command: 'add',
+    but maybe one of the following ones is a better match: ...` -- and **socat
+    still exits 0**, so `result.returncode == 0` was true and the function
+    logged "Added IP x to runtime map".
+  * **`#0` is not a valid map id.** Ids are assigned at config-parse time and
+    move on every config regeneration; on the live edge `blocked_ips.map` is
+    id 37 and `trusted_ips.map` is 10. There is no id 0. Any hardcoded number
+    is wrong -- the map must be referenced by its FILE PATH, which is what
+    haproxy.cfg itself names in `map_ip(/etc/haproxy/blocked_ips.map,0)`.
+
+And a third silence that makes a response-body check alone insufficient:
+`@1 add map #0 <ip> 1` returns an **empty body**, exit 0, and adds nothing
+anywhere -- while `@1 del map #0 <ip>` answers `Unknown map identifier.`. The
+add path can therefore only be trusted after reading the entry back.
+
+IP blocking still worked, because `update_blocked_ips_map()` rewrites
+`/etc/haproxy/blocked_ips.map` and the callers reload HAProxy, which re-reads
+it. The FILE is authoritative; this is the no-reload fast path, and it has
+never once run while reporting that it did.
+
+What it enforces
+----------------
+  1. The command strings actually sent: `@1` prefix first, map referenced by
+     PATH and never by `#<id>`, and the value `1` that
+     `map_ip(...,0) -m int gt 0` requires.
+  2. Every captured rejection is classified as FAILURE (returns False), not
+     success -- including the two that carry no error text at all.
+  3. Success is only reported when the entry reads back in the state asked
+     for. Not the exit status, not an empty reply.
+  4. No source in this repo builds a map command with a `#<id>` reference.
+     Comments may describe the old form; code may not use it.
+
+Runs fully offline: `_cli_send()` is replaced, so no socket, no socat, no
+HAProxy, no network.
+
+Running
+-------
+    python3 scripts/test-runtime-map-contract.py
+"""
+
+import ast
+import io
+import os
+import re
+import sys
+import glob
+import logging
+import tempfile
+import unittest
+
+MODULE_DIR = os.path.abspath(
+    os.environ.get('HAPROXY_MANAGER_DIR',
+                   os.path.join(os.path.dirname(os.path.abspath(__file__)), '..'))
+)
+os.chdir(MODULE_DIR)
+sys.path.insert(0, MODULE_DIR)
+
+_LOG_DIR = tempfile.mkdtemp(prefix='haproxy-mgr-test-logs-')
+_real_file_handler = logging.FileHandler
+logging.FileHandler = (
+    lambda filename, *a, **kw: _real_file_handler(
+        os.path.join(_LOG_DIR, os.path.basename(filename)), *a, **kw)
+)
+
+import haproxy_manager  # noqa: E402
+
+logging.getLogger().setLevel(logging.CRITICAL)
+
+MAP = haproxy_manager.BLOCKED_IPS_MAP_PATH
+IP = '192.0.2.77'
+
+# ---------------------------------------------------------------------------
+# Responses captured VERBATIM from the haproxy-manager container on the live
+# edge (HAProxy 3.0.11, 2026-08-22). socat exited 0 for every single one of
+# them, which is the entire reason none of this is decided on exit status.
+# ---------------------------------------------------------------------------
+
+# What the MASTER socket answers to an unprefixed `add map ...` -- i.e. the
+# reply the old code read as success.
+MASTER_REJECTS_ADD = """\
+Unknown command: 'add', but maybe one of the following ones is a better match:
+  @!<pid>                                 : send a command to the <pid> process
+  @master                                 : send a command to the master process
+  hard-reload                             : achieve a hard-reload (-st) of haproxy
+  reload                                  : achieve a soft-reload (-sf) of haproxy
+  user                                    : lower the level of the current CLI session to user
+  help [<command>]                        : list matching or all commands
+  prompt [timed]                          : toggle interactive mode with prompt
+  quit                                    : disconnect
+"""
+
+MASTER_REJECTS_DEL = MASTER_REJECTS_ADD.replace("'add'", "'del'")
+
+# `@1 del map #0 <ip>` / `@1 get map /etc/haproxy/nope.map <ip>`.
+UNKNOWN_MAP_IDENTIFIER = 'Unknown map identifier. Please use #<id> or <file>.\n'
+
+# `@1 add map <map> <ip>` with the value omitted (as the old docs showed).
+ADD_MAP_MISSING_VALUE = (
+    "'add map' expects three parameters (map identifier, key and value) or one "
+    "parameter (map identifier) and a payload\n")
+
+# `@1 del map <map> <ip>` for a key the runtime map does not hold.
+KEY_NOT_FOUND = 'Key not found.\n'
+
+# A successful mutation. THIS IS THE WHOLE PROBLEM: it is byte-for-byte what
+# `@1 add map #0 <ip> 1` also returns while adding nothing at all.
+MUTATION_OK = ''
+
+GET_FOUND = ('type=ip, case=sensitive, found=yes, idx=tree, key="%s", '
+             'value="1", type="str"\n' % IP)
+GET_NOT_FOUND = 'type=ip, case=sensitive, found=no\n'
+# `@1 get map #0 <ip>` -- "found", but with no value. haproxy.cfg matches with
+# `-m int gt 0`, so an entry like this does NOT block.
+GET_FOUND_NO_VALUE = ('type=ip, case=sensitive, found=yes, idx=tree, key="%s", '
+                      'value=none\n' % IP)
+
+SHOW_MAP_SAMPLE = (
+    '0x7f6e5a788700 101.36.109.130 1\n'
+    '0x7f6e5a788780 101.47.140.218 1\n'
+    '0x7f6e5a1fd500 %s 1\n' % IP)
+
+
+class FakeSocket(object):
+    """Replaces _cli_send(). Records every command, answers from a script.
+
+    `script` is a list of (substring, response) pairs, consulted in order; the
+    first whose substring appears in the command wins. Anything unmatched is
+    an explicit test bug, not a silent default.
+    """
+
+    def __init__(self, script):
+        self.script = script
+        self.sent = []
+
+    def __call__(self, command, socket_path, timeout):
+        self.sent.append(command)
+        for needle, response in self.script:
+            if needle in command:
+                return response
+        raise AssertionError('test script has no response for %r' % command)
+
+
+def with_socket(script):
+    """Install a FakeSocket for the duration of a `with` block."""
+    class _Ctx(object):
+        def __enter__(self):
+            self.fake = FakeSocket(script)
+            self._real = haproxy_manager._cli_send
+            haproxy_manager._cli_send = self.fake
+            return self.fake
+
+        def __exit__(self, *exc):
+            haproxy_manager._cli_send = self._real
+            return False
+    return _Ctx()
+
+
+# Every command the happy path needs, in the order the code issues them.
+HAPPY_ADD = [('get map', GET_FOUND), ('add map', MUTATION_OK)]
+HAPPY_DEL = [('get map', GET_NOT_FOUND), ('del map', MUTATION_OK)]
+
+
+class CommandsAreWellFormed(unittest.TestCase):
+    """Guard 1: the exact bytes on the wire.
+
+    Both original defects are visible here and nowhere else -- a `#0` map
+    reference and a missing `@1` are perfectly ordinary-looking Python.
+    """
+
+    def test_add_sends_worker_prefixed_path_referenced_command(self):
+        with with_socket(HAPPY_ADD) as fake:
+            self.assertTrue(haproxy_manager.add_ip_to_runtime_map(IP))
+        self.assertEqual(fake.sent[0], '@1 add map %s %s 1' % (MAP, IP))
+
+    def test_del_sends_worker_prefixed_path_referenced_command(self):
+        with with_socket(HAPPY_DEL) as fake:
+            self.assertTrue(haproxy_manager.remove_ip_from_runtime_map(IP))
+        self.assertEqual(fake.sent[0], '@1 del map %s %s' % (MAP, IP))
+
+    def test_every_command_is_tried_on_the_worker_first(self):
+        """`/tmp/haproxy-cli` is the MASTER socket; bare map commands 404."""
+        for run in (lambda: haproxy_manager.add_ip_to_runtime_map(IP),
+                    lambda: haproxy_manager.remove_ip_from_runtime_map(IP)):
+            with with_socket(HAPPY_ADD + HAPPY_DEL) as fake:
+                run()
+            for command in fake.sent:
+                self.assertTrue(command.startswith('@1 '),
+                                '%r is missing the @1 worker prefix' % command)
+
+    def test_no_command_references_a_map_by_id(self):
+        """Map ids move on every config regeneration. Path, always."""
+        script = HAPPY_ADD + HAPPY_DEL + [('show map', SHOW_MAP_SAMPLE)]
+        with with_socket(script) as fake:
+            haproxy_manager.add_ip_to_runtime_map(IP)
+            haproxy_manager.remove_ip_from_runtime_map(IP)
+            haproxy_manager.runtime_map_keys(MAP)
+        for command in fake.sent:
+            self.assertNotRegex(
+                command, r'\bmap\s+#',
+                '%r references a map by id; ids are not stable' % command)
+            self.assertIn(MAP, command,
+                          '%r does not name the map file' % command)
+
+    def test_add_carries_the_value_the_config_matches_on(self):
+        """`map_ip(...,0) -m int gt 0`: a valueless entry does not block."""
+        self.assertEqual(haproxy_manager.BLOCKED_IPS_MAP_VALUE, '1')
+        with with_socket(HAPPY_ADD) as fake:
+            haproxy_manager.add_ip_to_runtime_map(IP)
+        self.assertTrue(fake.sent[0].endswith(' %s 1' % IP),
+                        '%r has no value; HAProxy rejects it' % fake.sent[0])
+
+
+class RejectionIsFailure(unittest.TestCase):
+    """Guard 2+3: nothing may report success unless the map really changed.
+
+    Every response below was returned by the live socket with **exit code 0**.
+    The old code returned True for all of them.
+    """
+
+    def test_master_socket_rejection_of_add_is_failure(self):
+        with with_socket([('get map', GET_NOT_FOUND), ('add map', MASTER_REJECTS_ADD)]):
+            self.assertFalse(haproxy_manager.add_ip_to_runtime_map(IP))
+
+    def test_master_socket_rejection_of_del_is_failure(self):
+        with with_socket([('get map', GET_FOUND), ('del map', MASTER_REJECTS_DEL)]):
+            self.assertFalse(haproxy_manager.remove_ip_from_runtime_map(IP))
+
+    def test_unknown_map_identifier_is_failure(self):
+        with with_socket([('get map', GET_NOT_FOUND), ('add map', UNKNOWN_MAP_IDENTIFIER)]):
+            self.assertFalse(haproxy_manager.add_ip_to_runtime_map(IP))
+        with with_socket([('get map', GET_FOUND), ('del map', UNKNOWN_MAP_IDENTIFIER)]):
+            self.assertFalse(haproxy_manager.remove_ip_from_runtime_map(IP))
+
+    def test_missing_value_rejection_is_failure(self):
+        """Carries no marker word at all -- caught by 'a mutation says nothing'."""
+        with with_socket([('get map', GET_NOT_FOUND), ('add map', ADD_MAP_MISSING_VALUE)]):
+            self.assertFalse(haproxy_manager.add_ip_to_runtime_map(IP))
+
+    def test_silent_noop_add_is_failure(self):
+        """The `#0` failure mode: accepted, empty reply, nothing added.
+
+        Nothing in the response distinguishes this from success. Only the
+        read-back does -- which is why the read-back is not optional.
+        """
+        with with_socket([('get map', GET_NOT_FOUND), ('add map', MUTATION_OK)]):
+            self.assertFalse(haproxy_manager.add_ip_to_runtime_map(IP))
+
+    def test_add_that_lands_without_a_value_is_failure(self):
+        with with_socket([('get map', GET_FOUND_NO_VALUE), ('add map', MUTATION_OK)]):
+            self.assertFalse(haproxy_manager.add_ip_to_runtime_map(IP))
+
+    def test_del_that_leaves_the_key_behind_is_failure(self):
+        with with_socket([('get map', GET_FOUND), ('del map', MUTATION_OK)]):
+            self.assertFalse(haproxy_manager.remove_ip_from_runtime_map(IP))
+
+    def test_key_not_found_on_del_is_the_requested_end_state(self):
+        """Not a failure: the runtime map already lacks the key."""
+        with with_socket([('get map', GET_NOT_FOUND), ('del map', KEY_NOT_FOUND)]):
+            self.assertTrue(haproxy_manager.remove_ip_from_runtime_map(IP))
+
+    def test_verified_success_is_reported_as_success(self):
+        with with_socket(HAPPY_ADD):
+            self.assertTrue(haproxy_manager.add_ip_to_runtime_map(IP))
+        with with_socket(HAPPY_DEL):
+            self.assertTrue(haproxy_manager.remove_ip_from_runtime_map(IP))
+
+    def test_a_runtime_failure_never_raises_into_the_request_handler(self):
+        """The map file + reload still enforces the block; degrade, don't 500."""
+        with with_socket([('get map', GET_NOT_FOUND), ('add map', MASTER_REJECTS_ADD)]):
+            self.assertIs(haproxy_manager.add_ip_to_runtime_map(IP), False)
+
+    def test_mutations_are_checked_for_an_empty_body_not_a_marker_list(self):
+        """_HAPROXY_CLI_ERROR_MARKERS can only know rejections already seen."""
+        with self.assertRaises(haproxy_manager.HaproxyCliError):
+            with with_socket([('add map', 'something nobody has ever seen\n')]):
+                haproxy_manager.haproxy_cli('add map %s x 1' % MAP,
+                                            worker=True, expect_empty=True)
+
+    def test_the_new_error_markers_are_recognised(self):
+        for response in (UNKNOWN_MAP_IDENTIFIER, KEY_NOT_FOUND,
+                         MASTER_REJECTS_ADD):
+            self.assertTrue(haproxy_manager._cli_response_is_error(response),
+                            '%r must be classified as an error' % response[:40])
+        for response in (GET_FOUND, GET_NOT_FOUND, SHOW_MAP_SAMPLE):
+            self.assertFalse(haproxy_manager._cli_response_is_error(response),
+                             '%r is data, not an error' % response[:40])
+
+    def test_socat_exit_zero_carries_no_information(self):
+        """FakeSocket never signals failure any other way, and neither did socat."""
+        with with_socket([('get map', GET_NOT_FOUND), ('add map', MASTER_REJECTS_ADD)]) as fake:
+            self.assertFalse(haproxy_manager.add_ip_to_runtime_map(IP))
+        self.assertTrue(fake.sent, 'the command was sent and "succeeded" at the '
+                                   'process level; only the body says otherwise')
+
+
+class ReadBack(unittest.TestCase):
+    """The read-back primitives the guarantees above rest on."""
+
+    def test_lookup_reports_found_with_value(self):
+        with with_socket([('get map', GET_FOUND)]):
+            self.assertEqual(haproxy_manager.runtime_map_lookup(MAP, IP),
+                             (True, '1'))
+
+    def test_lookup_reports_not_found(self):
+        with with_socket([('get map', GET_NOT_FOUND)]):
+            self.assertEqual(haproxy_manager.runtime_map_lookup(MAP, IP),
+                             (False, None))
+
+    def test_lookup_raises_on_a_rejected_reference(self):
+        with with_socket([('get map', UNKNOWN_MAP_IDENTIFIER)]):
+            with self.assertRaises(haproxy_manager.HaproxyCliError):
+                haproxy_manager.runtime_map_lookup(MAP, IP)
+
+    def test_keys_parses_show_map_output(self):
+        with with_socket([('show map', SHOW_MAP_SAMPLE)]):
+            self.assertEqual(
+                haproxy_manager.runtime_map_keys(MAP),
+                {'101.36.109.130', '101.47.140.218', IP})
+
+    def test_empty_map_is_not_a_rejection(self):
+        with with_socket([('show map', '')]):
+            self.assertEqual(haproxy_manager.runtime_map_keys(MAP), set())
+
+    def test_rejected_show_map_still_raises(self):
+        with with_socket([('show map', UNKNOWN_MAP_IDENTIFIER)]):
+            with self.assertRaises(haproxy_manager.HaproxyCliError):
+                haproxy_manager.runtime_map_keys(MAP)
+
+
+MAP_BY_ID_RE = re.compile(r'\b(?:add|del|clear|show|get)\s+map\s+#')
+
+
+class NoSourceBuildsAMapIdCommand(unittest.TestCase):
+    """Guard 4: `map #<id>` may be described in comments, never executed.
+
+    Scanning string literals rather than raw text is deliberate -- the whole
+    reason this bug is documented at length in the source is so the next reader
+    does not reintroduce it, and a plain grep would fail on those comments.
+    """
+
+    def test_no_python_string_literal_builds_a_map_id_command(self):
+        tree = ast.parse(io.open('haproxy_manager.py', encoding='utf-8').read())
+        offenders = [
+            node.value for node in ast.walk(tree)
+            if isinstance(node, ast.Constant) and isinstance(node.value, str)
+            and MAP_BY_ID_RE.search(node.value)
+            and not (ast.get_docstring(tree) == node.value)
+        ]
+        # Docstrings are string literals too; exclude any literal that is a
+        # docstring of a module/class/function.
+        docstrings = set()
+        for node in ast.walk(tree):
+            if isinstance(node, (ast.Module, ast.ClassDef, ast.FunctionDef,
+                                 ast.AsyncFunctionDef)):
+                doc = ast.get_docstring(node, clean=False)
+                if doc:
+                    docstrings.add(doc)
+        offenders = [o for o in offenders if o not in docstrings]
+        self.assertEqual(offenders, [],
+                         'these string literals build a map command with an '
+                         'unstable #<id> reference')
+
+    def test_no_shell_or_template_code_line_uses_a_map_id(self):
+        targets = (glob.glob('scripts/*.sh') + glob.glob('templates/*.tpl')
+                   + ['Dockerfile'])
+        offenders = []
+        for path in targets:
+            if not os.path.exists(path):
+                continue
+            for lineno, line in enumerate(
+                    io.open(path, encoding='utf-8').read().splitlines(), 1):
+                if line.lstrip().startswith('#'):
+                    continue  # a comment describing the old form is fine
+                if MAP_BY_ID_RE.search(line):
+                    offenders.append('%s:%d: %s' % (path, lineno, line.strip()))
+        self.assertEqual(offenders, [],
+                         'map ids are assigned at config-parse time and move; '
+                         'reference the map file by path')
+
+
+if __name__ == '__main__':
+    unittest.main(verbosity=2)

scripts/test-stick-table-contract.py

@@ -0,0 +1,475 @@
+#!/usr/bin/env python3
+"""Contract test: what the stick tables STORE vs what the consumers READ.
+
+Why this file exists
+--------------------
+`/api/security/stats` and `scripts/show-tarpit-ips.sh` spent their whole
+existence reporting "Scan Count", "offense count" and "BLOCKED" figures parsed
+out of `gpc0` and `gpc1`. **No stick table in this repo has ever stored a
+general-purpose counter.** Every one of those numbers was fabricated, and an
+operator was making decisions on them.
+
+Nothing caught it, because each layer failed silently in a different way:
+
+  * `int(parts[3])` on a positional split hit `exp=368842`, raised ValueError,
+    and the loop `continue`d -- so the endpoint answered `active_threats: 0`
+    with an empty list. "No threats" and "the parser is broken" looked
+    identical.
+  * The command went to the MASTER CLI socket without the `@1` worker prefix.
+    HAProxy answered `Unknown command: 'show', but maybe one of the following
+    ones is a better match: ...` and **socat still exited 0**, so the
+    `returncode != 0` guard never fired. The reported `total_tracked_ips` was
+    the line count of that help text (8) while the real table held 388 entries.
+  * The shell consumers wrote `gpc0=${gpc0:-0}`, so a field that does not exist
+    rendered as a confident zero.
+
+The durable fix is not "parse better" -- it is making the template and its
+consumers unable to drift apart without something going red. That is this file.
+
+What it enforces
+----------------
+  1. `STICK_TABLE_FIELD_CONTRACT` in haproxy_manager.py equals, exactly and in
+     both directions, the `store` clauses in the rendered templates.
+  2. Every shell consumer's `EXPECTED_FIELDS=(...)` array equals the contract.
+  3. A captured sample of REAL `show table` output from the live edge parses to
+     exactly the contract's fields plus the entry metadata -- so the contract
+     describes reality, not just itself.
+  4. The loud-failure behaviour: `read_stick_table()` RAISES on a rejected
+     command, on a non-table response, and on a row missing a contract field.
+     It must never answer zeros. Guard 4 is the one that would have caught the
+     original bug on day one.
+  5. No `store` clause names a general-purpose counter, and no template tracks
+     one -- the state this repo is actually in, asserted rather than assumed.
+
+Assertions about the TEMPLATES go through `rule_lines()`, which strips comments
+before matching. These templates quote their own rules in prose at length; a
+bare `assertIn` over the rendered text passes just as happily against a rule
+that has been commented out. Same lesson, and same helper, as
+scripts/test-wpadmin-gate.py.
+
+Runs fully offline -- no HAProxy, no socket, no network.
+
+Running
+-------
+    python3 scripts/test-stick-table-contract.py
+"""
+
+import os
+import re
+import sys
+import logging
+import tempfile
+import unittest
+
+MODULE_DIR = os.path.abspath(
+    os.environ.get('HAPROXY_MANAGER_DIR',
+                   os.path.join(os.path.dirname(os.path.abspath(__file__)), '..'))
+)
+os.chdir(MODULE_DIR)
+sys.path.insert(0, MODULE_DIR)
+
+_LOG_DIR = tempfile.mkdtemp(prefix='haproxy-mgr-test-logs-')
+_real_file_handler = logging.FileHandler
+logging.FileHandler = (
+    lambda filename, *a, **kw: _real_file_handler(
+        os.path.join(_LOG_DIR, os.path.basename(filename)), *a, **kw)
+)
+
+import haproxy_manager  # noqa: E402
+
+
+# ---------------------------------------------------------------------------
+# A captured sample of REAL output, so this test can assert against reality
+# without a live socket.
+#
+# Provenance: `echo "@1 show table web" | socat stdio /tmp/haproxy-cli` inside
+# the haproxy-manager container on whp01, 2026-08-22, HAProxy 3.0.11. Rows
+# trimmed for length; format byte-for-byte as emitted.
+#
+# Note what is and is NOT here: no gpc0, no gpc1, no gpc(N), no gpc_rate, no
+# glitch_rate. Note also that field windows come back in MILLISECONDS
+# (`conn_rate(10000)`), not the `10s` the template writes -- a consumer that
+# labels the raw number "10s" is off by a factor of 1000.
+# ---------------------------------------------------------------------------
+LIVE_TABLE_SAMPLE = """\
+# table: web, type: ip, size:204800, used:388
+0x7f6e5447e728: key=17.58.57.102 use=0 exp=368842 shard=0 conn_rate(10000)=0 conn_cur=0 http_req_rate(10000)=0 http_err_rate(30000)=0
+0x7f6e541cf7c8: key=43.173.182.9 use=0 exp=171404 shard=0 conn_rate(10000)=0 conn_cur=0 http_req_rate(10000)=0 http_err_rate(30000)=0
+0x7f6e5547f528: key=95.129.255.180 use=0 exp=590639 shard=0 conn_rate(10000)=1 conn_cur=0 http_req_rate(10000)=1 http_err_rate(30000)=0
+0x7f6e5447e488: key=5.9.105.254 use=0 exp=556277 shard=0 conn_rate(10000)=0 conn_cur=0 http_req_rate(10000)=0 http_err_rate(30000)=1
+"""
+
+# The verbatim reply the MASTER CLI socket gives to an unprefixed worker
+# command. Captured the same way. socat exits 0 on this -- which is the entire
+# reason haproxy_cli() inspects the body.
+MASTER_SOCKET_REJECTION = """\
+Unknown command: 'show', but maybe one of the following ones is a better match:
+  show cli level                          : display the level of the current CLI session
+  show cli sockets                        : dump list of cli sockets
+  show proc                               : show processes status
+  show startup-logs                       : report logs emitted during HAProxy startup
+  show version                            : show version of the current process
+  help [<command>]                        : list matching or all commands
+  prompt [timed]                          : toggle interactive mode with prompt
+  quit                                    : disconnect
+"""
+
+# Anything matching these in a `store` clause is a general-purpose counter.
+GPC_PATTERN = re.compile(r'\bgpc|glitch')
+
+# Shell consumers that must declare their field expectations as a single
+# EXPECTED_FIELDS array, and the table each one reads.
+SHELL_CONSUMERS = {
+    'scripts/show-edge-ip-rates.sh': 'web',
+}
+
+EXPECTED_FIELDS_RE = re.compile(r'^\s*EXPECTED_FIELDS=\(([^)]*)\)\s*$', re.M)
+
+
+def rule_lines(cfg, needle):
+    """Comment-stripped config lines containing `needle`.
+
+    A line that is entirely a comment is dropped; a line mixing config with a
+    trailing comment is truncated at the first ' #' before matching. Without
+    this, every assertion below would pass against a rule that had been
+    commented out but whose text survived in the surrounding prose -- and these
+    templates quote their own rules in prose constantly. See
+    scripts/test-wpadmin-gate.py, where a mutation audit proved the point.
+    """
+    out = []
+    for raw in cfg.split('\n'):
+        stripped = raw.strip()
+        if stripped and not stripped.startswith('#'):
+            code = stripped.split(' #', 1)[0].rstrip()
+            if code and needle in code:
+                out.append(code)
+    return out
+
+
+def render_listener():
+    return haproxy_manager.template_env.get_template('hap_listener.tpl').render(
+        crt_path='/etc/haproxy/certs',
+        suspension_enabled=False,
+        coraza_spoe_backend=None,
+    )
+
+
+def render_security_tables():
+    return haproxy_manager.template_env.get_template('hap_security_tables.tpl').render()
+
+
+def stick_tables_from(cfg):
+    """{table name: (field, ...)} for every stick-table declared in `cfg`.
+
+    A stick-table takes the name of the frontend/backend/listen section that
+    declares it -- that name is what `show table <name>` wants, so the section
+    header is part of the contract, not incidental. Section headers and
+    stick-table lines are both read comment-stripped.
+    """
+    tables = {}
+    section = None
+    for raw in cfg.split('\n'):
+        stripped = raw.strip()
+        if not stripped or stripped.startswith('#'):
+            continue
+        code = stripped.split(' #', 1)[0].rstrip()
+        header = re.match(r'^(frontend|backend|listen)\s+(\S+)', code)
+        if header:
+            section = header.group(2)
+            continue
+        if code.startswith('stick-table'):
+            store = re.search(r'\bstore\s+(\S+)', code)
+            if not store:
+                raise AssertionError(
+                    'stick-table in section %r has no `store` clause: %r'
+                    % (section, code))
+            if section is None:
+                raise AssertionError(
+                    'stick-table declared outside any section: %r' % code)
+            # store is a comma-separated list; each item is name or name(window)
+            fields = tuple(item.split('(')[0]
+                           for item in store.group(1).split(','))
+            tables[section] = fields
+    return tables
+
+
+def all_template_stick_tables():
+    tables = {}
+    for cfg in (render_listener(), render_security_tables()):
+        for name, fields in stick_tables_from(cfg).items():
+            if name in tables:
+                raise AssertionError('stick table %r declared twice' % name)
+            tables[name] = fields
+    return tables
+
+
+class StickTableContract(unittest.TestCase):
+    """Guard 1 + 5: the templates and the Python contract, held together."""
+
+    def setUp(self):
+        self.templates = all_template_stick_tables()
+        self.contract = haproxy_manager.STICK_TABLE_FIELD_CONTRACT
+
+    def test_templates_actually_declare_stick_tables(self):
+        """Guard the guard: an empty parse would make every other check vacuous."""
+        self.assertTrue(self.templates,
+                        'parsed no stick tables out of the templates at all -- '
+                        'stick_tables_from() is broken, not the templates')
+
+    def test_same_table_names(self):
+        self.assertEqual(
+            sorted(self.templates), sorted(self.contract),
+            'STICK_TABLE_FIELD_CONTRACT and the templates disagree on WHICH '
+            'stick tables exist. Add/remove the table in both places.')
+
+    def test_same_fields_per_table(self):
+        for table in sorted(self.templates):
+            with self.subTest(table=table):
+                self.assertEqual(
+                    sorted(self.templates[table]),
+                    sorted(self.contract.get(table, ())),
+                    "stick table %r stores %s but STICK_TABLE_FIELD_CONTRACT "
+                    "claims %s. Whichever is wrong, a consumer is about to read "
+                    "a field that is never populated -- which is the bug this "
+                    "test exists for." % (table,
+                                          list(self.templates[table]),
+                                          list(self.contract.get(table, ()))))
+
+    def test_web_table_is_the_one_the_api_reads(self):
+        self.assertIn('web', self.contract)
+        self.assertIn('web', self.templates)
+
+    def test_no_general_purpose_counters_are_stored(self):
+        """The state of the world today, asserted rather than assumed.
+
+        If a gpc/glitch counter is ever genuinely added to a template, this
+        test is the place to update -- and updating it forces whoever does so
+        to also add the field to STICK_TABLE_FIELD_CONTRACT (test_same_fields_
+        per_table) and to the shell consumers (test_shell_consumers_match_
+        contract). That chain is the point: a counter cannot appear in a
+        consumer without existing in the table, and cannot appear in the table
+        without the consumers being updated.
+        """
+        for table, fields in self.templates.items():
+            for field in fields:
+                self.assertIsNone(
+                    GPC_PATTERN.search(field),
+                    'stick table %r now stores %r. Update this test, '
+                    'STICK_TABLE_FIELD_CONTRACT, and every consumer.'
+                    % (table, field))
+
+    def test_track_sc_counters_have_a_table_each(self):
+        """Every `track-scN ... table X` names a table that really exists.
+
+        A typo here is invisible to `haproxy -c` only in the sense that it is
+        NOT -- but it is invisible to the consumers, which would query a table
+        that is never written.
+        """
+        cfg = render_listener()
+        for line in rule_lines(cfg, 'track-sc'):
+            named = re.search(r'\btable\s+(\S+)', line)
+            if named:
+                self.assertIn(
+                    named.group(1), self.templates,
+                    'track-sc rule references undeclared table %r: %r'
+                    % (named.group(1), line))
+
+    def test_sc_counter_indices_fit_haproxys_limit(self):
+        """sc0/sc1/sc2 are all HAProxy gives us by default.
+
+        `tune.stick-counters` defaults to 3. A `track-sc3` without raising it
+        is a config-time failure, and the templates' own comments assume the
+        limit -- so assert it rather than leaving it as folklore.
+        """
+        cfg = render_listener() + '\n' + render_security_tables()
+        raised = rule_lines(cfg, 'tune.stick-counters')
+        limit = 3
+        if raised:
+            limit = int(re.search(r'(\d+)', raised[-1]).group(1))
+        for line in rule_lines(cfg, 'track-sc'):
+            idx = int(re.search(r'track-sc(\d+)', line).group(1))
+            self.assertLess(
+                idx, limit,
+                'track-sc%d exceeds tune.stick-counters (%d): %r'
+                % (idx, limit, line))
+
+
+class ShellConsumerContract(unittest.TestCase):
+    """Guard 2: the shell consumers cannot drift from the contract."""
+
+    def test_shell_consumers_match_contract(self):
+        for path, table in sorted(SHELL_CONSUMERS.items()):
+            with self.subTest(script=path):
+                full = os.path.join(MODULE_DIR, path)
+                self.assertTrue(os.path.exists(full),
+                                '%s is missing; it is a declared consumer of '
+                                'stick table %r' % (path, table))
+                with open(full) as fh:
+                    src = fh.read()
+                m = EXPECTED_FIELDS_RE.search(src)
+                self.assertIsNotNone(
+                    m, '%s must declare its field expectations once as a '
+                       'single-line `EXPECTED_FIELDS=(a b c)` array so this '
+                       'test can hold it to the template' % path)
+                declared = sorted(m.group(1).split())
+                self.assertEqual(
+                    declared,
+                    sorted(haproxy_manager.STICK_TABLE_FIELD_CONTRACT[table]),
+                    '%s reads %s but stick table %r stores %s'
+                    % (path, declared, table,
+                       sorted(haproxy_manager.STICK_TABLE_FIELD_CONTRACT[table])))
+
+    def test_retired_script_no_longer_parses_phantom_counters(self):
+        """show-tarpit-ips.sh may explain gpc0/gpc1; it may not extract them.
+
+        The shim is allowed -- encouraged -- to name the fields in prose so an
+        operator who runs it learns why its numbers went away. What it must not
+        do is go back to pulling values out of them.
+        """
+        path = os.path.join(MODULE_DIR, 'scripts/show-tarpit-ips.sh')
+        if not os.path.exists(path):
+            self.skipTest('show-tarpit-ips.sh has been removed outright')
+        with open(path) as fh:
+            lines = fh.readlines()
+        for raw in lines:
+            stripped = raw.strip()
+            if not stripped or stripped.startswith('#'):
+                continue
+            code = stripped.split(' #', 1)[0]
+            self.assertIsNone(
+                re.search(r"grep -o ['\"]?gpc|gpc[0-9]*=\$|sc_get_gpc|sc-inc-gpc", code),
+                'show-tarpit-ips.sh is extracting a general-purpose counter '
+                'again: %r' % stripped)
+
+
+class LiveSampleParses(unittest.TestCase):
+    """Guard 3: the contract describes real HAProxy output, not just itself."""
+
+    def test_header_parses(self):
+        header, entries = self._read()
+        self.assertEqual(header['name'], 'web')
+        self.assertEqual(header['type'], 'ip')
+        self.assertEqual(header['size'], 204800)
+        self.assertEqual(header['used'], 388)
+        self.assertEqual(len(entries), 4)
+
+    def test_sample_fields_are_exactly_contract_plus_metadata(self):
+        expected = set(haproxy_manager.STICK_TABLE_FIELD_CONTRACT['web'])
+        expected |= set(haproxy_manager.STICK_TABLE_ENTRY_META)
+        _, entries = self._read()
+        for line, fields in entries:
+            self.assertEqual(
+                set(fields), expected,
+                'real `show table web` output carries %s, contract+metadata '
+                'expects %s. Row: %r'
+                % (sorted(fields), sorted(expected), line))
+
+    def test_key_is_the_ip_not_the_allocation_pointer(self):
+        """The original bug read parts[0] -- the `0x...:` pointer -- as the IP."""
+        _, entries = self._read()
+        ips = [f['key']['value'] for _, f in entries]
+        self.assertIn('95.129.255.180', ips)
+        for ip in ips:
+            self.assertFalse(ip.startswith('0x'),
+                             'parsed a memory address as an IP: %r' % ip)
+
+    def test_windows_are_milliseconds(self):
+        """HAProxy reports `conn_rate(10000)` for a `conn_rate(10s)` store.
+
+        Asserted because labelling that raw 10000 as "10s" (or as seconds) is
+        an easy and completely silent way to be wrong by 1000x in the panel.
+        """
+        _, entries = self._read()
+        _, fields = entries[0]
+        self.assertEqual(fields['conn_rate']['window_ms'], 10000)
+        self.assertEqual(fields['http_req_rate']['window_ms'], 10000)
+        self.assertEqual(fields['http_err_rate']['window_ms'], 30000)
+        self.assertIsNone(fields['conn_cur']['window_ms'],
+                          'conn_cur is a gauge, not a rate; it has no window')
+
+    def test_values_are_the_real_ones(self):
+        _, entries = self._read()
+        by_ip = {f['key']['value']: f for _, f in entries}
+        self.assertEqual(by_ip['95.129.255.180']['http_req_rate']['value'], '1')
+        self.assertEqual(by_ip['5.9.105.254']['http_err_rate']['value'], '1')
+        self.assertEqual(by_ip['17.58.57.102']['http_req_rate']['value'], '0')
+
+    def _read(self):
+        return _read_table_from(LIVE_TABLE_SAMPLE)
+
+
+class FailsLoudly(unittest.TestCase):
+    """Guard 4: every way this can go wrong must raise, never return zeros.
+
+    This is the guard that would have caught the original bug immediately. Each
+    case below is a real response the old code accepted silently.
+    """
+
+    def test_master_socket_rejection_is_not_data(self):
+        """The exact reply that used to be reported as `total_tracked_ips: 8`."""
+        with self.assertRaises(haproxy_manager.HaproxyCliError) as ctx:
+            _read_table_from(MASTER_SOCKET_REJECTION)
+        self.assertIn('not a stick-table dump', str(ctx.exception))
+
+    def test_socat_exit_zero_does_not_mean_success(self):
+        """haproxy_cli() must reject on the BODY, not the exit status.
+
+        socat returns 0 for every response above -- the rejection is only ever
+        visible in the text.
+        """
+        self.assertTrue(
+            haproxy_manager._cli_response_is_error(MASTER_SOCKET_REJECTION))
+        self.assertTrue(
+            haproxy_manager._cli_response_is_error('No such table\n'))
+        self.assertTrue(
+            haproxy_manager._cli_response_is_error('Permission denied\n'))
+        self.assertFalse(
+            haproxy_manager._cli_response_is_error(LIVE_TABLE_SAMPLE))
+
+    def test_missing_contract_field_raises_and_names_it(self):
+        """A field the table stopped storing must not silently become 0."""
+        degraded = LIVE_TABLE_SAMPLE.replace(' http_err_rate(30000)=0', '')
+        with self.assertRaises(haproxy_manager.HaproxyCliError) as ctx:
+            _read_table_from(degraded)
+        msg = str(ctx.exception)
+        self.assertIn('http_err_rate', msg,
+                      'the error must name the missing field')
+        self.assertIn('drifted', msg,
+                      'the error must say what actually went wrong')
+
+    def test_empty_response_raises(self):
+        with self.assertRaises(haproxy_manager.HaproxyCliError):
+            _read_table_from('')
+
+    def test_row_without_key_raises(self):
+        broken = LIVE_TABLE_SAMPLE.replace('key=17.58.57.102 ', '')
+        with self.assertRaises(haproxy_manager.HaproxyCliError) as ctx:
+            _read_table_from(broken)
+        self.assertIn('no key=', str(ctx.exception))
+
+    def test_unknown_table_raises_before_touching_the_socket(self):
+        """Querying a table with no contract is a programming error, not a 0."""
+        with self.assertRaises(haproxy_manager.HaproxyCliError) as ctx:
+            haproxy_manager.read_stick_table('does_not_exist')
+        self.assertIn('no field contract', str(ctx.exception))
+
+    def test_empty_table_is_not_an_error(self):
+        """A table with zero entries is a legitimate, distinguishable result."""
+        header, entries = _read_table_from(
+            '# table: web, type: ip, size:204800, used:0\n')
+        self.assertEqual(header['used'], 0)
+        self.assertEqual(entries, [])
+
+
+def _read_table_from(response, table='web'):
+    """Run read_stick_table() against a canned response instead of a socket."""
+    real = haproxy_manager.haproxy_cli
+    haproxy_manager.haproxy_cli = lambda cmd, worker=False, timeout=None: response
+    try:
+        return haproxy_manager.read_stick_table(table)
+    finally:
+        haproxy_manager.haproxy_cli = real
+
+
+if __name__ == '__main__':
+    unittest.main(verbosity=2)

templates/hap_backend.tpl

@@ -4,7 +4,7 @@ backend {{ name }}-backend
     option forwardfor
     # Pass the real client IP to backend (from proxy headers or direct connection)
     # This is crucial for container-level logging and security tools
-    http-request add-header X-CLIENT-IP %[var(txn.real_ip)]
+    http-request set-header X-CLIENT-IP %[var(txn.real_ip)]
     http-request set-header X-Real-IP %[var(txn.real_ip)]
     http-request set-header X-Forwarded-For %[var(txn.real_ip)]
     http-request set-header X-Forwarded-Proto https if { ssl_fc }
@@ -29,7 +29,7 @@ backend {{ name }}-sse-backend
 
     option forwardfor
     # Pass the real client IP to backend (from proxy headers or direct connection)
-    http-request add-header X-CLIENT-IP %[var(txn.real_ip)]
+    http-request set-header X-CLIENT-IP %[var(txn.real_ip)]
     http-request set-header X-Real-IP %[var(txn.real_ip)]
     http-request set-header X-Forwarded-For %[var(txn.real_ip)]
     http-request set-header X-Forwarded-Proto https if { ssl_fc }

templates/hap_backend_http_check.tpl

@@ -4,7 +4,7 @@ backend {{ name }}-backend
     option httpchk
     # Pass the real client IP to backend (from proxy headers or direct connection)
     # This is crucial for container-level logging and security tools
-    http-request add-header X-CLIENT-IP %[var(txn.real_ip)]
+    http-request set-header X-CLIENT-IP %[var(txn.real_ip)]
     http-request set-header X-Real-IP %[var(txn.real_ip)]
     http-request set-header X-Forwarded-For %[var(txn.real_ip)]
     {% if ssl_enabled %}http-request set-header X-Forwarded-Proto https if { ssl_fc }{% endif %}

templates/hap_backend_longlived.tpl

@@ -14,7 +14,7 @@ backend {{ name }}-backend
     timeout tunnel 6h
     timeout http-keep-alive 6h
     option forwardfor
-    http-request add-header X-CLIENT-IP %[var(txn.real_ip)]
+    http-request set-header X-CLIENT-IP %[var(txn.real_ip)]
     http-request set-header X-Real-IP %[var(txn.real_ip)]
     http-request set-header X-Forwarded-For %[var(txn.real_ip)]
     http-request set-header X-Forwarded-Proto https if { ssl_fc }
@@ -31,7 +31,7 @@ backend {{ name }}-sse-backend
     timeout tunnel 6h
     timeout http-keep-alive 6h
     option forwardfor
-    http-request add-header X-CLIENT-IP %[var(txn.real_ip)]
+    http-request set-header X-CLIENT-IP %[var(txn.real_ip)]
     http-request set-header X-Real-IP %[var(txn.real_ip)]
     http-request set-header X-Forwarded-For %[var(txn.real_ip)]
     http-request set-header X-Forwarded-Proto https if { ssl_fc }

templates/hap_backend_websocket.tpl

@@ -7,7 +7,7 @@ backend {{ name }}-backend
     timeout tunnel 6h
     timeout http-keep-alive 6h
     option forwardfor
-    http-request add-header X-CLIENT-IP %[var(txn.real_ip)]
+    http-request set-header X-CLIENT-IP %[var(txn.real_ip)]
     http-request set-header X-Real-IP %[var(txn.real_ip)]
     http-request set-header X-Forwarded-For %[var(txn.real_ip)]
     http-request set-header X-Forwarded-Proto https if { ssl_fc }
@@ -24,7 +24,7 @@ backend {{ name }}-sse-backend
     timeout tunnel 6h
     timeout http-keep-alive 6h
     option forwardfor
-    http-request add-header X-CLIENT-IP %[var(txn.real_ip)]
+    http-request set-header X-CLIENT-IP %[var(txn.real_ip)]
     http-request set-header X-Real-IP %[var(txn.real_ip)]
     http-request set-header X-Forwarded-For %[var(txn.real_ip)]
     http-request set-header X-Forwarded-Proto https if { ssl_fc }

templates/hap_listener.tpl

@@ -582,7 +582,9 @@ frontend web
 
     # IP blocking using map file (manual blocks only)
     # Map file format: /etc/haproxy/blocked_ips.map contains "<ip_or_cidr> 1" per line
-    # Runtime updates: echo "add map #0 IP_ADDRESS 1" | socat stdio /var/run/haproxy.sock
+    # Runtime updates (worker command, map referenced by PATH -- "#<id>" ids
+    # move on every config regeneration and "#0" silently adds nothing):
+    #   echo "@1 add map /etc/haproxy/blocked_ips.map IP_ADDRESS 1" | socat stdio /tmp/haproxy-cli
     # Checks the real client IP (from headers if present, otherwise src)
     # map_ip() converter supports both single IPs and CIDR ranges (e.g., 192.168.1.0/24)
     acl is_blocked_ip var(txn.real_ip),map_ip(/etc/haproxy/blocked_ips.map,0) -m int gt 0