4 Commits

Include renames

Author	SHA1	Message	Date
shadowdaoandClaude Opus 5	c8d16b6990	fix(ip-blocking): the runtime map fast path has never once run ... add_ip_to_runtime_map() and remove_ip_from_runtime_map() sent `add map #0 <ip> 1` / `del map #0 <ip>` to /tmp/haproxy-cli and returned True whenever socat exited 0. Neither command has ever worked, on any deployment, for the entire life of the feature -- while logging "Added IP x to runtime map" every single time. Two independent defects: * NO `@1` PREFIX. /tmp/haproxy-cli is HAProxy's MASTER CLI socket; map commands are worker commands. Captured verbatim on whp01: $ echo "add map #0 192.0.2.77 1" | socat stdio /tmp/haproxy-cli Unknown command: 'add', but maybe one of the following ones is a better match: @!<pid> : send a command to the <pid> process ... $ echo $? 0 socat exits 0 on the rejection, so `result.returncode == 0` was true. Same silence PR #7 fixed on the `show table` path. * `#0` IS NOT A VALID MAP ID. Ids are assigned at config-parse time and move on every config regeneration -- `@1 show map` on whp01 reports blocked_ips.map as 37 and trusted_ips.map as 10. There is no id 0. Hardcoding any number is wrong; the map is referenced by FILE PATH, which is what haproxy.cfg itself names in map_ip(/etc/haproxy/blocked_ips.map,0). And a third silence, which is why a response-body check alone is not enough here: `@1 add map #0 <ip> 1` returns an EMPTY body, exit 0, and adds nothing to any map -- while `@1 del map #0 <ip>` and `@1 show map #0` both answer `Unknown map identifier.`. On the add path the reply is byte-for-byte identical to success. Only reading the entry back can tell them apart. IP blocking itself was never broken: update_blocked_ips_map() rewrites /etc/haproxy/blocked_ips.map and the callers reload HAProxy, which re-reads it. That path is untouched and stays authoritative. What was broken is the no-reload fast path, plus every report that it had worked. * haproxy_manager.py: both functions send `@1 add|del map /etc/haproxy/blocked_ips.map <ip> [1]` and READ THE ENTRY BACK with `get map` before returning True. runtime_map_lookup()/runtime_map_keys() are the read-back primitives. `sync_blocked_ips` loses `clear map #0` (which the master socket rejected just as loudly and just as invisibly) and verifies the whole set with one `show map` instead of counting commands that did not visibly complain; it answers 207 + `runtime_map_synced: false` when the runtime map does not match the database. * haproxy_cli() grows `expect_empty=True` for MUTATING commands: HAProxy answers those with nothing on success, so an empty body is the success and ANY non-empty body is a rejection. That is stricter than the marker list on purpose -- markers only recognise rejections someone has already seen, and it catches `'add map' expects three parameters ...`, which matches nothing. HaproxyCliError carries `.responses` so `del map` answering `Key not found.` (the requested end state) is told apart from a real failure without regex. * The four callers capture the boolean instead of discarding it and report `runtime_map_updated` / `runtime_map_failures` in the API response and the operation log. A runtime failure degrades to "enforced on the reload that already happens two lines later" -- never to an unblocked IP, never to a 500. * scripts/test-runtime-map-contract.py (offline, 26 tests) asserts the bytes on the wire (`@1` first, map by path, value `1`), classifies every captured response, and scans the repo's Python string literals and shell/template code lines for `#<id>` map references -- comments may describe the old form, code may not use it. Verified to fail on each defect reintroduced separately: no `@1` (3 failures), `#0` (4), no read-back (2), trust-the- reply (1). * The `#0` form is also corrected in IP_BLOCKING_API.md, MIGRATION_GUIDE.md and the comment in templates/hap_listener.tpl -- where every copy of it additionally omitted the `1`, which `-m int gt 0` needs to match. The only template change is a comment; `haproxy -c` on the live rendered config with it applied is clean (HAProxy 3.0.11, warnings unchanged). Verified on whp01 against the running container (docker cp + SIGHUP, no recreate). Before: both functions returned True and logged success while `@1 get map` answered `found=no` and entry_cnt stayed at 263. After: the fixed add lands with value "1" and the remove takes it out again; the old command form is now classified as a failure; a `#0` map reference returns False via the read-back. End to end through the API, `runtime_map_updated: true`, and /api/blocked-ips/sync -- which used to be a no-op reporting a full sync -- reports 264/264 verified present. The runtime path was isolated from the reload that normally follows it: with NO map-file write and NO reload (same haproxy worker pid throughout), adding 100.123.171.78 (whp01's own netbird overlay address -- not a customer IP, not in the is_local ranges) to the runtime map alone flipped a live site from HTTP 200 to 403, and removing it flipped it back to 200. That is the fast path working for the first time. All test IPs were removed afterwards: 0 rows in blocked_ips, 0 lines in the map file, entry_cnt back to 263. Six customer sites, the panel /health and `haproxy -c` are byte-identical to the baseline taken before the change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-22 11:15:01 -07:00
shadowdao	71f4b9ef05	Add CIDR notation support for IP blocking ... - Update map file format to include value (IP/CIDR 1) - Fix HAProxy template to use map_ip() for CIDR support - Update runtime map commands to include value - Document CIDR range blocking in API documentation - Support blocking entire network ranges (e.g., 192.168.1.0/24) This allows blocking compromised ISP ranges and other large-scale attacks.	2025-11-17 12:07:32 -08:00
shadowdaoandClaude	7869b81f27	CRITICAL FIX: Migrate HAProxy IP blocking from ACL to map files ... **Problem Solved:** - HAProxy ACL 64-word limit caused config parsing failures - "too many words, truncating after word 64" error - Complete service outage when >64 IPs were blocked - Error: "no such ACL : 'is_blocked'" broke all traffic routing **Solution: HAProxy Map Files (v1.6+)** - ✅ Unlimited IP addresses (no word limits) - ✅ Runtime updates without config reloads - ✅ Better performance (hash table vs linear search) - ✅ Safer config management with validation & rollback **Technical Implementation:** **Map File Integration:** - `/etc/haproxy/blocked_ips.map` stores all blocked IPs - `http-request deny status 403 if { src -f /etc/haproxy/blocked_ips.map }` - Runtime updates: `echo "add map #0 IP" | socat stdio /var/run/haproxy.sock` **Safety Features Added:** - `create_backup()` - Automatic config/map backups before changes - `validate_haproxy_config()` - Config validation before applying - `restore_backup()` - Automatic rollback on failures - `reload_haproxy_safely()` - Safe reload with validation pipeline **Runtime Management:** - `update_blocked_ips_map()` - Sync database to map file - `add_ip_to_runtime_map()` - Immediate IP blocking without reload - `remove_ip_from_runtime_map()` - Immediate IP unblocking **New API Endpoints:** - `POST /api/config/reload` - Safe config reload with rollback - `POST /api/blocked-ips/sync` - Sync database to runtime map **Template Changes:** - Replaced ACL method: `acl is_blocked src IP1 IP2...` (64 limit) - With map method: `http-request deny if { src -f blocked_ips.map }` (unlimited) **Backwards Compatibility:** - Existing API endpoints unchanged (GET/POST/DELETE /api/blocked-ips) - Database schema unchanged - Automatic migration on first config generation **Performance Improvements:** - O(1) hash table lookups vs O(n) linear ACL search - No config reloads needed for IP changes - Supports millions of IPs if needed - Memory efficient external file storage **Documentation:** - Complete migration guide in MIGRATION_GUIDE.md - Updated API documentation with new endpoints - Runtime management examples - Troubleshooting guide **Production Safety:** - All changes include automatic backup/restore - Config validation prevents bad deployments - Runtime updates avoid service interruption - Comprehensive error logging and monitoring This fixes the critical production outage caused by ACL word limits while providing a more scalable and performant IP blocking solution. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-08-22 08:31:17 -07:00
shadowdaoandClaude	ca37a68255	Add IP blocking functionality to HAProxy Manager ... - Add blocked_ips database table to store blocked IP addresses - Implement API endpoints for IP blocking management: - GET /api/blocked-ips: List all blocked IPs - POST /api/blocked-ips: Block an IP address - DELETE /api/blocked-ips: Unblock an IP address - Update HAProxy configuration generation to include blocked IP ACLs - Create blocked IP page template for denied access - Add comprehensive API documentation for WHP integration - Include test script for IP blocking functionality - Update .gitignore with Python patterns - Add CLAUDE.md for codebase documentation 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-08-21 18:32:47 -07:00