cloud-hosting-platform/haproxy-manager-base
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
e33167159db8a45df6b1c65dee987de563d08dc9
haproxy-manager-base/CLAUDE.md
T
shadowdaoandClaude Opus 5 b6a62e7f9f fix(security-stats): stop reporting counters the stick tables never stored 
/api/security/stats and scripts/show-tarpit-ips.sh reported "Scan Count",
"offense count" and BLOCKED/TARPITTED status parsed from gpc0/gpc1. No stick
table in this repo has ever stored a general-purpose counter -- the `web` table
stores conn_cur, conn_rate(10s), http_req_rate(10s), http_err_rate(30s), and
the two brute-force tables store http_req_rate(60s). Every one of those figures
was fabricated, and an operator was making decisions on them.

Three independent silences kept it alive:

  * `int(parts[3])` on a positional split hit `exp=368842`, raised ValueError,
    and the loop `continue`d -- so the endpoint always answered
    `active_threats: 0` with an empty list. Live on whp01 it also reported
    parts[0], the `0x...:` allocation pointer, as the source IP.
  * The command was sent to /tmp/haproxy-cli WITHOUT the `@1` worker prefix.
    That is the MASTER CLI socket, which answers "Unknown command: 'show' ..."
    -- and socat still exits 0, so the `returncode != 0` guard never fired.
    `total_tracked_ips` was the line count of that help text (8) while the real
    table held 388 entries.
  * The shell consumers wrote `gpc0=${gpc0:-0}`, rendering a field that does
    not exist as a confident zero.

Report what the tables actually store, rather than adding gpc counters to make
the old semantics real. Adding them would mean editing hap_listener.tpl -- the
one change here with a silent-total-outage failure mode -- to rebuild
enforcement history that the edge access log (shipped 2026.08.8, on the host at
/var/log/haproxy.log) already records per request, with status codes,
termination states and request references the stick table could never hold.

  * haproxy_manager.py: STICK_TABLE_FIELD_CONTRACT names what each table
    stores. haproxy_cli() sends worker commands with `@1`, falls back to the
    bare form for a plain stats socket, and inspects the RESPONSE BODY because
    socat's exit status is worthless here. parse_stick_table_entry() reads
    name=value / name(window_ms)=value pairs by NAME, never by position.
    read_stick_table() RAISES -- naming the field -- when a row is missing a
    contract field, instead of defaulting it to 0.
  * /api/security/stats returns the four real counters with their windows, the
    true `used:` count, and no invented threat_level/blocked/offense_count.
    Fewer numbers, all of them real.
  * scripts/show-edge-ip-rates.sh replaces the fabricated report; the four
    expected fields are declared once as EXPECTED_FIELDS and drive the parser.
    show-tarpit-ips.sh becomes a shim that explains why its numbers are gone
    and points at where tarpit events actually live.
  * monitor-attacks.sh loses fourteen fabricated "threat" categories and a
    composite threat score, all permanently zero; its access-log section now
    says the log is on the host instead of silently printing nothing.
  * haproxy_tarpit_config.txt -- the never-shipped design sketch these counters
    were copied from -- gets a NOT IMPLEMENTED banner.
  * scripts/test-stick-table-contract.py (offline, 21 tests) holds the
    templates' `store` clauses, STICK_TABLE_FIELD_CONTRACT and every consumer
    to each other, and asserts each loud-failure path against the real captured
    responses. Template and consumers can no longer drift apart quietly.

No template is touched, so haproxy.cfg is unchanged.

Verified on whp01: total_tracked_ips now tracks `used:` exactly (511 vs the
table's 511, was 8 vs 388), and per-IP values match `show table web key <ip>`
field for field. haproxy PIDs unmoved, `haproxy -c` warnings unchanged, five
customer sites HTTP 200.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 10:48:49 -07:00

146 lines
7.5 KiB
Markdown
Raw Blame History

# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Development Commands
### Testing
- **API Testing**: `./scripts/test-api.sh` - Tests all API endpoints with optional authentication
- **Certificate Request Testing**: `./scripts/test-certificate-request.sh` - Tests certificate generation endpoints
- **Stick-table contract**: `python3 scripts/test-stick-table-contract.py` - offline; holds the templates' `store` clauses, `STICK_TABLE_FIELD_CONTRACT`, and every consumer to each other. Run it after touching any `stick-table` line.
- **Manual Testing**: Run `curl` commands against `http://localhost:8000` endpoints as shown in README.md
### Reading stick tables (and why it is easy to get silently wrong)
`/tmp/haproxy-cli` is HAProxy's **master** CLI socket. Worker commands
(`show table`, `show map`, `add map`, ...) need an `@1` prefix. Without it
HAProxy answers `Unknown command: 'show', ...` **and socat still exits 0** — so
an exit-status check passes and the help text gets parsed as data. Always use
`haproxy_cli(cmd, worker=True)` in Python, which inspects the response body.
Stick-table entries are `name=value` / `name(window_ms)=value` pairs, not fixed
columns; the first token is an allocation pointer (`0x...:`), not the key. Parse
by NAME, and treat a missing field as an ERROR — never default it to `0`. The
`web` table stores only `conn_cur`, `conn_rate`, `http_req_rate`,
`http_err_rate`; it holds **no history and no counter of past blocks**. What was
actually denied/tarpitted is in the edge access log on the **host** at
`/var/log/haproxy.log` (shipped 2026.08.8), not in any stick table.
This is written down because `/api/security/stats` and `show-tarpit-ips.sh`
reported "Scan Count"/"BLOCKED" figures parsed from `gpc0`/`gpc1` — fields no
stick table has ever stored — for their entire existence. See the header of
`haproxy_tarpit_config.txt` and the contract test.
### Running the Application
- **Docker Build**: `docker build -t haproxy-manager .`
- **Local Development**: `python haproxy_manager.py` (requires HAProxy, certbot, and dependencies installed)
- **Container Run**: See README.md for various docker run configurations
### Monitoring and Debugging
- **Error Monitoring**: `./scripts/monitor-errors.sh` - Monitor application error logs
- **External Monitoring**: `./scripts/monitor-errors-external.sh` - External monitoring script
- **Health Check**: `curl http://localhost:8000/health`
- **Log Files**:
- `/var/log/haproxy-manager.log` - General application logs
- `/var/log/haproxy-manager-errors.log` - Error logs for alerting
## Architecture Overview
### Core Components
1. **haproxy_manager.py** - Main Flask application providing:
- RESTful API for HAProxy configuration management
- SQLite database integration for domain/backend storage
- Let's Encrypt certificate automation
- HAProxy configuration generation from Jinja2 templates
- Optional API key authentication via `HAPROXY_API_KEY` environment variable
2. **Database Schema** - SQLite database with three main tables:
- `domains` - Domain configurations with SSL settings
- `backends` - Backend service definitions linked to domains
- `backend_servers` - Individual servers within backend groups
3. **Template System** - Jinja2 templates for HAProxy configuration generation:
- `hap_header.tpl` - Global HAProxy settings, defaults, and HTTP/2 tuning
- `hap_backend.tpl` - Backend server definitions
- `hap_listener.tpl` - Frontend listener configurations with rate limiting
- `hap_letsencrypt.tpl` - SSL certificate configurations
- `hap_security_tables.tpl` - Stats frontend and security stick tables
- Template override support for custom backend configurations
4. **Certificate Management** - Automated SSL certificate handling:
- Let's Encrypt integration with certbot
- Self-signed certificate fallback for development
- Certificate renewal automation via cron
- Certificate download endpoints for external services
### Configuration Flow
1. Domain added via `/api/domain` endpoint → Database updated
2. `generate_config()` function → Reads database, renders Jinja2 templates → Writes `/etc/haproxy/haproxy.cfg`
3. HAProxy reload via socket API (`/tmp/haproxy-cli`) or process restart
4. SSL certificate generation via Let's Encrypt or self-signed fallback
### Key Design Patterns
- **Template-driven configuration**: HAProxy config generated from modular Jinja2 templates
- **Database-backed state**: All configuration persisted in SQLite for reliability
- **API-first design**: All operations exposed via REST endpoints
- **Process monitoring**: Health checks and automatic HAProxy restart capabilities
- **Comprehensive logging**: Operation logging with error alerting support
### Authentication & Security
- Optional API key authentication controlled by `HAPROXY_API_KEY` environment variable
- All API endpoints (except `/health` and `/`) require Bearer token when API key is set
- Certificate private keys combined with certificates in HAProxy-compatible format
- Default backend page for unmatched domains instead of exposing HAProxy errors
### Rate Limiting & Connection Limits (hap_listener.tpl)
- **Stick table**: `type ip size 200k expire 10m` tracking `conn_cur`, `conn_rate(10s)`, `http_req_rate(10s)`, `http_err_rate(30s)`
- Tracks real client IP via `var(txn.real_ip)` to work correctly behind Cloudflare/proxies
- **Rate limit thresholds**:
- Tarpit at 3000 req/10s (300 req/s)
- Hard block (deny) at 5000 req/10s (500 req/s)
- Connection rate limit: 500/10s
- Concurrent connection limit: 500
- Error rate limit: 100/30s
- **Whitelist bypasses** (exempt from rate limits):
- `is_local` — RFC1918 private address ranges
- `is_trusted_ip` — source IPs listed in `trusted_ips.list`
- `is_whitelisted` — real IPs (from proxy headers) matched in `trusted_ips.map`
### Trusted IP Whitelist Files
- `trusted_ips.list` — Source IP whitelist for rate limit bypass (one CIDR/IP per line)
- `trusted_ips.map` — Real IP whitelist for proxy-header matching (format: `<IP> 1`)
- Both files are baked into the Docker image via `COPY` in the Dockerfile
- Ship as comment-only templates (no real IPs). Add trusted IPs locally and do **not** commit them — this repo is mirrored publicly. Entries persist in the `/etc/haproxy` named volume across recreates
### Timeout Hardening (hap_header.tpl)
- `timeout http-request`: 300s -> 30s (slowloris protection)
- `timeout connect`: 120s -> 10s
- `timeout client`: 10m -> 5m
- `timeout http-keep-alive`: 120s -> 30s
### HTTP/2 Protection (hap_header.tpl)
- `tune.h2.fe.max-total-streams 2000` — limits total streams per HTTP/2 connection
- `tune.h2.fe.glitches-threshold 50` — CVE-2023-44487 Rapid Reset protection
### Stats Frontend (hap_security_tables.tpl)
- HAProxy stats page bound to `127.0.0.1:8404` (localhost only, accessible inside container)
- Template: `templates/hap_security_tables.tpl`
### Deployment Context
- Designed to run as Docker container with persistent volumes for certificates and configurations
- Exposes ports 80 (HTTP), 443 (HTTPS), and 8000 (management API/UI)
- Stats page on port 8404 (localhost only inside container)
- Management interface on port 8000 should be firewall-protected in production
- Dockerfile HEALTHCHECK verifies both port 8000 (Flask API) and port 80 (HAProxy), with `start-period=60s` and `timeout=10s`
- Supports deployment on servers with git directory at `/root/whp` and web file sync via rsync to `/docker/whp/web/`
- HAProxy is version 3.0.11
Reference in New Issue View Git Blame Copy Permalink