Commit Graph
4 Commits
Author SHA1 Message Date
shadowdaoandClaude Opus 5 b6a62e7f9f fix(security-stats): stop reporting counters the stick tables never stored
/api/security/stats and scripts/show-tarpit-ips.sh reported "Scan Count",
"offense count" and BLOCKED/TARPITTED status parsed from gpc0/gpc1. No stick
table in this repo has ever stored a general-purpose counter -- the `web` table
stores conn_cur, conn_rate(10s), http_req_rate(10s), http_err_rate(30s), and
the two brute-force tables store http_req_rate(60s). Every one of those figures
was fabricated, and an operator was making decisions on them.

Three independent silences kept it alive:

  * `int(parts[3])` on a positional split hit `exp=368842`, raised ValueError,
    and the loop `continue`d -- so the endpoint always answered
    `active_threats: 0` with an empty list. Live on whp01 it also reported
    parts[0], the `0x...:` allocation pointer, as the source IP.
  * The command was sent to /tmp/haproxy-cli WITHOUT the `@1` worker prefix.
    That is the MASTER CLI socket, which answers "Unknown command: 'show' ..."
    -- and socat still exits 0, so the `returncode != 0` guard never fired.
    `total_tracked_ips` was the line count of that help text (8) while the real
    table held 388 entries.
  * The shell consumers wrote `gpc0=${gpc0:-0}`, rendering a field that does
    not exist as a confident zero.

Report what the tables actually store, rather than adding gpc counters to make
the old semantics real. Adding them would mean editing hap_listener.tpl -- the
one change here with a silent-total-outage failure mode -- to rebuild
enforcement history that the edge access log (shipped 2026.08.8, on the host at
/var/log/haproxy.log) already records per request, with status codes,
termination states and request references the stick table could never hold.

  * haproxy_manager.py: STICK_TABLE_FIELD_CONTRACT names what each table
    stores. haproxy_cli() sends worker commands with `@1`, falls back to the
    bare form for a plain stats socket, and inspects the RESPONSE BODY because
    socat's exit status is worthless here. parse_stick_table_entry() reads
    name=value / name(window_ms)=value pairs by NAME, never by position.
    read_stick_table() RAISES -- naming the field -- when a row is missing a
    contract field, instead of defaulting it to 0.
  * /api/security/stats returns the four real counters with their windows, the
    true `used:` count, and no invented threat_level/blocked/offense_count.
    Fewer numbers, all of them real.
  * scripts/show-edge-ip-rates.sh replaces the fabricated report; the four
    expected fields are declared once as EXPECTED_FIELDS and drive the parser.
    show-tarpit-ips.sh becomes a shim that explains why its numbers are gone
    and points at where tarpit events actually live.
  * monitor-attacks.sh loses fourteen fabricated "threat" categories and a
    composite threat score, all permanently zero; its access-log section now
    says the log is on the host instead of silently printing nothing.
  * haproxy_tarpit_config.txt -- the never-shipped design sketch these counters
    were copied from -- gets a NOT IMPLEMENTED banner.
  * scripts/test-stick-table-contract.py (offline, 21 tests) holds the
    templates' `store` clauses, STICK_TABLE_FIELD_CONTRACT and every consumer
    to each other, and asserts each loud-failure path against the real captured
    responses. Template and consumers can no longer drift apart quietly.

No template is touched, so haproxy.cfg is unchanged.

Verified on whp01: total_tracked_ips now tracks `used:` exactly (511 vs the
table's 511, was 8 vs 388), and per-IP values match `show table web key <ip>`
field for field. haproxy PIDs unmoved, `haproxy -c` warnings unchanged, five
customer sites HTTP 200.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 10:48:49 -07:00
shadowdaoandClaude 31801a6c1d Make scan detection more targeted to avoid false positives
HAProxy Manager Build and Push / Build-and-Push (push) Successful in 52s
Major changes to prevent legitimate users from being blocked:

1. Increased thresholds significantly:
   - Initial trigger: 10 → 25 errors
   - Medium level: 20 → 40 errors
   - High level: 35 → 60 errors
   - Critical level: 50 → 100 errors

2. Only count actual scan attempts as errors:
   - Script files: .php, .asp, .jsp, .cgi, .pl, .py, .rb, .sh
   - Admin paths: /wp-admin, /phpmyadmin, /adminer
   - Config files: .env, .git, .htaccess, .ini, .yml
   - Backup files: .backup, .bak, .sql, .dump
   - Known vulnerable paths: /cgi-bin, /fckeditor

3. Explicitly exclude legitimate assets from counting:
   - Images: .jpg, .png, .gif, .svg, .webp
   - Fonts: .woff, .woff2, .ttf, .eot, .otf
   - Static: .css, .js, .map, .pdf
   - Common paths: /static/, /assets/, /fonts/, /images/

4. Still count all 401/403 errors (auth failures are suspicious)

This prevents missing fonts, images, CSS files from triggering blocks
while still catching actual vulnerability scanners.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-25 12:39:15 -07:00
shadowdaoandClaude e54b4b4afe Implement progressive protection: tarpit → silent-drop → block
HAProxy Manager Build and Push / Build-and-Push (push) Successful in 1m20s
- Set tarpit timeout to 10 seconds for initial offenders
- Use silent-drop for obvious scanners (35+ errors) and repeat offenders
- Silent-drop immediately closes connection without response
- Keep 429 block for critical threats (50+ errors)

Protection levels:
- 10-19 errors: 10s tarpit
- 20-34 errors: 10s tarpit (first), silent-drop (repeat)
- 35-49 errors: silent-drop
- 50+ errors: 429 block
- Burst attacks: 10s tarpit (first), silent-drop (repeat)

Updated monitoring script to show correct status based on new logic.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-25 06:42:09 -07:00
shadowdaoandClaude b88da4c58f Implement HAProxy tarpit escalation and CLI monitoring
HAProxy Manager Build and Push / Build-and-Push (push) Successful in 51s
- Add full tarpit escalation logic with gpc1 tracking (levels 0-3)
- Implement progressive delays: 2-5s → 8-15s → 20-45s → 60s
- Increase initial threshold from 5 to 10 errors (more tolerant)
- Reduce tracking duration from 2h to 1h (faster cleanup)
- Add show-tarpit-ips.sh script for monitoring tarpitted IPs via CLI
- Script shows IP, scan count, escalation level, and tarpit status

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-24 19:33:10 -07:00