Files
haproxy-manager-base/IP_BLOCKING_API.md
shadowdaoandClaude Opus 5 c8d16b6990 fix(ip-blocking): the runtime map fast path has never once run
add_ip_to_runtime_map() and remove_ip_from_runtime_map() sent
`add map #0 <ip> 1` / `del map #0 <ip>` to /tmp/haproxy-cli and returned True
whenever socat exited 0. Neither command has ever worked, on any deployment,
for the entire life of the feature -- while logging "Added IP x to runtime map"
every single time. Two independent defects:

  * NO `@1` PREFIX. /tmp/haproxy-cli is HAProxy's MASTER CLI socket; map
    commands are worker commands. Captured verbatim on whp01:

        $ echo "add map #0 192.0.2.77 1" | socat stdio /tmp/haproxy-cli
        Unknown command: 'add', but maybe one of the following ones is a better match:
          @!<pid>   : send a command to the <pid> process
          ...
        $ echo $?
        0

    socat exits 0 on the rejection, so `result.returncode == 0` was true. Same
    silence PR #7 fixed on the `show table` path.
  * `#0` IS NOT A VALID MAP ID. Ids are assigned at config-parse time and move
    on every config regeneration -- `@1 show map` on whp01 reports
    blocked_ips.map as 37 and trusted_ips.map as 10. There is no id 0.
    Hardcoding any number is wrong; the map is referenced by FILE PATH, which
    is what haproxy.cfg itself names in map_ip(/etc/haproxy/blocked_ips.map,0).

And a third silence, which is why a response-body check alone is not enough
here: `@1 add map #0 <ip> 1` returns an EMPTY body, exit 0, and adds nothing to
any map -- while `@1 del map #0 <ip>` and `@1 show map #0` both answer
`Unknown map identifier.`. On the add path the reply is byte-for-byte identical
to success. Only reading the entry back can tell them apart.

IP blocking itself was never broken: update_blocked_ips_map() rewrites
/etc/haproxy/blocked_ips.map and the callers reload HAProxy, which re-reads it.
That path is untouched and stays authoritative. What was broken is the
no-reload fast path, plus every report that it had worked.

  * haproxy_manager.py: both functions send `@1 add|del map
    /etc/haproxy/blocked_ips.map <ip> [1]` and READ THE ENTRY BACK with
    `get map` before returning True. runtime_map_lookup()/runtime_map_keys()
    are the read-back primitives. `sync_blocked_ips` loses `clear map #0`
    (which the master socket rejected just as loudly and just as invisibly) and
    verifies the whole set with one `show map` instead of counting commands
    that did not visibly complain; it answers 207 + `runtime_map_synced: false`
    when the runtime map does not match the database.
  * haproxy_cli() grows `expect_empty=True` for MUTATING commands: HAProxy
    answers those with nothing on success, so an empty body is the success and
    ANY non-empty body is a rejection. That is stricter than the marker list on
    purpose -- markers only recognise rejections someone has already seen, and
    it catches `'add map' expects three parameters ...`, which matches nothing.
    HaproxyCliError carries `.responses` so `del map` answering `Key not found.`
    (the requested end state) is told apart from a real failure without regex.
  * The four callers capture the boolean instead of discarding it and report
    `runtime_map_updated` / `runtime_map_failures` in the API response and the
    operation log. A runtime failure degrades to "enforced on the reload that
    already happens two lines later" -- never to an unblocked IP, never to a
    500.
  * scripts/test-runtime-map-contract.py (offline, 26 tests) asserts the bytes
    on the wire (`@1` first, map by path, value `1`), classifies every captured
    response, and scans the repo's Python string literals and shell/template
    code lines for `#<id>` map references -- comments may describe the old
    form, code may not use it. Verified to fail on each defect reintroduced
    separately: no `@1` (3 failures), `#0` (4), no read-back (2), trust-the-
    reply (1).
  * The `#0` form is also corrected in IP_BLOCKING_API.md, MIGRATION_GUIDE.md
    and the comment in templates/hap_listener.tpl -- where every copy of it
    additionally omitted the `1`, which `-m int gt 0` needs to match.

The only template change is a comment; `haproxy -c` on the live rendered config
with it applied is clean (HAProxy 3.0.11, warnings unchanged).

Verified on whp01 against the running container (docker cp + SIGHUP, no
recreate). Before: both functions returned True and logged success while
`@1 get map` answered `found=no` and entry_cnt stayed at 263. After: the fixed
add lands with value "1" and the remove takes it out again; the old command
form is now classified as a failure; a `#0` map reference returns False via the
read-back. End to end through the API, `runtime_map_updated: true`, and
/api/blocked-ips/sync -- which used to be a no-op reporting a full sync --
reports 264/264 verified present.

The runtime path was isolated from the reload that normally follows it: with
NO map-file write and NO reload (same haproxy worker pid throughout), adding
100.123.171.78 (whp01's own netbird overlay address -- not a customer IP, not
in the is_local ranges) to the runtime map alone flipped a live site from
HTTP 200 to 403, and removing it flipped it back to 200. That is the fast path
working for the first time. All test IPs were removed afterwards: 0 rows in
blocked_ips, 0 lines in the map file, entry_cnt back to 263. Six customer
sites, the panel /health and `haproxy -c` are byte-identical to the baseline
taken before the change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 11:15:01 -07:00

557 lines
16 KiB
Markdown

# IP Blocking API Documentation
This document describes the IP blocking functionality added to HAProxy Manager, which allows WHP (Web Hosting Platform) to manage blocked IP addresses through the API.
## Overview
The IP blocking feature allows administrators to:
- Block specific IP addresses or CIDR ranges from accessing any sites managed by HAProxy
- Unblock previously blocked IP addresses or CIDR ranges
- View all currently blocked IP addresses and CIDR ranges
- Track who blocked an IP/CIDR and when
When an IP is blocked (or falls within a blocked CIDR range), visitors from that IP address will receive a 403 Forbidden response.
### CIDR Range Support
The IP blocking system supports CIDR notation for blocking entire network ranges:
- **Single IP**: `192.168.1.100` (blocks only this IP)
- **CIDR Range**: `192.168.1.0/24` (blocks 256 IPs from 192.168.1.0 to 192.168.1.255)
- **Common CIDR Masks**:
- `/32` - Single IP (1 address)
- `/24` - Standard subnet (256 addresses)
- `/16` - Large network (65,536 addresses)
- `/8` - Very large network (16,777,216 addresses)
## Features
- **Runtime IP blocking**: Changes take effect immediately without HAProxy restarts
- **Map file based**: No ACL word limits, supports unlimited blocked IPs
- **Safe configuration management**: Automatic validation and rollback on failures
- **Runtime map synchronization**: Keep database and HAProxy runtime in sync
- **Audit logging**: All operations are logged for monitoring and compliance
## API Endpoints
### Authentication
All IP blocking endpoints require API key authentication when `HAPROXY_API_KEY` is set:
```bash
Authorization: Bearer your-api-key
```
### 1. Get All Blocked IPs
Retrieve a list of all currently blocked IP addresses.
**Endpoint:** `GET /api/blocked-ips`
**Response:**
```json
[
{
"id": 1,
"ip_address": "192.168.1.100",
"reason": "Suspicious activity detected",
"blocked_at": "2024-01-15 10:30:00",
"blocked_by": "WHP Admin Panel"
},
{
"id": 2,
"ip_address": "10.0.0.50",
"reason": "Brute force attempts",
"blocked_at": "2024-01-15 11:45:00",
"blocked_by": "Security System"
}
]
```
**Example Request:**
```bash
curl -X GET http://localhost:8000/api/blocked-ips \
-H "Authorization: Bearer your-api-key"
```
### 2. Block an IP Address
Add an IP address to the blocked list.
**Endpoint:** `POST /api/blocked-ips`
**Request Body:**
```json
{
"ip_address": "192.168.1.100",
"reason": "Suspicious activity detected",
"blocked_by": "WHP Admin Panel"
}
```
**Parameters:**
- `ip_address` (required): The IP address or CIDR range to block (e.g., "192.168.1.100" or "192.168.1.0/24")
- `reason` (optional): Reason for blocking (default: "No reason provided")
- `blocked_by` (optional): Who/what initiated the block (default: "API")
**Response:**
```json
{
"status": "success",
"blocked_ip_id": 1,
"message": "IP 192.168.1.100 has been blocked"
}
```
**Error Responses:**
- `400 Bad Request`: IP address is missing
- `409 Conflict`: IP address is already blocked
- `500 Internal Server Error`: Configuration generation failed
**Example Request (Single IP):**
```bash
curl -X POST http://localhost:8000/api/blocked-ips \
-H "Authorization: Bearer your-api-key" \
-H "Content-Type: application/json" \
-d '{
"ip_address": "192.168.1.100",
"reason": "Multiple failed login attempts",
"blocked_by": "WHP Security Module"
}'
```
**Example Request (CIDR Range):**
```bash
curl -X POST http://localhost:8000/api/blocked-ips \
-H "Authorization: Bearer your-api-key" \
-H "Content-Type: application/json" \
-d '{
"ip_address": "192.168.1.0/24",
"reason": "DDoS attack from compromised ISP",
"blocked_by": "WHP Security Module"
}'
```
### 3. Unblock an IP Address or CIDR Range
Remove an IP address or CIDR range from the blocked list.
**Endpoint:** `DELETE /api/blocked-ips`
**Request Body:**
```json
{
"ip_address": "192.168.1.100"
}
```
**Parameters:**
- `ip_address` (required): The IP address or CIDR range to unblock (must match exactly as it was blocked)
**Response:**
```json
{
"status": "success",
"message": "IP 192.168.1.100 has been unblocked"
}
```
**Error Responses:**
- `400 Bad Request`: IP address is missing
- `404 Not Found`: IP address not found in blocked list
- `500 Internal Server Error`: Configuration generation failed
**Example Request:**
```bash
curl -X DELETE http://localhost:8000/api/blocked-ips \
-H "Authorization: Bearer your-api-key" \
-H "Content-Type: application/json" \
-d '{"ip_address": "192.168.1.100"}'
```
## Integration with WHP
### PHP Integration Example
Here's how to integrate the IP blocking API into WHP using PHP:
```php
<?php
class HAProxyIPBlocker {
private $apiUrl;
private $apiKey;
public function __construct($apiUrl, $apiKey) {
$this->apiUrl = rtrim($apiUrl, '/');
$this->apiKey = $apiKey;
}
/**
* Get all blocked IPs
*/
public function getBlockedIPs() {
return $this->makeRequest('GET', '/api/blocked-ips');
}
/**
* Block an IP address
*/
public function blockIP($ipAddress, $reason = null, $blockedBy = 'WHP Control Panel') {
$data = [
'ip_address' => $ipAddress,
'reason' => $reason ?: 'Blocked via WHP Control Panel',
'blocked_by' => $blockedBy
];
return $this->makeRequest('POST', '/api/blocked-ips', $data);
}
/**
* Unblock an IP address
*/
public function unblockIP($ipAddress) {
$data = ['ip_address' => $ipAddress];
return $this->makeRequest('DELETE', '/api/blocked-ips', $data);
}
/**
* Make API request
*/
private function makeRequest($method, $endpoint, $data = null) {
$url = $this->apiUrl . $endpoint;
$headers = [
'Authorization: Bearer ' . $this->apiKey,
'Content-Type: application/json'
];
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $method);
if ($data) {
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data));
}
$response = curl_exec($ch);
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
$result = json_decode($response, true);
if ($httpCode >= 200 && $httpCode < 300) {
return ['success' => true, 'data' => $result];
} else {
return ['success' => false, 'error' => $result['message'] ?? 'Unknown error', 'code' => $httpCode];
}
}
}
// Usage example:
$haproxyBlocker = new HAProxyIPBlocker('http://haproxy-manager:8000', 'your-api-key-here');
// Block an IP
$result = $haproxyBlocker->blockIP('192.168.1.100', 'Spam detection', 'WHP Anti-Spam Module');
if ($result['success']) {
echo "IP blocked successfully: " . $result['data']['message'];
} else {
echo "Error: " . $result['error'];
}
// Get all blocked IPs
$blockedIPs = $haproxyBlocker->getBlockedIPs();
if ($blockedIPs['success']) {
foreach ($blockedIPs['data'] as $ip) {
echo "Blocked IP: {$ip['ip_address']} - Reason: {$ip['reason']}\n";
}
}
// Unblock an IP
$result = $haproxyBlocker->unblockIP('192.168.1.100');
if ($result['success']) {
echo "IP unblocked successfully";
}
?>
```
### WHP Control Panel Integration
To add IP blocking management to the WHP control panel:
1. **Create a management interface page** (`/admin/ip-blocking.php`):
```php
<?php
// Initialize the HAProxy IP Blocker
$haproxyBlocker = new HAProxyIPBlocker(
getenv('HAPROXY_MANAGER_URL') ?: 'http://haproxy-manager:8000',
getenv('HAPROXY_API_KEY')
);
// Handle form submissions
if ($_SERVER['REQUEST_METHOD'] === 'POST') {
if (isset($_POST['action'])) {
switch ($_POST['action']) {
case 'block':
$ip = filter_var($_POST['ip_address'], FILTER_VALIDATE_IP);
if ($ip) {
$result = $haproxyBlocker->blockIP(
$ip,
$_POST['reason'] ?? '',
$_SESSION['admin_username'] ?? 'WHP Admin'
);
$message = $result['success']
? "IP {$ip} has been blocked"
: "Error: " . $result['error'];
}
break;
case 'unblock':
$ip = filter_var($_POST['ip_address'], FILTER_VALIDATE_IP);
if ($ip) {
$result = $haproxyBlocker->unblockIP($ip);
$message = $result['success']
? "IP {$ip} has been unblocked"
: "Error: " . $result['error'];
}
break;
}
}
}
// Get current blocked IPs
$blockedIPs = $haproxyBlocker->getBlockedIPs();
?>
<!DOCTYPE html>
<html>
<head>
<title>IP Blocking Management - WHP</title>
</head>
<body>
<h1>IP Blocking Management</h1>
<?php if (isset($message)): ?>
<div class="alert"><?= htmlspecialchars($message) ?></div>
<?php endif; ?>
<!-- Block IP Form -->
<h2>Block an IP Address</h2>
<form method="POST">
<input type="hidden" name="action" value="block">
<label>IP Address: <input type="text" name="ip_address" required pattern="\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}"></label><br>
<label>Reason: <input type="text" name="reason" size="50"></label><br>
<button type="submit">Block IP</button>
</form>
<!-- Currently Blocked IPs -->
<h2>Currently Blocked IPs</h2>
<table border="1">
<thead>
<tr>
<th>IP Address</th>
<th>Reason</th>
<th>Blocked By</th>
<th>Blocked At</th>
<th>Action</th>
</tr>
</thead>
<tbody>
<?php if ($blockedIPs['success']): ?>
<?php foreach ($blockedIPs['data'] as $ip): ?>
<tr>
<td><?= htmlspecialchars($ip['ip_address']) ?></td>
<td><?= htmlspecialchars($ip['reason']) ?></td>
<td><?= htmlspecialchars($ip['blocked_by']) ?></td>
<td><?= htmlspecialchars($ip['blocked_at']) ?></td>
<td>
<form method="POST" style="display:inline">
<input type="hidden" name="action" value="unblock">
<input type="hidden" name="ip_address" value="<?= htmlspecialchars($ip['ip_address']) ?>">
<button type="submit">Unblock</button>
</form>
</td>
</tr>
<?php endforeach; ?>
<?php endif; ?>
</tbody>
</table>
</body>
</html>
```
2. **Environment Configuration**
Add these environment variables to your WHP configuration:
```bash
# HAProxy Manager API Configuration
HAPROXY_MANAGER_URL=http://haproxy-manager:8000
HAPROXY_API_KEY=your-secure-api-key-here
```
3. **Automatic Blocking Integration**
You can automatically block IPs based on certain criteria:
```php
// Example: Auto-block after multiple failed login attempts
function handleFailedLogin($username, $ipAddress) {
global $haproxyBlocker;
// Track failed attempts (implement your own logic)
$failedAttempts = getFailedAttempts($ipAddress);
if ($failedAttempts >= 5) {
$haproxyBlocker->blockIP(
$ipAddress,
"5+ failed login attempts for user: {$username}",
"WHP Security System"
);
// Log the blocking action
error_log("Auto-blocked IP {$ipAddress} due to multiple failed login attempts");
}
}
```
## How It Works
1. **Database Storage**: Blocked IPs are stored in the SQLite database table `blocked_ips`
2. **HAProxy Configuration**: When an IP is blocked/unblocked, the HAProxy configuration is regenerated
3. **ACL Rules**: HAProxy uses ACL rules to check if a source IP is in the blocked list
4. **Blocked Page**: Blocked IPs are served a custom "Access Denied" page via the default backend
## Testing
To test the IP blocking functionality:
```bash
# Block your test IP
curl -X POST http://localhost:8000/api/blocked-ips \
-H "Authorization: Bearer your-api-key" \
-H "Content-Type: application/json" \
-d '{"ip_address": "YOUR_TEST_IP", "reason": "Testing"}'
# Try to access a site (you should see the blocked page)
curl -H "X-Forwarded-For: YOUR_TEST_IP" http://localhost
# Unblock the IP
curl -X DELETE http://localhost:8000/api/blocked-ips \
-H "Authorization: Bearer your-api-key" \
-H "Content-Type: application/json" \
-d '{"ip_address": "YOUR_TEST_IP"}'
```
## Notes
- IP blocks are applied globally to all domains managed by HAProxy
- Changes take effect immediately without HAProxy restarts (runtime updates)
- Blocked IPs are persistent across HAProxy restarts (stored in database and map file)
- Map files support unlimited IPs (no ACL word limit restrictions)
- Consider implementing rate limiting on the API endpoints to prevent abuse
## New API Endpoints (Map File Era)
### 4. Safe Configuration Reload
Safely reload the HAProxy configuration with validation and automatic rollback.
**Endpoint:** `POST /api/config/reload`
**Response:**
```json
{
"status": "success",
"message": "HAProxy configuration reloaded safely"
}
```
**Error Response:**
```json
{
"status": "error",
"message": "Safe reload failed: Config validation failed: ..."
}
```
**Example Request:**
```bash
curl -X POST http://localhost:8000/api/config/reload \
-H "Authorization: Bearer your-api-key"
```
### 5. Sync Runtime Map
Synchronize blocked IPs from database to HAProxy runtime map.
**Endpoint:** `POST /api/blocked-ips/sync`
**Response:**
```json
{
"status": "success",
"message": "Synced 150/150 IPs to runtime map",
"total_ips": 150,
"synced_ips": 150
}
```
**Example Request:**
```bash
curl -X POST http://localhost:8000/api/blocked-ips/sync \
-H "Authorization: Bearer your-api-key"
```
## Runtime Map Commands
For advanced users, you can interact directly with HAProxy's runtime API:
Three things about these commands are easy to get wrong, and each one fails
**silently** (socat exits 0 either way — the rejection, if any, is only in the
response body):
* `/tmp/haproxy-cli` is HAProxy's **master** CLI socket. Map commands are
worker commands and need the `@1` prefix. Without it the reply is
`Unknown command: 'add', ...`.
* Reference the map by its **file path**, never by `#<id>`. Ids are assigned at
config-parse time and move on every config regeneration (on a live edge,
`blocked_ips.map` is id 37, `trusted_ips.map` is 10 — there is no id 0).
Worse, `@1 add map #0 <ip> 1` returns an **empty** reply and adds nothing.
* Entries must carry the value `1`. `haproxy.cfg` matches with
`map_ip(...,0) -m int gt 0`, so a valueless entry does not block. (`add map`
with no value is rejected: `'add map' expects three parameters ...`.)
```bash
MAP=/etc/haproxy/blocked_ips.map
# Add IP to runtime (immediate effect)
echo "@1 add map $MAP 192.168.1.100 1" | socat stdio /tmp/haproxy-cli
# Remove IP from runtime
echo "@1 del map $MAP 192.168.1.100" | socat stdio /tmp/haproxy-cli
# Confirm what actually happened (do not trust the exit status)
echo "@1 get map $MAP 192.168.1.100" | socat stdio /tmp/haproxy-cli
# Clear all blocked IPs from runtime
echo "@1 clear map $MAP" | socat stdio /tmp/haproxy-cli
# Show all runtime map entries, and the map ids currently in use
echo "@1 show map $MAP" | socat stdio /tmp/haproxy-cli
echo "@1 show map" | socat stdio /tmp/haproxy-cli
```
The runtime map is a **fast path only**. `/etc/haproxy/blocked_ips.map` is
authoritative: HAProxy re-reads it on reload, so a failed runtime command
delays a block until the next reload rather than losing it.
## Migration from ACL Method
If you're upgrading from the old ACL-based method:
1. **Automatic**: Just update the HAProxy Manager code - it will automatically migrate
2. **Validation**: The new system includes automatic config validation and rollback
3. **No Downtime**: Runtime updates mean no service interruptions
4. **Scalable**: No more 64 IP limit - handle thousands of blocked IPs