cloud-hosting-platform/haproxy-manager-base
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Client-supplied IP headers are trusted at the edge: X-Real-IP / CF-Connecting-IP / X-Forwarded-For are not stripped from untrusted peers #10

New Issue
Closed
opened 2026-09-18 16:03:09 +00:00 by jknapp · 1 comment
jknapp commented 2026-09-18 16:03:09 +00:00
Owner
Copy Link
Copy Source

Summary

The deployed haproxy.cfg resolves txn.real_ip from client-supplied headers without first stripping them from untrusted peers. Any anonymous visitor can therefore choose which IP address the edge believes they are, for every site behind this HAProxy.

This is a divergence between the repo's templates and what is actually running — the strip that trusted_proxies.list documents is not present in the live configuration.

Reproduction

Against any vhost behind this edge, as an ordinary internet client:

curl -H 'X-Real-IP: 203.0.113.99'        https://<any-vhost>/...
curl -H 'CF-Connecting-IP: 198.51.100.7' https://<any-vhost>/...

The backend receives X-Real-IP: 203.0.113.99 and 198.51.100.7 respectively. Verified 2026-09-18 against ols-webmail.dnspegasus.net, whose backend logs the resolved client address:

"path":"/healthz","client_ip":"203.0.113.99","peer_ip":"172.29.0.10","client_ip_source":"x-real-ip"
"path":"/healthz","client_ip":"198.51.100.7","peer_ip":"172.29.0.10","client_ip_source":"x-real-ip"

Neither address has any relationship to the real source.

Cause

/etc/haproxy/haproxy.cfg, the real-IP block in frontend web (around line 128):

acl has_cf_connecting_ip req.hdr(CF-Connecting-IP) -m found
acl has_x_real_ip        req.hdr(X-Real-IP) -m found
acl has_x_forwarded_for  req.hdr(X-Forwarded-For) -m found

http-request set-var(txn.real_ip) req.hdr_ip(CF-Connecting-IP)   if has_cf_connecting_ip
http-request set-var(txn.real_ip) req.hdr_ip(X-Real-IP)          if !has_cf_connecting_ip has_x_real_ip
http-request set-var(txn.real_ip) req.hdr_ip(X-Forwarded-For,1)  if !has_cf_connecting_ip !has_x_real_ip has_x_forwarded_for
http-request set-var(txn.real_ip) src                            if !has_cf_connecting_ip !has_x_real_ip !has_x_forwarded_for

There is no preceding http-request del-header ... guarded on the source address. Grepping the running config for del-header, trusted_prox, cloudflare_ips, src_is_trusted and is_trusted_proxy returns nothing.

The intent exists in the repo but is not deployed. trusted_proxies.list states:

Additional trusted reverse proxies — peers permitted to set CF-Connecting-IP, X-Real-IP and X-Forwarded-For. Anything NOT matched here or in cloudflare_ips.list has those headers stripped before real-IP resolution. Referenced by templates/hap_listener.tpl.

So either hap_listener.tpl is not producing that section, or the running config predates it.

Impact

txn.real_ip is not merely informational. In the same config it drives:

Line Use Consequence when forged
track-sc0 var(txn.real_ip) connection/request rate limiting evade by rotating the header
track-sc1 ... wp_bruteforce WordPress login brute-force table evade entirely
map_ip(blocked_ips.map) IP blocking an existing block is bypassed by claiming another address
map_ip(trusted_ips.map) whitelisting an attacker may be able to claim whitelisted status
add-header X-CLIENT-IP / set-header X-Real-IP forwarded to every backend every tenant's own per-IP logic inherits the forged value

The last row is how this reaches applications: any backend doing per-IP rate limiting, lockout, geo rules or audit logging is working from an attacker-chosen address. It affects all ~36 registered vhosts, not one.

For a concrete example, the webmail backend implements an escalating per-IP brute-force lockout. With this, an attacker can both evade it (rotate the header) and weaponise it (forge a victim's address, fail five logins, lock the victim out).

Suggested fix

Strip all three headers unless src matches a trusted peer, before the resolution block:

acl from_trusted_proxy src -f /etc/haproxy/trusted_proxies.list
acl from_cloudflare    src -f /etc/haproxy/cloudflare_ips.list

http-request del-header CF-Connecting-IP if !from_cloudflare !from_trusted_proxy
http-request del-header X-Real-IP        if !from_cloudflare !from_trusted_proxy
http-request del-header X-Forwarded-For  if !from_cloudflare !from_trusted_proxy
http-request del-header X-Client-IP      # set by us below; never accepted inbound

Notes worth considering while fixing:

  • X-CLIENT-IP is added, not set (http-request add-header X-CLIENT-IP), so a client-supplied one survives alongside ours and a backend reading the first occurrence gets the attacker's. Either set-header it or delete it inbound.
  • Empty lists must fail closed. With both files empty, the correct behaviour is to strip from everyone and use src — not to trust everyone.
  • Worth a startup or reload check that warns when these headers arrive from peers not in either list, so a future divergence is visible rather than silent.
  • hap_listener.tpl should be checked against a freshly generated config to confirm the deployed output actually contains the strip.

Testing

After the fix, the reproduction above should show the backend receiving the real source address, and the spoofed values should be absent. Worth also confirming that genuine Cloudflare-fronted vhosts still resolve the visitor correctly, since those legitimately depend on CF-Connecting-IP.

Found while deploying ols-webmail.dnspegasus.net behind this edge. That deployment's own client-IP handling was designed on the assumption that this strip existed; the design is unchanged, but the assumption did not hold against the running configuration.

## Summary The deployed `haproxy.cfg` resolves `txn.real_ip` from client-supplied headers **without first stripping them from untrusted peers**. Any anonymous visitor can therefore choose which IP address the edge believes they are, for every site behind this HAProxy. This is a divergence between the repo's templates and what is actually running — the strip that `trusted_proxies.list` documents is not present in the live configuration. ## Reproduction Against any vhost behind this edge, as an ordinary internet client: ``` curl -H 'X-Real-IP: 203.0.113.99' https://<any-vhost>/... curl -H 'CF-Connecting-IP: 198.51.100.7' https://<any-vhost>/... ``` The backend receives `X-Real-IP: 203.0.113.99` and `198.51.100.7` respectively. Verified 2026-09-18 against `ols-webmail.dnspegasus.net`, whose backend logs the resolved client address: ``` "path":"/healthz","client_ip":"203.0.113.99","peer_ip":"172.29.0.10","client_ip_source":"x-real-ip" "path":"/healthz","client_ip":"198.51.100.7","peer_ip":"172.29.0.10","client_ip_source":"x-real-ip" ``` Neither address has any relationship to the real source. ## Cause `/etc/haproxy/haproxy.cfg`, the real-IP block in `frontend web` (around line 128): ``` acl has_cf_connecting_ip req.hdr(CF-Connecting-IP) -m found acl has_x_real_ip req.hdr(X-Real-IP) -m found acl has_x_forwarded_for req.hdr(X-Forwarded-For) -m found http-request set-var(txn.real_ip) req.hdr_ip(CF-Connecting-IP) if has_cf_connecting_ip http-request set-var(txn.real_ip) req.hdr_ip(X-Real-IP) if !has_cf_connecting_ip has_x_real_ip http-request set-var(txn.real_ip) req.hdr_ip(X-Forwarded-For,1) if !has_cf_connecting_ip !has_x_real_ip has_x_forwarded_for http-request set-var(txn.real_ip) src if !has_cf_connecting_ip !has_x_real_ip !has_x_forwarded_for ``` There is no preceding `http-request del-header ...` guarded on the source address. Grepping the running config for `del-header`, `trusted_prox`, `cloudflare_ips`, `src_is_trusted` and `is_trusted_proxy` returns nothing. The intent exists in the repo but is not deployed. `trusted_proxies.list` states: > Additional trusted reverse proxies — peers permitted to set CF-Connecting-IP, X-Real-IP and X-Forwarded-For. Anything NOT matched here or in cloudflare_ips.list has those headers stripped before real-IP resolution. Referenced by templates/hap_listener.tpl. So either `hap_listener.tpl` is not producing that section, or the running config predates it. ## Impact `txn.real_ip` is not merely informational. In the same config it drives: | Line | Use | Consequence when forged | | --- | --- | --- | | `track-sc0 var(txn.real_ip)` | connection/request rate limiting | evade by rotating the header | | `track-sc1 ... wp_bruteforce` | WordPress login brute-force table | evade entirely | | `map_ip(blocked_ips.map)` | IP blocking | **an existing block is bypassed by claiming another address** | | `map_ip(trusted_ips.map)` | whitelisting | **an attacker may be able to claim whitelisted status** | | `add-header X-CLIENT-IP` / `set-header X-Real-IP` | forwarded to every backend | every tenant's own per-IP logic inherits the forged value | The last row is how this reaches applications: any backend doing per-IP rate limiting, lockout, geo rules or audit logging is working from an attacker-chosen address. It affects all ~36 registered vhosts, not one. For a concrete example, the webmail backend implements an escalating per-IP brute-force lockout. With this, an attacker can both evade it (rotate the header) and weaponise it (forge a victim's address, fail five logins, lock the victim out). ## Suggested fix Strip all three headers unless `src` matches a trusted peer, **before** the resolution block: ``` acl from_trusted_proxy src -f /etc/haproxy/trusted_proxies.list acl from_cloudflare src -f /etc/haproxy/cloudflare_ips.list http-request del-header CF-Connecting-IP if !from_cloudflare !from_trusted_proxy http-request del-header X-Real-IP if !from_cloudflare !from_trusted_proxy http-request del-header X-Forwarded-For if !from_cloudflare !from_trusted_proxy http-request del-header X-Client-IP # set by us below; never accepted inbound ``` Notes worth considering while fixing: - **`X-CLIENT-IP` is added, not set** (`http-request add-header X-CLIENT-IP`), so a client-supplied one survives alongside ours and a backend reading the first occurrence gets the attacker's. Either `set-header` it or delete it inbound. - **Empty lists must fail closed.** With both files empty, the correct behaviour is to strip from everyone and use `src` — not to trust everyone. - Worth a **startup or reload check** that warns when these headers arrive from peers not in either list, so a future divergence is visible rather than silent. - `hap_listener.tpl` should be checked against a freshly generated config to confirm the deployed output actually contains the strip. ## Testing After the fix, the reproduction above should show the backend receiving the **real** source address, and the spoofed values should be absent. Worth also confirming that genuine Cloudflare-fronted vhosts still resolve the visitor correctly, since those legitimately depend on `CF-Connecting-IP`. --- _Found while deploying `ols-webmail.dnspegasus.net` behind this edge. That deployment's own client-IP handling was designed on the assumption that this strip existed; the design is unchanged, but the assumption did not hold against the running configuration._
jknapp commented 2026-09-19 02:15:21 +00:00
Author
Owner
Copy Link
Copy Source

Resolved.

  • The trusted-proxy strip for CF-Connecting-IP / X-Real-IP / X-Forwarded-For has been in main since 79a1b84 (2026-08-13) and in the published image since 2026-08-23. The affected edge (192.168.1.20) was still running 2026.06.3, which predates it.
  • PR #11 closes the remaining gap: X-CLIENT-IP is now set-header instead of add-header, so an inbound copy cannot survive.
  • The edge was upgraded to 2026.09.1 on 2026-09-18; the rendered config contains the del-header rules and set-header X-CLIENT-IP.

Not yet confirmed end to end: re-run the two spoof curls from the report and check the backend log shows the real source address.

Resolved. - The trusted-proxy strip for CF-Connecting-IP / X-Real-IP / X-Forwarded-For has been in main since 79a1b84 (2026-08-13) and in the published image since 2026-08-23. The affected edge (192.168.1.20) was still running 2026.06.3, which predates it. - PR #11 closes the remaining gap: X-CLIENT-IP is now set-header instead of add-header, so an inbound copy cannot survive. - The edge was upgraded to 2026.09.1 on 2026-09-18; the rendered config contains the del-header rules and set-header X-CLIENT-IP. Not yet confirmed end to end: re-run the two spoof curls from the report and check the backend log shows the real source address.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
jknapp whmcs-sam whp-cicd-user whp-deploy
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: cloud-hosting-platform/haproxy-manager-base#10