WIP: Ship the tools the VPN toggle grants capability for #28

Draft
jknapp wants to merge 4 commits from feat/vpn-tooling into main
4 changed files with 78 additions and 41 deletions
Showing only changes of commit 92d64cf252 - Show all commits
+14 -7
View File
@@ -204,7 +204,7 @@ docker exec stdout → tokio task → emit("terminal-output-{sessionId}") → li
- **`Dockerfile`** — Ubuntu 24.04 base with Claude Code, Node.js 22, Python 3.12, Rust, Docker CLI, git, gh, AWS CLI v2, ripgrep, pnpm, uv, ruff pre-installed, plus the shared
libraries a browser links against (see below) and the VPN tooling the `vpn_support_enabled`
toggle grants capability for (`iproute2`, `wireguard-tools`, `nftables`)
toggle grants capability for (`iproute2`, `wireguard-tools`, `iptables`)
- **Browser runtime libraries are baked in; browser *binaries* are not.** A layer runs
`npx --yes playwright@latest install-deps chromium` as root, so Playwright names its own
dependencies and the list cannot rot against Ubuntu 24.04's `t64` renames or a new Chromium
@@ -327,12 +327,19 @@ container is created once by a very long function where a dropped capability is
`/run`. Anything writing key material there inherits the problem — the same `docker commit`
hazard as `triple-c.git-token-hash` and the custom-env fingerprint, in a directory that looks
ephemeral and is not. A VPN client that does this should delete its key on teardown.
- **`wg-quick` full tunnels need `xt_CONNMARK` from the host kernel**, which WSL2 does not have and
a container cannot load; `Recommends: nftables | iptables` is also stripped by
`--no-install-recommends`, so `nftables` is baked explicitly. See the Dockerfile comment — the
short version is that shipping the backend fixes native Linux and Docker Desktop for Mac, nothing
fixes Docker Desktop for Windows, and adding the routes directly with `ip route` sidesteps it on
all three.
- **`iptables` is baked, and picking `nftables` instead would have been wrong.** `Recommends:
nftables | iptables` is stripped by `--no-install-recommends`, and `wg-quick` needs a backend for
any `AllowedIPs = 0.0.0.0/0`. `nftables` is the tempting choice — preferred by `wg-quick`, half
the size — but `wg-quick` picks nft *unconditionally* when present, and its nft ruleset needs
`nft_fib_ipv4`, which LinuxKit (Docker Desktop for Mac) does not build while it *does* build
`xt_CONNMARK`. Shipping nftables would therefore have forfeited Mac. See the Dockerfile comment;
the kernel-config evidence is quoted there.
- **Two `wg-quick` failures remain, and only one is ours to fix.** Full tunnels still need
`xt_CONNMARK`, which WSL2 before 6.6 lacks — nothing installable changes that. And every
provider's stock config carries a `DNS =` line that fails in `set_dns()` before any routing, so it
breaks split tunnels too; `openresolv` has no candidate on noble and `resolvconf` drags in
systemd-resolved, so that one is documented rather than fixed. Driving `wg` and `ip route`
directly avoids both, which is what the skill does.
### Container Lifecycle
+19 -9
View File
@@ -477,19 +477,19 @@ When enabled, the container is given the three things a VPN client needs to buil
the `NET_ADMIN` capability, the `/dev/net/tun` device, and the `net.ipv4.conf.all.src_valid_mark`
sysctl that WireGuard requires. This is **off by default**.
The `ip`, `wg` and `nft` commands ship in the container image so there is something able to use
The `ip`, `wg` and `iptables` commands ship in the container image so there is something able to use
them. If your project's container was created from an older base image it will not have them, and
`wg` will simply not be found — **migrating the project onto the current base image** is what picks
them up. Installing them by hand with `sudo apt install wireguard-tools` works in the meantime, but
lives in the writable layer, so it is undone by a **Reset** and by a migration.
them up. `sudo apt install iproute2 wireguard-tools iptables` works in the meantime, but lives in
the writable layer, so it is undone by a **Reset** and by a migration.
**This setting makes a tunnel possible; it does not make one.** Nothing is connected, no traffic is
redirected, and no tunnel is configured or started on your behalf. Enabling it and expecting the
container's traffic to start leaving through a VPN is the most common misreading of what it does —
configuring a tunnel and routing traffic into it remains yours to do.
Without it, a client such as PIA, WireGuard or OpenVPN installs and its daemon starts normally, but
the connection attempt **hangs until it times out** — a default container has no tun device to open
With the setting **off**, a client such as PIA or OpenVPN installs and its daemon starts normally,
but the connection attempt **hangs until it times out** — a default container has no tun device to open
and no permission to add an interface or a route, and most clients report that as a generic timeout
rather than a permissions error.
@@ -528,10 +528,19 @@ Things worth knowing:
address via the original gateway, or the tunnel's encrypted packets try to route through the
tunnel. Note that a health check which fetches an IP literal such as `1.1.1.1` passes cleanly
while DNS is broken — resolve a name instead.
- **`wg-quick` cannot bring up a full tunnel on Docker Desktop for Windows.** Its `Table=auto` mode
routes by firewall mark and needs `xt_CONNMARK` from the host kernel, which WSL2's does not have
and a container cannot load. Split tunnels (a specific `AllowedIPs`) work fine, as does adding
the routes yourself with `ip route`. Native Linux and Docker Desktop for Mac are unaffected.
- **Delete a client's key material when you tear a tunnel down.** Anything written under `/run` is
in the container's writable layer, and recreating or migrating the project runs `docker commit`
over it — so a WireGuard private key left there gets baked into the project's snapshot image and
copied forward from then on. This is not hypothetical; it has already happened here.
- **Strip the `DNS =` line from a provider's `.conf` before `wg-quick up`.** Every commercial
provider ships one, and `wg-quick` hands it to `resolvconf`, which is not installed — so it fails
at `resolvconf: command not found` and deletes the interface again. This happens before any
routing, so it takes **split tunnels down too**. Set the resolver another way instead, or drive
`wg` and `ip route` directly rather than going through `wg-quick`.
- **`wg-quick` full tunnels also need `xt_CONNMARK` from the host kernel.** Native Linux, Docker
Desktop for Mac and WSL2 kernels from 6.6 have it; older WSL2 kernels do not, and a container
cannot load one. There the answer is again to add the routes yourself with `ip route`, which
needs no firewall backend on any platform.
> This setting can only be changed when the container is stopped. Capabilities and devices are
> fixed when a container is created, so toggling it recreates the container on the next start.
@@ -1291,6 +1300,7 @@ The sandbox container (Ubuntu 24.04) comes pre-installed with:
| ruff | Latest | Python linter/formatter |
| Rust | Stable | Rust development (via rustup) |
| Docker CLI | Latest | Container management (when spawning is enabled) |
| iproute2, WireGuard tools, iptables | Latest | Building a tunnel (when VPN Support is enabled) |
| git | Latest | Version control |
| GitHub CLI (gh) | Latest | GitHub integration |
| AWS CLI | v2 | AWS services and Bedrock |
+1 -1
View File
@@ -135,7 +135,7 @@ pub const FEATURE_PROBES: &[(&str, &str)] = &[
("/usr/local/bin/triple-c-task-runner", "Scheduled task runner"),
("/usr/local/bin/triple-c-sso-refresh", "AWS SSO auto-refresh"),
("/opt/mission-control", "Mission Control (Flight Control)"),
("/usr/bin/wg", "VPN support (WireGuard tools)"),
("/usr/bin/wg", "VPN tooling (WireGuard, for the VPN Support toggle)"),
];
/// Headroom demanded on Docker's storage backend on top of the measured
+44 -24
View File
@@ -36,7 +36,7 @@ RUN for i in 1 2 3 4 5; do \
socat \
iproute2 \
wireguard-tools \
nftables \
iptables \
&& rm -rf /var/lib/apt/lists/*
# `libnss3-tools` above provides `certutil`. Chrome/Chromium read neither
@@ -45,7 +45,7 @@ RUN for i in 1 2 3 4 5; do \
# corporate CA, no matter what the system trust store says. entrypoint.sh
# degrades to a warning if it is ever missing.
# `iproute2`, `wireguard-tools` and `nftables` above are what the VPN support
# `iproute2`, `wireguard-tools` and `iptables` above are what the VPN support
# toggle (`vpn_support_enabled`) grants capability *for*. That toggle hands a
# project CAP_NET_ADMIN and /dev/net/tun; without `ip` there is then no way to
# add a route, and without `wg` no way to build the tunnel those two exist to
@@ -58,42 +58,62 @@ RUN for i in 1 2 3 4 5; do \
# — silently, since a VPN that fails to come up looks exactly like one that was
# never started.
#
# Measured against the *current base image*, not a bare ubuntu:24.04 — the base
# already ships libelf1t64, so measuring on bare ubuntu over-counts by ~209 kB:
# +9 packages, 5,614 kB on amd64 (4,153 kB of that is iproute2+wireguard-tools,
# 1,461 kB is nftables). The same set on arm64 is 7,422 kB, measured against
# ubuntu:24.04 since the arm64 base is not cached here.
# Measured against the *current base image*, since a bare ubuntu:24.04 also
# pulls libelf1t64 and netbase, which this base already has, and so over-reports
# by ~258 kB: **+12 packages, 7,203 kB on amd64**. The same set on arm64 is
# ~14.4 MB — the package list is identical on both arches, the binaries are
# simply larger (measured as 14.7 MB on arm64 ubuntu:24.04, less that 258 kB).
#
# ## Why `nftables` specifically
# ## Why `iptables`, and not `nftables`
#
# `wireguard-tools` declares `Recommends: nftables | iptables`, which the
# `--no-install-recommends` above strips. That is not cosmetic: `wg-quick`'s
# `add_default()` runs whenever a config has `AllowedIPs = 0.0.0.0/0` — i.e.
# every stock full-tunnel config every provider hands out — and it shells out to
# a firewall backend with no `type -p` guard. Measured without one:
# a firewall backend with no `type -p` guard. Measured with neither installed:
#
# [#] iptables-restore -n
# /usr/bin/wg-quick: line 32: iptables-restore: command not found
# wg-quick EXIT=127 (interface rolled back, split tunnels unaffected)
# wg-quick EXIT=127
#
# `nftables` rather than `iptables` because `wg-quick` prefers it (`if type -p
# nft`, so with both installed iptables is dead weight), it is the first
# alternative in the package's own Recommends, and it is roughly half the size.
# `nftables` looks like the better pick — wg-quick prefers it, it is first in
# that Recommends, it is half the size — and it is the wrong one. wg-quick picks
# nft *unconditionally* when present (`if type -p nft`, line 241), so installing
# it makes the iptables path unreachable; and its nft ruleset needs three
# expression families where the iptables path needs one. Isolating them on a
# WSL2 host, the two connmark rules install fine and this is what fails:
#
# This does NOT make `wg-quick`'s full-tunnel mode work everywhere. `Table=auto`
# routes by fwmark and needs connection-mark tracking from the *host* kernel:
# nft add rule ... fib saddr type != local drop
# Error: Could not process rule: No such file or directory
# ^^^^^^^^^^^^^^ needs nft_fib_ipv4
#
# Warning: Extension CONNMARK revision 0 not supported, missing kernel module?
# That matters because of how the two hosts we ship to are configured. From
# LinuxKit's kernel config — Docker Desktop for Mac, identical on both arches:
#
# WSL2's kernel has no `xt_CONNMARK` and containers have no /lib/modules to load
# one from, so on Docker Desktop for Windows `wg-quick up` on a full tunnel fails
# regardless of what is installed here. Native Linux and Docker Desktop for Mac
# have it. Shipping the backend is what makes the difference on those two;
# nothing shipped here can make the difference on WSL2, where the way out is to
# add the routes with `ip route` instead of going through `wg-quick` at all.
# CONFIG_NETFILTER_XT_CONNMARK=y <- the iptables path works
# # CONFIG_NFT_FIB_IPV4 is not set <- the nft path does not
#
# `iptables` is deliberately still NOT here: with `nftables` present `wg-quick`
# never reaches for it, so it would add size and firewall surface for nothing.
# So shipping `nftables` would forfeit the platform it was meant to fix. With
# `iptables`, full tunnels work on native Linux, on Docker Desktop for Mac, and
# on WSL2 kernels from 6.6 (which added xt_CONNMARK as a module). Only WSL2
# older than that is left out, and nothing installable here changes it — the way
# out there is to add the routes with `ip route` instead of using `wg-quick`,
# which is what the pia-vpn skill does on every platform.
#
# ## What this still does not fix
#
# `wireguard-tools` only *Suggests* `openresolv | resolvconf`, so neither is
# installed, and every provider's stock config carries a `DNS =` line. That
# fails in `set_dns()`, *before* the firewall step, so it takes split tunnels
# down too:
#
# [#] resolvconf -a wg0 -m 0 -x
# /usr/bin/wg-quick: line 32: resolvconf: command not found
#
# Deliberately not fixed here: `openresolv` has no installation candidate on
# noble, and `resolvconf` resolves only by pulling in systemd-resolved — a
# resolver daemon and systemd units, into a container with no systemd. Strip the
# `DNS =` line and set the resolver another way. Documented in HOW-TO-USE.md.
# Remove default ubuntu user to free UID 1000 for host-user remapping
RUN if id ubuntu >/dev/null 2>&1; then userdel -r ubuntu 2>/dev/null || userdel ubuntu; fi \