diff --git a/docs/superpowers/plans/2026-11-16-aws-phase7-ec2.md b/docs/superpowers/plans/2026-11-16-aws-phase7-ec2.md new file mode 100644 index 0000000..c51bf8c --- /dev/null +++ b/docs/superpowers/plans/2026-11-16-aws-phase7-ec2.md @@ -0,0 +1,95 @@ +# AWS Host Phase 7 (ECS on EC2) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** A project can run on its owner's dedicated EC2 capacity instead of Fargate, which makes Docker access, VPN support and the full Claude Code sandbox available on AWS; the capacity scales to zero when unused. + +**Architecture:** The onboarding module gains an optional per-user compute block: the user's own ECS cluster, an instance role and profile, a launch template on the ECS-optimized AMI, an Auto Scaling group (min 0) and a capacity provider with managed scaling, so instances exist only while a task needs one. An EC2 project registers an EC2-compatible task definition (awsvpc, the host's Docker socket and/or VPN device when enabled) and runs on that cluster through the capacity provider. Everything else (secret, EFS, agent tunnel, stop guard, idle stop, follower, port mappings) is phase 5–6 unchanged. + +**Tech Stack:** OpenTofu (user-onboard module, hashicorp/aws 6.73.0 pinned), Rust (aws-sdk-ecs/-ec2 read, pricing), bash entrypoint, React/TS. + +**Spec:** `docs/superpowers/specs/2026-11-13-aws-ecs-host-design.md` (Multi-user accounts → strict tenancy "Privileged and Docker allowed only on a per-user capacity provider"; ECS on EC2 vs Fargate; Feature compatibility "EC2 (dedicated)" column; Phase 7). Phase 5 and 6 rulings still bind. + +## User decisions (2026-11-16) + +- **Idle capacity scales to zero.** The Auto Scaling group has min 0; the capacity provider's managed scaling removes instances when no task needs them. Starts after idle take about 2–3 minutes longer (instance boot + image pull). +- **The instance type is set at onboarding** (default `m7i.large` for x86, `m7g.large` for ARM). Projects only choose Fargate or EC2. +- **No GPU in phase 7.** + +## Decisions this plan makes (rulings) + +- **E1. A per-user ECS cluster, not a capacity provider on the environment cluster.** `PutClusterCapacityProviders` replaces a cluster's whole list, and the environment module owns the shared cluster's list; two modules writing it would race and undo each other. The user's onboarding module creates `triple-c--` (the workspace-user policy's `ecs:cluster` condition `cluster/triple-c-*` already matches) with exactly one capacity provider. Strict isolation follows for free: nobody else's task can be placed on the user's instances. +- **E2. Compute is opt-in per onboarding.** New onboarding variables `ec2_instance_type` (empty = no EC2 compute, the default; the module creates nothing for EC2) and `ec2_max_instances` (1–4, default 1). Adding or removing it is an onboarding re-apply. `ONBOARD_MODULE_VERSION` 1 → 2 (re-apply shows "Update available"; existing users keep working on Fargate without it). +- **E3. Instance specifics:** ECS-optimized Amazon Linux 2023 AMI resolved at apply time from the public SSM parameter (`/aws/service/ecs/optimized-ami/amazon-linux-2023/recommended/image_id`, or `…/arm64/…` by the type's architecture); IMDSv2 required, hop limit 2 (tasks can't reach the instance role at hop 1 with awsvpc anyway — awsvpc tasks get no IMDS route; set `ECS_AWSVPC_BLOCK_IMDS=true` in the agent config); root volume gp3 50 GiB encrypted; instances in the environment's two subnets with the environment's tasks security group; `ECS_ENABLE_TASK_ENI` trunking is **not** enabled (one workspace per instance is the expected use; `ec2_max_instances` bounds concurrency). +- **E4. The instance role** `triple-c---instance` carries the managed-policy-equivalent inline rights for the ECS agent (register/poll/submit, ECR pull, logs) and SSM core (for ECS Exec on EC2), under the permissions boundary — which therefore grows: the boundary module gains those actions (boundary version bump; an account admin re-applies it once). Instance profile of the same name. +- **E5. Admin tier grows a third file** `aws/policies/environment-admin-compute.json` (the two existing files are at 5,805 and 3,340 of 6,144): `autoscaling:*` on `autoScalingGroup:*:autoScalingGroupName/triple-c-*` with tag conditions, EC2 launch templates tag-scoped (`triple-c:component`), `ecs:CreateCapacityProvider`/`DeleteCapacityProvider`/`UpdateCapacityProvider`/`PutClusterCapacityProviders` on `capacity-provider/triple-c-*` and `cluster/triple-c-*`, `iam:*InstanceProfile*` on `instance-profile/triple-c-*`, `iam:PassRole` to `ec2.amazonaws.com` for `role/triple-c-*`, `ssm:GetParameter` on the public AMI parameters, `ec2:RunInstances` (with launch-template + tag conditions), `iam:CreateServiceLinkedRole` for `autoscaling.amazonaws.com`. Destruction tag-scoped as in phase 3. MODULE_CALLS gains every new provider call (drift test). +- **E6. Docker access on EC2 = the host's socket.** `allow_docker_access` on an EC2 project mounts `/var/run/docker.sock` from the instance (host volume) at the same path, and the entrypoint's existing Docker-socket group step runs in aws mode when the socket exists. That is root on the user's own instance — acceptable only because the instance is theirs alone (E1); the UI says so. **Privileged mode is not offered** separately (nothing in Triple-C needs it beyond Docker/VPN). +- **E7. VPN on EC2 = the same three pieces as local** (`vpn_host_config` parity): `NET_ADMIN`, the `/dev/net/tun` device, and the `net.ipv4.conf.all.src_valid_mark=1` system control (ECS `systemControls`, allowed for awsvpc). The AMI must have the `tun` module (AL2023 does); the live test checks a `wg` interface can be created. +- **E8. The Claude Code sandbox is allowed on EC2** (bubblewrap's user namespaces work on the ECS-optimized AMI; verify live — if not, keep it off like Fargate, P9). +- **E9. EC2 task sizing:** CPU/memory become reservations that must fit the instance type (the app knows the type's vCPU/memory from a small built-in table for the default families, validated at save; unknown types → only a "must fit" warning), no `ephemeralStorage` (instance disk), no Spot (EC2 Spot is a later item). +- **E10. Cost estimate for EC2** = the instance type's on-demand hourly price (Pricing API `AmazonEC2`, Linux, shared tenancy) while running; shown with "instances scale to zero when no workspace is running". + +## Global Constraints + +- Phase 5/6 constraints hold (sdk_config only; three tag keys on per-user resources; names only in `aws/tofu_modules.rs`; P1; policy + checks + README together, ≤ 6,144 each; three-part command registration; serde defaults + TS mirrors; live AWS only via the scratchpad `awsx`, cleaned up, spend cap $10 overall). +- Module versions and pinning tests move together (HCL `template_version` ↔ Rust consts ↔ default tags). +- Fargate projects behave exactly as before (fingerprints of existing Fargate plans unchanged — pinned tests). +- Every per-user EC2 resource carries `triple-c:component=user-onboarding` via default tags (admin destruction scoping). + +## Review Focus + +1. A user without EC2 compute picks EC2 for a project: refused at save with "Ask an admin to add EC2 capacity to your onboarding" — Task 4/6. +2. First start after idle: the capacity provider has to launch an instance; the start waits (progress "Starting an instance (about 3 minutes)…") instead of timing out at the Fargate wait — Task 5. +3. A task that doesn't fit the instance type: refused at save, not stuck PROVISIONING forever — Task 4/5. +4. Removing EC2 compute from an onboarding while an EC2 project exists: the next start says why — Task 5. +5. Docker access on: `docker ps` in the workspace works and lists only the user's own instance's containers — Task 7 (live). + +--- + +### Task 1: Model and validation + +**Files:** `models/project.rs`, `lib/types.ts`, `lib/awsWorkspace.ts`, `commands/project_commands.rs` (validate_aws_update). +**Produces:** `AwsWorkspaceConfig.launch_type: LaunchType { Fargate (default), Ec2 }`; validation: Ec2 forbids `spot` and `ephemeral_gib` ≠ default is ignored; Docker access and VPN allowed only when `launch_type == Ec2` for AWS projects; `ec2_instance_spec(type) -> Option<(vcpu_units, memory_mib)>` table for m7i/m7g/c7i/c7g/r7i/r7g/t3/t4g common sizes (large…4xlarge); fit check reserve 256 MiB + 128 CPU units for the agent/OS. +- [ ] Tests: old JSON → Fargate; Ec2 + spot refused; Docker/VPN refused on Fargate AWS projects, allowed on Ec2; fit check pass/fail; unknown type → Ok with warning flag. +- [ ] Commit `feat(aws): EC2 launch type setting`. + +### Task 2: Policies — boundary and the admin compute file + +**Files:** `aws/policies/permissions-boundary.json`, new `aws/policies/environment-admin-compute.json`, `aws/terraform/permissions-boundary/*` (version), `aws/policies.rs` (+ checks, `policy_files(EnvironmentAdmin)` three files), `aws/README.md`, `aws/live_tests.rs` (admin role attaches all three). +**Produces:** E4/E5 statements; boundary version bump; checks in lock-step; destruction tag-scoped; sizes recorded. +- [ ] Tests: lock-step + evaluator; another owner's ASG/launch template/capacity provider denied; PassRole to ec2 only for `role/triple-c-*`; all files ≤ 6,144. +- [ ] Commit `feat(aws): admin and boundary permissions for per-user EC2 capacity`. + +### Task 3: Onboarding module — per-user EC2 compute + +**Files:** `aws/terraform/user-onboard/{variables.tf,main.tf,ec2.tf (new),outputs.tf}`, `.terraform.lock.hcl` if providers change (they shouldn't), `aws/tofu_modules.rs` (+ tests: version 2, variables, `MODULE_CALLS`, names `user_cluster_name(env, owner_key)`, `user_capacity_provider_name`, `user_instance_role_name`), `aws/environments.rs` (onboarding outputs: `ec2_cluster_arn`, `ec2_capacity_provider`, `ec2_instance_type` — readable how? see note). +**Produces:** E1–E4 resources gated on `ec2_instance_type != ""` (`count`); outputs; `onboarding_complete` still depends on everything. +**Note (E11):** workspace users can't read the state bucket, so the module also writes a per-user SSM record `/triple-c/users//` (`{schema:1, instance_type, cluster_arn, capacity_provider}`, String, tagged) that the workspace user can read (`ssm:GetParameter*` on `parameter/triple-c/*`). It lives outside `/triple-c/environments/` so environment detection (GetParametersByPath there) never sees it. Task 2 must make sure the admin tier can put/delete parameters under `parameter/triple-c/users/*` (tag-scoped like the environment record). +- [ ] Tests (HCL + Rust): resources present only with the variable set; ASG min 0, max `ec2_max_instances`, `protect_from_scale_in` true with managed termination protection; capacity provider managed scaling target 100; IMDSv2 required; AMI parameter by arch; user record field list pinned. +- [ ] Commit `feat(aws): optional per-user EC2 capacity in the onboarding module`. + +### Task 4: Onboarding UI + commands + +**Files:** `commands/environment_commands.rs` (onboard + admin bundle carry `ec2_instance_type`, `ec2_max_instances`), `components/settings/hosts/*` (onboarding dialog: "EC2 capacity: none | m7i.large | m7g.large | other…", max instances), `aws/environments.rs` (`OnboardingState` gains `ec2: Option` from the user record). +- [ ] Tests: vars passed through; re-apply keeps existing values unless changed; UI shows the user's compute. +- [ ] Commit `feat(aws): choose EC2 capacity when onboarding`. + +### Task 5: EC2 task definition and run + +**Files:** `aws/workspace.rs` (EC2 request: `requiresCompatibilities EC2`, awsvpc, no ephemeralStorage, Docker socket host volume + mount when enabled, VPN linuxParameters/devices/systemControls when enabled, sandbox per E8), `aws/tasks.rs` (run on the user's cluster with `capacityProviderStrategy=[{user CP, weight 1}]`; wait timeout 10 min for EC2 with the "Starting an instance…" progress while the task is PROVISIONING/PENDING without a container instance), `host/aws_host.rs` (cluster per project: environment cluster for Fargate, user cluster for EC2; reconcile/watch/stop use the project's cluster), `aws/runtime.rs` (user record read). +- [ ] Tests: Fargate request bytes unchanged (pinned fingerprints); EC2 request shape for each toggle combination; run request uses the user's cluster + CP; watch groups by cluster; missing user compute → clear error; wait progress strings. +- [ ] Commit `feat(aws): run projects on per-user EC2 capacity`. + +### Task 6: Container + UI for EC2 projects + +**Files:** `container/entrypoint.sh` (aws mode: Docker socket group step when `/var/run/docker.sock` exists; VPN skill install when `VPN_SUPPORT_ENABLED=1` as on Docker), `docker/aws_mode_tests.rs`, `HostSection.tsx` (Launch type: Fargate | EC2 (your instance: m7i.large) — disabled with the reason when the user has no EC2 compute; EC2 notes: scale-to-zero start delay; Docker access = root on your instance), `AccessSection.tsx`/`RuntimeSection.tsx` (Docker access / VPN enabled for EC2 projects), cost estimate (E10) in `aws/pricing.rs` + `AwsCostEstimate`. +- [ ] Tests: entrypoint gating; UI enable/disable per launch type; EC2 estimate arithmetic with a recorded `AmazonEC2` price fixture. +- [ ] Commit `feat(aws): EC2 projects in the UI, Docker access and VPN`. + +### Task 7: Live verification + +Via `scripts/aws-e2e-workspace.sh` (extend or add a test): onboard a workspace user with `ec2_instance_type = m7i.large`, start an EC2 project (Hybrid) with Docker access and VPN on: cold start time (instance launch), `docker ps` works, `ip link add wg0 type wireguard` (or `ip tuntap add`) works, bubblewrap check (E8), stop → capacity scales back to 0 instances within the managed-scaling window (record the time), second start time; remove; destroy; inventory incl. EC2 instances, ASGs, launch templates, capacity providers, instance profiles. Budget < $2. + +### Task 8: Docs + +Spec (phase 7 done, E1–E10 + user decisions + execution rulings, data model), `CLAUDE.md` (aws/, host/, container), `aws/README.md` (third admin file, boundary re-apply, EC2 compute, live results).