AWS host phase 4: environments and onboarding with OpenTofu #81

Merged
jknapp merged 24 commits from feat/aws-environments into main 2026-11-15 15:39:32 +00:00
Owner

Phase 4 of the AWS ECS/Fargate host (spec docs/superpowers/specs/2026-11-13-aws-ecs-host-design.md, plan docs/superpowers/plans/2026-11-14-aws-environment-stacks.md).

What it does

Settings → Hosts → each AWS connection gets an Environments panel. Pick a region to see Triple-C environments there (ready, update available, deploying, incomplete, externally managed). From it you can:

  • Deploy a new environment with OpenTofu, which the app downloads (pinned 1.13.9, SHA-256 + GPG-verified sums) and runs itself. State lives in triple-c-tfstate-<account>-<region> (versioned, encrypted, public access blocked, TLS-only, S3-native locks).
  • Update / repair / follow / cancel / unlock a deployment. Runs are detached (hidden --tofu-runner mode), so they survive the app closing and are re-followed on reopen.
  • Join an environment: admins self-onboard; workspace users save an admin bundle (module + backend + tfvars + README with exact commands).
  • The app destroys only incomplete deployments with no valid record and no onboarded users.

CloudFormation is gone. Modules: aws/terraform/{environment,user-onboard,permissions-boundary}. Policies: environment-admin.json split into + environment-admin-network.json; workspace-user.json drops CloudFormation reads and scopes access-point creation to the owner's file system.

Key decisions: per-user EFS file system with owner-only policy; public network mode (no NAT, $0) by default, private-nat and existing available; no ECR in v1; per-user role names from a region-scoped owner key.

Testing

  • cargo test --locked: 1156 + 3 passed (24 ignored live tests)
  • vitest: 1233 passed; tsc clean
  • Live AWS (us-west-2), as a role holding only the environment-admin policies: deploy → join → update → teardown, plus private-nat. Found and fixed three missing permissions and one detection bug. Spend ≈ $0.06; account left clean.

Not yet verified (before first release, not blocking merge)

  • State bucket creation with tags in us-east-1
  • existing-VPC live apply
  • DisassociateAddress denial against a live non-Triple-C association
  • Detached runner + reattach on Windows/macOS (Cancel unsupported on Windows)
  • Log group KMS encryption (phase 5)

🤖 Generated with Claude Code

Phase 4 of the AWS ECS/Fargate host (spec `docs/superpowers/specs/2026-11-13-aws-ecs-host-design.md`, plan `docs/superpowers/plans/2026-11-14-aws-environment-stacks.md`). ## What it does Settings → Hosts → each AWS connection gets an **Environments** panel. Pick a region to see Triple-C environments there (ready, update available, deploying, incomplete, externally managed). From it you can: - **Deploy** a new environment with OpenTofu, which the app downloads (pinned 1.13.9, SHA-256 + GPG-verified sums) and runs itself. State lives in `triple-c-tfstate-<account>-<region>` (versioned, encrypted, public access blocked, TLS-only, S3-native locks). - **Update / repair / follow / cancel / unlock** a deployment. Runs are detached (hidden `--tofu-runner` mode), so they survive the app closing and are re-followed on reopen. - **Join** an environment: admins self-onboard; workspace users save an admin bundle (module + backend + tfvars + README with exact commands). - The app destroys only incomplete deployments with no valid record and no onboarded users. CloudFormation is gone. Modules: `aws/terraform/{environment,user-onboard,permissions-boundary}`. Policies: `environment-admin.json` split into + `environment-admin-network.json`; `workspace-user.json` drops CloudFormation reads and scopes access-point creation to the owner's file system. Key decisions: per-user EFS file system with owner-only policy; `public` network mode (no NAT, $0) by default, `private-nat` and `existing` available; no ECR in v1; per-user role names from a region-scoped owner key. ## Testing - `cargo test --locked`: 1156 + 3 passed (24 ignored live tests) - `vitest`: 1233 passed; `tsc` clean - Live AWS (us-west-2), as a role holding only the environment-admin policies: deploy → join → update → teardown, plus private-nat. Found and fixed three missing permissions and one detection bug. Spend ≈ $0.06; account left clean. ## Not yet verified (before first release, not blocking merge) - State bucket creation with tags in us-east-1 - `existing`-VPC live apply - DisassociateAddress denial against a live non-Triple-C association - Detached runner + reattach on Windows/macOS (Cancel unsupported on Windows) - Log group KMS encryption (phase 5) 🤖 Generated with [Claude Code](https://claude.com/claude-code)
jknapp added 24 commits 2026-11-15 14:50:51 +00:00
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runner holds an exclusive lock on apply.lock for its life and writes
its own PID, so a reused PID is never a running deploy; start_detached
waits for the lock (or kills the spawn). Cancel signals the runner alone,
which forwards SIGINT to tofu, so an AppImage's FUSE server in the same
group is never hit. tofu gets exactly child_env's variables with any
AppImage launcher changes undone, an app-owned CLI config, endpoint URLs
from AWS config ignored, and no AWS config files for keys connections.
Keys connections without keys are an error; init runs with -json so its
errors become the summary; Windows allowlist and AWS_CA_BUNDLE added.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
start_detached stops waiting once the runner holds apply.lock and returns
Started { pid: Option } (no PID: Cancel unavailable), killing only a
runner that never took the lock nor finished. The check, clear, spawn and
wait run under an exclusive start.lock, and a concurrent start or a held
apply.lock is StartError::AlreadyRunning with nothing touched. The runner
installs its SIGINT forwarder before writing apply.pid.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Eleven commands (detect, deploy, update, repair, destroy-incomplete,
unlock, follow, cancel, onboard, boundary, admin bundle) that re-run
detection server-side before acting, start detached OpenTofu runs off
the async runtime, and stream each run's log on aws-deploy-progress
with one follower per deployment. State keys from the frontend are
parsed back into a Deployment; anything state_key() can't produce is
refused. The admin bundle is written into a folder picked from Rust and
never overwrites unrelated files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Followers hold their first event until the command returns its op id; a
re-follow replays the whole log. Unlock re-reads the lock and deletes it
only if its ETag is unchanged and it is still stale. A launch claims its
work folder so overlapping launches and follows are refused. Repair,
update and destroy refuse under a held lock; destroy's guard is a tested
helper; the boundary deploy refuses a boundary that exists without state
here; the state read for a VPC id is capped while streaming.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The work-folder registry is a pure Slots<T> whose launch, follow and
clean-up transitions each run under one lock acquisition, with tests;
HeadObject errors name the action as s3:GetObject (HeadObject).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
EnvironmentsPanel under each AWS connection: region select and detection,
environment rows with state, lock owner/age, module version and the
detected actions (Join, Update, Repair, Destroy, Follow, Unlock), typed
confirmations for Destroy and Unlock, self-onboarding for environment
admins and the onboarding bundle for everyone else (R15).
DeployEnvironmentModal (name rule mirrored from Rust, D4 networks with
cost lines, boundary-first step), AdminBundleModal (D5 bundles) and
DeployProgress, whose hook subscribes before invoking and buffers events
by state key until the op id is known.

awsErrorMessage moves to lib/aws-error; PolicyFiles to its own module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rows and actions belong to the scan they came from: a region change hides
the old rows, detection in flight disables actions, and every action and
confirmation targets scan.region. Runs, pending state and errors are keyed
by region as well as state key. A boundary run that fails in the apply now
offers the boundary bundle with its summary; waiting-for-admin shows only
after a bundle is saved; clearer Destroy copy. Tests for each, and no act()
warnings in the new tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- s3:TagResource with s3:CreateBucket: a create with tags authorizes it
  (same request-tag and in-account conditions).
- ec2:CreateSecurityGroup on the VPC: EC2 shows the VPC no request tags,
  so that side is unconditioned; the group itself still needs the tag.
- ec2:DisassociateAddress on the literal */* only (${*}/${*}): destroying
  the NAT gateway's address disassociates it after the association is
  gone. The evaluator learns ${*}.

Checks, MODULE_CALLS and aws/README.md (including the Environments
section and the phase-4 live results) change with them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tofu destroy leaves the state object behind, empty, so a destroyed
environment kept scanning as Incomplete and a removed onboarding as
Joined. Detection reads small state objects and drops those that manage
no resources. Also lets a test scan and create a throwaway state bucket
(scan_in_bucket, ensure_named_state_bucket).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The environment live tests apply the real modules through the real
detached runner (the built app binary) as a role holding only the admin
files: state bucket, public deploy, scan, self-onboard, update with no
changes, destroy; the conditional lock delete (412 on a stale ETag); and
a private-nat deploy. A guard destroys what is left as the account admin
and deletes the throwaway bucket. CLAUDE.md and the spec describe the
OpenTofu design (D2-D10) in place of CloudFormation; phase 4 is done.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Live tests never delete a boundary or bucket they did not create: the
  boundary goes only through its module's destroy (no direct-delete
  fallback), ownership is taken only on NoSuchEntity, and the bucket is
  deleted only if this run created it.
- Detection reads at most 200 state objects of up to 2 KiB to find empty
  ones, and a failed read keeps the entry instead of failing the scan.
- Docs: DisassociateAddress on a live association is expected-denied, not
  verified (probe listed); the e2e admin role carries no boundary, and
  why; the spec's tenancy text follows D2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Critical
- Owner key is sha256("<region>:<aws:userid>") (12 hex): IAM role names are
  account-global and environments regional, so the same user and env name
  in two regions no longer collide. Every caller, the onboarding module's
  owner_key validation, docs and a two-region test.

Important
- A failed onboarding is Incomplete, not Joined: Joined needs the task role
  and, where the bucket is readable, the state's new onboarding_complete
  output (depends_on every resource); a failed or errored local run or a
  stale lock is Incomplete. Join stays as the retry; Follow, Unlock (own
  key only, stale, conditional delete) and Push saved state work on the
  caller's own onboarding.
- errored.tfstate: the runner logs an Error line naming it, follow's
  summary says so, materialize refuses while it exists, and the new
  push_aws_saved_state command runs `tofu state push errored.tfstate`
  through the same runner (file set aside once pushed); UI on the row.
- Bundle README: the bucket is created with the component tag in
  --create-bucket-configuration (the policy's RequestTag condition), plus
  the TLS-only bucket policy; the boundary bundle names an account admin
  WITHOUT the environment-admin files.
- Destroy-incomplete refuses while any users/<env>/ state remains.
- existing network: data.aws_subnet lookups and a precondition (same VPC,
  different AZs) on the task security group.

Minor
- The TLS statement is added to an existing bucket policy, never put over it.
- Deploy progress is emitted to the main window only.
- materialize checks and writes under start.lock.
- UI: managed-elsewhere label and explanation, Incomplete explained,
  Repair/Destroy disabled under a held lock ("Unlock first").
- Cargo.toml libc comment; README: Cancel unavailable on Windows, session.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test(aws): live lifecycle checks Joined from the completion output and NotJoined after removal
Secret Scan / scan (push) Successful in 6s
Build App (Preview) / compute-version (pull_request) Successful in 5s
Secret Scan / scan (pull_request) Successful in 6s
Build App (Preview) / create-release (pull_request) Successful in 5s
Build App (Preview) / build-macos (pull_request) Successful in 5m28s
Build App (Preview) / test (pull_request) Successful in 7m16s
Build App (Preview) / build-windows (pull_request) Successful in 10m8s
Build App (Preview) / build-linux (pull_request) Successful in 14m12s
Build App (Preview) / prune-previews (pull_request) Successful in 4s
be6f4f05a4
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jknapp merged commit 2d094b4748 into main 2026-11-15 15:39:32 +00:00
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: CyberCoveLLC/Triple-C#81