AWS host phase 4: environments and onboarding with OpenTofu #81

Merged
jknapp merged 24 commits from feat/aws-environments into main 2026-11-15 15:39:32 +00:00
24 Commits
Author SHA1 Message Date
shadowdaoandClaude Opus 5.5 be6f4f05a4 test(aws): live lifecycle checks Joined from the completion output and NotJoined after removal
Secret Scan / scan (push) Successful in 6s
Build App (Preview) / compute-version (pull_request) Successful in 5s
Secret Scan / scan (pull_request) Successful in 6s
Build App (Preview) / create-release (pull_request) Successful in 5s
Build App (Preview) / build-macos (pull_request) Successful in 5m28s
Build App (Preview) / test (pull_request) Successful in 7m16s
Build App (Preview) / build-windows (pull_request) Successful in 10m8s
Build App (Preview) / build-linux (pull_request) Successful in 14m12s
Build App (Preview) / prune-previews (pull_request) Successful in 4s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 00:06:26 -08:00
shadowdaoandClaude Opus 5.5 6911b9de27 fix(aws): final review round for the environment stacks
Critical
- Owner key is sha256("<region>:<aws:userid>") (12 hex): IAM role names are
  account-global and environments regional, so the same user and env name
  in two regions no longer collide. Every caller, the onboarding module's
  owner_key validation, docs and a two-region test.

Important
- A failed onboarding is Incomplete, not Joined: Joined needs the task role
  and, where the bucket is readable, the state's new onboarding_complete
  output (depends_on every resource); a failed or errored local run or a
  stale lock is Incomplete. Join stays as the retry; Follow, Unlock (own
  key only, stale, conditional delete) and Push saved state work on the
  caller's own onboarding.
- errored.tfstate: the runner logs an Error line naming it, follow's
  summary says so, materialize refuses while it exists, and the new
  push_aws_saved_state command runs `tofu state push errored.tfstate`
  through the same runner (file set aside once pushed); UI on the row.
- Bundle README: the bucket is created with the component tag in
  --create-bucket-configuration (the policy's RequestTag condition), plus
  the TLS-only bucket policy; the boundary bundle names an account admin
  WITHOUT the environment-admin files.
- Destroy-incomplete refuses while any users/<env>/ state remains.
- existing network: data.aws_subnet lookups and a precondition (same VPC,
  different AZs) on the task security group.

Minor
- The TLS statement is added to an existing bucket policy, never put over it.
- Deploy progress is emitted to the main window only.
- materialize checks and writes under start.lock.
- UI: managed-elsewhere label and explanation, Incomplete explained,
  Repair/Destroy disabled under a held lock ("Unlock first").
- Cargo.toml libc comment; README: Cancel unavailable on Windows, session.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 00:01:32 -08:00
shadowdaoandClaude Opus 5.5 0b3b100a91 fix(aws): review round for the live environment tests and detection
- Live tests never delete a boundary or bucket they did not create: the
  boundary goes only through its module's destroy (no direct-delete
  fallback), ownership is taken only on NoSuchEntity, and the bucket is
  deleted only if this run created it.
- Detection reads at most 200 state objects of up to 2 KiB to find empty
  ones, and a failed read keeps the entry instead of failing the scan.
- Docs: DisassociateAddress on a live association is expected-denied, not
  verified (probe listed); the e2e admin role carries no boundary, and
  why; the spec's tenancy text follows D2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 23:38:21 -08:00
shadowdaoandClaude Opus 5.5 ba88323feb test(aws): live environment lifecycle with OpenTofu as an environment admin; docs
The environment live tests apply the real modules through the real
detached runner (the built app binary) as a role holding only the admin
files: state bucket, public deploy, scan, self-onboard, update with no
changes, destroy; the conditional lock delete (412 on a stale ETag); and
a private-nat deploy. A guard destroys what is left as the account admin
and deletes the throwaway bucket. CLAUDE.md and the spec describe the
OpenTofu design (D2-D10) in place of CloudFormation; phase 4 is done.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 23:31:49 -08:00
shadowdaoandClaude Opus 5.5 5f38e0d0d2 fix(aws): an empty state is no deployment
tofu destroy leaves the state object behind, empty, so a destroyed
environment kept scanning as Incomplete and a removed onboarding as
Joined. Detection reads small state objects and drops those that manage
no resources. Also lets a test scan and create a throwaway state bucket
(scan_in_bucket, ensure_named_state_bucket).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 23:31:49 -08:00
shadowdaoandClaude Opus 5.5 2f5305eeb1 fix(aws): admin policy gaps found by the live OpenTofu run
- s3:TagResource with s3:CreateBucket: a create with tags authorizes it
  (same request-tag and in-account conditions).
- ec2:CreateSecurityGroup on the VPC: EC2 shows the VPC no request tags,
  so that side is unconditioned; the group itself still needs the tag.
- ec2:DisassociateAddress on the literal */* only (${*}/${*}): destroying
  the NAT gateway's address disassociates it after the association is
  gone. The evaluator learns ${*}.

Checks, MODULE_CALLS and aws/README.md (including the Environments
section and the phase-4 live results) change with them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 23:31:49 -08:00
shadowdaoandClaude Opus 5.5 bc23fc2dee fix(aws): environments panel acts on the scanned region; runs keyed per region
Rows and actions belong to the scan they came from: a region change hides
the old rows, detection in flight disables actions, and every action and
confirmation targets scan.region. Runs, pending state and errors are keyed
by region as well as state key. A boundary run that fails in the apply now
offers the boundary bundle with its summary; waiting-for-admin shows only
after a bundle is saved; clearer Destroy copy. Tests for each, and no act()
warnings in the new tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 22:56:36 -08:00
shadowdaoandClaude Opus 5.5 cf9b5eab3a feat(aws): environments panel — detect, deploy, update, repair, join
EnvironmentsPanel under each AWS connection: region select and detection,
environment rows with state, lock owner/age, module version and the
detected actions (Join, Update, Repair, Destroy, Follow, Unlock), typed
confirmations for Destroy and Unlock, self-onboarding for environment
admins and the onboarding bundle for everyone else (R15).
DeployEnvironmentModal (name rule mirrored from Rust, D4 networks with
cost lines, boundary-first step), AdminBundleModal (D5 bundles) and
DeployProgress, whose hook subscribes before invoking and buffers events
by state key until the op id is known.

awsErrorMessage moves to lib/aws-error; PolicyFiles to its own module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 22:50:23 -08:00
shadowdaoandClaude Opus 5.5 a8b83ca63a fix(aws): follow checks the launch slot under the same lock it installs in
The work-folder registry is a pure Slots<T> whose launch, follow and
clean-up transitions each run under one lock acquisition, with tests;
HeadObject errors name the action as s3:GetObject (HeadObject).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 22:41:29 -08:00
shadowdaoandClaude Opus 5.5 964ab993c8 fix(aws): gate progress behind the op id, conditional unlock, launch slots
Followers hold their first event until the command returns its op id; a
re-follow replays the whole log. Unlock re-reads the lock and deletes it
only if its ETag is unchanged and it is still stale. A launch claims its
work folder so overlapping launches and follows are refused. Repair,
update and destroy refuse under a held lock; destroy's guard is a tested
helper; the boundary deploy refuses a boundary that exists without state
here; the state read for a VPC id is capped while streaming.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 22:38:18 -08:00
shadowdaoandClaude Opus 5.5 3324e3d6fe feat(aws): environment commands with streamed OpenTofu progress
Eleven commands (detect, deploy, update, repair, destroy-incomplete,
unlock, follow, cancel, onboard, boundary, admin bundle) that re-run
detection server-side before acting, start detached OpenTofu runs off
the async runtime, and stream each run's log on aws-deploy-progress
with one follower per deployment. State keys from the frontend are
parsed back into a Deployment; anything state_key() can't produce is
refused. The admin bundle is written into a folder picked from Rust and
never overwrites unrelated files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 22:29:57 -08:00
shadowdaoandClaude Opus 5.5 97e6a3e15a fix(aws): never offer Destroy on a working environment; surface credential 403s; keep bucket tags
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 22:20:34 -08:00
shadowdaoandClaude Opus 5.5 f26388969b feat(aws): state bucket, and detection of environments, locks and onboarding
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 22:16:05 -08:00
shadowdaoandClaude Opus 5.5 b7cdf02859 fix(aws): never kill a runner that holds the lock; serialise starts
start_detached stops waiting once the runner holds apply.lock and returns
Started { pid: Option } (no PID: Cancel unavailable), killing only a
runner that never took the lock nor finished. The check, clear, spawn and
wait run under an exclusive start.lock, and a concurrent start or a held
apply.lock is StartError::AlreadyRunning with nothing touched. The runner
installs its SIGINT forwarder before writing apply.pid.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 22:05:31 -08:00
shadowdaoandClaude Opus 5.5 9ad8b99d3c fix(aws): runner liveness by lock, Cancel via the runner, clean tofu env
The runner holds an exclusive lock on apply.lock for its life and writes
its own PID, so a reused PID is never a running deploy; start_detached
waits for the lock (or kills the spawn). Cancel signals the runner alone,
which forwards SIGINT to tofu, so an AppImage's FUSE server in the same
group is never hit. tofu gets exactly child_env's variables with any
AppImage launcher changes undone, an app-owned CLI config, endpoint URLs
from AWS config ignored, and no AWS config files for keys connections.
Keys connections without keys are an error; init runs with -json so its
errors become the summary; Windows allowlist and AWS_CA_BUNDLE added.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 22:00:41 -08:00
shadowdaoandClaude Opus 5.5 a000d64577 feat(aws): pinned OpenTofu download and a detached apply runner
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 21:46:43 -08:00
shadowdaoandClaude Opus 5.5 02c8616512 fix(aws): pin the state bucket to the account, tag-scope network parents, drop v1-unused grants
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 21:34:57 -08:00
shadowdaoandClaude Opus 5.5 7dbf568696 feat(aws): admin policies cover the Terraform modules; drop CloudFormation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 21:25:48 -08:00
shadowdaoandClaude Opus 5.5 31c0a918e6 fix(aws): EC2-safe SG descriptions, deny other EFS principals, format-check the record
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 21:08:32 -08:00
shadowdaoandClaude Opus 5.5 f061ddb9a4 feat(aws): onboarding and permissions-boundary Terraform modules
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 21:03:20 -08:00
shadowdaoandClaude Opus 5.5 6192b7c1e8 fix(aws): keep environment subnet AZs stable; state_key takes a Deployment
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 20:54:42 -08:00
shadowdaoandClaude Opus 5.5 3583bee31e feat(aws): environment Terraform module and naming
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 20:50:55 -08:00
shadowdaoandClaude Opus 5.5 fe7642ee2c docs: phase 4 plan switches to OpenTofu with S3 state
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 20:43:51 -08:00
shadowdaoandClaude Opus 5.5 f924040f9e docs: phase 4 plan (environment + onboarding stacks)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 20:34:00 -08:00