Take the review: cache the stopped probe, and never let it cost an answer
Secret Scan / scan (push) Successful in 6s
Build App (Preview) / compute-version (pull_request) Successful in 5s
Secret Scan / scan (pull_request) Successful in 4s
Build App (Preview) / create-release (pull_request) Successful in 1s
Build App (Preview) / build-macos (pull_request) Successful in 2m58s
Build App (Preview) / build-linux (pull_request) Successful in 4m43s
Build App (Preview) / build-windows (pull_request) Successful in 5m9s
Build App (Preview) / prune-previews (pull_request) Successful in 2s

Six findings, all real. The one that mattered: `getContainerStaleness` is
called from a `useEffect` that fires whenever the container settles, so
merely opening a stopped project's Overview now committed its whole writable
layer — 44 s on a real project, against ~3 s for the snapshot probe it
replaced. Shipping that would have traded one bad banner for a bad page.

A stopped container's writable layer cannot change, so the probe is exactly
cacheable: `STOPPED_MANIFEST_CACHE` keys on the container's `FinishedAt`,
which moves on every stop. Cold 2967 ms, warm 1 ms, measured. A live test
asserts the restart case as well as the hit, because a cache that failed to
invalidate would plan a migration against a filesystem the project no longer
has — verified by breaking the token and watching that assertion fail.

Skipping the probe for projects that are not stale looked like the cheaper
fix and is unsafe: the deltas would be empty while `probeSettled` stayed
true, and the migrate action in the project menu is not gated on the banner,
so the pre-flight would report nothing to copy while the backend was told to
copy nothing. That is the hazard `canMigrate`'s comment already warns about.
Not done, and written down so it is not tried again.

Also from the review:

- A failed commit no longer costs an answer the snapshot could have given.
  Before this feature a stopped project read its snapshot directly, so
  surfacing this error would have made the banner worse than it was — and
  the failure modes are where the fallback earns its keep: a full disk (the
  commit allocates the whole layer, the snapshot probe allocates nothing)
  and a 409 from a concurrent claim.
- The probe no longer commits while the project is claimed. The collision is
  not symmetric: the probe losing is a retryable `probe_error`, but
  `start_project_container` removes the old container with a hard `?`, so a
  remove that raced a commit would fail the user's Start with an opaque
  error. `stopped_probe_policy` reads `project_lock::held` and probes the
  snapshot instead, or defers with a message that says so.
- The cleanup-failure warning claimed the next probe of the same container
  would reclaim the leftover. Unique names made that false the moment they
  landed; it is `reap_probe_images` that collects it.
- The TS binding still called the command read-only, which is how the
  auto-refresh got added in the first place.
- CLAUDE.md still documented the stable `triple-c-probe-{cid}:latest` name
  this PR removed as unsafe.

548 unit tests, 752 frontend tests, 4 live-Docker tests. Clippy unchanged at
44 warnings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RSaoDLovVV2wmH4H8VVxz
This commit is contained in:
2026-09-10 20:48:10 -07:00
co-authored by Claude Opus 5
parent 307ea07409
commit 95a78fe9a3
4 changed files with 308 additions and 7 deletions
@@ -142,6 +142,45 @@ fn pick_probe_source(container_running: Option<bool>, snapshot_exists: bool) ->
}
}
/// Reported as `probe_error` when another operation owns the project and there
/// is no snapshot image to read instead. Deliberately not a claim about the
/// container: nothing is wrong with it, the answer is simply not safe to take
/// right now. See [`stopped_probe_policy`].
const PROJECT_BUSY: &str = "Another operation is running on this project, so its contents could not be inspected. Try again once it finishes.";
/// What to do about a stopped container, whose probe is the expensive one: it
/// commits the writable layer before it can read anything.
#[derive(Debug, PartialEq, Eq)]
enum StoppedProbe {
/// Commit and probe. The current answer, and the default.
Commit,
/// Probe the snapshot image instead. Less current — it lags the container by
/// everything installed since the last commit — but it allocates nothing and
/// touches nothing, which is what makes it the right answer while another
/// operation owns the container.
SnapshotInstead,
/// Report rather than guess.
Defer,
}
/// Pick what to do about a stopped container.
///
/// **Never commits while the project is claimed.** `get_container_staleness`
/// takes no [`crate::project_lock`] claim of its own, by design, so a commit
/// here can overlap a Recreate or Reset — and the collision is not symmetric.
/// The probe losing is harmless: a surfaced `probe_error` the user retries. The
/// *recreate* losing is not, because `start_project_container` removes the old
/// container with a hard `?`, so a non-404 from a remove that raced this commit
/// fails the whole Start with an opaque "Failed to remove container". Reading
/// the claim costs nothing and takes that failure off the table.
fn stopped_probe_policy(project_is_busy: bool, snapshot_exists: bool) -> StoppedProbe {
match (project_is_busy, snapshot_exists) {
(false, _) => StoppedProbe::Commit,
(true, true) => StoppedProbe::SnapshotInstead,
(true, false) => StoppedProbe::Defer,
}
}
/// Runs two filesystem probes (~3 s each) and is therefore meant to be called
/// on demand, not polled.
///
@@ -220,7 +259,44 @@ pub async fn get_container_staleness(
) {
(ProbeSource::RunningContainer, Some(id)) => mig::manifest_from_container(id).await,
(ProbeSource::StoppedContainer, Some(id)) => {
mig::manifest_from_stopped_container(id).await
let busy = crate::project_lock::held(&project_id).is_some();
match stopped_probe_policy(busy, snapshot_exists) {
StoppedProbe::Commit => {
match mig::manifest_from_stopped_container_cached(id).await {
Ok(m) => Ok(m),
// **Never let a failed commit cost an answer the
// snapshot could have given.** Before stopped
// containers were readable at all, a stopped project
// fell straight through to its snapshot, so surfacing
// this error where the snapshot exists would make the
// banner *worse* than it was — and the ways this fails
// are the ones where the fallback matters most: a full
// disk (the commit has to allocate the whole writable
// layer; the snapshot probe allocates nothing) and a
// 409 from an operation that claimed the project after
// the check above.
Err(e) if snapshot_exists => {
log::warn!(
"Probing the stopped container for project {} failed ({}) — \
falling back to its snapshot image, which may lag it",
project_id,
e
);
mig::manifest_from_image(&snapshot_image).await
}
Err(e) => Err(e),
}
}
StoppedProbe::SnapshotInstead => {
log::info!(
"Project {} is claimed by another operation — probing its snapshot image \
rather than committing the container",
project_id
);
mig::manifest_from_image(&snapshot_image).await
}
StoppedProbe::Defer => Err(PROJECT_BUSY.to_string()),
}
}
(ProbeSource::Snapshot, _) => mig::manifest_from_image(&snapshot_image).await,
// `container_running` is `Some` exactly when `container_id` is, so the
@@ -2073,6 +2149,23 @@ mod tests {
assert_eq!(pick_probe_source(None, false), ProbeSource::Nothing);
}
#[test]
fn a_stopped_container_is_committed_only_when_nothing_else_owns_the_project() {
assert_eq!(stopped_probe_policy(false, false), StoppedProbe::Commit);
assert_eq!(stopped_probe_policy(false, true), StoppedProbe::Commit);
}
#[test]
fn a_busy_project_falls_back_rather_than_racing_a_recreate() {
// The snapshot lags, but a stale answer beats failing someone's Start.
assert_eq!(
stopped_probe_policy(true, true),
StoppedProbe::SnapshotInstead
);
// Nothing to fall back to: say so instead of committing anyway.
assert_eq!(stopped_probe_policy(true, false), StoppedProbe::Defer);
}
#[test]
fn byte_sizes_read_the_way_a_disk_warning_should() {
assert_eq!(human_bytes(512), "512 B");