Address review findings: durability, stale container ids, honest toasts
Secret Scan / scan (push) Successful in 4s
Build App (Preview) / compute-version (pull_request) Successful in 3s
Secret Scan / scan (pull_request) Successful in 6s
Build App (Preview) / create-release (pull_request) Successful in 2s
Build App (Preview) / build-macos (pull_request) Successful in 2m38s
Build App (Preview) / build-windows (pull_request) Successful in 6m18s
Build App (Preview) / build-linux (pull_request) Successful in 7m48s
Build App (Preview) / prune-previews (pull_request) Successful in 1s
Secret Scan / scan (push) Successful in 4s
Build App (Preview) / compute-version (pull_request) Successful in 3s
Secret Scan / scan (pull_request) Successful in 6s
Build App (Preview) / create-release (pull_request) Successful in 2s
Build App (Preview) / build-macos (pull_request) Successful in 2m38s
Build App (Preview) / build-windows (pull_request) Successful in 6m18s
Build App (Preview) / build-linux (pull_request) Successful in 7m48s
Build App (Preview) / prune-previews (pull_request) Successful in 1s
An Opus review of the previous commit found several real gaps: - pending_cleanup::save used plain write-temp-then-rename, unlike migration_store's fsync'd write it claimed to mirror — a crash in that window left a truncated record that list() would skip forever, silently reproducing the exact bug this module exists to fix. Now matches migration_store's File::create/write_all/sync_all/rename/sync_dir shape, and the tests exercise the real save/list/clear functions against a temp dir instead of re-implementing their bodies inline. - remove_project and rebuild_project_container only ever looked at project.container_id, unlike every other container-destroying path in the codebase, which falls back to find_existing_container for exactly this race (a crash between creating a container and persisting its id). A miss here left a container that then blocked every subsequent volume removal with a 409, forever. Both now resolve the same way the rest of the codebase does, and record the container by its deterministic name rather than its id so a retry still has something that resolves. - remove_project's toast promised an automatic retry unconditionally, even when writing the pending-cleanup record itself failed (the one case where nothing will actually retry). ProjectRemovalReport now carries retry_scheduled, and the UI is honest about which case it's in. - remove_volumes_by_name now retries once after a short delay on a 409, since Docker releasing a volume's mount reference right after its container is removed is not always instantaneous, and this is exactly the sequence remove_project runs. - rebuild_project_container (Reset) returns ProjectResetOutcome so the UI can warn when Reset could not fully clear a project's volumes, instead of only logging it — the new container silently reuses old data otherwise, which is what Reset promises not to do. - retry_pending_cleanup_logged escalates a record's log level after it has failed for a week, since recorded_at was otherwise write-only. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FGjXq6fqtAFHdbhk4f3PfZ
This commit is contained in:
@@ -3585,7 +3585,7 @@ pub async fn remove_volumes_by_name(names: &[String]) -> Vec<String> {
|
||||
|
||||
let mut leftover = Vec::new();
|
||||
for vol in names {
|
||||
match docker.remove_volume(vol, None).await {
|
||||
match remove_one_volume_with_retry(&docker, vol).await {
|
||||
Ok(_) => log::info!("Removed volume {}", vol),
|
||||
Err(bollard::errors::Error::DockerResponseServerError {
|
||||
status_code: 404, ..
|
||||
@@ -3599,6 +3599,28 @@ pub async fn remove_volumes_by_name(names: &[String]) -> Vec<String> {
|
||||
leftover
|
||||
}
|
||||
|
||||
/// Remove one volume, retrying once after a short delay on a 409 ("volume is
|
||||
/// in use"). Docker releasing a volume's mount reference after the container
|
||||
/// using it is removed is not always instantaneous, so the very first call
|
||||
/// site of this — `remove_project`, whose container removal lands
|
||||
/// immediately before its volume removal — could otherwise turn an ordinary
|
||||
/// race into a permanent pending-cleanup record and an alarming toast for
|
||||
/// something that would have cleared itself half a second later.
|
||||
async fn remove_one_volume_with_retry(
|
||||
docker: &bollard::Docker,
|
||||
name: &str,
|
||||
) -> Result<(), bollard::errors::Error> {
|
||||
match docker.remove_volume(name, None).await {
|
||||
Err(bollard::errors::Error::DockerResponseServerError {
|
||||
status_code: 409, ..
|
||||
}) => {
|
||||
tokio::time::sleep(std::time::Duration::from_millis(500)).await;
|
||||
docker.remove_volume(name, None).await
|
||||
}
|
||||
other => other,
|
||||
}
|
||||
}
|
||||
|
||||
/// Check whether the existing container's configuration still matches the
|
||||
/// current project settings. Returns `true` when the container must be
|
||||
/// recreated (mounts or env vars differ).
|
||||
|
||||
Reference in New Issue
Block a user