Fix review findings: secrets in snapshots, URL spoofing, migration data loss
Adversarial review of the branch produced findings across four areas. This addresses them, plus the Windows CI environment. Secrets. commit_container_snapshot baked the container's full env into the per-project snapshot image, so the shared OAuth token — and the AWS keys, git token and gateway master key — outlived revocation and were readable via docker inspect. Verified against Engine 29.6 that a commit body's config merges over the container's: keys cannot be dropped but can be overwritten, so all of them now commit as KEY=. clear_claude_token additionally rewrites images from earlier builds and reports honestly when a tag could not be rewritten. The recommendation to move the token out of env entirely was not taken, with reasoning: apiKeyHelper is a different auth method that outranks CLAUDE_CODE_OAUTH_TOKEN rather than a transport for it, and no file-based delivery exists. The durable exposure — the image — is what is closed here. Separately noted, not fixed: entrypoint.sh captures the token into the scheduler's .env inside the persisted volume. URL spoofing. Three call sites reached openUrl with container-controlled strings, one of which the review missed (the WebLinksAddon handler). The sign-in URL was scraped from container output with a longest-match tie-break and no userinfo check, so claude.ai@evil.tld rendered as "claude.ai…" in a truncating element. There is now one sanitizer in front of every sink — scheme allowlist, no userinfo, C0/C1 and quote rejection, host allowlist for the sign-in case, first-match — and the origin renders un-truncated. The toast is keyed so a changed URL remounts, closing a bait-and-switch where the user read one URL and clicked another. Migration. The rollback pin was best-effort: a tag failure was logged and the migration continued past remove_container, after which the final commit overwrote the only copy of the old system layer. It now aborts before anything destructive and reads the tag back. /var was destroyed while the ordinary recreate path preserves it — making the "safe" alternative to Reset more destructive than Reset's alternative; data-bearing subtrees are now detected and disclosed in the pre-flight rather than copied, since tarring a live database onto a different base's packages is a corruption risk. resume_migration now verifies the migration-state label instead of reporting success for a container that never swapped. dismiss actually resolves the record rather than leaving the feature permanently refusing to migrate. Start and Reset are guarded while a migration is live. Lifecycle. The gateway no longer publishes on 0.0.0.0 — bind address and advertised URL are derived together so they cannot drift. Disabling it now stops it. App exit runs teardown concurrently under a budget with a visible shutting-down state instead of blocking for minutes. Auto-starts retry when Docker is not up yet, and the polling-recovery path now reconciles, so interrupted migrations are still recovered. Auth-bridge forwards are capped, closing a container-driven fd exhaustion. Windows CI. build-windows failed on this branch with "linker link.exe not found". The runner had no MSVC build tools and the workflow assumed a hand-provisioned machine, so a bare runner registers, accepts jobs and fails at link time after downloading the whole crate graph. The job now installs the VC++ workload when vswhere cannot find it, matching how it already conditionally installs Rust and Node. 192 Rust tests, 274 frontend tests, both builds clean, zero warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -23,11 +23,20 @@
|
||||
//! * The flow needs a way to deliver the pasted code, hence
|
||||
//! [`submit_claude_token_code`] and the stdin channel below. Without it the
|
||||
//! command would simply sit at the prompt until it timed out.
|
||||
//! * [`crate::auth_bridge`] is *not* required for this particular command,
|
||||
//! since there is no container-local callback to reach. It is still enabled
|
||||
//! for the duration (and restored afterwards) as designed: it costs nothing
|
||||
//! here and keeps the flow working if a future CLI version, or the plain
|
||||
//! `claude login` path, goes back to a loopback redirect.
|
||||
//! * [`crate::auth_bridge`] is **not involved**. There is no container-local
|
||||
//! listener to reach, so there is nothing for it to bridge.
|
||||
//!
|
||||
//! An earlier version turned the bridge on for the duration "in case a
|
||||
//! future CLI version goes back to a loopback redirect", and turned it off
|
||||
//! again afterwards. That was wrong twice over. The bridge flag is
|
||||
//! *persisted* to `projects.json`, and the restore only ran if the future
|
||||
//! completed — so a force-quit, kill or panic inside the 15-minute
|
||||
//! [`SETUP_TIMEOUT`] left it latched on, and the app re-armed an
|
||||
//! unauthenticated loopback port mirror on every subsequent launch, for a
|
||||
//! project whose owner had never opted in. And it bought nothing: the
|
||||
//! speculative future it defended against would need code changes here
|
||||
//! anyway. The bridge remains available as the per-project setting it always
|
||||
//! was; this command does not touch it.
|
||||
//!
|
||||
//! ## Handling of the token itself
|
||||
//!
|
||||
@@ -66,10 +75,27 @@ const SETUP_TIMEOUT: Duration = Duration::from_secs(15 * 60);
|
||||
const TOKEN_PREFIX: &str = "sk-ant-oat01-";
|
||||
|
||||
/// Minimum number of body characters after [`TOKEN_PREFIX`] for a match to be
|
||||
/// believed. Real tokens run to ~90 characters; this is set well below that but
|
||||
/// far above anything prose would produce, so documentation-style decoys like
|
||||
/// `sk-ant-oat01-...` or `sk-ant-oat01-<your-token>` are rejected.
|
||||
const MIN_TOKEN_BODY: usize = 32;
|
||||
/// believed. Real `setup-token` credentials run to ~90 body characters.
|
||||
///
|
||||
/// This must be close to that real length, not merely "longer than prose".
|
||||
/// The earlier value of 32 accepted a *fragment* of a token — which is what a
|
||||
/// line wrap produces if `stty cols 200` fails and the pty falls back to 80
|
||||
/// columns, splitting the value across two lines. Two bad things follow from
|
||||
/// accepting one:
|
||||
///
|
||||
/// * the fragment gets stored as if it were the credential, so every
|
||||
/// container authenticates with a token that cannot work, and the failure
|
||||
/// surfaces far from its cause;
|
||||
/// * [`SecretRedactor`] masks the leading fragment because it carries the
|
||||
/// `sk-ant-` marker, but the *tail* on the next line carries no marker and
|
||||
/// is emitted to the UI in clear.
|
||||
///
|
||||
/// Set below the real length by enough margin to survive a modest change in
|
||||
/// token format, and far above any fragment an 80-column wrap can produce (the
|
||||
/// prefix alone eats 13 columns, so the longest possible first line fragment is
|
||||
/// ~67). Rejecting a real-but-shorter token is a loud, recoverable failure —
|
||||
/// "printed no recognisable token" — whereas accepting a fragment is silent.
|
||||
const MIN_TOKEN_BODY: usize = 80;
|
||||
|
||||
/// Redaction is deliberately broader than extraction: anything shaped like an
|
||||
/// Anthropic credential is masked on its way to the UI, not just `oat01` ones.
|
||||
@@ -361,6 +387,20 @@ fn strip_ansi_prefix(bytes: &[u8]) -> (String, usize) {
|
||||
(out, i)
|
||||
}
|
||||
|
||||
/// Cap on the bytes [`AnsiStripper`] will hold waiting for a control sequence
|
||||
/// to terminate.
|
||||
///
|
||||
/// [`strip_ansi_prefix`] stops at the first *incomplete* sequence and the
|
||||
/// remainder is carried to the next chunk — which is correct while the
|
||||
/// sequence really is going to end, and unbounded when it never does. A single
|
||||
/// `ESC ]` with no BEL and no ST swallows everything the container prints for
|
||||
/// the rest of the 15-minute [`SETUP_TIMEOUT`], and the buffer grows with it;
|
||||
/// `transcript` and [`SecretRedactor::pending`] are both already capped, so
|
||||
/// this was the one remaining way to make the flow eat memory. Generous enough
|
||||
/// that no legitimate sequence comes close (OSC 8 hyperlinks are the longest
|
||||
/// thing Claude Code emits, in the low hundreds of bytes).
|
||||
const MAX_ANSI_CARRY: usize = 64 * 1024;
|
||||
|
||||
/// Stateful wrapper around [`strip_ansi_prefix`] that carries an incomplete
|
||||
/// trailing sequence over to the next chunk.
|
||||
#[derive(Default)]
|
||||
@@ -371,8 +411,29 @@ struct AnsiStripper {
|
||||
impl AnsiStripper {
|
||||
fn push(&mut self, chunk: &[u8]) -> String {
|
||||
self.carry.extend_from_slice(chunk);
|
||||
let (out, consumed) = strip_ansi_prefix(&self.carry);
|
||||
let (mut out, consumed) = strip_ansi_prefix(&self.carry);
|
||||
self.carry.drain(..consumed);
|
||||
|
||||
// Past the cap the leading sequence is not going to terminate. Drop
|
||||
// its introducer and re-strip: the bytes behind it are then treated as
|
||||
// ordinary text rather than discarded, so a token hiding inside a
|
||||
// runaway OSC still reaches the parser. Dropping one byte per
|
||||
// overflowing chunk is enough — the cap can only be re-crossed by a
|
||||
// fresh chunk, which re-enters here.
|
||||
if self.carry.len() > MAX_ANSI_CARRY {
|
||||
log::warn!(
|
||||
"`claude setup-token` emitted an unterminated control sequence \
|
||||
longer than {} bytes — treating it as text",
|
||||
MAX_ANSI_CARRY
|
||||
);
|
||||
}
|
||||
while self.carry.len() > MAX_ANSI_CARRY {
|
||||
self.carry.drain(..1);
|
||||
let (more, consumed) = strip_ansi_prefix(&self.carry);
|
||||
self.carry.drain(..consumed);
|
||||
out.push_str(&more);
|
||||
}
|
||||
|
||||
out
|
||||
}
|
||||
}
|
||||
@@ -606,65 +667,23 @@ pub async fn acquire_claude_token(
|
||||
*cancel_slot().lock().await = Some(cancel_tx);
|
||||
}
|
||||
|
||||
let bridge_was_enabled = project.auth_bridge_enabled;
|
||||
// No auth-bridge elevation here — see the module docs. `setup-token` has no
|
||||
// container-local callback to bridge, and the flag that would enable one is
|
||||
// *persisted*, so a crash anywhere inside the 15-minute timeout used to
|
||||
// leave an unauthenticated port mirror armed for good.
|
||||
emit_progress(
|
||||
&app_handle,
|
||||
&project_id,
|
||||
"Running `claude setup-token` — sign in at the URL below, then submit the code it gives you.",
|
||||
);
|
||||
|
||||
let result = async {
|
||||
// See the module docs: 2.1.226's `setup-token` redirects to an
|
||||
// Anthropic-hosted callback, so no container-local listener needs
|
||||
// bridging. Enabled anyway, per design, to cover CLI versions and login
|
||||
// paths that do use a loopback redirect. Temporary elevation — the
|
||||
// prior setting is restored below whatever happens.
|
||||
if !bridge_was_enabled {
|
||||
state
|
||||
.projects_store
|
||||
.set_auth_bridge_enabled(&project_id, true)?;
|
||||
emit_progress(
|
||||
&app_handle,
|
||||
&project_id,
|
||||
"Auth bridge enabled for the duration of login.",
|
||||
);
|
||||
}
|
||||
// Called unconditionally, and idempotent: the flag may already have
|
||||
// been on while the poller was not running (e.g. enabled before start).
|
||||
state
|
||||
.auth_bridge
|
||||
.start(
|
||||
project_id.clone(),
|
||||
container_id.clone(),
|
||||
app_handle.clone(),
|
||||
state.projects_store.clone(),
|
||||
)
|
||||
.await;
|
||||
let result =
|
||||
run_setup_token(&app_handle, &project_id, &container_id, input_rx, cancel_rx).await;
|
||||
|
||||
emit_progress(
|
||||
&app_handle,
|
||||
&project_id,
|
||||
"Running `claude setup-token` — sign in at the URL below, then submit the code it gives you.",
|
||||
);
|
||||
|
||||
run_setup_token(&app_handle, &project_id, &container_id, input_rx, cancel_rx).await
|
||||
}
|
||||
.await;
|
||||
|
||||
// Release the flow, then restore the bridge — both unconditionally, so a
|
||||
// failed or cancelled login leaves nothing latched on.
|
||||
// Release the flow. Nothing else needs unwinding: this command changes no
|
||||
// persisted state until the token itself is stored, which is the point.
|
||||
*pending_input().lock().await = None;
|
||||
*cancel_slot().lock().await = None;
|
||||
if !bridge_was_enabled {
|
||||
// Stop the poller first: it awaits teardown, so host ports are provably
|
||||
// released before the flag goes back.
|
||||
state.auth_bridge.stop(&project_id).await;
|
||||
if let Err(e) = state
|
||||
.projects_store
|
||||
.set_auth_bridge_enabled(&project_id, false)
|
||||
{
|
||||
log::warn!(
|
||||
"Failed to restore the auth bridge setting for project {}: {}",
|
||||
project_id,
|
||||
e
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
let token = result?;
|
||||
secure::store_claude_oauth_token(&token)?;
|
||||
@@ -736,14 +755,69 @@ pub async fn has_claude_token() -> Result<bool, String> {
|
||||
Ok(secure::has_claude_oauth_token())
|
||||
}
|
||||
|
||||
/// Forget the shared Claude token. Containers keep the injected value until
|
||||
/// each is next started, at which point the rotation-id label mismatch forces a
|
||||
/// recreation that blanks the env var.
|
||||
/// What [`clear_claude_token`] managed to reach. The keychain entry is always
|
||||
/// gone by the time this is returned — the rest is about copies of the token
|
||||
/// that live outside it.
|
||||
#[derive(Debug, Default, serde::Serialize)]
|
||||
pub struct ClearTokenOutcome {
|
||||
/// Snapshot images that were holding the token and have been rewritten.
|
||||
pub snapshots_scrubbed: Vec<String>,
|
||||
/// Images still holding it, with the reason each could not be rewritten.
|
||||
/// Non-empty means the revocation is **incomplete** and the UI must say so.
|
||||
pub snapshots_failed: Vec<String>,
|
||||
/// Rewritten, but the pre-rewrite image object could not be deleted because
|
||||
/// a container is still running off it. Worth mentioning, not worth
|
||||
/// alarming about — see `SnapshotScrubReport::superseded_retained`.
|
||||
pub snapshots_superseded: Vec<String>,
|
||||
/// Set when Docker could not be reached at all, so nothing is known about
|
||||
/// what is still on disk.
|
||||
pub docker_unavailable: Option<String>,
|
||||
}
|
||||
|
||||
/// Forget the shared Claude token.
|
||||
///
|
||||
/// Deleting the keychain entry is the easy half. The token also exists in two
|
||||
/// other places, and a "Revoke" button that leaves either of them behind is
|
||||
/// telling the user something untrue:
|
||||
///
|
||||
/// * **Running containers** hold it in their environment. That resolves
|
||||
/// itself: the rotation id in `triple-c.claude-token-version` no longer
|
||||
/// matches, so the next start recreates the container and
|
||||
/// `MANAGED_AUTH_KEYS` blanks the variable.
|
||||
/// * **Snapshot images** hold it in `Config.Env`, and nothing resolves that on
|
||||
/// its own — the image outlives every container built from it, and
|
||||
/// `docker image inspect` will keep printing a live ~1-year credential for
|
||||
/// as long as the image exists. New commits no longer bake it in (see
|
||||
/// [`crate::docker::container::commit_container_snapshot`]), but images
|
||||
/// committed by earlier builds have to be rewritten, which is what
|
||||
/// [`scrub_secrets_from_snapshots`] does here.
|
||||
///
|
||||
/// The keychain deletion is never rolled back if the scrub fails; a partially
|
||||
/// completed revocation is still better than none, and the outcome is reported
|
||||
/// so the UI can be explicit about what is left.
|
||||
#[tauri::command]
|
||||
pub async fn clear_claude_token() -> Result<(), String> {
|
||||
pub async fn clear_claude_token() -> Result<ClearTokenOutcome, String> {
|
||||
secure::delete_claude_oauth_token()?;
|
||||
log::info!("Cleared the shared Claude authentication token");
|
||||
Ok(())
|
||||
|
||||
let report = crate::docker::container::scrub_secrets_from_snapshots().await;
|
||||
if report.left_something_behind() {
|
||||
log::warn!(
|
||||
"Revoked the shared Claude token but {} snapshot image(s) may still contain it",
|
||||
report.failed.len()
|
||||
);
|
||||
}
|
||||
|
||||
Ok(ClearTokenOutcome {
|
||||
snapshots_scrubbed: report.scrubbed,
|
||||
snapshots_failed: report
|
||||
.failed
|
||||
.into_iter()
|
||||
.map(|(image, reason)| format!("{}: {}", image, reason))
|
||||
.collect(),
|
||||
snapshots_superseded: report.superseded_retained,
|
||||
docker_unavailable: report.unavailable,
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
@@ -938,5 +1012,72 @@ mod tests {
|
||||
visible.push_str(&s.push(b"1mb"));
|
||||
assert_eq!(visible, "ab");
|
||||
}
|
||||
}
|
||||
// ── Truncated credentials ────────────────────────────────────────────
|
||||
// `stty cols 200` runs before Claude Code starts precisely so the ~103
|
||||
// character token never wraps. If that fails, the pty falls back to 80
|
||||
// columns and the token arrives split across two lines. The old floor of
|
||||
// 32 body characters believed the first half.
|
||||
|
||||
#[test]
|
||||
fn a_token_shorter_than_a_real_one_is_rejected() {
|
||||
let fragment = format!("{}{}", TOKEN_PREFIX, "M".repeat(MIN_TOKEN_BODY - 1));
|
||||
assert_eq!(
|
||||
parse_setup_token(&format!("Your token: {}\n", fragment)),
|
||||
None
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_line_wrapped_token_yields_nothing_rather_than_half_a_credential() {
|
||||
let tok = token('N');
|
||||
// Column 12 is where `Your token: ` ends, so an 80-column pty breaks
|
||||
// the value 68 characters in.
|
||||
let wrapped = format!("Your token: {}\n{}\n", &tok[..68], &tok[68..]);
|
||||
assert_eq!(
|
||||
parse_setup_token(&wrapped),
|
||||
None,
|
||||
"a wrapped token must fail loudly, not be stored truncated"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_real_length_token_is_still_accepted() {
|
||||
// Guards the floor from being raised past what Anthropic actually mints.
|
||||
let tok = token('O');
|
||||
assert_eq!(tok.len() - TOKEN_PREFIX.len(), 90);
|
||||
assert!(90 > MIN_TOKEN_BODY);
|
||||
assert_eq!(parse_setup_token(&format!("{}\n", tok)), Some(tok));
|
||||
}
|
||||
|
||||
// ── Bounded buffering ────────────────────────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn an_unterminated_control_sequence_cannot_grow_the_carry_without_bound() {
|
||||
let mut s = AnsiStripper::default();
|
||||
// An OSC introducer with no BEL and no ST. `strip_ansi_prefix` cannot
|
||||
// know it has ended, so every byte after it is carried — for the whole
|
||||
// 15-minute timeout, if nothing caps it.
|
||||
let mut seen = s.push(b"\x1b]8;id=1;");
|
||||
for _ in 0..40 {
|
||||
seen.push_str(&s.push(&vec![b'A'; 4096]));
|
||||
}
|
||||
assert!(
|
||||
s.carry.len() <= MAX_ANSI_CARRY,
|
||||
"carry grew to {} bytes",
|
||||
s.carry.len()
|
||||
);
|
||||
assert!(!seen.is_empty(), "the withheld text must be released, not dropped");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_token_printed_after_a_runaway_sequence_is_still_found() {
|
||||
let tok = token('Q');
|
||||
let mut s = AnsiStripper::default();
|
||||
let mut seen = s.push(b"\x1b]8;id=1;");
|
||||
for _ in 0..40 {
|
||||
seen.push_str(&s.push(&vec![b'A'; 4096]));
|
||||
}
|
||||
seen.push_str(&s.push(format!("\nYour token: {}\n", tok).as_bytes()));
|
||||
assert_eq!(parse_setup_token(&seen), Some(tok));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -60,8 +60,8 @@ use crate::docker;
|
||||
use crate::docker::migration::{self as mig, Recovery};
|
||||
use crate::models::{
|
||||
ContainerStaleness, MigrationOptions, MigrationPhase, MigrationPlan, MigrationReport,
|
||||
MigrationState, PackageFailure, Project, ProjectStatus, MIGRATION_PHASE_AWAITING,
|
||||
MIGRATION_PHASE_INTERRUPTED,
|
||||
MigrationState, PackageFailure, Project, ProjectStatus, UnpreservedData,
|
||||
MIGRATION_PHASE_AWAITING, MIGRATION_PHASE_INTERRUPTED,
|
||||
};
|
||||
use crate::storage::migration_store;
|
||||
use crate::AppState;
|
||||
@@ -159,6 +159,20 @@ pub async fn get_container_staleness(
|
||||
&mig::bind_mount_exclusions(&project.paths),
|
||||
);
|
||||
out.outdated_package_count = mig::outdated_package_count(&from_manifest, &base_manifest);
|
||||
// Reported, never copied, and never silent — see `unpreserved_data`.
|
||||
out.unpreserved_data = mig::unpreserved_data(&from_manifest, &base_manifest);
|
||||
if !out.unpreserved_data.is_empty() {
|
||||
log::warn!(
|
||||
"Project {}: {} data-bearing subtree(s) under /var would not survive a migration: {}",
|
||||
project_id,
|
||||
out.unpreserved_data.len(),
|
||||
out.unpreserved_data
|
||||
.iter()
|
||||
.map(|d| d.path.as_str())
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ")
|
||||
);
|
||||
}
|
||||
|
||||
let (etc_only_container, etc_only_base) = mig::etc_deltas(&from_manifest, &base_manifest);
|
||||
if !etc_only_container.is_empty() || !etc_only_base.is_empty() {
|
||||
@@ -207,7 +221,12 @@ fn active_migrations() -> &'static std::sync::Mutex<std::collections::HashSet<St
|
||||
ACTIVE_MIGRATIONS.get_or_init(|| std::sync::Mutex::new(std::collections::HashSet::new()))
|
||||
}
|
||||
|
||||
fn is_migrating(project_id: &str) -> bool {
|
||||
/// Whether a migration for this project is running **in this process right
|
||||
/// now**. Every command that stops, removes or recreates the project's
|
||||
/// container has to consult it: the window between `remove_container` and the
|
||||
/// create that follows looks exactly like "no container", and an ordinary
|
||||
/// Start landing in it creates a second container under the same name.
|
||||
pub(crate) fn is_migrating(project_id: &str) -> bool {
|
||||
active_migrations()
|
||||
.lock()
|
||||
.unwrap_or_else(|e| e.into_inner())
|
||||
@@ -335,6 +354,22 @@ async fn fresh_migration(
|
||||
let npm_delta = mig::set_delta(&from_manifest.npm_global, &base_manifest.npm_global);
|
||||
let (missing_paths, _) = mig::missing_features(&from_manifest, &base_manifest);
|
||||
let payload_bytes = mig::verbatim_payload_bytes(&from_manifest, &verbatim);
|
||||
// Measured here as well as in the pre-flight probe, because this is the one
|
||||
// reading taken against the container that is about to be destroyed. It is
|
||||
// frozen into the plan so the finished report can still name it.
|
||||
let unpreserved = mig::unpreserved_data(&from_manifest, &base_manifest);
|
||||
if !unpreserved.is_empty() {
|
||||
log::warn!(
|
||||
"Project {}: migrating destroys {} data-bearing subtree(s) under /var: {}",
|
||||
project_id,
|
||||
unpreserved.len(),
|
||||
unpreserved
|
||||
.iter()
|
||||
.map(|d| format!("{} ({} files)", d.path, d.file_count))
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ")
|
||||
);
|
||||
}
|
||||
|
||||
emit_progress(&app_handle, &project_id, "Checking network and disk...");
|
||||
let env = mig::preflight_environment(&base_image).await?;
|
||||
@@ -367,6 +402,7 @@ async fn fresh_migration(
|
||||
npm_packages: npm_delta.clone(),
|
||||
verbatim_paths: verbatim.clone(),
|
||||
missing_paths,
|
||||
unpreserved_data: unpreserved,
|
||||
});
|
||||
migration_store::save(&project_id, &mstate)?;
|
||||
|
||||
@@ -408,22 +444,63 @@ async fn fresh_migration(
|
||||
let _ = docker::stop_container(&container_id).await;
|
||||
|
||||
emit_progress(&app_handle, &project_id, "Saving the current image...");
|
||||
docker::commit_container_snapshot(&container_id, &project).await?;
|
||||
if let Err(e) = docker::commit_container_snapshot(&container_id, &project).await {
|
||||
// The container is stopped but intact and `:latest` is untouched, so
|
||||
// the only repair needed is to stop claiming a migration is in flight.
|
||||
abandon_before_swap(&project_id, &state, None).await;
|
||||
return Ok(MigrationReport::failed_preflight(format!(
|
||||
"Could not save the container's current image: {}. Nothing was changed — the container is still on its previous image and can be started as usual.",
|
||||
e
|
||||
)));
|
||||
}
|
||||
|
||||
// The rollback pin. `docker tag` of a 5.49 GB image was measured at 0.036 s
|
||||
// and 0 bytes, so this is free to take and only costs disk if it is kept.
|
||||
//
|
||||
// It is *not* best-effort. The commit above is now the only copy of the old
|
||||
// system layer, the next step removes the container, and `finish_migration`
|
||||
// repoints `:latest` at the new lineage — so a migration that carries on
|
||||
// without a usable pin has quietly made itself irreversible while the UI
|
||||
// goes on offering "Roll back". Tag, read it back, and abort while aborting
|
||||
// still costs nothing.
|
||||
emit_progress(&app_handle, &project_id, "Pinning a rollback image...");
|
||||
let (repo, _) = mig::split_image_ref(&snapshot_image);
|
||||
let tag = mig::rollback_tag(&chrono::Utc::now());
|
||||
let rollback_ref = format!("{}:{}", repo, tag);
|
||||
if let Err(e) = mig::tag_image(&snapshot_image, &repo, &tag).await {
|
||||
log::warn!("Could not create the rollback tag: {} — continuing", e);
|
||||
let tag_result = mig::tag_image(&snapshot_image, &repo, &tag).await;
|
||||
let resolved = if tag_result.is_ok() {
|
||||
mig::image_id(&rollback_ref).await.unwrap_or(None)
|
||||
} else {
|
||||
mstate.rollback_image = Some(rollback_ref);
|
||||
migration_store::save(&project_id, &mstate)?;
|
||||
None
|
||||
};
|
||||
if let Err(why) = rollback_pin_verdict(
|
||||
tag_result.as_ref().err().map(|e| e.as_str()),
|
||||
resolved.as_deref(),
|
||||
) {
|
||||
abandon_before_swap(&project_id, &state, Some(&rollback_ref)).await;
|
||||
return Ok(MigrationReport::failed_preflight(why));
|
||||
}
|
||||
mstate.rollback_image = Some(rollback_ref.clone());
|
||||
if let Err(e) = migration_store::save(&project_id, &mstate) {
|
||||
// A pin nothing has recorded is a pin no recovery path can find, which
|
||||
// is the same hole as not having one. Still pre-swap, so stop here.
|
||||
abandon_before_swap(&project_id, &state, Some(&rollback_ref)).await;
|
||||
return Ok(MigrationReport::failed_preflight(format!(
|
||||
"Could not record the rollback image before replacing the container: {}. Nothing was changed — the container is still on its previous image.",
|
||||
e
|
||||
)));
|
||||
}
|
||||
|
||||
emit_progress(&app_handle, &project_id, "Recreating on the new base image...");
|
||||
docker::remove_container(&container_id).await?;
|
||||
if let Err(e) = docker::remove_container(&container_id).await {
|
||||
// Still pre-swap: the old container is the one in place and `:latest`
|
||||
// still holds its lineage.
|
||||
abandon_before_swap(&project_id, &state, Some(&rollback_ref)).await;
|
||||
return Ok(MigrationReport::failed_preflight(format!(
|
||||
"Could not remove the old container: {}. Nothing was changed — the container is still on its previous image and can be started as usual.",
|
||||
e
|
||||
)));
|
||||
}
|
||||
|
||||
let docker_socket = settings
|
||||
.docker_socket_path
|
||||
@@ -487,9 +564,63 @@ async fn resume_migration(
|
||||
));
|
||||
}
|
||||
};
|
||||
// The container being *there* is not evidence that the swap happened. If
|
||||
// `commit_container_snapshot` failed, the old, unmigrated container is
|
||||
// still in place — and resuming into it would replay the deltas onto the
|
||||
// container that already had them, commit it as the migrated image and
|
||||
// report success. `reconcile_migration` has always checked this label; the
|
||||
// path the Migrate button reaches did not.
|
||||
let labelled = container_label(&container_id, mig::LABEL_MIGRATION_STATE)
|
||||
.await
|
||||
.as_deref()
|
||||
== Some(mig::MIGRATION_LABEL_IN_PROGRESS);
|
||||
match resume_verdict(&mstate.phase, labelled) {
|
||||
ResumeVerdict::Proceed => {}
|
||||
ResumeVerdict::RefuseUnswapped => {
|
||||
// Exactly the `SelfHeal` case `reconcile_migration` handles, so
|
||||
// handle it the same way: the record describes work that never
|
||||
// landed, `:latest` still holds the old lineage, and clearing it
|
||||
// turns a dead end into "just run the update again".
|
||||
if let Some(ref reference) = mstate.rollback_image {
|
||||
let _ = mig::untag_image(reference).await;
|
||||
}
|
||||
let _ = migration_store::clear_staging(&project_id);
|
||||
let _ = migration_store::clear(&project_id);
|
||||
return Ok(MigrationReport::failed_preflight(
|
||||
"This project's container is still the original one — the interrupted update never got as far as replacing it, so there was nothing to resume. Nothing was changed and the record has been cleared: start the project as usual, or run the update again from the beginning.",
|
||||
));
|
||||
}
|
||||
ResumeVerdict::RefuseNotResumable => {
|
||||
return Ok(MigrationReport::failed_preflight(format!(
|
||||
"This project's migration record is in the '{}' state, which cannot be resumed. Confirm it or roll it back first.",
|
||||
mstate.phase
|
||||
)));
|
||||
}
|
||||
}
|
||||
|
||||
if !docker::is_container_running(&container_id).await.unwrap_or(false) {
|
||||
emit_progress(&app_handle, &project_id, "Starting the migrated container...");
|
||||
docker::start_container(&container_id).await?;
|
||||
if let Err(e) = docker::start_container(&container_id).await {
|
||||
// Leave the record alone — the container is still mid-swap and the
|
||||
// user's choices are unchanged — but do not leave the project
|
||||
// parked at a transitional status nothing will ever revisit.
|
||||
let _ = state
|
||||
.projects_store
|
||||
.update_status(&project_id, ProjectStatus::Stopped);
|
||||
return Ok(MigrationReport {
|
||||
phase: MigrationPhase::Failed,
|
||||
packages_requested: Vec::new(),
|
||||
packages_installed: Vec::new(),
|
||||
packages_failed: Vec::new(),
|
||||
paths_copied: Vec::new(),
|
||||
features_restored: Vec::new(),
|
||||
rollback_available: mstate.rollback_image.is_some(),
|
||||
message: format!(
|
||||
"The half-migrated container would not start ({}), so the update could not be resumed. Nothing was lost — your home directory and Claude config live in volumes that were never touched. Try again, or roll back.",
|
||||
e
|
||||
),
|
||||
});
|
||||
}
|
||||
}
|
||||
emit_progress(&app_handle, &project_id, "Resuming the interrupted migration...");
|
||||
finish_migration(project, mstate, container_id, app_handle, state).await
|
||||
@@ -592,6 +723,12 @@ async fn finish_migration(
|
||||
};
|
||||
mstate.report = Some(report.clone());
|
||||
let _ = migration_store::save(&project_id, &mstate);
|
||||
// The container really is up, so say so. Left at `Stopping` this
|
||||
// project would never be re-examined: `reconcile_project_statuses`
|
||||
// only looks at `Running` and `Error`.
|
||||
let _ = state
|
||||
.projects_store
|
||||
.update_status(&project_id, ProjectStatus::Running);
|
||||
return Ok(report);
|
||||
}
|
||||
|
||||
@@ -630,6 +767,7 @@ async fn finish_migration(
|
||||
&features_restored,
|
||||
¬es,
|
||||
rollback_available,
|
||||
&plan.unpreserved_data,
|
||||
),
|
||||
phase,
|
||||
packages_requested,
|
||||
@@ -642,11 +780,13 @@ async fn finish_migration(
|
||||
|
||||
mstate.phase = MIGRATION_PHASE_AWAITING.to_string();
|
||||
mstate.report = Some(report.clone());
|
||||
migration_store::save(&project_id, &mstate)?;
|
||||
|
||||
// Before the save, not after: a failed save must not leave the project
|
||||
// parked at `Stopping` with a container that is plainly running.
|
||||
let _ = state
|
||||
.projects_store
|
||||
.update_status(&project_id, ProjectStatus::Running);
|
||||
migration_store::save(&project_id, &mstate)?;
|
||||
|
||||
emit_progress(&app_handle, &project_id, "Migration finished.");
|
||||
Ok(report)
|
||||
}
|
||||
@@ -663,9 +803,28 @@ pub async fn confirm_migration(
|
||||
state: State<'_, AppState>,
|
||||
) -> Result<(), String> {
|
||||
let _ = &state;
|
||||
// Confirming drops the only way back. Doing that underneath a running
|
||||
// migration would delete the pin it is relying on mid-flight.
|
||||
let Some(_guard) = ActiveGuard::acquire(&project_id) else {
|
||||
return Err(
|
||||
"A container base update is running for this project right now. Wait for it to finish."
|
||||
.to_string(),
|
||||
);
|
||||
};
|
||||
let Some(mstate) = migration_store::load(&project_id)? else {
|
||||
return Ok(());
|
||||
};
|
||||
// An unfinished migration is not a thing that can be accepted: `:latest`
|
||||
// still points at the old lineage, so dropping the pin and the record here
|
||||
// would strand a container the app can no longer reason about.
|
||||
if mstate.phase == MIGRATION_PHASE_INTERRUPTED
|
||||
|| mstate.phase == crate::models::MIGRATION_PHASE_IN_PROGRESS
|
||||
{
|
||||
return Err(
|
||||
"This project's container base update never finished, so there is nothing to accept yet. Resume it or roll it back."
|
||||
.to_string(),
|
||||
);
|
||||
}
|
||||
if let Some(ref reference) = mstate.rollback_image {
|
||||
if let Err(e) = mig::untag_image(reference).await {
|
||||
log::warn!("Could not drop the rollback tag {}: {}", reference, e);
|
||||
@@ -688,6 +847,13 @@ pub async fn rollback_migration(
|
||||
app_handle: tauri::AppHandle,
|
||||
state: State<'_, AppState>,
|
||||
) -> Result<(), String> {
|
||||
let Some(_guard) = ActiveGuard::acquire(&project_id) else {
|
||||
return Err(
|
||||
"A container base update is running for this project right now. Wait for it to finish."
|
||||
.to_string(),
|
||||
);
|
||||
};
|
||||
|
||||
let mut project = state
|
||||
.projects_store
|
||||
.get(&project_id)
|
||||
@@ -704,6 +870,16 @@ pub async fn rollback_migration(
|
||||
.to_string(),
|
||||
);
|
||||
};
|
||||
// Check the image is really there *before* removing the container. The
|
||||
// record only says a tag was created; a prune, a `docker rmi` or a failed
|
||||
// tag from an older build can all leave the reference dangling, and by the
|
||||
// time the container is gone there is nothing left to put back.
|
||||
if mig::image_id(&rollback_ref).await?.is_none() {
|
||||
return Err(format!(
|
||||
"The rollback image '{}' no longer exists, so this update cannot be undone — it may have been pruned. Nothing was changed: the container is still running on the new base.",
|
||||
rollback_ref
|
||||
));
|
||||
}
|
||||
|
||||
emit_progress(&app_handle, &project_id, "Rolling back...");
|
||||
state.auth_bridge.stop(&project_id).await;
|
||||
@@ -842,6 +1018,112 @@ pub async fn reconcile_migration(project: &Project, app_handle: &tauri::AppHandl
|
||||
// Internals
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
/// Whether the rollback pin can be relied on for the rest of the migration.
|
||||
///
|
||||
/// Pure so the decision is testable without Docker. Both failure shapes matter:
|
||||
/// `docker tag` can fail outright, and it can also "succeed" against an image
|
||||
/// that is not there any more (a concurrent prune), in which case reading the
|
||||
/// new reference back is the only thing that catches it. Either way the answer
|
||||
/// is to stop *before* `remove_container`, which is the last moment at which
|
||||
/// stopping is free.
|
||||
fn rollback_pin_verdict(
|
||||
tag_error: Option<&str>,
|
||||
resolved_image_id: Option<&str>,
|
||||
) -> Result<(), String> {
|
||||
if let Some(e) = tag_error {
|
||||
return Err(format!(
|
||||
"Could not pin a rollback image, so the update was stopped before it could become irreversible: {}. Nothing was changed — the container is still on its previous image. Free some disk space or check the Docker daemon, then try again.",
|
||||
e
|
||||
));
|
||||
}
|
||||
match resolved_image_id {
|
||||
Some(id) if !id.is_empty() => Ok(()),
|
||||
_ => Err(
|
||||
"The rollback image was tagged but could not be read back, so rolling this update back could not be guaranteed. The update was stopped before it could become irreversible — nothing was changed, and the container is still on its previous image."
|
||||
.to_string(),
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
/// Undo the *bookkeeping* of a migration that gave up before the container
|
||||
/// swap. Everything real is still in place: the container exists, `:latest`
|
||||
/// still holds the old lineage, and the volumes were never involved.
|
||||
///
|
||||
/// The status reset is the load-bearing part. `reconcile_project_statuses`
|
||||
/// only ever re-examines projects it finds in `Running` or `Error`, so a
|
||||
/// project abandoned at `Stopping` stays there for good — the Start button
|
||||
/// stays disabled and nothing in the app ever puts it right.
|
||||
async fn abandon_before_swap(
|
||||
project_id: &str,
|
||||
state: &State<'_, AppState>,
|
||||
rollback_ref: Option<&str>,
|
||||
) {
|
||||
if let Some(reference) = rollback_ref {
|
||||
let _ = mig::untag_image(reference).await;
|
||||
}
|
||||
let _ = migration_store::clear_staging(project_id);
|
||||
let _ = migration_store::clear(project_id);
|
||||
let _ = state
|
||||
.projects_store
|
||||
.update_status(project_id, ProjectStatus::Stopped);
|
||||
}
|
||||
|
||||
/// What a user-initiated resume is allowed to do.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
enum ResumeVerdict {
|
||||
/// The swap landed — the container in place is the migrated one.
|
||||
Proceed,
|
||||
/// The container in place is the **original**. The swap never happened
|
||||
/// (typically `commit_container_snapshot` failed), so replaying into it and
|
||||
/// committing the result would silently stamp the unmigrated container as
|
||||
/// migrated and report success.
|
||||
RefuseUnswapped,
|
||||
/// The record is not one a resume can act on at all.
|
||||
RefuseNotResumable,
|
||||
}
|
||||
|
||||
/// Gate `resume_migration` on the same two signals `reconcile_migration` uses.
|
||||
///
|
||||
/// `reconcile_migration` has always checked the container's
|
||||
/// `triple-c.migration-state` label; the frontend-callable resume path did not,
|
||||
/// and that asymmetry is the bug: clicking Migrate on a record left behind by a
|
||||
/// failed commit "resumed" into the old container.
|
||||
fn resume_verdict(phase: &str, container_has_in_progress_label: bool) -> ResumeVerdict {
|
||||
match mig::decide_recovery(Some(phase), container_has_in_progress_label) {
|
||||
Recovery::OfferResumeOrRollback => ResumeVerdict::Proceed,
|
||||
Recovery::SelfHeal => ResumeVerdict::RefuseUnswapped,
|
||||
Recovery::None | Recovery::OfferConfirmOrRollback => ResumeVerdict::RefuseNotResumable,
|
||||
}
|
||||
}
|
||||
|
||||
/// Drop every host- and Docker-side trace of a migration for a project that is
|
||||
/// being destroyed or reset.
|
||||
///
|
||||
/// Reset and Remove both delete the snapshot image and the volumes, so a
|
||||
/// surviving migration record can only describe things that no longer exist —
|
||||
/// and its `:pre-migration-<ts>` tag would hold an entire snapshot image (3.8
|
||||
/// to 12.3 GB measured) alive with nothing left that could ever use it. The
|
||||
/// staged payload tar is the same story at a smaller scale.
|
||||
pub(crate) async fn purge_migration_artifacts(project_id: &str) {
|
||||
match migration_store::load(project_id) {
|
||||
Ok(Some(state)) => {
|
||||
if let Some(ref reference) = state.rollback_image {
|
||||
if let Err(e) = mig::untag_image(reference).await {
|
||||
log::warn!("Could not drop the rollback tag {}: {}", reference, e);
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(None) => return,
|
||||
Err(e) => log::warn!(
|
||||
"Could not read the migration record for {} while cleaning up: {}",
|
||||
project_id,
|
||||
e
|
||||
),
|
||||
}
|
||||
let _ = migration_store::clear_staging(project_id);
|
||||
let _ = migration_store::clear(project_id);
|
||||
}
|
||||
|
||||
fn default_docker_socket() -> String {
|
||||
if cfg!(target_os = "windows") {
|
||||
"//./pipe/docker_engine".to_string()
|
||||
@@ -865,6 +1147,23 @@ fn human_bytes(n: u64) -> String {
|
||||
}
|
||||
}
|
||||
|
||||
/// Create (or truncate) a file that only the current user can read.
|
||||
///
|
||||
/// On Unix the mode goes on at `open(2)` time so there is no window in which
|
||||
/// the file exists world-readable. Windows has no equivalent bit and inherits
|
||||
/// the directory ACL, which is already per-user under `%APPDATA%`.
|
||||
async fn create_private_file(path: &std::path::Path) -> std::io::Result<tokio::fs::File> {
|
||||
let mut opts = tokio::fs::OpenOptions::new();
|
||||
opts.write(true).create(true).truncate(true);
|
||||
#[cfg(unix)]
|
||||
{
|
||||
// `tokio::fs::OpenOptions::mode` is the inherent unix method — no
|
||||
// `OpenOptionsExt` import needed, and it is applied at `open(2)` time.
|
||||
opts.mode(0o600);
|
||||
}
|
||||
opts.open(path).await
|
||||
}
|
||||
|
||||
/// Tar the verbatim set out of the running container into a host file.
|
||||
///
|
||||
/// Mirrors `download_container_backup`'s stream-an-exec's-stdout-to-a-file
|
||||
@@ -913,7 +1212,12 @@ async fn stage_payload(
|
||||
let exec = docker::exec::create_attached_exec_as(container_id, cmd, false, "root", "/").await?;
|
||||
let mut output = exec.output;
|
||||
|
||||
let file = tokio::fs::File::create(&host_path)
|
||||
// 0600, not the default umask. This tar holds the whole of `/usr/local`,
|
||||
// `/opt`, `/srv` and every loose `/workspace` file — private keys, tokens
|
||||
// in scripts, whatever the user put there — sitting in a predictable path
|
||||
// under the data directory, possibly for the length of a long migration.
|
||||
// World-readable is the wrong default for that.
|
||||
let file = create_private_file(&host_path)
|
||||
.await
|
||||
.map_err(|e| format!("Failed to create the staging file: {}", e))?;
|
||||
let mut writer = tokio::io::BufWriter::new(file);
|
||||
@@ -1168,9 +1472,23 @@ fn shell_quote(s: &str) -> String {
|
||||
format!("'{}'", s.replace('\'', r#"'\''"#))
|
||||
}
|
||||
|
||||
/// The last ~400 bytes of a package manager's output, for a failure reason.
|
||||
///
|
||||
/// The offset is a **byte** offset into arbitrary apt-get/npm output, which
|
||||
/// routinely contains multi-byte UTF-8 (mirror names, `‘quoted’` package names,
|
||||
/// progress glyphs). Slicing straight at `len - 400` panics the moment that
|
||||
/// lands inside a character — turning a reportable per-package failure into a
|
||||
/// crash of the whole migration. Walk forward to the next boundary instead.
|
||||
fn tail(s: &str) -> String {
|
||||
const LIMIT: usize = 400;
|
||||
let t = s.trim();
|
||||
let start = t.len().saturating_sub(400);
|
||||
if t.len() <= LIMIT {
|
||||
return t.to_string();
|
||||
}
|
||||
let mut start = t.len() - LIMIT;
|
||||
while start < t.len() && !t.is_char_boundary(start) {
|
||||
start += 1;
|
||||
}
|
||||
t[start..].to_string()
|
||||
}
|
||||
|
||||
@@ -1221,11 +1539,27 @@ async fn auto_rollback(
|
||||
let _ = state
|
||||
.projects_store
|
||||
.set_container_id(&project.id, Some(id));
|
||||
// Report what is actually true: a container that was recreated but
|
||||
// would not start is Stopped, not Running.
|
||||
let _ = state.projects_store.update_status(
|
||||
&project.id,
|
||||
if restored {
|
||||
ProjectStatus::Running
|
||||
} else {
|
||||
ProjectStatus::Stopped
|
||||
},
|
||||
);
|
||||
}
|
||||
Err(e) => {
|
||||
log::error!("Could not recreate the previous container: {}", e);
|
||||
// The status is `Stopping` at this point and nothing else will
|
||||
// revisit it — `reconcile_project_statuses` only re-examines
|
||||
// `Running` and `Error`. There is no container, so `Stopped` is
|
||||
// both true and the state the Start button needs.
|
||||
let _ = state
|
||||
.projects_store
|
||||
.update_status(&project.id, ProjectStatus::Running);
|
||||
.update_status(&project.id, ProjectStatus::Stopped);
|
||||
}
|
||||
Err(e) => log::error!("Could not recreate the previous container: {}", e),
|
||||
}
|
||||
|
||||
if let Some(ref reference) = mstate.rollback_image {
|
||||
@@ -1264,6 +1598,7 @@ fn summarize(
|
||||
features: &[String],
|
||||
notes: &[String],
|
||||
rollback_available: bool,
|
||||
unpreserved: &[UnpreservedData],
|
||||
) -> String {
|
||||
let mut parts: Vec<String> = Vec::new();
|
||||
parts.push(match phase {
|
||||
@@ -1291,6 +1626,25 @@ fn summarize(
|
||||
for note in notes {
|
||||
parts.push(note.clone());
|
||||
}
|
||||
// Stated in the outcome as well as the pre-flight. A user who clicked
|
||||
// through the warning still has to be told, in the record that persists,
|
||||
// which directories are now empty — silence here is how someone discovers
|
||||
// an empty database a week later.
|
||||
if !unpreserved.is_empty() {
|
||||
parts.push(format!(
|
||||
"Service data under {} was not carried across and cannot be restored by reinstalling the package{}.",
|
||||
unpreserved
|
||||
.iter()
|
||||
.map(|d| d.path.as_str())
|
||||
.collect::<Vec<_>>()
|
||||
.join(", "),
|
||||
if rollback_available {
|
||||
" — roll back if you needed it"
|
||||
} else {
|
||||
""
|
||||
}
|
||||
));
|
||||
}
|
||||
parts.push(
|
||||
"Your home directory and Claude config live in volumes that were never touched, so the login, skills, transcripts and scheduled tasks are exactly as they were."
|
||||
.to_string(),
|
||||
@@ -1330,6 +1684,7 @@ mod tests {
|
||||
&["Auth bridge tunnel (socat)".to_string()],
|
||||
&[],
|
||||
true,
|
||||
&[],
|
||||
);
|
||||
assert!(msg.contains("never touched"));
|
||||
assert!(msg.contains("Auth bridge tunnel (socat)"));
|
||||
@@ -1346,12 +1701,115 @@ mod tests {
|
||||
&[],
|
||||
&[],
|
||||
false,
|
||||
&[],
|
||||
);
|
||||
assert!(msg.contains("obsolete-pkg"));
|
||||
assert!(msg.contains("cannot be undone"));
|
||||
assert!(msg.contains("never touched"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_summary_names_the_var_data_the_migration_destroyed() {
|
||||
// The pre-flight warned about it; the record that persists has to as
|
||||
// well, or the only trace of an emptied database is a dialog the user
|
||||
// clicked through minutes ago.
|
||||
let msg = summarize(
|
||||
MigrationPhase::Succeeded,
|
||||
&["postgresql".to_string()],
|
||||
&[],
|
||||
&[],
|
||||
&[],
|
||||
&[],
|
||||
true,
|
||||
&[UnpreservedData {
|
||||
path: "/var/lib/postgresql".to_string(),
|
||||
bytes: 41_000_000,
|
||||
file_count: 912,
|
||||
}],
|
||||
);
|
||||
assert!(msg.contains("/var/lib/postgresql"));
|
||||
assert!(msg.contains("cannot be restored by reinstalling the package"));
|
||||
}
|
||||
|
||||
// ── The rollback pin is not best-effort ─────────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn a_migration_refuses_to_continue_without_a_verified_rollback_pin() {
|
||||
// By this point the commit is the only copy of the old system layer and
|
||||
// `remove_container` is next, so anything short of a tag that reads
|
||||
// back has to stop the migration rather than warn and carry on.
|
||||
assert!(rollback_pin_verdict(None, Some("sha256:abc")).is_ok());
|
||||
|
||||
let err = rollback_pin_verdict(Some("no space left on device"), None).unwrap_err();
|
||||
assert!(err.contains("no space left on device"));
|
||||
assert!(err.contains("Nothing was changed"));
|
||||
|
||||
// Tagged, but the reference does not resolve — a concurrent prune, or a
|
||||
// daemon that accepted the call and did nothing.
|
||||
let err = rollback_pin_verdict(None, None).unwrap_err();
|
||||
assert!(err.contains("could not be read back"));
|
||||
assert!(err.contains("nothing was changed"));
|
||||
assert!(rollback_pin_verdict(None, Some("")).is_err());
|
||||
}
|
||||
|
||||
// ── Resume must prove the swap actually happened ────────────────────────
|
||||
|
||||
#[test]
|
||||
fn resuming_refuses_when_the_container_is_still_the_unmigrated_one() {
|
||||
use crate::models::MIGRATION_PHASE_IN_PROGRESS;
|
||||
|
||||
// The label is the only evidence the swap landed. Without it the
|
||||
// container in place is the original: replaying into it and committing
|
||||
// would stamp an unmigrated container as migrated and report success.
|
||||
assert_eq!(
|
||||
resume_verdict(MIGRATION_PHASE_INTERRUPTED, false),
|
||||
ResumeVerdict::RefuseUnswapped
|
||||
);
|
||||
assert_eq!(
|
||||
resume_verdict(MIGRATION_PHASE_IN_PROGRESS, false),
|
||||
ResumeVerdict::RefuseUnswapped
|
||||
);
|
||||
assert_eq!(
|
||||
resume_verdict(MIGRATION_PHASE_INTERRUPTED, true),
|
||||
ResumeVerdict::Proceed
|
||||
);
|
||||
assert_eq!(
|
||||
resume_verdict(MIGRATION_PHASE_IN_PROGRESS, true),
|
||||
ResumeVerdict::Proceed
|
||||
);
|
||||
// A finished migration is a confirm/rollback decision, never a resume.
|
||||
assert_eq!(
|
||||
resume_verdict(MIGRATION_PHASE_AWAITING, true),
|
||||
ResumeVerdict::RefuseNotResumable
|
||||
);
|
||||
assert_eq!(
|
||||
resume_verdict("who-knows", true),
|
||||
ResumeVerdict::RefuseNotResumable
|
||||
);
|
||||
}
|
||||
|
||||
// ── tail() ──────────────────────────────────────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn the_failure_tail_never_splits_a_multibyte_character() {
|
||||
// apt-get quotes package names with ‘…’ and mirrors have non-ASCII
|
||||
// names, so the 400-byte cut lands mid-character routinely. The old
|
||||
// slice panicked, turning one reportable package failure into a crash
|
||||
// of the whole migration.
|
||||
for pad in 0..8 {
|
||||
let s = format!("{}{}", "x".repeat(pad), "é".repeat(400));
|
||||
let t = tail(&s);
|
||||
assert!(s.ends_with(&t), "tail must be a suffix of the input");
|
||||
assert!(t.len() <= 400 + 1);
|
||||
}
|
||||
// Short input is returned whole, trimmed.
|
||||
assert_eq!(tail(" apt-get exited 100 "), "apt-get exited 100");
|
||||
assert_eq!(tail(""), "");
|
||||
// A single character wider than the window is still returned intact.
|
||||
let wide = "🐳".repeat(200);
|
||||
assert!(wide.ends_with(&tail(&wide)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_live_migration_is_distinguishable_from_a_crashed_one() {
|
||||
// The whole point: reconcile cannot tell them apart from the outside,
|
||||
|
||||
@@ -145,6 +145,11 @@ pub async fn remove_project(
|
||||
// before the container (and the project record) go away.
|
||||
state.auth_bridge.stop(&project_id).await;
|
||||
|
||||
// A migration record outliving its project leaks a state file, a staged
|
||||
// payload tar that can run to several GB, and a `:pre-migration-<ts>` tag
|
||||
// holding an entire snapshot image that nothing will ever reference again.
|
||||
crate::commands::migration_commands::purge_migration_artifacts(&project_id).await;
|
||||
|
||||
// Stop and remove container if it exists
|
||||
if let Some(ref project) = state.projects_store.get(&project_id) {
|
||||
if let Some(ref container_id) = project.container_id {
|
||||
@@ -215,6 +220,20 @@ pub async fn start_project_container(
|
||||
app_handle: tauri::AppHandle,
|
||||
state: State<'_, AppState>,
|
||||
) -> Result<Project, String> {
|
||||
// A migration removes the container and creates its replacement moments
|
||||
// later. Starting in that window finds no container, creates a second one
|
||||
// under the same name, and the migration's own create then fails on the
|
||||
// name conflict — which sends it into an auto-rollback that also cannot
|
||||
// create. The UI already refuses (`canMigrate` gates on the container being
|
||||
// stopped and no run being in flight); this is the same gate on the side
|
||||
// that actually owns the invariant.
|
||||
if crate::commands::migration_commands::is_migrating(&project_id) {
|
||||
return Err(
|
||||
"A container base update is running for this project. Wait for it to finish, then start the project."
|
||||
.to_string(),
|
||||
);
|
||||
}
|
||||
|
||||
let mut project = state
|
||||
.projects_store
|
||||
.get(&project_id)
|
||||
@@ -536,11 +555,28 @@ pub async fn rebuild_project_container(
|
||||
app_handle: tauri::AppHandle,
|
||||
state: State<'_, AppState>,
|
||||
) -> Result<Project, String> {
|
||||
// Reset deletes both volumes and the snapshot image. Doing that while a
|
||||
// migration is mid-flight pulls the ground out from under it and leaves an
|
||||
// orphan migration record pointing at images that no longer exist.
|
||||
if crate::commands::migration_commands::is_migrating(&project_id) {
|
||||
return Err(
|
||||
"A container base update is running for this project. Wait for it to finish before resetting."
|
||||
.to_string(),
|
||||
);
|
||||
}
|
||||
|
||||
let project = state
|
||||
.projects_store
|
||||
.get(&project_id)
|
||||
.ok_or_else(|| format!("Project {} not found", project_id))?;
|
||||
|
||||
// Reset supersedes any migration decision that was still pending: the
|
||||
// snapshot image and both volumes are about to go, so a surviving record
|
||||
// could only describe things that no longer exist — while its
|
||||
// `:pre-migration-<ts>` tag held a whole snapshot image (multiple GB) alive
|
||||
// with nothing left that could ever use it.
|
||||
crate::commands::migration_commands::purge_migration_artifacts(&project_id).await;
|
||||
|
||||
// The bridge is bound to the container that is about to be destroyed;
|
||||
// `start_project_container` below re-arms it against the new one.
|
||||
state.auth_bridge.stop(&project_id).await;
|
||||
@@ -588,7 +624,27 @@ pub async fn reconcile_project_statuses(
|
||||
}
|
||||
|
||||
for project in &projects {
|
||||
if project.status != ProjectStatus::Running && project.status != ProjectStatus::Error {
|
||||
// `Starting` and `Stopping` are in here as a backstop, not because
|
||||
// anything is expected to leave a project in one. They are transitional
|
||||
// states owned by an in-flight command, so a project still wearing one
|
||||
// is a project whose command died — a crash mid-start, or a migration
|
||||
// that bailed out between the stop and the swap. Skipping them, as this
|
||||
// loop used to, meant nothing in the app ever put such a project right:
|
||||
// it sat at "Stopping" with the Start button disabled, permanently.
|
||||
// Docker is the authority either way, so the check below is correct for
|
||||
// all four.
|
||||
if !matches!(
|
||||
project.status,
|
||||
ProjectStatus::Running
|
||||
| ProjectStatus::Error
|
||||
| ProjectStatus::Starting
|
||||
| ProjectStatus::Stopping
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
// ...but never for a project this process is actively migrating: the
|
||||
// container is legitimately absent for part of that run.
|
||||
if crate::commands::migration_commands::is_migrating(&project.id) {
|
||||
continue;
|
||||
}
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
use tauri::State;
|
||||
|
||||
use crate::docker;
|
||||
use crate::models::gateway_settings::GatewaySettings;
|
||||
use crate::models::AppSettings;
|
||||
use crate::AppState;
|
||||
|
||||
@@ -14,7 +15,94 @@ pub async fn update_settings(
|
||||
settings: AppSettings,
|
||||
state: State<'_, AppState>,
|
||||
) -> Result<AppSettings, String> {
|
||||
state.settings_store.update(settings)
|
||||
let before = state.settings_store.get();
|
||||
let saved = state.settings_store.update(settings)?;
|
||||
|
||||
// Persisting a setting is not the same as applying it. The gateway is the
|
||||
// one settings block that owns a *container*, so a saved change that the
|
||||
// running container doesn't reflect is a live desync, not a preference.
|
||||
reconcile_gateway(&before.gateway, &saved.gateway).await;
|
||||
|
||||
Ok(saved)
|
||||
}
|
||||
|
||||
/// What a settings save has to do to the gateway container to stay honest.
|
||||
///
|
||||
/// Kept separate from the IPC command and expressed over plain settings so the
|
||||
/// decision is testable without Docker.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
enum GatewayAction {
|
||||
/// Nothing to do.
|
||||
None,
|
||||
/// The gateway is off — a container left running must be stopped.
|
||||
StopIfRunning,
|
||||
/// The published shape moved. A *running* container is now serving on the
|
||||
/// old binding while status reports the new one, so it has to be recreated.
|
||||
RestartIfRunning,
|
||||
}
|
||||
|
||||
/// Whether the container's published shape (as opposed to a purely cosmetic
|
||||
/// field) changed. Provider, models and base URL all change the rendered
|
||||
/// LiteLLM config, which is only read at boot.
|
||||
fn gateway_shape_changed(before: &GatewaySettings, after: &GatewaySettings) -> bool {
|
||||
before.port != after.port
|
||||
|| before.provider.trim() != after.provider.trim()
|
||||
|| before.api_base.as_deref().unwrap_or("").trim()
|
||||
!= after.api_base.as_deref().unwrap_or("").trim()
|
||||
|| before.valid_models() != after.valid_models()
|
||||
}
|
||||
|
||||
fn gateway_action(before: &GatewaySettings, after: &GatewaySettings) -> GatewayAction {
|
||||
if !after.enabled {
|
||||
// Includes the case where it was already disabled: a container found
|
||||
// running while the feature is off should not stay up.
|
||||
return GatewayAction::StopIfRunning;
|
||||
}
|
||||
if gateway_shape_changed(before, after) {
|
||||
return GatewayAction::RestartIfRunning;
|
||||
}
|
||||
GatewayAction::None
|
||||
}
|
||||
|
||||
/// Apply [`gateway_action`]. Never fails the settings save: the settings *are*
|
||||
/// saved by this point, and a Docker hiccup must not make the UI think they
|
||||
/// weren't. Both paths are no-ops when no container exists, so this stays cheap
|
||||
/// on the overwhelmingly common "gateway not in use" save.
|
||||
async fn reconcile_gateway(before: &GatewaySettings, after: &GatewaySettings) {
|
||||
let action = gateway_action(before, after);
|
||||
if action == GatewayAction::None {
|
||||
return;
|
||||
}
|
||||
|
||||
let (exists, running) = match docker::gateway::gateway_container_presence().await {
|
||||
Ok(presence) => presence,
|
||||
// Docker down: there is nothing running to desync from.
|
||||
Err(e) => {
|
||||
log::debug!("Gateway reconcile skipped ({})", e);
|
||||
return;
|
||||
}
|
||||
};
|
||||
if !exists || !running {
|
||||
return;
|
||||
}
|
||||
|
||||
match action {
|
||||
GatewayAction::StopIfRunning => {
|
||||
log::info!("Model gateway disabled in settings — stopping the container");
|
||||
if let Err(e) = docker::gateway::stop_gateway_container().await {
|
||||
log::error!("Failed to stop the model gateway after it was disabled: {}", e);
|
||||
}
|
||||
}
|
||||
GatewayAction::RestartIfRunning => {
|
||||
log::info!("Model gateway settings changed — recreating the container");
|
||||
// The fingerprint no longer matches, so this stops, removes and
|
||||
// recreates with the new port/config in one step.
|
||||
if let Err(e) = docker::gateway::ensure_gateway_running(after).await {
|
||||
log::error!("Failed to apply the new model gateway settings: {}", e);
|
||||
}
|
||||
}
|
||||
GatewayAction::None => unreachable!(),
|
||||
}
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
@@ -115,3 +203,96 @@ pub async fn list_aws_profiles() -> Result<Vec<String>, String> {
|
||||
|
||||
Ok(profiles)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::models::gateway_settings::GatewayModel;
|
||||
|
||||
fn enabled_gateway() -> GatewaySettings {
|
||||
GatewaySettings {
|
||||
enabled: true,
|
||||
port: 4000,
|
||||
provider: "openai".to_string(),
|
||||
api_base: None,
|
||||
models: vec![GatewayModel {
|
||||
name: "gpt-5.1".to_string(),
|
||||
model_id: "gpt-5.1".to_string(),
|
||||
}],
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn disabling_the_gateway_stops_it() {
|
||||
// The bug: turning the toggle off only persisted `enabled: false` and
|
||||
// hid the Stop button, leaving a container serving with no way to stop
|
||||
// it.
|
||||
let before = enabled_gateway();
|
||||
let mut after = before.clone();
|
||||
after.enabled = false;
|
||||
assert_eq!(gateway_action(&before, &after), GatewayAction::StopIfRunning);
|
||||
// Still true when it was already off — a stray running container is
|
||||
// still a container that shouldn't be up.
|
||||
assert_eq!(gateway_action(&after, &after), GatewayAction::StopIfRunning);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn changing_the_port_reconciles_the_container() {
|
||||
// Otherwise status reports the new port while the container keeps the
|
||||
// old binding, and every project gets a broken ANTHROPIC_BASE_URL.
|
||||
let before = enabled_gateway();
|
||||
let mut after = before.clone();
|
||||
after.port = 4100;
|
||||
assert_eq!(
|
||||
gateway_action(&before, &after),
|
||||
GatewayAction::RestartIfRunning
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn config_changes_that_only_take_effect_at_boot_reconcile_too() {
|
||||
let before = enabled_gateway();
|
||||
|
||||
let mut provider = before.clone();
|
||||
provider.provider = "groq".to_string();
|
||||
assert_eq!(
|
||||
gateway_action(&before, &provider),
|
||||
GatewayAction::RestartIfRunning
|
||||
);
|
||||
|
||||
let mut api_base = before.clone();
|
||||
api_base.api_base = Some("https://example.test/v1".to_string());
|
||||
assert_eq!(
|
||||
gateway_action(&before, &api_base),
|
||||
GatewayAction::RestartIfRunning
|
||||
);
|
||||
|
||||
let mut models = before.clone();
|
||||
models.models[0].model_id = "gpt-4.1".to_string();
|
||||
assert_eq!(
|
||||
gateway_action(&before, &models),
|
||||
GatewayAction::RestartIfRunning
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn saving_an_unchanged_or_half_typed_gateway_touches_nothing() {
|
||||
let before = enabled_gateway();
|
||||
assert_eq!(gateway_action(&before, &before), GatewayAction::None);
|
||||
|
||||
// Whitespace-only edits don't reach the rendered config.
|
||||
let mut trimmed = before.clone();
|
||||
trimmed.provider = " openai ".to_string();
|
||||
trimmed.api_base = Some(" ".to_string());
|
||||
assert_eq!(gateway_action(&before, &trimmed), GatewayAction::None);
|
||||
|
||||
// A half-filled model row is skipped when rendering, so it must not
|
||||
// bounce a live container either.
|
||||
let mut half_typed = before.clone();
|
||||
half_typed.models.push(GatewayModel {
|
||||
name: "gpt".to_string(),
|
||||
model_id: String::new(),
|
||||
});
|
||||
assert_eq!(gateway_action(&before, &half_typed), GatewayAction::None);
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user