Compare commits

..
Author SHA1 Message Date
shadowdaoandClaude Opus 5 95a78fe9a3 Take the review: cache the stopped probe, and never let it cost an answer
Secret Scan / scan (push) Successful in 6s
Build App (Preview) / compute-version (pull_request) Successful in 5s
Secret Scan / scan (pull_request) Successful in 4s
Build App (Preview) / create-release (pull_request) Successful in 1s
Build App (Preview) / build-macos (pull_request) Successful in 2m58s
Build App (Preview) / build-linux (pull_request) Successful in 4m43s
Build App (Preview) / build-windows (pull_request) Successful in 5m9s
Build App (Preview) / prune-previews (pull_request) Successful in 2s
Six findings, all real. The one that mattered: `getContainerStaleness` is
called from a `useEffect` that fires whenever the container settles, so
merely opening a stopped project's Overview now committed its whole writable
layer — 44 s on a real project, against ~3 s for the snapshot probe it
replaced. Shipping that would have traded one bad banner for a bad page.

A stopped container's writable layer cannot change, so the probe is exactly
cacheable: `STOPPED_MANIFEST_CACHE` keys on the container's `FinishedAt`,
which moves on every stop. Cold 2967 ms, warm 1 ms, measured. A live test
asserts the restart case as well as the hit, because a cache that failed to
invalidate would plan a migration against a filesystem the project no longer
has — verified by breaking the token and watching that assertion fail.

Skipping the probe for projects that are not stale looked like the cheaper
fix and is unsafe: the deltas would be empty while `probeSettled` stayed
true, and the migrate action in the project menu is not gated on the banner,
so the pre-flight would report nothing to copy while the backend was told to
copy nothing. That is the hazard `canMigrate`'s comment already warns about.
Not done, and written down so it is not tried again.

Also from the review:

- A failed commit no longer costs an answer the snapshot could have given.
  Before this feature a stopped project read its snapshot directly, so
  surfacing this error would have made the banner worse than it was — and
  the failure modes are where the fallback earns its keep: a full disk (the
  commit allocates the whole layer, the snapshot probe allocates nothing)
  and a 409 from a concurrent claim.
- The probe no longer commits while the project is claimed. The collision is
  not symmetric: the probe losing is a retryable `probe_error`, but
  `start_project_container` removes the old container with a hard `?`, so a
  remove that raced a commit would fail the user's Start with an opaque
  error. `stopped_probe_policy` reads `project_lock::held` and probes the
  snapshot instead, or defers with a message that says so.
- The cleanup-failure warning claimed the next probe of the same container
  would reclaim the leftover. Unique names made that false the moment they
  landed; it is `reap_probe_images` that collects it.
- The TS binding still called the command read-only, which is how the
  auto-refresh got added in the first place.
- CLAUDE.md still documented the stable `triple-c-probe-{cid}:latest` name
  this PR removed as unsafe.

548 unit tests, 752 frontend tests, 4 live-Docker tests. Clippy unchanged at
44 warnings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RSaoDLovVV2wmH4H8VVxz
2026-09-10 20:48:10 -07:00
shadowdaoandClaude Opus 5 307ea07409 Read a stopped container instead of claiming there is nothing to read
Secret Scan / scan (push) Successful in 4s
Build App (Preview) / compute-version (pull_request) Successful in 5s
Secret Scan / scan (pull_request) Successful in 6s
Build App (Preview) / create-release (pull_request) Successful in 1s
Build App (Preview) / build-macos (pull_request) Successful in 2m55s
Build App (Preview) / build-linux (pull_request) Successful in 4m56s
Build App (Preview) / build-windows (pull_request) Successful in 5m53s
Build App (Preview) / prune-previews (pull_request) Successful in 1s
A project that was merely stopped reported "This project has no container
or snapshot image yet, so there is nothing to compare against the base
image" — with its container sitting right there — and Update stayed
disabled. Start it and the checks passed, which is the tell: the staleness
probe had only two sources, a *running* container via `docker exec` or the
project's snapshot image.

The snapshot is not a checkpoint. `commit_container_snapshot` runs only
before a container is destroyed (a config-change recreate) or inside a
migration, never on stop, so a project in daily use for a year can have no
snapshot at all — and five of the six projects on the box that reported
this had none. Absence of a snapshot was being read as absence of anything
to inspect.

So probe the stopped container directly: commit its writable layer to a
throwaway image, probe that, drop it. A stopped container now also outranks
the snapshot, for the same reason a running one already did — the snapshot
lags it by everything installed since the last commit. `pick_probe_source`
is the whole decision and is unit-tested; the message it used to emit now
describes only the case it is true of, no container and no snapshot.

Two things found on the way, both documented in CLAUDE.md:

`bollard` never hands back the image id from a commit — its `Commit` model
deserialises "ID" while the daemon sends "Id" — so the probe image has to be
tagged, and a tagged image is dangling-proof and therefore invisible to
`sweep_orphaned_snapshots`, `reap_stale_migration_pins` and
`scrub_secrets_from_snapshots` alike. Without a reaper of its own a crashed
probe would leak a multi-gigabyte image that nothing could ever reclaim, so
`reap_probe_images` runs at startup beside `reap_probe_containers`, age-gated
for the same reason that one is: `reference=` is daemon-wide and a second
instance's live probe matches the glob.

It removes by tag, never by image id: a force removal by id untags an image
everywhere, which is how a first draft of the reaper test deleted an
unrelated `alpine:latest`. Names are unique per call rather than stable per
container, because container ids do not survive a recreate and two
overlapping probes would otherwise fight over one tag.

Verified against the container that reported the bug: 13,365 paths and an
apt delta of cmake, ffmpeg, libobs-dev, qt6-base-dev and nine more — the
migration payload the Update flow could not see. 546 unit tests plus three
live-Docker tests pass; no new clippy warnings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RSaoDLovVV2wmH4H8VVxz
2026-09-10 19:24:05 -07:00
jknapp 3aec2998d8 Merge pull request 'Anchor the update channel tag, and stop shipping a duplicate AppImage' (#52) from fix/update-channel-durability into main
Build App / compute-version (push) Successful in 3s
Secret Scan / scan (push) Successful in 4s
Build App / build-macos (push) Successful in 2m44s
Build App / build-linux (push) Successful in 4m48s
Build App / build-windows (push) Successful in 4m52s
Build App / create-tag (push) Successful in 4s
Build App / sync-to-github (push) Successful in 7s
2026-09-03 16:52:47 +00:00
shadowdao 019fb403d5 Merge remote-tracking branch 'origin/main' into fix/update-channel-durability
Secret Scan / scan (push) Successful in 4s
Build App (Preview) / compute-version (pull_request) Successful in 3s
Secret Scan / scan (pull_request) Successful in 3s
Build App (Preview) / create-release (pull_request) Successful in 1s
Build App (Preview) / build-macos (pull_request) Successful in 2m42s
Build App (Preview) / build-linux (pull_request) Successful in 4m51s
Build App (Preview) / build-windows (pull_request) Successful in 4m54s
Build App (Preview) / prune-previews (pull_request) Successful in 1s
2026-09-03 09:41:22 -07:00
jknapp b21a568bf5 Merge pull request 'Install from the lockfile, so CI cannot be broken by someone else's release' (#53) from fix/ci-npm-lockfile into main
Build App / compute-version (push) Successful in 4s
Secret Scan / scan (push) Successful in 3s
Build App / build-macos (push) Successful in 2m42s
Build App / build-windows (push) Successful in 4m53s
Build App / build-linux (push) Successful in 5m0s
Build App / create-tag (push) Successful in 3s
Build App / sync-to-github (push) Successful in 13s
2026-09-03 16:41:15 +00:00
shadowdaoandClaude Opus 5 f41b1d9054 Install from the lockfile, so CI cannot be broken by someone else's release
Secret Scan / scan (push) Successful in 4s
Build App (Preview) / compute-version (pull_request) Successful in 3s
Secret Scan / scan (pull_request) Successful in 3s
Build App (Preview) / create-release (pull_request) Successful in 1s
Build App (Preview) / build-macos (pull_request) Successful in 2m40s
Build App (Preview) / build-windows (pull_request) Successful in 4m52s
Build App (Preview) / build-linux (pull_request) Successful in 5m0s
Build App (Preview) / prune-previews (pull_request) Successful in 2s
`build-linux` fails before `tauri build` runs, on every workflow, at "Install
frontend dependencies":

    npm error Cannot read properties of null (reading 'edgesOut')

Reproduced exactly on the first attempt by running the step's own commands
locally on the same Node 22.23.2 the runner installs. The debug log gives the
frame the CI output omits:

    at #loadPeerSet (.../@npmcli/arborist/lib/arborist/build-ideal-tree.js:1289:38)

It is a null dereference in npm 10.9.8's peer-set resolver, reached through
vite → @vitejs/devtools → @vitejs/devtools-vitest → vitest@* →
@vitest/browser-playwright → vitest@4.1.11 → jsdom@* → canvas.

**Nothing in this repo changed to cause it.** The step deleted
`package-lock.json` before installing, so every build re-resolved the entire
tree against the registry against ranges like `vitest@*`. A dependency
published a version that produces a peer graph npm cannot resolve, and our CI
broke — the same command succeeded fifteen hours earlier for 0.4.21. That is
the real defect: the build was never reproducible, and the crash is only how we
found out.

So Linux installs with `npm ci`, from the committed lockfile, like Windows
already did. macOS moves too — it kept the lockfile but still ran `npm
install`, which is free to re-resolve; all three platforms now install
identically and none can re-resolve mid-release.

**The reason the lockfile was being deleted is obsolete, not ignored.** 2d4fce9
removed it "to ensure correct platform-specific bindings", which was a real
problem once. The committed lockfile now records 25 rollup platform variants,
and `npm ci` on Linux installs precisely rollup-linux-x64-{gnu,musl} and
@esbuild/linux-x64 — checked directly. A comment on the step says so, and says
not to reach for deleting the lockfile again: if `npm ci` refuses, package.json
and the lockfile have genuinely diverged and the fix is to commit an updated
lockfile.

Verified from the resulting tree: `tsc --noEmit` clean, `npm run build`
successful, 752 tests across 62 files passing. The `npx tauri --version ||
npm install @tauri-apps/cli` fallback in the next step cannot reintroduce a
fresh resolution — the CLI is a pinned devDependency that `npm ci` installs, so
the fallback is unreachable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011YPqHpjV4EL6RNEwrRKqQm
2026-09-03 09:28:09 -07:00
shadowdaoandClaude Opus 5 d38736007f Take the re-review: distinguish "absent" from "unreachable"
Secret Scan / scan (push) Successful in 3s
Build App (Preview) / compute-version (pull_request) Successful in 3s
Secret Scan / scan (pull_request) Successful in 4s
Build App (Preview) / create-release (pull_request) Successful in 1s
Build App (Preview) / build-linux (pull_request) Failing after 2m3s
Build App (Preview) / build-macos (pull_request) Successful in 2m41s
Build App (Preview) / build-windows (pull_request) Successful in 4m56s
Build App (Preview) / prune-previews (pull_request) Skipped
Second review of this branch. Two blockers and one real defect I had papered
over with a true-but-misleading claim.

**`make_latest` was missing from the republish path.** The create path sends
`"make_latest": "false"` so the channel cannot displace the versioned release
on the releases page. The reuse path — taken on every run after the first —
omitted it, and the API's documented default for a publish transition is
`true`. So the second release would have quietly promoted `linux-latest` to
the repository's Latest release: a release whose own body says "for a specific
version, use the versioned releases instead". Now sent on both paths.
`tag_name` is re-sent deliberately and now says so in a comment — the API
removes the tag when a PATCH omits it, and this branch exists because a tag
disappeared.

**A transient Gitea error would have cost the whole release.** `curl -sf`
fails identically for "404, the tag is genuinely absent" and "503, Gitea is
briefly unreachable", and both landed in the create branch. Creating a tag that
already exists returns 409, which aborted the last step of `build-linux` — and
`create-tag` and `sync-to-github` both depend on it, so no version tag and no
GitHub sync at all. The failure message also read "the tag does not exist" when
Gitea had merely been unreachable. Now a `case` on the HTTP code — 200 leave
alone, 404 create, anything else fail loudly with the real code — the same
idiom `Upload to Gitea release` already uses two steps above. `422
already_exists` on the release POST is likewise a recoverable answer, not a
reason to lose a release.

**The empty `Categories=` was still shipping, and my claim hid it.** I wrote
that the guard "asserts the absence of an empty value rather than the presence
of any filled one" — true of the regex, false of the artifact. The AppDir root
`.desktop` is a *symlink* into usr/share/applications, so `sed -i` replaced the
link with a regular file and left the real entry empty; the guard globbed the
root only, so it saw the copy it had just written and passed. Verified on the
real artifact: two divergent entries, and the one that shipped was empty. Fixed
with `--follow-symlinks`, both locations globbed, and the guard turned into a
positive assertion over every entry — which also closes its missing-key and
unmatched-glob holes. Both entries now read `Categories=Development;Utility;`.

Also taken: the duplicate-AppImage check moves to a precondition, since as a
post-mortem it let the script repack and overwrite the versioned artifact
before failing, and it silently selected by glob order, i.e. the older version
— it now refuses in under a second; assets are deleted and re-uploaded one at
a time, because deleting both up front left a fresh AppImage with no .zsync if
the second upload failed, which silently stops every client; and the success
line no longer claims a fallback was kept when there was nothing to demote.

Left as informational, with the reasoning recorded rather than acted on:
`--retry-all-errors` retries permanent 4xx (fail-closed, matches the repo's
other upload steps); the release list is unpaginated (a GraphQL lookup by
pending tag name is the durable fix, but 7 releases is decades from the cliff,
and the 422 handling above covers the failure mode); process-substitution
failure is invisible to `mapfile` (fail-closed downstream).

Verified against the real 0.4.19 artifact — happy path, no AppImage, two
AppImages, and an AppDir rebuilt with the bundled library removed. shellcheck
clean at warning level on both scripts. appimagetool now reports the AppStream
metadata found.

Nothing here is CI-proven, and that is worth stating plainly: `build-linux`
fails on this branch before `tauri build` even runs, at "Install frontend
dependencies" with `npm error Cannot read properties of null (reading
'edgesOut')` — confirmed in the logs of jobs 5644 and 5636. Unrelated to this
change and tracked separately, but it means the finalizer has never executed
in CI on either commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011YPqHpjV4EL6RNEwrRKqQm
2026-09-03 09:17:34 -07:00
shadowdaoandClaude Opus 5 63f282bef6 Fix the review findings: never destroy a working anchor
Secret Scan / scan (push) Successful in 4s
Build App (Preview) / compute-version (pull_request) Successful in 3s
Secret Scan / scan (pull_request) Successful in 4s
Build App (Preview) / create-release (pull_request) Successful in 1s
Build App (Preview) / build-linux (pull_request) Failing after 1m49s
Build App (Preview) / build-macos (pull_request) Successful in 2m41s
Build App (Preview) / build-windows (pull_request) Successful in 4m55s
Build App (Preview) / prune-previews (pull_request) Skipped
An adversarial review of the previous commit found six real problems and
corrected one of my claims. Taking all of it.

**The anchoring could kill the channel it exists to protect.** It did
DELETE-then-POST so the tag would name the current build. If the POST failed
for any transient reason the script aborted having already deleted the anchor a
previous run put there, and the next mirror run pruned GitHub's copy — a
transient Gitea error converting a healthy channel into a dead one, which is
strictly worse than the step not existing. There was also a real window
between the two calls with no tag at all.

The DELETE bought nothing. The update string resolves the tag by *name* and the
assets hang off the release object, so nothing about the channel depends on
which commit the tag points at; moving it changes only the source-zip link.
It existed solely to get past a 409, since Gitea's POST /tags has no force
semantics. Now the tag is created if absent and otherwise left alone, which
removes the window too.

**My "no window where the two disagree" claim was wrong, and it is the third
time in this area I have asserted something I had not established.** The
release POST sets no `target_commitish`, so GitHub creates its tag at its own
default-branch HEAD, not at `GITEA_SHA`; the two agree only because
`sync_on_commit` pushes main minutes earlier. And the DELETE actively created
the window. What the ordering genuinely buys is narrower: if anchoring fails,
the script aborts before creating a GitHub release that would be orphaned.

**Orphaned drafts were invisible to the release lookup.** GitHub demotes a
release to a draft when its tag is deleted, and `/releases/tags/` never returns
drafts — precisely the state every mirror run left behind. The by-tag lookup
reported "absent" while 86 MB drafts accumulated, one per release. The lookup
now reads the authenticated list, republishes the newest, and deletes the rest.

**A guard that could not catch what it named.** The update-info assertion was
a substring match on the tag, so it passed for a wrong host, path, filename or
transport — verified: an `evil.example.com/.../linux-latest/...` string passes
the old check and fails the new one. Now a fixed full-string match.

Also from the review: an absent bundled library no longer exits early, because
that skipped the metadata *and* left `update-channel/` uncreated, killing the
publish step on a missing directory and taking the tag and mirror jobs with it;
the Categories guard asserts the absence of an empty value rather than the
presence of any filled one; the channel directory is cleared before use so a
stale zsync cannot satisfy an existence check while describing the previous
build; the AppImage count uses a glob array, since `ls | wc -l` aborted under
pipefail before the message it promised could print; uploads carry the
retry/http1.1 hardening this repo's other upload steps already learned to
need; verification compares served size against built size, because a status
code only proves something is served; and the release workflow now fails on
empty artifacts instead of publishing a release with no AppImage.

The metainfo file is installed as `Triple-C.appdata.xml`. appimagetool derives
the name it looks for from the .desktop basename, so under the id-based name it
warned the metadata was missing on every build while this script reported it
present. Now it prints "AppStream upstream metadata found in
usr/share/metainfo/Triple-C.appdata.xml" — the AppStream id inside the file is
unchanged and is what identifies the component.

Two review hypotheses did not hold and nothing was changed for them: `set -e`
does not abort on a failing `&&` list mid-script, and my claim of a `trap`
reassignment was wrong — there is one trap, installed once.

Verified against the real 0.4.19 artifact: exit 0, one AppImage beside the
release, channel pair in its own directory, appimagetool reporting the metadata
found, and the wayland fallback intact. Guards exercised individually — the
duplicate one bites, the exact-match one rejects an impostor carrying the tag,
the empty directory reports cleanly, and all four publisher preconditions
refuse rather than half-publishing. Header parsing for the size check was
tested against a real redirecting GitHub asset URL.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011YPqHpjV4EL6RNEwrRKqQm
2026-09-03 08:53:13 -07:00
shadowdaoandClaude Opus 5 d561ce03d5 Anchor the update channel tag, and stop shipping a duplicate AppImage
Secret Scan / scan (push) Successful in 6s
Build App (Preview) / compute-version (pull_request) Successful in 3s
Secret Scan / scan (pull_request) Successful in 4s
Build App (Preview) / create-release (pull_request) Successful in 1s
Build App (Preview) / build-linux (pull_request) Failing after 1m49s
Build App (Preview) / build-macos (pull_request) Successful in 2m57s
Build App (Preview) / build-windows (pull_request) Successful in 16m16s
Build App (Preview) / prune-previews (pull_request) Skipped
Two defects in the update channel, both visible in 0.4.20 and 0.4.21.

**The channel tag does not survive.** `publish-update-channel.sh` created the
GitHub release, uploaded both assets and verified each URL returned 200 — the
job log shows it succeeding at 00:38. By 13:04 the tag was gone and every
installed copy was checking a 404.

Gitea push-mirrors this repo to GitHub every four hours, and a mirror push
deletes remote refs with no local counterpart. `linux-latest` was created by
GitHub's release API and never existed as a Gitea tag, so the mirror removed
it. Versioned tags were never affected because `create-tag` creates them in
Gitea first.

So the tag is now anchored in Gitea, and before the GitHub release rather than
after, so there is no window where the two disagree. Its absence fails the
step instead of warning, because it is the only thing keeping the channel
alive. Worth stating plainly: publishing correctly is not evidence the channel
still works, and the verification that passed at 00:38 could not have caught a
failure that arrives twelve hours later.

**Every release carried the AppImage twice.** The channel's stable-named copy
sat beside the versioned one, where the release job's `*.AppImage` glob picked
it up — so v0.4.21 published `Triple-C_0.4.21_amd64.AppImage` and
`Triple-C_x86_64.AppImage`, byte-identical at 86,686,200 bytes each, and
`sync-to-github` copied both to the mirror. 80 MB of duplicate per release,
under a name that reads like a different build. That is how it was noticed.

The channel pair now lives in `bundle/appimage/update-channel/`, out of the
glob's reach, and a guard fails the build if more than one AppImage is left
beside the release. Verified by planting a second one: it fails.

One appimagetool quirk found while moving it — zsyncmake writes the .zsync
into the working directory, not beside the image it describes, so it has to be
collected rather than assumed in place. The existing guard caught that too.

Verified against the real 0.4.19 artifact: exactly one AppImage at top level,
the channel pair in its own directory, update string still resolving to the
fixed tag, and the wayland fallback intact.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011YPqHpjV4EL6RNEwrRKqQm
2026-09-03 08:29:58 -07:00
10 changed files with 1251 additions and 80 deletions
+32 -5
View File
@@ -299,8 +299,34 @@ jobs:
- name: Install frontend dependencies
working-directory: ./app
run: |
rm -rf node_modules package-lock.json
npm install
# `npm ci` — from the lockfile, never resolving afresh.
#
# This used to be `rm -rf node_modules package-lock.json && npm
# install`, which deleted the lockfile "to ensure correct
# platform-specific bindings" (2d4fce9). That made every build
# re-resolve the whole tree against the registry, so a dependency
# publishing a new version could break CI with no change to this
# repo — and one did. Deleting the lockfile then hit a null
# dereference in npm 10.9.8's arborist peer-set resolver:
#
# npm error Cannot read properties of null (reading 'edgesOut')
# at #loadPeerSet (.../build-ideal-tree.js:1289:38)
#
# reached through vite → @vitejs/devtools → @vitejs/devtools-vitest
# → vitest@* → @vitest/browser-playwright → jsdom@* → canvas.
# Reproduced exactly by removing the lockfile locally on the same
# Node 22.23.2 the runner installs.
#
# The binding worry is obsolete: the committed lockfile records 25
# rollup platform variants, and `npm ci` on Linux installs precisely
# rollup-linux-x64-{gnu,musl} and @esbuild/linux-x64. Verified, along
# with a clean tsc, a successful build and 752 passing tests from the
# resulting tree.
#
# Do not "fix" a future dependency error by deleting the lockfile
# again. If `npm ci` refuses, package.json and the lockfile have
# genuinely diverged, and the fix is to commit an updated lockfile.
npm ci
- name: Install Tauri CLI
working-directory: ./app
@@ -335,7 +361,6 @@ jobs:
run: |
mkdir -p artifacts
cp app/src-tauri/target/release/bundle/appimage/*.AppImage artifacts/ 2>/dev/null || true
cp app/src-tauri/target/release/bundle/appimage/*.zsync artifacts/ 2>/dev/null || true
ls -la artifacts/
# Assets, not workflow artifacts — see the note at the top of this file.
@@ -427,8 +452,10 @@ jobs:
- name: Install frontend dependencies
working-directory: ./app
run: |
rm -rf node_modules
npm install
# `npm ci` here too, so all three platforms install identically and
# none of them can re-resolve the tree mid-release. Windows already
# did. See the Linux job for what a fresh resolution cost us.
npm ci
- name: Install Tauri CLI
working-directory: ./app
+51 -6
View File
@@ -172,8 +172,34 @@ jobs:
- name: Install frontend dependencies
working-directory: ./app
run: |
rm -rf node_modules package-lock.json
npm install
# `npm ci` — from the lockfile, never resolving afresh.
#
# This used to be `rm -rf node_modules package-lock.json && npm
# install`, which deleted the lockfile "to ensure correct
# platform-specific bindings" (2d4fce9). That made every build
# re-resolve the whole tree against the registry, so a dependency
# publishing a new version could break CI with no change to this
# repo — and one did. Deleting the lockfile then hit a null
# dereference in npm 10.9.8's arborist peer-set resolver:
#
# npm error Cannot read properties of null (reading 'edgesOut')
# at #loadPeerSet (.../build-ideal-tree.js:1289:38)
#
# reached through vite → @vitejs/devtools → @vitejs/devtools-vitest
# → vitest@* → @vitest/browser-playwright → jsdom@* → canvas.
# Reproduced exactly by removing the lockfile locally on the same
# Node 22.23.2 the runner installs.
#
# The binding worry is obsolete: the committed lockfile records 25
# rollup platform variants, and `npm ci` on Linux installs precisely
# rollup-linux-x64-{gnu,musl} and @esbuild/linux-x64. Verified, along
# with a clean tsc, a successful build and 752 passing tests from the
# resulting tree.
#
# Do not "fix" a future dependency error by deleting the lockfile
# again. If `npm ci` refuses, package.json and the lockfile have
# genuinely diverged, and the fix is to commit an updated lockfile.
npm ci
- name: Install Tauri CLI
working-directory: ./app
@@ -200,10 +226,23 @@ jobs:
- name: Collect artifacts
run: |
mkdir -p artifacts
# The versioned AppImage only. The update channel's copy lives in
# bundle/appimage/update-channel/ precisely so this glob cannot pick
# it up and publish an 80 MB duplicate under a second name.
cp app/src-tauri/target/release/bundle/appimage/*.AppImage artifacts/ 2>/dev/null || true
cp app/src-tauri/target/release/bundle/appimage/*.zsync artifacts/ 2>/dev/null || true
ls -la artifacts/
# A green job that published nothing is the worst outcome available:
# the release exists, carries no AppImage, and nobody is told. The
# `|| true` above is there so a missing bundle does not mask the real
# error, which makes this check the thing that catches it.
shopt -s nullglob
collected=(artifacts/*)
if [ ${#collected[@]} -eq 0 ]; then
echo "No artifacts collected — the bundler produced nothing." >&2
exit 1
fi
- name: Upload to Gitea release
if: gitea.event_name == 'push'
env:
@@ -286,7 +325,11 @@ jobs:
if: gitea.event_name == 'push'
env:
GH_PAT: ${{ secrets.GH_PAT }}
run: bash scripts/publish-update-channel.sh artifacts
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
GITEA_SHA: ${{ gitea.sha }}
run: |
bash scripts/publish-update-channel.sh \
app/src-tauri/target/release/bundle/appimage/update-channel
build-macos:
runs-on: macos-latest
@@ -343,8 +386,10 @@ jobs:
- name: Install frontend dependencies
working-directory: ./app
run: |
rm -rf node_modules
npm install
# `npm ci` here too, so all three platforms install identically and
# none of them can re-resolve the tree mid-release. Windows already
# did. See the Linux job for what a fresh resolution cost us.
npm ci
- name: Install Tauri CLI
working-directory: ./app
+57
View File
@@ -436,6 +436,63 @@ security update. Migration is the non-destructive way out; Reset is the destruct
bump: churn on the old base, and it would consume the "you should migrate" signal without
migrating. `get_container_staleness` surfaces it; `migrate_project_to_base` acts on it.
- **A missing lineage label means "unknown, probe instead", never "stale".**
- **The snapshot image is not a checkpoint — never read its absence as "nothing to inspect".**
`commit_container_snapshot` runs only before a container is destroyed (a config-change recreate)
or inside a migration. **Never on stop.** So a project in daily use for a year can legitimately
have no `triple-c-snapshot-{id}:latest` at all, and one that has is stale by everything installed
since. `pick_probe_source` therefore reads a *stopped* container directly — commit its writable
layer to a unique `triple-c-probe-*` image, probe that, drop it — and ranks it **above** the snapshot,
for the same reason a running container already outranked it. Assuming a snapshot existed is what
made a stopped, never-recreated project report "no container or snapshot image yet" with its
container sitting right there, and left Update disabled on the projects furthest behind.
- **`bollard` never gives you the image id back from a commit.** Its `Commit` response model
deserialises `"ID"`; the daemon sends `"Id"`, so `commit_container` returns `id: None` every time
(verified: bollard 0.18.1, Engine 29.6). Neither long-standing commit site notices because both
discard the response — but it means any commit you need a *reference* to has to be **tagged**.
- **A tagged leftover is the one orphan no sweep can reach, so the probe image has its own reaper.**
`sweep_orphaned_snapshots` collects `dangling` + `triple-c.managed=true`; `reap_stale_migration_pins`
and `scrub_secrets_from_snapshots` both filter `triple-c-snapshot-*`. A `triple-c-probe-*` image is
tagged and so matches none of them, which would make a crashed probe a permanent multi-gigabyte
leak with no UI to find it. `reap_probe_images` runs at startup beside `reap_probe_containers` and
is **load-bearing, not tidying** — it is also what makes the probe image's unscrubbed writable
layer acceptable. Two rules it earned the hard way:
- **Age-gate it** (`PROBE_REAP_MIN_AGE_SECS`, same as the container reaper). `reference=` is
daemon-wide, so a second copy of the app has live probe images matching the glob.
- **Remove by tag, never by image id.** A `force` removal by id untags an image *everywhere*; a
fixture that tagged `alpine:latest` into this namespace deleted the user's alpine that way.
- **Probe image names are unique per call, and must stay that way.** A stable per-container name was
tried: container ids do not survive a recreate, so most leftovers were stranded permanently, and
two concurrent probes fought over one tag — whichever finished first force-removed the image the
other was still reading, reporting a bogus `probe_error` on a healthy project. `get_container_staleness`
takes no `project_lock` claim (the migration banner needs it to answer *during* a migration), so
uniqueness is what makes overlapping probes safe.
- **The stopped-container probe is cached per stop, and that is not an optimisation you may drop.**
`getContainerStaleness` is called from a `useEffect` that fires whenever the container settles, so
merely opening a stopped project's Overview probes it. Uncached that is a `docker commit` of the
whole writable layer per visit — measured at 44 s on a real project, against ~3 s for the snapshot
probe it replaced. `STOPPED_MANIFEST_CACHE` is keyed on the container's `FinishedAt`, which is
exact rather than merely plausible: nothing can write to a stopped container's writable layer, and
`FinishedAt` moves on every stop. A live test asserts the restart case, because a cache that
failed to invalidate would plan a migration against a filesystem the project no longer has.
- **Do not "skip the probe when the project is not stale" to save that cost.** It was tried. The
deltas would be empty while `probeSettled` (`!probing && staleness && !probe_error`) stayed *true*,
which leaves the migrate action in the project menu enabled — that action is not gated on the
banner — so the pre-flight would report nothing to copy while the backend was told to copy
nothing. That is the exact hazard `ProjectHome.tsx`'s `canMigrate` comment already warns about.
- **A failed stopped-container probe falls back to the snapshot whenever one exists.** Before this
feature a stopped project read its snapshot directly, so surfacing a commit failure where the
snapshot could have answered would make the banner *worse* than it was — and the failure modes are
exactly the ones where the fallback earns its keep: a full disk (the commit allocates the whole
writable layer; the snapshot probe allocates nothing) and a 409 from a concurrent claim.
- **`get_container_staleness` never commits while the project is claimed.** It takes no
`project_lock` claim itself, deliberately — the banner has to answer *during* a migration — so it
reads `project_lock::held` instead and probes the snapshot rather than the container. The
collision is not symmetric: the probe losing is a retryable `probe_error`, but
`start_project_container` removes the old container with a hard `?`, so a remove that raced a
commit would fail the user's Start with an opaque error.
- **An image's `Created` is the image's own, not its tag's.** Tagging an existing image gives you
that image's age; BuildKit stamps `docker build` output with a fixed epoch. Only `docker commit`
stamps *now* — which is what real probe images do, and what any fixture for them must do.
- **`:latest` keeps pointing at the old lineage until the final commit.** That is what makes every
crash before that point self-heal — `start_project_container` just recreates from the old
snapshot. After the container swap, the new container's `triple-c.migration-state=in-progress`
+213 -11
View File
@@ -92,8 +92,111 @@ fn pick_recorded_lineage(
.or_else(|| from_snapshot.filter(|v| !v.is_empty()))
}
/// Read-only. Runs two filesystem probes (~3 s each) and is therefore meant to
/// be called on demand, not polled.
/// Reported as `probe_error` when there is genuinely nothing to read: no
/// container, stopped or otherwise, and no snapshot image.
///
/// It used to be reported for a *stopped* container too, which was simply
/// untrue — the container was sitting right there — and it disabled Update on
/// exactly the long-lived projects that had never been recreated and so had no
/// snapshot to fall back on.
const NOTHING_TO_PROBE: &str = "This project has no container or snapshot image yet, so there is nothing to compare against the base image.";
/// Where [`get_container_staleness`] reads the project's *current* filesystem
/// from, in descending order of how current the answer is.
#[derive(Debug, PartialEq, Eq)]
enum ProbeSource {
/// `docker exec` into the live container. The only source that includes
/// everything installed since the last commit *in this session*.
RunningContainer,
/// Commit the stopped container's writable layer to a throwaway image and
/// probe that. Exactly as current as the container, which is what makes it
/// preferable to the snapshot — see below.
StoppedContainer,
/// A throwaway container from `triple-c-snapshot-<id>:latest`.
Snapshot,
/// Nothing to read: no container, no snapshot.
Nothing,
}
/// Pick the probe source. `container_running` is `None` when the project has no
/// container at all, `Some(false)` when it has a stopped one.
///
/// **A stopped container outranks the snapshot.** The snapshot image is not a
/// checkpoint — `commit_container_snapshot` runs only before a removal (a
/// config-change recreate) or inside a migration, so a project that has never
/// hit either has *no snapshot at all*, however long it has been in use, and
/// one that has is stale by everything installed since. The container's
/// writable layer is the truth in both cases. This is the same argument
/// [`mig::manifest_from_container`] already makes for the running case; it does
/// not stop applying when the container is stopped.
///
/// Getting this wrong is what made a stopped, never-recreated project report
/// "no container or snapshot image yet" — with its container sitting right
/// there — and left Update disabled on the projects that most needed it.
fn pick_probe_source(container_running: Option<bool>, snapshot_exists: bool) -> ProbeSource {
match (container_running, snapshot_exists) {
(Some(true), _) => ProbeSource::RunningContainer,
(Some(false), _) => ProbeSource::StoppedContainer,
(None, true) => ProbeSource::Snapshot,
(None, false) => ProbeSource::Nothing,
}
}
/// Reported as `probe_error` when another operation owns the project and there
/// is no snapshot image to read instead. Deliberately not a claim about the
/// container: nothing is wrong with it, the answer is simply not safe to take
/// right now. See [`stopped_probe_policy`].
const PROJECT_BUSY: &str = "Another operation is running on this project, so its contents could not be inspected. Try again once it finishes.";
/// What to do about a stopped container, whose probe is the expensive one: it
/// commits the writable layer before it can read anything.
#[derive(Debug, PartialEq, Eq)]
enum StoppedProbe {
/// Commit and probe. The current answer, and the default.
Commit,
/// Probe the snapshot image instead. Less current — it lags the container by
/// everything installed since the last commit — but it allocates nothing and
/// touches nothing, which is what makes it the right answer while another
/// operation owns the container.
SnapshotInstead,
/// Report rather than guess.
Defer,
}
/// Pick what to do about a stopped container.
///
/// **Never commits while the project is claimed.** `get_container_staleness`
/// takes no [`crate::project_lock`] claim of its own, by design, so a commit
/// here can overlap a Recreate or Reset — and the collision is not symmetric.
/// The probe losing is harmless: a surfaced `probe_error` the user retries. The
/// *recreate* losing is not, because `start_project_container` removes the old
/// container with a hard `?`, so a non-404 from a remove that raced this commit
/// fails the whole Start with an opaque "Failed to remove container". Reading
/// the claim costs nothing and takes that failure off the table.
fn stopped_probe_policy(project_is_busy: bool, snapshot_exists: bool) -> StoppedProbe {
match (project_is_busy, snapshot_exists) {
(false, _) => StoppedProbe::Commit,
(true, true) => StoppedProbe::SnapshotInstead,
(true, false) => StoppedProbe::Defer,
}
}
/// Runs two filesystem probes (~3 s each) and is therefore meant to be called
/// on demand, not polled.
///
/// **Not read-only, despite only reporting.** The stopped-container path commits
/// a throwaway image and force-removes it, which makes this a writer of a
/// `triple-c-probe-*` image and puts it in the class of thing
/// [`crate::project_lock`] exists for — and it takes no claim. That is
/// deliberate: this is what the migration banner calls to decide whether to
/// offer an update, including while a migration is in flight, so refusing it
/// under a claim would blank the banner exactly when it has the most to say.
/// The exposure is bounded to a surfaced error — a concurrent Recreate, Reset or
/// migration can remove the container out from under the commit, and the result
/// is a `probe_error` the user can retry, never a damaged container or a
/// mislabelled image. Two overlapping probes cannot collide either, because
/// probe image names are unique per call; see
/// [`crate::docker::container::get_probe_image_name`].
#[tauri::command]
pub async fn get_container_staleness(
project_id: String,
@@ -145,16 +248,62 @@ pub async fn get_container_staleness(
};
// ── Probes ───────────────────────────────────────────────────────────
let running = match &container_id {
Some(id) => docker::is_container_running(id).await.unwrap_or(false),
None => false,
let container_running = match &container_id {
Some(id) => Some(docker::is_container_running(id).await.unwrap_or(false)),
None => None,
};
let from_manifest = if running {
mig::manifest_from_container(container_id.as_ref().unwrap()).await
} else if docker::image_exists(&snapshot_image).await.unwrap_or(false) {
mig::manifest_from_image(&snapshot_image).await
} else {
Err("This project has no container or snapshot image yet, so there is nothing to compare against the base image.".to_string())
let snapshot_exists = docker::image_exists(&snapshot_image).await.unwrap_or(false);
let from_manifest = match (
pick_probe_source(container_running, snapshot_exists),
&container_id,
) {
(ProbeSource::RunningContainer, Some(id)) => mig::manifest_from_container(id).await,
(ProbeSource::StoppedContainer, Some(id)) => {
let busy = crate::project_lock::held(&project_id).is_some();
match stopped_probe_policy(busy, snapshot_exists) {
StoppedProbe::Commit => {
match mig::manifest_from_stopped_container_cached(id).await {
Ok(m) => Ok(m),
// **Never let a failed commit cost an answer the
// snapshot could have given.** Before stopped
// containers were readable at all, a stopped project
// fell straight through to its snapshot, so surfacing
// this error where the snapshot exists would make the
// banner *worse* than it was — and the ways this fails
// are the ones where the fallback matters most: a full
// disk (the commit has to allocate the whole writable
// layer; the snapshot probe allocates nothing) and a
// 409 from an operation that claimed the project after
// the check above.
Err(e) if snapshot_exists => {
log::warn!(
"Probing the stopped container for project {} failed ({}) — \
falling back to its snapshot image, which may lag it",
project_id,
e
);
mig::manifest_from_image(&snapshot_image).await
}
Err(e) => Err(e),
}
}
StoppedProbe::SnapshotInstead => {
log::info!(
"Project {} is claimed by another operation — probing its snapshot image \
rather than committing the container",
project_id
);
mig::manifest_from_image(&snapshot_image).await
}
StoppedProbe::Defer => Err(PROJECT_BUSY.to_string()),
}
}
(ProbeSource::Snapshot, _) => mig::manifest_from_image(&snapshot_image).await,
// `container_running` is `Some` exactly when `container_id` is, so the
// two arms above are the only ones those variants can reach. This arm
// is `ProbeSource::Nothing` — and now *only* that: it used to also
// swallow every stopped container, which is the bug.
(_, _) => Err(NOTHING_TO_PROBE.to_string()),
};
let (from_manifest, base_manifest) = match from_manifest {
@@ -1964,6 +2113,59 @@ mod tests {
assert_eq!(pick_recorded_lineage(some(""), None), None);
}
#[test]
fn a_stopped_container_is_probed_rather_than_reported_missing() {
// The regression: a container that exists but is stopped, with no
// snapshot ever taken, read as "nothing to compare against".
assert_eq!(
pick_probe_source(Some(false), false),
ProbeSource::StoppedContainer
);
}
#[test]
fn the_container_outranks_the_snapshot_whether_or_not_it_is_running() {
// The snapshot lags the container by everything installed since the
// last commit, in both states.
assert_eq!(
pick_probe_source(Some(true), true),
ProbeSource::RunningContainer
);
assert_eq!(
pick_probe_source(Some(false), true),
ProbeSource::StoppedContainer
);
}
#[test]
fn the_snapshot_is_the_fallback_only_once_the_container_is_gone() {
assert_eq!(pick_probe_source(None, true), ProbeSource::Snapshot);
}
#[test]
fn nothing_to_probe_is_reserved_for_no_container_and_no_snapshot() {
// The one case the "no container or snapshot image yet" message may
// still describe.
assert_eq!(pick_probe_source(None, false), ProbeSource::Nothing);
}
#[test]
fn a_stopped_container_is_committed_only_when_nothing_else_owns_the_project() {
assert_eq!(stopped_probe_policy(false, false), StoppedProbe::Commit);
assert_eq!(stopped_probe_policy(false, true), StoppedProbe::Commit);
}
#[test]
fn a_busy_project_falls_back_rather_than_racing_a_recreate() {
// The snapshot lags, but a stale answer beats failing someone's Start.
assert_eq!(
stopped_probe_policy(true, true),
StoppedProbe::SnapshotInstead
);
// Nothing to fall back to: say so instead of committing anyway.
assert_eq!(stopped_probe_policy(true, false), StoppedProbe::Defer);
}
#[test]
fn byte_sizes_read_the_way_a_disk_warning_should() {
assert_eq!(human_bytes(512), "512 B");
+140 -4
View File
@@ -3052,6 +3052,118 @@ fn blanked_secret_env() -> Vec<String> {
.collect()
}
/// Image-name prefix for the throwaway commit a staleness probe of a stopped
/// container makes. The reaper's only handle on a leftover — see
/// [`crate::docker::migration::reap_probe_images`] — so nothing else may use it.
pub const PROBE_IMAGE_PREFIX: &str = "triple-c-probe-";
/// The throwaway image a staleness probe of a **stopped** container commits to.
///
/// **Unique per call**, and both halves of the name earn their place: the
/// container id prefix makes a leftover traceable in `docker images`, and the
/// counter makes two overlapping probes independent.
///
/// An earlier version of this was deliberately *stable* per container, on the
/// theory that the next probe would move the tag off an abandoned image and
/// leave it dangling for [`sweep_orphaned_snapshots`]. That was wrong twice
/// over. A container id does not survive a recreate, so for most leftovers
/// there is no "next probe of the same container" and the image was stranded
/// permanently; and a stable name made two concurrent probes fight over one
/// tag, where whichever finished first force-removed the image the other was
/// still reading and turned a healthy project into a bogus `probe_error`.
/// Uniqueness fixes both, and [`crate::docker::migration::reap_probe_images`]
/// is what collects the leftovers instead.
pub fn get_probe_image_name(container_id: &str) -> String {
use std::sync::atomic::{AtomicU64, Ordering};
static SEQ: AtomicU64 = AtomicU64::new(0);
let short: String = container_id.chars().take(12).collect();
let nanos = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0);
format!(
"{}{}-{}-{}:latest",
PROBE_IMAGE_PREFIX,
short,
nanos,
SEQ.fetch_add(1, Ordering::Relaxed)
)
}
/// Commit a **stopped** container's filesystem to a throwaway image, returning
/// its name. The caller owns the image and must remove it.
///
/// This exists so a stopped project can be read at all. `docker exec` needs a
/// running container and the snapshot image is not a checkpoint — see
/// [`crate::commands::migration_commands`]'s probe-source pick — so without
/// this there is no way to see inside a project that is merely stopped.
///
/// ## Why it is tagged at all
///
/// An untagged commit would be tidier: untagged plus the `triple-c.managed=true`
/// that `docker commit` copies off the container is exactly the pair
/// [`sweep_orphaned_snapshots`] already collects, so a leftover would self-heal
/// with no new machinery. **It is not available.** `bollard`'s `Commit` response
/// model deserialises `"ID"` while the daemon sends `"Id"`, so
/// `commit_container` hands back `id: None` every time and there is no
/// reference left to probe. Neither existing commit site notices, because both
/// discard the response. Verified against Engine 29.6, bollard 0.18.1.
///
/// So the image needs a name, a tagged image is not dangling, and the sweep
/// therefore cannot be the safety net. [`crate::docker::migration::reap_probe_images`]
/// is, and [`get_probe_image_name`] carries the rest of that argument.
///
/// ## What is in the image, and what is not
///
/// `pause: false` because nothing is running — pausing a stopped container is
/// an error, the same reason [`recommit_without_secrets`]'s scratch commit
/// passes `false`.
///
/// Secrets are blanked from the env for the same reason
/// [`commit_container_snapshot`] blanks them: the commit bakes the container's
/// full ENV into the image, and "it only lives a few seconds" is not a property
/// this function can promise after a crash.
///
/// **The writable layer is committed unscrubbed, and that is unavoidable here.**
/// [`commit_container_snapshot`] runs [`scrub_writable_layer`] first precisely
/// because a commit stacks a layer and never rewrites one — but that scrub is a
/// `docker exec`, which is exactly what a stopped container cannot serve, and
/// scrubbing is not wanted anyway: the probe's whole job is to report the
/// filesystem as it actually is. What makes it acceptable is that this copies
/// bytes that are *already on this disk* in the container's own writable layer,
/// into an image that is never pushed, never created from, and reaped — so it
/// duplicates data inside one trust domain rather than widening it. That
/// argument depends on the reaping actually happening; treat
/// [`crate::docker::migration::reap_probe_images`] as load-bearing, not tidying.
pub async fn commit_container_for_probe(container_id: &str) -> Result<String, String> {
let docker = get_docker()?;
let image_name = get_probe_image_name(container_id);
let (repo, tag) = image_name
.rsplit_once(':')
.map(|(r, t)| (r.to_string(), t.to_string()))
.expect("get_probe_image_name always emits a tag");
docker
.commit_container(
CommitContainerOptions {
container: container_id.to_string(),
repo,
tag,
pause: false,
..Default::default()
},
Config::<String> {
env: Some(blanked_secret_env()),
..Default::default()
},
)
.await
.map_err(|e| format!("Failed to commit stopped container {}: {}", container_id, e))?;
Ok(image_name)
}
/// Whether `env` (an image's `Config.Env`) holds a non-empty value for any
/// name in [`SECRET_ENV_KEYS`].
fn env_holds_a_secret(env: &[String]) -> bool {
@@ -3518,9 +3630,10 @@ pub async fn remove_snapshot_image(project: &Project) -> Result<(), String> {
remove_image_by_name(&get_snapshot_image_name(project)).await
}
/// Remove a Docker image by name/tag, treating "does not exist" as success.
/// Shared by [`remove_snapshot_image`] and the pending-cleanup retry, which
/// only has the image name (the project record is already gone by then).
/// Remove a Docker image by name, tag or **id**, treating "does not exist" as
/// success. Shared by [`remove_snapshot_image`], the pending-cleanup retry
/// (which only has the image name the project record is already gone by
/// then), and the staleness probe's throwaway commit, which has only an id.
pub async fn remove_image_by_name(image_name: &str) -> Result<(), String> {
let docker = get_docker()?;
@@ -3536,7 +3649,7 @@ pub async fn remove_image_by_name(image_name: &str) -> Result<(), String> {
.await
{
Ok(_) => {
log::info!("Removed snapshot image {}", image_name);
log::info!("Removed image {}", image_name);
Ok(())
}
Err(bollard::errors::Error::DockerResponseServerError {
@@ -4464,6 +4577,29 @@ mod tests {
assert!(env_holds_a_secret(&env));
}
/// The probe image's name must be **unique per call**. A stable name was
/// tried and is wrong twice over: a container id does not survive a
/// recreate, so a crashed probe's leftover would never be reclaimed by "the
/// next probe of the same container"; and two concurrent probes sharing one
/// tag means whichever finishes first force-removes the image the other is
/// still reading. See `commit_container_for_probe` and `reap_probe_images`.
#[test]
fn probe_image_names_are_unique_per_call_and_reapable_by_prefix() {
let id = "75993e6d5e1ab473b029a408c5ff0339";
let a = get_probe_image_name(id);
let b = get_probe_image_name(id);
assert_ne!(a, b, "two probes of one container must not share a tag");
// The prefix is the reaper's only handle on a leftover, so every name
// has to carry it — and it must not be the snapshot namespace, which is
// what a project is rebuilt from.
assert!(a.starts_with(PROBE_IMAGE_PREFIX), "{}", a);
assert!(!a.starts_with("triple-c-snapshot-"), "{}", a);
// Traceable back to its container, which is the point of the prefix.
assert!(a.contains("75993e6d5e1a"), "{}", a);
assert!(a.ends_with(":latest"), "{}", a);
}
#[test]
fn the_scrub_report_only_claims_success_when_nothing_is_left() {
let clean = SnapshotScrubReport {
+461
View File
@@ -886,6 +886,100 @@ pub async fn reap_probe_containers() {
}
}
/// Remove throwaway images left behind by a staleness probe of a stopped
/// container — [`super::container::commit_container_for_probe`]'s commits.
///
/// **Load-bearing, not tidying.** A probe image is *tagged*, because bollard
/// gives no image id back from a commit and there has to be something to probe.
/// Tagged means not dangling, so [`super::container::sweep_orphaned_snapshots`]
/// — which collects every other kind of orphan this app can leave — will never
/// see one. Without this, a probe that dies between its commit and its own
/// cleanup (SIGKILL, a crash, a 409 from a concurrent remove) strands a
/// multi-gigabyte image that **no code path can ever reclaim**, and there is no
/// UI to find it either. That is the one leak in this app with no floor on it,
/// so this runs at startup beside [`reap_probe_containers`].
///
/// Age-gated for exactly the reason that one is: `reference=` is a daemon-wide
/// filter, so a second copy of the app probing a project on the same daemon has
/// images matching this glob, and removing one mid-capture fails that probe with
/// "No such image" — the bogus `probe_error` the staleness work exists to get
/// rid of. In-process state cannot see the other instance, so age is the only
/// brake, and [`PROBE_REAP_MIN_AGE_SECS`] is already the right one: a probe is a
/// `find` over a root filesystem, not a multi-minute job.
///
/// Never fails the caller. Housekeeping, like every other sweep here.
pub async fn reap_probe_images() {
use bollard::image::{ListImagesOptions, RemoveImageOptions};
let docker = match get_docker() {
Ok(d) => d,
Err(e) => {
log::warn!("Could not reap leftover probe images: {}", e);
return;
}
};
let filters = HashMap::from([(
"reference".to_string(),
vec![format!("{}*", super::container::PROBE_IMAGE_PREFIX)],
)]);
let images = match docker
.list_images(Some(ListImagesOptions {
all: false,
filters,
..Default::default()
}))
.await
{
Ok(images) => images,
Err(e) => {
log::warn!("Could not list leftover probe images: {}", e);
return;
}
};
let now = chrono::Utc::now().timestamp();
for image in images {
// Unlike a container summary, an image summary always carries a
// `Created`, so there is no unknown-age case to defend against here.
if now - image.created < PROBE_REAP_MIN_AGE_SECS {
log::info!(
"Leaving probe image {:?} alone — it is younger than {} minutes, so it may belong \
to another Triple-C instance's live probe",
image.repo_tags,
PROBE_REAP_MIN_AGE_SECS / 60
);
continue;
}
// By **tag**, never by image id. A `force` removal by id untags an
// image everywhere, so an id that happens to carry another name loses
// that name too — which is how a test fixture that tagged
// `alpine:latest` into this namespace deleted the user's alpine. A real
// leftover has exactly the one probe tag, so removing the tag removes
// the image; anything else keeps whatever other names it has.
for tag in image
.repo_tags
.iter()
.filter(|t| t.starts_with(super::container::PROBE_IMAGE_PREFIX))
{
log::info!("Removing leftover probe image {}", tag);
if let Err(e) = docker
.remove_image(
tag,
Some(RemoveImageOptions {
force: true,
noprune: false,
}),
None,
)
.await
{
log::warn!("Could not remove leftover probe image {}: {}", tag, e);
}
}
}
}
/// How old a `triple-c.probe=migration` container must be before
/// [`reap_probe_containers`] will force-remove it, in seconds.
///
@@ -993,6 +1087,119 @@ pub async fn manifest_from_container(container_id: &str) -> Result<Manifest, Str
Ok(parse_manifest(&out))
}
/// Cached stopped-container manifests, keyed by container id, each paired with
/// the container's `FinishedAt` at the time it was captured.
///
/// **Sound because a stopped container's writable layer cannot change.** Nothing
/// can write to it while it is not running, so a manifest captured after it
/// stopped stays true until it is started again — and `FinishedAt` moves on
/// every stop, which is what makes the key exact rather than merely plausible.
///
/// This exists because `get_container_staleness` is called from a `useEffect`
/// that fires whenever the container settles, so simply opening a stopped
/// project's Overview probes it. Uncached that meant a `docker commit` of the
/// whole writable layer per visit — measured at 44 s on a real project — where
/// before this feature the same visit cost one throwaway container or nothing at
/// all. A regression like that is not worth the answer it buys.
///
/// Capped, because a `Manifest` of a real container is a few MB: this only has
/// to serve "the project whose page is open", so a handful of entries is the
/// whole working set and the oldest is dropped past that.
static STOPPED_MANIFEST_CACHE: std::sync::Mutex<
Option<Vec<(String, String, Manifest)>>,
> = std::sync::Mutex::new(None);
/// How many stopped-container manifests [`STOPPED_MANIFEST_CACHE`] keeps.
const STOPPED_MANIFEST_CACHE_MAX: usize = 4;
/// `FinishedAt` for a container, the cache's validity token. `None` when it
/// cannot be read, which is never treated as a hit.
async fn container_finished_at(container_id: &str) -> Option<String> {
let docker = get_docker().ok()?;
docker
.inspect_container(container_id, None)
.await
.ok()?
.state?
.finished_at
.filter(|s| !s.is_empty())
}
/// Capture a [`Manifest`] from a **stopped** container, reusing a cached one
/// when the container has not been started since it was taken.
///
/// See [`STOPPED_MANIFEST_CACHE`] for why this is exact and why it is needed.
pub async fn manifest_from_stopped_container_cached(
container_id: &str,
) -> Result<Manifest, String> {
let finished_at = container_finished_at(container_id).await;
if let Some(token) = &finished_at {
let guard = STOPPED_MANIFEST_CACHE.lock();
if let Ok(cache) = guard {
if let Some(entries) = cache.as_ref() {
if let Some((_, _, manifest)) = entries
.iter()
.find(|(id, tok, _)| id == container_id && tok == token)
{
log::debug!(
"Reusing the cached manifest for stopped container {}",
container_id
);
return Ok(manifest.clone());
}
}
}
}
let manifest = manifest_from_stopped_container(container_id).await?;
// Only cacheable if the container's state could be read at all; an unknown
// `FinishedAt` means there is no token that could later be compared.
if let Some(token) = finished_at {
if let Ok(mut cache) = STOPPED_MANIFEST_CACHE.lock() {
let entries = cache.get_or_insert_with(Vec::new);
entries.retain(|(id, _, _)| id != container_id);
entries.push((container_id.to_string(), token, manifest.clone()));
while entries.len() > STOPPED_MANIFEST_CACHE_MAX {
entries.remove(0);
}
}
}
Ok(manifest)
}
/// Capture a [`Manifest`] from a **stopped** container.
///
/// Commits the container's writable layer to a throwaway image, probes that,
/// and removes it. This is as current as [`manifest_from_container`] — it reads
/// the same filesystem — and it is why a stopped project no longer has to fall
/// back to its snapshot image, which may not exist at all and lags the
/// container by everything installed since the last commit when it does.
///
/// The image is removed on every path, including a failed probe. See
/// [`super::container::commit_container_for_probe`] for what a crash in the
/// window between the two costs, and why it is bounded.
pub async fn manifest_from_stopped_container(container_id: &str) -> Result<Manifest, String> {
let image = super::container::commit_container_for_probe(container_id).await?;
let manifest = manifest_from_image(&image)
.await
.map_err(|e| format!("Probe of the stopped container did not complete: {}", e));
if let Err(e) = super::container::remove_image_by_name(&image).await {
log::warn!(
"Could not remove the staleness probe's throwaway image {}: {} — `reap_probe_images` \
collects it at the next app start; the orphan sweep never will, because it is tagged",
image,
e
);
}
manifest
}
/// The image ID (`sha256:…`) of a local image, or `None` if it is not present.
///
/// Deliberately the **ID**, not a repo digest: locally built images and custom
@@ -2146,4 +2353,258 @@ mod tests {
assert!(!pin_is_reapable("pre-migration-handmade", false, ancient, &now));
assert!(!pin_is_reapable("latest", false, ancient, &now));
}
// ── Live Docker ─────────────────────────────────────────────────────────
/// The cache serves a second read of an unchanged stopped container, and —
/// the half that matters — stops serving it the moment the container is
/// started and stopped again. If invalidation were wrong this would report a
/// filesystem the project no longer has, and a migration would be planned
/// against it.
///
/// ```text
/// cargo test -- --ignored --nocapture stopped_manifest_cache
/// ```
#[cfg(unix)]
#[tokio::test]
#[ignore = "needs a Docker daemon; creates, commits and removes a throwaway container"]
async fn the_stopped_manifest_cache_survives_a_reread_but_not_a_restart() {
fn docker_cli(args: &[&str]) -> String {
let out = std::process::Command::new("docker")
.args(args)
.output()
.expect("docker CLI");
assert!(
out.status.success(),
"docker {:?} failed: {}",
args,
String::from_utf8_lossy(&out.stderr)
);
String::from_utf8_lossy(&out.stdout).trim().to_string()
}
let image = std::env::var("TRIPLE_C_TEST_IMAGE")
.unwrap_or_else(|_| "ghcr.io/shadowdao/triple-c-sandbox:latest".to_string());
let first = format!("/opt/cache-marker-a-{}", std::process::id());
let second = format!("/opt/cache-marker-b-{}", std::process::id());
let id = docker_cli(&[
"run", "-d", "--label", "triple-c.managed=true",
"--entrypoint", "/bin/sh",
&image, "-c", "sleep 600",
]);
let cleanup = || {
let _ = std::process::Command::new("docker")
.args(["rm", "-f", &id])
.output();
};
docker_cli(&["exec", &id, "mkdir", "-p", &first]);
docker_cli(&["stop", "-t", "1", &id]);
let t0 = std::time::Instant::now();
let cold = manifest_from_stopped_container_cached(&id).await;
let cold_ms = t0.elapsed().as_millis();
let t1 = std::time::Instant::now();
let warm = manifest_from_stopped_container_cached(&id).await;
let warm_ms = t1.elapsed().as_millis();
// Restart, change the filesystem, stop again — `FinishedAt` moves.
docker_cli(&["start", &id]);
docker_cli(&["exec", &id, "mkdir", "-p", &second]);
docker_cli(&["stop", "-t", "1", &id]);
let after_restart = manifest_from_stopped_container_cached(&id).await;
cleanup();
let has = |m: &Manifest, p: &str| m.paths.iter().any(|e| e.path == p && e.is_dir());
let cold = cold.expect("cold read");
let warm = warm.expect("warm read");
let after_restart = after_restart.expect("read after restart");
assert!(has(&cold, &first), "cold read missed {}", first);
assert!(has(&warm, &first), "warm read missed {}", first);
println!("cold {} ms, warm {} ms", cold_ms, warm_ms);
assert!(
warm_ms * 5 < cold_ms.max(5),
"the second read cost {} ms against a cold {} ms — it re-committed \
instead of using the cache",
warm_ms,
cold_ms
);
// The restart must have invalidated it: the new directory has to show up.
assert!(
has(&after_restart, &second),
"a restart did not invalidate the cache — {} is missing, so this is \
a stale manifest of a filesystem the container no longer has",
second
);
assert!(has(&after_restart, &first), "the restart lost {}", first);
}
/// The reaper finds a leftover probe image by prefix and — crucially —
/// refuses to remove a young one, because that image may be another
/// Triple-C instance's live probe. Only a real daemon can say whether the
/// `reference=` glob matches the names `get_probe_image_name` produces.
///
/// The fixture is **committed**, not tagged and not built. An image's
/// `Created` is its own, not its tag's, so tagging something already on disk
/// into this namespace yields a fixture the reaper is right to call ancient
/// — and BuildKit stamps a fixed epoch on `docker build` output, so a built
/// one looks ancient too. A commit stamps *now*, verified against Engine
/// 29.6, which is also how real probe images get their age.
///
/// Both of those mistakes were made here first, and one of them deleted an
/// unrelated `alpine:latest` — which is why `reap_probe_images` removes by
/// tag rather than by image id.
///
/// ```text
/// cargo test -- --ignored --nocapture reaper_spares
/// ```
#[cfg(unix)]
#[tokio::test]
#[ignore = "needs a Docker daemon; builds and removes a throwaway image"]
async fn the_reaper_spares_a_probe_image_young_enough_to_be_someone_elses() {
use std::process::Command;
fn docker_out(args: &[&str]) -> std::process::Output {
Command::new("docker").args(args).output().expect("docker CLI")
}
let base = std::env::var("TRIPLE_C_TEST_IMAGE")
.unwrap_or_else(|_| "alpine:latest".to_string());
let name = crate::docker::container::get_probe_image_name("reapertest01234");
// A never-started container is enough to commit from, and leaves the
// daemon's run state alone entirely.
let created = docker_out(&["create", &base, "true"]);
assert!(
created.status.success(),
"could not create the fixture container from {}: {}",
base,
String::from_utf8_lossy(&created.stderr)
);
let cid = String::from_utf8_lossy(&created.stdout).trim().to_string();
let committed = docker_out(&["commit", "--pause=false", &cid, &name]);
let _ = docker_out(&["rm", "-f", &cid]);
assert!(
committed.status.success(),
"could not commit the fixture image: {}",
String::from_utf8_lossy(&committed.stderr)
);
reap_probe_images().await;
let still_there = Command::new("docker")
.args(["image", "inspect", &name])
.output()
.expect("docker image inspect")
.status
.success();
let _ = Command::new("docker").args(["rmi", &name]).output();
assert!(
still_there,
"a probe image committed seconds ago was reaped — that is another \
instance's live probe being broken, see PROBE_REAP_MIN_AGE_SECS"
);
}
/// A *stopped* container is readable, and what comes back is its writable
/// layer rather than the image it was created from. This is the whole point
/// of the function: the base image cannot answer it, and the project may
/// well have no snapshot image at all.
///
/// Also asserts the throwaway commit leaves nothing behind, which no unit
/// test can. It has to assert on the `triple-c-probe-*` tags specifically:
/// the probe image is *tagged*, so a leak never shows up as a dangling
/// image and a dangling-set assertion here would pass either way.
///
/// Ignored because it needs Docker and commits a container; run it with
///
/// ```text
/// cargo test -- --ignored --nocapture stopped_container
/// ```
#[cfg(unix)]
#[tokio::test]
#[ignore = "needs a Docker daemon; creates, commits and removes a throwaway container"]
async fn a_stopped_container_is_read_from_its_writable_layer() {
fn docker_cli(args: &[&str]) -> String {
let out = std::process::Command::new("docker")
.args(args)
.output()
.expect("docker CLI");
assert!(
out.status.success(),
"docker {:?} failed: {}",
args,
String::from_utf8_lossy(&out.stderr)
);
String::from_utf8_lossy(&out.stdout).trim().to_string()
}
fn probe_images() -> Vec<String> {
let mut ids: Vec<String> = docker_cli(&[
"images", "-q",
"--filter",
&format!("reference={}*", crate::docker::container::PROBE_IMAGE_PREFIX),
])
.lines()
.map(|l| l.trim().to_string())
.filter(|l| !l.is_empty())
.collect();
ids.sort();
ids
}
let image = std::env::var("TRIPLE_C_TEST_IMAGE")
.unwrap_or_else(|_| "ghcr.io/shadowdao/triple-c-sandbox:latest".to_string());
// A marker only the writable layer can carry, under a MANIFEST_ROOTS root.
let marker = format!("/opt/probe-marker-{}", std::process::id());
// Another instance's live probe images are allowed to exist; what must
// hold is that this probe adds none of its own.
let before = probe_images();
let id = docker_cli(&[
"run", "-d", "--label", "triple-c.managed=true",
"--entrypoint", "/bin/sh",
&image, "-c", "sleep 300",
]);
let cleanup = |id: &str| {
let _ = std::process::Command::new("docker")
.args(["rm", "-f", id])
.output();
};
docker_cli(&["exec", &id, "mkdir", "-p", &marker]);
docker_cli(&["stop", "-t", "1", &id]);
let result = manifest_from_stopped_container(&id).await;
cleanup(&id);
let manifest = result.expect("a stopped container must be probeable");
assert!(
manifest.paths.iter().any(|e| e.path == marker && e.is_dir()),
"the probe read the image, not the container's writable layer: {} missing",
marker
);
// Non-empty package sets prove the probe script really ran, rather than
// parsing an empty transcript into an empty-but-Ok manifest.
assert!(
!manifest.apt_manual.is_empty(),
"apt-mark showmanual came back empty, so the probe did not run"
);
assert_eq!(
probe_images(),
before,
"the throwaway probe image was not cleaned up"
);
}
}
+9 -1
View File
@@ -263,12 +263,20 @@ pub fn run() {
// logged warning rather than a failed start.
//
// Ordering matters. Probes are removed first because a probe holds
// an image open and the sweep will not force; pins are untagged
// an image open and the sweep will not force — both the probe
// containers and the probe images, the latter being the one orphan
// the sweep can never reach on its own; pins are untagged
// second so the images they were holding are dangling by the time
// the sweep lists them; the sweep runs last and collects both.
let projects_store_for_cleanup = projects_store_setup.clone();
tauri::async_runtime::spawn(async move {
crate::docker::reap_probe_containers().await;
// Probe *images* too, and for a sharper reason: a probe
// container merely pins an image the sweep then refuses to
// touch, whereas a leftover probe image is tagged and so
// nothing else in this app can ever collect it. See
// `reap_probe_images`.
crate::docker::reap_probe_images().await;
let reaped = crate::docker::reap_stale_migration_pins().await;
if reaped > 0 {
log::info!("Startup housekeeping dropped {} stale rollback pin(s)", reaped);
+15 -3
View File
@@ -350,8 +350,8 @@ export const sweepClaudeTokenSnapshots = () =>
// without deleting its volumes. Reset is the destructive alternative: it wipes
// ~/.claude, the OAuth credential, installed skills and every transcript.
//
// Flow: getContainerStaleness (read-only, ~6s — two filesystem probes, so call
// it on demand rather than polling) → migrateProjectToBase → the project sits
// Flow: getContainerStaleness (~6s — two filesystem probes, so call it on demand
// rather than polling) → migrateProjectToBase → the project sits
// in "awaiting-confirmation" while the user tries it → confirmMigration or
// rollbackMigration.
//
@@ -361,7 +361,19 @@ export const sweepClaudeTokenSnapshots = () =>
//
// Progress arrives on the existing `container-progress` event.
/** Read-only. Runs two container/image filesystem probes; not for polling. */
/**
* Runs two container/image filesystem probes; not for polling.
*
* **Not read-only, despite only reporting.** When the container is *stopped*
* the backend has to commit its writable layer to a throwaway image before it
* can read anything — `docker exec` needs a running container — so this writes
* (and then removes) an image. The result is cached per stop, so repeat calls
* while the container stays stopped are cheap, but the first one after each stop
* pays for a commit of the whole layer: seconds on a small project, tens of
* seconds on a large one. Do not add a caller that fires more often than "the
* container settled into a new state" without re-reading
* `get_container_staleness`'s doc comment first.
*/
export const getContainerStaleness = (projectId: string) =>
invoke<ContainerStaleness>("get_container_staleness", { projectId });
+91 -31
View File
@@ -91,6 +91,11 @@ HOOK="apprun-hooks/triple-c-wayland-fallback.sh"
APPIMAGE_TOOL_URL="https://github.com/AppImage/appimagetool/releases/download/continuous/appimagetool-x86_64.AppImage"
APP_ID="com.triple-c.desktop"
# The channel pair lives in its own directory. Left beside the versioned image
# they are picked up by the release job's `*.AppImage` glob, and every release
# then carries an eighty-megabyte byte-identical duplicate under a second name
# — which is exactly as confusing on a downloads page as it sounds.
CHANNEL_DIR="update-channel"
STABLE_NAME="Triple-C_x86_64.AppImage"
UPDATE_TAG="linux-latest"
UPDATE_INFO="zsync|https://github.com/shadowdao/triple-c/releases/download/${UPDATE_TAG}/${STABLE_NAME}.zsync"
@@ -98,8 +103,13 @@ CATEGORIES="Development;Utility;"
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
appdata_src="$repo_root/packaging/appimage/$APP_ID.appdata.xml"
# appimagetool looks for `<desktop basename>.appdata.xml` and warns the
# metadata is missing under any other name — while the script cheerfully
# reported it present. The AppStream id inside the file is unchanged and is
# what actually identifies the component; only the filename follows the tool.
appdata_installed_as="Triple-C.appdata.xml"
dir="${1:?usage: unbundle-wayland-client.sh <bundle/appimage directory>}"
dir="${1:?usage: finalize-appimage.sh <bundle/appimage directory>}"
cd "$dir"
shopt -s nullglob
@@ -109,6 +119,14 @@ if [ ${#images[@]} -eq 0 ]; then
echo "No .AppImage in $dir — nothing to do." >&2
exit 0
fi
# Refused here rather than after the repack: with two present the old position
# let the script download appimagetool, repack, overwrite the versioned
# artifact and write the channel pair, *then* fail — and it silently picked
# images[0], which is glob order, i.e. the older version.
if [ ${#images[@]} -ne 1 ]; then
echo "Expected 1 AppImage in $dir, found ${#images[@]}: ${images[*]}" >&2
exit 1
fi
appimage="${images[0]}"
here="$PWD"
@@ -120,15 +138,15 @@ echo "Inspecting $appimage"
( cd "$work" && "$here/$appimage" --appimage-extract >/dev/null )
root="$work/squashfs-root"
if [ ! -e "$root/usr/lib/$LIB" ]; then
# Not a failure: linuxdeploy may have stopped bundling it, which is the
# outcome this script exists to produce.
echo "$LIB is not bundled — leaving $appimage alone."
exit 0
fi
# The demotion and the metadata are independent jobs, and an absent library
# must not skip the second. An early exit here also left `update-channel/`
# uncreated, which killed the publish step on a missing directory and took the
# tag and mirror jobs down with it — a half-published release.
demoted=false
if [ -e "$root/usr/lib/$LIB" ]; then
mkdir -p "$root/$FALLBACK_DIR"
mv "$root/usr/lib/$LIB" "$root/$FALLBACK_DIR/$LIB"
mkdir -p "$root/$FALLBACK_DIR"
mv "$root/usr/lib/$LIB" "$root/$FALLBACK_DIR/$LIB"
cat > "$root/$HOOK" <<'HOOK_EOF'
#! /usr/bin/env bash
@@ -177,6 +195,11 @@ src = src.replace(
)
open(path, "w").write(src)
PATCH_EOF
fi
demoted=true
echo "Demoted $LIB to $FALLBACK_DIR."
else
echo "$LIB is not bundled — nothing to demote."
fi
# --- metadata -------------------------------------------------------------
@@ -188,22 +211,29 @@ version="$(printf '%s' "$appimage" | sed -n 's/.*_\([0-9][0-9.]*\)_.*/\1/p')"
if [ -f "$appdata_src" ]; then
mkdir -p "$root/usr/share/metainfo"
sed -e "s/@VERSION@/$version/" -e "s/@DATE@/$(date -u +%Y-%m-%d)/" \
"$appdata_src" > "$root/usr/share/metainfo/$APP_ID.appdata.xml"
"$appdata_src" > "$root/usr/share/metainfo/$appdata_installed_as"
echo "Added AppStream metadata for $version."
else
echo "No AppStream source at $appdata_src — skipping." >&2
fi
# linuxdeploy emits `Categories=` empty, which files the app nowhere.
for desktop in "$root"/*.desktop; do
#
# The AppDir root entry is a **symlink** into usr/share/applications, so a
# plain `sed -i` replaces the link with a regular file and leaves the real entry
# untouched — two divergent copies, of which the empty one is the one that
# actually ships and the filled one is the only one a root-only guard can see.
# `--follow-symlinks` writes through. Both locations are globbed because the
# layout is linuxdeploy's, not ours, and it is free to stop symlinking.
for desktop in "$root"/*.desktop "$root"/usr/share/applications/*.desktop; do
[ -e "$desktop" ] || continue
if grep -q "^Categories=$" "$desktop"; then
sed -i "s/^Categories=$/Categories=$CATEGORIES/" "$desktop"
echo "Filled in Categories for $(basename "$desktop")."
sed -i --follow-symlinks "s/^Categories=$/Categories=$CATEGORIES/" "$desktop"
echo "Filled in Categories for ${desktop#"$root"/}."
fi
done
echo "Demoted $LIB to $FALLBACK_DIR; repacking."
echo "Repacking."
tool="$work/appimagetool"
curl -fsSL -o "$tool" "$APPIMAGE_TOOL_URL"
@@ -211,13 +241,19 @@ chmod +x "$tool"
# --appimage-extract-and-run: CI runners generally have no FUSE.
# -u embeds the update string and writes "$STABLE_NAME.zsync" beside the image.
rm -rf "$CHANNEL_DIR"
mkdir -p "$CHANNEL_DIR"
ARCH=x86_64 "$tool" --appimage-extract-and-run \
-u "$UPDATE_INFO" "$root" "$STABLE_NAME" >/dev/null
chmod +x "$STABLE_NAME"
-u "$UPDATE_INFO" "$root" "$CHANNEL_DIR/$STABLE_NAME" >/dev/null
chmod +x "$CHANNEL_DIR/$STABLE_NAME"
# The versioned name is what the per-version release publishes; the stable one
# and its .zsync go to the rolling tag. Same bytes, two names.
cp "$STABLE_NAME" "$appimage"
# and its .zsync go to the rolling tag. Same bytes, two names, two places.
# zsyncmake writes the .zsync into the working directory, not beside the image
# it describes, so it has to be collected rather than assumed in place.
[ -e "$STABLE_NAME.zsync" ] && mv "$STABLE_NAME.zsync" "$CHANNEL_DIR/"
cp "$CHANNEL_DIR/$STABLE_NAME" "$appimage"
chmod +x "$appimage"
# The guards are the test. Each one is a way the repack could look like it
@@ -227,16 +263,28 @@ out="$check/squashfs-root"
fail() { echo "FAILED: $1" >&2; exit 1; }
[ -e "$out/usr/lib/$LIB" ] && fail "$LIB is still on the loader path."
[ -e "$out/$FALLBACK_DIR/$LIB" ] || fail "the fallback copy of $LIB is missing."
[ -e "$out/$HOOK" ] || fail "the fallback hook is missing."
grep -q "triple-c-wayland-fallback" "$out/AppRun" || fail "AppRun does not source the hook."
if [ "$demoted" = true ]; then
[ -e "$out/usr/lib/$LIB" ] && fail "$LIB is still on the loader path."
[ -e "$out/$FALLBACK_DIR/$LIB" ] || fail "the fallback copy of $LIB is missing."
[ -e "$out/$HOOK" ] || fail "the fallback hook is missing."
grep -q "triple-c-wayland-fallback" "$out/AppRun" || fail "AppRun does not source the hook."
fi
[ -x "$out/usr/bin/triple-c" ] || fail "no executable usr/bin/triple-c."
# An empty Categories or missing metadata ships an image a manager cannot file
# or describe, and both fail silently at runtime rather than at build time.
grep -q "^Categories=.\+" "$out"/*.desktop || fail "Categories is still empty."
[ -f "$appdata_src" ] && { [ -e "$out/usr/share/metainfo/$APP_ID.appdata.xml" ] \
# Asserted positively, over every entry: the earlier form checked only that no
# *root* file held an empty value, which passed while the real entry under
# usr/share/applications shipped empty, and also passed on a missing key.
desktops=0
for desktop in "$out"/*.desktop "$out"/usr/share/applications/*.desktop; do
[ -e "$desktop" ] || continue
desktops=$((desktops + 1))
grep -q "^Categories=$CATEGORIES$" "$desktop" \
|| fail "${desktop#"$out"/} does not carry Categories=$CATEGORIES."
done
[ "$desktops" -gt 0 ] || fail "the image contains no .desktop entry at all."
[ -f "$appdata_src" ] && { [ -e "$out/usr/share/metainfo/$appdata_installed_as" ] \
|| fail "AppStream metadata did not make it into the image."; }
# The update string is the difference between adoptable and updatable. It
@@ -245,13 +293,25 @@ grep -q "^Categories=.\+" "$out"/*.desktop || fail "Categories is still empty."
# the URL it fetched the .zsync from. That is exactly why the output is named
# for the fixed tag: a versioned name here resolves to the build the client
# already has.
[ -e "$STABLE_NAME" ] || fail "the stable-named image is missing."
[ -e "$STABLE_NAME.zsync" ] || fail "appimagetool wrote no $STABLE_NAME.zsync."
[ -e "$CHANNEL_DIR/$STABLE_NAME" ] || fail "the stable-named image is missing."
[ -e "$CHANNEL_DIR/$STABLE_NAME.zsync" ] || fail "appimagetool wrote no .zsync."
readelf -p .upd_info "$STABLE_NAME" 2>/dev/null | grep -q "$UPDATE_TAG" \
|| fail "the image carries no update information for the $UPDATE_TAG tag."
grep -aq "^Filename: $STABLE_NAME$" "$STABLE_NAME.zsync" \
readelf -p .upd_info "$CHANNEL_DIR/$STABLE_NAME" 2>/dev/null | grep -qF "$UPDATE_INFO" \
|| fail "the image does not carry exactly the expected update information."
grep -aq "^Filename: $STABLE_NAME$" "$CHANNEL_DIR/$STABLE_NAME.zsync" \
|| fail "the .zsync names something other than $STABLE_NAME."
echo "OK: $appimage prefers the host $LIB (fallback kept), carries AppStream"
echo " metadata, and updates from the $UPDATE_TAG tag via $STABLE_NAME.zsync."
# The versioned release must carry one AppImage, not two. This is the guard
# for the duplicate that shipped in 0.4.20 and 0.4.21.
shopt -s nullglob
beside=(*.AppImage)
shopt -u nullglob
[ "${#beside[@]}" -eq 1 ] \
|| fail "expected 1 AppImage beside the release, found ${#beside[@]}."
if [ "$demoted" = true ]; then
echo "OK: $appimage prefers the host $LIB (fallback kept) and carries"
else
echo "OK: $appimage had no bundled $LIB to demote, and carries"
fi
echo " AppStream metadata. Channel pair in $CHANNEL_DIR/, updating from $UPDATE_TAG."
+182 -19
View File
@@ -17,7 +17,22 @@
# It writes to GitHub rather than Gitea because that mirror is where updates
# are pulled from. Needs GH_PAT with contents write on the mirror.
#
# Usage: GH_PAT=... publish-update-channel.sh <directory holding the artifacts>
# **The tag has to exist in Gitea, not just on GitHub, and that is the whole
# reason this script touches Gitea at all.** Gitea push-mirrors this repo to
# GitHub, and a mirror push deletes remote refs that have no local counterpart.
# A tag created only by GitHub's release API therefore survives until the next
# mirror run and then vanishes — which is exactly what happened to 0.4.20 and
# 0.4.21: the release was created and both URLs verified 200 at 00:38, and the
# 13:04 mirror deleted the tag, leaving every installed copy checking a 404.
# Versioned tags never had this problem because `create-tag` creates them in
# Gitea first. So does this one, now, and before the GitHub release rather than
# after, so there is no window where the two disagree.
#
# Note what this means for verification: publishing correctly is not evidence
# the channel still works hours later. The Gitea tag is what makes it durable,
# so its absence is treated as a failure rather than a warning.
#
# Usage: GH_PAT=... GITEA_TOKEN=... GITEA_SHA=... publish-update-channel.sh <dir>
set -euo pipefail
@@ -26,7 +41,12 @@ TAG="linux-latest"
API="https://api.github.com/repos/$REPO"
ASSETS=("Triple-C_x86_64.AppImage" "Triple-C_x86_64.AppImage.zsync")
GITEA_API="${GITEA_API:-https://repo.anhonesthost.net/api/v1}"
GITEA_REPO="${GITEA_REPO:-CyberCoveLLC/Triple-C}"
: "${GH_PAT:?GH_PAT is required to publish the update channel}"
: "${GITEA_TOKEN:?GITEA_TOKEN is required to anchor the $TAG tag against the mirror}"
: "${GITEA_SHA:?GITEA_SHA is required to point the $TAG tag at this build}"
dir="${1:?usage: publish-update-channel.sh <artifacts directory>}"
cd "$dir"
@@ -35,46 +55,179 @@ for asset in "${ASSETS[@]}"; do
done
gh() { curl -sf -H "Authorization: Bearer $GH_PAT" -H "Accept: application/vnd.github+json" "$@"; }
tea() { curl -sf -H "Authorization: token $GITEA_TOKEN" -H "Content-Type: application/json" "$@"; }
# Status, not a boolean. `curl -sf` fails identically for "404, the tag is
# genuinely absent" and "503, Gitea is briefly unreachable", and treating the
# second as the first means POSTing over a tag that already exists, taking a
# 409, and aborting the last step of build-linux — which `create-tag` and
# `sync-to-github` both depend on. A transient blip would cost the release, not
# just the channel update. Same `case`-on-code idiom as `Upload to Gitea
# release` two steps above in the workflow. A refused connection reports 000
# and lands in the catch-all.
tea_code() { curl -s -o /dev/null -w '%{http_code}' -H "Authorization: token $GITEA_TOKEN" "$@"; }
echo "==> Looking for the $TAG release"
release="$(gh "$API/releases/tags/$TAG" 2>/dev/null || true)"
release_id="$(printf '%s' "$release" | python3 -c 'import sys,json;print(json.load(sys.stdin).get("id",""))' 2>/dev/null || true)"
# Anchor the tag in Gitea — see the header. **Created if absent, never moved.**
#
# An earlier version deleted and recreated it so the tag would name the current
# build. That was worse than useless: nothing about the channel depends on
# which commit the tag points at — the update string resolves the tag by *name*
# and the assets hang off the release object — while a DELETE followed by a
# failed POST destroys a working anchor and leaves a window in which a mirror
# run prunes GitHub's copy. A transient Gitea error would have converted a
# healthy channel into a dead one, which is strictly worse than this step not
# existing. Gitea's POST /tags has no force semantics, so the DELETE was only
# ever there to get around a 409; asking first removes the need.
echo "==> Anchoring the $TAG tag in Gitea"
anchor_probe="$(tea_code "$GITEA_API/repos/$GITEA_REPO/tags/$TAG")"
case "$anchor_probe" in
200)
echo " already anchored — left alone"
;;
404)
echo " creating it at ${GITEA_SHA:0:9}"
tea -X POST "$GITEA_API/repos/$GITEA_REPO/tags" \
-d "{\"tag_name\": \"$TAG\", \"target\": \"$GITEA_SHA\", \"message\": \"Rolling Linux update channel\"}" \
>/dev/null
;;
*)
echo "FAILED: Gitea answered $anchor_probe asking whether the $TAG tag exists." >&2
echo " Refusing to guess — creating it blindly would 409 over an" >&2
echo " existing tag and abort the release." >&2
exit 1
;;
esac
# Not best-effort. Without this tag the mirror removes GitHub's and the
# channel dies silently somewhere between now and four hours from now. Reported
# by code, so "Gitea was unreachable" cannot masquerade as "the tag is gone".
anchor_code="$(tea_code "$GITEA_API/repos/$GITEA_REPO/tags/$TAG")"
[ "$anchor_code" = "200" ] || {
echo "FAILED: the $TAG tag is not readable in Gitea (HTTP $anchor_code);" >&2
echo " without it the mirror would delete GitHub's copy." >&2
exit 1
}
# Look through the authenticated list rather than /releases/tags/, which never
# returns drafts. That matters here specifically: GitHub demotes a published
# release to a draft when its tag is deleted, which is the state every mirror
# run left behind, so the by-tag lookup reports "absent" while orphaned drafts
# sit there holding 86 MB each. Reuse the newest and delete the rest, or they
# accumulate one per release forever.
echo "==> Looking for the $TAG release (drafts included)"
all_releases="$(gh "$API/releases?per_page=100")"
mapfile -t existing < <(printf '%s' "$all_releases" | python3 -c '
import sys, json
tag = sys.argv[1]
rs = [r for r in json.load(sys.stdin) if r.get("tag_name") == tag]
rs.sort(key=lambda r: r.get("created_at",""), reverse=True)
for r in rs:
print(r["id"])
' "$TAG")
release_id="${existing[0]:-}"
for stale in "${existing[@]:1}"; do
echo " deleting orphaned duplicate release $stale"
gh -X DELETE "$API/releases/$stale" >/dev/null || true
done
if [ -n "$release_id" ]; then
# A draft has no tag and serves no download URL, so it has to be republished.
echo " reusing release $release_id"
# `make_latest` is not optional here even though this release already exists.
# Publishing a draft is a publish transition, where the API's documented
# default is `true` — so omitting it would quietly promote this channel to
# the repository's "Latest release" and bury the versioned release a person
# actually wants from the releases page.
#
# `tag_name` is re-sent deliberately, and must be: the API removes the tag
# when a PATCH omits it. Given this whole change exists because a tag
# disappeared, that is an expensive line to tidy away.
gh -X PATCH "$API/releases/$release_id" \
-d "{\"tag_name\": \"$TAG\", \"draft\": false, \"make_latest\": \"false\"}" >/dev/null
release="$(gh "$API/releases/$release_id")"
fi
if [ -z "$release_id" ]; then
echo "==> Creating it"
# Not a prerelease, but deliberately not the "latest" release either: this
# tag is a channel, and it must never displace the versioned release a
# person lands on from the releases page.
release="$(gh -X POST "$API/releases" -d "$(python3 -c '
body_json="$(python3 -c '
import json
print(json.dumps({
"tag_name": "'"$TAG"'",
"name": "Linux update channel",
"body": "Rolling AppImage build that Triple-Cs in-app updater reads. "
"body": "Rolling AppImage build that Triple-C\u2019s in-app updater reads. "
"The two files here are replaced on every release; for a specific "
"version, use the versioned releases instead.",
"draft": False,
"prerelease": False,
"make_latest": "false",
}))')")"
}))')"
# `already_exists` is a benign, recoverable answer, not a reason to abort the
# last step of build-linux and lose the release with it. It means a release
# for this tag exists but the listing above did not show it — a draft that has
# sunk past the first page, since a draft's created_at is frozen while newer
# releases push it down. Re-ask by tag and carry on.
create_body="$(mktemp)"
create_code="$(curl -s -o "$create_body" -w '%{http_code}' \
-H "Authorization: Bearer $GH_PAT" -H "Accept: application/vnd.github+json" \
-X POST "$API/releases" -d "$body_json")"
case "$create_code" in
201)
release="$(cat "$create_body")"
;;
422)
if grep -q "already_exists" "$create_body"; then
echo " a release for $TAG already exists but was not listed — reusing it"
release="$(gh "$API/releases/tags/$TAG")"
else
echo "FAILED: GitHub rejected the release (422):" >&2
cat "$create_body" >&2
rm -f "$create_body"
exit 1
fi
;;
*)
echo "FAILED: creating the $TAG release returned $create_code:" >&2
cat "$create_body" >&2
rm -f "$create_body"
exit 1
;;
esac
rm -f "$create_body"
release_id="$(printf '%s' "$release" | python3 -c 'import sys,json;print(json.load(sys.stdin)["id"])')"
fi
echo "==> Removing superseded assets from release $release_id"
printf '%s' "$release" | python3 -c '
# One asset at a time, delete immediately followed by upload. Deleting both up
# front leaves the channel holding a fresh AppImage and no .zsync if the second
# upload fails, and a client that cannot fetch the .zsync simply stops updating
# — no error anyone here would see.
asset_ids="$(printf '%s' "$release" | python3 -c '
import sys, json
keep = set(sys.argv[1:])
out = {}
for a in json.load(sys.stdin).get("assets", []):
if a["name"] in keep:
print(a["id"])
' "${ASSETS[@]}" | while read -r asset_id; do
[ -n "$asset_id" ] || continue
gh -X DELETE "$API/releases/assets/$asset_id" >/dev/null || true
done
out[a["name"]] = a["id"]
print(json.dumps(out))
' "${ASSETS[@]}")"
# --retry/--max-time/--http1.1 for the reason the Gitea upload steps in this
# repo carry them: real mid-stream failures on large assets (curl 92 and 28).
for asset in "${ASSETS[@]}"; do
stale_id="$(printf '%s' "$asset_ids" | python3 -c 'import sys,json;print(json.load(sys.stdin).get(sys.argv[1],""))' "$asset")"
if [ -n "$stale_id" ]; then
echo "==> Replacing $asset (dropping superseded asset $stale_id)"
gh -X DELETE "$API/releases/assets/$stale_id" >/dev/null || true
fi
echo "==> Uploading $asset ($(du -h "$asset" | cut -f1))"
curl -sf -X POST \
curl -sf --http1.1 --retry 5 --retry-all-errors --retry-delay 5 --max-time 900 \
-X POST \
-H "Authorization: Bearer $GH_PAT" \
-H "Content-Type: application/octet-stream" \
--data-binary "@$asset" \
@@ -84,12 +237,22 @@ done
# The updater is only as good as this URL, and a silent failure here means
# every installed copy quietly stops updating. Confirm both are actually
# fetchable at the address the AppImage was built to check.
# Size as well as status: a 200 only proves something is served at the
# address, not that it is this build. GitHub accepting a truncated upload
# would pass a status-only check and then fail every client's checksum.
echo "==> Verifying the published URLs"
for asset in "${ASSETS[@]}"; do
url="https://github.com/$REPO/releases/download/$TAG/$asset"
code="$(curl -s -o /dev/null -w '%{http_code}' -L "$url")"
[ "$code" = "200" ] || { echo "FAILED: $url returned $code" >&2; exit 1; }
echo " $code $url"
local_size="$(stat -c %s "$asset")"
headers="$(curl -sIL "$url" | tr -d '\r')"
code="$(printf '%s\n' "$headers" | awk '/^HTTP\//{c=$2} END{print c}')"
served="$(printf '%s\n' "$headers" | awk 'tolower($1)=="content-length:"{n=$2} END{print n}')"
[ "$code" = "200" ] || { echo "FAILED: $url returned ${code:-no status}" >&2; exit 1; }
[ "$served" = "$local_size" ] \
|| { echo "FAILED: $url serves ${served:-unknown} bytes, built $local_size." >&2; exit 1; }
echo " $code $served bytes $url"
done
echo "OK: $TAG updated."
echo "OK: $TAG updated, and anchored in Gitea so the mirror preserves it."