get_container_staleness failed on every container start/stop with OUTPUT_LIMIT: Command output exceeded 8388608 bytes and was abandoned. manifest_from_container runs manifest_script() through exec_oneshot_as, which has an
8 MiB cap. On a real container the flat find /workspace -xdev walk was 8,594,010 bytes by
itself: about 94k entries from git worktrees, node_modules and vendored trees sitting in the
writable layer next to the bind mounts. The same manifest feeds fresh_migration (where it
fails with ?) and finish_migration.
Design
Why not prune by name? I measured it on that container (read-only): pruning node_modules
and .git only got it down to 6.3 MB / 68k entries. Most of what was left was a vendored
phpMyAdmin (about 5k files) in each of the eight worktrees. A list of names would never be
complete.
What this PR does: walk /workspace to depth 2 and summarise below that.
SUMMARISED_ROOTS = ["/workspace"], SUMMARY_DEPTH = 2. Each non-empty directory at
depth 2 is emitted by find as one S entry. A new ###SUMMARY section gives its size
from du -sbx.
The S line comes from find, not du. If du fails, the size is 0 and only the disk
estimate is affected. The directory is never dropped.
Empty directories at depth 2 are still printed as ordinary d entries, so the empty-tree
filter behaves as before.
Why depth 2 and not 1: depth 1 holds the bind-mount points. -xdev lists them without
descending, but du -x run on a mount point would walk the whole host repository.
Measured on the same container: the /workspace part drops from 8.6 MB to about 25 kB
(377 + 138 lines).
Why this loses nothing for migration: below /workspace/<x> every per-path filter in compute_verbatim_paths already has no effect:
dpkg owns nothing there;
there are no COPY_EXCLUSIONS there;
bind targets are exactly /workspace/<mount_name>, and a mount name cannot contain /;
the base ships an empty /workspace (checked).
So the verbatim set was already pruned to first-level entries, and it comes out the same.
Migration's tar -T archives /workspace/<x> recursively, so the contents of a summarised
directory are still copied in full.
Changes in compute_verbatim_paths:
A summary counts as content. is_dir() is false for S, so a dir holding only summaries
is not treated as an empty tree and dropped.
A summary is not subject to the "already in the base" check. That check is per path, and a
summary hides which paths are inside. Skipping it could drop user files under any subtree a
future base also ships. Copying is safe because restore uses --skip-old-files.
verbatim_payload_bytes counts the summary's du size.
Graceful degradation:
manifest_from_container now uses a new exec_oneshot_as_limited with MANIFEST_OUTPUT_LIMIT = 32 MiB. The other roots (/usr/local, /opt, ...) are still
walked file by file.
If the limit is still hit, describe_manifest_failure gives an actionable message instead
of the buffer error, the same approach as describe_listing_failure.
CLAUDE.md gets a bullet in the migration section explaining all this.
Tests
New unit tests:
parsing S entries and ###SUMMARY sizes;
a summary with no size line is kept;
summaries count as content and are counted in the payload;
a summary the base also has is still copied;
the script no longer walks /workspace flat;
describe_manifest_failure;
a Linux-only test that runs the real walk against a 2,000-file temp tree.
New ignored live test a_large_workspace_is_probed_without_overflowing_and_still_copied:
a scratch container off the base, with 80k files (9.15 MB as a flat walk) next to a bind
mount. It checks that the probe succeeds, does not cross the mount, gets summary sizes,
and that the verbatim set is the eight worktree dirs. It fails on the old behaviour
(verified by reverting the two constants) and passes now.
cargo test: 884 passed, 0 failed, 5 ignored. cargo clippy --all-targets adds no new
warnings.
Not addressed (pre-existing)
stage_payload's tar has no --one-file-system. Valid mount names are single path
components, so no bind mount can sit inside a copied dir, but nothing in tar enforces that.
run_throwaway (the image and stopped-container probes) reads logs with no size cap. It is
unaffected by this bug and gets smaller output from this change as well.
## Problem
`get_container_staleness` failed on every container start/stop with
`OUTPUT_LIMIT: Command output exceeded 8388608 bytes and was abandoned`.
`manifest_from_container` runs `manifest_script()` through `exec_oneshot_as`, which has an
8 MiB cap. On a real container the flat `find /workspace -xdev` walk was 8,594,010 bytes by
itself: about 94k entries from git worktrees, `node_modules` and vendored trees sitting in the
writable layer next to the bind mounts. The same manifest feeds `fresh_migration` (where it
fails with `?`) and `finish_migration`.
## Design
**Why not prune by name?** I measured it on that container (read-only): pruning `node_modules`
and `.git` only got it down to 6.3 MB / 68k entries. Most of what was left was a vendored
phpMyAdmin (about 5k files) in each of the eight worktrees. A list of names would never be
complete.
**What this PR does: walk `/workspace` to depth 2 and summarise below that.**
- `SUMMARISED_ROOTS = ["/workspace"]`, `SUMMARY_DEPTH = 2`. Each non-empty directory at
depth 2 is emitted by `find` as one `S` entry. A new `###SUMMARY` section gives its size
from `du -sbx`.
- The `S` line comes from `find`, not `du`. If `du` fails, the size is 0 and only the disk
estimate is affected. The directory is never dropped.
- Empty directories at depth 2 are still printed as ordinary `d` entries, so the empty-tree
filter behaves as before.
- Why depth 2 and not 1: depth 1 holds the bind-mount points. `-xdev` lists them without
descending, but `du -x` run on a mount point would walk the whole host repository.
- Measured on the same container: the `/workspace` part drops from **8.6 MB to about 25 kB**
(377 + 138 lines).
**Why this loses nothing for migration:** below `/workspace/<x>` every per-path filter in
`compute_verbatim_paths` already has no effect:
- dpkg owns nothing there;
- there are no `COPY_EXCLUSIONS` there;
- bind targets are exactly `/workspace/<mount_name>`, and a mount name cannot contain `/`;
- the base ships an empty `/workspace` (checked).
So the verbatim set was already pruned to first-level entries, and it comes out the same.
Migration's `tar -T` archives `/workspace/<x>` recursively, so the contents of a summarised
directory are still copied in full.
Changes in `compute_verbatim_paths`:
- A summary counts as content. `is_dir()` is false for `S`, so a dir holding only summaries
is not treated as an empty tree and dropped.
- A summary is not subject to the "already in the base" check. That check is per path, and a
summary hides which paths are inside. Skipping it could drop user files under any subtree a
future base also ships. Copying is safe because restore uses `--skip-old-files`.
- `verbatim_payload_bytes` counts the summary's `du` size.
**Graceful degradation:**
- `manifest_from_container` now uses a new `exec_oneshot_as_limited` with
`MANIFEST_OUTPUT_LIMIT = 32 MiB`. The other roots (`/usr/local`, `/opt`, ...) are still
walked file by file.
- If the limit is still hit, `describe_manifest_failure` gives an actionable message instead
of the buffer error, the same approach as `describe_listing_failure`.
`CLAUDE.md` gets a bullet in the migration section explaining all this.
## Tests
- New unit tests:
- parsing `S` entries and `###SUMMARY` sizes;
- a summary with no size line is kept;
- summaries count as content and are counted in the payload;
- a summary the base also has is still copied;
- the script no longer walks `/workspace` flat;
- `describe_manifest_failure`;
- a Linux-only test that runs the real walk against a 2,000-file temp tree.
- New ignored live test `a_large_workspace_is_probed_without_overflowing_and_still_copied`:
a scratch container off the base, with 80k files (9.15 MB as a flat walk) next to a bind
mount. It checks that the probe succeeds, does not cross the mount, gets summary sizes,
and that the verbatim set is the eight worktree dirs. **It fails on the old behaviour**
(verified by reverting the two constants) and passes now.
- `cargo test`: 884 passed, 0 failed, 5 ignored. `cargo clippy --all-targets` adds no new
warnings.
## Not addressed (pre-existing)
- `stage_payload`'s `tar` has no `--one-file-system`. Valid mount names are single path
components, so no bind mount can sit inside a copied dir, but nothing in tar enforces that.
- `run_throwaway` (the image and stopped-container probes) reads logs with no size cap. It is
unaffected by this bug and gets smaller output from this change as well.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
get_container_staleness failed on every start/stop of a real container
with "OUTPUT_LIMIT: Command output exceeded 8388608 bytes": the manifest's
flat `find /workspace` walk alone was 8.6 MB (94k entries of worktrees,
node_modules, vendored trees in the writable layer). Pruning node_modules
and .git by name was measured on that container and only reached 6.3 MB.
- Walk /workspace two levels deep; each non-empty directory at depth 2 is
one `S` entry, sized from a new ###SUMMARY section (`du -sbx`). The
entry comes from find, so a failed du costs the size, never the dir.
Measured on the same container: 8.6 MB -> ~25 kB.
- compute_verbatim_paths: summaries count as content (not an empty tree)
and are exempt from presence-in-base, since a summary hides which paths
are inside; restore's --skip-old-files keeps the per-file guarantee.
The verbatim set is unchanged: it already pruned to /workspace/<x>.
- manifest_from_container gets its own 32 MiB budget via a new
exec_oneshot_as_limited, and an OUTPUT_LIMIT is reported as an
actionable sentence (describe_manifest_failure) instead of the buffer
message.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- One find pass piped through awk replaces find + du: listing and sizes
can no longer disagree under concurrent writes, and /workspace is walked
once. A summary's size is now the sum of %s over its non-directories,
exactly what verbatim_payload_bytes summed per file, so the disk gate
neither under-counts (no du-missing -> 0) nor over-counts dir inodes.
- A depth-2 dir with no file at any depth is reported as a plain `d`, so
an empty scaffold (a/b/c with no files) is dropped as before instead of
becoming a verbatim root.
- The pipeline prints ###SUMMARISED-OK only when awk succeeds;
check_manifest_complete refuses a manifest without it in both
manifest_from_container and manifest_from_image, so a failed walk can
never read as an empty /workspace.
- The presence-in-base exemption for summaries now logs a warning if the
base ever ships a summarised path, and an ignored live test pins that
the base ships nothing under /workspace.
- plain_walk() emits nothing for no roots instead of `find -xdev` (= /).
- Overflow message names /var/www and /etc too.
- exec_oneshot_limited takes a user and returns the exit code; the
duplicate exec_oneshot_as_limited is gone. MAX_ONESHOT_OUTPUT doc fixed.
- The Linux walk test now echoes a following section and asserts it
parses, asserts exact sizes and the empty-scaffold case.
Measured on the container that overflowed: whole manifest 0.99 MB.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jknapp
merged commit e5b73d8fa5 into main2026-11-03 00:31:39 +00:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Problem
get_container_stalenessfailed on every container start/stop withOUTPUT_LIMIT: Command output exceeded 8388608 bytes and was abandoned.manifest_from_containerrunsmanifest_script()throughexec_oneshot_as, which has an8 MiB cap. On a real container the flat
find /workspace -xdevwalk was 8,594,010 bytes byitself: about 94k entries from git worktrees,
node_modulesand vendored trees sitting in thewritable layer next to the bind mounts. The same manifest feeds
fresh_migration(where itfails with
?) andfinish_migration.Design
Why not prune by name? I measured it on that container (read-only): pruning
node_modulesand
.gitonly got it down to 6.3 MB / 68k entries. Most of what was left was a vendoredphpMyAdmin (about 5k files) in each of the eight worktrees. A list of names would never be
complete.
What this PR does: walk
/workspaceto depth 2 and summarise below that.SUMMARISED_ROOTS = ["/workspace"],SUMMARY_DEPTH = 2. Each non-empty directory atdepth 2 is emitted by
findas oneSentry. A new###SUMMARYsection gives its sizefrom
du -sbx.Sline comes fromfind, notdu. Ifdufails, the size is 0 and only the diskestimate is affected. The directory is never dropped.
dentries, so the empty-treefilter behaves as before.
-xdevlists them withoutdescending, but
du -xrun on a mount point would walk the whole host repository./workspacepart drops from 8.6 MB to about 25 kB(377 + 138 lines).
Why this loses nothing for migration: below
/workspace/<x>every per-path filter incompute_verbatim_pathsalready has no effect:COPY_EXCLUSIONSthere;/workspace/<mount_name>, and a mount name cannot contain/;/workspace(checked).So the verbatim set was already pruned to first-level entries, and it comes out the same.
Migration's
tar -Tarchives/workspace/<x>recursively, so the contents of a summariseddirectory are still copied in full.
Changes in
compute_verbatim_paths:is_dir()is false forS, so a dir holding only summariesis not treated as an empty tree and dropped.
summary hides which paths are inside. Skipping it could drop user files under any subtree a
future base also ships. Copying is safe because restore uses
--skip-old-files.verbatim_payload_bytescounts the summary'sdusize.Graceful degradation:
manifest_from_containernow uses a newexec_oneshot_as_limitedwithMANIFEST_OUTPUT_LIMIT = 32 MiB. The other roots (/usr/local,/opt, ...) are stillwalked file by file.
describe_manifest_failuregives an actionable message insteadof the buffer error, the same approach as
describe_listing_failure.CLAUDE.mdgets a bullet in the migration section explaining all this.Tests
Sentries and###SUMMARYsizes;/workspaceflat;describe_manifest_failure;a_large_workspace_is_probed_without_overflowing_and_still_copied:a scratch container off the base, with 80k files (9.15 MB as a flat walk) next to a bind
mount. It checks that the probe succeeds, does not cross the mount, gets summary sizes,
and that the verbatim set is the eight worktree dirs. It fails on the old behaviour
(verified by reverting the two constants) and passes now.
cargo test: 884 passed, 0 failed, 5 ignored.cargo clippy --all-targetsadds no newwarnings.
Not addressed (pre-existing)
stage_payload'starhas no--one-file-system. Valid mount names are single pathcomponents, so no bind mount can sit inside a copied dir, but nothing in tar enforces that.
run_throwaway(the image and stopped-container probes) reads logs with no size cap. It isunaffected by this bug and gets smaller output from this change as well.
🤖 Generated with Claude Code