Two independent sets of fixes.
## Disk: stop the growth, no UI this round
The dangling-snapshot sweep was already correct and was never the leak. The
leak is that every `docker commit` **stacks** a layer and nothing compacts one:
a file deleted after it has been committed becomes a whiteout, not free bytes.
24 conditions trigger recreation+commit, so changing one settings field costs a
multi-gigabyte layer for the life of the project. One project was measured with
14 stacked commit layers, ~5.1 GB above its base.
* **Scrub the writable layer before every commit** (`docker/container.rs`,
`SNAPSHOT_SCRUB_PATHS` / `scrub_writable_layer`). The one moment those bytes
are still free to drop is before the commit that captures them. Measured on
one container's 4.48 GB pending layer: 3.0 GB of agent scratchpad under
`/tmp/claude-*`, the terminal drag-drop staging area (256 MiB per file, with
no `rm` for it anywhere in the repo), a PNG per pasted image, and the apt
lists/cache/logs that `browser_view/install.rs` and `triple-c-playwright-heal`
leave behind with no `apt-get clean`. A hardcoded list, never a heuristic:
`/workspace/{mount_name}` is a host bind mount and nothing here may reach one,
and the three `/tmp` globs cannot select the read-only `.host-ca`/`.host-aws`
mounts. Failure is a log line — a scrub must never block a snapshot.
* **Cap container logs** (`capped_log_config`). There was no `LogConfig`
anywhere, so containers ran on the daemon's unbounded `json-file` default.
Deliberately *not* wired into `container_needs_recreation`: participating
would recreate every project once, and a recreation costs a commit, which is
the thing being fixed. Picked up on the next natural recreation.
* **Make superseded base images sweepable** (`container/Dockerfile`). It carried
no `LABEL` at all, so `orphan_sweep_filters`' `dangling` + `triple-c.managed`
pair provably could not match one — ~11.9 GB observed stranded. Stamping
`triple-c.managed=true` is the whole fix; the sweep needed no change.
`create_container` writes the new `triple-c.base` key explicitly empty, or
Docker's label inheritance plus `docker commit` would make every snapshot
claim to be a base image. `force: false` stays, and now says why.
* **Sweep at startup** (`lib.rs`), not only after recreation: probes first
(a probe pins an image the unforced sweep then refuses), pins second, sweep
last. `sweep_orphaned_snapshots_logged` exists because all three callers threw
the report away — `reclaimed_bytes`, `failed` and `unavailable` included.
* **Reap migration leftovers.** `rollback_migration` retagged and orphaned the
migrated snapshot with no sweep. Stale `pre-migration-*` pins are now
age-reaped by scanning the tag pattern rather than trusting the state file —
`migration_store::load` reports an unparseable record as absent, which
stranded a 4-12 GB pin nothing could name again; `load` now moves a corrupt
record aside so `has_record` is trustworthy. A pin whose migration is still
awaiting confirmation is never reaped at any age. The probe container's
removal was a plain statement after an await, so a dropped future (an app quit
mid-migration) leaked a container pinning a multi-gigabyte image; it is a
`Drop` guard now, with `reap_probe_containers` for the case where the process
itself dies.
* **Prune scheduler logs.** `remove` deleted a task's JSON but never its log
directory, and the task runner appended uncapped `claude -p` output.
* **Fix the delete copy.** It said "the container, config volume, and stored
credentials"; it removes *both* volumes and the snapshot image.
No prune UI, and no unfiltered `prune_images`/`prune_volumes` anywhere — the
daemon is shared with the user's unrelated work.
## Claude Code settings: two invented keys, one inverted default, one sticky bug
Verified against code.claude.com/docs/en/settings-reference.md and env-vars.md.
* `effort` -> **`effortLevel`**, the key Claude Code actually reads; the old one
was written and silently ignored. `xhigh` added to the dropdown.
* `focusMode` -> **`viewMode: "focus"`**. `focusMode` was invented. The real key
does exactly what the existing UI hint already described.
* **Session recap was inverted.** Claude Code's recap is on by default, so
`CLAUDE_CODE_ENABLE_AWAY_SUMMARY=1`-when-enabled was a no-op and the control
could never turn the recap *off*. The field is renamed to
`session_recap_disabled` rather than reused: reusing the name with the
opposite meaning would have read every stored `enable_session_recap: false` —
which is every project that never touched the control — as "the user turned
this off".
* **The stickiness, which is the important one.** Keys were emitted only when
non-default, and the entrypoint *merges* into a settings.json on a persisted
volume, so switching a setting off omitted its key, the merge preserved the
stale on-value, and the setting stayed on until a destructive Reset. The fix
already existed in the same file — the sandbox block is emitted
unconditionally for exactly this reason — and is now applied to all five keys.
A key whose neutral state is *unset* (`tui`, `effortLevel`, `viewMode`,
`awaySummaryEnabled`) is emitted as JSON `null` and the entrypoint deletes it,
because a stand-in value is not neutral: `tui: "default"` pins the classic
renderer where unset lets Claude Code choose, and `viewMode: "default"`
overrides the user's own sticky `/focus` choice.
* The same stickiness existed, unnoticed, in the **env vars**: `docker commit`
bakes container env into the snapshot image, so a `=1` written once rode it
forever. All four are now emitted on every create, extracted into
`claude_code_env_vars` and unit tested. Two use an empty value for "off"
rather than `0`, because they outrank a setting the user can change from
inside their own container and Triple-C's default must not overrule a
`/config` choice it never asked about.
* TUI mode is now a genuine three-way choice (automatic / classic / fullscreen),
which the always-emitted key makes both necessary and possible.
`merge_claude_code_settings` is untouched by choice: a project-level OFF still
cannot override a globally-ON setting.
Tests: 364 frontend (+5), 308 Rust (+23), covering the scrub path list and
script, log rotation, pin reaping, and that toggling a setting off actually
clears a previously-set ON value.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GBq2rGum6GX7xXgsas1fDc
250 lines
11 KiB
Bash
250 lines
11 KiB
Bash
#!/bin/bash
|
|
# triple-c-task-runner — Executes a scheduled task via Claude Code agent
|
|
# Called by cron with a task ID argument. Handles locking, logging,
|
|
# notifications, one-time task cleanup, and log pruning.
|
|
|
|
set -uo pipefail
|
|
|
|
SCHEDULER_DIR="${HOME}/.claude/scheduler"
|
|
TASKS_DIR="${SCHEDULER_DIR}/tasks"
|
|
LOGS_DIR="${SCHEDULER_DIR}/logs"
|
|
NOTIFICATIONS_DIR="${SCHEDULER_DIR}/notifications"
|
|
RUNNING_DIR="${SCHEDULER_DIR}/running"
|
|
ENV_FILE="${SCHEDULER_DIR}/.env"
|
|
|
|
TASK_ID="${1:-}"
|
|
|
|
if [ -z "$TASK_ID" ]; then
|
|
echo "Usage: triple-c-task-runner <task-id>" >&2
|
|
exit 1
|
|
fi
|
|
|
|
TASK_FILE="${TASKS_DIR}/${TASK_ID}.json"
|
|
LOCK_FILE="${SCHEDULER_DIR}/.lock-${TASK_ID}"
|
|
|
|
if [ ! -f "$TASK_FILE" ]; then
|
|
echo "Task file not found: $TASK_FILE" >&2
|
|
exit 1
|
|
fi
|
|
|
|
# ── Acquire lock (prevent overlapping runs of the same task) ─────────────────
|
|
exec 200>"$LOCK_FILE"
|
|
if ! flock -n 200; then
|
|
echo "Task $TASK_ID is already running, skipping." >&2
|
|
exit 0
|
|
fi
|
|
|
|
# ── Source saved environment ─────────────────────────────────────────────────
|
|
# The env file is a snapshot taken by the entrypoint, which runs as root. A
|
|
# snapshot written before the entrypoint stopped capturing HOME still carries
|
|
# HOME=/root, and `set -a` would apply it to `claude` below — which then finds no
|
|
# credential under /root/.claude and exits with "Not logged in". The env file
|
|
# lives on the home volume, so those stale copies outlive an image update until
|
|
# the container is restarted; keep our own HOME regardless of what it says.
|
|
if [ -f "$ENV_FILE" ]; then
|
|
REAL_HOME="${HOME:-/home/claude}"
|
|
set -a
|
|
# shellcheck disable=SC1090
|
|
source "$ENV_FILE"
|
|
set +a
|
|
HOME="$REAL_HOME"
|
|
fi
|
|
|
|
# ── Read task definition ────────────────────────────────────────────────────
|
|
PROMPT=$(jq -r '.prompt' "$TASK_FILE")
|
|
WORKING_DIR=$(jq -r '.working_dir // "/workspace"' "$TASK_FILE")
|
|
TASK_NAME=$(jq -r '.name' "$TASK_FILE")
|
|
TASK_TYPE=$(jq -r '.type' "$TASK_FILE")
|
|
|
|
# ── Resolve permission mode ─────────────────────────────────────────────────
|
|
# TRIPLE_C_PERMISSION_MODE is injected into the container by Triple-C from the
|
|
# project's permission setting. Keep this mapping in sync with
|
|
# PermissionMode::cli_args() in app/src-tauri/src/models/project.rs.
|
|
# NOTE: headless `claude -p` runs cannot answer a permission prompt, so any
|
|
# mode other than "bypass" means the task may stop early when Claude Code asks
|
|
# for permission. Unset or unrecognized values pass no flag (Claude's default).
|
|
PERMISSION_ARGS=()
|
|
case "${TRIPLE_C_PERMISSION_MODE:-}" in
|
|
plan) PERMISSION_ARGS=(--permission-mode plan) ;;
|
|
acceptEdits) PERMISSION_ARGS=(--permission-mode acceptEdits) ;;
|
|
bypass) PERMISSION_ARGS=(--dangerously-skip-permissions) ;;
|
|
*) PERMISSION_ARGS=() ;;
|
|
esac
|
|
|
|
# ── Prepare log directory ───────────────────────────────────────────────────
|
|
TASK_LOG_DIR="${LOGS_DIR}/${TASK_ID}"
|
|
mkdir -p "$TASK_LOG_DIR"
|
|
|
|
TIMESTAMP=$(date +"%Y%m%d-%H%M%S")
|
|
LOG_FILE="${TASK_LOG_DIR}/${TIMESTAMP}.log"
|
|
|
|
# ── Publish run state ───────────────────────────────────────────────────────
|
|
# A scheduled run is detached — cron has no terminal, and the app fires it as a
|
|
# detached exec — so without this there is no way to tell a task that is still
|
|
# thinking from one that died, and a long run reads as a stall. `list`, `status`
|
|
# and the app's Automation tab all read this file.
|
|
#
|
|
# flock above is what actually prevents overlapping runs; this is purely an
|
|
# observability record, which is why readers verify the pid rather than trust
|
|
# the file. The EXIT trap covers the crash paths (OOM, container stop, SIGTERM)
|
|
# that would otherwise leave a task looking like it had been running for days.
|
|
mkdir -p "$RUNNING_DIR"
|
|
RUN_STATE="${RUNNING_DIR}/${TASK_ID}.json"
|
|
trap 'rm -f "$RUN_STATE"' EXIT
|
|
jq -n \
|
|
--arg pid "$$" \
|
|
--arg started "$(date +%s)" \
|
|
--arg log "$LOG_FILE" \
|
|
--arg name "$TASK_NAME" \
|
|
'{pid: ($pid | tonumber), started_epoch: ($started | tonumber), log: $log, name: $name}' \
|
|
> "$RUN_STATE"
|
|
|
|
# ── Execute Claude agent ────────────────────────────────────────────────────
|
|
{
|
|
echo "=== Task: $TASK_NAME ($TASK_ID) ==="
|
|
echo "=== Started: $(date) ==="
|
|
echo "=== Working dir: $WORKING_DIR ==="
|
|
echo "=== Prompt: $PROMPT ==="
|
|
echo "=== Permission mode: ${TRIPLE_C_PERMISSION_MODE:-default} ==="
|
|
echo ""
|
|
} > "$LOG_FILE"
|
|
|
|
EXIT_CODE=0
|
|
if [ -d "$WORKING_DIR" ]; then
|
|
cd "$WORKING_DIR"
|
|
# ${arr[@]+"${arr[@]}"} keeps an empty array safe under `set -u`
|
|
claude -p "$PROMPT" ${PERMISSION_ARGS[@]+"${PERMISSION_ARGS[@]}"} >> "$LOG_FILE" 2>&1 || EXIT_CODE=$?
|
|
else
|
|
echo "Error: working directory '$WORKING_DIR' does not exist" >> "$LOG_FILE"
|
|
EXIT_CODE=1
|
|
fi
|
|
|
|
{
|
|
echo ""
|
|
echo "=== Finished: $(date) ==="
|
|
echo "=== Exit code: $EXIT_CODE ==="
|
|
} >> "$LOG_FILE"
|
|
|
|
# ── Cap the size of this run's log ──────────────────────────────────────────
|
|
# `claude -p` output is unbounded — a task told to walk a large tree can emit
|
|
# hundreds of megabytes in one run — and the pruning below counts *files*, not
|
|
# bytes, so twenty logs of any size are twenty logs. One chatty task can
|
|
# therefore fill the home volume, which is also where ~/.claude and the OAuth
|
|
# credential live.
|
|
#
|
|
# The tail is the half worth keeping: `claude -p` writes its answer at the end,
|
|
# and the footer just appended carries the exit code that `status` and the app
|
|
# both grep for. So an oversize log is rewritten as a marker line plus its last
|
|
# MAX_LOG_BYTES rather than being deleted or capped from the front. This runs
|
|
# before the notification below so the summary is taken from the capped file.
|
|
#
|
|
# Best effort throughout: the run's real result is already recorded, so a
|
|
# failure here must not change the exit status. Note that `run` may be tailing
|
|
# this file — it has already streamed everything up to here, and nothing is
|
|
# appended after this point, so replacing the inode is invisible to it.
|
|
MAX_LOG_BYTES=$(( 5 * 1024 * 1024 ))
|
|
LOG_BYTES=$(wc -c < "$LOG_FILE" 2>/dev/null || echo 0)
|
|
if [ "${LOG_BYTES:-0}" -gt "$MAX_LOG_BYTES" ]; then
|
|
TRUNC_FILE="${LOG_FILE}.trunc"
|
|
if {
|
|
echo "=== Log truncated: $(( LOG_BYTES - MAX_LOG_BYTES )) bytes dropped from the start (cap ${MAX_LOG_BYTES} bytes) ==="
|
|
tail -c "$MAX_LOG_BYTES" "$LOG_FILE"
|
|
} > "$TRUNC_FILE" 2>/dev/null; then
|
|
mv -f "$TRUNC_FILE" "$LOG_FILE" 2>/dev/null || rm -f "$TRUNC_FILE"
|
|
else
|
|
rm -f "$TRUNC_FILE"
|
|
fi
|
|
fi
|
|
|
|
# ── Write notification ──────────────────────────────────────────────────────
|
|
mkdir -p "$NOTIFICATIONS_DIR"
|
|
NOTIFY_FILE="${NOTIFICATIONS_DIR}/${TASK_ID}_${TIMESTAMP}.notify"
|
|
|
|
if [ $EXIT_CODE -eq 0 ]; then
|
|
STATUS="SUCCESS"
|
|
else
|
|
STATUS="FAILED (exit code $EXIT_CODE)"
|
|
fi
|
|
|
|
# Extract a summary (last 10 meaningful lines before the footer)
|
|
SUMMARY=$(grep -v "^===" "$LOG_FILE" | grep -v "^$" | tail -n 10)
|
|
|
|
cat > "$NOTIFY_FILE" <<NOTIFY
|
|
Task: $TASK_NAME ($TASK_ID)
|
|
Status: $STATUS
|
|
Time: $(date)
|
|
Type: $TASK_TYPE
|
|
|
|
Summary:
|
|
$SUMMARY
|
|
NOTIFY
|
|
|
|
# ── One-time task cleanup ───────────────────────────────────────────────────
|
|
if [ "$TASK_TYPE" = "once" ]; then
|
|
rm -f "$TASK_FILE"
|
|
# Rebuild crontab to remove the completed one-time task
|
|
/usr/local/bin/triple-c-scheduler list > /dev/null 2>&1 || true
|
|
# Direct crontab rebuild (in case scheduler list doesn't trigger it)
|
|
TMP_CRON=$(mktemp)
|
|
echo "# Triple-C scheduled tasks — managed by triple-c-scheduler" > "$TMP_CRON"
|
|
echo "# Do not edit manually; changes will be overwritten." >> "$TMP_CRON"
|
|
echo "" >> "$TMP_CRON"
|
|
for tf in "$TASKS_DIR"/*.json; do
|
|
[ -f "$tf" ] || continue
|
|
local_enabled=$(jq -r '.enabled' "$tf")
|
|
[ "$local_enabled" = "true" ] || continue
|
|
local_schedule=$(jq -r '.schedule' "$tf")
|
|
local_id=$(jq -r '.id' "$tf")
|
|
echo "$local_schedule /usr/local/bin/triple-c-task-runner $local_id" >> "$TMP_CRON"
|
|
done
|
|
crontab "$TMP_CRON" 2>/dev/null || true
|
|
rm -f "$TMP_CRON"
|
|
fi
|
|
|
|
# ── Prune old logs (keep 20 per task) ───────────────────────────────────────
|
|
LOG_COUNT=$(find "$TASK_LOG_DIR" -name "*.log" -type f 2>/dev/null | wc -l)
|
|
if [ "$LOG_COUNT" -gt 20 ]; then
|
|
find "$TASK_LOG_DIR" -name "*.log" -type f | sort | head -n $((LOG_COUNT - 20)) | xargs rm -f
|
|
fi
|
|
|
|
# ── Reap log dirs of tasks that no longer exist ─────────────────────────────
|
|
# `triple-c-scheduler remove` deletes a task's log dir with the task, but a
|
|
# one-time task deletes its own task file above, so `remove` can never be run
|
|
# for it — nothing knows the id any more — and its directory would sit on the
|
|
# home volume forever. This is the sweep for that case.
|
|
#
|
|
# Deliberately delayed rather than done in the cleanup above: the run that just
|
|
# finished has only just written the sole record of itself, `run` and the app's
|
|
# Automation tab may still be tailing it, and `logs --id` keeps working for a
|
|
# task whose file is gone. So a dir is reaped only once nothing in it has been
|
|
# touched for LOG_RETENTION_DAYS, and never while a run is publishing state for
|
|
# that id. The sweep rides on task runs, so a container whose only task was
|
|
# one-time keeps that one directory until something else runs.
|
|
#
|
|
# Same paranoia as reap_task_logs() in triple-c-scheduler: the id comes from a
|
|
# directory name and is re-validated before it is used to build an `rm -rf`
|
|
# path, so no empty or path-bearing name can reach beyond $LOGS_DIR.
|
|
LOG_RETENTION_DAYS=7
|
|
for ORPHAN_DIR in "$LOGS_DIR"/*/; do
|
|
[ -d "$ORPHAN_DIR" ] || continue
|
|
ORPHAN_ID=$(basename "$ORPHAN_DIR")
|
|
[[ "$ORPHAN_ID" =~ ^[A-Za-z0-9][A-Za-z0-9_-]*$ ]] || continue
|
|
[ -f "${TASKS_DIR}/${ORPHAN_ID}.json" ] && continue
|
|
[ -f "${RUNNING_DIR}/${ORPHAN_ID}.json" ] && continue
|
|
# Anything modified inside the window keeps the whole directory.
|
|
[ -n "$(find "$ORPHAN_DIR" -mmin "-$(( LOG_RETENTION_DAYS * 1440 ))" -print -quit 2>/dev/null)" ] && continue
|
|
rm -rf -- "${LOGS_DIR:?}/${ORPHAN_ID}"
|
|
done
|
|
|
|
# ── Prune old notifications (keep 50 total) ─────────────────────────────────
|
|
NOTIFY_COUNT=$(find "$NOTIFICATIONS_DIR" -name "*.notify" -type f 2>/dev/null | wc -l)
|
|
if [ "$NOTIFY_COUNT" -gt 50 ]; then
|
|
find "$NOTIFICATIONS_DIR" -name "*.notify" -type f | sort | head -n $((NOTIFY_COUNT - 50)) | xargs rm -f
|
|
fi
|
|
|
|
# Release lock
|
|
flock -u 200
|
|
rm -f "$LOCK_FILE"
|
|
|
|
exit $EXIT_CODE
|