extract: opt-in mail tree (IMPORT_INCLUDE_MAIL=true) + mail_included marker

WHP's cPanel mailbox import needs cpmove-*/homedir/mail, which extract.sh
always excluded: in sanitization-container mode every mailbox was queued
with no Maildir and "verified" empty (B5, whp02 canary 2026-10-20).

- IMPORT_INCLUDE_MAIL=true (exactly "true"; default off) keeps the mail
  tree in the extract. Default behaviour is unchanged. The DANGEROUS
  symlink strips still apply inside the mail tree (scan-symlinks.php lists
  every archive entry).
- scan-files.php --skip-mail (passed by entrypoint.sh with the opt-in):
  clamscan --exclude-dir for the mail tree, the file count skips it, and a
  hit inside it is never quarantined. Logged with the reason: mail is
  copied verbatim into the customer's own mailbox and verified against the
  source, and multi-GB maildirs would dominate the run.
- report.json gains "mail_included" (true only with the opt-in) so the
  panel can tell this image from an older one, which never writes the key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
shadowdaoandClaude Opus 5.5 committed 2026-10-20 09:04:44 -07:00
1 parent 08b995a29c
commit 8217397154
5 files changed
+111 -22

No files matched your search

+28 -14
View File
@@ -4,6 +4,10 @@
#
# Usage: extract.sh <tarball> <dest> <username> [<actions_out>]
#
# Env: IMPORT_INCLUDE_MAIL "true" = extract cpmove-*/homedir/mail too
# (WHP's mailbox import needs the Maildirs).
# Anything else (default) = skip the mail tree.
#
# Calls scripts/lib/scan-symlinks.php first, then untars the cpmove with
# every DANGEROUS-classified symlink entry stripped via tar --exclude.
# The stripped-symlinks list is written as JSON to <actions_out> (default
@@ -21,6 +25,11 @@ DEST="${2:?usage: extract.sh <tarball> <dest> <username> [<actions_out>]}"
USERNAME="${3:?usage: extract.sh <tarball> <dest> <username> [<actions_out>]}"
ACTIONS_OUT="${4:-${DEST}/.cpanel-importer-stripped-symlinks.json}"
IMPORT_INCLUDE_MAIL="${IMPORT_INCLUDE_MAIL:-false}"
# Resolve sibling scripts relative to this file (/scripts in the image) so
# tests/run-tests.sh can run the same script from a checkout.
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ts() { date -u +'%Y-%m-%dT%H:%M:%SZ'; }
log() { printf '[%s] extract: %s\n' "$(ts)" "$*"; }
@@ -31,7 +40,7 @@ mkdir -p "$DEST"
log "scanning tarball for dangerous symlinks (cpmove vector check)"
SYMLINK_REPORT=$(mktemp -p /tmp scan-symlinks.XXXXXX.json)
if ! php /scripts/lib/scan-symlinks.php \
if ! php "$SCRIPT_DIR/lib/scan-symlinks.php" \
--tarball "$TARBALL" \
--username "$USERNAME" \
--report "$SYMLINK_REPORT"; then
@@ -86,19 +95,24 @@ if [[ "$DANGEROUS_COUNT" -gt 0 ]]; then
done < "$EXCLUDES_FILE"
fi
# Also skip the cpanel user's mail spool entirely. WHP doesn't import
# email (mail-import is a panel-side roadmap item, not a sandbox-mode
# step) so extracting + scanning the mailbox tree wastes time and disk:
# on real customer accounts the mail dir often dwarfs everything else
# (10+ GB of historical mbox/maildir). The exclude pattern uses tar's
# fnmatch globs and covers both maildir + spool layouts at any depth
# under homedir/. Logged as an info line so it shows up in the panel
# import log alongside the symlink strips.
{
echo "cpmove-*/homedir/mail"
echo "cpmove-*/homedir/mail/*"
} >> "$EXCLUDES_FILE"
log "skipping mail tree (cpmove-*/homedir/mail) — WHP does not import cPanel mail"
# Mail tree (cpmove-*/homedir/mail). Skipped by default: a plain site
# import doesn't need it, and on real customer accounts the mail dir often
# dwarfs everything else (10+ GB of historical maildir). The exclude
# pattern uses tar's fnmatch globs and covers the whole tree at any depth.
#
# IMPORT_INCLUDE_MAIL=true (WHP passes it only when the import copies
# mailboxes) extracts it. The DANGEROUS-symlink strips computed above apply
# to the mail tree exactly as to the rest of the archive: scan-symlinks.php
# lists every entry in the tarball, mail included.
if [[ "$IMPORT_INCLUDE_MAIL" == "true" ]]; then
log "including mail tree (cpmove-*/homedir/mail) — IMPORT_INCLUDE_MAIL=true"
else
{
echo "cpmove-*/homedir/mail"
echo "cpmove-*/homedir/mail/*"
} >> "$EXCLUDES_FILE"
log "skipping mail tree (cpmove-*/homedir/mail) — IMPORT_INCLUDE_MAIL not set"
fi
# --- extract --------------------------------------------------------------