fix(ols): scope the htaccess watcher to docroots, stop lswsctrl status log spam
ols-htaccess-watcher.sh matched .htaccess by basename only, so ANY .htaccess under a tenant (WordPress plugin guard files, not just the docroot OLS reads) triggered a full graceful restart. Measured on whp01 over 24h: 63 restarts, 0 of them from a docroot .htaccess actually changing — all from Wordfence/W3TC/ WPForms/etc. self-healing files, mostly on tenants that aren't even on this tier. Now matches the full path (%w%f) against */public_html/.htaccess, the only .htaccess OLS ever reads, and logs which path triggered each restart. entrypoint-shared-ols.sh's 3s supervisor poll called `lswsctrl status`, which appends a line to lsrestart.log on every invocation. Measured on whp01: 1,819,286 status lines vs 2,429 real restarts in a 96 MB, never-rotated log. ols_running() now checks the process table directly (pgrep -f 'lshttpd - main', verified against the litespeedtech/openlitespeed base image) instead of shelling out to a logging tool on a fixed timer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -15,6 +15,20 @@
|
||||
## runs it and the panel monitors it (check-ols-htaccess-watcher.php).
|
||||
set -uo pipefail
|
||||
|
||||
## WATCH_ROOT is deliberately left as the host-wide /mnt/users, not narrowed to
|
||||
## the shared-OLS tenant set, even though that set IS derivable in-container
|
||||
## (render-shared-ols-config.sh's $SITES_ROOT/*/site.meta VHROOT= is exactly
|
||||
## that list). Narrowing it would mean handing inotifywait a fixed argv list of
|
||||
## VHROOT dirs at process start — and inotifywait cannot be told to watch a NEW
|
||||
## directory once running. The panel provisions sites onto this container live,
|
||||
## between renders; a site added after the watcher started would then sit
|
||||
## outside every watch until the next container restart, i.e. exactly the
|
||||
## silent-failure mode (spec 7) this script exists to prevent, now for brand
|
||||
## new tenants instead of none. Doing this safely needs a reload path (SIGHUP
|
||||
## re-exec off the current site.meta list, coordinated with
|
||||
## render-shared-ols-config.sh) that does not exist yet and is its own change.
|
||||
## So: WATCH_ROOT stays broad, and correctness comes entirely from the path
|
||||
## match below, which is sufficient on its own.
|
||||
WATCH_ROOT="${OLS_WATCH_ROOT:-/mnt/users}"
|
||||
DEBOUNCE="${OLS_HTACCESS_DEBOUNCE:-15}" # coalesce window (s)
|
||||
FLOOR="${OLS_HTACCESS_FLOOR:-60}" # min seconds between restarts
|
||||
@@ -24,16 +38,17 @@ last_restart=0
|
||||
log() { echo "ols-htaccess-watcher: $*" >&2; }
|
||||
|
||||
do_restart() {
|
||||
path="$1"
|
||||
now=$(date +%s)
|
||||
if [ $((now - last_restart)) -lt "$FLOOR" ]; then
|
||||
log "within ${FLOOR}s floor — coalescing, skipping restart"
|
||||
log "within ${FLOOR}s floor — coalescing, skipping restart ($path)"
|
||||
return
|
||||
fi
|
||||
if "$LSWSCTRL" restart >/dev/null 2>&1; then
|
||||
last_restart=$now
|
||||
log "graceful restart issued (.htaccess change)"
|
||||
log "graceful restart issued — $path changed"
|
||||
else
|
||||
log "WARNING: lswsctrl restart failed"
|
||||
log "WARNING: lswsctrl restart failed ($path)"
|
||||
fi
|
||||
}
|
||||
|
||||
@@ -41,18 +56,34 @@ if ! command -v inotifywait >/dev/null 2>&1; then
|
||||
log "FATAL: inotifywait not installed (inotify-tools)"; exit 1
|
||||
fi
|
||||
mkdir -p "$WATCH_ROOT"
|
||||
log "watching $WATCH_ROOT for .htaccess changes (debounce=${DEBOUNCE}s floor=${FLOOR}s)"
|
||||
log "watching $WATCH_ROOT for docroot (public_html) .htaccess changes (debounce=${DEBOUNCE}s floor=${FLOOR}s)"
|
||||
|
||||
## -m monitor, -r recursive. We filter to .htaccess in the read loop rather than
|
||||
## --include so this works on older inotify-tools too. modify/create/delete/move
|
||||
## all matter (delete of .htaccess also changes rewrite behavior).
|
||||
inotifywait -m -r -e modify,create,delete,move "$WATCH_ROOT" --format '%f' 2>/dev/null |
|
||||
while read -r fname; do
|
||||
case "$fname" in
|
||||
.htaccess) ;;
|
||||
## -m monitor, -r recursive. We filter in the read loop rather than --include
|
||||
## so this works on older inotify-tools too. modify/create/delete/move all
|
||||
## matter (delete of .htaccess also changes rewrite behavior).
|
||||
##
|
||||
## --format '%w%f' (full path), NOT '%f' (basename only). OLS reads .htaccess
|
||||
## (RewriteFile) only from a vhost's DOCROOT — VHROOT, i.e.
|
||||
## /mnt/users/<user>/<domain>/public_html (see render-shared-ols-config.sh /
|
||||
## entrypoint-lsphp.sh) — never anything below it. A basename-only match fires
|
||||
## for ANY .htaccess anywhere under a tenant, at any depth, and WordPress
|
||||
## plugins write plenty of those that OLS never opens: measured on whp01 over
|
||||
## 24h, this watcher fired 63 restarts, of which the docroot .htaccess actually
|
||||
## changed in 0. All 28 distinct files behind those 63 were plugin guard files
|
||||
## — Wordfence self-healing waf/views/vendor/tmp/models/lib/.htaccess, W3 Total
|
||||
## Cache writing one per cached URL under wp-content/cache/page_enhanced/, plus
|
||||
## WPForms/Gravity Forms/UpdraftPlus/Groundhogg/WP Staging upload guards — and
|
||||
## most of those tenants are on the shared Apache tier (cac-fpm), not this OLS
|
||||
## tier at all, so their cache churn was restarting the OLS serving 15 unrelated
|
||||
## tenants for no reason. Matching the full path down to /public_html/.htaccess
|
||||
## is what actually ties a change to something OLS will reread.
|
||||
inotifywait -m -r -e modify,create,delete,move "$WATCH_ROOT" --format '%w%f' 2>/dev/null |
|
||||
while read -r path; do
|
||||
case "$path" in
|
||||
*/public_html/.htaccess) ;;
|
||||
*) continue ;;
|
||||
esac
|
||||
## A tenant .htaccess changed. Coalesce the save-burst, then restart ONCE.
|
||||
## A tenant DOCROOT .htaccess changed. Coalesce the save-burst, then restart ONCE.
|
||||
##
|
||||
## The coalesce is HARD-BOUNDED to DEBOUNCE seconds: a previous version blocked
|
||||
## on `read -t DEBOUNCE` which, on a busy multi-tenant server, never timed out
|
||||
@@ -69,5 +100,5 @@ while read -r fname; do
|
||||
break # ~2s of total quiet — the burst has settled
|
||||
fi
|
||||
done
|
||||
do_restart
|
||||
do_restart "$path"
|
||||
done
|
||||
|
||||
Reference in New Issue
Block a user