fix(ols): scope the htaccess watcher to docroots, stop lswsctrl status log spam
ols-htaccess-watcher.sh matched .htaccess by basename only, so ANY .htaccess under a tenant (WordPress plugin guard files, not just the docroot OLS reads) triggered a full graceful restart. Measured on whp01 over 24h: 63 restarts, 0 of them from a docroot .htaccess actually changing — all from Wordfence/W3TC/ WPForms/etc. self-healing files, mostly on tenants that aren't even on this tier. Now matches the full path (%w%f) against */public_html/.htaccess, the only .htaccess OLS ever reads, and logs which path triggered each restart. entrypoint-shared-ols.sh's 3s supervisor poll called `lswsctrl status`, which appends a line to lsrestart.log on every invocation. Measured on whp01: 1,819,286 status lines vs 2,429 real restarts in a 96 MB, never-rotated log. ols_running() now checks the process table directly (pgrep -f 'lshttpd - main', verified against the litespeedtech/openlitespeed base image) instead of shelling out to a logging tool on a fixed timer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -75,19 +75,47 @@ term_handler() {
|
||||
}
|
||||
trap term_handler TERM INT
|
||||
|
||||
## Variable + here-string, not a pipe into `grep -qi` — see the long note on the
|
||||
## identical function in entrypoint-litespeed.sh: `grep -q` closing the pipe on
|
||||
## a match can leave the writer dying 141, and `set -o pipefail` (line 14) turns
|
||||
## that into "OLS is down" *because* the running line matched. The reason is
|
||||
## structural (a pipefail script must not pipe into an early-exit reader), not
|
||||
## that this particular output is small; and the here-string is safe here for
|
||||
## the separate reason that `lswsctrl status` is far below the size at which
|
||||
## bash spills a here-string to a temp file. A non-zero `lswsctrl` still counts
|
||||
## as not running, as pipefail made it count before.
|
||||
## NOT `lswsctrl status` (unlike the otherwise-identical function in
|
||||
## entrypoint-litespeed.sh). `lswsctrl` appends a timestamped line to
|
||||
## logs/lsrestart.log on EVERY invocation it makes, including `status` — and
|
||||
## this loop polls every 3s forever. Measured on whp01: lsrestart.log is 96 MB,
|
||||
## holding 1,819,286 `status` lines against 2,429 real `restart` lines; at one
|
||||
## poll per 3s that's ~63 days of continuous polling, which is exactly the
|
||||
## file's age, and it isn't rotated on any host (whp01/whp02/sdbees all growing
|
||||
## at ~1.5 MB/day). So: check liveness directly instead of shelling out to a
|
||||
## tool whose logging is a side effect we don't want on a fixed timer.
|
||||
##
|
||||
## Verified (docker run litespeedtech/openlitespeed:1.8.4-lsphp83, the exact
|
||||
## base this image is built FROM — see Dockerfile.shared-ols): the running main
|
||||
## process shows in `ps` as `openlitespeed (lshttpd - main)`, one PID, always
|
||||
## present while OLS is up and absent the instant it is killed (checked via
|
||||
## `ps aux` immediately after `kill -9` on the main PID). `pgrep -f` matches
|
||||
## against the full command line, and no other process on this image's `ps`
|
||||
## output contains that string, so this cannot cross-match an unrelated
|
||||
## process. It also cannot self-match: pgrep excludes its own PID by default,
|
||||
## and the invoking process here is bash executing this script file, whose own
|
||||
## argv never contains the pattern text (only the *source lines* of this script
|
||||
## do, which `pgrep -f` never sees).
|
||||
##
|
||||
## Deliberately NOT the pidfile (/tmp/lshttpd/lshttpd.pid, confirmed present in
|
||||
## the same probe): pidfiles are known to go stale across a crash (verified —
|
||||
## after `kill -9` the file still held the dead PID), and treating a stale PID
|
||||
## as "alive" if the kernel ever reuses that number is a false positive this
|
||||
## supervisor cannot afford (see below). `pgrep -f` reads the live process
|
||||
## table, so there is no staleness window to reason about.
|
||||
##
|
||||
## Conservative on both failure directions, which matters because this is a
|
||||
## supervisor predicate, not a metric: a false negative makes start_ols() run
|
||||
## `lswsctrl start` against an already-running OLS — verified against the same
|
||||
## probe base image, that is NOT a no-op, it sends SIGUSR1 to the live main
|
||||
## process, i.e. the same graceful self-restart QUIC.cloud IP refreshes trigger
|
||||
## (see entrypoint-litespeed.sh's note on that handoff) — a brief, zero-
|
||||
## downtime blip at worst. A false positive is worse: it leaves a genuinely
|
||||
## dead OLS un-revived until some later poll happens to notice. So if this
|
||||
## predicate is ever in doubt it should err toward reporting "not running", not
|
||||
## "running".
|
||||
ols_running() {
|
||||
local st
|
||||
st=$(/usr/local/lsws/bin/lswsctrl status 2>/dev/null) || return 1
|
||||
grep -qi 'running with pid' <<<"$st"
|
||||
pgrep -f 'lshttpd - main' >/dev/null 2>&1
|
||||
}
|
||||
|
||||
MAX_STARTS=5
|
||||
|
||||
Reference in New Issue
Block a user