Josh Knapp 8a86beac73 feat: clear stale certbot lock files before each ACME run + at startup ...

certbot uses fasteners (fcntl-based locking) to serialize concurrent
invocations. The kernel auto-releases fcntl locks when the holding
process exits, but the .certbot.lock FILES persist on disk — and we've
seen real cases where subsequent runs report "Another instance of
Certbot is already running" even when no certbot process is alive.
Observed during the 2026-05-09 bundling rollout when a hung worker
held a lock across container-internal Python crashes.

When SSL is blocked on a customer site, this is high-impact: the
certbot lock can sit stale until somebody manually deletes it.

clear_stale_certbot_locks():
  - probes each known lock path with fcntl.LOCK_NB
  - if the lock is unheld → file is stale → delete it
  - if the lock IS held → leave it alone (real certbot is running)

Wired in:
  - container startup (init block)
  - /api/ssl single-domain handler
  - /api/ssl/bundle handler
  - /api/certificates/renew handler

Safe to call repeatedly; never deletes a lock a real process holds, so
can never trigger concurrent certbot runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-09 12:09:19 -07:00

88 KiB

Raw Blame History

View Raw