fix(session): recover stalled video subscriptions with a keyframe-forcing watchdog

Camera sources in the OBS plugin were dropping out at random and never
recovering, while the same players stayed healthy in browser talkback.
Measured on the live server: every OBS plugin subscriber racked up
862-1099 nackMisses (retransmit requests for packets the SFU had already
aged out of its send buffer -- unrecoverable loss) and sat at plis == 2
for a multi-hour session, versus 0/adaptive-PLI for a browser subscriber
in the same room. The pinned client-sdk-cpp (1.10.1) exposes no
PLI/keyframe-request API at all, so a decoder that lost a frame that way
had no way to resync for the rest of the show.

Add a stall-recovery watchdog: StallWatchdog (session_types.h/.cpp) is a
pure, fake-clock-testable class that decides when a video subscription
has gone too long (2000ms, kStallRecoveryTimeout) without a decoded
frame reaching OBS. LiveKitSession::Impl polls it from a dedicated
thread and, through the existing command queue (never touching the SDK
off the worker thread), toggles the publication's setEnabled(false)/
setEnabled(true) -- the one lever this SDK exposes that makes the SFU
restart delivery, and a restart always begins with a keyframe.

Repeated attempts against the same stall back off exponentially
(2s/4s/8s/16s/30s-capped, mirroring the shape of the adapter's own
reconnect backoff) so a genuinely gone publisher is retried on a bounded
cadence instead of hammered every 2 seconds. A muted, disabled or
unsubscribed track, an audio-only source, or a disconnected session
never arms the watchdog: new onTrackMuted/onTrackUnmuted handlers
suspend and resume its clock, always re-baselining from "now" rather
than a stale timestamp, so un-muting after a long legitimate camera-off
period cannot read as a multi-minute stall. Each attempt, and eventual
recovery, is logged through a new DiagnosticHandler that the OBS adapter
maps onto obs_log at the same severities the file already uses for other
notable events.

Also investigated (not changed): the setVideoQuality(HIGH) pin added for
an earlier simulcast-resize bug. The SDK's own docs and the FFI binary's
wire-protocol strings show setVideoQuality only bounds spatial/simulcast
layer selection (UpdateTrackSettings.quality), never temporal layers or
frame rate, which the SFU's own congestion control governs independently
-- so this pin is unlikely to be the cause of the packet-loss symptom,
and is probably an inert no-op now that publishing is pinned server-side
to a single spatial layer (L1T3). Left in place since that is not fully
unambiguous from the SDK alone. Full writeup in
.stall-recovery-report.md (untracked, not part of this commit).

Adds 3 new headless StallWatchdog tests (fires-after-threshold, does-
not-fire-when-muted/disabled, backs-off-rather-than-loops) to
test_session.cpp; caught a real bug during development where
onFrameDelivered() reset the backoff duration but not the
next-attempt-allowed timestamp, throttling a just-recovered stream
against its own stale backoff. Built and all 6 CTest suites pass
(124/124 checks in test_session); also verified clean under
ThreadSanitizer with warning counts unchanged from the pre-change
baseline.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

core/include/stplugin/session.h

@@ -11,6 +11,7 @@ You may obtain a copy of the License at
 
 #pragma once
 
+#include <chrono>
 #include <memory>
 #include <string>
 
@@ -54,6 +55,22 @@ struct SessionConfig {
     int connect_timeout_ms = 15000;
 };
 
+/// How long the stall-recovery watchdog waits for a decoded video frame
+/// before treating the subscription as stalled and forcing a fresh
+/// keyframe (see StallWatchdog's comment in session_types.h for why this
+/// exists -- unrecoverable packet loss with no PLI/keyframe-request API in
+/// the pinned SDK). Long enough that ordinary jitter never trips it (a
+/// healthy 30fps subscription delivers a frame at least every ~33ms);
+/// short enough a director barely has time to notice before it recovers.
+constexpr std::chrono::milliseconds kStallRecoveryTimeout{2000};
+
+/// Ceiling for the backoff between repeated recovery attempts against the
+/// SAME stall. Starts at kStallRecoveryTimeout and doubles each attempt, so
+/// a genuinely gone publisher (crashed encoder, dead upstream network) is
+/// retried every 2s, 4s, 8s, ... 30s rather than hammered every 2 seconds
+/// for the rest of the show.
+constexpr std::chrono::milliseconds kStallRecoveryMaxBackoff{30000};
+
 /// Wraps livekit::Room for exactly one subscribed slot.
 ///
 /// Threading contract, which the OBS adapter depends on:
@@ -67,6 +84,11 @@ struct SessionConfig {
 ///     call.
 ///   - The state handler is invoked from whichever thread observed the
 ///     change. It must not block and must not call back into this object.
+///   - The diagnostic handler (currently just the stall-recovery watchdog,
+///     see kStallRecoveryTimeout below) may be invoked from the internal
+///     command-queue worker thread or a video reader thread. Same rules as
+///     the state handler: must not block, must not call back into this
+///     object.
 ///   - All handlers must be installed before connect(); they are not
 ///     synchronised against a running session.
 class LiveKitSession {
@@ -80,6 +102,7 @@ public:
     void setVideoHandler(VideoFrameHandler handler);
     void setAudioHandler(AudioFrameHandler handler);
     void setStateHandler(SessionStateHandler handler);
+    void setDiagnosticHandler(DiagnosticHandler handler);
 
     /// Connect and start subscribing. Returns true once the room is up; the
     /// selected slot's tracks may still arrive later (or not at all, if the

core/include/stplugin/session_types.h

@@ -19,6 +19,7 @@ You may obtain a copy of the License at
 // self-consistent -- all live here, and LiveKitSession is the (much thinner)
 // piece that wires real SDK callbacks into them.
 
+#include <chrono>
 #include <cstddef>
 #include <cstdint>
 #include <functional>
@@ -137,6 +138,82 @@ private:
     bool has_audio_ = false;
 };
 
+// ---------------------------------------------------------------------------
+// Stall-recovery watchdog (pure timing/decision logic)
+// ---------------------------------------------------------------------------
+
+/// Decides WHEN to force a fresh keyframe on an already-subscribed video
+/// track. It does not touch LiveKit or OBS at all -- LiveKitSession::Impl
+/// (session.cpp) is what actually carries the decision out
+/// (RemoteTrackPublication::setEnabled(false) then setEnabled(true)), and
+/// only from its SDK command-queue thread, the same rule every other SDK
+/// interaction in that file already follows.
+///
+/// Why this exists: measured on the live server, comparing subscribers in
+/// the same LiveKit room over the same 30-minute window, every OBS plugin
+/// connection racked up ~862-1099 nackMisses and ~2200-3100 nackRepeated --
+/// nackMisses means the subscriber asked the SFU to retransmit a packet
+/// that had already aged out of its send buffer, i.e. unrecoverable loss --
+/// while a browser subscriber in the same room saw 0 and 0. A decoder that
+/// loses a frame that way cannot resync without a fresh keyframe. Every OBS
+/// plugin connection also sat at `plis` == 2 for a multi-hour session (a
+/// browser adapts and asks for keyframes normally), and grepping the pinned
+/// client-sdk-cpp (1.10.1) headers turns up no PLI/keyframe-request API at
+/// all. So today, once that happens, the source just stays broken for the
+/// rest of the show -- "drops at random and never recovers". Toggling the
+/// subscription off and back on is the one lever this SDK exposes that
+/// forces the SFU to stop and restart delivery of the track, and a restart
+/// always begins with a keyframe. This class is the "have we gone too long
+/// without a frame, and is it still worth trying again" clock behind that
+/// lever; see LiveKitSession::Impl::recoverVideo() for where it is pulled.
+///
+/// Not thread-safe on its own, deliberately -- same contract as
+/// SessionStateMachine above: LiveKitSession::Impl owns the lock (in
+/// practice the same state_mutex that guards `machine`).
+class StallWatchdog {
+public:
+    StallWatchdog(std::chrono::milliseconds timeout, std::chrono::milliseconds max_backoff);
+
+    /// Call whenever whether a frame could legitimately arrive right now
+    /// changes: true once a video track is subscribed (and unmuted), false
+    /// on detach/unsubscribe/mute/disconnect. Flipping to false always
+    /// clears all timing state; flipping back to true always starts a
+    /// brand-new grace period rather than measuring from a stale timestamp.
+    /// That is what stops an unmute -- or an ordinary publisher swap --
+    /// from firing the INSTANT it resumes, off a "last frame" that might
+    /// actually be minutes old: a muted, disabled or unsubscribed track, an
+    /// audio-only source, or a disconnected session must never trip this.
+    void setExpectingFrames(bool expecting, std::chrono::steady_clock::time_point now);
+
+    /// Call every time a decoded video frame is actually delivered.
+    void onFrameDelivered(std::chrono::steady_clock::time_point now);
+
+    /// Call periodically (finer-grained than the configured timeout).
+    /// Returns true exactly when a recovery attempt should be made right
+    /// now; each true also arms the backoff before the next one is even
+    /// considered, so a caller polling in a tight loop still cannot fire
+    /// back-to-back attempts against a publisher that never comes back --
+    /// see the class comment: hammering every 2 seconds forever against a
+    /// genuinely gone publisher is worse than a frozen source.
+    bool poll(std::chrono::steady_clock::time_point now);
+
+    /// Attempts made since the current stall started (since the last frame,
+    /// or since expecting-frames most recently became true). Reset by
+    /// onFrameDelivered and by setExpectingFrames.
+    int attemptsThisStall() const { return attempts_; }
+
+private:
+    std::chrono::milliseconds timeout_;
+    std::chrono::milliseconds max_backoff_;
+
+    bool expecting_ = false;
+    bool have_baseline_ = false;
+    std::chrono::steady_clock::time_point baseline_{};
+    std::chrono::milliseconds backoff_{};
+    std::chrono::steady_clock::time_point next_attempt_allowed_{};
+    int attempts_ = 0;
+};
+
 // ---------------------------------------------------------------------------
 // Frames handed to the OBS adapter
 // ---------------------------------------------------------------------------
@@ -177,4 +254,13 @@ using VideoFrameHandler = std::function<void(const VideoFrameData &)>;
 using AudioFrameHandler = std::function<void(const AudioFrameData &)>;
 using SessionStateHandler = std::function<void(SessionState state, const std::string &detail)>;
 
+/// Severity for LiveKitSession's own diagnostic log lines (currently just
+/// the stall-recovery watchdog). Kept separate from OBS's LOG_* levels and
+/// from the SDK's own livekit::LogLevel so core/ stays free of any OBS
+/// dependency -- the adapter maps this onto obs_log the same way it already
+/// maps livekit::LogLevel (see plugin-main.cpp's livekit log bridge).
+enum class DiagnosticLevel { Info, Warning };
+
+using DiagnosticHandler = std::function<void(DiagnosticLevel level, const std::string &message)>;
+
 } // namespace stplugin