Files
obs-streamer-tools-plugin/core/include/stplugin/session.h
T
shadowdaoandClaude Sonnet 5 3f2933ae3f
Build / macOS (macos-latest) (push) Successful in 1m7s
Build / Linux (ubuntu-24.04) (push) Successful in 1m14s
Build / macOS (macos-latest) (pull_request) Successful in 53s
Build / Linux (ubuntu-24.04) (pull_request) Successful in 1m19s
Build / Windows (windows-latest) (push) Failing after 3m25s
Build / Windows (windows-latest) (pull_request) Failing after 3m0s
fix(session): recover stalled video subscriptions with a keyframe-forcing watchdog
Camera sources in the OBS plugin were dropping out at random and never
recovering, while the same players stayed healthy in browser talkback.
Measured on the live server: every OBS plugin subscriber racked up
862-1099 nackMisses (retransmit requests for packets the SFU had already
aged out of its send buffer -- unrecoverable loss) and sat at plis == 2
for a multi-hour session, versus 0/adaptive-PLI for a browser subscriber
in the same room. The pinned client-sdk-cpp (1.10.1) exposes no
PLI/keyframe-request API at all, so a decoder that lost a frame that way
had no way to resync for the rest of the show.

Add a stall-recovery watchdog: StallWatchdog (session_types.h/.cpp) is a
pure, fake-clock-testable class that decides when a video subscription
has gone too long (2000ms, kStallRecoveryTimeout) without a decoded
frame reaching OBS. LiveKitSession::Impl polls it from a dedicated
thread and, through the existing command queue (never touching the SDK
off the worker thread), toggles the publication's setEnabled(false)/
setEnabled(true) -- the one lever this SDK exposes that makes the SFU
restart delivery, and a restart always begins with a keyframe.

Repeated attempts against the same stall back off exponentially
(2s/4s/8s/16s/30s-capped, mirroring the shape of the adapter's own
reconnect backoff) so a genuinely gone publisher is retried on a bounded
cadence instead of hammered every 2 seconds. A muted, disabled or
unsubscribed track, an audio-only source, or a disconnected session
never arms the watchdog: new onTrackMuted/onTrackUnmuted handlers
suspend and resume its clock, always re-baselining from "now" rather
than a stale timestamp, so un-muting after a long legitimate camera-off
period cannot read as a multi-minute stall. Each attempt, and eventual
recovery, is logged through a new DiagnosticHandler that the OBS adapter
maps onto obs_log at the same severities the file already uses for other
notable events.

Also investigated (not changed): the setVideoQuality(HIGH) pin added for
an earlier simulcast-resize bug. The SDK's own docs and the FFI binary's
wire-protocol strings show setVideoQuality only bounds spatial/simulcast
layer selection (UpdateTrackSettings.quality), never temporal layers or
frame rate, which the SFU's own congestion control governs independently
-- so this pin is unlikely to be the cause of the packet-loss symptom,
and is probably an inert no-op now that publishing is pinned server-side
to a single spatial layer (L1T3). Left in place since that is not fully
unambiguous from the SDK alone. Full writeup in
.stall-recovery-report.md (untracked, not part of this commit).

Adds 3 new headless StallWatchdog tests (fires-after-threshold, does-
not-fire-when-muted/disabled, backs-off-rather-than-loops) to
test_session.cpp; caught a real bug during development where
onFrameDelivered() reset the backoff duration but not the
next-attempt-allowed timestamp, throttling a just-recovered stream
against its own stale backoff. Built and all 6 CTest suites pass
(124/124 checks in test_session); also verified clean under
ThreadSanitizer with warning counts unchanged from the pre-change
baseline.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-20 19:04:14 -07:00

142 lines
5.7 KiB
C++

/*
streamer-tools OBS Camera Plugin - LiveKit session wrapper
Copyright (C) 2026 CyberCoveLLC <jknapp85@gmail.com>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
*/
#pragma once
#include <chrono>
#include <memory>
#include <string>
#include "stplugin/session_types.h"
namespace stplugin {
struct SessionConfig {
/// LiveKit websocket URL, from POST /api/obs/:slug/token.
std::string ws_url;
/// LiveKit JWT, from the same call.
std::string token;
/// The slot's participant identity to subscribe to.
std::string participant_identity;
/// Pixel format requested from the SDK. I420 costs no conversion on
/// either side.
PixelFormat video_format = PixelFormat::I420;
/// Ring-buffer depth for decoded video. Non-zero means the SDK drops the
/// OLDEST frame when the queue is full, which is the structural answer to
/// the stale-frame-after-publisher-swap bug that motivated this plugin
/// (see the design doc's Approach section): a stalled consumer can only
/// ever fall this far behind, and what it then sees is the newest frame,
/// not a backlog.
std::size_t video_queue_capacity = 3;
/// Same for audio. A little deeper because audio frames are 10ms each.
std::size_t audio_queue_capacity = 20;
bool subscribe_audio = true;
/// False for an audio-only source (the soundboard, say): the wanted
/// video track is never attached (no AttachVideo command posted), and
/// its publication is explicitly disabled server-side (RemoteTrack-
/// Publication::setEnabled(false)) so the SFU stops sending it at all --
/// not just "decoded and discarded here", genuinely not delivered.
bool subscribe_video = true;
/// How long connect() waits for the room to come up before giving up.
int connect_timeout_ms = 15000;
};
/// How long the stall-recovery watchdog waits for a decoded video frame
/// before treating the subscription as stalled and forcing a fresh
/// keyframe (see StallWatchdog's comment in session_types.h for why this
/// exists -- unrecoverable packet loss with no PLI/keyframe-request API in
/// the pinned SDK). Long enough that ordinary jitter never trips it (a
/// healthy 30fps subscription delivers a frame at least every ~33ms);
/// short enough a director barely has time to notice before it recovers.
constexpr std::chrono::milliseconds kStallRecoveryTimeout{2000};
/// Ceiling for the backoff between repeated recovery attempts against the
/// SAME stall. Starts at kStallRecoveryTimeout and doubles each attempt, so
/// a genuinely gone publisher (crashed encoder, dead upstream network) is
/// retried every 2s, 4s, 8s, ... 30s rather than hammered every 2 seconds
/// for the rest of the show.
constexpr std::chrono::milliseconds kStallRecoveryMaxBackoff{30000};
/// Wraps livekit::Room for exactly one subscribed slot.
///
/// Threading contract, which the OBS adapter depends on:
/// - connect() and disconnect() are blocking and must be called from an
/// ordinary thread. They must NOT be called from inside a handler this
/// class invokes: the SDK documents that Room::disconnect() deadlocks if
/// called from a room event callback, and Room's own callback registration
/// is not re-entrant either.
/// - The video and audio handlers are invoked on dedicated reader threads,
/// one per track. Frame pointers are valid only for the duration of the
/// call.
/// - The state handler is invoked from whichever thread observed the
/// change. It must not block and must not call back into this object.
/// - The diagnostic handler (currently just the stall-recovery watchdog,
/// see kStallRecoveryTimeout below) may be invoked from the internal
/// command-queue worker thread or a video reader thread. Same rules as
/// the state handler: must not block, must not call back into this
/// object.
/// - All handlers must be installed before connect(); they are not
/// synchronised against a running session.
class LiveKitSession {
public:
LiveKitSession();
~LiveKitSession();
LiveKitSession(const LiveKitSession &) = delete;
LiveKitSession &operator=(const LiveKitSession &) = delete;
void setVideoHandler(VideoFrameHandler handler);
void setAudioHandler(AudioFrameHandler handler);
void setStateHandler(SessionStateHandler handler);
void setDiagnosticHandler(DiagnosticHandler handler);
/// Connect and start subscribing. Returns true once the room is up; the
/// selected slot's tracks may still arrive later (or not at all, if the
/// camera is dark), which is reported through hasVideo()/the state
/// handler rather than as a connect failure.
bool connect(const SessionConfig &config);
/// Tear everything down. Safe to call when never connected, and safe to
/// call twice.
void disconnect();
SessionState state() const;
std::string stateDetail() const;
bool hasVideo() const;
bool hasAudio() const;
/// True when connected but the slot is not publishing: the source should
/// show its placeholder, not an error.
bool waitingForCamera() const;
/// Monotonic counters, for logging and for the adapter to tell "connected
/// but silent" from "never started".
std::uint64_t videoFrameCount() const;
std::uint64_t audioFrameCount() const;
/// Process-wide SDK init/teardown. Reference-counted, so several sources
/// can each hold one. The OBS adapter calls these from obs_module_load /
/// obs_module_unload.
static void globalInitialize();
static void globalShutdown();
private:
struct Impl;
std::unique_ptr<Impl> impl_;
};
} // namespace stplugin