Build / macOS (macos-latest) (push) Successful in 1m7s
Build / Linux (ubuntu-24.04) (push) Successful in 1m14s
Build / macOS (macos-latest) (pull_request) Successful in 53s
Build / Linux (ubuntu-24.04) (pull_request) Successful in 1m19s
Build / Windows (windows-latest) (push) Failing after 3m25s
Build / Windows (windows-latest) (pull_request) Failing after 3m0s
Camera sources in the OBS plugin were dropping out at random and never recovering, while the same players stayed healthy in browser talkback. Measured on the live server: every OBS plugin subscriber racked up 862-1099 nackMisses (retransmit requests for packets the SFU had already aged out of its send buffer -- unrecoverable loss) and sat at plis == 2 for a multi-hour session, versus 0/adaptive-PLI for a browser subscriber in the same room. The pinned client-sdk-cpp (1.10.1) exposes no PLI/keyframe-request API at all, so a decoder that lost a frame that way had no way to resync for the rest of the show. Add a stall-recovery watchdog: StallWatchdog (session_types.h/.cpp) is a pure, fake-clock-testable class that decides when a video subscription has gone too long (2000ms, kStallRecoveryTimeout) without a decoded frame reaching OBS. LiveKitSession::Impl polls it from a dedicated thread and, through the existing command queue (never touching the SDK off the worker thread), toggles the publication's setEnabled(false)/ setEnabled(true) -- the one lever this SDK exposes that makes the SFU restart delivery, and a restart always begins with a keyframe. Repeated attempts against the same stall back off exponentially (2s/4s/8s/16s/30s-capped, mirroring the shape of the adapter's own reconnect backoff) so a genuinely gone publisher is retried on a bounded cadence instead of hammered every 2 seconds. A muted, disabled or unsubscribed track, an audio-only source, or a disconnected session never arms the watchdog: new onTrackMuted/onTrackUnmuted handlers suspend and resume its clock, always re-baselining from "now" rather than a stale timestamp, so un-muting after a long legitimate camera-off period cannot read as a multi-minute stall. Each attempt, and eventual recovery, is logged through a new DiagnosticHandler that the OBS adapter maps onto obs_log at the same severities the file already uses for other notable events. Also investigated (not changed): the setVideoQuality(HIGH) pin added for an earlier simulcast-resize bug. The SDK's own docs and the FFI binary's wire-protocol strings show setVideoQuality only bounds spatial/simulcast layer selection (UpdateTrackSettings.quality), never temporal layers or frame rate, which the SFU's own congestion control governs independently -- so this pin is unlikely to be the cause of the packet-loss symptom, and is probably an inert no-op now that publishing is pinned server-side to a single spatial layer (L1T3). Left in place since that is not fully unambiguous from the SDK alone. Full writeup in .stall-recovery-report.md (untracked, not part of this commit). Adds 3 new headless StallWatchdog tests (fires-after-threshold, does- not-fire-when-muted/disabled, backs-off-rather-than-loops) to test_session.cpp; caught a real bug during development where onFrameDelivered() reset the backoff duration but not the next-attempt-allowed timestamp, throttling a just-recovered stream against its own stale backoff. Built and all 6 CTest suites pass (124/124 checks in test_session); also verified clean under ThreadSanitizer with warning counts unchanged from the pre-change baseline. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
142 lines
5.7 KiB
C++
142 lines
5.7 KiB
C++
/*
|
|
streamer-tools OBS Camera Plugin - LiveKit session wrapper
|
|
Copyright (C) 2026 CyberCoveLLC <jknapp85@gmail.com>
|
|
|
|
Licensed under the Apache License, Version 2.0 (the "License");
|
|
you may not use this file except in compliance with the License.
|
|
You may obtain a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
*/
|
|
|
|
#pragma once
|
|
|
|
#include <chrono>
|
|
#include <memory>
|
|
#include <string>
|
|
|
|
#include "stplugin/session_types.h"
|
|
|
|
namespace stplugin {
|
|
|
|
struct SessionConfig {
|
|
/// LiveKit websocket URL, from POST /api/obs/:slug/token.
|
|
std::string ws_url;
|
|
/// LiveKit JWT, from the same call.
|
|
std::string token;
|
|
/// The slot's participant identity to subscribe to.
|
|
std::string participant_identity;
|
|
|
|
/// Pixel format requested from the SDK. I420 costs no conversion on
|
|
/// either side.
|
|
PixelFormat video_format = PixelFormat::I420;
|
|
|
|
/// Ring-buffer depth for decoded video. Non-zero means the SDK drops the
|
|
/// OLDEST frame when the queue is full, which is the structural answer to
|
|
/// the stale-frame-after-publisher-swap bug that motivated this plugin
|
|
/// (see the design doc's Approach section): a stalled consumer can only
|
|
/// ever fall this far behind, and what it then sees is the newest frame,
|
|
/// not a backlog.
|
|
std::size_t video_queue_capacity = 3;
|
|
|
|
/// Same for audio. A little deeper because audio frames are 10ms each.
|
|
std::size_t audio_queue_capacity = 20;
|
|
|
|
bool subscribe_audio = true;
|
|
|
|
/// False for an audio-only source (the soundboard, say): the wanted
|
|
/// video track is never attached (no AttachVideo command posted), and
|
|
/// its publication is explicitly disabled server-side (RemoteTrack-
|
|
/// Publication::setEnabled(false)) so the SFU stops sending it at all --
|
|
/// not just "decoded and discarded here", genuinely not delivered.
|
|
bool subscribe_video = true;
|
|
|
|
/// How long connect() waits for the room to come up before giving up.
|
|
int connect_timeout_ms = 15000;
|
|
};
|
|
|
|
/// How long the stall-recovery watchdog waits for a decoded video frame
|
|
/// before treating the subscription as stalled and forcing a fresh
|
|
/// keyframe (see StallWatchdog's comment in session_types.h for why this
|
|
/// exists -- unrecoverable packet loss with no PLI/keyframe-request API in
|
|
/// the pinned SDK). Long enough that ordinary jitter never trips it (a
|
|
/// healthy 30fps subscription delivers a frame at least every ~33ms);
|
|
/// short enough a director barely has time to notice before it recovers.
|
|
constexpr std::chrono::milliseconds kStallRecoveryTimeout{2000};
|
|
|
|
/// Ceiling for the backoff between repeated recovery attempts against the
|
|
/// SAME stall. Starts at kStallRecoveryTimeout and doubles each attempt, so
|
|
/// a genuinely gone publisher (crashed encoder, dead upstream network) is
|
|
/// retried every 2s, 4s, 8s, ... 30s rather than hammered every 2 seconds
|
|
/// for the rest of the show.
|
|
constexpr std::chrono::milliseconds kStallRecoveryMaxBackoff{30000};
|
|
|
|
/// Wraps livekit::Room for exactly one subscribed slot.
|
|
///
|
|
/// Threading contract, which the OBS adapter depends on:
|
|
/// - connect() and disconnect() are blocking and must be called from an
|
|
/// ordinary thread. They must NOT be called from inside a handler this
|
|
/// class invokes: the SDK documents that Room::disconnect() deadlocks if
|
|
/// called from a room event callback, and Room's own callback registration
|
|
/// is not re-entrant either.
|
|
/// - The video and audio handlers are invoked on dedicated reader threads,
|
|
/// one per track. Frame pointers are valid only for the duration of the
|
|
/// call.
|
|
/// - The state handler is invoked from whichever thread observed the
|
|
/// change. It must not block and must not call back into this object.
|
|
/// - The diagnostic handler (currently just the stall-recovery watchdog,
|
|
/// see kStallRecoveryTimeout below) may be invoked from the internal
|
|
/// command-queue worker thread or a video reader thread. Same rules as
|
|
/// the state handler: must not block, must not call back into this
|
|
/// object.
|
|
/// - All handlers must be installed before connect(); they are not
|
|
/// synchronised against a running session.
|
|
class LiveKitSession {
|
|
public:
|
|
LiveKitSession();
|
|
~LiveKitSession();
|
|
|
|
LiveKitSession(const LiveKitSession &) = delete;
|
|
LiveKitSession &operator=(const LiveKitSession &) = delete;
|
|
|
|
void setVideoHandler(VideoFrameHandler handler);
|
|
void setAudioHandler(AudioFrameHandler handler);
|
|
void setStateHandler(SessionStateHandler handler);
|
|
void setDiagnosticHandler(DiagnosticHandler handler);
|
|
|
|
/// Connect and start subscribing. Returns true once the room is up; the
|
|
/// selected slot's tracks may still arrive later (or not at all, if the
|
|
/// camera is dark), which is reported through hasVideo()/the state
|
|
/// handler rather than as a connect failure.
|
|
bool connect(const SessionConfig &config);
|
|
|
|
/// Tear everything down. Safe to call when never connected, and safe to
|
|
/// call twice.
|
|
void disconnect();
|
|
|
|
SessionState state() const;
|
|
std::string stateDetail() const;
|
|
bool hasVideo() const;
|
|
bool hasAudio() const;
|
|
/// True when connected but the slot is not publishing: the source should
|
|
/// show its placeholder, not an error.
|
|
bool waitingForCamera() const;
|
|
|
|
/// Monotonic counters, for logging and for the adapter to tell "connected
|
|
/// but silent" from "never started".
|
|
std::uint64_t videoFrameCount() const;
|
|
std::uint64_t audioFrameCount() const;
|
|
|
|
/// Process-wide SDK init/teardown. Reference-counted, so several sources
|
|
/// can each hold one. The OBS adapter calls these from obs_module_load /
|
|
/// obs_module_unload.
|
|
static void globalInitialize();
|
|
static void globalShutdown();
|
|
|
|
private:
|
|
struct Impl;
|
|
std::unique_ptr<Impl> impl_;
|
|
};
|
|
|
|
} // namespace stplugin
|