stplugin::LiveKitSession wraps livekit::Room for exactly one subscribed slot: connect with the wsUrl/lkToken the API client minted, find the chosen participant's camera (and microphone), and hand decoded frames to callback-shaped handlers the OBS adapter can consume directly. Two architectural decisions worth recording, both forced by reading the SDK rather than guessed: 1. Frames come from VideoStream/AudioStream::fromTrack with our own reader threads, NOT from Room::setOnVideoFrameCallback. The dispatcher API is keyed by (participant identity, track NAME), which we cannot know before the track is published -- and disassembling liblivekit.so confirms that both Room::setOnVideoFrameCallback and the dispatcher's own setOnVideoFrameCallback merely record the registration: neither starts a reader for a track that is already subscribed. Registering after the subscription event, which is the only time the track name exists, would therefore have silently produced no video. Taking the shared_ptr<Track> straight off the TrackSubscribedEvent sidesteps the name entirely, and lets us pick the camera by TrackSource (streamer-tools publishes cameras as Source.Camera and screenshares separately -- apps/web/src/avatar/ publish.ts), which is what we actually mean. 2. Every stream operation runs on one owned worker thread, never on a room event thread. The SDK documents that Room::disconnect() from inside a delegate callback deadlocks, and Room's own event dispatch holds a mutex, so delegate callbacks only ever enqueue a command here. VideoStream::Options::capacity is set (3 frames) so the SDK's queue is a drop-oldest ring buffer: a stalled consumer can only fall three frames behind, and what it then sees is the newest frame rather than a backlog. That is the structural answer to the stale-media bug that motivated this plugin. The pure decision-making -- the state machine, track selection, frame geometry validation -- lives in session_types.h/.cpp with no LiveKit or OBS types, so it is unit-testable headlessly (81 checks in test_session, including the publisher-swap and reconnect transitions, plus the real connect() failure paths against the real SDK: unreachable host, garbage token, incomplete config, and destruction mid-connect). test_integration_livekit is the test that proves media actually flows. It publishes a synthetic camera and microphone into a real LiveKit room using the same SDK, subscribes through LiveKitSession, and asserts on the exact fields the OBS adapter will dereference. It skips (exit 0) unless STPLUGIN_IT_* is set, so the three build runners stay green; scripts/livekit-dev-room.py mints the tokens for a local `livekit-server --dev`. Verified locally against livekit-server 1.13.6 in dev mode: integration_livekit: 36 video frames, 323 audio frames, 10 state changes integration_livekit: 32 checks passed covering: connect; subscribe to the named participant's camera; 320x240 I420 frames with three planes, non-null plane pointers and strides >= the frame's own width; 48kHz audio; unpublish -> hasVideo() false, state stays Connected (a dark camera is the placeholder state, never an error) and NO further frames arrive from the dead publisher; republish -> video resumes; clean disconnect. One real finding from that run, now handled: WebRTC ramps a new subscription up from a downscaled spatial layer, so the first frames after (re)subscribing legitimately arrive smaller than what is being published. The OBS adapter must cope with a mid-stream resolution change; the test asserts per-frame geometry rather than the publisher's, and separately asserts the stream does reach full size. Full suite: ctest -> 6/6 passed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RL8abRmgFXkVASHkkqiJbE
119 lines
4.4 KiB
C++
119 lines
4.4 KiB
C++
/*
|
|
streamer-tools OBS Camera Plugin - LiveKit session wrapper
|
|
Copyright (C) 2026 CyberCoveLLC <jknapp85@gmail.com>
|
|
|
|
This program is free software; you can redistribute it and/or modify
|
|
it under the terms of the GNU General Public License as published by
|
|
the Free Software Foundation; either version 2 of the License, or
|
|
(at your option) any later version.
|
|
|
|
This program is distributed in the hope that it will be useful,
|
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
GNU General Public License for more details.
|
|
|
|
You should have received a copy of the GNU General Public License along
|
|
with this program. If not, see <https://www.gnu.org/licenses/>
|
|
*/
|
|
|
|
#pragma once
|
|
|
|
#include <memory>
|
|
#include <string>
|
|
|
|
#include "stplugin/session_types.h"
|
|
|
|
namespace stplugin {
|
|
|
|
struct SessionConfig {
|
|
/// LiveKit websocket URL, from POST /api/obs/:slug/token.
|
|
std::string ws_url;
|
|
/// LiveKit JWT, from the same call.
|
|
std::string token;
|
|
/// The slot's participant identity to subscribe to.
|
|
std::string participant_identity;
|
|
|
|
/// Pixel format requested from the SDK. I420 costs no conversion on
|
|
/// either side.
|
|
PixelFormat video_format = PixelFormat::I420;
|
|
|
|
/// Ring-buffer depth for decoded video. Non-zero means the SDK drops the
|
|
/// OLDEST frame when the queue is full, which is the structural answer to
|
|
/// the stale-frame-after-publisher-swap bug that motivated this plugin
|
|
/// (see the design doc's Approach section): a stalled consumer can only
|
|
/// ever fall this far behind, and what it then sees is the newest frame,
|
|
/// not a backlog.
|
|
std::size_t video_queue_capacity = 3;
|
|
|
|
/// Same for audio. A little deeper because audio frames are 10ms each.
|
|
std::size_t audio_queue_capacity = 20;
|
|
|
|
bool subscribe_audio = true;
|
|
|
|
/// How long connect() waits for the room to come up before giving up.
|
|
int connect_timeout_ms = 15000;
|
|
};
|
|
|
|
/// Wraps livekit::Room for exactly one subscribed slot.
|
|
///
|
|
/// Threading contract, which the OBS adapter depends on:
|
|
/// - connect() and disconnect() are blocking and must be called from an
|
|
/// ordinary thread. They must NOT be called from inside a handler this
|
|
/// class invokes: the SDK documents that Room::disconnect() deadlocks if
|
|
/// called from a room event callback, and Room's own callback registration
|
|
/// is not re-entrant either.
|
|
/// - The video and audio handlers are invoked on dedicated reader threads,
|
|
/// one per track. Frame pointers are valid only for the duration of the
|
|
/// call.
|
|
/// - The state handler is invoked from whichever thread observed the
|
|
/// change. It must not block and must not call back into this object.
|
|
/// - All handlers must be installed before connect(); they are not
|
|
/// synchronised against a running session.
|
|
class LiveKitSession {
|
|
public:
|
|
LiveKitSession();
|
|
~LiveKitSession();
|
|
|
|
LiveKitSession(const LiveKitSession &) = delete;
|
|
LiveKitSession &operator=(const LiveKitSession &) = delete;
|
|
|
|
void setVideoHandler(VideoFrameHandler handler);
|
|
void setAudioHandler(AudioFrameHandler handler);
|
|
void setStateHandler(SessionStateHandler handler);
|
|
|
|
/// Connect and start subscribing. Returns true once the room is up; the
|
|
/// selected slot's tracks may still arrive later (or not at all, if the
|
|
/// camera is dark), which is reported through hasVideo()/the state
|
|
/// handler rather than as a connect failure.
|
|
bool connect(const SessionConfig &config);
|
|
|
|
/// Tear everything down. Safe to call when never connected, and safe to
|
|
/// call twice.
|
|
void disconnect();
|
|
|
|
SessionState state() const;
|
|
std::string stateDetail() const;
|
|
bool hasVideo() const;
|
|
bool hasAudio() const;
|
|
/// True when connected but the slot is not publishing: the source should
|
|
/// show its placeholder, not an error.
|
|
bool waitingForCamera() const;
|
|
|
|
/// Monotonic counters, for logging and for the adapter to tell "connected
|
|
/// but silent" from "never started".
|
|
std::uint64_t videoFrameCount() const;
|
|
std::uint64_t audioFrameCount() const;
|
|
|
|
/// Process-wide SDK init/teardown. Reference-counted, so several sources
|
|
/// can each hold one. The OBS adapter calls these from obs_module_load /
|
|
/// obs_module_unload.
|
|
static void globalInitialize();
|
|
static void globalShutdown();
|
|
|
|
private:
|
|
struct Impl;
|
|
std::unique_ptr<Impl> impl_;
|
|
};
|
|
|
|
} // namespace stplugin
|