shadowdaoandClaude Sonnet 5
af7d2c6d24
Live testing (2026-09-07) showed two real problems in one root cause:
LiveKit's default subscriber behavior lets the SFU switch simulcast
layers on its own bandwidth/adaptive logic, and this plugin never told
it not to. For a real camera, that showed up as the OBS source's
received frame size visibly hopping between 320x180/640x360/1280x720
mid-show -- OBS's async video source resizes to match, breaking any
manual crop/position a director had set up. For the soundboard (a
Camera-source track that exists only to satisfy RTMP's video
requirement -- Soundboard.tsx -- with no real visual content), the
same instability, plus the video showing at all, was pure noise: there
was no way to pull just its audio.
Both come from RemoteTrackPublication (livekit/remote_track_publication.h
in the pinned SDK), on the exact publication object TrackSubscribedEvent
and attachExistingTracks already hand this code:
- setVideoQuality(VideoQuality::HIGH) on every wanted video track,
unconditionally, so the SFU always sends the top simulcast layer
instead of switching layers underneath a source with no
rendered-size hint to give it (this is a native subscriber, not a
sized <video> element).
- A new SessionConfig::subscribe_video (mirrors subscribe_audio):
when false, the wanted video track is never attached, and its
publication is explicitly setEnabled(false) -- the SFU stops
sending it, not just "decoded and discarded here". Wired to a new
"Audio only (no video)" checkbox in the source's properties.
Both call sites (a fresh TrackSubscribedEvent, and attachExistingTracks
sweeping tracks already up when the session starts watching) go
through one new handleWantedVideoTrack() so they can't drift apart.
Not unit-testable without a real LiveKit connection (RemoteTrackPublication
isn't fakeable, matching why test_integration_livekit.cpp already needs a
real server) -- verified instead by a full local build against real
libobs-dev + the pinned SDK (clean compile, all 6 existing tests still
pass) and CI. The actual behavioral fix -- stable resolution, no video
for an audio-only source -- needs the same real-OBS verification every
other claim in this repo's "What is verified, and how" section does.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RL8abRmgFXkVASHkkqiJbE
119 lines
4.4 KiB
C++
119 lines
4.4 KiB
C++
/*
|
|
streamer-tools OBS Camera Plugin - LiveKit session wrapper
|
|
Copyright (C) 2026 CyberCoveLLC <jknapp85@gmail.com>
|
|
|
|
Licensed under the Apache License, Version 2.0 (the "License");
|
|
you may not use this file except in compliance with the License.
|
|
You may obtain a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
*/
|
|
|
|
#pragma once
|
|
|
|
#include <memory>
|
|
#include <string>
|
|
|
|
#include "stplugin/session_types.h"
|
|
|
|
namespace stplugin {
|
|
|
|
struct SessionConfig {
|
|
/// LiveKit websocket URL, from POST /api/obs/:slug/token.
|
|
std::string ws_url;
|
|
/// LiveKit JWT, from the same call.
|
|
std::string token;
|
|
/// The slot's participant identity to subscribe to.
|
|
std::string participant_identity;
|
|
|
|
/// Pixel format requested from the SDK. I420 costs no conversion on
|
|
/// either side.
|
|
PixelFormat video_format = PixelFormat::I420;
|
|
|
|
/// Ring-buffer depth for decoded video. Non-zero means the SDK drops the
|
|
/// OLDEST frame when the queue is full, which is the structural answer to
|
|
/// the stale-frame-after-publisher-swap bug that motivated this plugin
|
|
/// (see the design doc's Approach section): a stalled consumer can only
|
|
/// ever fall this far behind, and what it then sees is the newest frame,
|
|
/// not a backlog.
|
|
std::size_t video_queue_capacity = 3;
|
|
|
|
/// Same for audio. A little deeper because audio frames are 10ms each.
|
|
std::size_t audio_queue_capacity = 20;
|
|
|
|
bool subscribe_audio = true;
|
|
|
|
/// False for an audio-only source (the soundboard, say): the wanted
|
|
/// video track is never attached (no AttachVideo command posted), and
|
|
/// its publication is explicitly disabled server-side (RemoteTrack-
|
|
/// Publication::setEnabled(false)) so the SFU stops sending it at all --
|
|
/// not just "decoded and discarded here", genuinely not delivered.
|
|
bool subscribe_video = true;
|
|
|
|
/// How long connect() waits for the room to come up before giving up.
|
|
int connect_timeout_ms = 15000;
|
|
};
|
|
|
|
/// Wraps livekit::Room for exactly one subscribed slot.
|
|
///
|
|
/// Threading contract, which the OBS adapter depends on:
|
|
/// - connect() and disconnect() are blocking and must be called from an
|
|
/// ordinary thread. They must NOT be called from inside a handler this
|
|
/// class invokes: the SDK documents that Room::disconnect() deadlocks if
|
|
/// called from a room event callback, and Room's own callback registration
|
|
/// is not re-entrant either.
|
|
/// - The video and audio handlers are invoked on dedicated reader threads,
|
|
/// one per track. Frame pointers are valid only for the duration of the
|
|
/// call.
|
|
/// - The state handler is invoked from whichever thread observed the
|
|
/// change. It must not block and must not call back into this object.
|
|
/// - All handlers must be installed before connect(); they are not
|
|
/// synchronised against a running session.
|
|
class LiveKitSession {
|
|
public:
|
|
LiveKitSession();
|
|
~LiveKitSession();
|
|
|
|
LiveKitSession(const LiveKitSession &) = delete;
|
|
LiveKitSession &operator=(const LiveKitSession &) = delete;
|
|
|
|
void setVideoHandler(VideoFrameHandler handler);
|
|
void setAudioHandler(AudioFrameHandler handler);
|
|
void setStateHandler(SessionStateHandler handler);
|
|
|
|
/// Connect and start subscribing. Returns true once the room is up; the
|
|
/// selected slot's tracks may still arrive later (or not at all, if the
|
|
/// camera is dark), which is reported through hasVideo()/the state
|
|
/// handler rather than as a connect failure.
|
|
bool connect(const SessionConfig &config);
|
|
|
|
/// Tear everything down. Safe to call when never connected, and safe to
|
|
/// call twice.
|
|
void disconnect();
|
|
|
|
SessionState state() const;
|
|
std::string stateDetail() const;
|
|
bool hasVideo() const;
|
|
bool hasAudio() const;
|
|
/// True when connected but the slot is not publishing: the source should
|
|
/// show its placeholder, not an error.
|
|
bool waitingForCamera() const;
|
|
|
|
/// Monotonic counters, for logging and for the adapter to tell "connected
|
|
/// but silent" from "never started".
|
|
std::uint64_t videoFrameCount() const;
|
|
std::uint64_t audioFrameCount() const;
|
|
|
|
/// Process-wide SDK init/teardown. Reference-counted, so several sources
|
|
/// can each hold one. The OBS adapter calls these from obs_module_load /
|
|
/// obs_module_unload.
|
|
static void globalInitialize();
|
|
static void globalShutdown();
|
|
|
|
private:
|
|
struct Impl;
|
|
std::unique_ptr<Impl> impl_;
|
|
};
|
|
|
|
} // namespace stplugin
|