stplugin::LiveKitSession wraps livekit::Room for exactly one subscribed slot:
connect with the wsUrl/lkToken the API client minted, find the chosen
participant's camera (and microphone), and hand decoded frames to
callback-shaped handlers the OBS adapter can consume directly.
Two architectural decisions worth recording, both forced by reading the SDK
rather than guessed:
1. Frames come from VideoStream/AudioStream::fromTrack with our own reader
threads, NOT from Room::setOnVideoFrameCallback. The dispatcher API is
keyed by (participant identity, track NAME), which we cannot know before
the track is published -- and disassembling liblivekit.so confirms that
both Room::setOnVideoFrameCallback and the dispatcher's own
setOnVideoFrameCallback merely record the registration: neither starts a
reader for a track that is already subscribed. Registering after the
subscription event, which is the only time the track name exists, would
therefore have silently produced no video. Taking the shared_ptr<Track>
straight off the TrackSubscribedEvent sidesteps the name entirely, and
lets us pick the camera by TrackSource (streamer-tools publishes cameras
as Source.Camera and screenshares separately -- apps/web/src/avatar/
publish.ts), which is what we actually mean.
2. Every stream operation runs on one owned worker thread, never on a room
event thread. The SDK documents that Room::disconnect() from inside a
delegate callback deadlocks, and Room's own event dispatch holds a mutex,
so delegate callbacks only ever enqueue a command here.
VideoStream::Options::capacity is set (3 frames) so the SDK's queue is a
drop-oldest ring buffer: a stalled consumer can only fall three frames
behind, and what it then sees is the newest frame rather than a backlog.
That is the structural answer to the stale-media bug that motivated this
plugin.
The pure decision-making -- the state machine, track selection, frame
geometry validation -- lives in session_types.h/.cpp with no LiveKit or OBS
types, so it is unit-testable headlessly (81 checks in test_session,
including the publisher-swap and reconnect transitions, plus the real
connect() failure paths against the real SDK: unreachable host, garbage
token, incomplete config, and destruction mid-connect).
test_integration_livekit is the test that proves media actually flows. It
publishes a synthetic camera and microphone into a real LiveKit room using
the same SDK, subscribes through LiveKitSession, and asserts on the exact
fields the OBS adapter will dereference. It skips (exit 0) unless
STPLUGIN_IT_* is set, so the three build runners stay green;
scripts/livekit-dev-room.py mints the tokens for a local `livekit-server
--dev`.
Verified locally against livekit-server 1.13.6 in dev mode:
integration_livekit: 36 video frames, 323 audio frames, 10 state changes
integration_livekit: 32 checks passed
covering: connect; subscribe to the named participant's camera; 320x240
I420 frames with three planes, non-null plane pointers and strides >= the
frame's own width; 48kHz audio; unpublish -> hasVideo() false, state stays
Connected (a dark camera is the placeholder state, never an error) and NO
further frames arrive from the dead publisher; republish -> video resumes;
clean disconnect.
One real finding from that run, now handled: WebRTC ramps a new subscription
up from a downscaled spatial layer, so the first frames after (re)subscribing
legitimately arrive smaller than what is being published. The OBS adapter
must cope with a mid-stream resolution change; the test asserts per-frame
geometry rather than the publisher's, and separately asserts the stream does
reach full size.
Full suite: ctest -> 6/6 passed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RL8abRmgFXkVASHkkqiJbE
shadowdaoandClaude Sonnet 5
80904a3e85