shadowdaoandClaude Sonnet 5
80904a3e85
stplugin::LiveKitSession wraps livekit::Room for exactly one subscribed slot: connect with the wsUrl/lkToken the API client minted, find the chosen participant's camera (and microphone), and hand decoded frames to callback-shaped handlers the OBS adapter can consume directly. Two architectural decisions worth recording, both forced by reading the SDK rather than guessed: 1. Frames come from VideoStream/AudioStream::fromTrack with our own reader threads, NOT from Room::setOnVideoFrameCallback. The dispatcher API is keyed by (participant identity, track NAME), which we cannot know before the track is published -- and disassembling liblivekit.so confirms that both Room::setOnVideoFrameCallback and the dispatcher's own setOnVideoFrameCallback merely record the registration: neither starts a reader for a track that is already subscribed. Registering after the subscription event, which is the only time the track name exists, would therefore have silently produced no video. Taking the shared_ptr<Track> straight off the TrackSubscribedEvent sidesteps the name entirely, and lets us pick the camera by TrackSource (streamer-tools publishes cameras as Source.Camera and screenshares separately -- apps/web/src/avatar/ publish.ts), which is what we actually mean. 2. Every stream operation runs on one owned worker thread, never on a room event thread. The SDK documents that Room::disconnect() from inside a delegate callback deadlocks, and Room's own event dispatch holds a mutex, so delegate callbacks only ever enqueue a command here. VideoStream::Options::capacity is set (3 frames) so the SDK's queue is a drop-oldest ring buffer: a stalled consumer can only fall three frames behind, and what it then sees is the newest frame rather than a backlog. That is the structural answer to the stale-media bug that motivated this plugin. The pure decision-making -- the state machine, track selection, frame geometry validation -- lives in session_types.h/.cpp with no LiveKit or OBS types, so it is unit-testable headlessly (81 checks in test_session, including the publisher-swap and reconnect transitions, plus the real connect() failure paths against the real SDK: unreachable host, garbage token, incomplete config, and destruction mid-connect). test_integration_livekit is the test that proves media actually flows. It publishes a synthetic camera and microphone into a real LiveKit room using the same SDK, subscribes through LiveKitSession, and asserts on the exact fields the OBS adapter will dereference. It skips (exit 0) unless STPLUGIN_IT_* is set, so the three build runners stay green; scripts/livekit-dev-room.py mints the tokens for a local `livekit-server --dev`. Verified locally against livekit-server 1.13.6 in dev mode: integration_livekit: 36 video frames, 323 audio frames, 10 state changes integration_livekit: 32 checks passed covering: connect; subscribe to the named participant's camera; 320x240 I420 frames with three planes, non-null plane pointers and strides >= the frame's own width; 48kHz audio; unpublish -> hasVideo() false, state stays Connected (a dark camera is the placeholder state, never an error) and NO further frames arrive from the dead publisher; republish -> video resumes; clean disconnect. One real finding from that run, now handled: WebRTC ramps a new subscription up from a downscaled spatial layer, so the first frames after (re)subscribing legitimately arrive smaller than what is being published. The OBS adapter must cope with a mid-stream resolution change; the test asserts per-frame geometry rather than the publisher's, and separately asserts the stream does reach full size. Full suite: ctest -> 6/6 passed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RL8abRmgFXkVASHkkqiJbE
79 lines
3.0 KiB
Python
79 lines
3.0 KiB
Python
#!/usr/bin/env python3
|
|
"""Mint LiveKit JWTs for the integration test against a local dev server.
|
|
|
|
Usage:
|
|
livekit-server --dev --bind 127.0.0.1 &
|
|
eval "$(python3 scripts/livekit-dev-room.py)"
|
|
ctest --test-dir build -R test_integration_livekit --output-on-failure
|
|
|
|
Prints shell `export` lines for the four STPLUGIN_IT_* variables
|
|
core/tests/test_integration_livekit.cpp looks for. With no arguments it uses
|
|
`livekit-server --dev`'s built-in devkey/secret credentials.
|
|
|
|
Standard library only (hmac + hashlib + base64) -- there is deliberately no
|
|
pip install step here, so this runs anywhere the repo is checked out.
|
|
"""
|
|
|
|
import argparse
|
|
import base64
|
|
import hashlib
|
|
import hmac
|
|
import json
|
|
import os
|
|
import time
|
|
|
|
|
|
def b64url(raw: bytes) -> str:
|
|
return base64.urlsafe_b64encode(raw).rstrip(b"=").decode("ascii")
|
|
|
|
|
|
def mint(api_key: str, api_secret: str, identity: str, room: str, *, publish: bool, subscribe: bool) -> str:
|
|
now = int(time.time())
|
|
header = {"alg": "HS256", "typ": "JWT"}
|
|
claims = {
|
|
"iss": api_key,
|
|
"sub": identity,
|
|
"name": identity,
|
|
"nbf": now - 10,
|
|
"exp": now + 3600,
|
|
"video": {
|
|
"room": room,
|
|
"roomJoin": True,
|
|
"canPublish": publish,
|
|
"canSubscribe": subscribe,
|
|
"canPublishData": False,
|
|
},
|
|
}
|
|
signing_input = f"{b64url(json.dumps(header, separators=(',', ':')).encode())}." \
|
|
f"{b64url(json.dumps(claims, separators=(',', ':')).encode())}"
|
|
signature = hmac.new(api_secret.encode(), signing_input.encode(), hashlib.sha256).digest()
|
|
return f"{signing_input}.{b64url(signature)}"
|
|
|
|
|
|
def main() -> None:
|
|
parser = argparse.ArgumentParser(description=__doc__)
|
|
parser.add_argument("--url", default=os.environ.get("LIVEKIT_URL", "ws://127.0.0.1:7880"))
|
|
parser.add_argument("--api-key", default=os.environ.get("LIVEKIT_API_KEY", "devkey"))
|
|
parser.add_argument("--api-secret", default=os.environ.get("LIVEKIT_API_SECRET", "secret"))
|
|
parser.add_argument("--room", default="obs-plugin-it")
|
|
parser.add_argument("--publisher-identity", default="cam-test")
|
|
parser.add_argument("--subscriber-identity", default="obs:obs-plugin-it:test")
|
|
args = parser.parse_args()
|
|
|
|
publish_token = mint(args.api_key, args.api_secret, args.publisher_identity, args.room,
|
|
publish=True, subscribe=False)
|
|
# Deliberately the same grant shape the real server mints for the plugin
|
|
# (apps/server/src/obs/plugin.routes.ts -> mintCaptionsToken): subscribe
|
|
# only, never publish.
|
|
subscribe_token = mint(args.api_key, args.api_secret, args.subscriber_identity, args.room,
|
|
publish=False, subscribe=True)
|
|
|
|
print(f'export STPLUGIN_IT_URL="{args.url}"')
|
|
print(f'export STPLUGIN_IT_PUBLISH_TOKEN="{publish_token}"')
|
|
print(f'export STPLUGIN_IT_SUBSCRIBE_TOKEN="{subscribe_token}"')
|
|
print(f'export STPLUGIN_IT_PUBLISHER_IDENTITY="{args.publisher_identity}"')
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|