4 Commits

Include renames

Author	SHA1	Message	Date
shadowdaoandClaude Sonnet 5	3f2933ae3f	fix(session): recover stalled video subscriptions with a keyframe-forcing watchdog ... Camera sources in the OBS plugin were dropping out at random and never recovering, while the same players stayed healthy in browser talkback. Measured on the live server: every OBS plugin subscriber racked up 862-1099 nackMisses (retransmit requests for packets the SFU had already aged out of its send buffer -- unrecoverable loss) and sat at plis == 2 for a multi-hour session, versus 0/adaptive-PLI for a browser subscriber in the same room. The pinned client-sdk-cpp (1.10.1) exposes no PLI/keyframe-request API at all, so a decoder that lost a frame that way had no way to resync for the rest of the show. Add a stall-recovery watchdog: StallWatchdog (session_types.h/.cpp) is a pure, fake-clock-testable class that decides when a video subscription has gone too long (2000ms, kStallRecoveryTimeout) without a decoded frame reaching OBS. LiveKitSession::Impl polls it from a dedicated thread and, through the existing command queue (never touching the SDK off the worker thread), toggles the publication's setEnabled(false)/ setEnabled(true) -- the one lever this SDK exposes that makes the SFU restart delivery, and a restart always begins with a keyframe. Repeated attempts against the same stall back off exponentially (2s/4s/8s/16s/30s-capped, mirroring the shape of the adapter's own reconnect backoff) so a genuinely gone publisher is retried on a bounded cadence instead of hammered every 2 seconds. A muted, disabled or unsubscribed track, an audio-only source, or a disconnected session never arms the watchdog: new onTrackMuted/onTrackUnmuted handlers suspend and resume its clock, always re-baselining from "now" rather than a stale timestamp, so un-muting after a long legitimate camera-off period cannot read as a multi-minute stall. Each attempt, and eventual recovery, is logged through a new DiagnosticHandler that the OBS adapter maps onto obs_log at the same severities the file already uses for other notable events. Also investigated (not changed): the setVideoQuality(HIGH) pin added for an earlier simulcast-resize bug. The SDK's own docs and the FFI binary's wire-protocol strings show setVideoQuality only bounds spatial/simulcast layer selection (UpdateTrackSettings.quality), never temporal layers or frame rate, which the SFU's own congestion control governs independently -- so this pin is unlikely to be the cause of the packet-loss symptom, and is probably an inert no-op now that publishing is pinned server-side to a single spatial layer (L1T3). Left in place since that is not fully unambiguous from the SDK alone. Full writeup in .stall-recovery-report.md (untracked, not part of this commit). Adds 3 new headless StallWatchdog tests (fires-after-threshold, does- not-fire-when-muted/disabled, backs-off-rather-than-loops) to test_session.cpp; caught a real bug during development where onFrameDelivered() reset the backoff duration but not the next-attempt-allowed timestamp, throttling a just-recovered stream against its own stale backoff. Built and all 6 CTest suites pass (124/124 checks in test_session); also verified clean under ThreadSanitizer with warning counts unchanged from the pre-change baseline. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-09-20 19:04:14 -07:00
shadowdaoandClaude Sonnet 5	af7d2c6d24	fix: pin video quality to stop OBS source resizing; add audio-only mode ... Live testing (2026-09-07) showed two real problems in one root cause: LiveKit's default subscriber behavior lets the SFU switch simulcast layers on its own bandwidth/adaptive logic, and this plugin never told it not to. For a real camera, that showed up as the OBS source's received frame size visibly hopping between 320x180/640x360/1280x720 mid-show -- OBS's async video source resizes to match, breaking any manual crop/position a director had set up. For the soundboard (a Camera-source track that exists only to satisfy RTMP's video requirement -- Soundboard.tsx -- with no real visual content), the same instability, plus the video showing at all, was pure noise: there was no way to pull just its audio. Both come from RemoteTrackPublication (livekit/remote_track_publication.h in the pinned SDK), on the exact publication object TrackSubscribedEvent and attachExistingTracks already hand this code: - setVideoQuality(VideoQuality::HIGH) on every wanted video track, unconditionally, so the SFU always sends the top simulcast layer instead of switching layers underneath a source with no rendered-size hint to give it (this is a native subscriber, not a sized <video> element). - A new SessionConfig::subscribe_video (mirrors subscribe_audio): when false, the wanted video track is never attached, and its publication is explicitly setEnabled(false) -- the SFU stops sending it, not just "decoded and discarded here". Wired to a new "Audio only (no video)" checkbox in the source's properties. Both call sites (a fresh TrackSubscribedEvent, and attachExistingTracks sweeping tracks already up when the session starts watching) go through one new handleWantedVideoTrack() so they can't drift apart. Not unit-testable without a real LiveKit connection (RemoteTrackPublication isn't fakeable, matching why test_integration_livekit.cpp already needs a real server) -- verified instead by a full local build against real libobs-dev + the pinned SDK (clean compile, all 6 existing tests still pass) and CI. The actual behavioral fix -- stable resolution, no video for an audio-only source -- needs the same real-OBS verification every other claim in this repo's "What is verified, and how" section does. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RL8abRmgFXkVASHkkqiJbE	2026-09-07 10:43:54 -07:00
shadowdaoandClaude Sonnet 5	969b8db94a	license: relicense first-party code from GPL-2.0-or-later to Apache-2.0 ... Owner sign-off: replace root LICENSE with Apache License 2.0, add a root NOTICE file, and swap the GPL-2.0 boilerplate header in every first-party core/ and obs-adapter/ source file for a short Apache-2.0 notice. This resolves review finding C2 (GPLv2 top-level LICENSE vs. the vendored Apache-2.0 LiveKit SDK is a license-compatibility violation): the whole repo is now Apache-2.0, matching LiveKit, so there's no GPL/Apache clash left. Updated the README Status gate and the CI workflow comment to reflect that C2 is resolved, while leaving the C1 WebRTC/OpenH264 patent/royalty gate untouched -- that question is still open and still blocks release. third_party/ stays under its own upstream licenses; only this project's own code changed hands. All 6 CTest suites still pass after the header swap. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RL8abRmgFXkVASHkkqiJbE	2026-09-07 04:44:16 -07:00
shadowdaoandClaude Sonnet 5	80904a3e85	Add the LiveKit session wrapper, verified end-to-end against a real room ... stplugin::LiveKitSession wraps livekit::Room for exactly one subscribed slot: connect with the wsUrl/lkToken the API client minted, find the chosen participant's camera (and microphone), and hand decoded frames to callback-shaped handlers the OBS adapter can consume directly. Two architectural decisions worth recording, both forced by reading the SDK rather than guessed: 1. Frames come from VideoStream/AudioStream::fromTrack with our own reader threads, NOT from Room::setOnVideoFrameCallback. The dispatcher API is keyed by (participant identity, track NAME), which we cannot know before the track is published -- and disassembling liblivekit.so confirms that both Room::setOnVideoFrameCallback and the dispatcher's own setOnVideoFrameCallback merely record the registration: neither starts a reader for a track that is already subscribed. Registering after the subscription event, which is the only time the track name exists, would therefore have silently produced no video. Taking the shared_ptr<Track> straight off the TrackSubscribedEvent sidesteps the name entirely, and lets us pick the camera by TrackSource (streamer-tools publishes cameras as Source.Camera and screenshares separately -- apps/web/src/avatar/ publish.ts), which is what we actually mean. 2. Every stream operation runs on one owned worker thread, never on a room event thread. The SDK documents that Room::disconnect() from inside a delegate callback deadlocks, and Room's own event dispatch holds a mutex, so delegate callbacks only ever enqueue a command here. VideoStream::Options::capacity is set (3 frames) so the SDK's queue is a drop-oldest ring buffer: a stalled consumer can only fall three frames behind, and what it then sees is the newest frame rather than a backlog. That is the structural answer to the stale-media bug that motivated this plugin. The pure decision-making -- the state machine, track selection, frame geometry validation -- lives in session_types.h/.cpp with no LiveKit or OBS types, so it is unit-testable headlessly (81 checks in test_session, including the publisher-swap and reconnect transitions, plus the real connect() failure paths against the real SDK: unreachable host, garbage token, incomplete config, and destruction mid-connect). test_integration_livekit is the test that proves media actually flows. It publishes a synthetic camera and microphone into a real LiveKit room using the same SDK, subscribes through LiveKitSession, and asserts on the exact fields the OBS adapter will dereference. It skips (exit 0) unless STPLUGIN_IT_* is set, so the three build runners stay green; scripts/livekit-dev-room.py mints the tokens for a local `livekit-server --dev`. Verified locally against livekit-server 1.13.6 in dev mode: integration_livekit: 36 video frames, 323 audio frames, 10 state changes integration_livekit: 32 checks passed covering: connect; subscribe to the named participant's camera; 320x240 I420 frames with three planes, non-null plane pointers and strides >= the frame's own width; 48kHz audio; unpublish -> hasVideo() false, state stays Connected (a dark camera is the placeholder state, never an error) and NO further frames arrive from the dead publisher; republish -> video resumes; clean disconnect. One real finding from that run, now handled: WebRTC ramps a new subscription up from a downscaled spatial layer, so the first frames after (re)subscribing legitimately arrive smaller than what is being published. The OBS adapter must cope with a mid-stream resolution change; the test asserts per-frame geometry rather than the publisher's, and separately asserts the stream does reach full size. Full suite: ctest -> 6/6 passed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RL8abRmgFXkVASHkkqiJbE	2026-09-06 21:39:49 -07:00