# obs-streamer-tools-plugin Native OBS Studio source plugin that pulls streamer-tools camera feeds directly from LiveKit over WebRTC, replacing the current SRT/RTSP-via-VLC-or- Media-Source path for directors. Full design: `docs/superpowers/specs/2026-09-06-obs-camera-plugin-design.md` in the `streamer-tools` repo. ## Status The plugin is **functionally complete on Linux and verified end to end there** (module loads into real libobs, connects to a real LiveKit server through the real streamer-tools API shape, and pushes decoded frames into `obs_source_output_video`/`_audio`). It has **not** been run in the OBS GUI on any platform, and macOS/Windows have only ever been built by CI, never loaded. See "What is verified, and how" below for exactly what that means, and "Testing this by hand" for what a human still needs to do. ## Layout ``` cmake/LiveKitSDK.cmake - downloads + unpacks the pinned client-sdk-cpp release core/ - core library (C++17, no OBS dependency, headless-testable) include/stplugin/ core.h version + ConnectionConfig json.h small strict JSON reader http.h injectable HTTP client interface api_client.h the two /api/obs/:slug/* calls session_types.h media/state types + the pure session logic session.h LiveKitSession, the livekit::Room wrapper src/ http_curl.cpp libcurl backend (Linux/macOS) http_winhttp.cpp WinHTTP backend (Windows) tests/ dependency-free CTest suites obs-adapter/ - thin OBS glue (C++) src/plugin-main.cpp obs_source_info, properties UI, frame output data/locale/en-US.ini scripts/livekit-dev-room.py - mints tokens for the integration test third_party/livekit/ - redistribution notices for the LiveKit binaries .gitea/workflows/build.yml - 3-platform CI matrix ``` ## How it works 1. The operator fills in the streamer-tools server URL, room slug and read key, and picks a camera from the dropdown. 2. The source's own worker thread calls `POST /api/obs/:slug/token?key=…` to mint a hidden, subscribe-only LiveKit token (identity `obs::` — a fresh nonce per mint, so two OBS installs watching the same room can never kick each other). 3. `LiveKitSession` connects `livekit::Room` to the returned `wsUrl`, waits for the chosen participant's `Source.Camera` video track (and their microphone), and reads decoded frames off `VideoStream`/`AudioStream`. 4. The adapter hands those straight to `obs_source_output_video` / `obs_source_output_audio`. Nothing on the OBS UI thread ever blocks on the network. The one deliberate exception is the "Refresh camera list" button, which the operator pressed and is waiting on; it uses a shortened 5s timeout. ### Design decisions worth knowing before changing this - **Frames come from `VideoStream::fromTrack` with our own reader threads, not from `Room::setOnVideoFrameCallback`.** The dispatcher API is keyed by (participant identity, track *name*), which is only knowable once the track is published — and disassembly of `liblivekit.so` 1.10.1 confirms that neither `Room::setOnVideoFrameCallback` nor the dispatcher's own version starts a reader for an already-subscribed track; they only record the registration. Registering at the only moment the name exists would therefore have silently produced no video. - **Every stream operation runs on one owned worker thread**, never on a LiveKit room event thread: `Room::disconnect()` from inside a delegate callback is documented to deadlock. - **`VideoStream::Options::capacity` is 3**, making the SDK queue a drop-oldest ring buffer. A stalled consumer can only fall three frames behind and then sees the *newest* frame, not a backlog — the structural answer to the stale-media bug that motivated this plugin. - **Video and audio are both timestamped with `os_gettime_ns()` at arrival.** The SDK gives video a WebRTC capture timestamp and audio none; mixing two epochs inside one OBS source would guarantee A/V drift. This relies on the SDK's jitter buffering having already aligned them — the assumption the design doc flags for verification on real hardware. **Still unverified.** - **WebRTC changes resolution mid-stream.** Observed directly in the integration test: the first frames after (re)subscribing arrive at a downscaled spatial layer before ramping to the published size. The adapter passes each frame's own geometry through, and logs geometry changes. ## Building Linux (the platform that is fully verified): ``` sudo apt-get install -y cmake ninja-build libobs-dev libcurl4-openssl-dev cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release cmake --build build ctest --test-dir build --output-on-failure ``` The configure step downloads the pinned `client-sdk-cpp` release (~13 MB) into `build/_deps/livekit-sdk`. Point `-DSTPLUGIN_LIVEKIT_SDK_DIR=` at a persistent directory to cache it across builds; `-DSTPLUGIN_LIVEKIT_SDK_VERSION` and `-DSTPLUGIN_LIVEKIT_SDK_TRIPLE` override the pin and the release triple. The build stages a runnable layout into `build/package/`: ``` build/package/bin/streamer-tools-camera.so (RPATH=$ORIGIN) build/package/bin/liblivekit.so build/package/bin/liblivekit_ffi.so build/package/data/locale/en-US.ini build/package/licenses/... ``` `build/package/bin` is what gets installed — the module resolves the LiveKit libraries from `$ORIGIN` / `@loader_path`, not from the build tree. ## Testing this by hand **Nobody has yet run this in the OBS GUI. That test is still outstanding on all three platforms.** To do it on Linux: ``` mkdir -p ~/.config/obs-studio/plugins/streamer-tools-camera cp -r build/package/bin build/package/data \ ~/.config/obs-studio/plugins/streamer-tools-camera/ obs ``` Then: Sources → `+` → "streamer-tools Camera" → fill in the server URL, room slug and read key from the room's settings page → "Refresh camera list" → pick a camera. Check `~/.config/obs-studio/logs/` for `[streamer-tools-camera] connected to …` and `[streamer-tools-camera] video frame WxH I420`. What to look for that automated testing could not answer: - Does video actually *appear*, right way up, right colours? - Is A/V sync acceptable? (see the timestamp caveat above) - What is the end-to-end latency versus the existing egress path? - Does a publisher restarting mid-show recover cleanly on screen? ### Running the LiveKit integration test `core/tests/test_integration_livekit` publishes a synthetic camera into a real room and subscribes to it through the wrapper. It skips unless `STPLUGIN_IT_*` is set: ``` livekit-server --dev --bind 127.0.0.1 & eval "$(python3 scripts/livekit-dev-room.py)" ctest --test-dir build -R test_integration_livekit --output-on-failure ``` ## What is verified, and how Verified on Ubuntu 24.04 (libobs 30.0.2, client-sdk-cpp 1.10.1, livekit-server 1.13.6 in dev mode): | Claim | How it was checked | |---|---| | The pinned LiveKit SDK links and is callable | `test_livekit_smoke`: `initialize()`/`shutdown()` round-trip, header version asserted equal to the CMake pin | | The JSON reader handles real and hostile input | `test_json`, 158 checks, including truncated bodies, HTML error pages, binary garbage, lone surrogates, and a depth-limit case | | The API client parses the real response shapes and every error branch | `test_api_client`, 121 checks, against a fake HTTP client **and** a real loopback HTTP server driving the actual platform backend | | A dead/stalled/garbage server cannot hang or crash the plugin | loopback cases: truncated JSON, connection closed with no reply, non-HTTP bytes, dead port, stalled server cut off by the client timeout | | Session state transitions, track selection, frame geometry | `test_session`, 81 checks, plus real `connect()` failures against the real SDK | | **Media actually flows** | `test_integration_livekit` against a real LiveKit server: 36 video frames + 323 audio frames, correct I420 geometry and plane pointers, publisher unpublish → `hasVideo()` false with **no further frames from the dead publisher**, republish → video resumes | | **The module loads into real libobs and pushes frames** | a headless libobs harness (`obs_startup` + `obs_reset_audio`/`obs_reset_video` + `obs_open_module`) driving the built module against a stand-in streamer-tools API in front of a real LiveKit server. Log: `connected to ws://… watching cam-test` then `video frame 640x360 I420`; the camera dropdown populated as `Test Camera` / `Dark Camera (offline)`; status `connected`; clean destroy and unload | | A wrong read key is reported, not silently swallowed | same harness with a bad key: status `unknown room slug, or the read key is wrong or has been rotated`, warning info type, retry with backoff, no crash | **Not verified anywhere:** - The OBS GUI, on any platform. No human has looked at this in OBS. - macOS and Windows beyond "CI compiles and the core tests pass". The WinHTTP backend has never run against a real streamer-tools server. - A/V sync and end-to-end latency against the existing egress path. - Behaviour against the real production streamer-tools server (only against a stand-in serving the same shapes). - Token expiry after an hour. Expiry is handled *reactively*: a fatal disconnect makes the worker mint a fresh token and reconnect. The design doc's "proactively refreshed before expiry" is **not** implemented — `client-sdk-cpp` 1.10.1 exposes no way to hand a live `Room` a new token. ## CI `.gitea/workflows/build.yml` runs on every push, matrixed across the three runners available to this repo under the `CyberCoveLLC` org. | Job | `runs-on` | Runner | |---|---|---| | `linux` | `ubuntu-latest` | `gitea-runner.internal.cloud-hosting.io` (Global) | | `macos` | `macos-latest` | `home-mac` (Global) | | `windows` | `windows-latest` | `winvm-builder` (org-scoped) | Linux uses Ubuntu's `libobs-dev` and builds the real OBS adapter. macOS and Windows use the `obsproject/obs-plugintemplate` buildspec bootstrap, trimmed to drop `qt6` (this plugin's properties UI is plain `obs_properties_*`), with `obs-studio.version` pinned no newer than what Linux builds against — OBS rejects a module built against a newer libobs than the one running it.