obs-streamer-tools-plugin/README.md

# obs-streamer-tools-plugin

Native OBS Studio source plugin that pulls streamer-tools camera feeds
directly from LiveKit over WebRTC, replacing the current SRT/RTSP-via-VLC-or-
Media-Source path for directors. Full design:
`docs/superpowers/specs/2026-09-06-obs-camera-plugin-design.md` in the
`streamer-tools` repo.

## Status

The plugin is **functionally complete on Linux and verified end to end there**
(module loads into real libobs, connects to a real LiveKit server through the
real streamer-tools API shape, and pushes decoded frames into
`obs_source_output_video`/`_audio`). It has **not** been run in the OBS GUI on
any platform, and macOS/Windows have only ever been built by CI, never loaded.
See "What is verified, and how" below for exactly what that means, and
"Testing this by hand" for what a human still needs to do.

## Layout

```
cmake/LiveKitSDK.cmake   - downloads + unpacks the pinned client-sdk-cpp release
core/                    - core library (C++17, no OBS dependency, headless-testable)
  include/stplugin/
    core.h                 version + ConnectionConfig
    json.h                 small strict JSON reader
    http.h                 injectable HTTP client interface
    api_client.h           the two /api/obs/:slug/* calls
    session_types.h        media/state types + the pure session logic
    session.h              LiveKitSession, the livekit::Room wrapper
  src/
    http_curl.cpp          libcurl backend (Linux/macOS)
    http_winhttp.cpp       WinHTTP backend (Windows)
  tests/                   dependency-free CTest suites
obs-adapter/             - thin OBS glue (C++)
  src/plugin-main.cpp      obs_source_info, properties UI, frame output
  data/locale/en-US.ini
scripts/livekit-dev-room.py - mints tokens for the integration test
third_party/livekit/     - redistribution notices for the LiveKit binaries
.gitea/workflows/build.yml  - 3-platform CI matrix
```

## How it works

1. The operator fills in the streamer-tools server URL, room slug and read key,
   and picks a camera from the dropdown.
2. The source's own worker thread calls `POST /api/obs/:slug/token?key=…` to
   mint a hidden, subscribe-only LiveKit token
   (identity `obs:<slug>:<nonce>` — a fresh nonce per mint, so two OBS
   installs watching the same room can never kick each other).
3. `LiveKitSession` connects `livekit::Room` to the returned `wsUrl`, waits for
   the chosen participant's `Source.Camera` video track (and their microphone),
   and reads decoded frames off `VideoStream`/`AudioStream`.
4. The adapter hands those straight to `obs_source_output_video` /
   `obs_source_output_audio`.

Nothing on the OBS UI thread ever blocks on the network. The one deliberate
exception is the "Refresh camera list" button, which the operator pressed and
is waiting on; it uses a shortened 5s timeout.

### Design decisions worth knowing before changing this

- **Frames come from `VideoStream::fromTrack` with our own reader threads, not
  from `Room::setOnVideoFrameCallback`.** The dispatcher API is keyed by
  (participant identity, track *name*), which is only knowable once the track
  is published — and disassembly of `liblivekit.so` 1.10.1 confirms that
  neither `Room::setOnVideoFrameCallback` nor the dispatcher's own version
  starts a reader for an already-subscribed track; they only record the
  registration. Registering at the only moment the name exists would therefore
  have silently produced no video.
- **Every stream operation runs on one owned worker thread**, never on a
  LiveKit room event thread: `Room::disconnect()` from inside a delegate
  callback is documented to deadlock.
- **`VideoStream::Options::capacity` is 3**, making the SDK queue a
  drop-oldest ring buffer. A stalled consumer can only fall three frames
  behind and then sees the *newest* frame, not a backlog — the structural
  answer to the stale-media bug that motivated this plugin.
- **Video and audio are both timestamped with `os_gettime_ns()` at arrival.**
  The SDK gives video a WebRTC capture timestamp and audio none; mixing two
  epochs inside one OBS source would guarantee A/V drift. This relies on the
  SDK's jitter buffering having already aligned them — the assumption the
  design doc flags for verification on real hardware. **Still unverified.**
- **WebRTC changes resolution mid-stream.** Observed directly in the
  integration test: the first frames after (re)subscribing arrive at a
  downscaled spatial layer before ramping to the published size. The adapter
  passes each frame's own geometry through, and logs geometry changes.

## Building

Linux (the platform that is fully verified):

```
sudo apt-get install -y cmake ninja-build libobs-dev libcurl4-openssl-dev
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build
ctest --test-dir build --output-on-failure
```

The configure step downloads the pinned `client-sdk-cpp` release (~13 MB) into
`build/_deps/livekit-sdk`. Point `-DSTPLUGIN_LIVEKIT_SDK_DIR=<path>` at a
persistent directory to cache it across builds; `-DSTPLUGIN_LIVEKIT_SDK_VERSION`
and `-DSTPLUGIN_LIVEKIT_SDK_TRIPLE` override the pin and the release triple.

The build stages a runnable layout into `build/package/`:

```
build/package/bin/streamer-tools-camera.so   (RPATH=$ORIGIN)
build/package/bin/liblivekit.so
build/package/bin/liblivekit_ffi.so
build/package/data/locale/en-US.ini
build/package/licenses/...
```

`build/package/bin` is what gets installed — the module resolves the LiveKit
libraries from `$ORIGIN` / `@loader_path`, not from the build tree.

## Testing this by hand

**Nobody has yet run this in the OBS GUI. That test is still outstanding on
all three platforms.** To do it on Linux:

```
mkdir -p ~/.config/obs-studio/plugins/streamer-tools-camera
cp -r build/package/bin build/package/data \
      ~/.config/obs-studio/plugins/streamer-tools-camera/
obs
```

Then: Sources → `+` → "streamer-tools Camera" → fill in the server URL, room
slug and read key from the room's settings page → "Refresh camera list" → pick
a camera. Check `~/.config/obs-studio/logs/` for
`[streamer-tools-camera] connected to …` and `[streamer-tools-camera] video
frame WxH I420`.

What to look for that automated testing could not answer:
- Does video actually *appear*, right way up, right colours?
- Is A/V sync acceptable? (see the timestamp caveat above)
- What is the end-to-end latency versus the existing egress path?
- Does a publisher restarting mid-show recover cleanly on screen?

### Running the LiveKit integration test

`core/tests/test_integration_livekit` publishes a synthetic camera into a real
room and subscribes to it through the wrapper. It skips unless
`STPLUGIN_IT_*` is set:

```
livekit-server --dev --bind 127.0.0.1 &
eval "$(python3 scripts/livekit-dev-room.py)"
ctest --test-dir build -R test_integration_livekit --output-on-failure
```

## What is verified, and how

Verified on Ubuntu 24.04 (libobs 30.0.2, client-sdk-cpp 1.10.1,
livekit-server 1.13.6 in dev mode):

| Claim | How it was checked |
|---|---|
| The pinned LiveKit SDK links and is callable | `test_livekit_smoke`: `initialize()`/`shutdown()` round-trip, header version asserted equal to the CMake pin |
| The JSON reader handles real and hostile input | `test_json`, 158 checks, including truncated bodies, HTML error pages, binary garbage, lone surrogates, and a depth-limit case |
| The API client parses the real response shapes and every error branch | `test_api_client`, 121 checks, against a fake HTTP client **and** a real loopback HTTP server driving the actual platform backend |
| A dead/stalled/garbage server cannot hang or crash the plugin | loopback cases: truncated JSON, connection closed with no reply, non-HTTP bytes, dead port, stalled server cut off by the client timeout |
| Session state transitions, track selection, frame geometry | `test_session`, 81 checks, plus real `connect()` failures against the real SDK |
| **Media actually flows** | `test_integration_livekit` against a real LiveKit server: 36 video frames + 323 audio frames, correct I420 geometry and plane pointers, publisher unpublish → `hasVideo()` false with **no further frames from the dead publisher**, republish → video resumes |
| **The module loads into real libobs and pushes frames** | a headless libobs harness (`obs_startup` + `obs_reset_audio`/`obs_reset_video` + `obs_open_module`) driving the built module against a stand-in streamer-tools API in front of a real LiveKit server. Log: `connected to ws://…  watching cam-test` then `video frame 640x360 I420`; the camera dropdown populated as `Test Camera` / `Dark Camera (offline)`; status `connected`; clean destroy and unload |
| A wrong read key is reported, not silently swallowed | same harness with a bad key: status `unknown room slug, or the read key is wrong or has been rotated`, warning info type, retry with backoff, no crash |

**Not verified anywhere:**
- The OBS GUI, on any platform. No human has looked at this in OBS.
- macOS and Windows beyond "CI compiles and the core tests pass". The WinHTTP
  backend has never run against a real streamer-tools server.
- A/V sync and end-to-end latency against the existing egress path.
- Behaviour against the real production streamer-tools server (only against a
  stand-in serving the same shapes).
- Token expiry after an hour. Expiry is handled *reactively*: a fatal
  disconnect makes the worker mint a fresh token and reconnect. The design
  doc's "proactively refreshed before expiry" is **not** implemented —
  `client-sdk-cpp` 1.10.1 exposes no way to hand a live `Room` a new token.

## CI

`.gitea/workflows/build.yml` runs on every push, matrixed across the three
runners available to this repo under the `CyberCoveLLC` org.

| Job | `runs-on` | Runner |
|---|---|---|
| `linux` | `ubuntu-latest` | `gitea-runner.internal.cloud-hosting.io` (Global) |
| `macos` | `macos-latest` | `home-mac` (Global) |
| `windows` | `windows-latest` | `winvm-builder` (org-scoped) |

Linux uses Ubuntu's `libobs-dev` and builds the real OBS adapter. macOS and
Windows use the `obsproject/obs-plugintemplate` buildspec bootstrap, trimmed
to drop `qt6` (this plugin's properties UI is plain `obs_properties_*`), with
`obs-studio.version` pinned no newer than what Linux builds against — OBS
rejects a module built against a newer libobs than the one running it.