obs-streamer-tools-plugin/README.md

# obs-streamer-tools-plugin

Native OBS Studio source plugin that pulls streamer-tools camera feeds
directly from LiveKit over WebRTC, replacing the current SRT/RTSP-via-VLC-or-
Media-Source path for directors. Full design:
`docs/superpowers/specs/2026-09-06-obs-camera-plugin-design.md` in the
`streamer-tools` repo.

## Status

The plugin is **functionally complete on Linux and verified end to end there**
(module loads into real libobs, connects to a real LiveKit server through the
real streamer-tools API shape, and pushes decoded frames into
`obs_source_output_video`/`_audio`).

It has **not been run in the OBS GUI on any platform.** macOS builds the real
module in CI but its artifact is not yet loadable (see the macOS packaging gap
under CI). Windows has not yet completed a build with the current fixes.

See "What is verified, and how" below for exactly what has and has not been
checked, and "Testing this by hand" for what a human still needs to do.

## Layout

```
cmake/LiveKitSDK.cmake   - downloads + unpacks the pinned client-sdk-cpp release
core/                    - core library (C++17, no OBS dependency, headless-testable)
  include/stplugin/
    core.h                 version + ConnectionConfig
    json.h                 small strict JSON reader
    http.h                 injectable HTTP client interface
    api_client.h           the two /api/obs/:slug/* calls
    session_types.h        media/state types + the pure session logic
    session.h              LiveKitSession, the livekit::Room wrapper
  src/
    http_curl.cpp          libcurl backend (Linux/macOS)
    http_winhttp.cpp       WinHTTP backend (Windows)
  tests/                   dependency-free CTest suites
obs-adapter/             - thin OBS glue (C++)
  src/plugin-main.cpp      obs_source_info, properties UI, frame output
  data/locale/en-US.ini
scripts/livekit-dev-room.py - mints tokens for the integration test
third_party/livekit/     - redistribution notices for the LiveKit binaries
.gitea/workflows/build.yml  - 3-platform CI matrix
```

## How it works

1. The operator fills in the streamer-tools server URL, room slug and read key,
   and picks a camera from the dropdown.
2. The source's own worker thread calls `POST /api/obs/:slug/token?key=…` to
   mint a hidden, subscribe-only LiveKit token
   (identity `obs:<slug>:<nonce>` — a fresh nonce per mint, so two OBS
   installs watching the same room can never kick each other).
3. `LiveKitSession` connects `livekit::Room` to the returned `wsUrl`, waits for
   the chosen participant's `Source.Camera` video track (and their microphone),
   and reads decoded frames off `VideoStream`/`AudioStream`.
4. The adapter hands those straight to `obs_source_output_video` /
   `obs_source_output_audio`.

Nothing on the OBS UI thread ever blocks on the network. The one deliberate
exception is the "Refresh camera list" button, which the operator pressed and
is waiting on; it uses a shortened 5s timeout.

### Design decisions worth knowing before changing this

- **Frames come from `VideoStream::fromTrack` with our own reader threads, not
  from `Room::setOnVideoFrameCallback`.** The dispatcher API is keyed by
  (participant identity, track *name*), which is only knowable once the track
  is published — and disassembly of `liblivekit.so` 1.10.1 confirms that
  neither `Room::setOnVideoFrameCallback` nor the dispatcher's own version
  starts a reader for an already-subscribed track; they only record the
  registration. Registering at the only moment the name exists would therefore
  have silently produced no video.
- **Every stream operation runs on one owned worker thread**, never on a
  LiveKit room event thread: `Room::disconnect()` from inside a delegate
  callback is documented to deadlock.
- **`VideoStream::Options::capacity` is 3**, making the SDK queue a
  drop-oldest ring buffer. A stalled consumer can only fall three frames
  behind and then sees the *newest* frame, not a backlog — the structural
  answer to the stale-media bug that motivated this plugin.
- **Video and audio are both timestamped with `os_gettime_ns()` at arrival.**
  The SDK gives video a WebRTC capture timestamp and audio none; mixing two
  epochs inside one OBS source would guarantee A/V drift. This relies on the
  SDK's jitter buffering having already aligned them — the assumption the
  design doc flags for verification on real hardware. **Still unverified.**
- **WebRTC changes resolution mid-stream.** Observed directly in the
  integration test: the first frames after (re)subscribing arrive at a
  downscaled spatial layer before ramping to the published size. The adapter
  passes each frame's own geometry through, and logs geometry changes.

## Building

Linux (the platform that is fully verified). `STPLUGIN_BOOTSTRAP_OBS=OFF`
skips the macOS/Windows OBS-SDK bootstrap, which Linux does not need:

```
sudo apt-get install -y cmake ninja-build libobs-dev libcurl4-openssl-dev
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DSTPLUGIN_BOOTSTRAP_OBS=OFF
cmake --build build
ctest --test-dir build --output-on-failure
```

The configure step downloads the pinned `client-sdk-cpp` release (~13 MB) into
`build/_deps/livekit-sdk`. Point `-DSTPLUGIN_LIVEKIT_SDK_DIR=<path>` at a
persistent directory to cache it across builds; `-DSTPLUGIN_LIVEKIT_SDK_VERSION`
and `-DSTPLUGIN_LIVEKIT_SDK_TRIPLE` override the pin and the release triple.

The build stages a runnable layout into `build/package/`:

```
build/package/bin/64bit/streamer-tools-camera.so   (RPATH=$ORIGIN)
build/package/bin/64bit/liblivekit.so
build/package/bin/64bit/liblivekit_ffi.so
build/package/data/locale/en-US.ini
build/package/licenses/...
```

That is exactly the layout OBS searches on Linux and Windows —
`<config>/obs-studio/plugins/<name>/bin/64bit` plus a sibling `data/`, per
`AddExtraModulePaths()` in obs-studio's `UI/window-basic-main.cpp` — so
`build/package/` is a straight drop-in. The module resolves the LiveKit
libraries from `$ORIGIN` (verified: `ldd` on the staged copy resolves both
to `bin/64bit/`), not from the build tree. macOS is not this shape; see the
macOS packaging gap under CI.

## Testing this by hand

**Nobody has yet run this in the OBS GUI. That test is still outstanding on
all three platforms.** To do it on Linux:

```
mkdir -p ~/.config/obs-studio/plugins/streamer-tools-camera
cp -r build/package/bin build/package/data \
      ~/.config/obs-studio/plugins/streamer-tools-camera/
obs
```

(That yields `.../streamer-tools-camera/bin/64bit/streamer-tools-camera.so`
and `.../streamer-tools-camera/data/locale/en-US.ini`, which is what OBS
looks for.)

Then: Sources → `+` → "streamer-tools Camera" → fill in the server URL, room
slug and read key from the room's settings page → "Refresh camera list" → pick
a camera. Check `~/.config/obs-studio/logs/` for
`[streamer-tools-camera] connected to …` and `[streamer-tools-camera] video
frame WxH I420`.

What to look for that automated testing could not answer:
- Does video actually *appear*, right way up, right colours?
- Is A/V sync acceptable? (see the timestamp caveat above)
- What is the end-to-end latency versus the existing egress path?
- Does a publisher restarting mid-show recover cleanly on screen?

### Running the LiveKit integration test

`core/tests/test_integration_livekit` publishes a synthetic camera into a real
room and subscribes to it through the wrapper. It skips unless
`STPLUGIN_IT_*` is set:

```
livekit-server --dev --bind 127.0.0.1 &
eval "$(python3 scripts/livekit-dev-room.py)"
ctest --test-dir build -R test_integration_livekit --output-on-failure
```

## What is verified, and how

Verified on Ubuntu 24.04 (libobs 30.0.2, client-sdk-cpp 1.10.1,
livekit-server 1.13.6 in dev mode):

| Claim | How it was checked |
|---|---|
| The pinned LiveKit SDK links and is callable | `test_livekit_smoke`: `initialize()`/`shutdown()` round-trip, header version asserted equal to the CMake pin |
| The JSON reader handles real and hostile input | `test_json`, 158 checks, including truncated bodies, HTML error pages, binary garbage, lone surrogates, and a depth-limit case |
| The API client parses the real response shapes and every error branch | `test_api_client`, 121 checks, against a fake HTTP client **and** a real loopback HTTP server driving the actual platform backend |
| A dead/stalled/garbage server cannot hang or crash the plugin | loopback cases: truncated JSON, connection closed with no reply, non-HTTP bytes, dead port, stalled server cut off by the client timeout |
| Session state transitions, track selection, frame geometry | `test_session`, 81 checks, plus real `connect()` failures against the real SDK |
| **Media actually flows** | `test_integration_livekit` against a real LiveKit server: 36 video frames + 323 audio frames, correct I420 geometry and plane pointers, publisher unpublish → `hasVideo()` false with **no further frames from the dead publisher**, republish → video resumes |
| **The module loads into real libobs and pushes frames** | a headless libobs harness (`obs_startup` + `obs_reset_audio`/`obs_reset_video` + `obs_open_module`) driving the built module against a stand-in streamer-tools API in front of a real LiveKit server. Log: `connected to ws://…  watching cam-test` then `video frame 640x360 I420`; the camera dropdown populated as `Test Camera` / `Dark Camera (offline)`; status `connected`; clean destroy and unload |
| A wrong read key is reported, not silently swallowed | same harness with a bad key: status `unknown room slug, or the read key is wrong or has been rotated`, warning info type, retry with backoff, no crash |
| Changing the selected camera reconnects cleanly | same harness: switch to a dark slot and back. Each switch mints a fresh `obs:<room>:<nonce>` identity and reconnects; video returns; status stays `connected`; no crash, no stale frame |
| Two sources in one OBS process | same harness with a second source added: both connect with distinct nonce identities, both receive frames, both tear down cleanly |

**Not verified anywhere:**
- The OBS GUI, on any platform. No human has looked at this in OBS.
- macOS beyond "CI builds and links the real module and the core tests pass".
  Its artifact is a bare `.so` with a relative libobs install name and will
  not load in OBS.app — see the macOS packaging gap under CI.
- Windows beyond "the core library and the WinHTTP backend compile and their
  tests pass", from runs predating the current fixes. The WinHTTP backend has
  never run against a real streamer-tools server, only against the loopback
  test server in `test_api_client`.
- A/V sync and end-to-end latency against the existing egress path.
- Behaviour against the real production streamer-tools server (only against a
  stand-in serving the same shapes).
- Token expiry after an hour. Expiry is handled *reactively*: a fatal
  disconnect makes the worker mint a fresh token and reconnect. The design
  doc's "proactively refreshed before expiry" is **not** implemented —
  `client-sdk-cpp` 1.10.1 exposes no way to hand a live `Room` a new token.

## CI

`.gitea/workflows/build.yml` runs on every push, matrixed across the three
runners available to this repo under the `CyberCoveLLC` org.

| Job | `runs-on` | Runner | State |
|---|---|---|---|
| `linux` | `ubuntu-24.04` | `localhost.localdomain` | **Green.** Builds the real adapter against Ubuntu's libobs-dev 30.0.2, runs all six test suites, uploads `build/package` as an artifact |
| `macos` | `macos-latest` | `home-mac` (Global) | **Green.** Builds libobs 30.0.2 from source, then the real adapter; 6/6 tests; artifact uploaded. But see the macOS packaging gap below |
| `windows` | `windows-latest` | `winvm-builder` (org-scoped) | **Unconfirmed** — see below |

The Linux job is pinned to `ubuntu-24.04` rather than `ubuntu-latest`: this
instance's two Linux runners answer `ubuntu-latest` with different releases,
and 22.04's `libobs-dev` is OBS 27 — a different API surface, and the LiveKit
SDK's own `linux-x64` asset does not even link there (hence the
`ubuntu-22.04` SDK triple; see `cmake/LiveKitSDK.cmake`).

macOS and Windows use the `obsproject/obs-plugintemplate` buildspec
bootstrap, trimmed to drop `qt6` (this plugin's properties UI is plain
`obs_properties_*`), with `obs-studio.version` pinned to 30.0.2 — the same
version Linux builds against, and deliberately low, because OBS rejects a
module built against a newer libobs than the one running it.

**Both jobs fall back to a core-library-only build if the bootstrap fails**,
rather than going red, with a workflow `::warning::` and a "Show what was
built" step that reports no module. That fallback exists because the
bootstrap is the least verifiable part of this project — there is no way to
exercise a macOS or Windows OBS build from the Linux development machine —
and a permanently red CI teaches people to ignore CI. **Do not remove the
warning:** a green job that quietly stopped building the plugin is worse than
a red one.

### Where the macOS bootstrap actually got to

Six CI iterations, each fixing a real failure visible in the logs:

1. Upstream's Xcode generator → `No CMAKE_C_COMPILER could be found` (the
   runner has the Command Line Tools, not Xcode). Switched to Ninja.
2. OBS's SDK version regex only matches a full-Xcode SDK path. Synthesised a
   `MacOSX.platform/Developer/SDKs/MacOSX<ver>.sdk` symlink to the same SDK.
3. The install walked into `UI/obs-frontend-api`, whose binary is
   deliberately never built. The install's exit code is now tolerated.
4. Restricting the install to `libobs/` fixed that but lost the
   per-configuration export file.
5. `xattr -r -d com.apple.quarantine` followed the SDK symlink into the
   read-only system SDK. Symlink moved to the build directory; the xattr step
   is no longer fatal.
6. `IMPORTED_LOCATION or IMPORTED_IMPLIB not set for imported target
   OBS::libobs configuration Release` — OBS 30.0.2 installs
   `libobsTargets.cmake` without the per-config file that carries the library
   path. The top-level `CMakeLists.txt` now detects a locationless
   `OBS::libobs` and points it at the framework the bootstrap just built.

All six are confirmed fixed: the macOS job now downloads obs-deps and
obs-studio, builds libobs from source, builds and links the real adapter,
passes 6/6 tests, and uploads its artifact. `otool -L` on the result shows it
linked against libobs and `@rpath/liblivekit.dylib`.

### macOS packaging gap (known, unfixed)

**The macOS artifact will not load in OBS.app as it stands.** Two reasons,
neither of which CI can catch, because CI only proves it compiles and links:

1. It is a bare `streamer-tools-camera.so`. OBS on macOS loads plugins as
   `<name>.plugin` bundles (`Contents/MacOS/<name>`, `Contents/Resources/`,
   an `Info.plist`), which is what obs-plugintemplate's
   `cmake/macos/helpers.cmake` builds and which this project deliberately did
   not vendor.
2. `otool -L` shows the libobs dependency recorded as the relative path
   `libobs/libobs.framework/Versions/A/libobs`, inherited from the
   from-source libobs's own install name. A real plugin needs
   `@rpath/libobs.framework/Versions/A/libobs` plus an `LC_RPATH` pointing at
   `OBS.app/Contents/Frameworks`.

Fixing this means either vendoring the template's macOS bundle helpers or
adding an `install_name_tool` pass and a bundle layout — bounded work, but
work that has to be done and checked on an actual Mac. It is deliberately not
attempted here rather than guessed at.

### Where the Windows bootstrap got to

Windows is by far the slowest job — the `lukka/get-cmake` step alone takes
7-15 minutes on `winvm-builder`, and the runner serialises jobs, so a burst of
pushes leaves a queue that takes an hour to drain. One confirmed bug of its
own was found and fixed: upstream passes `-A x64,version=<Windows SDK>` to the
OBS sub-configure, and with a current CMake that `,version=` suffix reappears
verbatim in the sub-build's `CMAKE_VS_PLATFORM_NAME` — which obs-studio's own
dependency downloader uses as the architecture, sending it after
`windows-deps-2023-11-03-x64,version=10.0.26100.0.zip`:

```
string sub-command JSON member 'hashes windows-x64,version=10.0.26100.0' not found
Unable to download .../windows-deps-2023-11-03-x64,version=10.0.26100.0.zip
```

Plain `-A x64` now. **No Windows run has yet completed with that fix in
place**, so Windows should be treated as unverified beyond "the core library
and the WinHTTP backend compile and their tests pass", which earlier runs did
show. Expect further iterations there of the same kind the macOS bootstrap
needed.