Compositing graphics straight into a WebRTC video source

RenderingJune 2026

Every guest in Castalong arrives as a native OBS source whose frames come off a WebRTC track. On-screen graphics — the name lower-third, a branding frame around the webcam, a scrolling ticker — are drawn into those frames, on the streamer's machine, so they follow the guest wherever you switch them. This note is how that path works and why it's built the way it is.

The old way, and why we left it

The first version was the obvious one: a transparent browser source rendering an HTML overlay, fed by the control panel. Two problems showed up fast. OBS's embedded browser (CEF) does not reliably consume EventSource/SSE, so the overlay ended up polling the server a few times a second — which became roughly 85% of the access log and a load source we had to tune fail2ban around. And the overlay was a separate, canvas-fixed source: it didn't travel with a guest when you re-routed them between scene slots.

So graphics moved into the plugin, and the design goal became: no server round-trips, and the overlay belongs to the guest, not to a screen position.

Definitions as data, over the data channel

The overlay is sent as a tiny JSON definition, not as rendered pixels, over the LiveKit data channel the plugin already uses for routing commands (topic castalong):

{ "type": "gfx", "guest": "Alice",
  "name":   { "show": true, "l1": "Alice", "accent": "#0d9488", "pos": "bl" },
  "frame":  { "url": "https://.../frame.png" },
  "ticker": { "show": true, "text": "..." } }

It's keyed by guest name, the same key the switcher routes on, so "travels with the guest" falls out for free: whichever source is currently rendering Alice composites Alice's overlay. The control surface publishes it straight to the plugin through the SFU, so there is no graphics state on the server and nothing to poll.

Rasterise off the pump; blend on it

The video pump is hot — it runs per frame, per guest. It must never allocate-heavy, never lock, never rasterise text. So the work is split:

• When a def (or the routed guest) changes, an off-pump task rasterises it into an immutable OverlaySnapshot — a set of straight-alpha RGBA layers plus an optional animated ticker — and publishes it with ArcSwap.
• The pump does a single lock-free load() of the current snapshot and blends it onto the frame. The only per-frame animation is the ticker's x-offset; its glyphs are rasterised once.

Because the snapshot is immutable and swapped atomically, the pump always sees a fully-formed overlay or the previous one — never a half-built combination of name, frame and visibility. That's exactly the consistency you want for video.

One pixel format for the whole life of the source

The guest path now always outputs BGRA: I420 → scale/letterbox → BGRA → fade → blend → output. That dropped the I420 fast-path, but it also fixed a real bug. The source used to flip between transparent-BGRA (when idle) and I420 (when live) on every punch, and that runtime format toggle re-initialises OBS's async source machinery. Committing to BGRA for the source's whole lifetime removes the toggle. The cost is one I420→BGRA pass per frame per guest — sub-millisecond at 720p — using the same limited-range BT.709 matrix OBS would use, so colours match.

No C-backed graphics libraries

The plugin links libwebrtc, which is /MT + crt-static on Windows, and it also builds for Linux. A C-backed 2D or font library would fight both. So the renderer is deliberately dependency-light: fontdue for glyph rasterising, a bundled OFL font via include_bytes! (no system-font discovery), hand-rolled rectangles, a stride-aware RGBA-over-BGRA blend, and the pure-Rust image/png decode for the branding frame (fetched once by URL and cached). Everything that touches pixels is Rust we control.