Real-time Systems

The media and transport layer that moves a frame from the host to your screen in a few milliseconds.

Section
06 / 12
Read
1 min
Updated

Media

FFmpegmedia engine

FFmpeg handles encoding and decoding on the host, with hardware paths for NVENC, QuickSync, AMF and VideoToolbox, and H.264, HEVC and AV1 where the hardware and the browser both agree.

Opusaudio

Low latency audio in both directions, for the session sound and for voice in a room.

Transport

WebRTC

WebRTC for one to one sessions. ICE finds the shortest path, DTLS-SRTP encrypts it, and a TURN relay steps in only when the network refuses a direct route.

QUIC and Media over QUIC

QUIC with TLS 1.3 built in, carrying Media over QUIC. One publish, many subscribers, and no head of line blocking between them.

Why two transports

WhenPickReason
One person drivingWebRTCShortest possible path, direct where the network allows
Several people watchingMedia over QUICFan out through a relay, so a slow viewer stalls only their own picture
LAN or VPNMedia over QUIC directNo relay hop at all when the browser can reach the host

Input

ViGEmBuswindows

ViGEmBus presents a virtual Xbox or PlayStation controller to games and carries rumble back.

Input driverwindows

Alluno's own input driver delivers keyboard and mouse events at the device level, which is what raw input titles expect.

Which transport to pick, and why