4 ms·

What's the state of the art in distributing WebRTC to 100k+ clients? When I was more into the low latency streaming space a few years ago, it felt like WebRTC

What's the state of the art in distributing WebRTC to 100k+ clients? When I was more into the low latency streaming space a few years ago, it felt like WebRTC was there when it came to << 1 second latency, but the infrastructure to actually distribute it was not really. I think Cloudflare (and maybe some other vendors) were working on creating some standard, has it landed? Can I run my own horizontally scalable WebRTC broadcaster (are there open source implementations)? Something like Low-Latency HLS or CMAF was at like < 5 second latency, but was on the other hand stupidly easy to distribute widely (just static files on a plain old CDN / http server).

disqard 2y ago
I'm unaware of WebRTC being used for this sort of "multicast" context, especially with such a high "fanout" -- all I've used/built are p2p contexts like video/data. Not knowledgeable here, but also interested if someone actually knows the answer to your question...
- gigachadbro 2y ago
  I previously was involved in a service where WebRTC was used for a fairly large number of N streams to M viewers. Like ~75 streams total and 500+ viewers. Solution space was public sector incident command (think police officers with multiple cameras on their body streaming to command center).
- rattt 2y ago
  Microsoft's now shut down Mixer (formerly Beam.pro) primarly used WebRTC and had streams with up to 100k viewers at some point all via WebRTC, but can't find much info about it anymore now sadly.
gorbypark 2y ago
I think state of the art in this space is WHIP/WHEP, which is what this project is implementing. I'm guessing 100k+ clients would be pretty difficult for Broadcast Box (out of the box), however Cloudflare has a beta stream service that's has a tagline of "Sub-second latency live streaming (using WHIP) and playback (using WHEP) to unlimited concurrent viewers." https://developers.cloudflare.com/stream/webrtc-beta/ https://developers.cloudflare.com/stream/webrtc-beta/
Sean-Der 2y ago
Open Source doesn’t have an instant scalability offering yet. I hope someone takes it on! Quite a few companies offer this a service though. Cloudflare, Twitch’s IVS, Dolby Millicast, Phenix RTS… LiveKit did a write up on how they built theirs https://blog.livekit.io/scaling-webrtc-with-distributed-mesh/ https://blog.livekit.io/scaling-webrtc-with-distributed-mesh...
- mannyv 2y ago
  Interesting that "but a particular session can only be hosted by one instance" Why is that? Is that a protocol issue with WebRTC or an implementation issue with the way WebRTC servers are written? There are lots of ways to share session state across servers.
  - Sean-Der 2y ago
    To build this quickly I would chain Broadcast Box instances. Have it look like a n-ary tree and spin up another broadcast box instance when you need it. `OBS -> Broadcast Box -> Broadcast Box -> Broadcast Box -> Viewer` To start I would do WHIP between the servers as well. More optimization can be done, but it would be a good quick start.

disqard 2y ago

I'm unaware of WebRTC being used for this sort of "multicast" context, especially with such a high "fanout" -- all I've used/built are p2p contexts like video/data. Not knowledgeable here, but also interested if someone actually knows the answer to your question...

gigachadbro 2y ago
I previously was involved in a service where WebRTC was used for a fairly large number of N streams to M viewers. Like ~75 streams total and 500+ viewers. Solution space was public sector incident command (think police officers with multiple cameras on their body streaming to command center).
rattt 2y ago
Microsoft's now shut down Mixer (formerly Beam.pro) primarly used WebRTC and had streams with up to 100k viewers at some point all via WebRTC, but can't find much info about it anymore now sadly.

gorbypark 2y ago

I think state of the art in this space is WHIP/WHEP, which is what this project is implementing. I'm guessing 100k+ clients would be pretty difficult for Broadcast Box (out of the box), however Cloudflare has a beta stream service that's has a tagline of "Sub-second latency live streaming (using WHIP) and playback (using WHEP) to unlimited concurrent viewers." https://developers.cloudflare.com/stream/webrtc-beta/ https://developers.cloudflare.com/stream/webrtc-beta/

Sean-Der 2y ago

Open Source doesn’t have an instant scalability offering yet. I hope someone takes it on! Quite a few companies offer this a service though. Cloudflare, Twitch’s IVS, Dolby Millicast, Phenix RTS… LiveKit did a write up on how they built theirs https://blog.livekit.io/scaling-webrtc-with-distributed-mesh/ https://blog.livekit.io/scaling-webrtc-with-distributed-mesh...

mannyv 2y ago
Interesting that "but a particular session can only be hosted by one instance" Why is that? Is that a protocol issue with WebRTC or an implementation issue with the way WebRTC servers are written? There are lots of ways to share session state across servers.
- Sean-Der 2y ago
  To build this quickly I would chain Broadcast Box instances. Have it look like a n-ary tree and spin up another broadcast box instance when you need it. `OBS -> Broadcast Box -> Broadcast Box -> Broadcast Box -> Viewer` To start I would do WHIP between the servers as well. More optimization can be done, but it would be a good quick start.