9 ms·
Ultra Low Latency WebRTC Streaming – Open-Source Media Server
- kodablah 7y agoWhat makes this approach lower latency than, say, a centralized TURN server in one of those clouds?
- adreamingsoul 7y agoFrom my understanding, the current challenge with ultra-low-latency is scale. Will it scale to 100, 1000, 100,000, concurrent users? At the moment, everyone in the industry is working to figure out how to provide a viewer experience that is similar, or better than the current expectations of live streaming. Live streaming is hard. Ultra-low-latency is just as hard. However, I'm still not sure why we need ultra-low-latency. Will it improve the live experience? Maybe. Aside from live-action sports or significant events, do we as content consumers really need ultra-low-latency?
- Jhsto 7y ago> However, I'm still not sure why we need ultra-low-latency. Simply for passive viewing probably not. For live streaming in which the viewer is also using actuators, e.g., gamepads, wheel, etc., for controlling the stream capture device, then yes.
- alias_neo 7y agoYou've just done a stellar job of making "gaming" sound like a tedious day job. I'm impressed.
- Jhsto 7y agoYeah I tried too fancy words. What I meant to say that there might be applications beyond gaming, like driving a remote car, vacuum cleaner, or who knows what.
- alias_neo 7y agoYour choice of words were perfect, I just found it humorous, as a gamer, it was the first thing that came to mind!
- donalhunt 7y agoindustries serving the sports industry / creating products related to sports will be very interested... not your average consumer though...
- stcredzero 7y agoI'm interested in low latency WebRTC datachannel. Does this apply to that as well?
- pedrocr 7y agoHow low is ultra-low-latency that it's relevant for sports? Does anyone really care if their broadcast is 30 seconds delayed from live? I subscribe to both Formula 1 and MotoGP online streaming services and couldn't care less if the broadcasts were even multiple minutes delayed. They may very well be already and I wouldn't know. As far as I can tell they both just use standard CDNs delivering DASH or similar (basically just short video files played in sequence). That seems more than enough for sports. Multiple minutes may result in getting real time information from elsewhere (e.g., Twitter, local radio) but anywhere from 10 to 30 seconds seems fine.
- toast0 7y agoIf your friends are watching the same events in other places; you basically need your video to be less delayed than your friends' viewing plus the latency of them texting you (friend observes something, types something, sends, then whatever network and push delay on the way to you). If you're not watching with someone else, it doesn't make that much difference; but 30 seconds is plenty for someone to cheer for a goal and the notification to distract you from seeing it.
- pedrocr 7y agoThat's already an issue with radio vs TV which is where I'm getting the ~10 to 30 second value. Normal cable TV already has a bunch of delay and someone listening on the radio is often 15+ seconds ahead of you. This can be disruptive. If that person is at a bar or other gathering they will disrupt the viewing because they will cheer (or not cheer) well ahead of the event so the suspense is gone. I've seen this happen. So given that what you are competing against (cable TV) already has a significant delay I doubt a normal web stream can't reach the same or better latency with just standard HTTP download of video files.
- daniel-cussen 7y agoYes. 30 seconds is too much. In Santiago, during national soccer matches everyone would watch, goals would reach us through the sound of neighbors rejoicing sooner than they would from actual television coverage because the satellite dish had a latency of say 3 seconds over cable.
- harlanji 7y agoI did coding livestreams and spoke with the audience. The lower latency the better. Others have mentioned control of things, similar idea. HLS over Cloudflare surprisingly beat YouTube and matched Twitch in my side-by-side tests—I built an Open Source self-hosted live streaming system. Currently live sports have about 40-85 seconds of latency and BBC is shooting for 1.5 for the next olympics with ip multicast techniques [1]. 1. http://www.wired.co.uk/article/england-vs-croatia-live-stream-bbc-iplayer-itv http://www.wired.co.uk/article/england-vs-croatia-live-strea...
- Waterluvian 7y agoUltra low latency absolutely improves the streaming experience for community streams like on Twitch. When everyone is +-20 seconds of each other the chat doesn't line up well.
- AceJohnny2 7y agoImplying there's value to Twitch chat?
- deleted 7y ago[deleted]
- harshitaneja 7y agoGoogle Stadia(and other online game streaming) and other systems where actuators would be used remotely like drone control systems and others would greatly benefit from ultra low latency.
- deleted 7y ago[deleted]
- fabioyy 7y agohqtrivia
- maxmcd 7y agoI think even with turn there's lots of chatter back and forth. Connected clients can NACK for frames and ask for a bandwidth reduction. I think the expected optimal implementation is that the group of WebRTC participants normalize towards an optimal bitrate. For 1->n streaming you just want very high quality broadcast->server and then presumably you want the downstream clients to not have the option to send packets back to the broadcaster. Transcoding is hard though, so consumers might be stuck with just the source video quality. I assume this lib just passes the frames on from the source (twitch did just that through almost all of its pre-acquisition growth). Some of my details might be a little off, but I did spend a few months trying to build something very similar to this with gstreamer. WebRTC was hard to grok.
- mekya 7y agoActually it is not lower latency than centralized TURN server. The difference from TURN is that Ant Media Server behaves like peer to each connected clients. Not relaying the video/audio as TURN. Additionally, it can create adaptive bitrates on the server side. It can record streams and control other things on server side.
- fabioyy 7y agoThe problem is that webrtc is a very intensive CPU ( each connection have its own encription ). How many users can this solution handle per server?
- ec109685 7y agoHow is encryption expense different from normal https streaming from a cdn?
- est 7y agomost https connections are ephemeral and limited to connections per domain. WebRTC can spawn large number of p2p connections.
- fabioyy 7y agopacket size of webrtc(rtp) is VERY VERY VERY small, and since they are UDP, each individual packet must be encrypted. ( i can't just encode a big chunk of data and send to the network like tls ). in fact, for scaling a solution that uses "near realtime broadcast", you probably don't want encryption at all. ( afaik you can't disable webrtc encryption. ) in my tests i can get 3x more performance just using rtmp/rtsp. ( ~8000 video streaming connections on a aws c5.xlarge at 800kbit/s ).
- maxmcd 7y agoIt looks like they support RTMP and a few other streaming methods. Maybe it's just WebRTC for ingress and then it converts to other formats. Given the advertised 500ms of latency I imagine it's not WebRTC->WebRTC.
- MuffinFlavored 7y agoHow does one-to-many streaming work? If 256 clients connect to one streamer, how does the streamer not need to send his/her stream 256 times?
- deleted 7y ago[deleted]
- pault 7y agoThat's what the server is for.
- rkagerer 7y ago500ms is low latency? What's usual?
- naikrovek 7y agoMultiple seconds.
- BeefySwain 7y agoThe best Vimeo/Livestream can do is about 30 seconds from the time the frame hits their server, as a reference.
- rasz 7y agoTwitch random 1080@60 6K viewers stream >Latency To Broadcaster: 7.71 sec. >Latency Mode: Low Latency
- deleted 7y ago[deleted]
- isostatic 7y agoAs far as I'm concerned, 500ms is normal latency (once setting aside network delay -- clearly a 200ms one way trip from Europe to NZ will add to that) I'd say ultra low latency would be sub-frame, so <40ms. Low latency on the 1 frame to half a second, normal upto say 1.5s, and high latency beyond that. But that's for point to point or multicast circuits, and ISPs don't like multicast as they can't charge for it.
- dboreham 7y agoAnd here I was blissfully unaware that I'm experiencing reality 30 seconds delayed. Had no idea, other than obviously there's some # frames delay needed to do motion compression, and some transit latency.
- kwindla 7y agoThis looks to be nicely architected. Looking forward to digging in. In answer to the questions about TURN, this approach won't be lower latency than TURN but will scale better. TURN servers are just (nearly) passive relays, so the sending client needs to set up as many outbound streams as there are receiving clients in the session. The advantage to the TURN approach is that you can do end-to-end encryption. The server this post is about "forwards" streams. It's in a class of infrastructure traditionally called a "Selective Forwarding Unit." So the sending client can send one stream, and the SFU copies the stream and sends it to the receiving clients. The server needs more CPU and outgoing bandwidth as the receiver count grows, but the sending client doesn't need much of either. Once you have streams piping through a server, you can do lots of other things, of course. This server can transcode to several formats, including HLS and RTMP, and can copy streams internally across a cluster to scale more than a single machine can. Media servers are really fun to work on: lots of small(-ish) hard problems that touch low-level network protocols, memory and cpu optimization, architecture (because eventually you want a clean plugins interface, etc.). If you're interested in this stuff, check out Mediasoup, an open source WebRTC SFU with a very nice design in which all the low-level stuff is c++ and all the high-level interfaces are exposed as nodejs objects.[0] [0]-https://mediasoup.org/ https://mediasoup.org/
- sansnomme 7y agoHow's the performance and features of mediasoup compared to this? I have been looking for an easy to use DFU.
- kwindla 7y agoHaven't benchmarked ant media server, but mediaSoup performance is very good. "Easy to use" is in the eye of the beholder. Mediasoup isn't a packaged solution; you need to write some javascript to get a server up and running. From this perspective it's a framework that sits at the same level as, say, Express. But the API is flexible and powerful. It's exactly what you want if you're building an SFU cluster for a specific use case and want to decide on the architecture yourself. Here's excellent "1:N broadcast" sample code: [0] With that sample code, you can broadcast a single WebRTC stream to ~500 clients on one vCPU of an AWS c5.large instance. To scale to more clients you need to use more than one vCPU. The new mediasoup v3 APIs make this pretty easy. There are now "pipe transports" and "plain rtp transports" for moving the low-level rtp bits around between processes and between machines.[1] [0]-https://github.com/michaelfig/mediasoup-broadcast-example https://github.com/michaelfig/mediasoup-broadcast-example [1]-https://mediasoup.org/documentation/v3/differences-between-v2-and-v3/ https://mediasoup.org/documentation/v3/differences-between-v...
- gugagore 7y agoThere's also possible latency savings from having the video encoder be aware of network conditions on a per-frame basis: http://ex.camera http://ex.camera but I don't think it's found its way to the "mainstream".
- hexane360 7y agoThere's also this which tried to do something similar: https://snr.stanford.edu/salsify/ https://snr.stanford.edu/salsify/ Edit: It's the same author
- selim17 7y agoTest AMS Live Demo in https://antmedia.io/livedemo/ https://antmedia.io/livedemo/ AMS Github Pages: https://github.com/ant-media/Ant-Media-Server https://github.com/ant-media/Ant-Media-Server Thanks.
- rv11 7y agoIt will help apps which require interaction from users, like google stadia's game streaming. for other use cases, I think one does not care even if the feed is delayed by few seconds.
- IshKebab 7y agoAnother interesting use case someone mentioned here before is the world cup - you don't want a penalty shoot-out ruined by your neighbours cheering a few seconds before you see the goal. A few seconds would also such for live streaming where the streamer interacts in chat.
- deleted 7y ago[deleted]
- DiseasedBadger 7y agoI've heard enough of this anti-anonymity, anti-privacy protocol.
- mgamache 7y agoI've used Wowza (video streaming server) for years and this is a direct competitor. The pricing is a little higher for Wowza, but Wowza is a mature product with tons of options for web streaming. The weakness of Wowza has been its support for WebRTC. I am using it, but it's not easy to stream from RTSP/RTMP to WebRTC. It also doesn't scale out for WebRTC