12 ms·
Show HN: OBS Live-streaming with 120ms latency
- Anunayj 2y agoWhy is low latency livestream so hard, while at the same time Cloud Gaming Tech like Nvidia Gamestream and such can have such a flawless experience? I've used Moonlight + Nvidia Gamestream with ~40ms RTT and couldn't feel a difference in competitive shooters, so total latency must be pretty low. Does it have something to do with the bandwidth requirements? (1 stream v/s potentially hundreds)
- redox99 2y agoA lot of it is buffering to work around crappy connections. Cloud gaming requires low latency so buffering is kept to a minimum.
- user_7832 2y agoSlightly tangential, do you think Moonlight (with say Sunshine) is good enough for proper work? I've used a few "second screens" apps like spacedesk on my iPad but generally when the resolution is good enough for text, it's too laggy for scrolling (and vice-versa). (For more details, I'm planning to stream from an old laptop after turning it into a hackintosh. I'm hoping staying on the home network's going to help with latency.)
- aappleby 2y agoAbsolutely yes, especially if you use a GPU with a recent video encoder. I have all my PCs connected to each other via Moonlight+Sunshine and the latency on the local network is unnoticeable. I code on my Linux workstation from my laptop, play games on the gaming PC from my workstation, etcetera and it is all basically perfect.
- user_7832 2y agoThank you! When you say GPU with a recent video encoder, do you mean them separately (i.e. don't use software/cpu streaming; and use an efficient encoder), or do you mean use a GPU that supports recent encoders? I'm afraid my Intel HD 520 isn't particularly new and likely doesn't support modern encoders.
- aappleby 2y agoA GPU with a dedicated encoder for x264 or preferably x265/AV1. You can do without it, but you'll spend a core or two on software encoding and the overhead will add a few tens of ms of lag. With full hardware capture and encode (default on windows, can require tweaking on Linux ) it's virtually free resource-wise.
- ThatPlayer 2y agoYou're off by one letter: the codec is h264/h265. x264 and x265 are the CPU encoding softwares for the codec.
- tymscar 2y agoWithout a doubt! I use moonlight for ten+ hours a week. I never use it for gaming and it never failed once.
- wmf 2y agoCloud gaming is streaming from a server in a data center to one nearby client. Twitch-style live streaming is from a client, to a data center, to a CDN, to multiple clients.
- spongebobstoes 2y agoTLDR, there are a lot of moving pieces, but people are working on it at the moment. I try to summarize below what some of the challenges are Bandwidth requirements are a big one. For broadcasts you want your assets to be cacheable in CDN and on device, and without custom edge + client code + custom media package, that means traditional urls which each contain a short (eg 2s) mp4 segment of the stream. The container format used is typically mp4, and you cannot write the mp4 metadata without knowing the size of each frame, which you don't know until encoding finishes. Let's call this "segment packaging latency". To avoid this, it's necessary to use (typically invent) a new protocol other than DASH/HLS + mp4. Also need cache logic on the CDN to handle this new format. For smooth playback without interruptions, devices want to buffer as much as possible, especially for unreliable connections. Let's call this "playback buffer latency". Playback buffer latency can be minimized by writing a custom playback client, it's just a lot of work. Then there is the ABR part, where there is a manifest being fetches that contains a list of all available bitrates. This needs to be updated, devices need to fetch it and then fetch the next content. Let's call this "manifest rtt latency". Lastly (?) there is the latency from video encoding itself. For the most efficient encoding / highest quality, B-frames should be used. But those are "lookahead" frames, and a typical 3 frame lookahead already adds ~50 ms at 60fps. Not to mention the milliseconds spent doing the encoding calculations themselves. Big players are rewriting large parts of the stack to have lower latency, including inventing new protocols other than DASH/HLS for streaming, to avoid the manifest RTT latency hit.
- mannyv 2y agoFor HLS you can use mpeg ts, but mp4 is also an option (with the problem you talk about). IMO one of the issues is that transcoding to lower resolutions usually happens on the server side. That takes time. If the client transcoded that latency would go away (mostly).
- est 2y agoGPU would do transcoding, build network packet and copy data via PCI-E, all using hardware, avoid memory copy. OBS+WebRTC is mostly software doing heavy-lifting. Imagine if the camera would build WebRTC UDP packets directly and zero-copy to NIC, that would lower latency quite a bit.
- KeplerBoy 2y agoI wouldn't be surprised to learn that Nvidia is doing exactly that on their cloud: Compressing the video on the GPU using NVENC, building a package around it and then passing it to a NIC under the same PCIe switch (mellanox used to call that peerdirect) and sending it on its way. The tech is all there, it just requires some arcane knowledge.
- imtringued 2y ago"arcane knowledge" is too strong of a phrase. You need someone who is familiar with Nvidia hardware and is willing to write software that only works on Nvidia hardware.
- KeplerBoy 2y agoIt is arcane as in information how all of this works on their specific hardware is not publicly available, but probably widespread within.
- kierank 2y agoThis is premature optimisation. The bus bandwidth and latency needed to get a few Mbps of compressed video to the PC is microscopic. It's completely unnecessary to lock yourself into NVIDIA just to create some UDP packets.
- KeplerBoy 2y agoI was talking about Nvidia's Cloud gaming offer (GeForce Now). For them it's certainly not a premature optimization.
- numpad0 2y agoBecause there are so many middlemen in series buffering frames, and also because access circuit between user terminal to nearest CDN is jittery too. The latency must be few times over max jitter for a par course user experience.
- mavamaarten 2y agoIn my head the answer is simple, moonlight is a one-to-one stream, while broadcasting to many clients at once is a whole different setup.
- Hikikomori 2y agoThere's no way that people cannot tell the difference, I can with various streaming methods from my pc to my shield/tv, with wire in the same house. Mouse to photon latency of a good pc will be in the range of 10-20ms, best case you're doubling or tripling that.
- booi 2y agoI can feel 40ms for sure and there’s no way you play a competitive shooter with a 40ms delay. Hell even a Bluetooth mouse gets annoying. Maybe if you’re playing Microsoft Office it’s ok.
- Guillaume86 2y agoNah to be fair it's fine for a lot of games which are also played on old gen consoles with terrible gamepad to TV latency. Sure twitchy multiplayers are definitely not some of them. I'm not big on competitive multiplayer, only Rocket League and I can't do this over local streaming. Pretty much anything else I play is ok though.
- ravenhappy 2y agoYou, my dear Internet friend, are confidently expressing your lack of experience. No one who has played multiplayer LAN games, or low latency Internet games, could or would ever say that streaming gaming, such as the dead stadia, or moonlight, whatever, are comparable to the alternative, Nah, they couldn't.
- the-smug-one 2y agoI don't think that I could feel the difference between 40ms and 10ms RTT when playing something like DOTA2 or AoE2.
- Hikikomori 2y agoMost online games use client side prediction, so any input made by the client happens almost instantly on the client and it feels really good, and can be rollbacked if the server disagrees. If you stream your game remote with 40ms it will add 40ms to your input and that just feels bad (not to mention jitter, especially if you're on semi-congested wifi), but its not unplayable or even that noticeable in many games. Would I play some casual Dota like that? Sure. But not high ranked games.
- deleted 2y ago[deleted]
- filleokus 2y agoWhat's the state of the art in distributing WebRTC to 100k+ clients? When I was more into the low latency streaming space a few years ago, it felt like WebRTC was there when it came to << 1 second latency, but the infrastructure to actually distribute it was not really. I think Cloudflare (and maybe some other vendors) were working on creating some standard, has it landed? Can I run my own horizontally scalable WebRTC broadcaster (are there open source implementations)? Something like Low-Latency HLS or CMAF was at like < 5 second latency, but was on the other hand stupidly easy to distribute widely (just static files on a plain old CDN / http server).
- disqard 2y agoI'm unaware of WebRTC being used for this sort of "multicast" context, especially with such a high "fanout" -- all I've used/built are p2p contexts like video/data. Not knowledgeable here, but also interested if someone actually knows the answer to your question...
- gigachadbro 2y agoI previously was involved in a service where WebRTC was used for a fairly large number of N streams to M viewers. Like ~75 streams total and 500+ viewers. Solution space was public sector incident command (think police officers with multiple cameras on their body streaming to command center).
- rattt 2y agoMicrosoft's now shut down Mixer (formerly Beam.pro) primarly used WebRTC and had streams with up to 100k viewers at some point all via WebRTC, but can't find much info about it anymore now sadly.
- gorbypark 2y agoI think state of the art in this space is WHIP/WHEP, which is what this project is implementing. I'm guessing 100k+ clients would be pretty difficult for Broadcast Box (out of the box), however Cloudflare has a beta stream service that's has a tagline of "Sub-second latency live streaming (using WHIP) and playback (using WHEP) to unlimited concurrent viewers." https://developers.cloudflare.com/stream/webrtc-beta/ https://developers.cloudflare.com/stream/webrtc-beta/
- kragen 2y agoyou know, almost every time i try to talk with my family on jitsi, there's some kind of glitch. they can't see my screen, or i can't see theirs, or they can see it but only in super low resolution, or they have the camera turned on but i can't see it, or we all get kicked off, or something. can broadcast box allow us to use obs studio (or some other free, open-source software) to stream to each other, without relying on a proprietary server? i don't need 100k+ clients, i'd be satisfied with reliable connectivity between 2–4 clients! and i could run a server outside of nat i'm not going to get 120ms latency though. i'm in argentina, they're mostly in the usa, and i have 200+ milliseconds of latency over the internet to anything in the usa if broadcast box isn't what i'm looking for, is there something else? i already know about zoom, google, and teams, but those all make us vulnerable to proprietary servers
- gorbypark 2y agoHaving discovered this 15 minutes ago I can't really speak in absolutes, but firing up OBS and using their test server, I think this would allow you to do exactly what you want. It's trivial to get OBS set up to stream to this, and while I haven't tried it, the docs say that it supports multiple streams to the same stream key. I guess it would just show each stream side by side in the client? I think you could run your own server, run an instance of the front end and distribute the instructions to setup OBS to your family.
- bubblesnort 2y agoJitsi is actually one of the better options. My only beef is that it got bloated with features I don't need.
- kragen 2y agoi suspect that the problems with jitsi are maybe problems with webrtc in general, because i've experienced very similar things on the proprietary services. my limited experience with obs studio, though, has been flawless, and it seems to be the main daily driver for lots of people who make their living streaming?
- taskforcegemini 2y ago
- Laesx 2y agoI've been using OvenMediaEngine for years which seems to be the same thing as this, but this looks pretty promising I'll give it a shot
- AhilHack1500yt 2y ago[flagged]
- chgs 2y agohttps://www.obe.tv/how-to-lie-about-latency/ https://www.obe.tv/how-to-lie-about-latency/
- imtringued 2y agoMy ancient latency claims of at most ~200ms (in practice ~100ms) are based around this test: https://humanbenchmark.com/tests/reactiontime https://humanbenchmark.com/tests/reactiontime Yes. Me clicking the mouse, the software transmitting the mouse click to a datacenter, software rendering the web browser using AVX2, software encoding the stream, sending it to the local browser and decoding it on the screen, shining photons in my eyes, me clicking the button a second time (which also needs to be transmitted to the datacenter), gets me around ~400ms over WebRTC on the reaction time benchmark vs ~200ms on the local computer. I'm not even trying. It's a janky as hell solution that is about to fall apart the moment you look at it funny. Also, I hate ffmpeg for streams that last longer than a day. The latency creep of streams lasting weeks is horrible.
- ta1243 2y agoI've had streams lasting years without any latency creep using professional encoders
- half-kh-hacker 2y ago> We've shown that many measurements of latency [...] ignore the full capture and playback pipeline In the repo linked in OP is a screenshot showing a wall clock next to its playback on the streaming site -- that's end-to-end to me. So how is this relevant?
- kierank 2y agoBecause latency is a distribution and these photos are often selected at the best-case P0 end of all the encode/decode processes whereas actually what matters is the worst case P99. A proper implementation will make sure the worst-case latency is accounted for and not cherry-pick the best case.
- 2y ago
- piyushtechsavy 2y agoBroadcasting over WebRTC can be if the source is single. But in case of a scenario of multiple source, like take a conference of 100 people as example, will it be smooth?
- eqvinox 2y ago> Sub-second Latency > Broadcast Box uses WebRTC for broadcast and playback. By using WebRTC instead of RTMP and HLS you get the fastest experience possible. Nothing in RTMP prevents you from achieving low latency; it's the software stack around it that determines latency for both RTMP and WebRTC. Only HLS does have some built-in deficiencies that cause extra latency.
- imtringued 2y agoWebRTC is basically just signalling around RTP in a browser compatible manner. The problem with RTMP is that nobody is using flash based video players anymore so "RTMP and HLS" as a bundle is non-negotiable in web browsers.
- Sean-Der 2y agoI agree, if you are in a perfect network! If you have any packet loss, changes in throughput, or roaming it breaks down. RTMP for me is preventing that next generation of use cases.
- thih9 2y agoAssuming I run this on a vps, what would be the bandwidth cost?
- cheschire 2y agoEight. Could you add more to your question? Where did you already do research? What providers are you considering? How much bandwidth is “free” with the tier of VPS you’re paying for? Etc This is a question that sales reps spend lots of work hours trying to help clients work out.
- blitzar 2y agoClearly the answer is 43.
- deleted 2y ago[deleted]
- Sean-Der 2y agoI created Broadcast Box originally as a reference server to test OBS against. It was way easier for people to test my WebRTC/WHIP PRs against. Seeing people use it I am seeing the benefits/excitement more. * low latency means you have a relationship with your audience. These intimate broadcasts are a new medium. * Simulcast means it is way cheaper to run a streaming site. No more running ffmpeg/generating transcodes server side. * AV1/H265/Opus means users with lower bandwidth can now broadcast. Users with enough bandwidth can stream at quality levels they couldn’t before * UDP gives us IRL/Roaming streams. No custom setup for re-connects. * Multi-track lets you send multiple video feeds or languages at once * E2E Encryption means that P2P distribution could be a thing
- dangoodmanUT 2y agoJust pouring through the code, one thing I think it's missing is transcoding to multiple (lower) resolutions as well so weaker connections can watch. Is that correct? I took a brief look a the WHIP protocol and it seems like maybe it's just a matter of converting the frames before writing?
- mannyv 2y agoI believe that's handled on the OBS side.
- Sean-Der 2y agoThis is all done client side. OBS sends up multiple renditions, the server is just in charge of forwarding the specific layer. I think this is better in a few ways. * Lower latency - The extra decode + encode adds enough latency that you start to lose the real-time latency. * Better Quality - You get generational loss from the transcoding. You get better quality only encoding one times. Streaming services also optimize for cost. Having broadcasters control the encoding quality of everything makes for a better experience. * Security/Trust - Servers shouldn't be able to modify video at all. I would like for broadcast services to eventually offer E2E encryption. It feels wrong to me that a streaming service is able to modify video however it pleases.
- nubinetwork 2y agoSo can I use this to record multiple streamers, add commentary and tracker overlays, to stream it back to twitch? I've been wanting to make something like this for a while...
- svpcom 2y agohttps://github.com/svpcom/wfb-ng https://github.com/svpcom/wfb-ng (and cheap CCTV cameras with OpenIPC firmware) allow to broadcast video with < 50ms glass-to-glass latency. https://www.youtube.com/watch?v=4tlPJSQA6HA https://www.youtube.com/watch?v=4tlPJSQA6HA
- mckirk 2y agoIf this does what it says it does I'll be a very happy user. Playing RPGs 'together' with somebody over the internet isn't that much fun if they are a second or more behind in what's going on. I've actually looked for a solution to this problem (low-latency P2P streaming) quite some time ago, and couldn't get it to work with just OBS because of strange bugs and other issues, so I really appreciate you including this use-case :)
- mrguyorama 2y agoSteam has this functionality built in, including streaming keyboard and mouse and controllers over the open internet. It's called "remote play together"
- Sean-Der 2y agoI am excited for you to use it! If you run into bugs or have any cool ideas join the discord and would love to chat https://discord.gg/An5jjhNUE3 https://discord.gg/An5jjhNUE3
- monocularvision 2y agoI see the following statement in the README: “You could also use P2P to pull other broadcasters into your stream. No special configuration or servers required anymore to get sub-second co-streams.” I currently have this setup for doing a co-stream with a friend and it is terrible: 1. Friend is running OBS to capture his gameplay. 2. Friend has OBS streaming to a Raspberry Pi I have running at my house. 3. The Raspberry Pi is running nginx configured to accept the RTMP stream. 4. I run OBS on another machine to capture my gameplay, add overlays, etc. 5. My OBS has an input source using VLC to capture the stream from the Raspberry Pi. The setup is awful. Video is pretty delayed and it often just stops working. I would love to look into this project but after reading through the README, I am unclear how I would use this for my setup. Any pointers?
- Sean-Der 2y agoI would have your friend stream to Broadcast Box. You can run it on the Raspberry Pi! Then I would pull that source into your OBS. You have two options * Pull Broadcast Box as a browser source * Use my PR that adds WHEP Sources https://github.com/obsproject/obs-studio/pull/10353 https://github.com/obsproject/obs-studio/pull/10353 For now I would do Browser Source (is easier/no custom builds). In the future when WHEP is merged that is the way to go. If you get stuck jump in the Discord and happy to help debug! https://discord.gg/An5jjhNUE3 https://discord.gg/An5jjhNUE3
- monocularvision 2y agoSounds great! We will give that a try. Already hopped on the Discord, I will let you know how it goes.
- ranger_danger 2y agoIMO This has the same problems inherent to WebRTC itself, which is that a large percentage of users in the world are behind symmetric or CG NAT and so TURN relay servers are needed, but never actually defined by the application, because there are no good free ones. Personally I've never been able to use any WebRTC-enabled service for this reason, nor anyone else I know.
- Sean-Der 2y agoMy public instance runs on a world addressable host and doesn't have TURN. I have no problem using it from my phone and it is behind a CG NAT. No TURN/Relay servers are needed. > I've never been able to use any WebRTC-enabled service for this reason, nor anyone else I know Is this a browser/agent issue? You aren't able to use Google Meet or Zoom in the browser?
- ranger_danger 2y agoLive-streaming in one direction doesn't need a relay because the users aren't uploading... And google/zoom use their own relays.
- Sean-Der 2y agoGoogle doesn't use any STUN/TURN servers I just went and confirmed. The following is the RTCPeerConfiguration used. ``` https://meet.google.com/***-***-\*\ https://meet.google.com/***-***-\*\*, { iceServers: [], iceTransportPolicy: all, bundlePolicy: max-bundle, rtcpMuxPolicy: require, iceCandidatePoolSize: 0 }, ```
- ranger_danger 2y agoIt may not use it if you don't explicitly need it, there could be out-of-band detection that adds a TURN server or some other OOB relay if it's needed. I've also seen comments online that say Chrome in particular can support TCP for WebRTC which would negate the need for a relay (as normally only DTLS over UDP is used). But based on my understanding of how WebRTC and NAT works, using the typical UDP approach for bidirectional communication over symmetric or CGNAT absolutely should not work, barring some other method of NAT traversal such as a browser-based UPnP client or such.
- iod 2y agoPeople interested in this project might also be interested in Cloudflare's webrtc streaming service¹ as a cloud hosted solution to this same problem. "Sub-second latency live streaming (using WHIP) and playback (using WHEP) to unlimited concurrent viewers." Using the same OBS WHIP plugin, you can just point to Cloudflare instead. Their target pricing model is $1 per 1000 minutes.² Which equates to $0.06 per hour streamed. ¹ https://developers.cloudflare.com/stream/webrtc-beta https://developers.cloudflare.com/stream/webrtc-beta ² https://blog.cloudflare.com/webrtc-whip-whep-cloudflare-stream https://blog.cloudflare.com/webrtc-whip-whep-cloudflare-stre...
- gorbypark 2y agoThey mention "$1 per 1000 minutes of video delivered", does that mean per viewer? $0.06 per hour per viewer seems like a lot, although I have no idea if that's "competitive" or inline with actual bandwidth costs.
- iod 2y agoAs per the direct quote in my parent comment, by "unlimited concurrent viewers", I can only assume that they mean what it says as taken by the text from the links of both ¹⁺²
- freedomben 2y agoMuch more of a meta question, but why is live-streaming preferred by some people? Live streaming just seems to have so many downsides to me: 1. Requires real-time presence 2. No editing (meaning less efficient use of time for the viewer) 3. No client-side speeding up/skipping irrelevant parts 4. No possibility of index or table of contents What are some use cases where live streaming is better?
- ratedgene 2y agoLive interaction with a community is pretty big.
- deleted 2y ago[deleted]
- femtozer 2y agoI see one significant upside: combined with a chat, you can ask the content creator questions directly
- fwip 2y agoInteraction with viewers is the primary one. There is also the "event" nature of streaming, where the audience is excited to react to things together in real-time, like when people Tweet when watching the Superbowl or a show's season finale. Even for mundane streams, the "chat" will happily talk amongst themselves reacting to whatever the streamer is doing, which can feel like hanging out with friends. There's also times where the streamer is discussing something that is happening now or just happening (world events, the Superbowl, a game update, etc), where viewers are excited about the content right now and don't want to wait for the traditional record->edit->release cycle.
- freedomben 2y agoAh thank you, this is the missing piece! That makes a lot of sense.
- soulofmischief 2y agoSome viewers prefer livestreaming, and we see streamers catering to those audiences. It depends on the streamer, but often times there is much stronger sense of community when there is a live, topic-oriented chat, especially when the streamer engages with the chat. This is not something you can satisfactorily replicate with prerecorded streams. Streaming is also much more low-cost to produce, editing can often represent an unwanted source of complexity and loss of creative control.
- gkhartman 2y agoAs someone unfamiliar with video broadcasting latencies, how does this compare to alternatives? Also, what are the hardware specs used to achieve the 120ms measurement?
- Sean-Der 2y agoThe alternative today is ~few seconds. This is from my T420 Thinkpad on Linux. The only significant input I can think is my RTT to the server. ``` Reply from 165.227.221.230: bytes=32 time=37ms TTL=49 Reply from 165.227.221.230: bytes=32 time=37ms TTL=49 ```
- jjbinx007 2y agoVdo.ninja is also webrtc and low latency but I'm not sure of exact numbers.
- Sean-Der 2y agovdo.ninja is a fantastic project, can’t recommend it enough. It made the dream of ‘co-streaming’ available to home users and not just those with hardware/pro setups.
- Terretta 2y agoSome firms were broadcasting live feeds with sub-second (~200ms) latencies to broad audiences not long after the dot com bust. There've always been (a majority of) broadcasters that had seconds to half minute (!) delays. Very few understood, or even today understand, how to tune every single touchpoint (including what's safe to do, and what one must not do.) Having an ultra low latency “video toaster” / broadcast mixer is a critical piece for sure. For us the motivations to work on this started with "backstage feeds" synced with live TV that needed to look simultaneous to the home cable viewer (think World Wrestling Entertainment) as well as Wall Street calls that needed to be (perceptually) in sync with dial-ins. And of course, Times Square NYE ball drops! In reality, almost nothing matters that way. For vast majority of content, the viewer is consuming only your stream, without some more real time channel at the same time. In most cases, the viewer cares far more about visual quality than the delay.