12 ms·

Welcome to Project Lightspeed. This is a project that allows anyone to easily deploy their own sub-second latency live-streaming server. In its current state yo

Welcome to Project Lightspeed. This is a project that allows anyone to easily deploy their own sub-second latency live-streaming server. In its current state you can stream from OBS [1] and watch that stream back from any desktop browser. This has been a super fun project which has taught me more than any other project I have done. It uses Rust, Go and React and can be deployed fairly easily on a very lightweight server. For example, I have been doing my test streams on a $5 Digital Ocean droplet and the CPU usage is at around 20%. Granted not a lot of people are watching however it is more lightweight than a solution such as Janus. The point of this project is twofold. First I wanted to learn more about WebRTC and real-time communication in general. Second, I wanted to provide a platform where people can setup their own little live-stream environment. Maybe you just want a place where you and some friends can hang out or you are sick of the main-stream platforms. Anyhow as of writing this post it is v0.1.0 and considered an MVP release. In the coming months I (and hopefully some of you :)) will be adding more features and trying to flesh this out into as much of a full featured platform as possible. Feel free to take a look at the repo and let me know what you all think :) [1] Open Broadcast Software (OBS): https://obsproject.com/ https://obsproject.com/

12things 6y ago
A very interesting project, can you elaborate more on how it is getting sub second latency and why youtube/twitch seem to have more than a few seconds of delay.
- GRVYDEV 6y ago
  YouTube and twitch use RTMP which operates over TCP. This means that each time we send a packet we need to ensure that it’s been received which adds latency overhead. Lightspeed uses the FTL protocol which operates over udp thus reducing the latency overhead
  - samstave 6y ago
    So assuming you have packet loss - you just get a paused/blank stream until the flow continues? How does it handle any network issues. Would it ever be possible to route two stream via different paths to the client and let the client just accept the first packet from either and drop the other in order to add some redundancy to delivery?
    - GRVYDEV 6y ago
      This wouldn’t actually solve anything since WebRTC can handle packet loss. The loss is going to be coming from OBS -> Server which unfortunately there isn’t much I can do about that since it uses UDP
      - samstave 6y ago
        Ah thanks. A diagram of the setup on your page would be helpful...
    - midasuni 6y ago
      Multiple streams is BAU with RTP - using smpte-2022-7, or most of the time just firing the packet different times on different routing tables. Sometimes network paths die. This could be a dodgy router in a third party network that drops streams for 150ms at a time, or a bgp recalculation that knocks it out for maybe a minute or so. In both cases you need to have multiple routes to keep your latency low.
- imtringued 6y ago
  HTTP video streams are split into segments and those segments are delivered whole. Larger segments are easier to cache and scale to bigger number of viewers. The other factor is that youtube and twitch spend more CPU time on compression to achieve lower bitrates.
simlevesque 6y ago
Could it be possible to use WebRTC to have listeners become seeders/repeaters so you could have more listener without impacting too much your own CPU ? Something like AceStream.
- GRVYDEV 6y ago
  Yup! You could develop a web of WebRTC relays
ghostie_plz 6y ago
Thanks for sharing! I was actually just looking for something like this earlier today, can't wait to try it out.
- GRVYDEV 6y ago
  I can’t wait for you to try it out haha!
rlyshw 6y ago
This project is pretty cool, I’ll tinker around with it tomorrow. As an aside, I’ve noticed you’re building out your own stream protocol stack (FTL/LightSpeed). What’s the reasoning there? Seems slightly inconvenient to have to “hack” OBS to make the output stream work. Will FTL support be merged into OBS in the future? If you’re just trying to avoid the latency of RTMP then I might suggest considering the existing SRT protocol[1]. It’s been open source for a while and is well-established(native support in OBS core and optional in FFMPEG). Seems to already solve a lot of the transport-level stuff that you’re working on with FTL. [1] Secure Reliable Transport: https://github.com/Haivision/srt https://github.com/Haivision/srt
- GRVYDEV 6y ago
  So FTL is supported by OBS and was used by Mixer. I’m interested to move to SRT in the future since you’re correct, FTL support will be going away eventually
- GRVYDEV 6y ago
  Also the work required to adapt what I have to SRT is non trivial and I would rather have something that works right now and then build in SRT support in the future
- GRVYDEV 6y ago
  Also, you’re not “hacking” obs. oBS is compiled with the FTL sdk you just have to tell it to use it.
  - rlyshw 6y ago
    Got it. I’m fresh on FTL, first I’ve really dug into it so apologies for the ignorance. I’m in the industry of stream transport and mostly work with SRT. Looking forward to tinkering with this!
    - GRVYDEV 6y ago
      No worries :) enjoy and please let me know what you find!
oblio 6y ago
Really cool! You say you use Rust and Go. Why both? I imagine Rust for super performance sensitive stuff and Go for the rest, for faster development?
- codetrotter 6y ago
  This was going to be my question too basically. Hoping to see OP answer this and in particular what I would like to see them comment on is how they divide their codebase between these two languages. Which parts are being implemented in which of those languages and why. Personally a huge proponent of Rust.
- rektide 6y ago
  I think it was a matter of what libraries were available where. Namely, Lightspeed-webrtc uses the extremely popular & robust Go library Pion[1] for webrtc. It's a little over 500 lines. The Rust Lightspeed-ingest[2] server is also ~500 lines of code, and primarily handshakes the FTL protocol used to communicate with OBS. There is a Pion port to rust[3] that is in progress. I am not sure the state of this work. Pion is used quite extensively by many many projects; I'm not sure if the rust webrtc-rs port has any notable users yet. As I began by saying, I expect the trustability & extensiveness of Pion is what lead to lightspeed-webrtc being written in Go. [1] https://pion.ly/ https://pion.ly/ [2] https://github.com/GRVYDEV/Lightspeed-ingest https://github.com/GRVYDEV/Lightspeed-ingest [3] https://github.com/webrtc-rs/webrtc https://github.com/webrtc-rs/webrtc
morsch 6y ago
Awesome! I was looking for something like this when trying to play a local multiplayer game via the Internet in an early lockdown. There are, or were, no good turnkey solutions for this. Twitch and Youtube have 5-10s latency, which is often not good enough. Mixer promised (and presumably delivered) ~1s latency using the FTL protocol you use, but they had a wait list of a couple of days or weeks, and of course now, they don't exist anymore. Even Steam Play Together, ostensibly built for this purpose, wasn't low latency enough in my limited experience (this really surprised me, so maybe I'm doing it wrong). The easiest solution, use the share desktop function of whatever video conference tool, almost works, but they universally seemed reduce the frame rate, which is ok for presentations but unsuitable for games (also, no audio). My solution was to output OBS to a virtual webcam device and use Jitsi Meet. A bit roundabout, but it worked wonderfully. Ideally, I'd forgo the DO droplet, and just run everything locally. 20% of a small droplet is even less of a modern desktop computer's CPU. Which leaves upload bandwidth for broadcast, which depends on your connection and how many people you need to be able to stream to.
- ossopite 6y ago
  I've played local multiplayer games over parsec (https://parsec.app/ https://parsec.app/), it works great and is pretty much turnkey
  - morsch 6y ago
    Yes, I forgot about Parsec, that's a good suggestion. I remember trying it, and not getting it to do what I want, unfortunately I don't remember why. I think I was stuck in the "Arcade", when all I wanted was to share my desktop or one window. It certainly looks like exactly what I was looking for.
    - Warchamp7 6y ago
      Arcade is like a quick match menu for finding a game to play. You can share any screen or desktop from the main app and have friends connect
  - 6y ago
justaj 6y ago
Thanks for making this project possible. Have you tried streaming via Tor? Is it possible to achieve streaming via Tor even if subsecond delay can't be achieved?
- Lucasoato 6y ago
  Wouldn't you have bandwidth problems by streaming via Tor?
  - justaj 6y ago
    I guess it depends on what you stream. It would have to be very low-bandwidth video or just audio.
dodgepong 6y ago
You should know the OBS team plans on deprecating FTL since Mixer was the only major player who ever used it, not to mention the fact that the server side of the tech is closed. The FTL implementation in OBS is buggy, and keeping it maintained is not worth the effort for a non-standard transport protocol.
- GRVYDEV 6y ago
  Yes I am aware however there is a new service called Glimesh that is utilizing FTL so I dont think it is going to disappear tomorrow. Also I have implemented the server side so it does not matter that it is closed.
  - dodgepong 6y ago
    Sure, but Glimesh are going to be looking at moving away from FTL themselves. See the discussion on this PR: https://github.com/obsproject/obs-studio/pull/3834 https://github.com/obsproject/obs-studio/pull/3834
    - GRVYDEV 6y ago
      Yup I have spoken to them about that as well however that move wont be for a while
      - dodgepong 6y ago
        Well, I think it's possible FTL will be gone from OBS by the end of 2021 so I guess they need to figure out what they are doing sooner rather than later. There will probably be a post about it on the OBS Github soon.
captn3m0 6y ago
Any way to push non-OBS content to Lightspeed? I've been trying to run a sub-second latency game-livestream (ala twitch plays), and I'd rather run it on a headless server without OBS.
- xd1936 6y ago
  It should work, as long as you're sending it to the server over FTL[1], which is a pretty new and uncommon protocol developed by the now-defunct Mixer. 1. https://github.com/mixer/ftl-sdk https://github.com/mixer/ftl-sdk
  - naikrovek 6y ago
    It's kind of amazing how active that GitHub org is given that Mixer is gone...
rektide 6y ago
The ingress components is interesting to me. It takes the OBS stream, via the FTL protocol, & converts it into something for WebRTC to use, yes? What drove you to use FTL protocol for ingestion? Did you consider alternatives like RTMP, which I believe OBS also supports? I hand't heard of FTL before. Apparently it was a protocol used in Microsoft's now-defunct game-streaming service, Mixer. I found some discussion of the various streaming protocols here[1], which included some description of FTL. I guess it comes down to latency. That would still have left SRT on the table, yes? Great project. Such a key area of connectivity for us all. So glad you did this. Thanks. [1] https://restream.io/blog/streaming-protocols/ https://restream.io/blog/streaming-protocols/
- GRVYDEV 6y ago
  I went with FTL instead of RTMP for the sake of latency. Also FTL gives me a stream of RTP packets which can go directly into WebRTC meaning I have to do 0 processing of the packets where as with RTMP I would have to convert them into RTP packets. Also SRT is interesting however it is wildly complicated and does not use RTP meaning I would have to figure out how it works and then convert whatever it gives me into RTP packets for WebRTC

12things 6y ago

A very interesting project, can you elaborate more on how it is getting sub second latency and why youtube/twitch seem to have more than a few seconds of delay.

GRVYDEV 6y ago
YouTube and twitch use RTMP which operates over TCP. This means that each time we send a packet we need to ensure that it’s been received which adds latency overhead. Lightspeed uses the FTL protocol which operates over udp thus reducing the latency overhead
- samstave 6y ago
  So assuming you have packet loss - you just get a paused/blank stream until the flow continues? How does it handle any network issues. Would it ever be possible to route two stream via different paths to the client and let the client just accept the first packet from either and drop the other in order to add some redundancy to delivery?
  - GRVYDEV 6y ago
    This wouldn’t actually solve anything since WebRTC can handle packet loss. The loss is going to be coming from OBS -> Server which unfortunately there isn’t much I can do about that since it uses UDP
    - samstave 6y ago
      Ah thanks. A diagram of the setup on your page would be helpful...
  - midasuni 6y ago
    Multiple streams is BAU with RTP - using smpte-2022-7, or most of the time just firing the packet different times on different routing tables. Sometimes network paths die. This could be a dodgy router in a third party network that drops streams for 150ms at a time, or a bgp recalculation that knocks it out for maybe a minute or so. In both cases you need to have multiple routes to keep your latency low.
imtringued 6y ago
HTTP video streams are split into segments and those segments are delivered whole. Larger segments are easier to cache and scale to bigger number of viewers. The other factor is that youtube and twitch spend more CPU time on compression to achieve lower bitrates.

simlevesque 6y ago

Could it be possible to use WebRTC to have listeners become seeders/repeaters so you could have more listener without impacting too much your own CPU ? Something like AceStream.

GRVYDEV 6y ago
Yup! You could develop a web of WebRTC relays

ghostie_plz 6y ago

Thanks for sharing! I was actually just looking for something like this earlier today, can't wait to try it out.

GRVYDEV 6y ago
I can’t wait for you to try it out haha!

rlyshw 6y ago

This project is pretty cool, I’ll tinker around with it tomorrow. As an aside, I’ve noticed you’re building out your own stream protocol stack (FTL/LightSpeed). What’s the reasoning there? Seems slightly inconvenient to have to “hack” OBS to make the output stream work. Will FTL support be merged into OBS in the future? If you’re just trying to avoid the latency of RTMP then I might suggest considering the existing SRT protocol[1]. It’s been open source for a while and is well-established(native support in OBS core and optional in FFMPEG). Seems to already solve a lot of the transport-level stuff that you’re working on with FTL. [1] Secure Reliable Transport: https://github.com/Haivision/srt https://github.com/Haivision/srt

GRVYDEV 6y ago
So FTL is supported by OBS and was used by Mixer. I’m interested to move to SRT in the future since you’re correct, FTL support will be going away eventually
GRVYDEV 6y ago
Also the work required to adapt what I have to SRT is non trivial and I would rather have something that works right now and then build in SRT support in the future
GRVYDEV 6y ago
Also, you’re not “hacking” obs. oBS is compiled with the FTL sdk you just have to tell it to use it.
- rlyshw 6y ago
  Got it. I’m fresh on FTL, first I’ve really dug into it so apologies for the ignorance. I’m in the industry of stream transport and mostly work with SRT. Looking forward to tinkering with this!
  - GRVYDEV 6y ago
    No worries :) enjoy and please let me know what you find!

oblio 6y ago

Really cool! You say you use Rust and Go. Why both? I imagine Rust for super performance sensitive stuff and Go for the rest, for faster development?

codetrotter 6y ago
This was going to be my question too basically. Hoping to see OP answer this and in particular what I would like to see them comment on is how they divide their codebase between these two languages. Which parts are being implemented in which of those languages and why. Personally a huge proponent of Rust.
rektide 6y ago
I think it was a matter of what libraries were available where. Namely, Lightspeed-webrtc uses the extremely popular & robust Go library Pion[1] for webrtc. It's a little over 500 lines. The Rust Lightspeed-ingest[2] server is also ~500 lines of code, and primarily handshakes the FTL protocol used to communicate with OBS. There is a Pion port to rust[3] that is in progress. I am not sure the state of this work. Pion is used quite extensively by many many projects; I'm not sure if the rust webrtc-rs port has any notable users yet. As I began by saying, I expect the trustability & extensiveness of Pion is what lead to lightspeed-webrtc being written in Go. [1] https://pion.ly/ https://pion.ly/ [2] https://github.com/GRVYDEV/Lightspeed-ingest https://github.com/GRVYDEV/Lightspeed-ingest [3] https://github.com/webrtc-rs/webrtc https://github.com/webrtc-rs/webrtc

morsch 6y ago

Awesome! I was looking for something like this when trying to play a local multiplayer game via the Internet in an early lockdown. There are, or were, no good turnkey solutions for this. Twitch and Youtube have 5-10s latency, which is often not good enough. Mixer promised (and presumably delivered) ~1s latency using the FTL protocol you use, but they had a wait list of a couple of days or weeks, and of course now, they don't exist anymore. Even Steam Play Together, ostensibly built for this purpose, wasn't low latency enough in my limited experience (this really surprised me, so maybe I'm doing it wrong). The easiest solution, use the share desktop function of whatever video conference tool, almost works, but they universally seemed reduce the frame rate, which is ok for presentations but unsuitable for games (also, no audio). My solution was to output OBS to a virtual webcam device and use Jitsi Meet. A bit roundabout, but it worked wonderfully. Ideally, I'd forgo the DO droplet, and just run everything locally. 20% of a small droplet is even less of a modern desktop computer's CPU. Which leaves upload bandwidth for broadcast, which depends on your connection and how many people you need to be able to stream to.

ossopite 6y ago
I've played local multiplayer games over parsec (https://parsec.app/ https://parsec.app/), it works great and is pretty much turnkey
- morsch 6y ago
  Yes, I forgot about Parsec, that's a good suggestion. I remember trying it, and not getting it to do what I want, unfortunately I don't remember why. I think I was stuck in the "Arcade", when all I wanted was to share my desktop or one window. It certainly looks like exactly what I was looking for.
  - Warchamp7 6y ago
    Arcade is like a quick match menu for finding a game to play. You can share any screen or desktop from the main app and have friends connect
- 6y ago

justaj 6y ago

Thanks for making this project possible. Have you tried streaming via Tor? Is it possible to achieve streaming via Tor even if subsecond delay can't be achieved?

Lucasoato 6y ago
Wouldn't you have bandwidth problems by streaming via Tor?
- justaj 6y ago
  I guess it depends on what you stream. It would have to be very low-bandwidth video or just audio.

dodgepong 6y ago

You should know the OBS team plans on deprecating FTL since Mixer was the only major player who ever used it, not to mention the fact that the server side of the tech is closed. The FTL implementation in OBS is buggy, and keeping it maintained is not worth the effort for a non-standard transport protocol.

GRVYDEV 6y ago
Yes I am aware however there is a new service called Glimesh that is utilizing FTL so I dont think it is going to disappear tomorrow. Also I have implemented the server side so it does not matter that it is closed.
- dodgepong 6y ago
  Sure, but Glimesh are going to be looking at moving away from FTL themselves. See the discussion on this PR: https://github.com/obsproject/obs-studio/pull/3834 https://github.com/obsproject/obs-studio/pull/3834
  - GRVYDEV 6y ago
    Yup I have spoken to them about that as well however that move wont be for a while
    - dodgepong 6y ago
      Well, I think it's possible FTL will be gone from OBS by the end of 2021 so I guess they need to figure out what they are doing sooner rather than later. There will probably be a post about it on the OBS Github soon.

captn3m0 6y ago

Any way to push non-OBS content to Lightspeed? I've been trying to run a sub-second latency game-livestream (ala twitch plays), and I'd rather run it on a headless server without OBS.

xd1936 6y ago
It should work, as long as you're sending it to the server over FTL[1], which is a pretty new and uncommon protocol developed by the now-defunct Mixer. 1. https://github.com/mixer/ftl-sdk https://github.com/mixer/ftl-sdk
- naikrovek 6y ago
  It's kind of amazing how active that GitHub org is given that Mixer is gone...

rektide 6y ago

The ingress components is interesting to me. It takes the OBS stream, via the FTL protocol, & converts it into something for WebRTC to use, yes? What drove you to use FTL protocol for ingestion? Did you consider alternatives like RTMP, which I believe OBS also supports? I hand't heard of FTL before. Apparently it was a protocol used in Microsoft's now-defunct game-streaming service, Mixer. I found some discussion of the various streaming protocols here[1], which included some description of FTL. I guess it comes down to latency. That would still have left SRT on the table, yes? Great project. Such a key area of connectivity for us all. So glad you did this. Thanks. [1] https://restream.io/blog/streaming-protocols/ https://restream.io/blog/streaming-protocols/

GRVYDEV 6y ago
I went with FTL instead of RTMP for the sake of latency. Also FTL gives me a stream of RTP packets which can go directly into WebRTC meaning I have to do 0 processing of the packets where as with RTMP I would have to convert them into RTP packets. Also SRT is interesting however it is wildly complicated and does not use RTP meaning I would have to figure out how it works and then convert whatever it gives me into RTP packets for WebRTC