5 ms·

I wrote this to solve a few things I cared about. * I want to show people that native WebRTC players can be a thing. I hope this encourages hangouts/discord/$x

I wrote this to solve a few things I cared about. * I want to show people that native WebRTC players can be a thing. I hope this encourages hangouts/discord/$x to implement WHIP and WHEP it would let people do so much more * I wanted to make low latency sharing easier. I saw the need for this working on adding WebRTC to OBS and Broadcast Box[0] * I wanted to show devs what a great ecosystem exists for WebRTC. Lots of great implementations in different languages. * Was a bit of a ‘frustration project’. I saw a company claiming only their proprietary protocol can do latency this low. So I thought ‘screw you I will make an open source version!’ [0] https://github.com/glimesh/broadcast-box https://github.com/glimesh/broadcast-box

Sean-Der 2y ago
Another things that is in this realm. I am adding native co-streaming/conferencing to OBS [0]. OBS can send WebRTC, next I want to make receiving work well. Between that and Simulcast I hope to make real-time video dramatically cheaper and easier [0] https://docs.google.com/document/d/1Ed2Evze1ZJHY-1f4tYzqNZfx2NIVD2HMLbyVScsTxnM/edit https://docs.google.com/document/d/1Ed2Evze1ZJHY-1f4tYzqNZfx...
- ta988 2y ago
  This would be fabulous, thank you so much for working on that. What kind of latency does dual encoding (on client then on receiver again) adds? Are there codecs that can have multiple streams on the same image (as in zones of independent streams on the video surface)?
  - Sean-Der 2y ago
    It definitely adds latency, not enough to be a bad experience. We have vdo.ninja today and Twitch's Stream Together. Those both do the 'dual encoding' and it is a good enough experience that users are doing it!
- xeonmc 2y ago
  Is it possible to extend OBS into a p2p video chat client?
  - Sean-Der 2y ago
    I don't believe it is possible today, but I am working on it! It would be great for users trying to co-stream. I created OBS2Browser[0] that is a good first step in the direction. [0] https://github.com/Sean-Der/OBS2Browser https://github.com/Sean-Der/OBS2Browser
tamimio 2y ago
> I saw a company claiming only their proprietary protocol Did the company have a “ripple” in its name? Curious
- Sean-Der 2y ago
  Let me find it again! I saw it on LinkedIn and it was such a bullshit promo thing
  - tamimio 2y ago
    I remember around two years ago, we got in touch with a company—without mentioning the name but it has "ripple" in it—and after an hour-long seminar, NDA, password-protected binaries, and other BS, they barely delivered ~150ms latency..
slashink 2y ago
Hey Sean, we both worked at Twitch Video but I left just as you were joining. I currently work on the Discord video stack and am somewhat curious about how you imagine Discord leveraging WHIP/WHEP. Do you see it as a way for these clients to broadcast outwards to services like Twitch or more as an interoperability tool?
- Sean-Der 2y ago
  Users what to send WHIP into discord. The lack of control on screen sharing today is frustrating. Users want to capture via another tool and control bitrate/resolution. Most Broadcast Box users tell me that’s their reason for switching off discord. ——— With WHEP I want to see easier co-streaming. I should be able to connect a room to my OBS instance and everyone’s video auto show up. I don’t have this figured out yet. Would love your opinion and feedback. Wanna comment on the doc or would love to talk 1:1 ! siobud.com/meeting
- eggsome 2y ago
  What's the plan for Wayland compatibility? For a little while I was able to share a single app - but not the full desktop. Now I can't share anything from Ubuntu 24.04 when using Wayland :(
  - palata 2y ago
    I had to tweak a few things to make it work on my Alpine Linux. This tutorial helped a lot: https://github.com/emersion/xdg-desktop-portal-wlr/wiki/%22It-doesn't-work%22-Troubleshooting-Checklist https://github.com/emersion/xdg-desktop-portal-wlr/wiki/%22I...
makapuf 2y ago
Great things are accomplished by spite programming! https://hackaday.com/2018/01/03/spite-thrift-and-the-virtues-of-an-affordable-logic-analyzer/ https://hackaday.com/2018/01/03/spite-thrift-and-the-virtues...
Karrot_Kream 2y ago
Thanks the code is really useful to read through.
mcwiggin2 2y ago
This is awesome. I would love if you had some examples on how to use AntMedia as a source. I am mostly in video engineering so the read the source comes slower to me. This would be really handy in many cases.
mwcampbell 2y ago
Is the restriction to NVIDIA necessary for the low latency?
- Sean-Der 2y ago
  Nope! I want to add all the other flows. nvidia is the absolute lowest I believe. I wanted to do it first to know if it was worth building.
namibj 2y ago
Any plans on integrating L4S with e.g. Tetrys-based FEC and using a way where the congestion feedback from L4S acts on the quantizer/rate-factor instead of directly on bitrate? It's much more appropriate to do perceptual fairness than strict bitrate fairness. Happy to have a chat on this btw; you can best catch me on discord.
- stefan_ 2y ago
  Depends on the network, surely? Lots of applications for low latency video where you are not sharing the channel, but it has a fixed bandwidth.
  - namibj 2y ago
    E.g. "Low Latency DOCSIS"[0] and related, WiFi[1], support it and with the former it's about non-exclusive scarce uplink capacity where cross-customer capacity sharing may rely on post-hoc analysis of flow behavior to check for abuse, switching to forced fairness if caught by such heuristics. For downstream it's even more natural to have shared capacity with enough congestion to matter, but often only the WiFi side would have a large discretionary range for bandwidth scheduling/allocation to matter much. Apple already opportunistically uses L4S with TCP-Prague and there are real-world deployments/experiments [2] with end-to-end L4S. Fixed- [0]: https://github.com/cablelabs/lld https://github.com/cablelabs/lld [1] relevant excerpt from [0]: Applications that send large volumes of traffic that need low latency, but that are responsive to congestion in the network. These applications can benefit from using a technology known as "Low Latency, Low Loss, Scalable Throughput (L4S)". Support for this technology is including in the LLD feature set, but is beyond the scope of what we have in this repository. Information on L4S can be found in this IETF draft architecture. [2]https://www.vodafone.com/news/technology/no-lag-gaming-vodafone-and-nokia-test-new-technology-to-eliminate-gaming-and-video-lag https://www.vodafone.com/news/technology/no-lag-gaming-vodaf...
cchance 2y ago
Question, 30ms latency sounds amazing but how does it actually compare to "the standard" sharing tools for desktops, like do you know what the latency on say MSRDP is as comparison or VNC?
- Sean-Der 2y ago
  I doubt the protocol itself makes a big difference. I bet you can get 30ms with VNC. The difference with BitWHIP. * Can play WebRTC in browser. That makes things easier to use. * simpler/hackable software. BitWHIP is simple and uses nvenc etc… if you use nvenc with VNC I bet you can get the same experience

Sean-Der 2y ago

Another things that is in this realm. I am adding native co-streaming/conferencing to OBS [0]. OBS can send WebRTC, next I want to make receiving work well. Between that and Simulcast I hope to make real-time video dramatically cheaper and easier [0] https://docs.google.com/document/d/1Ed2Evze1ZJHY-1f4tYzqNZfx2NIVD2HMLbyVScsTxnM/edit https://docs.google.com/document/d/1Ed2Evze1ZJHY-1f4tYzqNZfx...

ta988 2y ago
This would be fabulous, thank you so much for working on that. What kind of latency does dual encoding (on client then on receiver again) adds? Are there codecs that can have multiple streams on the same image (as in zones of independent streams on the video surface)?
- Sean-Der 2y ago
  It definitely adds latency, not enough to be a bad experience. We have vdo.ninja today and Twitch's Stream Together. Those both do the 'dual encoding' and it is a good enough experience that users are doing it!
xeonmc 2y ago
Is it possible to extend OBS into a p2p video chat client?
- Sean-Der 2y ago
  I don't believe it is possible today, but I am working on it! It would be great for users trying to co-stream. I created OBS2Browser[0] that is a good first step in the direction. [0] https://github.com/Sean-Der/OBS2Browser https://github.com/Sean-Der/OBS2Browser

tamimio 2y ago

> I saw a company claiming only their proprietary protocol Did the company have a “ripple” in its name? Curious

Sean-Der 2y ago
Let me find it again! I saw it on LinkedIn and it was such a bullshit promo thing
- tamimio 2y ago
  I remember around two years ago, we got in touch with a company—without mentioning the name but it has "ripple" in it—and after an hour-long seminar, NDA, password-protected binaries, and other BS, they barely delivered ~150ms latency..

slashink 2y ago

Hey Sean, we both worked at Twitch Video but I left just as you were joining. I currently work on the Discord video stack and am somewhat curious about how you imagine Discord leveraging WHIP/WHEP. Do you see it as a way for these clients to broadcast outwards to services like Twitch or more as an interoperability tool?

Sean-Der 2y ago
Users what to send WHIP into discord. The lack of control on screen sharing today is frustrating. Users want to capture via another tool and control bitrate/resolution. Most Broadcast Box users tell me that’s their reason for switching off discord. ——— With WHEP I want to see easier co-streaming. I should be able to connect a room to my OBS instance and everyone’s video auto show up. I don’t have this figured out yet. Would love your opinion and feedback. Wanna comment on the doc or would love to talk 1:1 ! siobud.com/meeting
eggsome 2y ago
What's the plan for Wayland compatibility? For a little while I was able to share a single app - but not the full desktop. Now I can't share anything from Ubuntu 24.04 when using Wayland :(
- palata 2y ago
  I had to tweak a few things to make it work on my Alpine Linux. This tutorial helped a lot: https://github.com/emersion/xdg-desktop-portal-wlr/wiki/%22It-doesn't-work%22-Troubleshooting-Checklist https://github.com/emersion/xdg-desktop-portal-wlr/wiki/%22I...

makapuf 2y ago

Great things are accomplished by spite programming! https://hackaday.com/2018/01/03/spite-thrift-and-the-virtues-of-an-affordable-logic-analyzer/ https://hackaday.com/2018/01/03/spite-thrift-and-the-virtues...

Karrot_Kream 2y ago

Thanks the code is really useful to read through.

mcwiggin2 2y ago

This is awesome. I would love if you had some examples on how to use AntMedia as a source. I am mostly in video engineering so the read the source comes slower to me. This would be really handy in many cases.

mwcampbell 2y ago

Is the restriction to NVIDIA necessary for the low latency?

Sean-Der 2y ago
Nope! I want to add all the other flows. nvidia is the absolute lowest I believe. I wanted to do it first to know if it was worth building.

namibj 2y ago

Any plans on integrating L4S with e.g. Tetrys-based FEC and using a way where the congestion feedback from L4S acts on the quantizer/rate-factor instead of directly on bitrate? It's much more appropriate to do perceptual fairness than strict bitrate fairness. Happy to have a chat on this btw; you can best catch me on discord.

stefan_ 2y ago
Depends on the network, surely? Lots of applications for low latency video where you are not sharing the channel, but it has a fixed bandwidth.
- namibj 2y ago
  E.g. "Low Latency DOCSIS"[0] and related, WiFi[1], support it and with the former it's about non-exclusive scarce uplink capacity where cross-customer capacity sharing may rely on post-hoc analysis of flow behavior to check for abuse, switching to forced fairness if caught by such heuristics. For downstream it's even more natural to have shared capacity with enough congestion to matter, but often only the WiFi side would have a large discretionary range for bandwidth scheduling/allocation to matter much. Apple already opportunistically uses L4S with TCP-Prague and there are real-world deployments/experiments [2] with end-to-end L4S. Fixed- [0]: https://github.com/cablelabs/lld https://github.com/cablelabs/lld [1] relevant excerpt from [0]: Applications that send large volumes of traffic that need low latency, but that are responsive to congestion in the network. These applications can benefit from using a technology known as "Low Latency, Low Loss, Scalable Throughput (L4S)". Support for this technology is including in the LLD feature set, but is beyond the scope of what we have in this repository. Information on L4S can be found in this IETF draft architecture. [2]https://www.vodafone.com/news/technology/no-lag-gaming-vodafone-and-nokia-test-new-technology-to-eliminate-gaming-and-video-lag https://www.vodafone.com/news/technology/no-lag-gaming-vodaf...

cchance 2y ago

Question, 30ms latency sounds amazing but how does it actually compare to "the standard" sharing tools for desktops, like do you know what the latency on say MSRDP is as comparison or VNC?

Sean-Der 2y ago
I doubt the protocol itself makes a big difference. I bet you can get 30ms with VNC. The difference with BitWHIP. * Can play WebRTC in browser. That makes things easier to use. * simpler/hackable software. BitWHIP is simple and uses nvenc etc… if you use nvenc with VNC I bet you can get the same experience