5 ms·

WebRTC for the Curious

223 points by sebg 2y ago

valorzard 2y ago
WebRTC is lowkey one of the most underrated technologies out there, ESPECIALLY WebRTC data channels. WebRTC data channels are basically “UDP on the web” but they have lots of controls to change how reliable they are, so they can be used as TCP style connections as well. I still don’t fully understand why more people don’t use them over something like QUIC. (I think I’ve asked this question before here, but I wasn’t really satisfied with the answers) I sadly switched off of using them, but mostly because the ecosystem around there is super underdeveloped compared to the ecosystem around QUIC/quinn. There is a LOT of boilerplate involved that feels unnecessary. But, if you’re making a multiplayer game in the web, it’s basically the best technology to use cuz it already works. And if you use a library like libdatachannel or pion, you could make the game in Go or C++ and compile it for both Steam and the web! Here’s a project I did with them that shows off compiled for both web and desktop: https://github.com/ValorZard/gopher-combat https://github.com/ValorZard/gopher-combat
- ronsor 2y ago
  > I still don’t fully understand why more people don’t use them over something like QUIC. > the ecosystem around there is super underdeveloped compared to the ecosystem around QUIC/quinn > There is a LOT of boilerplate involved that feels unnecessary I think you just answered your own question and even gave the answers I would've given.
  - valorzard 2y ago
    Well it’s a bit of a chicken or the egg situation right? None of those issues will get better if no one uses it.
- torginus 2y ago
  > I still don’t fully understand why more people don’t use them over something like QUIC Dealing with NAT traversal especially with full-cone NATs is difficult and expensive - you have to maintain dedicated infrastructure of TURN servers for NAT and you have to proxy all your traffic through that, it's quite the overhead, especially since IPv4 addresses and bandwidth on AWS don't come cheap.
  - torginus 2y ago
    Here comes the inevitable HN downvote when for once I talk about something I'm professionally employed to do.
  - remram 2y ago
    That's only if you do peer-to-peer. QUIC can't even do peer-to-peer, so I don't understand the critique.
- Sean-Der 2y ago
  I am so happy that WebRTC has so many implementations. Every developer thinks differently, it's good to have a implementation that matches your mindset. * https://github.com/shinyoshiaki/werift-webrtc https://github.com/shinyoshiaki/werift-webrtc (Typescript) * https://github.com/pion/webrtc https://github.com/pion/webrtc (Golang) * https://github.com/webrtc-rs/webrtc https://github.com/webrtc-rs/webrtc (Rust) * https://github.com/algesten/str0m https://github.com/algesten/str0m (Rust) * hhttps://github.com/sepfy/libpeer https://github.com/sepfy/libpeer (C/Embedded) * https://webrtc.googlesource.com/src/ https://webrtc.googlesource.com/src/ (C++) * https://github.com/sipsorcery-org/sipsorcery https://github.com/sipsorcery-org/sipsorcery (C#) * https://github.com/paullouisageneau/libdatachannel https://github.com/paullouisageneau/libdatachannel (C++) * https://github.com/elixir-webrtc https://github.com/elixir-webrtc (Elixir) * https://github.com/aiortc/aiortc https://github.com/aiortc/aiortc (Python) * GStreamer’s webrtcbin (C) See https://github.com/sipsorcery/webrtc-echoes https://github.com/sipsorcery/webrtc-echoes for examples of some running against each other.
  - esafak 2y ago
    Anyone used a JVM implementation?
  - lostmsu 2y ago
    Sadly, SipSorcery is lagging behind as of now. We only have H.264 and VP8 support for video, data channels only have partial implementation (only reliable ordered), and the performance of data channels is far from what it could be. But it is a great starting point for anyone working in C#.
  - shay_ker 2y ago
    Unfortunately, there's massive drift between WebRTC implementations. The status quo is Google's, which is used by Chrome: https://webrtc.googlesource.com/src/ https://webrtc.googlesource.com/src/ And everyone else has to play catch-up. A big source of the drift is having a common library for SDP parsing, but also necessary features like BWE, different encodings, etc. For example, aiortc, python's WebRTC implementation, isn't quite at the level most would want. It isn't necessarily easy to tell, without clear benchmarks, which implementation is at-par with Google's.
- moffkalast 2y ago
  The horrible boilerplate and complexity is why people don't use it. Getting even simple usage examples seems to be a tall order for some use cases and LLMs are hopeless in trying to help with it. P2P connections are also often blocked by ISPs for whatever reason, making it impossible to use without a fallback TURN server which defeats the entire purpose of the thing if you wanted to do scalable multiplayer without the server infrastructure. You're left sending over the whole stream with double the latency and have to eat all the bandwidth.
- spencerflem 2y ago
  Im excited for WebTransport for similar reasons. Being peer to peer is really cool though
- aspbee555 2y ago
  I have been working on a quic/webrtc application for a while, webrtc capabilities are severely underrated since everything pairs it with TCP signaling defeating the power of webrtc
- numa7numa7 2y ago
  I have no knowledge of any of these but do you think webRTC might be superior to websockets for a multiplayer game at this point?
  - Sean-Der 2y ago
    Depends! If you want unordered/non-sequential it could help. If you don’t care about those things could be more work for no value.
Sean-Der 2y ago
I hope everyone enjoys WebRTC for the Curious (I am one of the authors). If you have any ideas on how to make it better I would love to hear :) WebRTC is absolutely magical. Having 'Web' in the name does it a disservice. I see it used to. * Remotely control excavators * Security Cameras * Connect autonomous robots in factories * Low latency streaming from sail boats and rocket ships * And all the use cases you expect (Conferencing, Telehealth etc..)
- fidotron 2y ago
  My experience is WebRTC has an on-ramp problem, which this is partly to address, but it is significantly helped if you are thrown in the deep end to work on it with someone that knows the quirks already. The big thing I got from that process was to stop being afraid of reading the SDP, because a huge amount of the problems you will run into are really the result of SDP oddities. It is credit to people like the libwebrtc maintainers that the complex morass of MediaStreamTrack processing is as solid and performant as it is these days. (And extendable). I share the view that it should form the basis of real time communication, humans involved or not, and a/v media involved or not. There seems to be some progress on applying absolute timestamps to frames, for example, however, at some point if we want to have rocket ships using it (and I do too) we will eventually need to have some way to reconcile divergent clocks used at different sources! Sean is modestly not mentioning Pion here, which is the lower level library many golang people reach to for webrtc components, and deservedly so.
  - Sean-Der 2y ago
    I need to finish it but 'Reading the SDP' I hope to solve with https://pe.pion.ly/ https://pe.pion.ly/ (https://github.com/pion/explainer https://github.com/pion/explainer). You can just your Offer/Answer in and interactively explore what has been configured. WebRTC's complexity can be frustrating. I believe it is inherent to how many things it is trying to solve. If an alternative arrives that solves everything WebRTC does, it will end up being just as complex. Or maybe not! Time will tell.
  - quantadev 2y ago
    For JavaScript developers "simple-peer" npm package is very popular and makes WebRTC very simple. I don't think its' an on-ramp problem, holding it back, it's just that most apps don't necessarily benefit from P2P coms, and also I'd hazard a guess not many people know about WebRTC, yet. Just guessing.
adhamsalama 2y ago
I like this book, but it doesn't contain any code examples, so it wasn't useful to me. I ended up adopting code from the High Performance Browser Networking book and some code examples by Google that were written like 8 years ago. It was painful to replace the outdated APIs with new ones and rewrite in TypeScript, but I eventually did it. https://github.com/adhamsalama/webrtc/ https://github.com/adhamsalama/webrtc/ Then I used it with WebAssembly to run a distributed SQLite database in the browser for peer-to-peer collaboration. https://github.com/adhamsalama/sqlite-wasm-webrtc https://github.com/adhamsalama/sqlite-wasm-webrtc Probably the coolest projects I have ever done, and they contain almost no backend code at all (I am a backend dev).
- Sean-Der 2y ago
  I’m glad you had fun with WebRTC. It’s super cool tech. If you are ever back in WebRTC land would loved to help https://pion.ly/discord https://pion.ly/discord The book is vendor agnostic. I wanted it to be timeless and vendor agnostic. The lack of code made all publishers I approached reject it :( I would love to see a hard copy someday. I had hoped if I put no code in it the WebRTC community would feel more comfortable with it. If I made it Pion specific, would greatly reduce its reach.
  - adhamsalama 2y ago
    Can I ask what you mean by vendor agnostic? Don't the WebRTC APIs exist in all browsers?
    - Sean-Der 2y ago
      You have multiple implementations (Go, Python/Typescript) and servers (Janus/Jitsi) I was afraid if I pushed any software in particular it would dissuade other groups from using it
      - adhamsalama 2y ago
        I just wrote it using the browser APIs and a small signaling server using WebSockets. That's it.
amelius 2y ago
Can webrtc technology be used to pierce through corporate firewalls?
- Sean-Der 2y ago
  Yep! ICE lets you try a bunch of different ports and protocols. So you can have a remote peer and try to contact it via UDP/TCP/TLS. You can even attempt via multiple interfaces (Wifi and 5G). You can then measure the packet loss/latency across these different paths and figure out which one is best.
- curious_curios 2y ago
  Likely yes, via TURN servers.
- dboreham 2y ago
  No. It doesn't allow anything that can't be done by a browser behind a firewall connecting to a regular web server outside the firewall.
  - Sean-Der 2y ago
    WebRTC can create a NAT Mapping via STUN (UDP). https://webrtcforthecurious.com/docs/03-connecting/#nat-mapping https://webrtcforthecurious.com/docs/03-connecting/#nat-mapp... that is pretty powerful compared to Client/Server HTTP. Dependent on the firewall, but most I have seen allow NAT mapping with different behaviors.
  - ranger_danger 2y ago
    To my knowledge, most regular web servers either cannot speak DTLS at all or only have limited support for it, but all modern browsers work with it just fine for WebRTC.
hnlurker22 2y ago
Thank you for not calling it WebRTC for Dummies
- pavlov 2y ago
  The “for Dummies” branding is a trademark of the publishing company Wiley, so you need a contract with them to use it.
  - hnlurker22 2y ago
    Tell me you wrote a for dummies book without telling you wrote a for dummies book
quantadev 2y ago
For all those into WebRTC, I just ran across this Reddit topic (link below), and the very latest post is one about a Chat App that was apparently just completed last week: https://www.reddit.com/r/WebRTC/comments/1jwwfj5/quanta_chat_all_new_version_rewritten_from_scratch/ https://www.reddit.com/r/WebRTC/comments/1jwwfj5/quanta_chat... It mentions Nostr, and makes me realize both Nostr and WebRTC can work well together. I haven't checked, but I'd guess there are Nostr apps using WebRTC to send Nostr messages around. I mean Nostr is basically just a crypto-signed JSON object.

223 points by sebg 2y ago

WebRTC is lowkey one of the most underrated technologies out there, ESPECIALLY WebRTC data channels. WebRTC data channels are basically “UDP on the web” but they have lots of controls to change how reliable they are, so they can be used as TCP style connections as well. I still don’t fully understand why more people don’t use them over something like QUIC. (I think I’ve asked this question before here, but I wasn’t really satisfied with the answers) I sadly switched off of using them, but mostly because the ecosystem around there is super underdeveloped compared to the ecosystem around QUIC/quinn. There is a LOT of boilerplate involved that feels unnecessary. But, if you’re making a multiplayer game in the web, it’s basically the best technology to use cuz it already works. And if you use a library like libdatachannel or pion, you could make the game in Go or C++ and compile it for both Steam and the web! Here’s a project I did with them that shows off compiled for both web and desktop: https://github.com/ValorZard/gopher-combat https://github.com/ValorZard/gopher-combat

ronsor 2y ago
> I still don’t fully understand why more people don’t use them over something like QUIC. > the ecosystem around there is super underdeveloped compared to the ecosystem around QUIC/quinn > There is a LOT of boilerplate involved that feels unnecessary I think you just answered your own question and even gave the answers I would've given.
- valorzard 2y ago
  Well it’s a bit of a chicken or the egg situation right? None of those issues will get better if no one uses it.
torginus 2y ago
> I still don’t fully understand why more people don’t use them over something like QUIC Dealing with NAT traversal especially with full-cone NATs is difficult and expensive - you have to maintain dedicated infrastructure of TURN servers for NAT and you have to proxy all your traffic through that, it's quite the overhead, especially since IPv4 addresses and bandwidth on AWS don't come cheap.
- torginus 2y ago
  Here comes the inevitable HN downvote when for once I talk about something I'm professionally employed to do.
- remram 2y ago
  That's only if you do peer-to-peer. QUIC can't even do peer-to-peer, so I don't understand the critique.
Sean-Der 2y ago
I am so happy that WebRTC has so many implementations. Every developer thinks differently, it's good to have a implementation that matches your mindset. * https://github.com/shinyoshiaki/werift-webrtc https://github.com/shinyoshiaki/werift-webrtc (Typescript) * https://github.com/pion/webrtc https://github.com/pion/webrtc (Golang) * https://github.com/webrtc-rs/webrtc https://github.com/webrtc-rs/webrtc (Rust) * https://github.com/algesten/str0m https://github.com/algesten/str0m (Rust) * hhttps://github.com/sepfy/libpeer https://github.com/sepfy/libpeer (C/Embedded) * https://webrtc.googlesource.com/src/ https://webrtc.googlesource.com/src/ (C++) * https://github.com/sipsorcery-org/sipsorcery https://github.com/sipsorcery-org/sipsorcery (C#) * https://github.com/paullouisageneau/libdatachannel https://github.com/paullouisageneau/libdatachannel (C++) * https://github.com/elixir-webrtc https://github.com/elixir-webrtc (Elixir) * https://github.com/aiortc/aiortc https://github.com/aiortc/aiortc (Python) * GStreamer’s webrtcbin (C) See https://github.com/sipsorcery/webrtc-echoes https://github.com/sipsorcery/webrtc-echoes for examples of some running against each other.
- esafak 2y ago
  Anyone used a JVM implementation?
- lostmsu 2y ago
  Sadly, SipSorcery is lagging behind as of now. We only have H.264 and VP8 support for video, data channels only have partial implementation (only reliable ordered), and the performance of data channels is far from what it could be. But it is a great starting point for anyone working in C#.
- shay_ker 2y ago
  Unfortunately, there's massive drift between WebRTC implementations. The status quo is Google's, which is used by Chrome: https://webrtc.googlesource.com/src/ https://webrtc.googlesource.com/src/ And everyone else has to play catch-up. A big source of the drift is having a common library for SDP parsing, but also necessary features like BWE, different encodings, etc. For example, aiortc, python's WebRTC implementation, isn't quite at the level most would want. It isn't necessarily easy to tell, without clear benchmarks, which implementation is at-par with Google's.
moffkalast 2y ago
The horrible boilerplate and complexity is why people don't use it. Getting even simple usage examples seems to be a tall order for some use cases and LLMs are hopeless in trying to help with it. P2P connections are also often blocked by ISPs for whatever reason, making it impossible to use without a fallback TURN server which defeats the entire purpose of the thing if you wanted to do scalable multiplayer without the server infrastructure. You're left sending over the whole stream with double the latency and have to eat all the bandwidth.
spencerflem 2y ago
Im excited for WebTransport for similar reasons. Being peer to peer is really cool though
aspbee555 2y ago
I have been working on a quic/webrtc application for a while, webrtc capabilities are severely underrated since everything pairs it with TCP signaling defeating the power of webrtc
numa7numa7 2y ago
I have no knowledge of any of these but do you think webRTC might be superior to websockets for a multiplayer game at this point?
- Sean-Der 2y ago
  Depends! If you want unordered/non-sequential it could help. If you don’t care about those things could be more work for no value.

Sean-Der 2y ago

I hope everyone enjoys WebRTC for the Curious (I am one of the authors). If you have any ideas on how to make it better I would love to hear :) WebRTC is absolutely magical. Having 'Web' in the name does it a disservice. I see it used to. * Remotely control excavators * Security Cameras * Connect autonomous robots in factories * Low latency streaming from sail boats and rocket ships * And all the use cases you expect (Conferencing, Telehealth etc..)

fidotron 2y ago
My experience is WebRTC has an on-ramp problem, which this is partly to address, but it is significantly helped if you are thrown in the deep end to work on it with someone that knows the quirks already. The big thing I got from that process was to stop being afraid of reading the SDP, because a huge amount of the problems you will run into are really the result of SDP oddities. It is credit to people like the libwebrtc maintainers that the complex morass of MediaStreamTrack processing is as solid and performant as it is these days. (And extendable). I share the view that it should form the basis of real time communication, humans involved or not, and a/v media involved or not. There seems to be some progress on applying absolute timestamps to frames, for example, however, at some point if we want to have rocket ships using it (and I do too) we will eventually need to have some way to reconcile divergent clocks used at different sources! Sean is modestly not mentioning Pion here, which is the lower level library many golang people reach to for webrtc components, and deservedly so.
- Sean-Der 2y ago
  I need to finish it but 'Reading the SDP' I hope to solve with https://pe.pion.ly/ https://pe.pion.ly/ (https://github.com/pion/explainer https://github.com/pion/explainer). You can just your Offer/Answer in and interactively explore what has been configured. WebRTC's complexity can be frustrating. I believe it is inherent to how many things it is trying to solve. If an alternative arrives that solves everything WebRTC does, it will end up being just as complex. Or maybe not! Time will tell.
- quantadev 2y ago
  For JavaScript developers "simple-peer" npm package is very popular and makes WebRTC very simple. I don't think its' an on-ramp problem, holding it back, it's just that most apps don't necessarily benefit from P2P coms, and also I'd hazard a guess not many people know about WebRTC, yet. Just guessing.

adhamsalama 2y ago

I like this book, but it doesn't contain any code examples, so it wasn't useful to me. I ended up adopting code from the High Performance Browser Networking book and some code examples by Google that were written like 8 years ago. It was painful to replace the outdated APIs with new ones and rewrite in TypeScript, but I eventually did it. https://github.com/adhamsalama/webrtc/ https://github.com/adhamsalama/webrtc/ Then I used it with WebAssembly to run a distributed SQLite database in the browser for peer-to-peer collaboration. https://github.com/adhamsalama/sqlite-wasm-webrtc https://github.com/adhamsalama/sqlite-wasm-webrtc Probably the coolest projects I have ever done, and they contain almost no backend code at all (I am a backend dev).

Sean-Der 2y ago
I’m glad you had fun with WebRTC. It’s super cool tech. If you are ever back in WebRTC land would loved to help https://pion.ly/discord https://pion.ly/discord The book is vendor agnostic. I wanted it to be timeless and vendor agnostic. The lack of code made all publishers I approached reject it :( I would love to see a hard copy someday. I had hoped if I put no code in it the WebRTC community would feel more comfortable with it. If I made it Pion specific, would greatly reduce its reach.
- adhamsalama 2y ago
  Can I ask what you mean by vendor agnostic? Don't the WebRTC APIs exist in all browsers?
  - Sean-Der 2y ago
    You have multiple implementations (Go, Python/Typescript) and servers (Janus/Jitsi) I was afraid if I pushed any software in particular it would dissuade other groups from using it
    - adhamsalama 2y ago
      I just wrote it using the browser APIs and a small signaling server using WebSockets. That's it.

amelius 2y ago

Can webrtc technology be used to pierce through corporate firewalls?

Sean-Der 2y ago
Yep! ICE lets you try a bunch of different ports and protocols. So you can have a remote peer and try to contact it via UDP/TCP/TLS. You can even attempt via multiple interfaces (Wifi and 5G). You can then measure the packet loss/latency across these different paths and figure out which one is best.
curious_curios 2y ago
Likely yes, via TURN servers.
dboreham 2y ago
No. It doesn't allow anything that can't be done by a browser behind a firewall connecting to a regular web server outside the firewall.
- Sean-Der 2y ago
  WebRTC can create a NAT Mapping via STUN (UDP). https://webrtcforthecurious.com/docs/03-connecting/#nat-mapping https://webrtcforthecurious.com/docs/03-connecting/#nat-mapp... that is pretty powerful compared to Client/Server HTTP. Dependent on the firewall, but most I have seen allow NAT mapping with different behaviors.
- ranger_danger 2y ago
  To my knowledge, most regular web servers either cannot speak DTLS at all or only have limited support for it, but all modern browsers work with it just fine for WebRTC.

hnlurker22 2y ago

Thank you for not calling it WebRTC for Dummies

pavlov 2y ago
The “for Dummies” branding is a trademark of the publishing company Wiley, so you need a contract with them to use it.
- hnlurker22 2y ago
  Tell me you wrote a for dummies book without telling you wrote a for dummies book

quantadev 2y ago

For all those into WebRTC, I just ran across this Reddit topic (link below), and the very latest post is one about a Chat App that was apparently just completed last week: https://www.reddit.com/r/WebRTC/comments/1jwwfj5/quanta_chat_all_new_version_rewritten_from_scratch/ https://www.reddit.com/r/WebRTC/comments/1jwwfj5/quanta_chat... It mentions Nostr, and makes me realize both Nostr and WebRTC can work well together. I haven't checked, but I'd guess there are Nostr apps using WebRTC to send Nostr messages around. I mean Nostr is basically just a crypto-signed JSON object.