6 ms·

STUNner: A Kubernetes ingress gateway for WebRTC

83 points by Sean-Der 4y ago

arianvanp 4y ago
This solves a lot of problems i was having deploying our media application at customers! Wish it existed back then. Had to jump through a lot of hoops to deploy TURN server infra on kubernetes and it was not pretty at all.
Uptrenda 4y ago
As chance would have it I've recently implemented a Turn client and a STUN client. Turn = a way to proxy UDP or TCP cons from host A to host B. STUN = a protocol that shows what your externally mapped port and IP is. From this you can inter what NAT type you have if any. My understanding is that when it comes to WebRTC, the thing that glues everything together is 'ICE' which decides where to get information from, what candidates to use, what strategy to use to connect. So in other words there is a server you use to exchange candidates? I am curious because I don't see how Turn or STUN by itself would be that useful for WebRTC and the project page doesn't mention Ice?
- arccy 4y ago
  ICE is just the negotiation that happens over a side channel, usually as part your application server (since it needs to know which clients want to talk to each other). It doesn't really make sense to have separate ICE infra.
- stagas 4y ago
  It mentions ICE[0] and provides a library[1] to simplify its configuration. [0]: https://github.com/l7mp/stunner#configuring-webrtc-clients-to-reach-stunner https://github.com/l7mp/stunner#configuring-webrtc-clients-t... [1]: https://www.npmjs.com/package/@l7mp/stunner-auth-lib https://www.npmjs.com/package/@l7mp/stunner-auth-lib
- solar-ice 4y ago
  The server you use to exchange ICE candidates ("you can try connecting to me here") is probably, in most cases, your media server - the thing you are hiding behind this TURN server. It probably exchanges them via a protocol like WHIP[0]. Probably, you'd try to configure it to present only one candidate here - the STUNner's TURN server. [0] https://www.ietf.org/archive/id/draft-ietf-wish-whip-02.html https://www.ietf.org/archive/id/draft-ietf-wish-whip-02.html
- xg15 4y ago
  Seems to me, they really missed out on naming the project "Turner" as it's really more about the TURN part than the rest. Essentially, they are making creative use of TURN servers as reverse proxies. My understanding of the whole thing is as follows: WebRTC was originally designed for direct P2P connections between browsers - and ICE was a grab-bag of strategies to get around NATs and firewalls. One "strategy" was TURN, which was essentially "give up and relay the peers through a 3rd party server". At some point, people started to (ab)use WebRTC for traditional client/server media streaming. Browsers don't care if the other peer sits in a data center. You can ignore most of ICE in that scenario, as you have full control over one peer (the media server) and therefore can ensure it is reachable - in particular, you don't need STUN or TURN servers to make it work. However, much of the design is still assuming P2P, so it doesn't play well with Kubernetes which is all designed for client/server. The trick is now that they expose a special TURN server which can act as a unified point of entry into the Kubernetes cluster and make the whole infrastructure much more usable in a Kubernetes context.
  - rg0now 4y ago
    <Spoiler alert: I'm one of the authors behind STUNner> Wow, this is a perfect summary, thanks! In fact, must better than what we could ever come up with, nice! Indeed, the idea is to reuse the STUN/TURN protocol suite (note that under the hood the TURN protocol is just an extension to STUN, hence the name STUNner) in order to ingest WebRTC traffic into Kubernetes in a browser-compliant way. ICE is not (necessarily) a concern for STUNner: ICE is just a negotiation protocol for WebRTC clients and servers to arrive to a common understanding of the addresses and ports used to exchange media, and STUNner is the dataplane that makes that communication happen. And that's maybe the main point here: STUNner is just the dataplane and we still lack a nice control plane. STUNner is already quite usable on its own (but you may need a bit of a reading and manual configuration to make it work), just like how you can use a manually configured Envoy to implement a simpler ingress gateway but of course using it via Istio or Gloo is so much more convenient! So the next step will be to expose STUNner to the Kubernetes control plane wrapped in the Kubernetes Gateway API. We plan this for the next STUNner release, so bear with us! Or better yet, come and join us, we wan use all help!
- metadat 4y ago
  Are your TURN and / or STUN implementations publicly available? There are also nice existing FOSS implementations: (Golang) Pion TURN - https://github.com/pion/turn https://github.com/pion/turn (Rust) Port of Pion TURN - https://github.com/webrtc-rs/turn https://github.com/webrtc-rs/turn and https://webrtc.rs/ https://webrtc.rs/ (Rust) TURN server and client - https://github.com/sile/rusturn https://github.com/sile/rusturn (Python) TURN Server - https://github.com/m32/sturn https://github.com/m32/sturn (Javascript) Node.js TURN server - https://github.com/Atlantis-Software/node-turn https://github.com/Atlantis-Software/node-turn (C) TURN server - https://github.com/Atlantis-Software/node-turn https://github.com/Atlantis-Software/node-turn Additional reading if you care to refresh on WebRTC terminology like STUN, TURN, ICE, etc: https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/Protocols https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/... https://www.twilio.com/docs/stun-turn/faq https://www.twilio.com/docs/stun-turn/faq
throw0101a 4y ago
People sometimes ask "What's the advantage of IPv6?". Well, not having to futz around with STUN/NAT due to NAT is one improvement. See "How NAT traversal works": * https://tailscale.com/blog/how-nat-traversal-works/ https://tailscale.com/blog/how-nat-traversal-works/ * https://news.ycombinator.com/item?id=30707711 https://news.ycombinator.com/item?id=30707711 (2022) * https://news.ycombinator.com/item?id=24241105 https://news.ycombinator.com/item?id=24241105 (2020) And no, just because a device has a publicly IPv6 address does not mean it is actually / necessarily accessible from the public Internet. Firewalls do not stop working with IPv6. My iMac lives behind my Asus RT-AC68U; I can ping my iMac's IPv6 address: * https://www.subnetonline.com/pages/ipv6-network-tools/online-ipv6-ping.php https://www.subnetonline.com/pages/ipv6-network-tools/online... But I cannot connect to SSH from the Internet (but from my laptop SSH works), or port 80 (if I launch 'nc -6 -l 80'): * https://www.subnetonline.com/pages/ipv6-network-tools/online-ipv6-port-scanner.php https://www.subnetonline.com/pages/ipv6-network-tools/online... You can have something with a 2000/3 unicast, non-link-local IPv6 address, and decide via policy whether to allow connections to it, either with an on-host firewall, or with an upstream network device: * https://www.asus.com/support/FAQ/1013638/ https://www.asus.com/support/FAQ/1013638/ To repeat: Firewalls do not stop working with IPv6. And yes, there is still some layer of software that is needed for allowing network connections if the default firewall policy is default deny all (except for replies to 'internally' originated connections): * https://en.wikipedia.org/wiki/Port_Control_Protocol https://en.wikipedia.org/wiki/Port_Control_Protocol * http://upnp.org/specs/arch/UPnP-arch-AnnexAIPv6-v1.pdf http://upnp.org/specs/arch/UPnP-arch-AnnexAIPv6-v1.pdf But not having to deal with address weirdness is one less complication.
- giancarlostoro 4y ago
  I don't care about open ports, I care more about exposing my IP to internet strangers. I'm having flashbacks to Skype days where other gamers would get your IP and use malicious tools to basically nuke your internet. Skype was really dumb about leaking your IP to total internet strangers. I would basically just login to a less known account when this would happen because somebody got mad I did this or that in a video game. I hear Discord direct calls will also leak your IP so I never accept calls from anyone I don't personally know.

83 points by Sean-Der 4y ago

This solves a lot of problems i was having deploying our media application at customers! Wish it existed back then. Had to jump through a lot of hoops to deploy TURN server infra on kubernetes and it was not pretty at all.

Uptrenda 4y ago

As chance would have it I've recently implemented a Turn client and a STUN client. Turn = a way to proxy UDP or TCP cons from host A to host B. STUN = a protocol that shows what your externally mapped port and IP is. From this you can inter what NAT type you have if any. My understanding is that when it comes to WebRTC, the thing that glues everything together is 'ICE' which decides where to get information from, what candidates to use, what strategy to use to connect. So in other words there is a server you use to exchange candidates? I am curious because I don't see how Turn or STUN by itself would be that useful for WebRTC and the project page doesn't mention Ice?

arccy 4y ago
ICE is just the negotiation that happens over a side channel, usually as part your application server (since it needs to know which clients want to talk to each other). It doesn't really make sense to have separate ICE infra.
stagas 4y ago
It mentions ICE[0] and provides a library[1] to simplify its configuration. [0]: https://github.com/l7mp/stunner#configuring-webrtc-clients-to-reach-stunner https://github.com/l7mp/stunner#configuring-webrtc-clients-t... [1]: https://www.npmjs.com/package/@l7mp/stunner-auth-lib https://www.npmjs.com/package/@l7mp/stunner-auth-lib
solar-ice 4y ago
The server you use to exchange ICE candidates ("you can try connecting to me here") is probably, in most cases, your media server - the thing you are hiding behind this TURN server. It probably exchanges them via a protocol like WHIP[0]. Probably, you'd try to configure it to present only one candidate here - the STUNner's TURN server. [0] https://www.ietf.org/archive/id/draft-ietf-wish-whip-02.html https://www.ietf.org/archive/id/draft-ietf-wish-whip-02.html
xg15 4y ago
Seems to me, they really missed out on naming the project "Turner" as it's really more about the TURN part than the rest. Essentially, they are making creative use of TURN servers as reverse proxies. My understanding of the whole thing is as follows: WebRTC was originally designed for direct P2P connections between browsers - and ICE was a grab-bag of strategies to get around NATs and firewalls. One "strategy" was TURN, which was essentially "give up and relay the peers through a 3rd party server". At some point, people started to (ab)use WebRTC for traditional client/server media streaming. Browsers don't care if the other peer sits in a data center. You can ignore most of ICE in that scenario, as you have full control over one peer (the media server) and therefore can ensure it is reachable - in particular, you don't need STUN or TURN servers to make it work. However, much of the design is still assuming P2P, so it doesn't play well with Kubernetes which is all designed for client/server. The trick is now that they expose a special TURN server which can act as a unified point of entry into the Kubernetes cluster and make the whole infrastructure much more usable in a Kubernetes context.
- rg0now 4y ago
  <Spoiler alert: I'm one of the authors behind STUNner> Wow, this is a perfect summary, thanks! In fact, must better than what we could ever come up with, nice! Indeed, the idea is to reuse the STUN/TURN protocol suite (note that under the hood the TURN protocol is just an extension to STUN, hence the name STUNner) in order to ingest WebRTC traffic into Kubernetes in a browser-compliant way. ICE is not (necessarily) a concern for STUNner: ICE is just a negotiation protocol for WebRTC clients and servers to arrive to a common understanding of the addresses and ports used to exchange media, and STUNner is the dataplane that makes that communication happen. And that's maybe the main point here: STUNner is just the dataplane and we still lack a nice control plane. STUNner is already quite usable on its own (but you may need a bit of a reading and manual configuration to make it work), just like how you can use a manually configured Envoy to implement a simpler ingress gateway but of course using it via Istio or Gloo is so much more convenient! So the next step will be to expose STUNner to the Kubernetes control plane wrapped in the Kubernetes Gateway API. We plan this for the next STUNner release, so bear with us! Or better yet, come and join us, we wan use all help!
metadat 4y ago
Are your TURN and / or STUN implementations publicly available? There are also nice existing FOSS implementations: (Golang) Pion TURN - https://github.com/pion/turn https://github.com/pion/turn (Rust) Port of Pion TURN - https://github.com/webrtc-rs/turn https://github.com/webrtc-rs/turn and https://webrtc.rs/ https://webrtc.rs/ (Rust) TURN server and client - https://github.com/sile/rusturn https://github.com/sile/rusturn (Python) TURN Server - https://github.com/m32/sturn https://github.com/m32/sturn (Javascript) Node.js TURN server - https://github.com/Atlantis-Software/node-turn https://github.com/Atlantis-Software/node-turn (C) TURN server - https://github.com/Atlantis-Software/node-turn https://github.com/Atlantis-Software/node-turn Additional reading if you care to refresh on WebRTC terminology like STUN, TURN, ICE, etc: https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/Protocols https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/... https://www.twilio.com/docs/stun-turn/faq https://www.twilio.com/docs/stun-turn/faq

throw0101a 4y ago

People sometimes ask "What's the advantage of IPv6?". Well, not having to futz around with STUN/NAT due to NAT is one improvement. See "How NAT traversal works": * https://tailscale.com/blog/how-nat-traversal-works/ https://tailscale.com/blog/how-nat-traversal-works/ * https://news.ycombinator.com/item?id=30707711 https://news.ycombinator.com/item?id=30707711 (2022) * https://news.ycombinator.com/item?id=24241105 https://news.ycombinator.com/item?id=24241105 (2020) And no, just because a device has a publicly IPv6 address does not mean it is actually / necessarily accessible from the public Internet. Firewalls do not stop working with IPv6. My iMac lives behind my Asus RT-AC68U; I can ping my iMac's IPv6 address: * https://www.subnetonline.com/pages/ipv6-network-tools/online-ipv6-ping.php https://www.subnetonline.com/pages/ipv6-network-tools/online... But I cannot connect to SSH from the Internet (but from my laptop SSH works), or port 80 (if I launch 'nc -6 -l 80'): * https://www.subnetonline.com/pages/ipv6-network-tools/online-ipv6-port-scanner.php https://www.subnetonline.com/pages/ipv6-network-tools/online... You can have something with a 2000/3 unicast, non-link-local IPv6 address, and decide via policy whether to allow connections to it, either with an on-host firewall, or with an upstream network device: * https://www.asus.com/support/FAQ/1013638/ https://www.asus.com/support/FAQ/1013638/ To repeat: Firewalls do not stop working with IPv6. And yes, there is still some layer of software that is needed for allowing network connections if the default firewall policy is default deny all (except for replies to 'internally' originated connections): * https://en.wikipedia.org/wiki/Port_Control_Protocol https://en.wikipedia.org/wiki/Port_Control_Protocol * http://upnp.org/specs/arch/UPnP-arch-AnnexAIPv6-v1.pdf http://upnp.org/specs/arch/UPnP-arch-AnnexAIPv6-v1.pdf But not having to deal with address weirdness is one less complication.

giancarlostoro 4y ago
I don't care about open ports, I care more about exposing my IP to internet strangers. I'm having flashbacks to Skype days where other gamers would get your IP and use malicious tools to basically nuke your internet. Skype was really dumb about leaking your IP to total internet strangers. I would basically just login to a less known account when this would happen because somebody got mad I did this or that in a video game. I hear Discord direct calls will also leak your IP so I never accept calls from anyone I don't personally know.