8 ms·
Building video chat into my personal website using WebRTC, WebSockets, and Go
- regularemployee 6y agoI tried this myself too and when I try p2p with 4 people, out of 10 tests about 50% of the time I won't be able to see all 4 people or someone wouldn't be able to see all 4 people. It was really hard to make p2p work and debugging the ice connections was even harder.
- moron4hire 6y agoThat's probably Network Address Translation (NAT), which requires TURN (a fancy name for a central relay for all media) to "punch through". TURN literally stands for "Traversal Using Relay around NAT". And it's just a traditional, centralized. non-p2p fallback for people on paternalistic networks that don't allow them to create UDP connections or TCP connections on any ports other than 80 or 443. Which, as it turns out, is a lot of users. I've seen estimates in the range of 10 to 20% of users. Which means, for a random selection of 7 users, you pretty much have a 50/50 chance of not being able to peer everyone using just STUN.
- kwindla 6y agoI think it is likely also bandwidth and cpu issues with mesh peer-to-peer. Unless you're capping the video bitrate, the browser will try to use whatever the browser's default target is, for each connection. On Chrome that's 3mb/s, which is a lot of network bandwidth, and turns out to be a lot of cpu as well just shuffling those packets through the encoding->sending->bandwidth-estimation and receiving->decoding->rendering pipelines. Capping the video bitrate is more complicated and confusing than it should be. It's better now that the browser implementations are all more or less closing in on "WebRTC 1.0" compliance. But you still need to reach into either the raw SDP you are exchanging during signaling, or the RTCPeerConnection objects, and set the encoding bitrate target. The SaaS platforms that offer WebRTC APIs and infrastructure all do a lot of work under the covers to set bitrate caps, track constraints (resolution, for example), and other bits and pieces of WebRTC config that work well on a wide variety of networks, devices, and browsers.
- throw14082020 6y agoI always assumed everyone is behind NAT, you're saying on 10 to 20% of people are, and therefore only they need TURN. I'd love to see where you got that number. If I were to guess, the problem GP is facing is bandwidth, a mesh network uses exponentially more bandwidth. For each user, the bandwidth is linear, N more people requires N more bandwidth. This is fine for downloads, but uploading N more can be much more challenging for certain networks.
- cma 6y agoHe's mistaken that NAT always requires TURN. Consumer NAT typically still allows incoming UDP, using STUN/punch-through, or TCP with uPNP support. He maybe meant to talk about only about more restrictive NAT situations or campus/corporate/ISP/nation-state/scientology-compound firewalls.
- throw14082020 6y agoIm not sure about UDP hole punching and how it relates to WebRTC, i don't see it being talked about much. In general hole punching is a rare thing to hear, I cannot find much resources about it. And uPNP (Universal Plug and Play) sounds like its for device discovery in the same local network, so again, it doesn't sound related to webRTC, we can connect directly with each other on the same local network anyway.
- mypalmike 6y agoHole punching in the context of webrtc is usually referred to as ICE. UPNP has a number of functions, including forwarding of WAN packets to a specific LAN device.
- deleted 6y ago[deleted]
- mypalmike 6y agoThe 10-20 percent would reflect people with symmetric NAT, which is rather common with mobile networks, corporate NAT, etc. Symmetric NAT requires TURN relays. Typical home router configurations are not symmetric NAT, and usually work with just STUN.
- Sean-Der 6y agoThe only numbers I have ever seen published are Whereby's[0] they saw 17% used TURN. There is a little more nuance then just paternalistic networks though. In same cases like NAT Mapping exhaustion you just can't give an individual user multiple long lived mappings. Address Dependendent filtering/mapping also makes sense in some cases. It makes P2P harder, but does give you the ability to provide your users more sessions at least! https://medium.com/the-making-of-whereby/what-kind-of-turn-server-is-being-used-d67dbfc2ff5d https://medium.com/the-making-of-whereby/what-kind-of-turn-s...
- kwindla 6y agoWe see about the same numbers Whereby does at Daily, globally across our whole user base. Bounces around a little but is usually just under 20%. Way more for customers that are mostly serving corporate users, of course (firewalls). And more for mobile-heavy user populations. Actually, that's a good reminder that it would be nice to understand the mobile data networks breakdown in more detail. Most of the US mobile data networks require TURN, as far as I remember when I last looked at this. But I don't know if that's true everywhere in the world.
- Sean-Der 6y agoThat is awesome, thanks for sharing :) Lots of little details and they all effect each other. I really enjoy networking because of this. I wonder if we come back to this in 10 years what this number will be. Linux on the desktop and IPv6 is just around the corner...
- mekkkkkk 6y agoThinking about it, v4 addresses and oil have a lot in common. The exhaustion/depletion is coming, but we keep finding ways to circumvent it. For oil you got fracking and sand. For v4 you see wider adoption of wide/carrier-grade NAT. In both cases it temporarily solves the supply problem really effectively. However it also ruins the environment.
- deneb150 6y agoDebugging this is indeed quite hard. I have only used it for 1 on 1 p2p calls.
- Sean-Der 6y agoWebRTC + networking is frustrating. IMO it is a leaky abstraction. There was a hope that ICE+TURN would work everywhere and users would never need to worry. That isn't true so we need to do a better job educating developers about what/why things went wrong. I am working on a Open Source book that includes a WebRTC networking chapter[0]. Would love your opinions/feedback if this would have actually been helpful when learning this stuff! [0] https://webrtcforthecurious.com/docs/03-connecting/ https://webrtcforthecurious.com/docs/03-connecting/
- parhamn 6y agoWonderful and very well written, thanks! I too experimented with a p2p golang webchat setup. All the jargon was confusing and very hard to look up. This post has already given me much more clarity!!
- VWWHFSfQ 6y agoiOS and Safari is riddled with WebRTC bugs like this. Sounds similar to my experience. Everything consistently works great in Chrome and Firefox and then only kinda works Safari. worst browser on the planet
- nobleach 6y agoI've jokingly referred to Safari as SafarIE for the last 5 years. It does tick all these boxes: 1. Backed by an OS manufacturer that doesn't care about the web 2. Spends more time working on features that suit itself than meeting standards agreed upon by a body of which they're a part. 3. The only sanctioned/allowed browser on their platform (MS didn't even achieve this holy grail) 4. Lagging behind most other popular browsers by years in some cases But due to it being the ONLY browser that'll run on iOS, I have no choice but to dumb down user experience for it. This year's lovely issue has been MediaRecorder - but supposedly that's made it into the most recent release.
- mguerville 6y agoI use firefox on iOS, it’s even set as my default browser. Curious what you mean by “safari is the only browser that works in iOS”? For in browser video chat? Genuinely curious, what are the limitations?
- danielheath 6y agoFirefox on iOS uses the safari js runtime and the safari layout engine.
- mguerville 6y agoAh I see, thanks I had no idea (I assume most average users are in the same situation). I assume IpadOS is the same, and only MacOS allows a true non safari browser.
- VWWHFSfQ 6y agoFirefox and chrome on iOS are just re-skins of safari. Apple doesn't allow other browser backends
- yawnxyz 6y agoI thought I was just dumb for not being able to figure this out, but turns out it's just a hard problem for everyone haha
- 6510 6y agoYeah, it made me feel like I had to read the manual but there wasn't any. Seeing such a bad one reminded me how spoiled we are with beautiful abstractions.
- meheleventyone 6y agoThe worst one for me so far is mDNS not working on my local network so the one circumstance you should basically be able to guarantee an easy P2P connection doesn’t work.
- tschellenbach 6y agoI feel like part of the reason why software engineering projects are hard to estimate is this. So you want to add video chat.. - this blogpost: 100 loc - pion (open source): 100k loc? - dolby.io/ agora: I'm guessing >1m loc - zoom.... even more?
- deleted 6y ago[deleted]
- detaro 6y agoI'm not that convinced. I mean yes, if you literally just estimate "add video chat" without any research or further clarification, but that'd get you in hell in every other discipline too. EDIT: I guess one part might be that people are less likely to recognize specializations than in other disciplines?
- rodgerd 6y agoYeah. "I want a shed". OK, let's build a shed. "Oh yeah, and maybe some power, IDK maybe three phase. Also PoE and smart lighting and some insulation and enough room for a CNC machine and..."
- spenczar5 6y agoYou might have a big shed, but it wouldn’t be a very complicated one. Electrical supply is extremely well-abstracted and clearly defined. Software really is worse than other disciplines in how complexity leaks out everywhere.
- VWWHFSfQ 6y agoTo be fair this is a very simplistic project just to get the 2-way chat working. There are no fallbacks for anything that doesn't work correctly. There are also numerous edge cases especially iOS and Safari that would add 1,000 more lines of code to properly account for. Not too mention all the features that people actually want like muting, toggle video, noise detection/cancellation. So yeah, setting up a P2P video chat in 2021 is somewhat easy. Until it's not.
- ejb503 6y agoNot quite so easy as the blog makes out... didn't see any mention of turn and stun servers, and multi-peer adds layers of complexity... To stably build a negotiation system you'll probably need an infrastructure of websockets and some kind of nosql db to handle identity and other quirks around negotiation... Example... how do you handle refresh from a new tab or after the connection has dropped... some kind of device signature is probably needed too!! (We've just spent a year building this for ecommerce @ https://yown.it https://yown.it) BIG thumbs up for the interest in WebRTC though enormous potential...
- throw14082020 6y agoWebRTC is complicated, its been around for a while and support in browsers have not been great in the past, which might be why Zoom first used WebSockets for video. They use WebRTC now though, and WebRTC is fine now, it is the standard, but potential is not the right word. Have a look at WebTransport to see a future alternative with potential. For those who are interested, the technical term is signalling (not negotiation), and there are many providers that will help with that (ably.com, pubnub.com, pusher.com), you don't need to build your own infrastructure. WebSockets is also just one option. Using a SFU/ MCU is almost a requirement for multi person calls, becoming more important for bigger groups. I had a look at yown.it, I don't know what it does, your description of it is a bit vague. Those problems you mention are not hard to solve: "device signature"? You just set a cookie. Connection dropped? Cookie got you covered. New tab? Cookie got you covered. Refresh? Cookie you got covered. Other interesting technologies are: Twilio's network traversal service: https://www.twilio.com/stun-turn https://www.twilio.com/stun-turn Agora's higher level products (e.g. video call, voice call) https://www.agora.io/en https://www.agora.io/en
- socceroos 6y agoAnother great alternative: https://jitsi.org/projects/ https://jitsi.org/projects/
- ejb503 6y agoEssentially we enable in-browser comms (including but not limited to WebRTC for video and audio streams on top of storefronts). Given we allow anonymous connections, we need to associate each WebRTC connection with user defined data (read user profile). It's not quite as simple as "a cookie" because one user can have multiple devices, updated user information has to sync across the other connections and for a smooth experience you have to have synced connection statuses. We did look at syncing all this with RTC data channels, problem... you can't get message history and you also can't depend on the channel until after a successful negotiation, which again for us is only part of the larger infrastructure... This forces the use of a parallel comms system such as websockets, allowing for event based synchronisation as well as the organisation of the WebRTC metadata both pre and post connection... Most people don't want "naked javascript" with two faces on it, and WebRTC is a fantastic tool for video and audio streaming, however it is limited in its wider use (which is perfectly fine it does enough!)... I think the problem is that people associate "video chat" with simply the media streaming, whereas the reality is that integrating it into a feature rich front end framework is significantly more complicated, and not simply a case of "adding a cookie" The difference between the solutions you posted and websockets is as far as I can tell, "your own websockets" or "pay someone else to run your websockets".
- neophyt3 6y agono mention of turn servers, also p2p does not scale well since its mesh network... good article, but this wont make it to production alone
- jqpabc123 6y agoAnyone have something similar for screen sharing?
- tomcooks 6y agoStream screen to webrtc via ffmpeg
- Sean-Der 6y agoInstead of `getUserMedia` replace with `getDisplayMedia`. If you are looking for a native option use [0] or [1] and you can send anything from ffmpeg to webrtc. ffmpeg itself doesn't support WebRTC so need to use something for the last part. [0] https://github.com/rviscarra/webrtc-remote-screen https://github.com/rviscarra/webrtc-remote-screen [1] https://github.com/pion/webrtc/tree/master/examples/rtp-to-webrtc https://github.com/pion/webrtc/tree/master/examples/rtp-to-w...
- tppiotrowski 6y agoHere’s a similar project I did w/screenshare https://github.com/ted-piotrowski/ted-piotrowski.github.io https://github.com/ted-piotrowski/ted-piotrowski.github.io
- jqpabc123 6y agoThanks for all the replies.
- cblconfederate 6y agoi wonder if webrtc was built to be intentionally complex or if a better standard would make adoption easier, perhaps in conjunction with a standard server (like we have httpd for html)
- emehrkay 6y agoIm doing exactly this, but will use it with a raspberrypi with a 7 inch touch screen as my doorbell. Someone hits the link on the screen, it hits the server, server texts me a link to join the video session and thats it really. I got the core code going (I used a simple tornado [python] implementation as it has web sockets built in) This is the version of the js code that I got going (I couldn't reason about straight inline scripting, I had to make unnecessary classes. you dont need them) https://gist.github.com/emehrkay/1ea9a87a91e00b27843d9b71a3cce96c https://gist.github.com/emehrkay/1ea9a87a91e00b27843d9b71a3c... You also need to tell nginx to serve the wss connection with http 1.1 or the handshakes fail ``` location /websocket/path { proxy_pass http://whateverSiteDotCom; http://whateverSiteDotCom; proxy_http_version 1.1; proxy_set_header Connection "upgrade"; proxy_set_header Upgrade $http_upgrade; proxy_set_header Origin ''; } ```
- deneb150 6y agoLooks nice, here's the final working JS file from this blog post's example, no classes: https://github.com/m-butterfield/mattbutterfield.com/blob/882fc4b7aa3fe7f59a0d8acf0623d20e6e48c87a/app/controllers/js/video.js https://github.com/m-butterfield/mattbutterfield.com/blob/88... A bit dense, and could use some error handling...but it actually seems to work fine!
- emehrkay 6y agoThis looks pretty straight-forward and makes me wonder if im doing too much work creating and juggling two RTCPeerConnection connections. I will try your approach
- emehrkay 6y agoThis code works fine in iOS safari (and desktop chrome, Firefox, and safari). I've seen a few comments on how mobile safari has pitfalls or is the new IE, but there isnt any specific code there for safari. it all just works
- sscarduzio 6y agoI really like the video doorbell project, can you make it OSS and put it on a Github repo?
- Naac 6y ago...and GCP, and pub/sub/ and... So a little click baity title. If the backend wasn't distributed the title would be a little more apt.
- mfbx9da4 6y agoNice, did you setup a turn server?
- mro_name 6y agonice and concise, chapeau.
- SilurianWenlock 6y agoI cant see this video feature on his personal website?