7 ms·
Show HN: Pure C WebRTC implementation for embedded devices
- rytill 7y agoSean, where were your libraries a year ago ...? Having implemented the entire WebRTC spec a couple times now, what wisdoms have you learned that can be shared with us mortals? If you could write a protocol for the same use-case right now that instantly became as popular and standard as WebRTC, what would be different about it?
- Sean-Der 7y agoI am so glad you like them! Working on Pion and the KVS WebRTC stuff has been super rewarding because people want it. I have built so many things that flopped, so this is a welcome change of pace. Thank you so much for being part of the community, talking to people on the Pion slack about WebRTC is the highpoint of my week :) > Having implemented the entire WebRTC spec a couple times now, what wisdoms have you learned that can be shared with us mortals? The biggest thing I learned during all this was community/collaboration. With all the RFCs I could grind it out and learn them/write unit tests, but working with people took a lot more. With Pion I wanted to build a community/get people excited. I don't always get it right, but I love when I can figure out where mine and other peoples interests lie. The next big one is convincing people that WebRTC/P2P is important. It has been really hard to convince the Go community that this stuff is important, I have gotten LOTS of talk rejections. I feel so strongly that WebRTC/P2P is the future I want to see. I just want to see an internet that is resilient to monoliths going down and encourages direct collaboration between people. > If you could write a protocol for the same use-case right now that instantly became as popular and standard as WebRTC, what would be different about it? I think WebRTC as a protocol is really wonderful. I love that it took established/working technologies and wrapped them up into a nice package. This has let me bridge WebRTC with lots of other interesting projects/hacks. I think that is why WebRTC will eventually eat the RTC/VoIP world, since it is a standard and uses existing technologies it works everywhere. I would like to see lots of changes to WebRTC as an API though. The W3C/Javascript protocol doesn't allow you access to enough things, I think the more things developers can tweak the more interesting things we will see built. This was a slide I did for the W3C https://i.imgur.com/vHTnlFY.png https://i.imgur.com/vHTnlFY.png it hasn't gotten much reception yet, but I am going to keep trying :)
- deleted 7y ago[deleted]
- billconan 7y agoyes. WebRTC doesn't allow changing the video buffering behavior. it is suitable for conference meetings, but not so ideal for remote control.
- j1elo 7y ago> Media via Datachannels is being chosen more and more (can be HW accelerated) That piqued my curiosity. Could you please expand on this? What kind of hw accel can be applied to video when it is sent through DataChannels instead of the standard video channel?
- Sean-Der 7y agoThe encryption for DataChannels is HW accelerated while video is not. I was able to convince people to change this one! So a developer can opt-in to the accelerated profile. https://bugs.chromium.org/p/chromium/issues/detail?id=713701#c20 https://bugs.chromium.org/p/chromium/issues/detail?id=713701...
- j1elo 7y agoNice speedup! However it seems that AES-GCM comes at a cost of considerable increases in packet sizes, from what I could read in the link. Have you observed an increase of packet loss and latencies in real-world networks? (esp. with third world connections) The improvements in CPU usage are clear for the server, but the choice doesn't seem so obvious to me EDIT: I meant that, for the quality of end-user experience (who doesn't care about server load), it doesn't seem like an immediately obvious better choice to just use GCM, until some empiric testing shows if the potential increase of packet loss causes noticeable problems, or on the contrary, doesn't. It would be very cool reading any observations in this regard.
- Sean-Der 7y agoSorry I haven't done any measurements myself. Agree, it is a trade-off for sure, if you are doing video and packets are already ~1300 bytes it probably makes sense. For Opus only you are really blowing up your payload. You might see CPU improvements on the client side as well though! If you have a fixed set of clients it might be something worth evaluating. I personally don't have any real-word deploys I can measure. The AES-GCM work came around because a Pion user mentioned it me, so then I added it/tried to push the issue in the IETF/W3C.
- PaulBGD_ 7y agoSean is phenominal, when I was writing a WebRTC service a year ago the Pion implementation looked really really good, so good that I learned Golang just to use Pions!
- Sean-Der 7y agoAppreciate the kind words :) Your bug reports were super helpful. Getting users like you are phenomenal, many people just get frustrated and leave without telling me. I really enjoyed getting your perspective on pion/webrtc as a new Go developer, stuff like that improves the project a lot!
- roberte3 7y agoWow, this is something that I have been looking for ~a year! Great job, can't wait to dust that project idea off the shelf!
- Sean-Der 7y agoAwesome! If you need any help with your project I am happy to help :)
- throwGuardian 7y agoLooking through the documentation, there is no mention of WebRtc data channel. 1. Does this SDK support WebRtc data channel. 2. Does the AWS Kinesis WebRTC provide a managed service for WebRTC data, without the need to use audio/video. 3. Aside from signaling, and the 30% if instances that true P2P connections fail to get established, is all traffic truly P2P?
- Sean-Der 7y ago1.) Yes it does! It has OnDataChannel, you can set it on the PeerConnection. createDataChannel is a WIP, hopefully this weekend. 2.) Yes. The WebRTC peers can do Data only if they want. We provide STUN/TURN servers, and works with any compliant WebRTC implementation. This repo exists just to serve embedded devices (which was harder before) 3.) Yes it is fully P2P. Even when going via TURN it is E2E encrypted, Amazon relays the traffic but it can’t decrypt it only the peers can.
- throwGuardian 7y agoThank you for the detailed response.
- gfodor 7y agoVery cool! One thing I often ask about in WebRTC threads is if anyone has come across any native erlang or elixir implementations in the wild of the stack (not just signalling.) I've been hoping for a solution like this to pop up in that ecosystem, but so far I haven't found anything. The latest versions of OTP now support DTLS as part of the main library, so it seems more possible now than ever! For others who are looking for similar WebRTC implementations, these are also out there: - https://github.com/pion/webrtc https://github.com/pion/webrtc - https://github.com/aiortc/aiortc https://github.com/aiortc/aiortc
- j1elo 7y agoMembrane [1] is a project that aims to create a pipeline-based media framework, much like GStreamer, but more to the point (smaller scope) and written for the Erlang VM. We could foresee a WebRTC module existing at some point in the future, but right now the project seems to be just in its infancy. I found it while dreaming about writing an Erlang-based WebRTC media server like Kurento, for which this kind of frameworks are invaluable, and currently GStreamer is the king. [1]: https://www.membraneframework.org/ https://www.membraneframework.org/
- mdonahoe 7y agoFun fact: OP was the creator of Pion https://www.linkedin.com/in/sean-dubois/ https://www.linkedin.com/in/sean-dubois/
- kolencherry 7y agoShiguredo‘s Sora SFU is the only complete Erlang WebRTC implementation that I’m familiar with. It’s a closed source commercial product.
- Sean-Der 7y agoI would love to help with this! I don't know Erlang, but have been looking for an excuse to learn :) If someone is able to lead/maintain the project I can help answer questions/dev work. I just know how to write idiomatic Erlang. I bet we could implement ice-lite, SRTP and SDP parsing in a few weeks easily.
- tehlike 7y agoNice. I have been on and off woth gstreamer, and combining all of amazon stack for video processing, it makes a lot of sense for surveillance applications!
- Riverheart 7y agoSo name aside, this would work with any compliant Webrtc provider? Edit: And is this library officially supported or more R&D?
- Sean-Der 7y agoYes! It implements the standard PeerConnection API. The only code that is KVS specific is the signaling code (which you can easily not use) Officially supported. KVS is a public service with developers full time working on this library.
- Riverheart 7y agoThanks!
- dysoncdn 7y agoThank you for making pion!
- Sean-Der 7y agoThank you for using it! I love seeing all the stuff people have built. If there is anything I can do to make it better I would love to hear. I am always on https://pion.ly/slack https://pion.ly/slack
- mdonahoe 7y agoI was just digging into this repo on my own, so I did a double-take when your link took me there again! Thanks so much for this.
- Matthias247 7y agoWow, that's a huge effort and codebase! Congratulations for releasing it! I like the consistent coding style, the fact that cmake is used, and that sanitizers are used! I see the title mentions embedded systems. What systems are target of the SDK besides embedded linux? I looked for some kind of platform abstraction layer which would be useful for that, but couldn't find it so far. But actually not sure whether it would be worthwhile at all to run something like WebRTC on a MCU with a RTOS. Protocols that complex often require a ton of dynamic memory, which is not that compatible with the common no-allocation policies on embedded platforms.
- Sean-Der 7y agoThank you! I actually don't know the embedded world at all. My background is WebRTC/userland programming. My answers aren't going to be great, but this is where my learning is at today :) > What systems are target of the SDK besides embedded linux I had a few conversations with embedded devs and they said if I wrote this in Pure C (and avoided GCC-isms) it would work well for. Most of the people I have worked with so far are doing embedded Linux, a few are doing non-Linux but I just get bug reports from them. They tell me their C compiler chokes on this etc... > I looked for some kind of platform abstraction layer We have a library called PiC[0] that does some of that. I don't think it is as low level as you are thinking though! It has portable atomics, and allows people to provide their own allocators stuff like that. > not sure whether it would be worthwhile at all to run something like WebRTC on a MCU with a RTOS Yea I think it would be pretty hard. Getting usrsctp and OpenSSL on those devices seem pretty insane. I have looked at OpenSSL alternatives, but for usrsctp I think we will need to solve it ourselves. I want to explore bringing down the footprint more, but the project is really driven by customer demands and no one has explicitly asked yet. [0] https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/tree/master/kinesis-video-pic/src/utils/src https://github.com/awslabs/amazon-kinesis-video-streams-prod...
- yc-kraln 7y agoThere are a few widely-used embedded SSL stacks, of which mbed TLS is probably one worth targeting or adding support for (wolfSSL is another.) So long as this requires OpenSSL it's gonna basically be stuck in Linux.
- saurik 7y agoI am very curious to understand better how this stacks up against the usual C++ WebRTC implementation when code size isn't an issue, particularly with respect to flexibility. (I work on a project that does VPN services over WebRTC, am doing stuff like WebRTC tunneled over WebRTC, have WebRTC routing over LwIP, and we are working on improving the congestion control algorithms.) My stripped down compile of that library--which admittedly is dropping most of the audio/video functionality as I only care about DataChannels, so maybe if I were doing video I'd be freaking out at the size of the stack (though I would imagine the video codecs would dominate the size, no?)--is clocking in at about 7MB, which is much larger than 200kB but nowhere near the point where I start to care: I mean, the default buffer size per channel in the library is a crazy large 16MB buffer, so I find myself not caring much about the size of my text segment ;P. (Are embedded systems really that constrained these days?) One thing I have enjoyed about the C++ build is that it has tons and tons of knobs for me to mess with in the binding API... I have to reimplement the network detection and am using two separate socket implementations (which is all pretty trivial as just about everything can be subclassed and replaced), I mess with the offers and answers before handing them back/forth through the stack (which is easy as there is an object-oriented model for all of the data structures; I mean, it isn't a great model, but it is at least a model), I am able to easily assign work to occur on specific threads (and have been able to share a signaling thread while splitting networking threads across backends), and I at times reach into the heart of the system (without any code modifications, though with a really hilarious template trick to help me access private C++ class members) to pull up the SCTP stack internals. (Note that with only one arguable-exception we do everything we do currently without any modifications to WebRTC's code.) Is this implementation going to be better for me, in maybe having fewer layers of abstraction between me and the underlying state machines, or worse, due to trying to target "embedded systems" in a programming language that tends to be less flexible (and so between those two considerations maybe leading me to have less ability to customize it without having to rewrite or hack on it directly)? I ask because I know a lot of other people see C and think "ah, finally this is simple", but I have not had any issues with the C++ codebase (the secret is to not even try to use Google's build system: just take all the .cc files you need--and only the ones you need--and link them together). (Oh, maybe even really simple: I bet this isn't going to support Windows?)
- 7y ago
- teleforce 7y agoHi Sean, any recommendations for references or books to learn about webrtc since most of the books on webrtc seem rather old?
- pjmlp 7y agoAt least Amazon seems to care about supporting sanitizers on the build, so I wonder how much it has been validated through fuzzing and similar security validation processes. Which is kind of relevant if such implementation is to be burned into a piece of hardware, that might never see any kind of updates.
- tincholio 7y agoBesides the awesomeness of Sean's work, it's refreshing to see a HN thread where all the comments are so positive!
- teleforce 7y agoIntel has just released Open WebRTC Toolkit (OWT) [1]. It is an end to end audio/video communication development toolkit based on Webrtc, optimized for Intel Architecture. [1] https://01.org/open-webrtc-toolkit https://01.org/open-webrtc-toolkit
- ptsneves 7y agoHello Sean Great work! What are your thoughts on Janus webrtc signaling server and do you consider targeting it? I am building a drone fleet manager with live video and integration with other video sources and have been working with Janus because it allows me to be cloud/premises independent. What are your thoughts on cloud independence for webrtc?