4 ms·

TLDR: WebRTC supports e2e encryption between peers. full mesh p2p videoconferencing is very inefficient for large conferences so an SFU (Selective Forwarding U

TLDR: WebRTC supports e2e encryption between peers. full mesh p2p videoconferencing is very inefficient for large conferences so an SFU (Selective Forwarding Unit) is used - a server acting as a peer that redistributes the streams to call parties. This SFU is basically a middleman - it decrypts the stream, and reincrypts it as it forwards - meaning that there is no e2e encryption. This is a current limitation of webrtc mediastreams. There is discussion underway to fix this with "Insertable Streams", allowing you to transform streams (e.g. insert an additional encryption layer). MY THOUGHTS: Zoom already bypass the usual webrtc mediastreams - instead they use wasm ffmpeg to record from a canvas and send that data over a data-channel to their SFU. It seems to me that they could instead encrypt this data before sending it over the stream, and distribute keys directly between peers via a full mesh - this would give them full e2e. As an aside, the ability to implement something like this is SEVERELY hampered by apples decision not to support webm. I simply do not understand why they refuse to do so. If they supported webm (and the MediaRecorder API) then there would be no need to perform unaccelerated wasm-based patent-encumbered video encoding in the browser in order to provide e2e encrypted media streams via sfu. DISCLAIMER: I don't work for zoom, but I do work on a product that handles e2e webrtc via SFU using similar mechanisms to that I explained above.

aloer 7y ago
What does record from a canvas mean in this context? Is that html canvas? If so, how are they filling it with local screen content?
- supermatt 7y ago
  HTML canvas, yes. They use a webgl context (but you can also do similar with a 2d context) containing a video texture (from the mediastream - video/webcam/screencap/etc). You can find a number of examples by searching "getUserMedia" and "canvas" - for example: https://webrtc.github.io/samples/src/content/getusermedia/canvas/ https://webrtc.github.io/samples/src/content/getusermedia/ca... edit: apologies, here they read the pixel data from the video element, but the principle is the same. One reason for using a canvas intermediary (or even multiple) is to provide multiple sizes, or perform some processing, such as background removal, for example.
- jeremija 7y ago
  I believe it is the HTML canvas. It is very simple to capture a video frame using canvas - see this commented out code: https://github.com/jeremija/peer-calls/blob/6b865a78bbb4e64e981212b0a0e741741f8f9196/src/client/components/Input.tsx#L61-L72 https://github.com/jeremija/peer-calls/blob/6b865a78bbb4e64e...
Aachen 7y ago
I know webrtc works p2p when possible and does some form of encryption, but I didn't know it claims e2e encryption. Is this for real, like, any standard browser implementation can generate keys for e2ee and display them to the user for verification?
- supermatt 7y ago
  Heres some information: https://rtcweb-wg.github.io/security-arch/#rfc.section.4.3 https://rtcweb-wg.github.io/security-arch/#rfc.section.4.3 I dont believe there is a way to retrieve the keys via javascript, but you can get a fingerprint from the SessionDescription. These fingerprints are used to validate the certificates during the DTLS handshake, so displaying those should be sufficient for manual verification. Note that there may be multiple independently encrypted streams per peer, so you may have to display a few. Its not quite as simple as a single fingerprint, usually. I think this is probably an artifact of deriving the SRTP key from the DTLS key. I would much prefer the option to manage SRTP encryption keys independently, which would simplify this somewhat.
- kreetx 7y ago
  Yes, webrtc when done in p2p/mesh, is end-to-end encrypted, even when using TURN servers in between (these are relay servers when direct p2p is not possible). Unfortunately mesh doesn't work out well when it's a conference call (many callers), so MCUs, SFUs, etc come to play.

aloer 7y ago

What does record from a canvas mean in this context? Is that html canvas? If so, how are they filling it with local screen content?

supermatt 7y ago
HTML canvas, yes. They use a webgl context (but you can also do similar with a 2d context) containing a video texture (from the mediastream - video/webcam/screencap/etc). You can find a number of examples by searching "getUserMedia" and "canvas" - for example: https://webrtc.github.io/samples/src/content/getusermedia/canvas/ https://webrtc.github.io/samples/src/content/getusermedia/ca... edit: apologies, here they read the pixel data from the video element, but the principle is the same. One reason for using a canvas intermediary (or even multiple) is to provide multiple sizes, or perform some processing, such as background removal, for example.
jeremija 7y ago
I believe it is the HTML canvas. It is very simple to capture a video frame using canvas - see this commented out code: https://github.com/jeremija/peer-calls/blob/6b865a78bbb4e64e981212b0a0e741741f8f9196/src/client/components/Input.tsx#L61-L72 https://github.com/jeremija/peer-calls/blob/6b865a78bbb4e64e...

Aachen 7y ago

I know webrtc works p2p when possible and does some form of encryption, but I didn't know it claims e2e encryption. Is this for real, like, any standard browser implementation can generate keys for e2ee and display them to the user for verification?

supermatt 7y ago
Heres some information: https://rtcweb-wg.github.io/security-arch/#rfc.section.4.3 https://rtcweb-wg.github.io/security-arch/#rfc.section.4.3 I dont believe there is a way to retrieve the keys via javascript, but you can get a fingerprint from the SessionDescription. These fingerprints are used to validate the certificates during the DTLS handshake, so displaying those should be sufficient for manual verification. Note that there may be multiple independently encrypted streams per peer, so you may have to display a few. Its not quite as simple as a single fingerprint, usually. I think this is probably an artifact of deriving the SRTP key from the DTLS key. I would much prefer the option to manage SRTP encryption keys independently, which would simplify this somewhat.
kreetx 7y ago
Yes, webrtc when done in p2p/mesh, is end-to-end encrypted, even when using TURN servers in between (these are relay servers when direct p2p is not possible). Unfortunately mesh doesn't work out well when it's a conference call (many callers), so MCUs, SFUs, etc come to play.