14 ms·

How to build large-scale end-to-end encrypted group video calls

371 points by frankjr 5y ago

remus 5y ago
I don't know what their threat model is but it's interesting that they don't seem too bothered about reducing meta data collection potential on the server. I bet you could put together some pretty interesting graphs of who is talking to who, how much they talk and when.
- tptacek 5y ago
  Their messaging substrate is Signal itself, for whatever that's worth, so at least the signaling component of the system should inherit the guarantees Signal already makes. But it's a good question.
- pthatcherg 5y ago
  Author here. The server doesn't know who is in the call. It can't build any graph.
  - codethief 5y ago
    How does this work? I mean, I suppose the server must at least know the IP addresses of the participants?
    - pthatcherg 5y ago
      Yes, it knows the IP addresses of the clients. The other thing it knows is that you have provided a proof that you are a member of a the group conversation associated with the group call (basically a proof that you are allowed to join the call). And that proof is based on Signal's "zkgroup" system. That's it. Once the server has the proof, it forwards packets for you. And it doesn't learning about you from then proof (other than what the proof proves: that you can join).
      - godelski 5y ago
        But isn't knowing the IP a decently good identifier? I know signal is trying to be as trustless as possible, but since IPs don't change often can't this be used to deanonymize people relatively easily? I mean supposing the server became hostile/hijacked?
        codethief 5y ago
        This was already the case before group video calls and, yes, IPs can be used as identifiers. I don't think this can be prevented on a technical level[0] – you will always need to trust the server here. Signal does have a good track record, though: https://signal.org/bigbrother/ https://signal.org/bigbrother/ [0]: Unless you do p2p routing of video calls – with the known downsides regarding traffic, performance and NAT. But even then you would probably still need a server as rendez-vous point.
  - up6w6 5y ago
    I'm interested in a explanation for that too. Signal recently introduced the same sender key method for groups [1] and, correct me if I'm wrong, this is first formal mention of the "Selective Forwarding" method we have. I get that Signal's open-source implementation doesn't store this type of metadata, but is very misleading to see the founder of Signal tweeting that "doesn't have access to the data" [2] while it's not that much better than Whatsapp in the technical side. [1] https://community.signalusers.org/t/sender-key-feature/33599/6 https://community.signalusers.org/t/sender-key-feature/33599... [2] https://twitter.com/moxie/status/1457005910136549379 https://twitter.com/moxie/status/1457005910136549379
  - walrus01 5y ago
    DPI of the network traffic at the ISPs upstream of the server side (Whether ordered by US national security letter or other) would collect some very interesting data on ASN/IP traffic origins and time of day usage patterns, however. Or even something much less than DPI and just high sampling rate netflow.
- deleted 5y ago
  [deleted]
Naac 5y ago
>> There is no off the shelf software that would allow us to support calls of that size while ensuring that all communication is end-to-end encrypted, so we built our own open source Signal Calling Service to do the job But wasn't there Jitsi? [0] I think its great we have competition among Free Software projects so that both can improve. But sometimes I feel like maybe duplicated efforts create two 5/10 solutions. Instead what we really want is one 8/10 solution, or better. [0] https://meet.jit.si/ https://meet.jit.si/
- estaseuropano 5y ago
  While I love jitsi, i don't think it is E2E?
  - dest 5y ago
    AFAIK it's E2E for 1:1 video chats, but not when more are there.
    - bilal4hmed 5y ago
      Jitsi does support e2ee for groups as well https://jitsi.org/e2ee-in-jitsi/ https://jitsi.org/e2ee-in-jitsi/
      - deleted 5y ago
        [deleted]
  - Naac 5y ago
    AFAIK this was a work in progress[0]. I am not sure what the status of this is now. [0] https://jitsi.org/blog/e2ee/ https://jitsi.org/blog/e2ee/
  - jkepler 5y ago
    I think Jitsi group calls can be end to end encrypted, provided all participants use Chromium 83, per https://jitsi.org/security/ https://jitsi.org/security/.
  - 5y ago
wyager 5y ago
How does signal get money to cover costs of running compute-intensive services?
- keewee7 5y ago
  One of the the WhatsApp founders, Brian Acton, donated $100 million to them as an unsecured loan due to be repaid in 2068: https://en.wikipedia.org/wiki/Brian_Acton#Signal https://en.wikipedia.org/wiki/Brian_Acton#Signal https://en.wikipedia.org/wiki/Signal_(software)#Developers_and_funding https://en.wikipedia.org/wiki/Signal_(software)#Developers_a...
  - sorenjan 5y ago
    How long does that last? Telegram uses a few hundred million dollars each year, although they are significantly larger. > As Telegram approaches 500 million active users, many of you are asking the question – who is going to pay to support this growth? After all, more users mean more expenses for traffic and servers. A project of our size needs at least a few hundred million dollars per year to keep going. https://t.me/durov/142 https://t.me/durov/142
    - new_stranger 5y ago
      > needs at least a few hundred million dollars per year to keep going I'm pretty sure that is not server cost. This is probably the standard approach of companies hiring tons of personal and spending tens of thousands or hundreds of thousands on ads every single day.
- sandstrom 5y ago
  They recently added support for in-app donations: https://www.theverge.com/2021/12/2/22814934/signal-launches-in-app-sustainer-program-donations-messaging-secure https://www.theverge.com/2021/12/2/22814934/signal-launches-... I hope they'll take it a step further and require payment for certain functionality (maybe video calls?, or desktop client support?).
johnisgood 5y ago
Great, now they should just stop using telephone numbers as identifiers.
- maxwell 5y ago
  What do you suggest?
  - sam_lowry_ 5y ago
    A login+password, like in IRC.
    - tptacek 5y ago
      IRC tracks metadata serverside!
      - johnisgood 5y ago
        I do not think that OP was referring to implementing it the same, or even in a similar way, but to use a username/password pair. OP is free to correct me if I am wrong though. In any case, elimination of metadata done right is the way Ricochet[1] does it. The recipient can calculate the sender's contact ID based on the public key, and authenticate it by verifying the signature on the request. This proves that the sender can publish the hidden service represented by their contact ID. You can read more about it here: https://github.com/ricochet-im/ricochet/blob/master/doc/design.md https://github.com/ricochet-im/ricochet/blob/master/doc/desi...! [1] https://github.com/ricochet-im/ricochet https://github.com/ricochet-im/ricochet (maintained fork: https://github.com/blueprint-freespeech/ricochet-refresh https://github.com/blueprint-freespeech/ricochet-refresh)
  - zamadatix 5y ago
    Signal has had standard usernames on the roadmap for years.
  - johnisgood 5y ago
    Usernames work. You could even use UUIDs these days as QR is an increasingly common way of sharing data. But yeah, usernames would be a great improvement.
1vuio0pswjnm7 5y ago
"Full mesh: Each call participant sends its media (audio and video) directly to each other call participant. This works for very small calls, but does not scale to many participants. Most people just don't have an Internet connection fast enough to send 40 copies of their video at the same time. Server mixing: Each call participant sends its media to a server. The server "mixes" the media together and sends it to each participant. This works with many participants, but is not compatible with end-to-end encryption because it requires that the server be able to view and alter the media. Selective Forwarding: Each participant sends its media to a server. The server "forwards" the media to other participants without viewing or altering it. This works with many participants, and is compatible with end-to-end-encryption." Imagine an end user who is interested in "very small calls" with friends and family. She is not interested in communicating to an infinitely large audience ("broadcasting"). She never has group calls on Signal with 40 people, even ones with Grandmother and full extended family are well under 40 people. We have to use our imagination because this user does not actually exist. The imaginary user reads this blog post and she thinks to herself "Full mesh sounds like the best design. There is less/no reliance on a third party, traffic does not need to be sent to an additional third party server." With full mesh, there is no need to mention the caveat "without viewing or altering it", or selectively choosing not to forward it to certain recipients. Full mesh seems to give the user the most control and require the least dependence on third party servers (not necessarily "no dependence", but the least dependance). Then she reads this line: "Because Signal must have end-to-end encryption and scale to many participants, we use selective forwarding."
- prophesi 5y ago
  Just an FYI full mesh would still require communicating with a third-party server, at the very least for initial networking when joining/leaving a group call. The whole point of E2E encryption is so that passing data through a third party shouldn't matter in the first place. And lastly, even when you have just a 1:1 video chat, sending and receiving full resolution/quality multimedia can still be way too much for some peoples' internet connections. UX is extremely important for Signal, as unreliable video chat is a surefire way for those less caring about privacy to hop back over to a privacy-violating alternative. I feel sorry for those working on bringing security/privacy to everyone, as they have to appease power users and privacy absolutists, along with one's grandmother and the TikTok generation.
sneak 5y ago
They have the bandwidth for relaying video streams to 40 people but won't let me send full res jpegs in 1:1 messages? And no, I can't just rebuild my client, because I'm on iOS and non-official builds won't receive push notifications from the official developers.
- Vinnl 5y ago
  That's not really related to this article, but I can select photo quality if I send a photo on Android. Appears to have been added in May.
  - sneak 5y ago
    The article specifically mentions that they operate the infrastructure for relaying encrypted video streams for up to 40 participants. I can also select media quality on iOS. My options are "compressed way too much" and "compressed too much". I assume you have the same options. I would like to be able to attach images as files and have them come though unmodified. It is a general purpose communications tool, it should not be editorializing over my attachments. I use Signal to communicate privately with my attorney. Why does anyone think tampering with evidence in transit is okay? Apple also doesn't support open source in the App Store, so I can't fix the problem myself.
    - hiq 5y ago
      > I would like to be able to attach images as files and have them come though unmodified. It is a general purpose communications tool, it should not be editorializing over my attachments. It's a usability trade-off, it's better for a lot of people to compress these at least slightly. The main real problem is awareness, people have to know about this so that they can put their files into .zip instead of plain .jpg.
      - sneak 5y ago
        They added the ability to make it recompress less, but did not include a "please do not alter my files" option.
benlivengood 5y ago
To scale to thousands (is this even useful?) of e2e users build a tree of participants who can remix each other's video. Pick a handy mixing ratio like 4:1 or 9:1 (a square helps, since they compose nicely if downscaled to a grid vs. active talker stays fullscreen) and nodes with the highest bandwidth and lowest latency take M-1 streams and add it to their own to make an M:1 mix which can be forwarded to a node closer to the root which produces another M:1 stream, and the root sends a single mixed stream down the tree until every participant has the mix. Max bandwidth at each node is M down and M up. Minimal spanning tree with max M edges per node recomputed as participants leave and join. Build 3 or 4 distinct trees and leave the connections open for more rapid switching if intermediate nodes stop participating.
- wolverine876 5y ago
  > To scale to thousands (is this even useful?) Think of a citizens' forum, or people watching a sporting event together.
JoeAltmaier 5y ago
Oh this all brings back memories, of Sococo in the 2000's. We faced all these problems and had similar solutions to them all. We even had a rapidly adapting network make-and-break recovery layer. You unplug your laptop from a wired connection, switch to wireless - we recovered in milliseconds. You heard barely a click. The encryption issue is fun - we had a rotate-key message in-band. The receiver loaded new keys and tried them in sequence to ease the turnover time - out-of-order packets etc could make it ambiguous for a short while which key to use. A cache and aging keys out made it work pretty well. Remixing on user stations proved to be problematic (mentioned elsewhere on this thread). You'd think if 6 people at one site were conferencing with a dozen elsewhere, you could elect one at each site to mix-and-forward. But corporate networks made it hard to determine who was 'adjacent' - they were often layered and without uPNP (is that what the router protocol is called?) you couldn't tell if somebody at the next desk was even in your company. We had up to 100 people in a conference, and our enter-the-conference time was on the order of 100ms. Click into an all-hands, and be able to hear everybody before you finger left the mouse button. It was wonderful. Sococo today is a sad shadow of that. They went open-source and lost all our IP instantly. Just another WebRTC client last I knew.
- narush 5y ago
  > They went open-source and lost all our IP instantly. Can you explain what this means? Like - other people copied your work? Genuinely wondering, OSS noob here...
  - JoeAltmaier 5y ago
    There was little or nothing in WebRTC to match what we'd spend 5 years creating. So they were back to 1-5 people in a conference, with 1-3 second connect times, and no resilience to network changes. The excuse they gave was "We can't rely on 6 people in Iowa for our core IP". So they switched to some open source mix node that was the pet project of 2 guys in Italy. Two academics, who gave it hardly any attention. And it had zero IP; just a collection of APIs stitched together to give you the impression of having a mix node. We said all that at the time. But such was the power of the magic words "Open Source" that it all bounced off their mental shields.
BitPirate 5y ago
Are there any plans to add VP9 support?
- pthatcherg 5y ago
  Author here. It's on the long list of things to do, yes. But we have many things before it on the list :).
  - KennyBlanken 5y ago
    Hopefully proper client sync and cross-platform migration are on the list. Two of the biggest issues for me with Signal have always been the incredibly poor client sync which I've heard is intentionally crippled to the point of being useless because of some hand-wave-y privacy reasons. Also, the inability to move between Android and iOS. I lost years worth of conversations when I switched from Android. Worse, one day my Signal client on Android started crashing on startup. I emailed support@ with logcat files and never heard a peep. That lack of support for probably one of the most serious use issues one could have left a permanent bad taste in my mouth. I don't care whatever paranoid argument someone at Signal has for why I can't sync my desktop client to my phone. Stop being paternalistic and let me decide if the tradeoff security-wise is worth it to me, please.
kitkat_new 5y ago
Next step: decentralizing encrypted group calls [0] [0]: https://2021.commcon.xyz/talks/extending-matrix-s-e2ee-calls-to-multiparty https://2021.commcon.xyz/talks/extending-matrix-s-e2ee-calls...
- a-dub 5y ago
  decentralization would be interesting for the purposes of keeping operating costs down in some cases. can imagine some sort of mode for just a few participants on mobile networks only where all the high bandwidth stuff remains fully peer to peer.
  - pthatcherg 5y ago
    Author here. We are considering making 2-person group calls switch to p2p under the hood. I don't know if that that will ever expand to 3 or more.
    - a-dub 5y ago
      cool! i suppose even with limited metadata with e2ee, you could probably do a quick study to get a sense for how many calls could switch (how many are 2 party, how many aren't behind nats, etc) and maybe even get a sense for how much could be saved in operation costs. interesting privacy considerations now that i think of it though. p2p would trivially leak the existence of a call between two devices, but i suppose that given the right taps it would be possible to correlate through server streams based on timing data anyhow.
      - codethief 5y ago
        > but i suppose that given the right taps it would be possible to correlate through server streams based on timing data anyhow Probably, but it'd be much harder I suppose.
    - Arathorn 5y ago
      pthatcherg: this is super cool; on the Matrix side we hadn't spotted that you'd written your own SFU. Would you have any objection to us trying to use it as a decentralised Matrix SFU, as per the design in https://github.com/matrix-org/matrix-doc/blob/matthew/group-voip/proposals/3401-group-voip.md https://github.com/matrix-org/matrix-doc/blob/matthew/group-...? (Obviously we'd do so within the constraints of AGPLv3).
kodablah 5y ago
I have been quite happy with some of my SFU code on top of the Rust wrapper for Mediasoup[0] which I understand from this post doesn't have congestion control. I would love to give this a try, but I fear including a library w/ AGPL license. The license choice is not worth hashing out here, and I'm sure Signal's goal isn't necessarily developer adoption compared to openness, but it will hurt adoption. 0 - https://docs.rs/mediasoup/latest/mediasoup/ https://docs.rs/mediasoup/latest/mediasoup/
- pthatcherg 5y ago
  Author here. MediaSoup was one of the two SFUs that did seem have congestion control at the time. Sorry if that wasn't clear.
Thaxll 5y ago
Looking at the loop, what prevent a slow client to block the iteration? I understand that it's UDP so fire and forget but shoudn't there be a thread pool to handle that in parallel?
- pthatcherg 5y ago
  Author here. What is in the article is a very simplified version of the code. The code that runs in production uses many threads. See here: https://github.com/signalapp/Signal-Calling-Service/blob/main/src/udp_server.rs https://github.com/signalapp/Signal-Calling-Service/blob/mai...
tadeegan 5y ago
As always, incredible work signal team!
shams93 5y ago
Skip the SFU and use media paging with the most recent 1.0 webrtc api you may not need an SFU, in fact the SFU could be a major budget destroyer when p2p meshes using participant paging is much better.
- gfodor 5y ago
  Can you elaborate on this?
e12e 5y ago
Isn't it somewhat disconcerting that the sequence of media isn't protected by authenticated e2e? Essentially any delay and re-ordering is invisible?
bexp 5y ago
It would be great if author could elaborate on performance bottlenecks of existing SFU implementations and posted some benchmarks or even better some profiling graphs. Given that reference webrtc is written in C++ I wonder what percentage of performance gains was attributed to rust ?
wolverine876 5y ago
As someone who doesn't develop video codecs, the bandwidth consumed by sending multiple video resolutions seems a poor trade-off for the bandwidth saved by receiving a reduced resolution. (This isn't a criticism, but curiosity - I'm certain the developers thought of it.) Is it possible (or is it already done) to compress the data by sending the lowest resolution in full, and then for each higher resolution, send only the delta between it and the one immediately beneath it (i.e., resolution layer 5 would contain only the delta between it and resolution layer 4)? Perhaps because of E2EE the final output would have to be reconstituted on the fly client-side (i.e., because the server lacks visibility into the data), which might be asking too much. Then again, the client is generating all these resolutions.
- TD-Linux 5y ago
  The tradeoff makes sense mostly for many-party calls - for a 1:1 call sending multiple resolutions is a waste, but for a many participant call the downlink bandwidth usage becomes much larger than the uplink. You layer idea is called SVC and is mentioned in the article. But VP8 doesn't support it, it is waiting on them to upgrade to VP9 or AV1.
  - wolverine876 5y ago
    > The tradeoff makes sense mostly for many-party calls - for a 1:1 call sending multiple resolutions is a waste, but for a many participant call the downlink bandwidth usage becomes much larger than the uplink. Excellent insight, thanks. > You layer idea is called SVC and is mentioned in the article. But VP8 doesn't support it, it is waiting on them to upgrade to VP9 or AV1. Thanks. I noticed SVC in the article but somehow overlooked that detail.
vlovich123 5y ago
One thing that’s not clear to me (and maybe this has to do with how vp8 works?). In h264/h265 you have I frames (which are standalone) and P frames. If I recall correctly, P frames require having decoded the immediately preceding frame or video corruption occurs until a new I frame is received. With SFU selectively choosing which resolution stream to send a frame from next, how does this work? Is the client encoding each frame as an I frame? Is there feedback from the SFU to tell the client which stream its using so that the client knows to emit I frames (and thus generating a few corrupted frames until the stream is resynced)? Does vp8 work differently?
rvz 5y ago
This engineering post is all for nothing given that Signal is still a centralized messaging service and still requires your phone number to sign in/up which means users are still vulnerable to SIM swapping attacks. Hence this, the service can be 'shutdown' at any time, making Signal even more uninteresting to use. (Probably affected by the Log4J RCE stuff) So from me, No thanks and no deal.
- DarylZero 5y ago
  In theory the code could be made reusable, but it probably was structured in a very much un-reusable way and will never be available to other projects. Everything has to be some giant monolithic integration because if you engineered it properly as a reusable library, it wouldn't provide any business benefit to the authors. It's almost a form of closed source by obfuscation through over-integrated design.
mderazon 5y ago
Challenge is to find 40 friends who use Signal. *( I love signal)

371 points by frankjr 5y ago

I don't know what their threat model is but it's interesting that they don't seem too bothered about reducing meta data collection potential on the server. I bet you could put together some pretty interesting graphs of who is talking to who, how much they talk and when.

tptacek 5y ago
Their messaging substrate is Signal itself, for whatever that's worth, so at least the signaling component of the system should inherit the guarantees Signal already makes. But it's a good question.
pthatcherg 5y ago
Author here. The server doesn't know who is in the call. It can't build any graph.
- codethief 5y ago
  How does this work? I mean, I suppose the server must at least know the IP addresses of the participants?
  - pthatcherg 5y ago
    Yes, it knows the IP addresses of the clients. The other thing it knows is that you have provided a proof that you are a member of a the group conversation associated with the group call (basically a proof that you are allowed to join the call). And that proof is based on Signal's "zkgroup" system. That's it. Once the server has the proof, it forwards packets for you. And it doesn't learning about you from then proof (other than what the proof proves: that you can join).
    - godelski 5y ago
      But isn't knowing the IP a decently good identifier? I know signal is trying to be as trustless as possible, but since IPs don't change often can't this be used to deanonymize people relatively easily? I mean supposing the server became hostile/hijacked?
      - codethief 5y ago
        This was already the case before group video calls and, yes, IPs can be used as identifiers. I don't think this can be prevented on a technical level[0] – you will always need to trust the server here. Signal does have a good track record, though: https://signal.org/bigbrother/ https://signal.org/bigbrother/ [0]: Unless you do p2p routing of video calls – with the known downsides regarding traffic, performance and NAT. But even then you would probably still need a server as rendez-vous point.
- up6w6 5y ago
  I'm interested in a explanation for that too. Signal recently introduced the same sender key method for groups [1] and, correct me if I'm wrong, this is first formal mention of the "Selective Forwarding" method we have. I get that Signal's open-source implementation doesn't store this type of metadata, but is very misleading to see the founder of Signal tweeting that "doesn't have access to the data" [2] while it's not that much better than Whatsapp in the technical side. [1] https://community.signalusers.org/t/sender-key-feature/33599/6 https://community.signalusers.org/t/sender-key-feature/33599... [2] https://twitter.com/moxie/status/1457005910136549379 https://twitter.com/moxie/status/1457005910136549379
- walrus01 5y ago
  DPI of the network traffic at the ISPs upstream of the server side (Whether ordered by US national security letter or other) would collect some very interesting data on ASN/IP traffic origins and time of day usage patterns, however. Or even something much less than DPI and just high sampling rate netflow.
deleted 5y ago
[deleted]

Naac 5y ago

>> There is no off the shelf software that would allow us to support calls of that size while ensuring that all communication is end-to-end encrypted, so we built our own open source Signal Calling Service to do the job But wasn't there Jitsi? [0] I think its great we have competition among Free Software projects so that both can improve. But sometimes I feel like maybe duplicated efforts create two 5/10 solutions. Instead what we really want is one 8/10 solution, or better. [0] https://meet.jit.si/ https://meet.jit.si/

estaseuropano 5y ago
While I love jitsi, i don't think it is E2E?
- dest 5y ago
  AFAIK it's E2E for 1:1 video chats, but not when more are there.
  - bilal4hmed 5y ago
    Jitsi does support e2ee for groups as well https://jitsi.org/e2ee-in-jitsi/ https://jitsi.org/e2ee-in-jitsi/
    - deleted 5y ago
      [deleted]
- Naac 5y ago
  AFAIK this was a work in progress[0]. I am not sure what the status of this is now. [0] https://jitsi.org/blog/e2ee/ https://jitsi.org/blog/e2ee/
- jkepler 5y ago
  I think Jitsi group calls can be end to end encrypted, provided all participants use Chromium 83, per https://jitsi.org/security/ https://jitsi.org/security/.
- 5y ago

wyager 5y ago

How does signal get money to cover costs of running compute-intensive services?

keewee7 5y ago
One of the the WhatsApp founders, Brian Acton, donated $100 million to them as an unsecured loan due to be repaid in 2068: https://en.wikipedia.org/wiki/Brian_Acton#Signal https://en.wikipedia.org/wiki/Brian_Acton#Signal https://en.wikipedia.org/wiki/Signal_(software)#Developers_and_funding https://en.wikipedia.org/wiki/Signal_(software)#Developers_a...
- sorenjan 5y ago
  How long does that last? Telegram uses a few hundred million dollars each year, although they are significantly larger. > As Telegram approaches 500 million active users, many of you are asking the question – who is going to pay to support this growth? After all, more users mean more expenses for traffic and servers. A project of our size needs at least a few hundred million dollars per year to keep going. https://t.me/durov/142 https://t.me/durov/142
  - new_stranger 5y ago
    > needs at least a few hundred million dollars per year to keep going I'm pretty sure that is not server cost. This is probably the standard approach of companies hiring tons of personal and spending tens of thousands or hundreds of thousands on ads every single day.
sandstrom 5y ago
They recently added support for in-app donations: https://www.theverge.com/2021/12/2/22814934/signal-launches-in-app-sustainer-program-donations-messaging-secure https://www.theverge.com/2021/12/2/22814934/signal-launches-... I hope they'll take it a step further and require payment for certain functionality (maybe video calls?, or desktop client support?).

johnisgood 5y ago

Great, now they should just stop using telephone numbers as identifiers.

maxwell 5y ago
What do you suggest?
- sam_lowry_ 5y ago
  A login+password, like in IRC.
  - tptacek 5y ago
    IRC tracks metadata serverside!
    - johnisgood 5y ago
      I do not think that OP was referring to implementing it the same, or even in a similar way, but to use a username/password pair. OP is free to correct me if I am wrong though. In any case, elimination of metadata done right is the way Ricochet[1] does it. The recipient can calculate the sender's contact ID based on the public key, and authenticate it by verifying the signature on the request. This proves that the sender can publish the hidden service represented by their contact ID. You can read more about it here: https://github.com/ricochet-im/ricochet/blob/master/doc/design.md https://github.com/ricochet-im/ricochet/blob/master/doc/desi...! [1] https://github.com/ricochet-im/ricochet https://github.com/ricochet-im/ricochet (maintained fork: https://github.com/blueprint-freespeech/ricochet-refresh https://github.com/blueprint-freespeech/ricochet-refresh)
- zamadatix 5y ago
  Signal has had standard usernames on the roadmap for years.
- johnisgood 5y ago
  Usernames work. You could even use UUIDs these days as QR is an increasingly common way of sharing data. But yeah, usernames would be a great improvement.

1vuio0pswjnm7 5y ago

"Full mesh: Each call participant sends its media (audio and video) directly to each other call participant. This works for very small calls, but does not scale to many participants. Most people just don't have an Internet connection fast enough to send 40 copies of their video at the same time. Server mixing: Each call participant sends its media to a server. The server "mixes" the media together and sends it to each participant. This works with many participants, but is not compatible with end-to-end encryption because it requires that the server be able to view and alter the media. Selective Forwarding: Each participant sends its media to a server. The server "forwards" the media to other participants without viewing or altering it. This works with many participants, and is compatible with end-to-end-encryption." Imagine an end user who is interested in "very small calls" with friends and family. She is not interested in communicating to an infinitely large audience ("broadcasting"). She never has group calls on Signal with 40 people, even ones with Grandmother and full extended family are well under 40 people. We have to use our imagination because this user does not actually exist. The imaginary user reads this blog post and she thinks to herself "Full mesh sounds like the best design. There is less/no reliance on a third party, traffic does not need to be sent to an additional third party server." With full mesh, there is no need to mention the caveat "without viewing or altering it", or selectively choosing not to forward it to certain recipients. Full mesh seems to give the user the most control and require the least dependence on third party servers (not necessarily "no dependence", but the least dependance). Then she reads this line: "Because Signal must have end-to-end encryption and scale to many participants, we use selective forwarding."

prophesi 5y ago
Just an FYI full mesh would still require communicating with a third-party server, at the very least for initial networking when joining/leaving a group call. The whole point of E2E encryption is so that passing data through a third party shouldn't matter in the first place. And lastly, even when you have just a 1:1 video chat, sending and receiving full resolution/quality multimedia can still be way too much for some peoples' internet connections. UX is extremely important for Signal, as unreliable video chat is a surefire way for those less caring about privacy to hop back over to a privacy-violating alternative. I feel sorry for those working on bringing security/privacy to everyone, as they have to appease power users and privacy absolutists, along with one's grandmother and the TikTok generation.

sneak 5y ago

They have the bandwidth for relaying video streams to 40 people but won't let me send full res jpegs in 1:1 messages? And no, I can't just rebuild my client, because I'm on iOS and non-official builds won't receive push notifications from the official developers.

Vinnl 5y ago
That's not really related to this article, but I can select photo quality if I send a photo on Android. Appears to have been added in May.
- sneak 5y ago
  The article specifically mentions that they operate the infrastructure for relaying encrypted video streams for up to 40 participants. I can also select media quality on iOS. My options are "compressed way too much" and "compressed too much". I assume you have the same options. I would like to be able to attach images as files and have them come though unmodified. It is a general purpose communications tool, it should not be editorializing over my attachments. I use Signal to communicate privately with my attorney. Why does anyone think tampering with evidence in transit is okay? Apple also doesn't support open source in the App Store, so I can't fix the problem myself.
  - hiq 5y ago
    > I would like to be able to attach images as files and have them come though unmodified. It is a general purpose communications tool, it should not be editorializing over my attachments. It's a usability trade-off, it's better for a lot of people to compress these at least slightly. The main real problem is awareness, people have to know about this so that they can put their files into .zip instead of plain .jpg.
    - sneak 5y ago
      They added the ability to make it recompress less, but did not include a "please do not alter my files" option.

benlivengood 5y ago

To scale to thousands (is this even useful?) of e2e users build a tree of participants who can remix each other's video. Pick a handy mixing ratio like 4:1 or 9:1 (a square helps, since they compose nicely if downscaled to a grid vs. active talker stays fullscreen) and nodes with the highest bandwidth and lowest latency take M-1 streams and add it to their own to make an M:1 mix which can be forwarded to a node closer to the root which produces another M:1 stream, and the root sends a single mixed stream down the tree until every participant has the mix. Max bandwidth at each node is M down and M up. Minimal spanning tree with max M edges per node recomputed as participants leave and join. Build 3 or 4 distinct trees and leave the connections open for more rapid switching if intermediate nodes stop participating.

wolverine876 5y ago
> To scale to thousands (is this even useful?) Think of a citizens' forum, or people watching a sporting event together.

JoeAltmaier 5y ago

Oh this all brings back memories, of Sococo in the 2000's. We faced all these problems and had similar solutions to them all. We even had a rapidly adapting network make-and-break recovery layer. You unplug your laptop from a wired connection, switch to wireless - we recovered in milliseconds. You heard barely a click. The encryption issue is fun - we had a rotate-key message in-band. The receiver loaded new keys and tried them in sequence to ease the turnover time - out-of-order packets etc could make it ambiguous for a short while which key to use. A cache and aging keys out made it work pretty well. Remixing on user stations proved to be problematic (mentioned elsewhere on this thread). You'd think if 6 people at one site were conferencing with a dozen elsewhere, you could elect one at each site to mix-and-forward. But corporate networks made it hard to determine who was 'adjacent' - they were often layered and without uPNP (is that what the router protocol is called?) you couldn't tell if somebody at the next desk was even in your company. We had up to 100 people in a conference, and our enter-the-conference time was on the order of 100ms. Click into an all-hands, and be able to hear everybody before you finger left the mouse button. It was wonderful. Sococo today is a sad shadow of that. They went open-source and lost all our IP instantly. Just another WebRTC client last I knew.

narush 5y ago
> They went open-source and lost all our IP instantly. Can you explain what this means? Like - other people copied your work? Genuinely wondering, OSS noob here...
- JoeAltmaier 5y ago
  There was little or nothing in WebRTC to match what we'd spend 5 years creating. So they were back to 1-5 people in a conference, with 1-3 second connect times, and no resilience to network changes. The excuse they gave was "We can't rely on 6 people in Iowa for our core IP". So they switched to some open source mix node that was the pet project of 2 guys in Italy. Two academics, who gave it hardly any attention. And it had zero IP; just a collection of APIs stitched together to give you the impression of having a mix node. We said all that at the time. But such was the power of the magic words "Open Source" that it all bounced off their mental shields.

BitPirate 5y ago

Are there any plans to add VP9 support?

pthatcherg 5y ago
Author here. It's on the long list of things to do, yes. But we have many things before it on the list :).
- KennyBlanken 5y ago
  Hopefully proper client sync and cross-platform migration are on the list. Two of the biggest issues for me with Signal have always been the incredibly poor client sync which I've heard is intentionally crippled to the point of being useless because of some hand-wave-y privacy reasons. Also, the inability to move between Android and iOS. I lost years worth of conversations when I switched from Android. Worse, one day my Signal client on Android started crashing on startup. I emailed support@ with logcat files and never heard a peep. That lack of support for probably one of the most serious use issues one could have left a permanent bad taste in my mouth. I don't care whatever paranoid argument someone at Signal has for why I can't sync my desktop client to my phone. Stop being paternalistic and let me decide if the tradeoff security-wise is worth it to me, please.

kitkat_new 5y ago

Next step: decentralizing encrypted group calls [0] [0]: https://2021.commcon.xyz/talks/extending-matrix-s-e2ee-calls-to-multiparty https://2021.commcon.xyz/talks/extending-matrix-s-e2ee-calls...

a-dub 5y ago
decentralization would be interesting for the purposes of keeping operating costs down in some cases. can imagine some sort of mode for just a few participants on mobile networks only where all the high bandwidth stuff remains fully peer to peer.
- pthatcherg 5y ago
  Author here. We are considering making 2-person group calls switch to p2p under the hood. I don't know if that that will ever expand to 3 or more.
  - a-dub 5y ago
    cool! i suppose even with limited metadata with e2ee, you could probably do a quick study to get a sense for how many calls could switch (how many are 2 party, how many aren't behind nats, etc) and maybe even get a sense for how much could be saved in operation costs. interesting privacy considerations now that i think of it though. p2p would trivially leak the existence of a call between two devices, but i suppose that given the right taps it would be possible to correlate through server streams based on timing data anyhow.
    - codethief 5y ago
      > but i suppose that given the right taps it would be possible to correlate through server streams based on timing data anyhow Probably, but it'd be much harder I suppose.
  - Arathorn 5y ago
    pthatcherg: this is super cool; on the Matrix side we hadn't spotted that you'd written your own SFU. Would you have any objection to us trying to use it as a decentralised Matrix SFU, as per the design in https://github.com/matrix-org/matrix-doc/blob/matthew/group-voip/proposals/3401-group-voip.md https://github.com/matrix-org/matrix-doc/blob/matthew/group-...? (Obviously we'd do so within the constraints of AGPLv3).

kodablah 5y ago

I have been quite happy with some of my SFU code on top of the Rust wrapper for Mediasoup[0] which I understand from this post doesn't have congestion control. I would love to give this a try, but I fear including a library w/ AGPL license. The license choice is not worth hashing out here, and I'm sure Signal's goal isn't necessarily developer adoption compared to openness, but it will hurt adoption. 0 - https://docs.rs/mediasoup/latest/mediasoup/ https://docs.rs/mediasoup/latest/mediasoup/

pthatcherg 5y ago
Author here. MediaSoup was one of the two SFUs that did seem have congestion control at the time. Sorry if that wasn't clear.

Thaxll 5y ago

Looking at the loop, what prevent a slow client to block the iteration? I understand that it's UDP so fire and forget but shoudn't there be a thread pool to handle that in parallel?

pthatcherg 5y ago
Author here. What is in the article is a very simplified version of the code. The code that runs in production uses many threads. See here: https://github.com/signalapp/Signal-Calling-Service/blob/main/src/udp_server.rs https://github.com/signalapp/Signal-Calling-Service/blob/mai...

tadeegan 5y ago

As always, incredible work signal team!

shams93 5y ago

Skip the SFU and use media paging with the most recent 1.0 webrtc api you may not need an SFU, in fact the SFU could be a major budget destroyer when p2p meshes using participant paging is much better.

gfodor 5y ago
Can you elaborate on this?

e12e 5y ago

Isn't it somewhat disconcerting that the sequence of media isn't protected by authenticated e2e? Essentially any delay and re-ordering is invisible?

bexp 5y ago

It would be great if author could elaborate on performance bottlenecks of existing SFU implementations and posted some benchmarks or even better some profiling graphs. Given that reference webrtc is written in C++ I wonder what percentage of performance gains was attributed to rust ?

wolverine876 5y ago

As someone who doesn't develop video codecs, the bandwidth consumed by sending multiple video resolutions seems a poor trade-off for the bandwidth saved by receiving a reduced resolution. (This isn't a criticism, but curiosity - I'm certain the developers thought of it.) Is it possible (or is it already done) to compress the data by sending the lowest resolution in full, and then for each higher resolution, send only the delta between it and the one immediately beneath it (i.e., resolution layer 5 would contain only the delta between it and resolution layer 4)? Perhaps because of E2EE the final output would have to be reconstituted on the fly client-side (i.e., because the server lacks visibility into the data), which might be asking too much. Then again, the client is generating all these resolutions.

TD-Linux 5y ago
The tradeoff makes sense mostly for many-party calls - for a 1:1 call sending multiple resolutions is a waste, but for a many participant call the downlink bandwidth usage becomes much larger than the uplink. You layer idea is called SVC and is mentioned in the article. But VP8 doesn't support it, it is waiting on them to upgrade to VP9 or AV1.
- wolverine876 5y ago
  > The tradeoff makes sense mostly for many-party calls - for a 1:1 call sending multiple resolutions is a waste, but for a many participant call the downlink bandwidth usage becomes much larger than the uplink. Excellent insight, thanks. > You layer idea is called SVC and is mentioned in the article. But VP8 doesn't support it, it is waiting on them to upgrade to VP9 or AV1. Thanks. I noticed SVC in the article but somehow overlooked that detail.

vlovich123 5y ago

One thing that’s not clear to me (and maybe this has to do with how vp8 works?). In h264/h265 you have I frames (which are standalone) and P frames. If I recall correctly, P frames require having decoded the immediately preceding frame or video corruption occurs until a new I frame is received. With SFU selectively choosing which resolution stream to send a frame from next, how does this work? Is the client encoding each frame as an I frame? Is there feedback from the SFU to tell the client which stream its using so that the client knows to emit I frames (and thus generating a few corrupted frames until the stream is resynced)? Does vp8 work differently?

rvz 5y ago

This engineering post is all for nothing given that Signal is still a centralized messaging service and still requires your phone number to sign in/up which means users are still vulnerable to SIM swapping attacks. Hence this, the service can be 'shutdown' at any time, making Signal even more uninteresting to use. (Probably affected by the Log4J RCE stuff) So from me, No thanks and no deal.

DarylZero 5y ago
In theory the code could be made reusable, but it probably was structured in a very much un-reusable way and will never be available to other projects. Everything has to be some giant monolithic integration because if you engineered it properly as a reusable library, it wouldn't provide any business benefit to the authors. It's almost a form of closed source by obfuscation through over-integrated design.

mderazon 5y ago

Challenge is to find 40 friends who use Signal. *( I love signal)