5 ms·

Public Signal Backups Testing

78 points by blendergeek 1y ago

ggm 1y ago
Prepare for the UK requests for mandatory key escrow into the data. Which I have high confidence signal will refuse to do. I merely observe that there's a duty cycle here across data which states and providers have to dance through each time.
godelski 1y ago
This is pretty awesome and an exciting feature. But I want to make 2 critiques here. 1) I think Signal is really bad at communicating with its community. Especially being an open source project I think it is important to be open in communication. I wish they wrote more blog posts and were announcing these things through blog posts (I really miss those blog posts, even the non-Moxie ones. They made it feel more communal and like you understood Signal's vision. Ultimately, that builds trust, which is ironically necessary when building trustless systems). Creates a good way to succinctly explain what the feature is, the end goal, and so on. You can easily add a cross link to any forum discussions. But jesus fucking christ, I really hate these private communities. No one likes making random accounts just to submit bug reports and it is a little insulting to make devs do it when GitHub exists (I can get why they do this but there might just be no optimal solution...). But man, the Signal community is particularly bad and off-putting... I'll add that this is an extra pain point being a security app. It should be expected that Signal users are suspicious of Signal. That should even be encouraged! But lack of communication often breeds conspiracies. People not knowing even a high level road map start believing that Signal is doing nothing while asking for money. It's a small team, so of course it is slower, but just a bit of transparency can do a lot to mitigate this. You don't need full blown PR, but PR to the tech nerds does seem necessary at this stage (can become general public when your average tech nerd is convinced Signal is more secure than Telegram and understand that Signal and Matrix are not solving the same problems). Right now it is the tech nerds that spread the conspiracies and the infighting just ends up making apps like WhatsApp more inviting. We're usually bickering over technical mostly non-issue things[0] 2) The post mentions > The ability to dynamically offload media so that Signal takes up less space on your phone, while still letting you download that media on the fly if you scroll back. I can tell you that the vast majority of my storage in Signal is created through dupes. As far as I can tell, every time you forward an image or other piece of media it creates a duplicate[1]. I'd get pretty good storage savings if these were soft links to one another or a COW system was used (think like BTRFS[2], but you could get the same effect without that filesystem[3]). Is there a security issue with this? If so, can someone explain? This kinda circles back to #1 because it is pretty hard to get answers to these types of things from the community as ultimately you frequently end up with results like asking on Reddit. Community answers are great for naive and unnuanced questions but the moment any technicality is brought in. But my point here is kinda about trying to better reach out to the community. I get that listening to users is noisy, but truth is that Signal's lack of metadata means they have fewer insights into user desires and concerns. Fortunately, Signal has more technically minded users than most apps, but there's no real good communication path with them. Honestly, I think there are dozens of good ideas in their community forums that go missed. Big reason I hate these community forums is that they also use popularity as a proxy for user desire. But requiring login makes that noisy as well as many features are going to be things people want but don't know they want, especially when there's some technical aspect involved. Here's a few examples: [4] "Airdrop": User presented an "Airdrop" like feature which benefits them, reduces Signal's bandwidth costs, and could ultimately create a pathway forward to decentralization if they decide to go that way. [5] Link Sanitization: This is straight up a privacy feature! You know when you share a YouTube link? Strip everything after the "?" because that's just tracking data. (You could solve the false positive issue by a default setting to sanitize or not and long press to get unsanitized link. Plenty of solutions to that). But Firefox and other privacy preserving platforms already have some solution to this. Signal really does need this to meet its own goals, and even naive users benefit from just a more visually appealing link (shorter) Both of these are things that I think most users would want or enjoy having, they both further Signal's main mission, YET no "average user" is really going to see these as things they want until they actually have them. Over the years I've seen plenty of ideas like this and even in a more technical space like Signal's community, these are going to be missed while things like Stickers and Social Media like features will rise to the top. It's good to get that information about the community but it's important to recognize how highly beneficial ideas can be missed. Hell, [5] isn't even too difficult to implement! (It's not trivial, but it's a very doable feature and imo has a large impact. Though I'm biased because I manually sanitize links before sending) [0] Guarantee to see a reply doing this. Which is fine, this is HN, its a space where we can do technical bickering. [1] When I export data, I get unique dupes and when I delete data my storage responds appropriately. Could be how storage is determined, but I strongly lean towards these being unique copies. [2] https://btrfs.readthedocs.io/en/latest/Introduction.html https://btrfs.readthedocs.io/en/latest/Introduction.html [3] If any Android or iOS engineers are reading this, please, for the love of god give us this. Should make security better too since you can do subvolumes and encrypt those. I'm sure Signal would love to get containerized and encrypted subvolumes. Hell, fucking containerize each app! Big security win. (I'm sure there are issues and I genuinely would love to hear what they are. Please educate me, this is not my domain but I'd like to know more) [4] https://community.signalusers.org/t/signal-airdrop/37402 https://community.signalusers.org/t/signal-airdrop/37402 [5] https://community.signalusers.org/t/clean-sent-links-strip-tracking-utm-parameters/16467 https://community.signalusers.org/t/clean-sent-links-strip-t...
- mdaniel 1y ago
  > But jesus fucking christ, I really hate these private communities. Also, I don't know if Discourse was dropped in its head as a baby but only through some determined use of the mouse wheel could one possibly discover that there are ... I don't even know ... unlimited additional pages of discussion hiding in that thread. Infinite scroll wasn't designed to be taken literally, holy shit At least with "Page 1 of 8675309" one can know in advance that they should pull up a chair. I guess the UX pattern is "hover over the '1h ago' URL to see the page number, n00b" But, back on topic: > We chose not to exclude all disappearing messages because uh-huh, so kind of defeats the purpose of disappearing if they're preserved, eh?
  - godelski 1y ago
    Discord is only marginally better than dedicated forums. Honestly, I'm not even sure it is... But boy am I annoyed at how frequently that is used too. I don't know if misspelled on purpose, but I think the original name is a more accurate description than the typo (Discourse). Because it sure does create a lot of discord (disagreement and lack of harmony) You might as well go to the Arch Linux forums or Stack Overflow. Where someone will berate you for bringing up an already discussed topic that is difficult to search for. And to rub salt into the wound, they almost never provide the cross-link... Back on topic (kinda): Man, I really do wish Signal would allow you to pin or bookmark messages. It sure beats the cluttered chaos of scrolling, searching, or forwarding to Notes To Self (which lacks a cross-link back but at least gives you precise search queries).
    - mdaniel 1y ago
      Yes, and only recently did I learn that Discord has a 100 join limit, so yea for everyone in the universe locking themselves in an anti-automation limited platform. As show-stoppingly annoying as joining every single Slack workspace in the universe is, at least they didn't ban my email after 100 workspaces I actually sponsor Zulip because that's the world I want to live in: sane threading, public HTML views of the chats, Apache 2 licensed, and a "we host it for you" for Open Source communities --- Back in the olden days, I actually would have submitted a PR to introduce that new behavior, but their "fuck you, it's our project" taught me that it's source available more than open source for any meaningful version of that
zeckalpha 1y ago
Back it up now so when the encryption is broken, it can be read!
- godelski 1y ago
  https://signal.org/blog/pqxdh/ https://signal.org/blog/pqxdh/ https://signal.org/docs/specifications/pqxdh/ https://signal.org/docs/specifications/pqxdh/
  - cassonmars 1y ago
    The backup is secured with "a strong key", implying that all PFS guarantees go out the window regardless of the PFS algorithm used to send the messages in the first place. Signal had great guarantees by how they both enforced a single client and was limited largely to screenshots as backups, now you'll never know if the person you're talking to has a full backup in the cloud, with metadata to match the actual conversation times, destroying the repudiability (i.e. plausible deniability) feature.
    - DaSHacka 1y ago
      Whats to say they didn't take screenshots of the conversation that got backed up to Google Photos or iCloud anyway? I dont think this changes anything, in regards to a malicious(/incompetent) recipient.
      - zeckalpha 1y ago
        Screenshots can easily be forged.
- gruez 1y ago
  If you think they've gone rogue and are working with the NSA or whatever, why can't they be doing the same thing with your e2e messages while in transit? What do they gain by getting it through backups?
  - 1y ago
XorNot 1y ago
So is this going to be self-hostable? Because what I'd really like is backups to go to my own server and definitely things like the media offload.
- theknarf 1y ago
  The article says that local-backups are still going to be a thing, and that it'll use the new format (so that you can restore ex an Android backup on iPhone and the other way around). Presumably you'll have to bring your own solution for storing the backup, but if its just a file / archive then you can store it however you store the rest of your backups.
  - XorNot 1y ago
    The issue isn't "just a file" - it's whether that file can be streamed off my device. Signal's backups are easily one of the largest files, and I don't want 3 copies of it on my phone (which is what backup currently does). I really want the backups to stream off to my own version of the backup service, and do the neat deferred media thing which would be great for keeping long range archives (i.e. by letting them get picked up by my server's regular snapshot to backblaze).
    - igor47 1y ago
      I currently configure signal to back up into a directory that's synced to my server via syncthing
      - XorNot 1y ago
        Which is what I used to do and the problems were Signal keeps two copies of the file by default and puts them on your device memory. So your device ends up having 3 copies of the same data, on the same device. This is rather impractical if you have a lot of content in Signal, which I do because my family use it extensively to send photos and videos (which is the sort of thing you generally really want to backup).
gojomo 1y ago
The post claims with regard to the cost of a cloud backup that "Local backups still exist" - but that's a lie, there's no local backup option on iOS.
- ezfe 1y ago
  iOS can back up to a Mac or PC.
  - SequoiaHope 1y ago
    Sadly not Linux tho.
    - eigenspace 1y ago
      If you sync an iPhone to a Linux machine, you can then save your backups on Linux.
    - Nextgrid 1y ago
      "libimobiledevice" has CLI tools that can backup/restore iOS devices on Linux.
      - gojomo 1y ago
        Do such backups retain Signal message history - unlike Apple's native local backups to MacOS?
        Nextgrid 1y ago
        This uses the same protocol as macOS, so same outcome. However (not sure if it helps in this case), keep in mind that different data is retained depending on whether the backup is encrypted or not (the tool supports encrypted backups too). So maybe check with encryption enabled and see if it carries over Signal messages?
        1y ago
sureglymop 1y ago
Good effort, but wouldn't it be better if such backups were split into n parts and distributed among n independent providers using something like shamir secret sharing where a backup could then be restored with k out of n parts? I trust Signal but I find this quite basic compared to just how much thought/work went into the Signal protocol comparatively.
snthpy 1y ago
The following two features are interesting to me. Where can I find more info on these? * Backups are structured so that the Signal service cannot associate a backup with a given user. * Payments are structured so that the Signal service cannot associate a payment with a given user.
- protimewaster 1y ago
  I'm curious about this as well. How can they enforce a quota without being able to associate the data with a user?
  - igor47 1y ago
    Maybe similar to how mullvad VPN does it? You get a secondary identifier distinct from your signal username, and payments and quota are associated with that identifier?

78 points by blendergeek 1y ago

ggm 1y ago

Prepare for the UK requests for mandatory key escrow into the data. Which I have high confidence signal will refuse to do. I merely observe that there's a duty cycle here across data which states and providers have to dance through each time.

godelski 1y ago

This is pretty awesome and an exciting feature. But I want to make 2 critiques here. 1) I think Signal is really bad at communicating with its community. Especially being an open source project I think it is important to be open in communication. I wish they wrote more blog posts and were announcing these things through blog posts (I really miss those blog posts, even the non-Moxie ones. They made it feel more communal and like you understood Signal's vision. Ultimately, that builds trust, which is ironically necessary when building trustless systems). Creates a good way to succinctly explain what the feature is, the end goal, and so on. You can easily add a cross link to any forum discussions. But jesus fucking christ, I really hate these private communities. No one likes making random accounts just to submit bug reports and it is a little insulting to make devs do it when GitHub exists (I can get why they do this but there might just be no optimal solution...). But man, the Signal community is particularly bad and off-putting... I'll add that this is an extra pain point being a security app. It should be expected that Signal users are suspicious of Signal. That should even be encouraged! But lack of communication often breeds conspiracies. People not knowing even a high level road map start believing that Signal is doing nothing while asking for money. It's a small team, so of course it is slower, but just a bit of transparency can do a lot to mitigate this. You don't need full blown PR, but PR to the tech nerds does seem necessary at this stage (can become general public when your average tech nerd is convinced Signal is more secure than Telegram and understand that Signal and Matrix are not solving the same problems). Right now it is the tech nerds that spread the conspiracies and the infighting just ends up making apps like WhatsApp more inviting. We're usually bickering over technical mostly non-issue things[0] 2) The post mentions > The ability to dynamically offload media so that Signal takes up less space on your phone, while still letting you download that media on the fly if you scroll back. I can tell you that the vast majority of my storage in Signal is created through dupes. As far as I can tell, every time you forward an image or other piece of media it creates a duplicate[1]. I'd get pretty good storage savings if these were soft links to one another or a COW system was used (think like BTRFS[2], but you could get the same effect without that filesystem[3]). Is there a security issue with this? If so, can someone explain? This kinda circles back to #1 because it is pretty hard to get answers to these types of things from the community as ultimately you frequently end up with results like asking on Reddit. Community answers are great for naive and unnuanced questions but the moment any technicality is brought in. But my point here is kinda about trying to better reach out to the community. I get that listening to users is noisy, but truth is that Signal's lack of metadata means they have fewer insights into user desires and concerns. Fortunately, Signal has more technically minded users than most apps, but there's no real good communication path with them. Honestly, I think there are dozens of good ideas in their community forums that go missed. Big reason I hate these community forums is that they also use popularity as a proxy for user desire. But requiring login makes that noisy as well as many features are going to be things people want but don't know they want, especially when there's some technical aspect involved. Here's a few examples: [4] "Airdrop": User presented an "Airdrop" like feature which benefits them, reduces Signal's bandwidth costs, and could ultimately create a pathway forward to decentralization if they decide to go that way. [5] Link Sanitization: This is straight up a privacy feature! You know when you share a YouTube link? Strip everything after the "?" because that's just tracking data. (You could solve the false positive issue by a default setting to sanitize or not and long press to get unsanitized link. Plenty of solutions to that). But Firefox and other privacy preserving platforms already have some solution to this. Signal really does need this to meet its own goals, and even naive users benefit from just a more visually appealing link (shorter) Both of these are things that I think most users would want or enjoy having, they both further Signal's main mission, YET no "average user" is really going to see these as things they want until they actually have them. Over the years I've seen plenty of ideas like this and even in a more technical space like Signal's community, these are going to be missed while things like Stickers and Social Media like features will rise to the top. It's good to get that information about the community but it's important to recognize how highly beneficial ideas can be missed. Hell, [5] isn't even too difficult to implement! (It's not trivial, but it's a very doable feature and imo has a large impact. Though I'm biased because I manually sanitize links before sending) [0] Guarantee to see a reply doing this. Which is fine, this is HN, its a space where we can do technical bickering. [1] When I export data, I get unique dupes and when I delete data my storage responds appropriately. Could be how storage is determined, but I strongly lean towards these being unique copies. [2] https://btrfs.readthedocs.io/en/latest/Introduction.html https://btrfs.readthedocs.io/en/latest/Introduction.html [3] If any Android or iOS engineers are reading this, please, for the love of god give us this. Should make security better too since you can do subvolumes and encrypt those. I'm sure Signal would love to get containerized and encrypted subvolumes. Hell, fucking containerize each app! Big security win. (I'm sure there are issues and I genuinely would love to hear what they are. Please educate me, this is not my domain but I'd like to know more) [4] https://community.signalusers.org/t/signal-airdrop/37402 https://community.signalusers.org/t/signal-airdrop/37402 [5] https://community.signalusers.org/t/clean-sent-links-strip-tracking-utm-parameters/16467 https://community.signalusers.org/t/clean-sent-links-strip-t...

mdaniel 1y ago
> But jesus fucking christ, I really hate these private communities. Also, I don't know if Discourse was dropped in its head as a baby but only through some determined use of the mouse wheel could one possibly discover that there are ... I don't even know ... unlimited additional pages of discussion hiding in that thread. Infinite scroll wasn't designed to be taken literally, holy shit At least with "Page 1 of 8675309" one can know in advance that they should pull up a chair. I guess the UX pattern is "hover over the '1h ago' URL to see the page number, n00b" But, back on topic: > We chose not to exclude all disappearing messages because uh-huh, so kind of defeats the purpose of disappearing if they're preserved, eh?
- godelski 1y ago
  Discord is only marginally better than dedicated forums. Honestly, I'm not even sure it is... But boy am I annoyed at how frequently that is used too. I don't know if misspelled on purpose, but I think the original name is a more accurate description than the typo (Discourse). Because it sure does create a lot of discord (disagreement and lack of harmony) You might as well go to the Arch Linux forums or Stack Overflow. Where someone will berate you for bringing up an already discussed topic that is difficult to search for. And to rub salt into the wound, they almost never provide the cross-link... Back on topic (kinda): Man, I really do wish Signal would allow you to pin or bookmark messages. It sure beats the cluttered chaos of scrolling, searching, or forwarding to Notes To Self (which lacks a cross-link back but at least gives you precise search queries).
  - mdaniel 1y ago
    Yes, and only recently did I learn that Discord has a 100 join limit, so yea for everyone in the universe locking themselves in an anti-automation limited platform. As show-stoppingly annoying as joining every single Slack workspace in the universe is, at least they didn't ban my email after 100 workspaces I actually sponsor Zulip because that's the world I want to live in: sane threading, public HTML views of the chats, Apache 2 licensed, and a "we host it for you" for Open Source communities --- Back in the olden days, I actually would have submitted a PR to introduce that new behavior, but their "fuck you, it's our project" taught me that it's source available more than open source for any meaningful version of that

zeckalpha 1y ago

Back it up now so when the encryption is broken, it can be read!

godelski 1y ago
https://signal.org/blog/pqxdh/ https://signal.org/blog/pqxdh/ https://signal.org/docs/specifications/pqxdh/ https://signal.org/docs/specifications/pqxdh/
- cassonmars 1y ago
  The backup is secured with "a strong key", implying that all PFS guarantees go out the window regardless of the PFS algorithm used to send the messages in the first place. Signal had great guarantees by how they both enforced a single client and was limited largely to screenshots as backups, now you'll never know if the person you're talking to has a full backup in the cloud, with metadata to match the actual conversation times, destroying the repudiability (i.e. plausible deniability) feature.
  - DaSHacka 1y ago
    Whats to say they didn't take screenshots of the conversation that got backed up to Google Photos or iCloud anyway? I dont think this changes anything, in regards to a malicious(/incompetent) recipient.
    - zeckalpha 1y ago
      Screenshots can easily be forged.
gruez 1y ago
If you think they've gone rogue and are working with the NSA or whatever, why can't they be doing the same thing with your e2e messages while in transit? What do they gain by getting it through backups?
- 1y ago

XorNot 1y ago

So is this going to be self-hostable? Because what I'd really like is backups to go to my own server and definitely things like the media offload.

theknarf 1y ago
The article says that local-backups are still going to be a thing, and that it'll use the new format (so that you can restore ex an Android backup on iPhone and the other way around). Presumably you'll have to bring your own solution for storing the backup, but if its just a file / archive then you can store it however you store the rest of your backups.
- XorNot 1y ago
  The issue isn't "just a file" - it's whether that file can be streamed off my device. Signal's backups are easily one of the largest files, and I don't want 3 copies of it on my phone (which is what backup currently does). I really want the backups to stream off to my own version of the backup service, and do the neat deferred media thing which would be great for keeping long range archives (i.e. by letting them get picked up by my server's regular snapshot to backblaze).
  - igor47 1y ago
    I currently configure signal to back up into a directory that's synced to my server via syncthing
    - XorNot 1y ago
      Which is what I used to do and the problems were Signal keeps two copies of the file by default and puts them on your device memory. So your device ends up having 3 copies of the same data, on the same device. This is rather impractical if you have a lot of content in Signal, which I do because my family use it extensively to send photos and videos (which is the sort of thing you generally really want to backup).

gojomo 1y ago

The post claims with regard to the cost of a cloud backup that "Local backups still exist" - but that's a lie, there's no local backup option on iOS.

ezfe 1y ago
iOS can back up to a Mac or PC.
- SequoiaHope 1y ago
  Sadly not Linux tho.
  - eigenspace 1y ago
    If you sync an iPhone to a Linux machine, you can then save your backups on Linux.
  - Nextgrid 1y ago
    "libimobiledevice" has CLI tools that can backup/restore iOS devices on Linux.
    - gojomo 1y ago
      Do such backups retain Signal message history - unlike Apple's native local backups to MacOS?
      - Nextgrid 1y ago
        This uses the same protocol as macOS, so same outcome. However (not sure if it helps in this case), keep in mind that different data is retained depending on whether the backup is encrypted or not (the tool supports encrypted backups too). So maybe check with encryption enabled and see if it carries over Signal messages?
        1y ago

sureglymop 1y ago

Good effort, but wouldn't it be better if such backups were split into n parts and distributed among n independent providers using something like shamir secret sharing where a backup could then be restored with k out of n parts? I trust Signal but I find this quite basic compared to just how much thought/work went into the Signal protocol comparatively.

snthpy 1y ago

The following two features are interesting to me. Where can I find more info on these? * Backups are structured so that the Signal service cannot associate a backup with a given user. * Payments are structured so that the Signal service cannot associate a payment with a given user.

protimewaster 1y ago
I'm curious about this as well. How can they enforce a quota without being able to associate the data with a user?
- igor47 1y ago
  Maybe similar to how mullvad VPN does it? You get a secondary identifier distinct from your signal username, and payments and quota are associated with that identifier?