3 ms·

I'm curious to see how this compares to tarsnap for encrypted backups. One cool thing that people might not know about tarsnap is that you can generate read-onl

by aorth 11y ago

I'm curious to see how this compares to tarsnap for encrypted backups. One cool thing that people might not know about tarsnap is that you can generate read-only subkeys of your main key! https://www.tarsnap.com/man-tarsnap-keymgmt.1.html https://www.tarsnap.com/man-tarsnap-keymgmt.1.html

cperciva 11y ago
I'm also curious. I hadn't seen S3QL before it appeared at the top of HN. Two things jump out at me from a quick glance at the site: 1. "S3QL splits files in blocks of a configurable size (default: 10 MB)" <-- this is quite a large block size and will result in significantly less deduplication than tarsnap's variable-length average 64kB blocks. (On the other hand, large blocks significantly reduce the amount of work and RAM needed; S3QL's tuning probably makes sense for a "live" filesystem.) 2. "all data can AES encrypted with a 256 bit key" <-- leaving aside the grammatical problem, I can't find anything beyond this about key management. At best this means that anyone who can decrypt data can encrypt it, and vice versa; but the usual rule about crypto documentation applies here: If people don't realize that the details matter, they're probably doing something wrong. (Unfortunately it's 1AM and I'm allergic to snakes, or else I would dig into the code to see exactly how their crypto works.)
a3_nm 11y ago
A difference with Tarsnap is that s3ql is free and open-source software (both the client and server), so you can also use it on your own server, your friends' servers, etc. I used s3ql to back up my stuff on servers where I had shell access and disk space but didn't trust root. By contrast, from what I understand, tarsnap requires you to use the official server.
witten 11y ago
Having used both S3QL and Tarsnap.. In my experience, S3QL's use of remote mounted filesystems is fundamentally unsuited to unattended backups. On many occasions, S3QL's mounted filesystem would break mid-backup due to network issues, and stay mounted and unable to recover. Then, from that point forward, no backups would ever work again without manual intervention because the mount point was already open. Additionally, S3QL would periodically issue new releases that didn't support old versions of the file format, or only supported them a set number of releases back. So if you didn't upgrade frequently enough, you'd find yourself with a release that refused to read your existing gigabytes of backup data. And then at that point, you have to do a binary search to find and recompile old releases in the vain attempt to resurrect your data and avoid having to do a full backup from scratch. Bottom line: Stay far away from S3QL and instead use Attic, Borg, or Tarsnap. Also, if you select Attic or Borg, check out Atticmatic: https://torsion.org/atticmatic/ https://torsion.org/atticmatic/
- Nikratio 11y ago
  Regarding the network issues: please file a bug report so that they can be fixed.

by aorth 11y ago

cperciva 11y ago

I'm also curious. I hadn't seen S3QL before it appeared at the top of HN. Two things jump out at me from a quick glance at the site: 1. "S3QL splits files in blocks of a configurable size (default: 10 MB)" <-- this is quite a large block size and will result in significantly less deduplication than tarsnap's variable-length average 64kB blocks. (On the other hand, large blocks significantly reduce the amount of work and RAM needed; S3QL's tuning probably makes sense for a "live" filesystem.) 2. "all data can AES encrypted with a 256 bit key" <-- leaving aside the grammatical problem, I can't find anything beyond this about key management. At best this means that anyone who can decrypt data can encrypt it, and vice versa; but the usual rule about crypto documentation applies here: If people don't realize that the details matter, they're probably doing something wrong. (Unfortunately it's 1AM and I'm allergic to snakes, or else I would dig into the code to see exactly how their crypto works.)

a3_nm 11y ago

A difference with Tarsnap is that s3ql is free and open-source software (both the client and server), so you can also use it on your own server, your friends' servers, etc. I used s3ql to back up my stuff on servers where I had shell access and disk space but didn't trust root. By contrast, from what I understand, tarsnap requires you to use the official server.

witten 11y ago

Having used both S3QL and Tarsnap.. In my experience, S3QL's use of remote mounted filesystems is fundamentally unsuited to unattended backups. On many occasions, S3QL's mounted filesystem would break mid-backup due to network issues, and stay mounted and unable to recover. Then, from that point forward, no backups would ever work again without manual intervention because the mount point was already open. Additionally, S3QL would periodically issue new releases that didn't support old versions of the file format, or only supported them a set number of releases back. So if you didn't upgrade frequently enough, you'd find yourself with a release that refused to read your existing gigabytes of backup data. And then at that point, you have to do a binary search to find and recompile old releases in the vain attempt to resurrect your data and avoid having to do a full backup from scratch. Bottom line: Stay far away from S3QL and instead use Attic, Borg, or Tarsnap. Also, if you select Attic or Borg, check out Atticmatic: https://torsion.org/atticmatic/ https://torsion.org/atticmatic/

Nikratio 11y ago
Regarding the network issues: please file a bug report so that they can be fixed.