8 ms·

So what's a good Linux consumer solution? We here are all capable of rolling our own hacks, sure. But I want a good clean friendly product I don't have to confi

So what's a good Linux consumer solution? We here are all capable of rolling our own hacks, sure. But I want a good clean friendly product I don't have to configure or think about. The real challenge in these things is the UX for restoring files. (For home-grown options, I've been using rsnapshot for years. But it really works best on a local disk and the rsync-over-ssh hacks I've done are the opposite of "clean consumer solution".)

X86BSD 9y ago
Tarsnap.
- dixie_land 9y ago
  Haha was just typing the same thing you beat me to it
- dchest 9y ago
  Agree! It now even has a nice GUI https://github.com/Tarsnap/tarsnap-gui https://github.com/Tarsnap/tarsnap-gui
- slavik81 9y ago
  Tarsnap is not reasonable alternative for most users. Consider that I was paying $5/mo for 200GB of storage from CrashPlan. From tarsnap, that would be $50/mo. And, frankly, my measly 200GB isn't much data. A 2TB HDD costs under $100 and could cost $500/mo to back up with tarsnap. It would still have been $5/mo with CrashPlan. I get that Colin is basically just charging the S3 costs plus a small margin, but the reality is that S3 is far too expensive to be used to store low-value consumer data.
  - X86BSD 9y ago
    I backup quite a bit and after deduplication with tarsnap it's not nearly as much.
    - slavik81 9y ago
      While I'm sure there would be minor savings, the price differential is so large that the data would need to be >90% duplicate to be cost-competitive even for small data sets.
    - res0nat0r 9y ago
      I've been using duplicacy and it supports a bunch of different remote endpoints. https://duplicacy.com/ https://duplicacy.com/
    - skrause 9y ago
      I hear that a lot about tarsnap, do you all mainly back up VMs or log files? The majority of my most important backup data is my photos. I don't see how tarsnap could duplicate (or even further compress) that.
      - zippergz 9y ago
        I evaluated tarsnap for my photos a while back, and this was the conclusion I came to. I would have been paying for basically the entire size of my photo library, which is quite large.
    - SimonPStevens 9y ago
      That's kind of another problem though. No way to predict costs in advance. All you can do is know the upper bound. I have approx 2.5tb on CrashPlan, so that gives me an upper bound of $625 per month. Maybe it might be 50% lower when deduped, maybe more, maybe less. Who knows. Not really a pricing model I can make a decision on. Tarsnap looks great for certain use cases. But it doesn't really make sense for home users with a large amount of data.
      - X86BSD 9y ago
        You can do a dry run with Tarsnap to see how much it would transfer and how much it would deduplicate before ever running for Realz(tm). Using the --dry-run flag.
      - 9y ago
  - deleted 9y ago
    [deleted]
  - votepaunchy 9y ago
    The most expensive S3 storage tier is $0.023/GB/month, or $4.60/month for 200 GB. That's quite a markup to $50.
    - slavik81 9y ago
      It seems the price of S3 has dropped significantly over the years while tarsnap has remained more or less constant. I guess my statement was out of date.
- paol 9y ago
  One of the options I looked up after getting the email from Crashplan. Unfortunately the price is crazy, it would cost over $500/month for the amount of data I have in Crashplan for $5/month. (Granted that product is going away, but even the upgrade path to Crashplan Pro will only cost $10/month.)
  - X86BSD 9y ago
    You can only say that after you have tried a dry run of Tarsnap to see what your actual usage would be. You can't take the rate Colin charges at face value because that is not what you will actually wind up paying.
stryk 9y ago
There's rsync.net which very explicitly supports rsync/zfs over SSH
- LeoPanthera 9y ago
  I use rsync.net with borg. Works great.
  - Freaky 9y ago
    They have a dedicated account for borg/attic: http://rsync.net/products/attic.html http://rsync.net/products/attic.html There's also restic, which works similarly, but has native support for cloud storage: https://restic.github.io/ https://restic.github.io/ - sadly it's claims of being efficient are rather undermined by the continued lack of compression support.
- metalliqaz 9y ago
  I have an account with rsync.net and I like them a lot for their Unix-friendly services. However, I unfortunately have to recommend that you NOT use rsync.net if you are on a Comcast/Xfinity cable modem. rsync.net has some kind of problem with the "Powerboost" feature. The first 10 megabytes of a transfer are fast as hell, then it drops to ~150kB/s. Have fun backing up gigabytes at that speed. There is something wrong with their traffic shaping. After much back-and-forth their tech support basically threw up their hands and gave up.
  - retSava 9y ago
    Do Comcast throttle torrents for you? Perhaps the traffic to rsync in some way looks like torrents to them. Or like spam/DOS or something like that. Or like a competitor. The things I've read about Comcast are not pretty, alas.
    - metalliqaz 9y ago
      No, my torrents routinely saturate my connection. It's definitely rsync.net. In the past they have tweaked their settings and been able to get it to work, but it has never stuck.
cknight 9y ago
The last one I tried was SpiderOak. I've got to be honest though, I wasn't a huge fan then, and it still looks like things haven't improved all that much recently: https://ctrl.blog/entry/review-spideroak https://ctrl.blog/entry/review-spideroak
ValentineC 9y ago
Some folks at r/DataHoarder recommend Duplicati: https://www.reddit.com/r/DataHoarder/comments/6vbcok/crashplan_for_home_ending_as_of_today_new_and/dlywgx7/ https://www.reddit.com/r/DataHoarder/comments/6vbcok/crashpl...
- zippergz 9y ago
  I'm pretty sure that whatever solution the r/datahoarder people go for is exactly the one I DON'T want to choose. Those are the exact people who make attractive pricing unsustainable.
  - metalliqaz 9y ago
    Maybe you don't like the way they abuse their accounts, but it doesn't mean their tools are bad.
    - zippergz 9y ago
      It does mean the tools are more likely to be driven out of business, though.
  - larrybud 9y ago
    Duplicati is backup software only; you bring-your-own-(local or cloud)-storage.
DCKing 9y ago
Duplicity [1] is a free software project that creates incremental backups and encrypts them using GPG. It can use Backblaze B2 [2] as a remote synchronization option, which when I looked at options last year was also the cheapest cloud storage provider anywhere (if this has changed, let me know!). Duplicity also supports a very wide array of other cloud storage services if you'd like to use another one [1]. The killer feature Duplicity has for consumer use is that it's probably the only (?) backup program on Linux that has an actual quality desktop GUI called Déja Dup [3] that's included by default in Ubuntu. It works somewhat similar to how Time Machine works on Macs. But if you need to backup headless systems the Duplicity command line interface works fine as well. When it comes to backup features it's not the most powerful tool compared to some other solutions like Attic or Borg, but I think the GUI and out-of-the-box (encrypted) integration with cloud services make it one of the most user friendly solutions on Linux. [1]: http://duplicity.nongnu.org/ http://duplicity.nongnu.org/ [2]: https://www.backblaze.com/b2/cloud-storage.html https://www.backblaze.com/b2/cloud-storage.html [3]: https://www.linux.com/learn/total-system-backup-and-recall-deja-dup https://www.linux.com/learn/total-system-backup-and-recall-d... [3b:] EDIT: Also Duplicati has a nice web GUI. Worth looking into as well.
- lima 9y ago
  While it works well, Duplicity is rather slow and only supports expensive full-incremental-full backup cycles. Modern incremental-forever backup tools like borg[1] (fork of Attic[2]) are much faster since they are based on block hashing, which also gets you deduplication for free. Basically, a backup is a set of hashes - this means that you can selectively delete or retain old backups without having to merge them. I built a >50 TB enterprise backup cluster with borg and it works extremely well. [1]: https://borgbackup.readthedocs.io/en/stable/ https://borgbackup.readthedocs.io/en/stable/ [2]: https://www.stavros.io/posts/holy-grail-backups/ https://www.stavros.io/posts/holy-grail-backups/
  - oconnore 9y ago
    Borg is amazing, except for the lack of public key encryption, which makes it unusable: https://github.com/borgbackup/borg/issues/672 https://github.com/borgbackup/borg/issues/672
nfriedly 9y ago
I've been really happy with Rockstor - http://rockstor.com/ http://rockstor.com/ - it's essentially like FreeNAS, but with Linux, Btrfs, and lower RAM requirements. I have a mirrored 2x6TB setup that all of my computers back up to. It shows up as an Apple TimeMachine, so MacBook backup and restore is easy; and then it also some media storage with automatic nightly/weekly/monthly snapshots - the monthly ones hang around for a year, the others are cleaned up sooner. For those, I have to un-hide the snapshot folders, but then I can just copy-paste any deleted files that I want to restore. It definitely takes some setup work, but I've been pretty happy overall. That server is in my basement, my work laptop also has CrashPlan for off-site backups, but that's a business plan, so it won't be affected by this. I do need to figure out an off-site backup for the rest of it...
paulcole 9y ago
I think the ship for "good clean friendly product I don't have to configure or think about" sailed with the mention of consumer Linux.
akulbe 9y ago
I don't think I'd call it a "consumer" solution really, but it works very well, once you set it up. rsync.net It's not the cheapest either, but I never had issues when I was using them.
XorNot 9y ago
I have a strong suspicion Linux support doesn't exist, because that user category are going to buy 1 unlimited plan, put it on their 20TB home SAN, and well...20TB of cloud storage is not cheap (price it out on backblaze business).

X86BSD 9y ago

Tarsnap.

dixie_land 9y ago
Haha was just typing the same thing you beat me to it
dchest 9y ago
Agree! It now even has a nice GUI https://github.com/Tarsnap/tarsnap-gui https://github.com/Tarsnap/tarsnap-gui
slavik81 9y ago
Tarsnap is not reasonable alternative for most users. Consider that I was paying $5/mo for 200GB of storage from CrashPlan. From tarsnap, that would be $50/mo. And, frankly, my measly 200GB isn't much data. A 2TB HDD costs under $100 and could cost $500/mo to back up with tarsnap. It would still have been $5/mo with CrashPlan. I get that Colin is basically just charging the S3 costs plus a small margin, but the reality is that S3 is far too expensive to be used to store low-value consumer data.
- X86BSD 9y ago
  I backup quite a bit and after deduplication with tarsnap it's not nearly as much.
  - slavik81 9y ago
    While I'm sure there would be minor savings, the price differential is so large that the data would need to be >90% duplicate to be cost-competitive even for small data sets.
  - res0nat0r 9y ago
    I've been using duplicacy and it supports a bunch of different remote endpoints. https://duplicacy.com/ https://duplicacy.com/
  - skrause 9y ago
    I hear that a lot about tarsnap, do you all mainly back up VMs or log files? The majority of my most important backup data is my photos. I don't see how tarsnap could duplicate (or even further compress) that.
    - zippergz 9y ago
      I evaluated tarsnap for my photos a while back, and this was the conclusion I came to. I would have been paying for basically the entire size of my photo library, which is quite large.
  - SimonPStevens 9y ago
    That's kind of another problem though. No way to predict costs in advance. All you can do is know the upper bound. I have approx 2.5tb on CrashPlan, so that gives me an upper bound of $625 per month. Maybe it might be 50% lower when deduped, maybe more, maybe less. Who knows. Not really a pricing model I can make a decision on. Tarsnap looks great for certain use cases. But it doesn't really make sense for home users with a large amount of data.
    - X86BSD 9y ago
      You can do a dry run with Tarsnap to see how much it would transfer and how much it would deduplicate before ever running for Realz(tm). Using the --dry-run flag.
    - 9y ago
- deleted 9y ago
  [deleted]
- votepaunchy 9y ago
  The most expensive S3 storage tier is $0.023/GB/month, or $4.60/month for 200 GB. That's quite a markup to $50.
  - slavik81 9y ago
    It seems the price of S3 has dropped significantly over the years while tarsnap has remained more or less constant. I guess my statement was out of date.
paol 9y ago
One of the options I looked up after getting the email from Crashplan. Unfortunately the price is crazy, it would cost over $500/month for the amount of data I have in Crashplan for $5/month. (Granted that product is going away, but even the upgrade path to Crashplan Pro will only cost $10/month.)
- X86BSD 9y ago
  You can only say that after you have tried a dry run of Tarsnap to see what your actual usage would be. You can't take the rate Colin charges at face value because that is not what you will actually wind up paying.

stryk 9y ago

There's rsync.net which very explicitly supports rsync/zfs over SSH

LeoPanthera 9y ago
I use rsync.net with borg. Works great.
- Freaky 9y ago
  They have a dedicated account for borg/attic: http://rsync.net/products/attic.html http://rsync.net/products/attic.html There's also restic, which works similarly, but has native support for cloud storage: https://restic.github.io/ https://restic.github.io/ - sadly it's claims of being efficient are rather undermined by the continued lack of compression support.
metalliqaz 9y ago
I have an account with rsync.net and I like them a lot for their Unix-friendly services. However, I unfortunately have to recommend that you NOT use rsync.net if you are on a Comcast/Xfinity cable modem. rsync.net has some kind of problem with the "Powerboost" feature. The first 10 megabytes of a transfer are fast as hell, then it drops to ~150kB/s. Have fun backing up gigabytes at that speed. There is something wrong with their traffic shaping. After much back-and-forth their tech support basically threw up their hands and gave up.
- retSava 9y ago
  Do Comcast throttle torrents for you? Perhaps the traffic to rsync in some way looks like torrents to them. Or like spam/DOS or something like that. Or like a competitor. The things I've read about Comcast are not pretty, alas.
  - metalliqaz 9y ago
    No, my torrents routinely saturate my connection. It's definitely rsync.net. In the past they have tweaked their settings and been able to get it to work, but it has never stuck.

cknight 9y ago

The last one I tried was SpiderOak. I've got to be honest though, I wasn't a huge fan then, and it still looks like things haven't improved all that much recently: https://ctrl.blog/entry/review-spideroak https://ctrl.blog/entry/review-spideroak

ValentineC 9y ago

Some folks at r/DataHoarder recommend Duplicati: https://www.reddit.com/r/DataHoarder/comments/6vbcok/crashplan_for_home_ending_as_of_today_new_and/dlywgx7/ https://www.reddit.com/r/DataHoarder/comments/6vbcok/crashpl...

zippergz 9y ago
I'm pretty sure that whatever solution the r/datahoarder people go for is exactly the one I DON'T want to choose. Those are the exact people who make attractive pricing unsustainable.
- metalliqaz 9y ago
  Maybe you don't like the way they abuse their accounts, but it doesn't mean their tools are bad.
  - zippergz 9y ago
    It does mean the tools are more likely to be driven out of business, though.
- larrybud 9y ago
  Duplicati is backup software only; you bring-your-own-(local or cloud)-storage.

DCKing 9y ago

Duplicity [1] is a free software project that creates incremental backups and encrypts them using GPG. It can use Backblaze B2 [2] as a remote synchronization option, which when I looked at options last year was also the cheapest cloud storage provider anywhere (if this has changed, let me know!). Duplicity also supports a very wide array of other cloud storage services if you'd like to use another one [1]. The killer feature Duplicity has for consumer use is that it's probably the only (?) backup program on Linux that has an actual quality desktop GUI called Déja Dup [3] that's included by default in Ubuntu. It works somewhat similar to how Time Machine works on Macs. But if you need to backup headless systems the Duplicity command line interface works fine as well. When it comes to backup features it's not the most powerful tool compared to some other solutions like Attic or Borg, but I think the GUI and out-of-the-box (encrypted) integration with cloud services make it one of the most user friendly solutions on Linux. [1]: http://duplicity.nongnu.org/ http://duplicity.nongnu.org/ [2]: https://www.backblaze.com/b2/cloud-storage.html https://www.backblaze.com/b2/cloud-storage.html [3]: https://www.linux.com/learn/total-system-backup-and-recall-deja-dup https://www.linux.com/learn/total-system-backup-and-recall-d... [3b:] EDIT: Also Duplicati has a nice web GUI. Worth looking into as well.

lima 9y ago
While it works well, Duplicity is rather slow and only supports expensive full-incremental-full backup cycles. Modern incremental-forever backup tools like borg[1] (fork of Attic[2]) are much faster since they are based on block hashing, which also gets you deduplication for free. Basically, a backup is a set of hashes - this means that you can selectively delete or retain old backups without having to merge them. I built a >50 TB enterprise backup cluster with borg and it works extremely well. [1]: https://borgbackup.readthedocs.io/en/stable/ https://borgbackup.readthedocs.io/en/stable/ [2]: https://www.stavros.io/posts/holy-grail-backups/ https://www.stavros.io/posts/holy-grail-backups/
- oconnore 9y ago
  Borg is amazing, except for the lack of public key encryption, which makes it unusable: https://github.com/borgbackup/borg/issues/672 https://github.com/borgbackup/borg/issues/672

nfriedly 9y ago

I've been really happy with Rockstor - http://rockstor.com/ http://rockstor.com/ - it's essentially like FreeNAS, but with Linux, Btrfs, and lower RAM requirements. I have a mirrored 2x6TB setup that all of my computers back up to. It shows up as an Apple TimeMachine, so MacBook backup and restore is easy; and then it also some media storage with automatic nightly/weekly/monthly snapshots - the monthly ones hang around for a year, the others are cleaned up sooner. For those, I have to un-hide the snapshot folders, but then I can just copy-paste any deleted files that I want to restore. It definitely takes some setup work, but I've been pretty happy overall. That server is in my basement, my work laptop also has CrashPlan for off-site backups, but that's a business plan, so it won't be affected by this. I do need to figure out an off-site backup for the rest of it...

paulcole 9y ago

I think the ship for "good clean friendly product I don't have to configure or think about" sailed with the mention of consumer Linux.

akulbe 9y ago

I don't think I'd call it a "consumer" solution really, but it works very well, once you set it up. rsync.net It's not the cheapest either, but I never had issues when I was using them.

XorNot 9y ago

I have a strong suspicion Linux support doesn't exist, because that user category are going to buy 1 unlimited plan, put it on their 20TB home SAN, and well...20TB of cloud storage is not cheap (price it out on backblaze business).