5 ms·
Ask HN: What is the most cost effective way to do backups?
I'm interesting in backing up some large VM images, say about 500 GB.
I want to optimize for lowest cost.
I would prefer to use a tool like rsync (and not an inflexible closed source GUI). So, things like Box don't appeal to me.
I'm doing this in case of catastrophic failure to my HD. It is a risk, of course, but I would prefer to pay less for a service that might "only be 90%" as reliable. I don't need 5 9s.
I see a variety of services, the best of which seem to be rsync.net which is priced at minimum of 700 GB for 1.5 cents a GB, so about $15/mo (https://www.rsync.net/pricing.html https://www.rsync.net/pricing.html)
I could run my own VM on Vultr and can't do better than that AFAICT.
I see services like pCloud which have a *lifetime* cost of $199 for 500B (https://www.pcloud.com/cloud-storage-pricing-plans.html?period=lifetime https://www.pcloud.com/cloud-storage-pricing-plans.html?peri...). That seems interesting, and I expect in five years they will say "Sorry, we are out of business!" I'm ok with that risk and will likely plan for that.
But, are there other options to consider? Is there a sweet spot for storage if I almost never need to put more files, can tolerate slow retrieval, and am OK with a discount because the company will probably go out of business?
I could just buy a used desktop off Craigslist with a 1TB drive for $200 and pay for the power myself. That's another option, but I wonder if there is some reason I can find a cheaper option.
- RainbowFriends 3y agohttps://www.tarsnap.com/ https://www.tarsnap.com/ or if you really want to DIY move AWS S3 to Glacier Deep Archive at $0.00099 per GB/month
- monsieurbanana 3y agoTarsnap would be very expensive for his use case
- xrd 3y agoMy original comment was going to read: "I really like tarsnap, and the pricing seems much worse than rsync. $0.25 / GB-month. For 500GB that would mean $125/mo, right?" Looks like I am totally wrong! https://www.tarsnap.com/deduplication-examples.html https://www.tarsnap.com/deduplication-examples.html The de-duplication says you can store terabytes for $10/mo. I'm glad you forced me to respond to your comment!
- danparsonson 3y agoHeavily dependent on the amount of duplication you have in your data though, yes? Maybe you can work that out ahead of time?
- DirectorKrennic 3y agoHow often do you plan to do backups? Continuously or periodically?
- xrd 3y agoVery, very, very infrequently. These are VMs that I will upload once and then not think about them for years in case my local machines die.
- sgc 3y agoIn which case a hdd + bank deposit box might be cheaper than other solutions.
- Dalewyn 3y agoBuy a HDD, backup to it, then put the HDD in a safe deposit box at your bank.
- activiation 3y agoA few years old VM might break when you update it if you ever have to restore it.
- electroly 3y agoYou know your requirements better than we do, but at first glance this appears to be a mistake. You can see what's going to happen: when you go to restore the years-old VM backup, everything is so out of date that it doesn't work and the updates fail. "Do it once and forget about it for years" is essentially always a mistake with backups.
- alexk307 3y agoWhy not get a RAID array and store them in your basement?
- warent 3y agoI think you're supposed to have a backup off prem in case of some disaster so you're not just losing everything at once
- avgDev 3y agoI paid $89 for 15 months of Microsoft Office 365. It gives you 1TB per user for up to 5 users. I use onedrive app on Iphone to backup images. Next step is to sync my NAS but I have not gotten around to that yet.
- hansc 3y agoMaybe not what you are looking for, but I do it the manual way. Low-end, but reliable once in a while: USB3.2 external SSD (example 60usd on NewEgg for SanDisk 1TB)
- obituary_latte 3y agoYou may already have some free storage from a service like Google or Apple (iCloud). You could use something like that and rsync. Otherwise, a cheap desktop +/ NAS in basement with rsync might be the cheapest option.
- quartz 3y agoCloud storage is a massive ripoff. Just go buy a 2TB drive at Best Buy for $120 and stick it in an external usb-c enclosure from amazon for $15.
- stavros 3y agoAnd then make sure to scrub it weekly to make sure the bits haven't rotted, and move it to another location every night to be sure it doesn't die in a house fire.
- s1mon 3y agoWhile it doesn't support Linux, Backblaze is cloud storage and it's $70/yr for unlimited storage. To me, that's not a rip off for very secure off site back up. I backup my Mac locally to an external drive with Time Machine, many things (Photos, Mail, etc) are also in iCloud, and I use Backblaze. If you only use local external drives, get more than one, and rotate them off site. All it takes is a fire or break-in and your computer and external backup drive could be gone. If you at least have a drive off site, there's a much higher chance that you'll be ok.
- ASalazarMX 3y agoCloud backups do help with disaster recovery. If your house burns or floods, you can recover your data anywhere without sifting ashes/fishing for your external drive.
- omniglottal 3y agoRestic is the tool and Glacier is the service if you want it cheap.
- ArtWomb 3y agoEre by Fate or Happy Accident. You have stumbled upon the major pain point all of us "cloud architects" are having today. Dumping VMs to cloud store. We end up paying a 100x premium on what disc actually costs today for the privilege of "infinite" persistence. There's gotta be a better way ;) Forget data centers, DNA is the Future of Data Storage https://interestingengineering.com/science/dna-data-storage-capsule https://interestingengineering.com/science/dna-data-storage-... DNA storage in thermoresponsive microcapsules https://www.nature.com/articles/s41565-023-01377-4 https://www.nature.com/articles/s41565-023-01377-4
- stavros 3y agoYou're not paying for the disk, you're paying for the disk to be there when you need it. > Ere by Fate or Happy Accident Doesn't "ere" mean "before", not "either"? > Forget data centers, DNA is the Future of Data Storage Giving a much more literal meaning to "bit rot".
- uncensoredjrk 3y agoRestic (only backing up important data) to Backblaze B2 has been the sweet spot for my use cases. If the full VM goes down, I'll spin up a new one and restore my data.
- briffle 3y agoExactly, setup scripts or some other way to automate the stuff you need from the bare OS, and skip backing up that whole OS, pagefile, etc.
- graiz 3y agoI use backblaze.com - less than $100/year for unlimited backup on my main machine. Their business backup is $5/TB/Month, significantly less than S3.
- andyjohnson0 3y agoI also use backblaze amd am happy with the price and service.
- version_five 3y agoAlso vouching for them.
- tzs 3y agoI don't know about their business backup product, but the last time I considered their personal backup product restoring files involved decrypting the files on their server to produce a zip file for you to download. Is that still the case, and does business backup also work like that?
- seanalltogether 3y agoSince you mention buying a desktop as a local solution. You can instead get a single drive NAS for less then $200, or better yet a 2 drive one that you can put into a RAID config.
- Octoth0rpe 3y agoAnecdata, but I've been very happy with pCloud. Notably it regularly goes on sale. I think I got my 2tb lifetime plan for ~$300, and it's been perfectly sufficient. I've had it for around 4 years so far, so it's more than paid for itself versus the dropbox subscriptions that it replaced.
- xrd 3y agoThat's good to hear, I had not heard of them before and was putting "they will go out of business in x years" into my risk assessment.
- leoedin 3y agoHow much trust do you have in the lifetime plan? Surely the only way that can work is if new customers keep arriving - if they ever stop gaining customers then the money stops flowing, and you'd better hope they didn't spend it all.
- Octoth0rpe 3y ago> How much trust do you have in the lifetime plan? Very low when I first bought into it, but yeah that was 4 years ago. I've MORE than paid off the up front cost. If they disappeared tomorrow, I'd be comfortable with what I paid. I'd note that my usage may imply a higher level of risk than others may be comfortable with. All of my data is fully local/synced on my personal machine, and not local/available as long as I have internet on phone/tablet. So the only way I'd lose data is if pCloud disappeared _and_ my personal machine blew up in the same day. If pcloud disappears, I'd just dump my local files into iCloud or dropbox or something; and if my personal machine blew up I'd replace it and hope I'd get the replacement before pCloud goes kaput :) Doesn't seem like a terrible strategy. Also, pcloud does have a couple of mechanisms for getting additional revenue from existing customers. For example, you can increase the size of your lifetime plan for an additional one time payment, or add their crypto option. Also, some people presumably may pay month to month. Lastly, the best thing I can do as a pcloud user who is banking on them not going out of business is to occasionally rep them in public so that they get more customers, which I'm doing right now :)
- throwawaaarrgh 3y agoWhat you want is an external SATA RAID array hot swap cage. Connect two drives and in your OS set up a mirrored array. If one drive fails the other keeps working (don't buy the same model/batch drive). Cheaper over time than monthly payment.
- hestefisk 3y agoRAID is not a backup guarantee. Use zfs.
- throwawaaarrgh 3y ago?????? How is ZFS better than mirroring??? For two drives??????
- sokoloff 3y agoWhat threats are you trying to mitigate against? Only primary hard drive failures? Ransomware? User/application error? Fire/Flood/Earthquake? If you only want 90% and don't care about local concentration of risk, it's going to be really hard to beat a $50 used computer with a SATA hard drive turned into a Linux box in your basement.
- taneq 3y agoIf you put it in someone else's basement and combine it with rsnapshot or rsync then you're pretty much 100% of the way there. My setup is: 1) Server runs rsnapshot to keep a local version history of all important files 2) Nightly snapshot of the server VM is saved to an offsite Proxmox backup server onto a ZFS volume which is mirrored between two physical drives. So lost files due to user error can be pulled back off hourly, daily or monthly snapshots, total loss of the server for any reason just means restoring from the backup server (which I have tested, in an emergency with a little fiddling the backup server can take over as main server for a while), and no single drive failure on the backup server will lose old backups. If we get hit by ransomware and all our files get encrypted then our nightly backups would start timing out. :P I also sync the whole tree to my local drive since I regularly need to work offline on remote sites, so I have totally separate fairly recent snapshots on two different laptops, only one of which is generally used at any given time. If anything takes out both premises and both laptops simultaneously, I'm probably in no condition to care about our project folders anyway. Did I miss anything?
- invertedohm 3y agoPersonally I just push to an offsite restic backup on a cheap Hetzner storage box. It's not super fast and only hosted in EU AFAICT but it fits the bill for being cheap and reliable enough. It's just there for me to retrieve sentimental-value things like photos in the off chance all my other backup methods have failed. Initial setup is a bit weird with their SSH key setup IIRC but manageable. https://www.hetzner.com/storage/storage-box https://www.hetzner.com/storage/storage-box
- xrd 3y agoMan, that pricing of 4 euro for 1TB is nice. I love this option.
- anotherhue 3y agoYou can sshfs or cifs to it. Latency hurts but it's good.
- rodolphoarruda 3y agoI think a good way to go would be to get an used desktop and backup your VMs to it using rsync. "Pure sync is not backup", yes, that's right. So for a real backup you can use a service like Wasabi, for example, which will cost you around $5/TB per month with no extras fees for data transfers. In case you lose any VMs, you could immediately access a local copy at your used desktop. In case your whole place/office is compromised, you could access Wasabi from a different location and download a copy of your VM. Disclaimer: I'm a Wasabi MSP.
- yjftsjthsd-h 3y agoIf you're running a spare desktop with some disks in it, just run ZFS on that, take snapshots after every rsync, and ideally initiate rsync from the backup machine and don't have credentials to it anywhere else so that it can't (easily) be compromised at the same time as the other systems in the event of malware. Though yes, you should also always have an offsite backup as well, which AFAIK wasabi is fine for (as long as you're aware of the 1TB/90d minimums).
- aBioGuy 3y agoThe cheapest and simplest strategy is to buy several external drives and leave them at work / someone else's house.
- activiation 3y agoOr even leave one in your car
- berkle4455 3y agoIf you're not worried about physical/location risk, just buy an external HDD or a NAS.
- kqr 3y agoWhat is your process for restoration testing? If you test restoring from backups frequently you can afford less durable backup media. If you test restoring only after a few years when shit has hit the fan you will want to pay for higher durability.
- FrancoisLp 3y agoBackblaze is really nice, I used it for large backups (cold backups)
- christophilus 3y agoYeah. Backblaze or Wasabi would be my choice if you want cloud options. If you want to just toss it onto a drive in your basement, that's probably the cheapest option long-term.
- hestefisk 3y agoSet up a physical server on hetzner with a bunch of large disks. Then use zfs send. Job done :)
- opengears 3y agoCan you elaborate on this setup a bit? Any good documentation out there to read?
- zendaven 3y agoIceDrive is a pCloud alternative you might want to look into. I personally use pCloud though.
- SirMaster 3y ago>I would prefer to use a tool like rsync (and not an inflexible closed source GUI). So, things like Box don't appeal to me. The good news is that there is a tool for this: rclone https://rclone.org/docs/ https://rclone.org/docs/ It's basically rsync for cloud services and even supports your example of the Box cloud service. You can do a lot of neat things with rclone like transparent encryption and even turning a cloud storage into a mounted local storage folder. The other option would just be something simple like a RasPi or similar miniPC with a pair of HDDs mirrored, either automatically or manually by just backing up to both, or having the pc copy one to the other, with some retention rules etc. Honestly for important data I would do both a cloud backup and local, unless you have too much data where cloud would be too expensive. In that case just keep the most important stuff in the cloud and keep that and the less important stuff local.
- xrd 3y agorclone is interesting, and 1filchier has insanely good pricing at $30/yr for 2 TB. https://1fichier.com/tarifs.html https://1fichier.com/tarifs.html Anyone tried that?
- baal80spam 3y agorclone is an amazing tool.
- DavideNL 3y agoBorg Backup + Hetzner StorageBox… - https://www.borgbackup.org/ https://www.borgbackup.org/ - https://www.hetzner.com/storage/storage-box https://www.hetzner.com/storage/storage-box And optionally: Borgmatic: - https://torsion.org/borgmatic/ https://torsion.org/borgmatic/
- simcop2387 3y agoDefinitely take a look at tools like borg and restic, they'll help make the backups easier to do and have advantages in encryption and deduplication. Borgmatic is a nicely configurable system on top of borg for handling retiring of old backups and file paths and such.
- deleted 3y ago[deleted]
- ph4te 3y agorclone or restic with Wasabi is my goto backup solution. Restic for snapshots, rclone for random data dirs.
- sam_lowry_ 3y agorestic to a Glacier-like storage or dedicated storage box at the likes of Blackbase or Hetzner. That would be around $5/TB/month, AFAIR.
- stunt 3y agoIf you don't need to backup regularly, use offline storage and take multiple copies. That's what I do for everything I run at home. But I don't need daily backup. I backup every quarter.
- plantroon 3y agokopia.io to onedrive family plan. Nothing beats the price and lack of restrictions, plus it's encrypted, compressed and deduped before being sent off.