10 ms·

How I Backup

115 points by twapi 3y ago

naniwaduni 3y ago
> I Backup Well, that's a fun casualty of the title filter!
- TonyTrapp 3y ago
  I backup, therefore I am.
- supriyo-biswas 3y ago
  There was also (Why you should) “Still love Telnet” from some time ago[1]. [1] https://news.ycombinator.com/item?id=36179850 https://news.ycombinator.com/item?id=36179850
- throw0101b 3y ago
  Is "Backup" an Ancient Roman name like Claudius perhaps?
kkfx 3y ago
I've read USB sticks and backup in the same phrase, something must be wrong.
- TonyTrapp 3y ago
  Given that it's just one out of several backup media, that's kinda fine, I guess? Just don't put all your eggs in one basket. Which is why I'd add at least one HDD to the mix, as all the local options appear to be flash-based. Flash memory can develop defects especially if it's not turned on for a while. For long-term backup storage, a HDD might be a better solution.
  - starquake 3y ago
    Redundant USB stick might be more reliable than a single USB stick. But for me a USB stick is just too unreliable. I prefer putting my eggs in soft cosy baskets. Not in concrete baskets full of holes.
- NikkiA 3y ago
  Depends on the stick, I have a couple of ~15 year old sticks that still work 100% reliably whenever I need a 4GB installation media, compared to HDDs that are less than 2 years old and riddled with remapped sectors :/ I think I've only ever thrown away 1 USB stick due to failure, microSD is my nemesis when it comes to putting 'reliable' and 'flash' in the same sentence, everything else seems to last just fine as long as it's not used constantly.
  - NikkiA 3y ago
    Just checked, my eldest is a 2007 lexar "twist n turn" 4GB, last formatted in 2019 with intel lan drivers on it, still 100% reliable, files on it verify ok.
- dist-epoch 3y ago
  USB sticks are much more reliable than hard drives.
  - TonyTrapp 3y ago
    That really depends on the stick. In Germany, there is a brand called Intenso that is unfortunately rather popular here, you'll find their low-priced USB sticks everywhere and I've had nothing but problems with the few I had to use. They often were much slower than advertised, sometimes they would just break down completely in the middle of copying files. It always felt like they try to source the cheapest flash chips that barely pass quality assurance. Not to mention all the "fake" USB sticks that report a larger size than what their actual capcity is, which you may only find out after it's too late. In contrast, I have had a single hard disk with a fatal failure in the last twenty years or so.
  - 1oooqooq 3y ago
    They are not. Most of this misconception is when people evaluated flash memory quality, when flash memory was expensive. Today flash memory is the most cheap stuff you can get your hands on. And quality nose dived, and nobody re-did the comparisons. Meanwhile HDD are not used anywhere but datacenters, and quality went up. For the same reasons flash storage was good quality before: low inventory only bought by quality conscious clients.
- Aachen 3y ago
  Why?
  - kkfx 3y ago
    Quality of most USB sticks (meaning not all, but almost all) on sale is crap. As a result you might get a stick working for many years, for all files you put on it, while many others with important silent data corruption in a little time. Of course you can use them with a "modern" filesystem with block-level checksums but... It's like playing Russian roulette. SSD tend to be better but are still good for fresh storage only, HDD are complicated, most have evolved toward crap, but good ones exists and are reliable and cheap and big enough for most usages. Long story short: choose three HDDs from good brands/series and use them normally the likelihood of multiple concurrent failures are scarce and the final price is not that high. For really important stuff a simple desktop tape reader and few tapes TOGETHER with HDDs complete the games. SSDs inside desktops/laptops/home-servers do their jobs well.
    - Aachen 3y ago
      I was of the opinion that the reliability of the storage medium does not matter much for backup purposes, so long as it's reliable enough to survive the days between a failure of another copy and the rebuilding of that. But you've convinced me that it's not a blanket statement one can make, due to silent data corruption. Even with systems that have checksumming, you're right that I'm not doing a full data read on my restic backup regularly. I guess it can still be an option to use regular USB sticks, given the right software and discipline to run its checks, but at that point, it's probably easier to use a more suitable storage medium, especially since HDDs are the cheapest option per GB. (Not that I personally use USB sticks for backup currently, but I do repurpose old hardware for things like my email server.) Thanks!
juangacovas 3y ago
Interesting. Just for sharing, for backup on Linux I've been using rsnapshot, restic (which accepts Rclone as interface) and ReaR (Relax and Recover) for full linux recovery; on Windows, commercial Acronis and recently UrBackup. Old good "pathsync" from Cockos also helps on LAN. Everything boils down to your amount of files or GBs when some solution becomes "too slow" doing incrementals and/or file processing. My two cents.
- rom1v 3y ago
  On Linux: https://www.borgbackup.org/ https://www.borgbackup.org/
michaelt 3y ago
> Whenever I turn on one computer, I sync it from the other. > Daily rsync to an encrypted Linux USB stick attached to the desktop. > Daily rsync to an encrypted MacOS USB stick attached to the laptop. > Daily rsync to an encrypted ZFS SSD inside the desktop. > Daily rsync to OpenBSD remote attached storage at vultr.com > [...] I do it a few times a day, especially if I’ve just made or saved something of value. And always right before I shut down the computer, which I do almost any time I step away from it for more than a few minutes. Have you ever noticed blog posts rarely go viral if they describe a normal setup, and are much more likely to do so if they describe a really complicated setup? I'm all for people coming up with needlessly complex homelab setups if they have fun doing so - but sometimes I wonder whether newbies see this sort of thing, and think Linux needs this level of complexity.
- megous 3y ago
  What complexity? rsync to 4 locations, likely from some script?
  - mosselman 3y ago
    It says in the post he runs them by hand.
    - megous 3y ago
      And how should he run them? He has to manipulate HW things to do the backup. It doesn't mean he has to do 10 steps for each of these backups. It can still be in a script.
  - pizza234 3y ago
    Describing systems in the article's terms ("5UP3R 51MPL3, BR0S") is disingenous. Backing up to multiple manually managed locations requires also maintaining those locations, which means spending time on system administration, which is not negligible (it's sysadmin work). At best, the sysadmin time is amortized, but in real world, it never really is. It's difficult to compare this strategy to actual backup programs, because it's not really a backup (it misses some functionalities of backup systems). If the independence requirement is removed (which can make the comparison improper), then for 10/20$ a month there are backup programs that require no maintenance at all, adding other functionalities at the same time (which can include user-owned external storage locations).
jbverschoor 3y ago
Doesn’t sound like backing up, but keeping your stuff massively in sync. What happens if you discover something is missing or corrupted after 6 months?
- Double_a_92 3y ago
  If it's not something super critical in my experience the simpler solution is better for longterm or fool-proof backups. E.g. if I use some special backup software that creates incremental archives or whatever, and then I die... My family would have absolutely no idea how to access that data. While with a normal NTFS partition and data on an external HDD they could just plug it in some PC and look at the family photos.
  - mmcgaha 3y ago
    I use a rsync script that does incrementals but keeps full copies of each backup. Although mine is significantly different, it is based on http://www.mikerubel.org/computers/rsync_snapshots/ http://www.mikerubel.org/computers/rsync_snapshots/ by Mike Rubel.
  - pizza234 3y ago
    It really depends on what we're comparing this to. If one compares it to a commercial solution that allows backing up to user-owned storages, then the commercial solution is simpler while having more functionality (including versioning).
- globular-toast 3y ago
  The ZFS clone should deal with that via regular snapshots.
  - dist-epoch 3y ago
    Only if you have infinite storage space.
    - deleted 3y ago
Aachen 3y ago
Two additional suggestions: 1. Test your backups. I see no testing procedure described, and although maybe the author's scripts have spot checking beyond what is described in the article, code will only ever catch the cases it was programmed to expect. Manual testing does not have to be frequent or extensive, but making sure the backups are still recoverable and complete (take five minutes to spot check some old and new files) every six months helps ensure no silent issues are occurring 2. Less importantly: use append-only backups to an independent system. While ransomware for consumers is comparatively rare these days, it would give me peace of mind. The author's deposit box mostly fulfils that purpose, but whether it gets filled with bogus data or wiped by the malware is a matter of luck in timing (with the odds heavily in their favor due to the irregular connection of the drive and the existence of another computer that the malware probably isn't programmed to expect). Of course, "append only" is a bit of a misnomer because eventually you'd run out of space, but the system should independently delete old data rather than being able to command it remotely (from the potentially infected system) to wipe anything. Restic has docs with considerations for data deletion in an append-only setup (disclose: I helped write them), which are also relevant if you don't use restic because they describe the method more than the specific implementation: https://restic.readthedocs.io/en/latest/060_forget.html#security-considerations-in-append-only-mode https://restic.readthedocs.io/en/latest/060_forget.html#secu...
ElDji 3y ago
My setup : - servers and laptops (laptops : except /home): restic to local minio instance + rclone to B2 storage - /home of laptops : kopia to b2 storage
DavidPiper 3y ago
Here I was starting to think 2 USB (platter) drives and the occasional rsync -uavPi is too much. All power to the author, that's a hell of a set up.
praptak 3y ago
Author: describes a simple setup which probably puts him in the top 0.1% of personal users in terms of resilience to data loss. HN: nitpick about potential problems which are present even in some industry grade data loss solutions.
- hef19898 3y ago
  And honestly, I never lost data in my whole life as a private user. Exceptions: Own stupidity in which case even back-ups don't help. And that included the time when HDDs had a tendency to fail. And what little data loss occured was actually a grand total of 3 hours of work to recreate it. I simply don't get the obbsession people have with data back ups for their private needs.
  - NikolaNovak 3y ago
    I had the great privilege of my hard drive dying 4th year university during exams / finals season. And so I had the great luck of paying "student rates" for a data recovery service. That was a well worth early lesson in life :) Most people just cannot imagine losing all their data. They think it cannot happen, or it won't matter, or it will all be fine. What we have is a problem of imagination. I run through a simple exercise with my non techie friends whave have endless faith in technology : "if I went to your phone and laptop with a hammer right now and smash them to pieces, how much would you cry?" (Some tear up right then and there ; others still think it'd be fine until I ask some follow up questions to instantiate the risk:). Personally, I have scanned photos of childhood and tax and immigration and education and legal records and books I purchased and work documents and client photos from side photography gig and a million other things it'd be a pain to lose. So I have a local NAS and pay a cloud backup service and have reached my comfort level of data loss risk :). "Never lost data my whole life " is pure luck, and I wish it continues, but it's a bad plan - it's a bit of "I played Russian roulette and I'm fine!" :->
    - stavros 3y ago
      I had my mobile phone die out of the blue. I sync everything, so I wasn't worried, but later on I realized I had lost some old ringtones and notification sounds I quite liked. Nothing terrible, but it definitely showed me the importance of testing your backups (whenever you buy a new phone).
lurtbancaster 3y ago
Question for rsync users on linux. Does your rsync preserve creation times --crtimes ? I'm aware ext4/3 filesystems don't store creation times. But I want to use rsync on Linux to sync files between an NTFS(supports creation times) and a Btrfs partition(also supports creation times) without losing the creation times. Currently rsync just sets the creation time to the modification time on the destination directory. Which is not what I want. When I use `--crtimes` I get a "This rsync does not support --crtimes (-N)" error. Weirdly, just using `cp -a` works. But it doesn' t have any of the checksumming and differential copy bells and whistles of rsync. How do you rsync on linux while preserving creation times(on supported file systems)?
- perlclutcher 3y ago
  I just migrated from a QNAP NAS to a custom build with ZFS. I have files with creation times dating back to the 90s on my original FAT16 filesystem and I didn't want to lose them, so I recently did a (way-too-)deep dive on this, I'll summarize my notes for you. First, ext4 actually does support creation times, called "crtime". But there's some internet confusion about it since this support predated linux kernel support, so you had to use ext4-specific tooling (on an unmounted filesystem) to access it, e.g.: debugfs -R 'stat <inode>' /dev/nvme0n1p1 debugfs -R -w 'set_inode_field <inode> crtime <new-time>' /dev/nvme0n1p1 The btrfs situation is similar, but btrfs called it "otime" (for some reason?). Linux 4.11 introduced kernel-level support unified across all filesystems, calling it "btime" (birth time). But the normal file syscalls only support reading btimes, not setting them to arbitrary values. And rsync on linux, as you saw, can't do anything the kernel doesn't have a syscall for. For a while the only option was to: 1. set the system clock 2. create a file (at which point the kernel sets btime to the system time, plus a few nanoseconds) 3. restore the system clock Obviously a huge hack, and needs root, but tools like s-tar automated it (search for "time storm" in this manpage): https://web.archive.org/web/20220331080358/http://schilytools.sourceforge.net/man/man1/star.1.html https://web.archive.org/web/20220331080358/http://schilytool... I almost gave up and spun up a Windows VM (since Windows has supported reading and writing creation times since the beginning). But then it clicked -- the kernel interface to the filesystem module takes the btime as an explicit parameter. So if you could find a kernel module that talks to the filesystem module directly (instead of going through the usual high-level file syscalls), you can pass along any btime you want. And there just so
prmoustache 3y ago
rsync for backup is a recipe for disaster if you don't realize you just rsynced and overwriten good data with corrupted or ransomwared data. If you want to use rsync, do rsync and snapshots...and those snapshots should automatically made on the remote destination (can be just a daily cron), only accessible read only from the source machine without having any administrative rights to tamper them.
bambax 3y ago
I use a similar, but I think simpler, setup. "Files I use and change every day" are on Dropbox, which gets synced to a NAS. Photo and video files are directly stored on the NAS and don't live anywhere else. The NAS is rsynced automatically every day to an external disk (connected via USB) of which there are three copies. One is in a safe at home, the other at my parents' house. The one in the safe is rotated every week, the one at my parents' every month. If my house gets robbed or even burns down it's likely the disk in the safe will survive and I lose only one week of data. If the house gets completely destroyed or the safe is cracked, then I can turn to the disk at my parents and lose one month. It's true that rsync is not a backup. But by default rsync only adds data. Delete is an option one has to turn on. My daily rsync does not have the delete option. I have a different script with delete, that I run manually. The theory was that I would review the delete process with a dry run before committing to it. In practice I don't do this and simply run the delete script occasionally, which ruins the purpose. But I guess I could be more careful if I really wanted to.
globular-toast 3y ago
Anyone know/recommend something like ZFS.rent in Europe?
- stavros 3y ago
  I use rsync.net, but I don't know if their `zfs receive` is encrypted: https://www.rsync.net/products/zfsintro.html https://www.rsync.net/products/zfsintro.html
cranium 3y ago
I use Syncthing on my PCs and NAS, and Backblaze to trivially do a 3-2-1 backup strategy. My work directories are synced on all devices. With my NAS up 24/7, I can always pick up the last changes when I switch computers. The irreplaceable documents (all my notes, taxes, company accounting,...) are also backed up by the NAS on Backblaze, keeping all versions. I occasionally create a snapshot on Backblaze for those files – should totally automate that but I'm lazy. I do agree though: it's more "syncing stuff between multiple places" than a real backup.
jopsen 3y ago
I've been happy with: https://kopia.io/ https://kopia.io/ Fairly easy to configure, does snapshots to S3 and has a icon in my tray I can watch :)
- 1oooqooq 3y ago
  kopia is one of those things are forever in my todo list while i use rsync daily.
- pizza234 3y ago
  It seems that Kopia doesn't support continuous backup, which may be a requirement for some users (detecting changes on a large number of files is resource-intensive).
oulipo 3y ago
I'm using : - Dropbox for "core files" (small subset of files I don't want to lose, mostly my notes, papers, etc) - Arq with S3 for backups (do a backup each hour) - An external SDD to do semi-frequent backups (mostly used in case I change laptop to ensure I have a backup of my files, but otherwise I mostly rely on Arq)
- Aachen 3y ago
  Blank lines prevent the bullet points running together
Toutouxc 3y ago
I keep my important files on my DIY NAS -- a cheap NUC with two external HDDs (Btrfs in RAID1). Every night at 1am a cron job takes a restic snapshot to a Backblaze bucket. I feel like bang for buck my solution isn't half bad. The NUC is also my home server (web, Pi-hole, Home Assistant, ...), so it was already there, the HDDs were cheap and I currently pay less than $3 a month for over 300 GB of data in the bucket.
matthews2 3y ago
Thanks for sharing! I'm going to go and test my backups now...
deepersprout 3y ago
I do something similar[1] with rsync --link-dest and hard links, so that I can go back in time if I delete something. OP seems to just have a copy of his actual files, but if he sees a file missing after 2 weeks after deleting it, he has no way of recovering it. [1]https://31337.it/incremental-backups-with-rsync/ https://31337.it/incremental-backups-with-rsync/
- rstuart4133 3y ago
  Or if you don't want to write a DIY solution you could use this (shameless plug): https://rsync-backup-ras.sourceforge.io/ https://rsync-backup-ras.sourceforge.io/
petters 3y ago
> here is how I do my backups. It takes me about ten seconds per day and five minutes per month to maintain > rsync everything to each of them, bringing one down to a safe deposit box downtown, and taking out the one that was there from my last visit Sounds like you are underestimating how much time you spend on backups. Sounds good though
- EvanAnderson 3y ago
  I assumed the author has to make the trip to the bank anyway. I integrate my off-site, offline backup storage with visits to my parents, for example.
TheCapeGreek 3y ago
My absolute final resort "solution" for data loss of keepsakes: Accept the impermanence of data, memories and memorabilia. Ideally I'll strive to keep that data backed up as long as I can, and will mourn its loss, but mentally I prepare myself that one day I might lose access in some way. Most importantly, that data is also 99% irrelevant to the next generation, maybe save for a handful of photos or videos ("what did grandpa look like?"). So in the grand scheme of things it's not like I have to be an epic archivist of my own life. I suppose it helps that this was done for me in the past - toxic family members threw out all my childhood family photos a decade ago.
- 72deluxe 3y ago
  "Toxic" seems quite the understatement! How sad. I went to school with a boy who had no photos from his youth. His parents took many photos but didn't get them developed, and when they took all the films to the pharmacy (here in the UK) to get them developed, the pharmacy sadly burned down and they didn't ever get any photos. Quite sad I thought.
JonChesterfield 3y ago
Everything that matters is checked into fossil. Box hosting that uses zfs. That rules out most failure modes other than gross incompetence, partially mitigated by a periodic backup to external drives. In the before times I put everything in Dropbox. Dropbox synced with an empty directory by mistake and deleted everything. Not going to use Dropbox again.
agubelu 3y ago
Ah yes, HN's automatic title editorializing strikes again, removing 33% of the title's wordcount and changing it from "click here to see my backup method" to "hey, I do backups".
1oooqooq 3y ago
that's a somewhat expensive setup to still do all the work of dealing with dead drives and manually doing replication and dealing with conflicts (I fear the pain of dealing with his 4way daily backup of the same data over desktop/laptop/2 pen drives) zfs rent = 4+TB drive + $10/mo (base 1TB) + $5*3 (extra 3TB) = drive + $25/mo hetzner = 5TB $10.9/mo vultr = $5/mo (probably more) 4x ~4TB drives = $800 assuming you replace drives every 5yrs: $3,254 or $54.23/mo. assuming the drives last 10yr which are common now: $5,708 or $47.56/mo.
- aChattuio 3y ago
  A 4tb drive costs 80-120€ 7€/month for 5y
  - 1oooqooq 3y ago
    that price is for barracudas. then you will be lucky to replace them every 2 years. wd gold and others that will last 5~10years are actually more expensive for 4TB ($500) because they are old stock for people that for some reason need that model. The sweet spot for top quality HDD is always 200~300. In whatever capacity du jour. (14~20TB today) nitipicking drive cost aside, the monthly services are the bulky of the cost.
aChattuio 3y ago
I do a zfs snapshot and than do a sync. I never delete those snapshots because the big items like images don't change.
jamesgreenleaf 3y ago
I've used Backblaze for years and it's worth the money. I've only done a full restore from it once after a laptop was bricked and it was seamless. There have been a few times that I needed a recently deleted file and it worked for that too.
p0w3n3d 3y ago
I'm using rdiff-backup to have versioned copies, however at some point of time trimming of the snapshots is required
djks422 3y ago
Are backup stored to Glacier is my house-burned-down archive for about $5/mo. It may cost a lot to get out, but at that point our family pictures and movies are gone anyway, so it would be worth the cost.
deleted 3y ago
[deleted]
eternityforest 3y ago
I use Vorta. I have two profiles, one for my home dir, and recently I added one for all dotfiles I would want to keep when setting up a new machine. It's only about 5 files but it avoids using yet another tool for dotfiles.

How I Backup

115 points by twapi 3y ago

naniwaduni 3y ago

> I Backup Well, that's a fun casualty of the title filter!

TonyTrapp 3y ago
I backup, therefore I am.
supriyo-biswas 3y ago
There was also (Why you should) “Still love Telnet” from some time ago[1]. [1] https://news.ycombinator.com/item?id=36179850 https://news.ycombinator.com/item?id=36179850
throw0101b 3y ago
Is "Backup" an Ancient Roman name like Claudius perhaps?

kkfx 3y ago

I've read USB sticks and backup in the same phrase, something must be wrong.

TonyTrapp 3y ago
Given that it's just one out of several backup media, that's kinda fine, I guess? Just don't put all your eggs in one basket. Which is why I'd add at least one HDD to the mix, as all the local options appear to be flash-based. Flash memory can develop defects especially if it's not turned on for a while. For long-term backup storage, a HDD might be a better solution.
- starquake 3y ago
  Redundant USB stick might be more reliable than a single USB stick. But for me a USB stick is just too unreliable. I prefer putting my eggs in soft cosy baskets. Not in concrete baskets full of holes.
NikkiA 3y ago
Depends on the stick, I have a couple of ~15 year old sticks that still work 100% reliably whenever I need a 4GB installation media, compared to HDDs that are less than 2 years old and riddled with remapped sectors :/ I think I've only ever thrown away 1 USB stick due to failure, microSD is my nemesis when it comes to putting 'reliable' and 'flash' in the same sentence, everything else seems to last just fine as long as it's not used constantly.
- NikkiA 3y ago
  Just checked, my eldest is a 2007 lexar "twist n turn" 4GB, last formatted in 2019 with intel lan drivers on it, still 100% reliable, files on it verify ok.
dist-epoch 3y ago
USB sticks are much more reliable than hard drives.
- TonyTrapp 3y ago
  That really depends on the stick. In Germany, there is a brand called Intenso that is unfortunately rather popular here, you'll find their low-priced USB sticks everywhere and I've had nothing but problems with the few I had to use. They often were much slower than advertised, sometimes they would just break down completely in the middle of copying files. It always felt like they try to source the cheapest flash chips that barely pass quality assurance. Not to mention all the "fake" USB sticks that report a larger size than what their actual capcity is, which you may only find out after it's too late. In contrast, I have had a single hard disk with a fatal failure in the last twenty years or so.
- 1oooqooq 3y ago
  They are not. Most of this misconception is when people evaluated flash memory quality, when flash memory was expensive. Today flash memory is the most cheap stuff you can get your hands on. And quality nose dived, and nobody re-did the comparisons. Meanwhile HDD are not used anywhere but datacenters, and quality went up. For the same reasons flash storage was good quality before: low inventory only bought by quality conscious clients.
Aachen 3y ago
Why?
- kkfx 3y ago
  Quality of most USB sticks (meaning not all, but almost all) on sale is crap. As a result you might get a stick working for many years, for all files you put on it, while many others with important silent data corruption in a little time. Of course you can use them with a "modern" filesystem with block-level checksums but... It's like playing Russian roulette. SSD tend to be better but are still good for fresh storage only, HDD are complicated, most have evolved toward crap, but good ones exists and are reliable and cheap and big enough for most usages. Long story short: choose three HDDs from good brands/series and use them normally the likelihood of multiple concurrent failures are scarce and the final price is not that high. For really important stuff a simple desktop tape reader and few tapes TOGETHER with HDDs complete the games. SSDs inside desktops/laptops/home-servers do their jobs well.
  - Aachen 3y ago
    I was of the opinion that the reliability of the storage medium does not matter much for backup purposes, so long as it's reliable enough to survive the days between a failure of another copy and the rebuilding of that. But you've convinced me that it's not a blanket statement one can make, due to silent data corruption. Even with systems that have checksumming, you're right that I'm not doing a full data read on my restic backup regularly. I guess it can still be an option to use regular USB sticks, given the right software and discipline to run its checks, but at that point, it's probably easier to use a more suitable storage medium, especially since HDDs are the cheapest option per GB. (Not that I personally use USB sticks for backup currently, but I do repurpose old hardware for things like my email server.) Thanks!

juangacovas 3y ago

Interesting. Just for sharing, for backup on Linux I've been using rsnapshot, restic (which accepts Rclone as interface) and ReaR (Relax and Recover) for full linux recovery; on Windows, commercial Acronis and recently UrBackup. Old good "pathsync" from Cockos also helps on LAN. Everything boils down to your amount of files or GBs when some solution becomes "too slow" doing incrementals and/or file processing. My two cents.

rom1v 3y ago
On Linux: https://www.borgbackup.org/ https://www.borgbackup.org/

michaelt 3y ago

> Whenever I turn on one computer, I sync it from the other. > Daily rsync to an encrypted Linux USB stick attached to the desktop. > Daily rsync to an encrypted MacOS USB stick attached to the laptop. > Daily rsync to an encrypted ZFS SSD inside the desktop. > Daily rsync to OpenBSD remote attached storage at vultr.com > [...] I do it a few times a day, especially if I’ve just made or saved something of value. And always right before I shut down the computer, which I do almost any time I step away from it for more than a few minutes. Have you ever noticed blog posts rarely go viral if they describe a normal setup, and are much more likely to do so if they describe a really complicated setup? I'm all for people coming up with needlessly complex homelab setups if they have fun doing so - but sometimes I wonder whether newbies see this sort of thing, and think Linux needs this level of complexity.

megous 3y ago
What complexity? rsync to 4 locations, likely from some script?
- mosselman 3y ago
  It says in the post he runs them by hand.
  - megous 3y ago
    And how should he run them? He has to manipulate HW things to do the backup. It doesn't mean he has to do 10 steps for each of these backups. It can still be in a script.
- pizza234 3y ago
  Describing systems in the article's terms ("5UP3R 51MPL3, BR0S") is disingenous. Backing up to multiple manually managed locations requires also maintaining those locations, which means spending time on system administration, which is not negligible (it's sysadmin work). At best, the sysadmin time is amortized, but in real world, it never really is. It's difficult to compare this strategy to actual backup programs, because it's not really a backup (it misses some functionalities of backup systems). If the independence requirement is removed (which can make the comparison improper), then for 10/20$ a month there are backup programs that require no maintenance at all, adding other functionalities at the same time (which can include user-owned external storage locations).

jbverschoor 3y ago

Doesn’t sound like backing up, but keeping your stuff massively in sync. What happens if you discover something is missing or corrupted after 6 months?

Double_a_92 3y ago
If it's not something super critical in my experience the simpler solution is better for longterm or fool-proof backups. E.g. if I use some special backup software that creates incremental archives or whatever, and then I die... My family would have absolutely no idea how to access that data. While with a normal NTFS partition and data on an external HDD they could just plug it in some PC and look at the family photos.
- mmcgaha 3y ago
  I use a rsync script that does incrementals but keeps full copies of each backup. Although mine is significantly different, it is based on http://www.mikerubel.org/computers/rsync_snapshots/ http://www.mikerubel.org/computers/rsync_snapshots/ by Mike Rubel.
- pizza234 3y ago
  It really depends on what we're comparing this to. If one compares it to a commercial solution that allows backing up to user-owned storages, then the commercial solution is simpler while having more functionality (including versioning).
globular-toast 3y ago
The ZFS clone should deal with that via regular snapshots.
- dist-epoch 3y ago
  Only if you have infinite storage space.
  - deleted 3y ago

Aachen 3y ago

Two additional suggestions: 1. Test your backups. I see no testing procedure described, and although maybe the author's scripts have spot checking beyond what is described in the article, code will only ever catch the cases it was programmed to expect. Manual testing does not have to be frequent or extensive, but making sure the backups are still recoverable and complete (take five minutes to spot check some old and new files) every six months helps ensure no silent issues are occurring 2. Less importantly: use append-only backups to an independent system. While ransomware for consumers is comparatively rare these days, it would give me peace of mind. The author's deposit box mostly fulfils that purpose, but whether it gets filled with bogus data or wiped by the malware is a matter of luck in timing (with the odds heavily in their favor due to the irregular connection of the drive and the existence of another computer that the malware probably isn't programmed to expect). Of course, "append only" is a bit of a misnomer because eventually you'd run out of space, but the system should independently delete old data rather than being able to command it remotely (from the potentially infected system) to wipe anything. Restic has docs with considerations for data deletion in an append-only setup (disclose: I helped write them), which are also relevant if you don't use restic because they describe the method more than the specific implementation: https://restic.readthedocs.io/en/latest/060_forget.html#security-considerations-in-append-only-mode https://restic.readthedocs.io/en/latest/060_forget.html#secu...

ElDji 3y ago

My setup : - servers and laptops (laptops : except /home): restic to local minio instance + rclone to B2 storage - /home of laptops : kopia to b2 storage

DavidPiper 3y ago

Here I was starting to think 2 USB (platter) drives and the occasional rsync -uavPi is too much. All power to the author, that's a hell of a set up.

praptak 3y ago

Author: describes a simple setup which probably puts him in the top 0.1% of personal users in terms of resilience to data loss. HN: nitpick about potential problems which are present even in some industry grade data loss solutions.

hef19898 3y ago
And honestly, I never lost data in my whole life as a private user. Exceptions: Own stupidity in which case even back-ups don't help. And that included the time when HDDs had a tendency to fail. And what little data loss occured was actually a grand total of 3 hours of work to recreate it. I simply don't get the obbsession people have with data back ups for their private needs.
- NikolaNovak 3y ago
  I had the great privilege of my hard drive dying 4th year university during exams / finals season. And so I had the great luck of paying "student rates" for a data recovery service. That was a well worth early lesson in life :) Most people just cannot imagine losing all their data. They think it cannot happen, or it won't matter, or it will all be fine. What we have is a problem of imagination. I run through a simple exercise with my non techie friends whave have endless faith in technology : "if I went to your phone and laptop with a hammer right now and smash them to pieces, how much would you cry?" (Some tear up right then and there ; others still think it'd be fine until I ask some follow up questions to instantiate the risk:). Personally, I have scanned photos of childhood and tax and immigration and education and legal records and books I purchased and work documents and client photos from side photography gig and a million other things it'd be a pain to lose. So I have a local NAS and pay a cloud backup service and have reached my comfort level of data loss risk :). "Never lost data my whole life " is pure luck, and I wish it continues, but it's a bad plan - it's a bit of "I played Russian roulette and I'm fine!" :->
  - stavros 3y ago
    I had my mobile phone die out of the blue. I sync everything, so I wasn't worried, but later on I realized I had lost some old ringtones and notification sounds I quite liked. Nothing terrible, but it definitely showed me the importance of testing your backups (whenever you buy a new phone).

lurtbancaster 3y ago

Question for rsync users on linux. Does your rsync preserve creation times --crtimes ? I'm aware ext4/3 filesystems don't store creation times. But I want to use rsync on Linux to sync files between an NTFS(supports creation times) and a Btrfs partition(also supports creation times) without losing the creation times. Currently rsync just sets the creation time to the modification time on the destination directory. Which is not what I want. When I use `--crtimes` I get a "This rsync does not support --crtimes (-N)" error. Weirdly, just using `cp -a` works. But it doesn' t have any of the checksumming and differential copy bells and whistles of rsync. How do you rsync on linux while preserving creation times(on supported file systems)?

perlclutcher 3y ago
I just migrated from a QNAP NAS to a custom build with ZFS. I have files with creation times dating back to the 90s on my original FAT16 filesystem and I didn't want to lose them, so I recently did a (way-too-)deep dive on this, I'll summarize my notes for you. First, ext4 actually does support creation times, called "crtime". But there's some internet confusion about it since this support predated linux kernel support, so you had to use ext4-specific tooling (on an unmounted filesystem) to access it, e.g.: debugfs -R 'stat <inode>' /dev/nvme0n1p1 debugfs -R -w 'set_inode_field <inode> crtime <new-time>' /dev/nvme0n1p1 The btrfs situation is similar, but btrfs called it "otime" (for some reason?). Linux 4.11 introduced kernel-level support unified across all filesystems, calling it "btime" (birth time). But the normal file syscalls only support reading btimes, not setting them to arbitrary values. And rsync on linux, as you saw, can't do anything the kernel doesn't have a syscall for. For a while the only option was to: 1. set the system clock 2. create a file (at which point the kernel sets btime to the system time, plus a few nanoseconds) 3. restore the system clock Obviously a huge hack, and needs root, but tools like s-tar automated it (search for "time storm" in this manpage): https://web.archive.org/web/20220331080358/http://schilytools.sourceforge.net/man/man1/star.1.html https://web.archive.org/web/20220331080358/http://schilytool... I almost gave up and spun up a Windows VM (since Windows has supported reading and writing creation times since the beginning). But then it clicked -- the kernel interface to the filesystem module takes the btime as an explicit parameter. So if you could find a kernel module that talks to the filesystem module directly (instead of going through the usual high-level file syscalls), you can pass along any btime you want. And there just so

prmoustache 3y ago

rsync for backup is a recipe for disaster if you don't realize you just rsynced and overwriten good data with corrupted or ransomwared data. If you want to use rsync, do rsync and snapshots...and those snapshots should automatically made on the remote destination (can be just a daily cron), only accessible read only from the source machine without having any administrative rights to tamper them.

bambax 3y ago

I use a similar, but I think simpler, setup. "Files I use and change every day" are on Dropbox, which gets synced to a NAS. Photo and video files are directly stored on the NAS and don't live anywhere else. The NAS is rsynced automatically every day to an external disk (connected via USB) of which there are three copies. One is in a safe at home, the other at my parents' house. The one in the safe is rotated every week, the one at my parents' every month. If my house gets robbed or even burns down it's likely the disk in the safe will survive and I lose only one week of data. If the house gets completely destroyed or the safe is cracked, then I can turn to the disk at my parents and lose one month. It's true that rsync is not a backup. But by default rsync only adds data. Delete is an option one has to turn on. My daily rsync does not have the delete option. I have a different script with delete, that I run manually. The theory was that I would review the delete process with a dry run before committing to it. In practice I don't do this and simply run the delete script occasionally, which ruins the purpose. But I guess I could be more careful if I really wanted to.

globular-toast 3y ago

Anyone know/recommend something like ZFS.rent in Europe?

stavros 3y ago
I use rsync.net, but I don't know if their `zfs receive` is encrypted: https://www.rsync.net/products/zfsintro.html https://www.rsync.net/products/zfsintro.html

cranium 3y ago

I use Syncthing on my PCs and NAS, and Backblaze to trivially do a 3-2-1 backup strategy. My work directories are synced on all devices. With my NAS up 24/7, I can always pick up the last changes when I switch computers. The irreplaceable documents (all my notes, taxes, company accounting,...) are also backed up by the NAS on Backblaze, keeping all versions. I occasionally create a snapshot on Backblaze for those files – should totally automate that but I'm lazy. I do agree though: it's more "syncing stuff between multiple places" than a real backup.

jopsen 3y ago

I've been happy with: https://kopia.io/ https://kopia.io/ Fairly easy to configure, does snapshots to S3 and has a icon in my tray I can watch :)

1oooqooq 3y ago
kopia is one of those things are forever in my todo list while i use rsync daily.
pizza234 3y ago
It seems that Kopia doesn't support continuous backup, which may be a requirement for some users (detecting changes on a large number of files is resource-intensive).

oulipo 3y ago

I'm using : - Dropbox for "core files" (small subset of files I don't want to lose, mostly my notes, papers, etc) - Arq with S3 for backups (do a backup each hour) - An external SDD to do semi-frequent backups (mostly used in case I change laptop to ensure I have a backup of my files, but otherwise I mostly rely on Arq)

Aachen 3y ago
Blank lines prevent the bullet points running together

Toutouxc 3y ago

I keep my important files on my DIY NAS -- a cheap NUC with two external HDDs (Btrfs in RAID1). Every night at 1am a cron job takes a restic snapshot to a Backblaze bucket. I feel like bang for buck my solution isn't half bad. The NUC is also my home server (web, Pi-hole, Home Assistant, ...), so it was already there, the HDDs were cheap and I currently pay less than $3 a month for over 300 GB of data in the bucket.

matthews2 3y ago

Thanks for sharing! I'm going to go and test my backups now...

deepersprout 3y ago

I do something similar[1] with rsync --link-dest and hard links, so that I can go back in time if I delete something. OP seems to just have a copy of his actual files, but if he sees a file missing after 2 weeks after deleting it, he has no way of recovering it. [1]https://31337.it/incremental-backups-with-rsync/ https://31337.it/incremental-backups-with-rsync/

rstuart4133 3y ago
Or if you don't want to write a DIY solution you could use this (shameless plug): https://rsync-backup-ras.sourceforge.io/ https://rsync-backup-ras.sourceforge.io/

petters 3y ago

> here is how I do my backups. It takes me about ten seconds per day and five minutes per month to maintain > rsync everything to each of them, bringing one down to a safe deposit box downtown, and taking out the one that was there from my last visit Sounds like you are underestimating how much time you spend on backups. Sounds good though

EvanAnderson 3y ago
I assumed the author has to make the trip to the bank anyway. I integrate my off-site, offline backup storage with visits to my parents, for example.

TheCapeGreek 3y ago

My absolute final resort "solution" for data loss of keepsakes: Accept the impermanence of data, memories and memorabilia. Ideally I'll strive to keep that data backed up as long as I can, and will mourn its loss, but mentally I prepare myself that one day I might lose access in some way. Most importantly, that data is also 99% irrelevant to the next generation, maybe save for a handful of photos or videos ("what did grandpa look like?"). So in the grand scheme of things it's not like I have to be an epic archivist of my own life. I suppose it helps that this was done for me in the past - toxic family members threw out all my childhood family photos a decade ago.

72deluxe 3y ago
"Toxic" seems quite the understatement! How sad. I went to school with a boy who had no photos from his youth. His parents took many photos but didn't get them developed, and when they took all the films to the pharmacy (here in the UK) to get them developed, the pharmacy sadly burned down and they didn't ever get any photos. Quite sad I thought.

JonChesterfield 3y ago

Everything that matters is checked into fossil. Box hosting that uses zfs. That rules out most failure modes other than gross incompetence, partially mitigated by a periodic backup to external drives. In the before times I put everything in Dropbox. Dropbox synced with an empty directory by mistake and deleted everything. Not going to use Dropbox again.

agubelu 3y ago

Ah yes, HN's automatic title editorializing strikes again, removing 33% of the title's wordcount and changing it from "click here to see my backup method" to "hey, I do backups".

1oooqooq 3y ago

that's a somewhat expensive setup to still do all the work of dealing with dead drives and manually doing replication and dealing with conflicts (I fear the pain of dealing with his 4way daily backup of the same data over desktop/laptop/2 pen drives) zfs rent = 4+TB drive + $10/mo (base 1TB) + $5*3 (extra 3TB) = drive + $25/mo hetzner = 5TB $10.9/mo vultr = $5/mo (probably more) 4x ~4TB drives = $800 assuming you replace drives every 5yrs: $3,254 or $54.23/mo. assuming the drives last 10yr which are common now: $5,708 or $47.56/mo.

aChattuio 3y ago
A 4tb drive costs 80-120€ 7€/month for 5y
- 1oooqooq 3y ago
  that price is for barracudas. then you will be lucky to replace them every 2 years. wd gold and others that will last 5~10years are actually more expensive for 4TB ($500) because they are old stock for people that for some reason need that model. The sweet spot for top quality HDD is always 200~300. In whatever capacity du jour. (14~20TB today) nitipicking drive cost aside, the monthly services are the bulky of the cost.

aChattuio 3y ago

I do a zfs snapshot and than do a sync. I never delete those snapshots because the big items like images don't change.

jamesgreenleaf 3y ago

I've used Backblaze for years and it's worth the money. I've only done a full restore from it once after a laptop was bricked and it was seamless. There have been a few times that I needed a recently deleted file and it worked for that too.

p0w3n3d 3y ago

I'm using rdiff-backup to have versioned copies, however at some point of time trimming of the snapshots is required

djks422 3y ago

Are backup stored to Glacier is my house-burned-down archive for about $5/mo. It may cost a lot to get out, but at that point our family pictures and movies are gone anyway, so it would be worth the cost.

deleted 3y ago

[deleted]

eternityforest 3y ago

I use Vorta. I have two profiles, one for my home dir, and recently I added one for all dotfiles I would want to keep when setting up a new machine. It's only about 5 files but it avoids using yet another tool for dotfiles.