25 ms·

Restic – Backups Done Right

• shimonabi 5y ago
  I've been using Restic for at least 4 years to store encrypted, incremental backups on Backblaze. Before that I used duplicity. I pay around 0.40$ a month to have a backup of around 70 GB.
  • bool3max 5y ago
    Backblaze is that cheap? Or am I just out of touch as storage prices go?
    • anderspitman 5y ago
      Most likely Backblaze B2
    • corndoge 5y ago
      B2 is super affordable and restic supports it ootb
• erulabs 5y ago
  I built our backup service for our users on top of restic! Works great :)
  • lucb1e 5y ago
    What kind of service, is it a commercial thing I can use or do you mean in a company or so?
    • erulabs 5y ago
      It's part of kubesail.com, which helps folks host at home (but works with any kubernetes cluster). We use restic as part of the backup service which launches a container on your machine to locally encrypt and then upload data. Restic works great to de-dupe files and minimize the size of the backup and restore.
• sheerun 5y ago
  I'm still waiting for backup tool that uses asymmetric encryption (data encrypted using public key, decrypted using private key) and having write-only server mode (so bad actor can't remove backups)
  • dividuum 5y ago
    The append-only mode can be implemented using https://github.com/restic/rest-server https://github.com/restic/rest-server or services like rsync.net that offer read-only zfs snapshots. Doesn’t solve the asymmetric crypto of course.
  • k8sToGo 5y ago
    Incremental and differential backups aren't possible that way. Also Borg has an append only mode.
    • lucb1e 5y ago
      Why would that not be possible? Especially since you say borg can do it, and borg is said to be incremental and deduplicated? Note that write-only or append-only is a bit of a misnomer, since reading the files is fine (you need the decryption key anyhow before they're of any use). It's about not being able to overwrite or remove backup data without some verification that you're not ransomware or similar.
    • rssoconnor 5y ago
      Tarsnap provides incremental and differential backups with public key cryptography. So it must be possible.
    • eps 5y ago
      Only if the backup side is unreadable (which is different from being append-only).
      • remram 5y ago
        I'm the GP's setup, it's not readable without the private key. You could do deduplication on the encrypted blocks I suppose (is that secure?)
• cyounkins 5y ago
  One of my favorite things about restic is not about the software, it's the community. People in the forums and bug reports are respectful and decent, and it's fantastic to see.
• lucb1e 5y ago
  Previously: 176 comments, 2 years ago https://news.ycombinator.com/item?id=21410833 https://news.ycombinator.com/item?id=21410833 Related: "Restic Cryptography" by FiloSottile (2017), 36 comments https://news.ycombinator.com/item?id=15131310 https://news.ycombinator.com/item?id=15131310
• alpaca128 5y ago
  Seems great, but I don't find the "Quickstart" section helpful. A demo video that shows 2-3 unknown commands without context is not a guide, same with links to the full documentation. Both make sense on the landing page, but both don't tell me how to get started quickly. Edit: interestingly the restic repo's Readme is better in that regard. It's the demo video but in text form with explanations: https://github.com/restic/restic#quick-start https://github.com/restic/restic#quick-start
  • lucb1e 5y ago
    Quickstart: repo=/mnt/mybackupdrive/somefolder restic init -r $repo restic backup /my/files -r $repo restic mount /mnt/backedup -r $repo cp -r /mnt/backedup/snapshots/latest/accidentally-removed-directory ~/restored Where the repository can also be sftp or various other things. There are a ton of other commands and options, but to literally just get started, this is all you need for making and restoring data from a backup.
    • Datagenerator 5y ago
      Same goes for S3 support, working great. Been using MinIO and SeaweedFS as target, works identical to native AWS.
    • yosito 5y ago
      > unknown commands without context is not a guide This quickstart assumes some context that not every person starting to use Restic has. It could be improved by offering some more context for each line: # RESTIC QUICKSTART FOR LINUX # Choose were you want to save your backups MYREPO=/mnt/mybackupdrive/somefolder # Initialize a new restic repo at your chosen backup location restic init -r $MYREPO # Backup your files to the newly created repo restic backup /my/files -r $MYREPO # To restore a backup, first mount the repo restic mount /mnt/backedup -r $MYREPO # Browse the latest backup at /mnt/backedup/snapshots/latest ls -la /mnt/backedup/snapshots/latest # Copy the files you want to restore out of the repo cp -r /mnt/backedup/snapshots/latest/directory-to-restore ~/restored
• alyandon 5y ago
  Really looking forward to the day I can ditch borg backup and switch over but really can't until https://github.com/restic/restic/issues/21 https://github.com/restic/restic/issues/21 is addressed. I have to pay for cloud storage and the lack of compression would easily double my costs based on my testing.
  • l33tman 5y ago
    Borg locally in combination with rclone to AWS (or other) works very well, compression and deblocking.
    • lucb1e 5y ago
      What is deblocking? Did you mean deduplication?
      • l33tman 5y ago
        Yes, dedup of blocks :)
  • andrepd 5y ago
    Why are you looking forward to ditching borg? I've started to use it as my backup solution and it works wonders.
    • ollien 5y ago
      My main gripe with borg, personally, is that it's push only. I want to be able to back up my VPS without having to have it ssh into my home network.
      • andrepd 5y ago
        Ah true, that's a good point. I can't use it to back up my Android phone, for instance.
        • 5y ago
• jacquesm 5y ago
  Backups done right? That's Tarsnap iirc. Previously on HN: https://news.ycombinator.com/item?id=21410833 https://news.ycombinator.com/item?id=21410833
  • lucb1e 5y ago
    That's like comparing rsync to google drive. One is an open source tool where you can use whatever back-end you want, the other is a service. (Which is fine, just different kinds of things.) However, in this case it's the open source tool that has a much easier user interface (I am actually proficient with tar, but still my tarsnap experience is like comparing 'restic backup /my/files --repo /mnt/backupdisk' with https://xkcd.com/1168/ https://xkcd.com/1168/)
    • NoraCodes 5y ago
      > One is an open source tool where you can use whatever back-end you want, the other is a service. What is your definition of "service" that makes tarsnap - a company that asks you to pay them over time to provide an, uh, service - not one?
      • aniforprez 5y ago
        They probably meant this article is the open source tool and tarsnap is the service
        • lucb1e 5y ago
          Indeed. Restic is just something you apt install and nobody provides you any service (you have to organise your storage space yourself); tarsnap is not simply free to use for yourself with your own storage. (Not saying it has to be free, but that's what makes it the definition of a service you have to purchase.)
  • gorgoiler 5y ago
    tarsnap is solid but has two major weakspots — it’s not price competitive, at least for the b2c product retailed from the site, and it is very slow.
• BrandoElFollito 5y ago
  I did not see anywhere mention of deduplication - is this implemented in Restic? (I use Borg and the feature is an absolute must for me)
  • lloeki 5y ago
    IIRC restic operates at the file level on the source side and then stores data in blobs, and unchanged blobs are deduplicated (it operates as a content addressed store, kind of like git), which means iterative backups are possible. EDIT: see "blob" https://restic.readthedocs.io/en/latest/100_references.html#design https://restic.readthedocs.io/en/latest/100_references.html#... and "efficient" https://github.com/restic/restic#design-principles https://github.com/restic/restic#design-principles
  • lucb1e 5y ago
    Yes, it deduplicates but does not compress. Since most large files (mp3, jpg, mp4, docx, etc.) are already compressed, this is not a big issue but it's not ideal either. If I remember correctly, another tool either does both or does compression instead of deduplication, this might have been borg or bup. In case that's something you care about.
    • l33tman 5y ago
      Typical borg run: Original size Compressed size Deduplicated size This archive: 167.99 GB 136.78 GB 53.85 MB All archives: 2.93 TB 2.38 TB 133.66 GB Sorry about the formatting. But the compression is not completely irrelevant. Dedup of blocks between files and backups is of course the absolutely most crucial part though.
      • lucb1e 5y ago
        Thanks for the real-world stats! I must say it took me a bit to read it though, let me format it for others (hopefully also narrow enough for mobile): compress data | dedup 1 backup: 168G 137G 54M Σ backups: 2.9T 2.4T 134G Please correct me if 'one backup' and 'all backups' is an incorrect interpretation of 'all archives'. I wasn't entirely sure what you mean by that but I think I get the point. So in conclusion, adding compression saves about 17% (a $10 monthly bill would be $8.28 instead, if you pay per GB) for you.
• beebeepka 5y ago
  Somewhat offtopic but this reminds me: what's the best way to backup a Raspberry Pi? Mine has a 128GB SD card and that makes backing up the whole thing kind of silly. There has to be a better way. I've lost a few months of data twice because of my ignorance
  • wglb 5y ago
    For small systems in my home lab, I use rsync from the small devices (e.g., Rpi, Intel NUC, etc) to the big system with a couple of 6TB drives.
    • _wldu 5y ago
      I do this too. And you can combine rsync with gocryptfs (in reverse mode) to get strong encryption of your backups as well. This is especially important if you are storing the backups on a remote/untrusted device. https://wiki.archlinux.org/title/Gocryptfs#Usage https://wiki.archlinux.org/title/Gocryptfs#Usage
      • noir_lord 5y ago
        I use rsync to backup my desktop (which is mostly used for development) pretty much just rsync /home to a big enough external LUKS drive. Has saved my bacon a few times, I don't care about incremental snapshots or delta's (though I've used both rsnapshot and rdiff successfully in the past), I'm covering the "if the SSD blows up, how long to chuck a new one in and be back up" case not the "I might need that file from 6mths ago". I also have syncthing setup via /home/<user>/Shared/{Personal,Work}/ on every machine and important stuff I just chuck in Shared/ and forget about/it's available wherever I need it at any point. I've had really ornate bulletproof snapshot based backups but honestly for my particular use case they where more hassle than they were worth, rsync does what I want every time and has never let me down.
  • cliffwarden 5y ago
    I like rdiff-backup. For me, it's a good middle-ground between rsync and a full backup solution. https://rdiff-backup.net/docs/features.html https://rdiff-backup.net/docs/features.html
• Coxa 5y ago
  How does this compare to Borg, which I am currently quite happy with.
  • ghgr 5y ago
    I switched from Borg to Restic because it nicely integrates with cloud backup services. I've been using restic+backblaze happily for the last 2 years.
    • nix23 5y ago
      And it's just a single go-binary, i just trow it on a win/bsd/linux machine create a key, start the backup. I love the simplicity of it, however for more complex plans i use git-annex.
    • Jeff_Brown 5y ago
      Does restic provide time travel like Borg does?
      • IceWreck 5y ago
        By time travel if you mean viewing/mounting your repo at various checkpoints then yes.
  • bccdee 5y ago
    I used to use borg and I migrated away from it to Restic when I somehow corrupted my backup archive. I dunno what I did, but I started getting "non-utf-8 filename" python errors every time I tried to access it. It might have had something to do with the archive being on a removable disk. Anyway! I'm happier with restic now. It's never crashed for me, and it has native cloud backends. But it's ultimately just another backup application.
    • sheerun 5y ago
      And I managed to corrupt Restic archive, so I switched to Borg
• sandreas 5y ago
  And yet again: A list of alternatives. attic (python) - https://github.com/jborg/attic https://github.com/jborg/attic borg (c) - https://github.com/borgbackup/borg https://github.com/borgbackup/borg bupstash (rust) - https://github.com/andrewchambers/bupstash https://github.com/andrewchambers/bupstash duplicacy (go) - https://github.com/gilbertchen/duplicacy https://github.com/gilbertchen/duplicacy duplicati (c#) - https://github.com/duplicati/duplicati https://github.com/duplicati/duplicati duplicity (python) - https://github.com/henrysher/duplicity https://github.com/henrysher/duplicity kopia (go) - https://github.com/kopia/kopia https://github.com/kopia/kopia nfreezer (python) - https://github.com/josephernest/nfreezer https://github.com/josephernest/nfreezer rdedup (rust) - https://github.com/dpc/rdedup https://github.com/dpc/rdedup restic (go) - https://github.com/restic/restic https://github.com/restic/restic rclone (go) - https://github.com/rclone/rclone https://github.com/rclone/rclone rsnapshot (perl) - https://github.com/rsnapshot/rsnapshot https://github.com/rsnapshot/rsnapshot snebu (c) - https://github.com/derekp7/snebu https://github.com/derekp7/snebu tarsnap (c) - https://github.com/Tarsnap/tarsnap https://github.com/Tarsnap/tarsnap I think there are many more out there (https://github.com/restic/others https://github.com/restic/others) - I personally use restic while technology wise (speed, only restore needs password) i would prefer rdedup which is an impressive piece of software but unfortunately without file iterator... :-)
  • ComodoHacker 5y ago
    Might add zpaq (C++) - http://mattmahoney.net/dc/zpaq.html http://mattmahoney.net/dc/zpaq.html to the list.
    • nix23 5y ago
      and especially lzip and lrzip then too. But zpaq is pure magic...a slow one but with massive compression.
  • nix23 5y ago
    attic is python and c borg is python and c bupstash is rust and c Maybe update your list a bit ;)
  • 5y ago
• k8sToGo 5y ago
  For the longest time I used to use borg. These days, however, I am using kopia[0] https://kopia.io/ https://kopia.io/ Quite happy with it.
  • jarenmf 5y ago
    Seems like an interesting project, what are the differences from borg?
    • k8sToGo 5y ago
      Well it is written in Go instead of Python. If I remember correctly, borg is single threaded and quite slow. Kopia is really fast. Other than performance, the biggest benefit is that multiple clients can write to one repository at once. There is a designated maintenance user, though.
    • mekster 5y ago
      When it comes to backup software, do yourself a favor and do extensive research on their stability and reliability. From what I learnt, borg has least problem of all the open source backup software with the most wanted features (encryption, dedup, compression and rotation) and others all have their quirks, be it huge memory usage, data loss to worst being corruption of the backup repository. Things may have improved since I've looked but you want user feedbacks saying so. It's too late to know that backup software has been failing when you need it as the original data is already unavailable, so choosing by the look of landing page is a bad way when it comes to backup. kopia is still new and last time I used about a year ago, it still had basic problems, so I would never use it in place of borg. restic, duplicati, duplicity and duplicacy all have some sort of problems especially when the repo gets large but of course there are cases where things are working fine. https://forum.rclone.org/t/rclone-as-destination-for-borgbackup/12717/9 https://forum.rclone.org/t/rclone-as-destination-for-borgbac... (Restic issues) https://www.reddit.com/r/Backup/comments/opu1ep/comment/h67n6s4/ https://www.reddit.com/r/Backup/comments/opu1ep/comment/h67n... (Kopia issues) https://forum.duplicacy.com/t/memory-usage/623/24 https://forum.duplicacy.com/t/memory-usage/623/24 (Duplicacy doing a big rewrite of the engine to fix memory issues.) https://forum.duplicati.com/t/is-duplicati-2-ready-for-production/4714 https://forum.duplicati.com/t/is-duplicati-2-ready-for-produ... (duplicati stability issues) https://www.reddit.com/r/unRAID/comments/eg0zpe/duplicati_seems_too_fragile_for_realworld_use/ https://www.reddit.com/r/unRAID/comments/eg0zpe/duplicati_se... (Another duplicati issues) borg also has change logs with any major reliability problems mentioned up front which gives you more confidence than serious bugs buried in GitHub issues like other tools. https://borgbackup.readthedocs.io/en/stable/changes.html https://borgbackup.readthedocs.io/en/stable/changes.html The only downside of borg is it can only target ssh host natively, but there are services like rsync.net (with special borg pricing), borgbase or you could locally run borg and rclone the entirety to anywhere you want.
• lights0123 5y ago
  Restic is great. Used to use Duplicati because of its GUI, but Restic is far faster, reliable, and restoring files via FUSE is extremely easy to use.
• anotherhue 5y ago
  Restic makes it trivial to use the very cost effective Backblaze B2 export \ B2_ACCOUNT_ID=123456 \ B2_ACCOUNT_KEY=DEADBEEF\ RESTIC_REPOSITORY=b2:myname-restic-myhost \ RESTIC_PASSWORD=CAFEBABE restic backup --verbose --host myhost \ --exclude /not_this \ /yes_this \ /and_this
• senotrusov 5y ago
  Restic is amazing. I like how Restic could be automated. I usually work in multiple Linux virtual machines and I have a Bash script to setup regular backup of all my many $HOMEs. I did not yet have the script for verification (did it manually just to be sure), but the rest is here if someone is interested https://github.com/senotrusov/sopkafile/blob/main/lib/ubuntu-workstation/backup.sh https://github.com/senotrusov/sopkafile/blob/main/lib/ubuntu...
  • tailspin2019 5y ago
    Cool repo! You've now sent me down a deep Sopka rabbit hole..
    • senotrusov 5y ago
      I'm really glad that you find it interesting, it was my solo project for the long time. Feel free to contact me on discord stan#9673 if you need any help or just for the general shell-scripting related chat :)
• deleted 5y ago
  [deleted]
  • deleted 5y ago
    [deleted]
• tailspin2019 5y ago
  I've always liked the look of Restic. I should really start using it to backup my Linux servers. However for desktop use, I've always really struggled with the idea of not having a UI for my backup client. I'm not afraid of the command line, but the idea of browsing backup archives without a GUI feels awkward to me. I wonder if there is room for some sort of add-on GUI for Restic for those that are more visual (unless such a thing already exists?)
  • cpach 5y ago
    I’ve thought a lot about this. One alternative solution could be this: Instead of backing the laptop up directly, sync it to a NAS. (Using rsync or a similar tool.) Then run Restic on the NAS and back up the data from the NAS to S3 or similar.
  • sandgiant 5y ago
    You can use `restic mount` to mount a repository and browse and restore backed up files using your file browser GUI of choice. Works quite well.
    • tailspin2019 5y ago
      Wow, that's awesome. I think many more people would use Restic if this was called out prominently on the home page. I've known about Restic for years and would likely be using it by now had I realised that! EDIT: Ah looks like it may not work on Windows. Part of the appeal of Restic, for me, would be being able to cover all of my Windows, Mac and Linux machines with the same system.
      • nightski 5y ago
        I'm guessing it would work in WSL.
    • cpach 5y ago
      Ooh, didn’t know. Sounds very useful!
  • 5y ago
• pawelduda 5y ago
  Oh yeah, restic is awesome _backup_prepare () { export $(sudo cat <protected_credentials_file> | xargs) } _backup_remove_old_snapshots () { restic forget -r <repo_name> --keep-weekly 10 } _backup_verify () { restic check -r <repo_name> } backup () { echo "---------------Scheduled backup time---------------" echo "" _backup_prepare restic -r <repo_name> --verbose --exclude="$HOME/snap" --exclude="$HOME/Android" --exclude="$HOME/.android" --exclude="$HOME/ApkProjects" backup ~/ echo "" echo "---------------Backup done, removing old snapshots---------------" echo "" _backup_remove_old_snapshots echo "" echo "---------------Old snapshots removed, verifying the data is restorable---------------" echo "" _backup_verify echo "" echo "---------------Backup done and verified!---------------" } Then in cron I just have this entry: 20 \* \* \* DISPLAY=:0 kitty -- /bin/zsh --login -c 'source ~/.zshrc; backup; read' So every time my PC is on @ 20:00, a shell window will pop-up, asking me for password and runs the backup :). Since they are incremental, it takes maybe 10-15 minutes top.
  • kkjjkgjjgg 5y ago
    My eyes immediately glazed over seeing the code, tbh. Not what I expected from a super simple backup solution.
    • number6 5y ago
      The relevant restic part is: restic -r <repo_name> --verbose --exclude="$HOME/snap" --exclude="$HOME/Android" --exclude="$HOME/.android" --exclude="$HOME/ApkProjects" backup ~/
    • pawelduda 5y ago
      Never mentioned it's super simple. It's DIY that I stitched together in 1-2 hrs of incremental upgrades and it's been running like this for months. To me it's simple enough but YMMV
      • kkjjkgjjgg 5y ago
        Sorry it wasn't meant as a criticism of your work. I just had hoped for something simpler from the "Backups Done Right" headline. To me if it requires complicated shell commands it is not really "done right".
• sandgiant 5y ago
  I've used restic for over four years. It's been rock solid on all platforms (mac/windows/linux). I've set it up on everything from beefy linux servers to 10 year old Windows machines (scheduled job running for 3+ years, pruning on a separate machine, never had any problems). The support for B2 and S3 compatible backends as well as rclone makes it a breeze to set up. The community and maintainers are also very friendly and helpful. Highly recommended!
• rcarmo 5y ago
  I have a Makefile to invoke restic on just about every Linux machine I have (even Raspberry Pis), usually to do incremental backups to Azure blob storage. It is a great, no-fuss, “forever” tool I’ve relied upon for years, and has made it trivial to clone or restore some pretty weird setups. Just mind you exclude node_modules and the like from it :)
  • lucb1e 5y ago
    Do you also run things like 'restic check' on it from time to time? I once managed to corrupt a restic repository after ctrl+c'ing the program and pulling a disk before it was apparently done syncing the write cache (I wanted to leave and catch a bus but noticed that this was still running), not sure if there are other conditions where that might happen. Better to know ahead of time to make a new backup before the primary copy breaks.
• mushufasa 5y ago
  "Once version 1.0.0 is released, we guarantee backward compatibility of all repositories within one major version; as long as we do not increment the major version, data can be read and restored. We strive to be fully backward compatible to all prior versions. During initial development (versions prior to 1.0.0), maintainers and developers will do their utmost to keep backwards compatibility and stability, although there might be breaking changes without increasing the major version. " they are on 0.12 now
  • lucb1e 5y ago
    For what it's worth, it has a good track record, but yes you're totally right that restic is not exactly commercial-grade software with proper guarantees. In one of the early talks at a local hackerspace, the author did also demo decrypting the data manually if something got somehow broken. The tool is just a layer on top of relatively straightforward cryptography. From what I remember (I looked at this in 2018 so forgive any errors), you'd have to write a script that iterates over the index where it says which blocks are in which files (since it's deduplicated) and decrypt it with some standard AES library. Perhaps as a security consultant this seems easier to me than it does to others, though, but it's not as if you're without hope if the tool did break, or as if you couldn't just download a previous version from the GitHub releases page.
  • 41b696ef1113 5y ago
    One of the reasons I prefer restic to borg is it is trivial to maintain a standalone copy of the executable. Wherever I put backups, I keep a copy of the restic executable used to generate the dump. For extra paranoid, could clone the restic source tree (with vendored dependencies). Go language backwards compatibility is such that I should always be able to read my data.
• nickcw 5y ago
  Restic is a great backup program. I use it (via rclone) to back up my laptop and it truly saved my bacon when I attached my main hard disk to a VM I was playing with and accidentally formatted it! Restic can use rclone to access cloud providers it can't access and that's something we worked together on. I use rclone directly for backing up media and files which don't change much, but restic nails that incremental backup at the cost of no longer mapping one file to one object on the storage which is where rclone shines. (rclone author)
  • aborsy 5y ago
    Will there be any issue if an rclone crypt remote is selected to be the backend for restic? Is there any interaction between these two layers of encryption, or are they completely independent?
    • IceWreck 5y ago
      Not OP but why would you want to use rclone crypt with restic if restic already does the encryption for you.
      • aborsy 5y ago
        There are sometimes vulnerabilities in software. For instance, popular rclone and tarsnap have had fatal vulnerabilities in the past. Also, I have a crypt set up for other stuff and rather deal with one remote. Otherwise, cascades encryption is generally not recommended.
        • saganus 5y ago
          Do you have any links about the tarsnap vulnerability? I've been using it for a few years and was not aware of it so would be great to know if any backups were affected.
          • cperciva 5y ago
            https://www.daemonology.net/blog/2011-01-18-tarsnap-critical-security-bug.html https://www.daemonology.net/blog/2011-01-18-tarsnap-critical... (2011)
• sneak 5y ago
  I used to religiously make backups, and I still rsync my homedir onto a server occasionally. That said, I keep ~/Documents and ~/dev these days in Syncthing directories, and one of my syncthing nodes is an Ubuntu LTS server with zfs, with the zfs-auto-snapshot package installed. I still run my old backup system periodically (once or twice a month) but I now think Syncthing is at a point of reliability where realtime cross-machine sync is now my primary safety net wrt "the machine in front of me has turned to entropy", versus some point-in-time backup.
  • dagenix 5y ago
    From the Syncthing FAQ (https://docs.syncthing.net/users/faq.html#is-syncthing-my-ideal-backup-application https://docs.syncthing.net/users/faq.html#is-syncthing-my-id...): > Is Syncthing my ideal backup application? > No. Syncthing is not a great backup application because all changes to your files (modifications, deletions, etc.) will be propagated to all your devices. You can enable versioning, but we encourage you to use other tools to keep your data safe from your (or our) mistakes.
    • diarrhea 5y ago
      I would argue in conjunction with zfs snapshots (can be every minute), it's safe and the caveat no longer applies.
      • dagenix 5y ago
        If you accidentally reformat the ZFS drive, that will delete all your files and snapshots and then Syncthing will replicate those deletions. I'm really not super familiar with Syncthing, but it sounds like it's niche is availability as opposed to durability.
        • multiply 5y ago
          Even if you format your drive and Syncthing copies over the empty files you will still have a history of snapshots on the other host running ZFS snapshots.
• johng 5y ago
  Emborg with borg is my favorite combo right now.
  • lucb1e 5y ago
    Care to elaborate on how it compares and what emborg is (I haven't seen that mentioned before)?
    • johng 5y ago
      It’s just a front end for borg. Makes things a bit more user friendly. https://emborg.readthedocs.io/en/latest/ https://emborg.readthedocs.io/en/latest/
  • _emacsomancer_ 5y ago
    Vorta is another nice Borg frontend.
    • johng 5y ago
      I use Emborg on servers with no GUI.
      • _emacsomancer_ 5y ago
        Right. I didn't mean to suggest Vorta as a 'competitor' to Emborg, just as another nice interface to Borg.
• Kavelach 5y ago
  A while ago, I wrote a blog post on how I prepared an automatic backup solution: https://kavela.ch/article/backups-on-linux.html https://kavela.ch/article/backups-on-linux.html
• 37ef_ced3 5y ago
  Why wouldn't you use tarsnap instead? https://www.tarsnap.com/ https://www.tarsnap.com/
  • lucb1e 5y ago
    The tarsnap subthread is here: https://news.ycombinator.com/item?id=29209658 https://news.ycombinator.com/item?id=29209658
  • encryptluks2 5y ago
    Well, for one Tarsnap is incredibly expensive. I get 6TB of cloud storage for roughly $65/year with unlimited transfer. Tarsnap would be in excess of $1500 per month.
    • FireBeyond 5y ago
      Holy crap. Yeah. 3TB, $750/month?!?
• mherrmann 5y ago
  I use restic to back up my 1TB external HDD to Backblaze for 5$/mo. It's great.
• peterbozso 5y ago
  Restic is a great piece of software. I use it for more than two years now to do encrypted backups of my home server into Backblaze B2. Took minutes to set up with a couple of lines of script, works like a charm since then. Highly recommended!
  • argiope 5y ago
    Same here! Really happy with the setup. Got an Odroid HC2 doing daily backups to Backblaze B2. The thing doesn't even sweat scanning around 1TB of data. I find the automated pruning of old snapshots also pretty sleek: restic forget --keep-daily 7 --keep-weekly 5 --keep-monthly 12 --keep-yearly 75
    • johntash 5y ago
      Just a heads up in case you aren't already aware, 'restic forget' does not automatically prune data. You also need to pass --prune or run 'restic prune' later. Other wise, your snapshots are dropped from the index but the data used by them still exists in the repo.
• guerby 5y ago
  Does restic have a user interface on windows/Mac?
• deleted 5y ago
  [deleted]
• cadence- 5y ago
  Can this be used to incrementally backup the whole / (root) directory? Or is this only for non-system folders with data?
  • lucb1e 5y ago
    > Can this be used to incrementally backup the whole / (root) directory? That's how I use it (with --one-file-system), but I haven't tried restoring that. I just check that my data is there (e.g. open some pictures) and call it a day. For me, if I need to reinstall the system anyhow (super rare), I'm also happy to just reinstall the OS, do a bunch of apt installs, restore a few files in /etc perhaps, and put my homedir back. So I can't vouch for whether it will store all special attributes on system files, but I would expect that common things like owner/group/mode are there. Fwiw, I once rsync'd a remote root partition to the current machine and that worked. Would not recommend to depend on this, but it apparently doesn't take much to be able to restore the root and boot from it :). You can also quite easily test this in a VM, if you'd like to make sure it works.
• sa1 5y ago
  I need a backup system in which files are never deleted, even if they are deleted locally. These files should also be easy and efficient to search for; you shouldn't have to go through all previous snapshots, trying to guess which snapshot a file might be in. Does restic do this?
  • colordrops 5y ago
    Yes on deletes, backups are incremental. Not sure about how easy to find files but I'd be surprised if there weren't commands for doing this.
    • sa1 5y ago
      I'm not sure, but seems to be an expensive operation on snapshot based architectures - have to search through all snapshots.
      • lucb1e 5y ago
        If you know the path, it would be $ restic mount --repo /mnt/backupdrive/myrepo /mnt/backedup $ echo /mnt/backedup/snapshots/*/path/to/file And it would print which snapshots it is contained in. If you don't know the path, I don't know how long `find -type f` takes so you might be right about that being inefficient. It is certainly not a use-case that I think restic ever had in mind. I also don't know of "backup" software to solve this, it seems a bit out of scope for most of them (they are meant to have a backup copy of your disk, not be a file manager with history). You might have more luck with tools like rsync (or Toucan, from the good old days where I used Windows and Portable Apps), those can certainly create only new files and never delete deleted files, but then you typically don't get encryption and deduplication.
        • sa1 5y ago
          Thanks, I’ve also thought that something built on rclone or rsync might be more suitable. I haven’t yet moved to restic, so it might be that find -f is good enough for me.
• deleted 5y ago
  [deleted]
• rmoriz 5y ago
  Doing sysadmin/devops for over 20 years now and restic is the easiest and most robust backup tool I've seen.
  • lucb1e 5y ago
    Robust in what way? If you mean user error then I would agree, it's super easy to manage backups and not do stupid things (I say this without much experience with other tools, fwiw). But as for file corruption, is it much different from the other standard solutions out there like borg or duplicity?
• jturpin 5y ago
  Restic is great, I use it to back up my Nextcloud data from my raspberry pi to two locations, a separate USB drive and a remote DigitalOcean space. Very easy to use and all encrypted. I wrote a blog post about it actually! https://compileandrun.com/2021-01-31-nextcloud-traefik-restic/ https://compileandrun.com/2021-01-31-nextcloud-traefik-resti...
• blunte 5y ago
  I was wondering how this compared to Bacula, which I setup for a company with significant amounts of data to backup (and which worked great, especially after I wrote a retention management + pruning tool). Here's an old discussion: https://news.ycombinator.com/item?id=19485783 https://news.ycombinator.com/item?id=19485783
• rcthompson 5y ago
  My current backup system involves having all my data on ZFS and then using sanoid for snapshots and syncoid for syncing those snapshots to my server that also runs ZFS. Would I gain anything significant from switching to something like Restic for backups?
  • mavhc 5y ago
    I use syncoid to sync to a server that only has 1 open port, ssh, which only allows one user, and that non root user runs rbash and can only do zfs receive, not zfs destroy. Hopefully that stumps the ransomware people trying to delete backups.
• goranerik 5y ago
  Truly suprised not seeing burp mentioned anywhere in these comments. Super reliable, laptop friendly and with lots of Nice features. https://github.com/grke/burp https://github.com/grke/burp
  • liveder 5y ago
    use it for the last three years. good so far!
• wfleming 5y ago
  Like others here, I'm a big fan of restic. I use it to backup to backblaze B2, and have systemd timer units to run it daily. I use it with pass (https://www.passwordstore.org/ https://www.passwordstore.org/) for secrets management, my wrapper script is at https://github.com/wfleming/dotfiles/blob/arch-linux/home_nodot/bin/backup.sh https://github.com/wfleming/dotfiles/blob/arch-linux/home_no... if it's useful for anyone.
• Malfunction92 5y ago
  I’ve been using Backblaze to back up my mac for the last four years. Would there be any advantage for me to look into a self-hosted alternative like Restic?
• dang 5y ago
  Past related threads: Saving a restic backup the hard way - https://news.ycombinator.com/item?id=28438430 https://news.ycombinator.com/item?id=28438430 - Sept 2021 (2 comments) Restic Cryptography (2017) - https://news.ycombinator.com/item?id=27471549 https://news.ycombinator.com/item?id=27471549 - June 2021 (5 comments) Backups with restic and systemd and prometheus export - https://news.ycombinator.com/item?id=27411713 https://news.ycombinator.com/item?id=27411713 - June 2021 (1 comment) Restic – Backups Done Right - https://news.ycombinator.com/item?id=21410833 https://news.ycombinator.com/item?id=21410833 - Oct 2019 (177 comments) Append-only backups with restic and rclone - https://news.ycombinator.com/item?id=19347188 https://news.ycombinator.com/item?id=19347188 - March 2019 (42 comments) Restic Cryptography - https://news.ycombinator.com/item?id=15131310 https://news.ycombinator.com/item?id=15131310 - Aug 2017 (36 comments) A performance comparison of Duplicacy, restic, Attic, and duplicity - https://news.ycombinator.com/item?id=14796936 https://news.ycombinator.com/item?id=14796936 - July 2017 (49 comments) Restic – Backups done right - https://news.ycombinator.com/item?id=10135430 https://news.ycombinator.com/item?id=10135430 - Aug 2015 (1 comment)
• neilv 5y ago
  Any opinions on Restic vs. Borg Backup (or Borgmatic), for encrypted backups (over SSH and to USB drives)?
  • sheerun 5y ago
    Both failed me with recovery at some point (corrupt indexes)
    • PenguinCoder 5y ago
      I had Borg fail me the same way with index corruption. Since moving to Restic I haven't had any issues backing up or restoring, and it seems quicker too.
  • _h9mb 5y ago
    Using Borg for more than a year I haven't had an issue yet. In the past I used a similar tool that had error-correction capability but the tool was buggy and slow when compared to Borg.
  • photon-torpedo 5y ago
    Why not use both? Recommendations are to have two backups anyway, one local and one remote. I use borg to make local backups to a USB HDD, and restic for remote backups in the cloud. Using different software guards against implementation bugs.
  • major--neither 5y ago
    both are fantastic. i started on borg, and moved part of my shit to restic so i use both. borg ui is a bit better, but you can normalize that away with a bunch of scripts.
• krick 5y ago
  Using the chance to ask: what is currently the cheapest, most efficient place for home PC backups (on linux)? Backblaze B2? Something else?
  • _def 5y ago
    Hard drives I guess (happy if someone can correct me on that)
  • brosciencecode 5y ago
    rsync.net is awesome. You get a Linux filesystem you can access via SSH. Many tools support it (like git-annex), and since it's ssh-able, you can script anything that doesn't work out of the box. They also have optional replication, and you can choose the location of both of the servers that host your files. Great support from actual engineers, used by huge companies for their offsite backups, and never been breached (to my knowledge). EDIT: their features page is a good read. https://www.rsync.net/cloudstorage.html https://www.rsync.net/cloudstorage.html
    • krick 5y ago
      Sounds very enticing, however $25/m for 1 TB seems rather pricey. Is replication that x2, or is it included?
      • AnonC 5y ago
        I'm not the GP. There's a slightly cheaper "borg backup" plan at rsync.net. [1]: https://www.rsync.net/products/borg.html https://www.rsync.net/products/borg.html
  • lucb1e 5y ago
    €0.40/TB/month or €120 per 5 years for 5TB: Old phone or a raspberry pi with a hard drive attached located at a friend's or family member's house. Any cloud service is going to be 5-10 times more at least, last I checked (2019), and that's already a lot better than five years before that (as a ratio of self-hosted to a managed service, so independent of raw storage prices). B2 and Glacier seem to be some of the cheaper options these days. Backblaze (the backup software) doesn't run on Linux and is closed source but they pinky promise to support "unlimited" backups for $5/month which is a really good deal if you both trust them and run Windows. Tarsnap is S3 pricing plus markup, but what you get is linux support, open source clients from a person the community trusts, write-only keys, and the hosting part is off your hands, and pay-as-you-go, which is quite a unique combination. Edit: B2 is $5.7/TB/month according to https://news.ycombinator.com/item?id=29209665 https://news.ycombinator.com/item?id=29209665 (`0.4/70*1000`)
• galaxyLogic 5y ago
  Can you backup your C-drive?
• goodpoint 5y ago
  Duplicity is excellent. Furthermore, Duplicity is written in Python so you get tracebacks and quick fixes are really easy to do. How does Restic compares to it?
  • mbirth 5y ago
    And how does it compare to duply, which let me boil down my duplicity setup to 8 lines config, 3 lines of globs (2 includes, 1 global exclude) and this line in my crontab: 30 4 * * * /usr/bin/duply myjob backup_purge_purgeFull --force > /tmp/duply_myjob.log
• StillBored 5y ago
  Hmm, anyone know if they fixed some of the exponential time issues it was having? I was really excited about some of the features and tried it a couple years ago, and it died before getting very far sync'ing my NAS to a usb3 jbod I plugged directly in to prime it for a rpi4 network attached backup. For something like that I would expect the backup speed to be a few hundred MB/sec and it quickly was just running in the 10's of MB/sec and getting slower. I've noted this before that my NAS has a tendency to kill a lot of backup applications that haven't really been stressed. Its about 50 something TB of data, made up of a fair number of compressed video files, family movie kinds of things and about ~7T of source files. The combination of a crapton of tiny files and 50T of data kills most of these more recent open source backup applications, which seem to be released with the "it can backup my laptop" testing. Also, I see someone mentions it sill doesn't have a compressed block option, which I really want because most of those source files compress to about 1/3rd of their space, which is important since my upload speeds are crummy (thanks spectrum!) I wonder how much of that deficiency is just lack of good go compression routines that can run at a few hundred MB/sec.
  • johntash 5y ago
    My main dataset that I back up with restic is around 11 million files or so, and fluctuates between 8-15TB of source data. I think I've run into similar issues testing out various backup tools too, but have settled on restic for now since it's been the most reliable. I'd definitely recommend trying restic again if it's been a couple years. Somewhat recently they made some nice speed improvements. It used to take me several days to do a forget+purge on my restic repo. Now it takes only a few hours (less than an actual backup takes). How many files are you backing up? I found the biggest issue for me was that almost every tool tries to keep the list of files in memory, so once you get into millions of files - it starts to require a lot more memory and can crash on low-resource machines.
• alexktz 5y ago
  Autorestic wraps restic in YAML config files, and for that I am very grateful. https://github.com/cupcakearmy/autorestic https://github.com/cupcakearmy/autorestic
• Arch-TK 5y ago
  What if I don't want to encrypt? I would like tarsnap/restic/whatever (but unlike tarsnap, I want to be able to self host) and I want it without encryption.
  • aborsy 5y ago
    Use a default password, eg, 123.
• Bellamy 5y ago
  What about compression? What does restic use under the hood?
• pabs3 5y ago
  There are a few downsides to using restic: No support for compression yet No support for deleting data from snapshots No support for continous backups (restic walks the directory tree for each backup). No support for resilience from disk errors using par2 or similar. No directory chunking, so backing up millions of files in one directory uses a lot of memory.
  • kzrdude 5y ago
    A simple duplicity then seems superior, unfortunately. Duplicitly seems solid but a bit complicated, possibly fragile (with so many components fitted together, like par2 stacking on top of any other storage).
• zxcvbn4038 5y ago
  For me the one to beat is IBM’s backup utility (known for years as ADSM (adstar storage manager) and later called TSM (tivoli storage manager). I’ve never seen a commercial or open source program that comes close. I’ve also never seen successful backups anyplace that did not have TSM. Usually the backups are corrupt, or nobody knows how to do a restore, or you need 1000x the storage capacity in order to restore the initial backup and all of the incremental backups until you reach a specific point in time. At places I worked with TSM it was so simple that individual users could fire up a gui and pull files out of the backup pool. On the backend we had massive IBM tape libraries and it was hypnotic to watch the robot jet around moving tapes in and out of drives and the storage slots. It never stopped moving either, when backups are restored were not happening it was busy consolidating the tapes, making copies of data from tapes that had been used too many times, or preparing copies to be sent off site. It was a full time job for someone to load new blanks when TSM requested and remove the offsite tapes and put them into a box for fedex to pick up. (The one thing that has not changed is that it’s still quicker to send massive amounts of data by fedex then it is to send it over the public internet)
  • fsh 5y ago
    I have used TSM (or ADSM or Spectrum Protect or whatever IBM calls it this week) quite a bit. The basic functionality and performance are not too bad. However, it clearly shows that the software originates in the 1980s. The client is written in C++ and really likes to leak memory. This becomes problematic when backing up more than a few million files. The official "fix" suggested by IBM is to configure a cronjob that restarts the scheduler once a day (seriously). TSM also has no support for deduplication, so good luck backing up large variable binary files such as VM images or project files (video, CAD, etc).
    • zxcvbn4038 5y ago
      I’m pretty sure it did originate in the 80s, it had an earlier name than ADSM, then was rebranded back when IBM was going to split itself into “baby blues”, then Lou Gerstner took over and stopped the split-up. Despite its faults it’s still the best I’ve ever encountered.
• axelthegerman 5y ago
  > Once version 1.0.0 is released, we guarantee backward compatibility of all repositories within one major version; as long as we do not increment the major version, data can be read and restored. We strive to be fully backward compatible to all prior versions. > During initial development (versions prior to 1.0.0), maintainers and developers will do their utmost to keep backwards compatibility and stability, although there might be breaking changes without increasing the major version. Hmm not sure I would like to try on a backup tool that might introduce breaking changes that easily
• major--neither 5y ago
  everything works fine in wsl2 too including the fuse mounts. go golang!
• helloworld11 5y ago
  I'm a bit confused. For all the description of different convoluted processes below, why not simply handle backup with something even simpler like Backblaz, SpiderOak or maybe sync.com? Seems a lot simpler than the Restic, Rclone and other methods described here.