6 ms·
If you prefer a similar approach, but as a single compiled binary, there's Restic: https://github.com/restic/restic https://github.com/restic/restic Update: ye
by nine_k 4y ago
If you prefer a similar approach, but as a single compiled binary, there's Restic: https://github.com/restic/restic https://github.com/restic/restic
Update: yet another take on basically the same approach, also as a self-contained binary: https://github.com/kopia/kopia https://github.com/kopia/kopia
- kova12 4y agoI recall trying to use restic instead of borg a couple years ago, and some major feature was unavailable. I don't recall what is was, I think it was compression, which made archives quite large, and required larger instances for backup.
- deleted 4y ago[deleted]
- anotherevan 4y agoIt probably was compression. The good news is compression is now available with Restic!
- RockRobotRock 4y agoDoes anyone have a take on Kopia vs Restic?
- btschaegg 4y agoA major factor why I wouldn't want to use Kopia (I looked into it) ist that it is opinionated with regards to how your system is set up (old-school unix FS layouts in a "pets, not cattle" way). It assumes the location of config files and does not allow you to change the backup path that's stored in a snapshot's metadata. That's bad if you want to use it 1) on NixOS (I don't want backup configs laying around in `~/.config`). As Indy famously said: "That belongs in a Nix expression!" Edit: On a second note: Why have mandatory config files at all? I don't have an issue with having the option or it being the default, but for my use case being able to specify the whole repository config via arguments sounds considerably more sane. 2) with ZFS snapshots (yes, I'm backing up `/path/to/dataset/.zfs/snapshot/<timestamp>/foo/bar`, but that should not be its path in the metadata!) OTOH, it seems to have the upside that you can apparently alter snapshots after the fact more easily (e.g. if you find out you shouldn't have backed up that gigantic VM image you just moved somewhere temporarily). I leave the decision on whether this is a footgun or not to you. And to be clear: The ZFS snaphot thing is also a pain with Restic, too. You can hack around it somewhat better with something like systemd-nspawn, but it really shouldn't be that hard.
- XorNot 4y agoZFS snapshot thing was also a huge pain with bup back in the day. I remember doing some work to add in a path graft system for it but it never got merged. Backup tool authors never seem to support this use case.
- Freaky 4y agoThis is why I made https://github.com/Freaky/zfsnapr https://github.com/Freaky/zfsnapr Instead of working out how to teach my backup tools about snapshots, I just mount them in a subtree and use that as a chroot env.
- cyounkins 4y agoFor ZFS snapshots with restic I don't use the timed snapshots and instead create a new specially named one. There's a code snippet at https://medium.com/@cyounkins/correct-backups-require-filesystem-snapshots-23062e2e7a15 https://medium.com/@cyounkins/correct-backups-require-filesy...
- proactivesvcs 4y agoFYI removing data from backup repositories (e.g. deleting files from a backup) is currently in the master branch of restic and will likely be in the next release.
- attentive 4y agoFor context, I am long time Kopia user. Looks like compression only added in the latest release of restic. No mounts on windows. No GUI? Lack of compression was likely the reason I went with Kopia.
- mekster 4y agoIn my book, backup tools don't count unless it has been used widely for a while without major issues repeatedly being reported. Kopia is too new for that state.
- Silhouette 4y agoThat's a reasonable point but the solution - as with all things backup - is diversification. Otherwise if everyone followed your logic then no new backup software or storage service could ever become established no matter how good it actually was. Given that all of the options being discussed here look technically better and in some cases more than an order of magnitude cheaper than other popular backup services and software discussed on HN in the past you could afford to run full redundant backups with multiple combinations of software and backing storage and still have more options for a much lower price than a few years ago.
- eli 4y agoIf you regularly actually test it, it should be fine. (Using a popular solution without testing it is riskier)
- g_p 4y agoKopia is pretty new, but looking through the documentation I felt it had some pretty good features that suggest it will be a good option in the future. A few highlights (for me at least): - They have implemented support for object locking and versioning to prevent a compromised set of S3 keys being used to erase backups. There seems to be a pull request outstanding to dynamically shift forward the compliance hold each time an object file is "deemed still used", but very few s3 bucket based backup tools seem to really think about resisting deletion and documenting best practice. I like to use minimum privilege and deny DeleteObject permission, but using versioning and object locking policies seems to be an interesting potential solution. - Inclusion of (optional) Reed Solomon error correction codes in backup objects. - a CLI verification option that will test a given % of your backup objects (at random) every time it's run, to give you opportunistic random verification to try find issues. - reasonably well documented CLI options for recovering data were anything to go wrong - as the developers mention, there are a wide range of ways to recover data. Interestingly, they appear to have actually implemented these, rather than leave them as theoretical. For example, CLI commands to interact with indexes, manifests, blobs, snapshots, etc. You don't need them normally, but having these available lets you look around a simple backup and understand the format and gain confidence in it, and that the tools work. The only feature it seems to me to be lacking is asymmetric encryption of backups, so you can keep the key needed for recovery away from the host you are backing up.
- binaryanomaly 4y agorestic is great and simple to use. Use it for archiving my backups to Google Cloud Storage.
- auxym 4y agorestic also seems to have better Windows support. Borg can can run in WSL but has seen limited testing under such, per their own docs.
- somishere 4y agoBig thumbs up for Kopia and its very simple GUI / strategy. Have been using it for a couple of years now to remote backup hard drives and working folders on a bunch of family macs to B2. Restored twice now - logic board & corrupt hd. Chose it after trialing both Borg and Restic for ease of use and storage cost. My monthly backblaze bill hovers around $1.40.
- crossroadsguy 4y ago1.40 for what kind of storage and transfer?
- somishere 4y agoStorage is only around 250gb on B2, significantly larger on device. Egress/other averages out fairly minimal, mainly class b transactions. https://help.backblaze.com/hc/en-us/articles/217667478-Understanding-B2-Pricing-Structure https://help.backblaze.com/hc/en-us/articles/217667478-Under...
- alwayslikethis 4y agoI tried Restic first due to its support of rclone. But it created many small files which is difficult to sync over my high-latency cloud service. I also saw an issue on Github about data loss, so I switched to using borg. Very satisfied with borg. Side note: they offer single compiled binaries for borg too.
- dimatura 4y agoYou can change the chunk size on restic, for what it's worth.
- noAnswer 4y agoI tested restic a few years ago. Wanted to like it, since they directly support OpenBSD. I backuped roughly 4TB of multi media data. It took a week. After that I did a re-run. No new data, so it only took a few minutes. Than I added a tiny txt file and let it run again. It took a week again! So I settled on borg. I use it for offsite backup. I did restores of individual files as well of whole snapshots after broken hard drives. There even was a time I used it with WSL to backup my parents data.
- aborsy 4y agoThese backup software are all incremental, work similarly and take similar times (difference is in details). If you lost locally cached metadata files keeping information on what has changed, they all have to scan the whole source again.
- nine_k 4y agoScanning 4TB for a week? It should not take as long even if you use magnetic tapes and change them manually. I suppose the latency of the remote server was too high.
- noAnswer 4y ago> work similarly and take similar times Except that I tested them and they didn't. I hadn't "lost locally cached metadata". And it didn't just scan all the source files again, it re-transferred them over the network. All 4TB of it! That's why it took so long. (At that time the sever was still local, because I wanted to make the initial run not over the Internet. So there wasn't even added latency.) I'm sure it was a bug, maybe even in combination with OpenBSD, which moste likely is fixed now. As I said it was years ago. I compared and was more happy with borg. Which stood the test of time and saved my ass multiple times.
- zephyr9 4y agoAnd Restic just got the ability to rewrite snapshots, i.e. retroactively remove material added to backup.
- raegis 4y agoDoes Restic do encryption well now? When I tried it years ago you had to type the password on the command line, and there was no interest in adding GnuPG support.
- wfriesen 4y agoI've been using it for a while now getting the password from a file.
- proactivesvcs 4y agoYou can also send the password via a password file or password command, by command line switch or environment variable.
- wdkrnls 4y agoDon't underestimate the value of a self contained binary. I used to use Borg and Attic. Then I needed to recover my backup only to find that all the python dependencies had gone out of sync on Arch Linux. After that I switched to Restic. Then I had a power failure during upgrade. After that I switched to Guix.
- ThomasWaldmann 4y agoYou could have just used the self-contained borg binary (download from github releases page). It contains everything needed (python, python libs, other libs) except glibc and related libs which must match the OS / kernel and therefore are intentionally not included.
- qwertygerty 4y agosurely,after fixing all the dependencies, and getting borg running again, you would've been able to access your backups?
- avodonosov 4y agoI read [1] that Restic, in contrast to Borg, supports deduplication between snapshots of different machines - attractive for me, in case I have copies of large files on several machines. Is that also true for Kopia? 1 - https://stickleback.dk/borg-or-restic/ https://stickleback.dk/borg-or-restic/
- tetha 4y agoMaybe I'm misunderstanding, but the deduplication of Borg happens at a repository level and after the client-side encryption. If several hosts are pushing archives containing the same files (or rather, file contents) into the same repository, they are deduplicated. We rely on this in some places in fact. We have a couple of redundant hosts all serving the same file set. This deduplication allows us to push backups from all systems without coordination and only the first backup of the same file set requires storage beyond metadata.
- nine_k 4y agoBut isn't that only possible if the private keys are the same across hosts? This may make sense, depending on your threat model.
- tetha 4y agoYes, all hosts accessing the borg repository need the encryption keys or the passphrase of the repository, since it relies on the homomorphic encryption of the repo. And yeah, we tend to partition our borg repos along functionality and needs of restore. For example, database backups and file store backups are split into different repos, but several file store hosts in the same cluster all write to the same borg repo. After all, if something allowed to compromise one file store host, it will most likely allow compromise of all identically setup file store hosts in that cluster. And on the other hand, in this way, we don't have to think about the backups if one of the file store hosts goes offline - the other two will just continue writing backups. This would be something that's really easy to forget and could bite very badly.
- avodonosov 4y ago
- PYTHONDJANGO 4y agoNote that restic has known unresolved problems with backing up to cifs shares due to some underlying go implementation details deep down in the stack: restic fails on repo mounted via CIFS/samba on Linux using go 1.14 build https://github.com/restic/restic/issues/2659 https://github.com/restic/restic/issues/2659 Of course there are some cifs issues in the borg tracker, too, but most of them seem to be based on unreliable network connections - while the restic problem seems to be related to golang details. With backups I would rather not trust a tool that has basic problems like accessing files on a network drive, but this is just me.