4 ms·
> That looks overly complicated In practice, it's simple enough. I have a backup script that does all the heavy lifting for borg and rsync. Syncthing operates
by vczf 6y ago
> That looks overly complicated
In practice, it's simple enough. I have a backup script that does all the heavy lifting for borg and rsync. Syncthing operates independently in the background, and I rarely log changes with git. The most tedious part of the process is logging new/changed/deleted files with fim.
> Why is there no single tool that can do all of those?
Good question, and I totally agree. I was actually looking for a solution that could do deduplicated incremental backups (versioning, like time machine) plus file integrity management. I couldn't find one. Nobody seems to care much about file integrity, and nowadays I'm actually unsure if this is a real problem for anyone except paranoid geeks.
Could also use a good frontend GUI that could notify you about changed files, allow you to easily mark directories for different levels of "tracking", easily browse past versions and restore them, etc. The CLI isn't doing any favors here.
> I mostly use rsync. Occasionally I destroy my backups by calling it wrongly, like missing a trailing slash. That could not happen if the copying and file integrity checking was combined in one tool
Try borg-backup, it'll be a massive improvement over plain rsync when it comes to backups.
> But it stores the database as json? That is a bad format to store file names. And written in Java it is probably going to be slow (I have over a million files in ~, and multiple copies of it in the backups, so I care a lot about performance)
Gzipped json on-disk, but I haven't noticed any problems with it over the last few years. I doubt java is bottlenecking the performance at all: it's limited by IO and CPU throughput for hashing the files. For everyday use, you don't need to hash every file in entirety to detect changes. There's a "fast" mode that checks a couple of blocks. You can also operate in a subdirectory of the repo to ignore files outside that subdirectory.
I also don't just use one massive fim repository. I have a .fim/ for each directory I care about (types of data) that a secondary scripts loops through for checking the status. That weakens the integrity guarantees since you can't log the movement of files across directories, but makes it easier to reorganize things.
> So many emojis in the commit log. Is that really necessary nowadays?
Yah, that's the most emojis I've ever seen in a git repo. Maybe 2017 was a different era.