3 ms·

I tested restic a few years ago. Wanted to like it, since they directly support OpenBSD. I backuped roughly 4TB of multi media data. It took a week. After that

I tested restic a few years ago. Wanted to like it, since they directly support OpenBSD. I backuped roughly 4TB of multi media data. It took a week. After that I did a re-run. No new data, so it only took a few minutes. Than I added a tiny txt file and let it run again. It took a week again! So I settled on borg. I use it for offsite backup. I did restores of individual files as well of whole snapshots after broken hard drives. There even was a time I used it with WSL to backup my parents data.

aborsy 4y ago
These backup software are all incremental, work similarly and take similar times (difference is in details). If you lost locally cached metadata files keeping information on what has changed, they all have to scan the whole source again.
- nine_k 4y ago
  Scanning 4TB for a week? It should not take as long even if you use magnetic tapes and change them manually. I suppose the latency of the remote server was too high.
- noAnswer 4y ago
  > work similarly and take similar times Except that I tested them and they didn't. I hadn't "lost locally cached metadata". And it didn't just scan all the source files again, it re-transferred them over the network. All 4TB of it! That's why it took so long. (At that time the sever was still local, because I wanted to make the initial run not over the Internet. So there wasn't even added latency.) I'm sure it was a bug, maybe even in combination with OpenBSD, which moste likely is fixed now. As I said it was years ago. I compared and was more happy with borg. Which stood the test of time and saved my ass multiple times.
  - bttger 4y ago
    This probably wasn't a bug. I experienced the same last week but it was because I ran the backup command from a different directory. I've created a note saying: `restic backup -v -r <location>/<repo_name> <source_dir>` first `cd` to the parent directory of the `source_dir` to avoid too many nested directories in the repository and conflicts in the mounting point path (so that the file change detection of restic works when mounting the drive to be backed up at another point)
shaicoleman 4y ago
restic has drastically improved peformance recently - increased pack size, new repository format, compression, configurable concurrency, etc. The performance issues you experienced may have been resolved.

aborsy 4y ago

These backup software are all incremental, work similarly and take similar times (difference is in details). If you lost locally cached metadata files keeping information on what has changed, they all have to scan the whole source again.

nine_k 4y ago
Scanning 4TB for a week? It should not take as long even if you use magnetic tapes and change them manually. I suppose the latency of the remote server was too high.
noAnswer 4y ago
> work similarly and take similar times Except that I tested them and they didn't. I hadn't "lost locally cached metadata". And it didn't just scan all the source files again, it re-transferred them over the network. All 4TB of it! That's why it took so long. (At that time the sever was still local, because I wanted to make the initial run not over the Internet. So there wasn't even added latency.) I'm sure it was a bug, maybe even in combination with OpenBSD, which moste likely is fixed now. As I said it was years ago. I compared and was more happy with borg. Which stood the test of time and saved my ass multiple times.
- bttger 4y ago
  This probably wasn't a bug. I experienced the same last week but it was because I ran the backup command from a different directory. I've created a note saying: `restic backup -v -r <location>/<repo_name> <source_dir>` first `cd` to the parent directory of the `source_dir` to avoid too many nested directories in the repository and conflicts in the mounting point path (so that the file change detection of restic works when mounting the drive to be backed up at another point)

shaicoleman 4y ago

restic has drastically improved peformance recently - increased pack size, new repository format, compression, configurable concurrency, etc. The performance issues you experienced may have been resolved.