3 ms·

I have came to the same conclusion and was surprised to find that RAR is the only popular archive format that included parity. Personally I use 7zip for compre

I have came to the same conclusion and was surprised to find that RAR is the only popular archive format that included parity. Personally I use 7zip for compression and par2[0] for parity. [0] https://en.wikipedia.org/wiki/Parchive https://en.wikipedia.org/wiki/Parchive

dark-star 4y ago
I have yet to find a conclusive analysis on how well RAR with recovery works for different failure modes. I mean I can guess it works pretty well for single-bit flips, but how about burst errors, how long can those be? Usually you want to have protection from at least 1 or 2 filesystem blocks, which can be 4 or 8k or even more, depending on the file system. How about repeating error patterns, data deletions, etc.?
- metadat 4y ago
  PAR files (parity) are battle tested, if you are really concerned about recovering from corruption. https://en.m.wikipedia.org/wiki/Parchive https://en.m.wikipedia.org/wiki/Parchive
brnt 4y ago
It appears lzip is resistant to single bit flips, but can't be configured with more resistance.
gsich 4y ago
I like the inbuilt parity as you don't have to hold on to another set of files.
Koffiepoeder 4y ago
I'm still looking for a full-fledged backup and archiving solution that has the following characteristics: 0) error-correction (not just detection!) 1) end-to-end encryption 2) deduplication 3) compression 4) cross-platform implementations 5) (at least one) user interface 6) open source Both Borg [0] and Restic [1] have long standing open issues for error-correction, but seem to consider it off strategy. I find that decision kind of strange, since to me the whole purpose of a backup solution is to restore your system to a correct state after any kind of incident. My current solution is an assembly of shell scripts that combine borg with par2, but I'm rather unhappy with it. For one, I trust my home-brewn solution rather faintly (i.e. similar to `don't roll your own crypto` I think there should be an adagium `don't roll your own back-up solutions`). In addition I think an error-correcting mechanism should be available also for the less technology-savvy. [0]: https://github.com/borgbackup/borg/issues/225 https://github.com/borgbackup/borg/issues/225 [1]: https://github.com/restic/restic/issues/256 https://github.com/restic/restic/issues/256
- Koffiepoeder 4y ago
  Paper/master thesis/nerd snipe idea: does availability of Reed-Salomon information of unencrypted files weaken the encryption of their encrypted counterparts?

dark-star 4y ago

I have yet to find a conclusive analysis on how well RAR with recovery works for different failure modes. I mean I can guess it works pretty well for single-bit flips, but how about burst errors, how long can those be? Usually you want to have protection from at least 1 or 2 filesystem blocks, which can be 4 or 8k or even more, depending on the file system. How about repeating error patterns, data deletions, etc.?

metadat 4y ago
PAR files (parity) are battle tested, if you are really concerned about recovering from corruption. https://en.m.wikipedia.org/wiki/Parchive https://en.m.wikipedia.org/wiki/Parchive

brnt 4y ago

It appears lzip is resistant to single bit flips, but can't be configured with more resistance.

gsich 4y ago

I like the inbuilt parity as you don't have to hold on to another set of files.

Koffiepoeder 4y ago

I'm still looking for a full-fledged backup and archiving solution that has the following characteristics: 0) error-correction (not just detection!) 1) end-to-end encryption 2) deduplication 3) compression 4) cross-platform implementations 5) (at least one) user interface 6) open source Both Borg [0] and Restic [1] have long standing open issues for error-correction, but seem to consider it off strategy. I find that decision kind of strange, since to me the whole purpose of a backup solution is to restore your system to a correct state after any kind of incident. My current solution is an assembly of shell scripts that combine borg with par2, but I'm rather unhappy with it. For one, I trust my home-brewn solution rather faintly (i.e. similar to `don't roll your own crypto` I think there should be an adagium `don't roll your own back-up solutions`). In addition I think an error-correcting mechanism should be available also for the less technology-savvy. [0]: https://github.com/borgbackup/borg/issues/225 https://github.com/borgbackup/borg/issues/225 [1]: https://github.com/restic/restic/issues/256 https://github.com/restic/restic/issues/256

Koffiepoeder 4y ago
Paper/master thesis/nerd snipe idea: does availability of Reed-Salomon information of unencrypted files weaken the encryption of their encrypted counterparts?