4 ms·

Please pick a topic and stick with it. You asked about corruption, specifically when "a small part of the whole archive file gets corrupted". I addressed that

by dalke 10y ago

Please pick a topic and stick with it. You asked about corruption, specifically when "a small part of the whole archive file gets corrupted". I addressed that point. I'm not going to cover all of the advantages and disadvantages between the two systems, because they are well-known and boring. You of course must choose the method that matches your expected corruption methods. A incomplete write is a different failure mode than you wanted me to address. Reed-Solomon isn't designed for that. Other methods are, and they are often used in databases. And you of course must choose a method which fits your access model. Just like you would of course choose tar over zip if you want to preserve all of the filesystem metadata on a unix-based machine. Zip doesn't handle hard links, for example. My point again is that data duplication causes well-known problems when there is a data mismatch, and while the duplication in zip as the result of its write-only/journalling format helps with some forms of error recovery, there are better methods for error recovery if that is your primary concern.

acqq 10y ago
What you call the "data duplication" is actually a "metadata redundancy" which results in the faster access to the list of all the files in the archive (a good example of the pre-computed pre-cached info). The format is good, if some library can't cope with the format, the problem is with the library making the corruption of the archive. And that archive is still, even after being corrupted by wrongly written tool, recoverable because of the existing redundancy. The concept of metadata redundancy isn't anything specific to the archives, the filesystems do it too, and it's always an engineering trade-off, and I consider the one in Zip format is a good one, based on the real-life examples I've given.
- dalke 10y ago
  I agree that it's a good format. It's an engineering compromise. The benefits you see also cause creshal's comment: > Beginning and the end, which continues to bite us in the ass until today, where we regularly stumble over bugs when one part of a toolchain uses one entry and the rest the other. Do you think creshal is mistaken? Of course they are due to bugs in the toolchain. That's a natural consequence of the file format. That's the whole point of creshal's comment. What do you think I'm trying to say, because I don't understand your responses based on what I thought I was trying to say.

by dalke 10y ago

acqq 10y ago

What you call the "data duplication" is actually a "metadata redundancy" which results in the faster access to the list of all the files in the archive (a good example of the pre-computed pre-cached info). The format is good, if some library can't cope with the format, the problem is with the library making the corruption of the archive. And that archive is still, even after being corrupted by wrongly written tool, recoverable because of the existing redundancy. The concept of metadata redundancy isn't anything specific to the archives, the filesystems do it too, and it's always an engineering trade-off, and I consider the one in Zip format is a good one, based on the real-life examples I've given.

dalke 10y ago
I agree that it's a good format. It's an engineering compromise. The benefits you see also cause creshal's comment: > Beginning and the end, which continues to bite us in the ass until today, where we regularly stumble over bugs when one part of a toolchain uses one entry and the rest the other. Do you think creshal is mistaken? Of course they are due to bugs in the toolchain. That's a natural consequence of the file format. That's the whole point of creshal's comment. What do you think I'm trying to say, because I don't understand your responses based on what I thought I was trying to say.