6 ms·
The articles main point seems to be complaining about ambiguity. > Scan from the back, find the end-of-central-directory-record and then use it to read through
by opless 3y ago
The articles main point seems to be complaining about ambiguity.
> Scan from the back, find the end-of-central-directory-record and then use it to read through the central directory, only looking at things the central directory references.
It was pretty obvious 30 years ago that this was the correct way to unzip. Streaming content wasn't even a thing back then (dialup being a luxury). It was the only way to ensure that a self extractor could work reliably.
- iforgotpassword 3y agoExactly, some of the quirks are just a result of designing this for these old systems. ZIP needed to work acceptably on machines with less than a megabyte of RAM. The file format contains ambiguity and redundancy yes, but given well behaving software it's the "good enough" of archive formats. Even in the *nix world we didn't get anything better, tar isn't even seekable and has its own idiosyncrasies, especially if you compress it because the compression happens afterwards and is applied to the whole archive.
- opless 3y agoTar IS seekable, as it was designed especially for tapes. The modern usage of it adds compression, which you have to undo first before doing your archive operations ;) Zip was > tar (and cpio) in that regard because you didn't have that two step process and only had to have enough spare disk space for the extracted file, rather than extracted file + decompressed tar
- iforgotpassword 3y agoTar is seekable in the literal sense, you have to seek around until you find the file you want. You have to look at each header, if it's not the file you want you skip over that one file and read the next header. Because you only know how long that one file is. So you cannot even know where the third file is without having processed the headers of the previous two. It's "read, skip, read, skip, read skip" until you maybe find the file you wanted. ZIP otoh has the central directory index where you can look things up much faster.
- masklinn 3y agoZip still > tar in that regard, since you can list archive contents (and extract individual files) without having to go through the entire thing every time, hence zip being suitable as a local file system (à la opendocument or ooxml).
- xxs 3y agoZip is terrible as a local files system, unless you consider read only (randomly, and not many files). Writing/append to a file requires some form of garbage organization/compaction - which requires rewrite of the entire CEN. Unlike the regular files/entries, CEN doesn't have a checksum, so corruption there is pretty bad. Each file being compressed individually means low compression rates, the deflate overhead might be acceptable for slow mediums (although in that case the memory overhead would be non-trivial).
- xxs 3y agoThe massive downsize of zip is that each file is compressed individually, which means very poor compression. Overall the compression rate is significantly lower, same with an attempt to decompress it all. It's only useful for random reads of few select files. Tar -> read, skip, read, till you find what you need, compression doesn't change much, as most stream compression algorithms have a flush method.
- tecleandor 3y agoWell, depending on the definition you could fit ZMODEM as an streaming + compression (RLE) protocol. I used it a lot almost 30 years ago!
- opless 3y agoAs I mentioned, dialup for many was a luxury.
- dale_glass 3y agoStreaming was a thing if you include tape
- opless 3y agoCassettes were mostly used on 8 bit machines ;) Tape drives were relatively expensive to floppy disks.
- Karellen 3y agoTape drives were used on a lot of early computers, including the hardware that ran early Unices (e.g. 16-bit PDP-11) and which tar/gzip were written for https://en.wikipedia.org/wiki/IBM_7-track https://en.wikipedia.org/wiki/IBM_7-track https://en.wikipedia.org/wiki/9-track_tape https://en.wikipedia.org/wiki/9-track_tape https://en.wikipedia.org/wiki/DECtape https://en.wikipedia.org/wiki/DECtape "tape" includes reel-to-reel systems as well as cassette.
- opless 3y agoAnd exactly none of them were used on contemporary machines that PKZIP ran on.
- Karellen 3y agoHuh. I wasn't aware that cassette tapes were used on the systems that PKZip ran on either. Pretty sure DOS didn't have tape drive support ordinarily, and I wasn't aware that PKZip was widely ported to anything that wasn't DOS or a clone/derivative?
- opless 3y agohttps://en.wikipedia.org/wiki/IBM_cassette_tape https://en.wikipedia.org/wiki/IBM_cassette_tape Sorry, but I guess you had to be there to have that knowledge?
- lifthrasiir 3y agoThe problem is not that the reading method should be obvious once you consider that era, but that the reading or writing method is not exactly specified and there can be a considerable variety in implementations. For example the OP complains about a passing mention (4.3.1): > Files MAY be added or replaced within a ZIP file, or deleted. While the author considers this to imply (but not acknowledge) that local file headers and central directory headers may disagree to each other, I'm not even sure because the specification never clearly confirms whether a random data can appear before any local file header. The only thing that has been acknowledged is SFX (4.1.9), and it is not even specified where the "extraction code" can be embedded. Can it be placed between two files? If such thing is indeed possible, shouldn't there be a provision to avoid ambiguity due to the stray data? Say, it could have said that deleting a file in place should mask all occurrences of local file header signature 0x04034b50 so that the vacant space can no longer interpreted as a file at all.
- amelius 3y agoSelf-extractors are too dangerous to run, except perhaps in a sandbox.
- tinus_hn 3y agoAlso the only way such a file can be editable which was a major design goal of the zip format and which is not really possible with Tar files.
- efreak 3y agoNot sure what you're saying here. Tar supports appending new files and new versions of files to an existing archive without rewriting the entire archive. I don't believe it supports marking files as deleted (not sure why, tbh) but you can add an empty file with the same name for non-tape storage to prevent extracting the original version. It does this by simply appending the new version of the file to the end of the archive, which is understandable as unlike zip files tar (tape archive) was designed explicitly for tape, where there is no random access. Unless you're referring to editing files in-place inside a tar file, which should actually be easier than doing so with a zip file (as long as the new version isn't longer than the old version, and you're not actually storing it on tape).
- tinus_hn 3y agoSo there’s three options: index at the beginning, index at the end and no index. Index at the beginning means it’s easy to look up, but impossible to add anything. No index, which is tar, means it’s impossible to tell what’s in it without reading the entire file. Index at the end, which is zip, means you have to do slightly more work to find the index, but then you can read it without reading the rest of the file, and move it further along the file if you want to extend the file and you can edit it or add to it. And if you want compression tar is just a compressed blob and the only thing you can do is read it from beginning to end. That’s just not very helpful unless the only things you want to do are packages a bunch of things and unpack all of them. A major use case but one met perfectly fine by hundreds of formats.