4 ms·

One important difference in practice is that zip files needs to be saved to disk to be extracted. gzip files on the other hand can be stream unzipped i.e curl h

One important difference in practice is that zip files needs to be saved to disk to be extracted. gzip files on the other hand can be stream unzipped i.e curl http://example.com/foo.tar.gz http://example.com/foo.tar.gz | tar zxvf - is possible but not with zip files. I am not sure if this is a limitation of the unzip tool. I would love to know if there is a work around to this.

tobias3 10y ago
Zip files can be streamed. See for example https://developer.android.com/reference/java/util/zip/ZipInputStream.html https://developer.android.com/reference/java/util/zip/ZipInp... It even allows you to stream the zip while creating it, i.e. you do not need to know the compressed size of the files before starting to write the compressed file content.
- deleted 10y ago
  [deleted]
- pierrec 10y ago
  I wonder how their API degrades when the stream doesn't support seeking (as is often the case over HTTP, or like in gp's example of using pipes). The whole doc you linked to, and the feature of on-the-fly decompression, seems to assume that the stream is seekable.
  - tobias3 10y ago
    InputStreams are not seekable (see https://developer.android.com/reference/java/io/InputStream.html https://developer.android.com/reference/java/io/InputStream.... ). You can also have a look at a zip file spec to see that they can be streamed: https://users.cs.jmu.edu/buchhofp/forensics/formats/pkzip.html https://users.cs.jmu.edu/buchhofp/forensics/formats/pkzip.ht...
    - pierrec 10y ago
      OK, makes sense. The headers at the end of the file are only required if you want to extract specific files out of order. Love the clarity of that spec, btw.
bluedino 10y ago
You can't do that with a .zip file because the file header information is actually at the end of the file. You could work around that by sending the header first, though.
- the_common_man 10y ago
  How do you send the header first? Is that a feature of the http framework?
  - bluedino 10y ago
    It'd be something you'd have to write yourself, of course. On the server side, read it and send it before the file. Maybe go through all your zips and copy the needed metadata to another location.
- greggman 10y ago
  You can't put the header first. It is an intentional feature of .zip that the header is at the end so you can update a large zip file by just appending a new header to the end. That way the entire file does not have to be rewritten. Just read the old header, append new files, append new header. This was important back in floppy disk days
  - bluedino 10y ago
    >> Just read the old header, append new files, append new header. This was important back in floppy disk days Don't forget 'overwrite old header' Important for floppy disks in two ways, one because of space constraints and two, because of how slow floppies were.
    - joosters 10y ago
      Don't forget 'overwrite old header' Why? The new one will become the 'real' one since it will be at the end of the modified file. So it doesn't really matter if you delete the old one. If you're really trying to squeeze file sizes down, you could reference the old 'header' from the new one, so that you do not have to list the entire archive's contents again.

tobias3 10y ago

Zip files can be streamed. See for example https://developer.android.com/reference/java/util/zip/ZipInputStream.html https://developer.android.com/reference/java/util/zip/ZipInp... It even allows you to stream the zip while creating it, i.e. you do not need to know the compressed size of the files before starting to write the compressed file content.

deleted 10y ago
[deleted]
pierrec 10y ago
I wonder how their API degrades when the stream doesn't support seeking (as is often the case over HTTP, or like in gp's example of using pipes). The whole doc you linked to, and the feature of on-the-fly decompression, seems to assume that the stream is seekable.
- tobias3 10y ago
  InputStreams are not seekable (see https://developer.android.com/reference/java/io/InputStream.html https://developer.android.com/reference/java/io/InputStream.... ). You can also have a look at a zip file spec to see that they can be streamed: https://users.cs.jmu.edu/buchhofp/forensics/formats/pkzip.html https://users.cs.jmu.edu/buchhofp/forensics/formats/pkzip.ht...
  - pierrec 10y ago
    OK, makes sense. The headers at the end of the file are only required if you want to extract specific files out of order. Love the clarity of that spec, btw.

bluedino 10y ago

You can't do that with a .zip file because the file header information is actually at the end of the file. You could work around that by sending the header first, though.

the_common_man 10y ago
How do you send the header first? Is that a feature of the http framework?
- bluedino 10y ago
  It'd be something you'd have to write yourself, of course. On the server side, read it and send it before the file. Maybe go through all your zips and copy the needed metadata to another location.
greggman 10y ago
You can't put the header first. It is an intentional feature of .zip that the header is at the end so you can update a large zip file by just appending a new header to the end. That way the entire file does not have to be rewritten. Just read the old header, append new files, append new header. This was important back in floppy disk days
- bluedino 10y ago
  >> Just read the old header, append new files, append new header. This was important back in floppy disk days Don't forget 'overwrite old header' Important for floppy disks in two ways, one because of space constraints and two, because of how slow floppies were.
  - joosters 10y ago
    Don't forget 'overwrite old header' Why? The new one will become the 'real' one since it will be at the end of the modified file. So it doesn't really matter if you delete the old one. If you're really trying to squeeze file sizes down, you could reference the old 'header' from the new one, so that you do not have to list the entire archive's contents again.