4 ms·

I wonder if some developer of an app working with ZIP files will have noticed this bug and had to update their app. ZIP is an unusually terrible file format bec

I wonder if some developer of an app working with ZIP files will have noticed this bug and had to update their app. ZIP is an unusually terrible file format because it has a footer rather than a header, so the truncation bug would lead to a ZIP file with two headers!

canucker2016 4y ago
Some ZIP-file handling apps go straight to the end of the ZIP file and work backwards looking for the central directory. I believe others will start from the beginning of the file and parse each local directory header and 1) build up their own central directory truth, or 2) stop once they've reached the start of the central directory. I recall that the footer-only format was to enable streaming out a ZIP file esp. when rewinding to the front of the file/stream was not possible, like sending over a network - at least that's one of the use cases I recall.
- ladberg 4y ago
  I believe the footer-only format was more about being able to make modifications by only appending to the zip file and not having to rewrite/shift any data.
  - canucker2016 4y ago
    ...the central directory needs to be shifted by the amount of the new compressed data + local directory header. But that's a lot less than if the central directory was at the beginning and the code had to grow the central directory and shift ALL the compressed data + local directory headers down.
    - ladberg 4y ago
      Yeah that might be the ideal way to do it, but technically you could just append the new entries and new central directory without even removing the old one. I'm not entirely sure what storage system that would make sense on, but presumably there was one back in the day.
      - canucker2016 4y ago
        3.5" floppies - 800K-1.44MB, small and slow. Better than cassette tapes though...
        cesarb 4y ago
        > 3.5" floppies - 800K-1.44MB, small and slow. I feel like a lot of people in this thread are missing a crucial part of the historical context: ZIP files can be split. That is, a single ZIP file can span more than one floppy disk. The entry for each compressed file in the central directory, which is always on the last floppy disk, has a field which says in which disk the compressed file can be found. If you had a multi-disk ZIP file, and wanted to extract a single file from it, you inserted the last disk, pointed the unpacking software at it, and it would tell you which disk(s) to insert (in sequence) to extract that file. A lot of the strange decisions of the ZIP file format make sense if you think about that use case: you have a directory full of files, which you want to compress into a set of floppy disks; and once you have that set of floppy disks, you want to extract a subset of the compressed files. You also want to be able to extract the undamaged files even when the last floppy disk (with the central directory) is damaged.
layer8 4y ago
Same for PDF. But at least that’s much less likely to create the issue of invisible non-deleted data as in the PNG case.

canucker2016 4y ago

Some ZIP-file handling apps go straight to the end of the ZIP file and work backwards looking for the central directory. I believe others will start from the beginning of the file and parse each local directory header and 1) build up their own central directory truth, or 2) stop once they've reached the start of the central directory. I recall that the footer-only format was to enable streaming out a ZIP file esp. when rewinding to the front of the file/stream was not possible, like sending over a network - at least that's one of the use cases I recall.

ladberg 4y ago
I believe the footer-only format was more about being able to make modifications by only appending to the zip file and not having to rewrite/shift any data.
- canucker2016 4y ago
  ...the central directory needs to be shifted by the amount of the new compressed data + local directory header. But that's a lot less than if the central directory was at the beginning and the code had to grow the central directory and shift ALL the compressed data + local directory headers down.
  - ladberg 4y ago
    Yeah that might be the ideal way to do it, but technically you could just append the new entries and new central directory without even removing the old one. I'm not entirely sure what storage system that would make sense on, but presumably there was one back in the day.
    - canucker2016 4y ago
      3.5" floppies - 800K-1.44MB, small and slow. Better than cassette tapes though...
      - cesarb 4y ago
        > 3.5" floppies - 800K-1.44MB, small and slow. I feel like a lot of people in this thread are missing a crucial part of the historical context: ZIP files can be split. That is, a single ZIP file can span more than one floppy disk. The entry for each compressed file in the central directory, which is always on the last floppy disk, has a field which says in which disk the compressed file can be found. If you had a multi-disk ZIP file, and wanted to extract a single file from it, you inserted the last disk, pointed the unpacking software at it, and it would tell you which disk(s) to insert (in sequence) to extract that file. A lot of the strange decisions of the ZIP file format make sense if you think about that use case: you have a directory full of files, which you want to compress into a set of floppy disks; and once you have that set of floppy disks, you want to extract a subset of the compressed files. You also want to be able to extract the undamaged files even when the last floppy disk (with the central directory) is damaged.

layer8 4y ago

Same for PDF. But at least that’s much less likely to create the issue of invisible non-deleted data as in the PNG case.