3 ms·

> Uncompressed .tar is a reasonable choice for this application Yes, uncompressed tar (with transfer compression, which is offered in HTTP) is an option for so

> Uncompressed .tar is a reasonable choice for this application Yes, uncompressed tar (with transfer compression, which is offered in HTTP) is an option for some amount of data. Till the point where it isn't. zip has similar benefits as tar(+transfer compression) but a later point where it fails for such a scenario.

chungy 6mo ago
Zip allows you to set compression algorithm on a per-file basis, including no compression.
- QuantumNomad_ 6mo ago
  You can achieve the same with tar if you individually compress the files before adding them to the tar ball instead of compressing the tar ball itself. I don’t see how that plus a small index of offsets would be notably more or less work to do from using a zip file.
  - chungy 6mo ago
    Zip has a central directory you could just query, instead of having to construct one in-memory by scanning the entire archive. That's significantly less work.
    - QuantumNomad_ 6mo ago
      I mean if they include a pre-made index with it. For example an uncompressed index at byte offset 0 in the tar ball that lists what is inside and their offsets. It would still be comparable amount of work to create software to do that with tar as to use a zip file, if fine grained compression levels etc is being used.
      - johannes1234321 6mo ago
        But then you are not using tar, you are doing your own file format atop of tar.
        QuantumNomad_ 6mo ago
        I suppose you are right about that. But it would still be a valid tar file that can be viewed and extracted with normal tools. Kind of similar to how a .docx file can be extracted as zip but still has additional structure to its contents.
        chungy 6mo ago
        What are you really proposing? That a first ".INDEX" entry be made that contains the offsets of all the other members? That could work in a backwards-compatible way (as long as no standard tar utility makes modifications to the archive...), but it's hamfisted. Just use Zip. It's already a well-known format with numerous implementations and already does the job that you want to do.

chungy 6mo ago

Zip allows you to set compression algorithm on a per-file basis, including no compression.

QuantumNomad_ 6mo ago
You can achieve the same with tar if you individually compress the files before adding them to the tar ball instead of compressing the tar ball itself. I don’t see how that plus a small index of offsets would be notably more or less work to do from using a zip file.
- chungy 6mo ago
  Zip has a central directory you could just query, instead of having to construct one in-memory by scanning the entire archive. That's significantly less work.
  - QuantumNomad_ 6mo ago
    I mean if they include a pre-made index with it. For example an uncompressed index at byte offset 0 in the tar ball that lists what is inside and their offsets. It would still be comparable amount of work to create software to do that with tar as to use a zip file, if fine grained compression levels etc is being used.
    - johannes1234321 6mo ago
      But then you are not using tar, you are doing your own file format atop of tar.
      - QuantumNomad_ 6mo ago
        I suppose you are right about that. But it would still be a valid tar file that can be viewed and extracted with normal tools. Kind of similar to how a .docx file can be extracted as zip but still has additional structure to its contents.
        chungy 6mo ago
        What are you really proposing? That a first ".INDEX" entry be made that contains the offsets of all the other members? That could work in a backwards-compatible way (as long as no standard tar utility makes modifications to the archive...), but it's hamfisted. Just use Zip. It's already a well-known format with numerous implementations and already does the job that you want to do.