3 ms·
As mentioned it's trivial across the spread of compression algorithms supporting this type of behaviour (`gzip`, `zstandard`, `zip`, ...), the header in `zip` m
by Smerity 3y ago
As mentioned it's trivial across the spread of compression algorithms supporting this type of behaviour (`gzip`, `zstandard`, `zip`, ...), the header in `zip` making it even more convenient as you note!
WARC as a format essentially states that unless you have good reason "record at a time" compression is the preferred[1].
The mixture of "technically possible" and "part of spec" is what makes it so useful - any generic WARC tool can support random access, there are explicit fields to index over (URL), and even non-conforming WARC files can be easily rewritten to add such a capability.
[1]: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.0/#record-at-time-compression https://iipc.github.io/warc-specifications/specifications/wa...