4 ms·
Actually zstd internally splits data into frames, and frames can indicate the decompressed data size. So if we control the compressor we can make it so that all
by kccqzy 1mo ago
Actually zstd internally splits data into frames, and frames can indicate the decompressed data size. So if we control the compressor we can make it so that all frames have the size information; it isn’t exactly seekable but at least it will not need to decompress the resource. https://python-zstandard.readthedocs.io/en/latest/concepts.html https://python-zstandard.readthedocs.io/en/latest/concepts.h...
Given how fast zstd can decompress, this may or may not actually be a win: the time spent waiting for I/O might be so large that the decompression can fit within the wait time.