3 ms·
This is an awesome blog post that is very well written, but the lack of incompressible performance analysis prevents It from providing a complete overview of zs
by ryao 10y ago
This is an awesome blog post that is very well written, but the lack of incompressible performance analysis prevents It from providing a complete overview of zstd.
Incompressible performance measurements are important for interactive/realtime workloads and the numbers are extremely interesting because they can differ dramatically from the average case measurements. LZ4 for instance has been measured at doing 10GB/sec on incompressible data on a single core of a modern Intel Xeon processor. At the other end of the spectrum is the worst case scenario for incompressible data where performance slows to a crawl. I do not recall any examples in this area, but the point is that it is possible for algorithms to have great average case performance and terrible worst case performance. Quick sort is probably the most famous example of that concept.
I have no reason to suspect that zstd has bad incompressible performance, but the omission of incompressible performance numbers is unfortunate.
- Cyan4973 10y agozstd goes at > 1 GB/s on uncompressible data. It has some fast heuristics for such cases too.