3 ms·

In this day and age, storage is not as important as IO, so I hate it when I see benchmarks with compression ratios + CPU times. That's not helpful. What I want

In this day and age, storage is not as important as IO, so I hate it when I see benchmarks with compression ratios + CPU times. That's not helpful. What I want to see is the total IO + CPU time across libraries for my specific IO and CPU constraints. Sure, it makes benchmarks more involved to display, as scalars are not enough anymore, you need multiple curves, but that's meaningful at least. To illustrate, if I have very fast IO, then I probably don't care of the compression ratio, it will be faster to download the raw payload and have 0 decompression cost. On the other end of the spectrum, if I have very slow IO, I would gladly have a much slower decompression algorithm but higher compression ratio for a faster overall timing. This is especially important because cloud storage for instance are rather cheap, but slow. Caches/CDNs are very fast, but storage is expensive. Etc

vlovich123 1y ago
In the specific examples mentioned, it almost doesn’t matter. It achieves nearly identical sizes (within 10-20%) that the transfer size of the image given typical sizes is nearly irrelevant unless you’re downloading on dial-up.
lifthrasiir 1y ago
> What I want to see is the total IO + CPU time across libraries for my specific IO and CPU constraints. This knob is typically called as a (compression) level, even though it isn't advertised as such because it is hard to translate level into your metric. Some libraries like zstd support IO-adaptive operations as a simpler alternative too.
hedora 1y ago
Read and write time are cpu_time/your_cpu_performance + data_size/your_bandwidth. The benchmark doesn't know what your CPU performance and bandwidth are (or will be on some future device). You should be able to multiply these costs out yourself.
philistine 1y ago
You're assuming that IO and CPU are equal. CPU speed is getting faster at a much higher rate than IO. It makes sense, one is infrastructure, the other is the silicon in your pocket. That's why you're getting downvoted, and why algos always push more and more to the CPU. Statistically, the correct posture is to expect the CPU to consistently get faster. I mean, if you make an image format now, you want it to work with CPUs in 30 years. Will IO speed gains overtake CPU improvements in 30 years? Most certainly not.

vlovich123 1y ago

In the specific examples mentioned, it almost doesn’t matter. It achieves nearly identical sizes (within 10-20%) that the transfer size of the image given typical sizes is nearly irrelevant unless you’re downloading on dial-up.

lifthrasiir 1y ago

> What I want to see is the total IO + CPU time across libraries for my specific IO and CPU constraints. This knob is typically called as a (compression) level, even though it isn't advertised as such because it is hard to translate level into your metric. Some libraries like zstd support IO-adaptive operations as a simpler alternative too.

hedora 1y ago

Read and write time are cpu_time/your_cpu_performance + data_size/your_bandwidth. The benchmark doesn't know what your CPU performance and bandwidth are (or will be on some future device). You should be able to multiply these costs out yourself.

philistine 1y ago

You're assuming that IO and CPU are equal. CPU speed is getting faster at a much higher rate than IO. It makes sense, one is infrastructure, the other is the silicon in your pocket. That's why you're getting downvoted, and why algos always push more and more to the CPU. Statistically, the correct posture is to expect the CPU to consistently get faster. I mean, if you make an image format now, you want it to work with CPUs in 30 years. Will IO speed gains overtake CPU improvements in 30 years? Most certainly not.