3 ms·
I counter your argument: I have constructed systems that did this. Disk read latency doesn't matter for streaming reads. The actual math for this for a produc
by dekhn 9y ago
I counter your argument: I have constructed systems that did this. Disk read latency doesn't matter for streaming reads.
The actual math for this for a product at scale is interesting; I built such a product and did the math, and we found that moderate compression gave higher rates. However, this is based on a production-class infrastructure.
- dekhn 9y agoI should also mention: using more CPU to decompress highly compressed data means you have less CPU available (on a multithreaded app in a resource-managed environment) for other work. Since after decompressing you're going to be doing a bunch of other work (and a lot of that work will be going on simultaneousy in other threads), using less CPU to decompress can give higher throughput. But, again, this is a very complex problem.