3 ms·
How is compression not a real workload? Zstd is an excellent codec that's rapidly proliferating - for example Chrome is integrating it at the moment. https://b
by nvm0n2 3y ago
How is compression not a real workload? Zstd is an excellent codec that's rapidly proliferating - for example Chrome is integrating it at the moment.
https://bugs.chromium.org/p/chromium/issues/detail?id=1246971 https://bugs.chromium.org/p/chromium/issues/detail?id=124697...
Relatively few servers run databases. Many more are running web servers and given the huge efforts web devs go to in order to optimize response sizes, much faster, lower power and lower latency compression seems like an instant win. All that's required is for web servers to integrate the zstd library and this QAT thing, and that can be enough to tip the balance especially for IO bound servers that are mostly just doing string interpolation and waiting for backends.
And you mention a DB with compressed disk storage. How is better compression not a huge win for that use case? Databases are usually disk IOP, latency and CPU power constrained, and this is a win for all of those cases.
Finally, clearly this tech can be applied to other algorithms not just zstd. Presumably they highlight zstd because the library is actively maintained and was willing to add this (probably single use) "plugin" API. Really they should have just integrated it directly instead of complicating things for every zstd user, but I guess it adds extra dependencies or increases code size or something. The wins are big enough that other codec libraries will probably use the same approach and eventually it'll be abstracted by APIs that are less code size sensitive.
Yes, AMD is doing great right now, but Intel have had an edge when it comes to specialized CPU features for a long time. For example AMD are only just now catching up to where Intel were with SGX 8 years ago.
- saagarjha 3y ago> For example AMD are only just now catching up to where Intel were with SGX 8 years ago. Ah yes, being hopelessly broken?
- nvm0n2 3y agoOther way around. AMD's early efforts were hard broken and needed a new CPU, SGX has never been as far as I know. Even when not technically "broken" AMD SEV was broken by design because it didn't protect all the necessary state. SGX was correctly designed from day one and was always patchable via their TCB recovery process = software upgrade, sufficiently well that it could be patched to recover from Spectre attacks despite nobody knowing about them at the time SGX was designed. But that was just an example. Intel have introduced a quite a few interesting CPU extensions and special bits of hardware that AMD had no answer for over the years.
- ksec 3y ago>How is compression not a real workload? Not parent, but I guess what he / she meant was total workload. E.g, If 100% of your workload, only 5% of that is compression, speeding it up by 5x will only give you a 4% reduction in total system workload. Which means other factor like clock speed, more core, cache etc in a different system could still win despite it lost the compression benchmarks.