3 ms·
This mentions something I've been wondering about: >Depending on the data, this can reduce the size of the database by 80% while keeping performance mostly the
by logicallee 4y ago
This mentions something I've been wondering about:
>Depending on the data, this can reduce the size of the database by 80% while keeping performance mostly the same (or even improving it, since the data to be read from disk is smaller).
"Even improving it" is a bold claim!
Modern CPU's are often hungry for data, aren't they? While being incredibly fast and mostly idle?
Is reading a smaller compressed file and decompressing it in the CPU's hyperfast cache before use faster than reading it uncompressed?
Maybe you can speed up an entire server, even one with lots of free space and no need for compression, by adding full disk compression anyway!
Unless the CPU's are at 100% utilization this could always be used. Since the CPU will then use the data, it might be fast enough to uncompress it at home faster than the bus can keep dropping it off at file read speeds.
This chart I found that goes up to 2000 suggests cache access had an increasing performance gap with even RAM bandwidth, to say nothing of SSD's[1]
It is from here[2]
(However, compressing a full disk can only help if there are extra CPU cycles.
For activity spikes when the CPU doesn't have extra cycles or the cache for it, both compressed and uncompressed versions could be available, at most doubling disk usage, so that the uncompressed file could be read instead.
This can ensure the CPU isn't burdened with a decompression task, since this only helps if the fastest caches are used for it and they might have better things to do than uncompress files.
However, whenever the CPU has idle cores and lots of cache, which is most of the time, it could request the compressed version. If it is busy it could ask for the original version which saves the decompression work.)
Can this help solve the problem of waiting for the disk?
[1] http://www.extremetech.com/wp-content/uploads/2014/08/CPU-DRAM.png http://www.extremetech.com/wp-content/uploads/2014/08/CPU-DR...
[2] https://www.extremetech.com/extreme/188776-how-l1-and-l2-cpu-caches-work-and-why-theyre-an-essential-part-of-modern-chips https://www.extremetech.com/extreme/188776-how-l1-and-l2-cpu...
- outworlder 4y ago> Maybe you can speed up an entire server, even one with lots of free space and no need for compression, by adding full disk compression anyway! Yes. Although not all filesystems support compression. One of the filesystems that does support and this is regularly benchmarked, is ZFS. It is also interesting that while you are definitely using more CPU to read a compressed stream, if that causes you to finish your task earlier, then it can be a net gain even from a CPU usage standpoint. eg. https://www.servethehome.com/the-case-for-using-zfs-compression/ https://www.servethehome.com/the-case-for-using-zfs-compress...