4 ms·
ANS is super fast and trivially parallelizable, faster than Huffman or especially arithmetic encoding, and is superior to Huffman at least as far as compression
by jhj 4y ago
ANS is super fast and trivially parallelizable, faster than Huffman or especially arithmetic encoding, and is superior to Huffman at least as far as compression ratios are concerned (symbol probabilities are not restricted to powers of 2, but unlike arithmetic encoding one is usually limited to something close to multiples of 2^-9 to 2^-13 for symbol probabilities, the limitation being due to required table sizes for rANS or tANS in high speed memories or in L1 cache).
It is fast because it can be machine word oriented (you can read/write whole machine word sizes at a time, not arbitrary/variable bit length sequences), and as a result you can interleave any number of independent (parallel) encoders in the same stream with just a (parallel) prefix sum to figure out where to write state values (as whole machine words) when renormalization needs to occur in individual encoders. It is super SIMD friendly as a result (see https://arxiv.org/pdf/1402.3392.pdf https://arxiv.org/pdf/1402.3392.pdf).
I for one got up to 400 GB/s throughput on A100 GPUs in my implementation (https://github.com/facebookresearch/dietgpu https://github.com/facebookresearch/dietgpu), which to my knowledge is/was the fastest entropy coder out there on a commercially available hardware platform, so fast that we're using it to losslessly compress data before sending over PCIe/NVLink/Infiniband/etc during large scale neural network training while still being a win. Nvidia's nvCOMP library also recently added similar techniques.
ANS can also self-synchronize as well, but chunking data into fixed >=4 KiB segments has fairly minimal compression size overhead (you need an index to tell you where each variable sized compressed chunk begins) and is faster.