3 ms·

I took a quick glance at the spec PDF. From what I can see it is a combined compressor that does: 1. Use pretty standard LZ77 sequence matching to previously

I took a quick glance at the spec PDF. From what I can see it is a combined compressor that does: 1. Use pretty standard LZ77 sequence matching to previously seen data. 2. A move to front encoding of the LZ77 references. 3. A canonical Huffman encoding of the MTFed references. So no new, revolutionary ideas. The MTF part is one of several ways of encoding the references. Given by the OP, the authors have spent time tweaking parameters (like the LZ77 window). But as far as I understand, the canonical Huffman encoder implies that the compressor can only operate on files, not streams.

BenoitP 8y ago
To me, the news is more that they are open-sourcing the hardware acceleration that goes with it. These sort of initiatives should be loudly celebrated; and I think they will be more common as the end of Moore's law progresses, and as hardware will become less and less generic. My bet: in less than 10 years we're going to see an open source kind of chips like the following: The JavaChip™, maintained by OpenJDK; available at your local cloud, provided by TSMC: * Zipline acceleration for your data needs * GC acceleration * Code and data caches modes specialized for small threads * UTF-8-based datagram dynamic parsing pattern * Explicit prefetcher control * The JVM's profiling data done in the hardware, more extensively * Collections.sort() as a hardware bitonic sort where appropriate * Facilities for predicate pushdown to the SSD / NIC * etc. Basically: all the specificities of your very specific programming model down in the hardware.