4 ms·
Density – Fast compression library
- dvh 12y agoWhy no comparsion with common contestants: gzip, bzip, dd?
- pykello 12y agoMaybe because they're known to be slow, specially bzip. http://catchchallenger.first-world.info//wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO http://catchchallenger.first-world.info//wiki/Quick_Benchmar...
- curun1r 12y agoIf that's the case, it would've been nice to see snappy included in the comparisons, since it seem appears targeted at the same use cases.
- biokoda 12y agoThose compression algorithms are not really designed for the type of work lz4/density is. It's basically shooting fish in a barrel. You can find plenty of lz4 benchmarks comparing it to other compression algorithms.
- Yver 12y agoThe README mentions Squash, where you can see the results of many more compressors. https://quixdb.github.io/squash-benchmark/ https://quixdb.github.io/squash-benchmark/
- corysama 12y agoThe graphs near the bottom of that page look very useful for understanding how density and other compressors compare. But, they also look dramatically different each time I refresh the page... Reporting from iOS8 Safari.
- analogj 12y agoThats because the testfile and testbed machine are chosen randomly each time the page reloads. That's mentioned in the blue note that at the top of each section."The default dataset is selected randomly."
- thechao 12y agoThis popped up on reddit/r/programming yesterday, so I'll just copy my comment from over there: How well does density [perform] when there's competition for the I$ and L$? LZ4 in theory is slower than a number of competitors (like snappy). However, LZ4 actually handily outperforms them when put into a "live" situations where it's running as a service alongside the 'real' program. The size of the in-memory data-structures is the real Achille's heal of most of the contenders to LZ4 I've seen.
- azinman2 12y agoInteresting. I've just been a casual observer to these types of things and haven't really considered total memory used but that's a big issue on devices like mobile, or even server environments where there many be many jobs concurrently. It's too bad that the Squash benchmark, linked in the github repo, doesn't have total peak memory usage. Do you know what the best contenders are as far as that's concerned?
- thechao 12y agoThe last time I researched this, to sufficient depth, was about 3 years ago. LZ4 was, hands down, the winner---nothing was even close.
- alexandre_m 12y agoBut, what is the Weissman Score??
- billyhoffman 12y agoMiddle Out.
- corysama 12y agoPlease note that the HN community takes a rather strict approach when moderating comments that contribute noise to the conversation. "Nice article!" comments are routinely downvoted. As is sarcasm, witticisms, memes, references and other styles of comments that occur frequently but do not contribute to the discussion. It's a knowingly doomed attempt to hold back the flood of noise that covers Reddit.
- willvarfar 12y agoExcelent commentary and a faster implementation by the legendary Charles Bloom: http://cbloomrants.blogspot.com/2015/03/03-25-15-density-chameleon.html http://cbloomrants.blogspot.com/2015/03/03-25-15-density-cha...
- jeffreyrogers 12y agoIt's funny, over the last week I've repeatedly come across things from people working at RAD Game Tools. It looks like they've got quite a good group of people working there.
- legulere 12y agoThey seem to be symmetric in compression and decompression. In most workloads you actually need to decompress way more often than compress.
- pipeep 12y agoI think this, or something like it, would be really useful with zram (Linux RAM compression), which currently uses LZO.
- jonbaer 12y agoPied Piper is real?
- DiThi 12y agoHow does it compare with Zstandard? http://fastcompression.blogspot.com.es/2015/01/zstd-stronger-compression-algorithm.html http://fastcompression.blogspot.com.es/2015/01/zstd-stronger...