4 ms·
That's reporting better figures with general compression than I'd expect from experience. Anyway, I'd expect to reach for zfp.io for arrays of floating point d
by gnufx 4y ago
That's reporting better figures with general compression than I'd expect from experience. Anyway, I'd expect to reach for zfp.io for arrays of floating point data.
It's certainly an old observation that the compression method you want to use is data-dependent, even for similar kinds of data. Long ago it was found that there was a dramatic difference in x-ray crystallography image data depending on whether they were taken with a lab source (low resolution) or at a synchrotron (high resolution -- sharper peaks).
- aras_p 4y agoI have a new post (https://aras-p.info/blog/2023/02/03/Float-Compression-5-Science/ https://aras-p.info/blog/2023/02/03/Float-Compression-5-Scie...) comparing zfp, fpzip, spdp, ndzip, streamvbyte libraries for compressing this same data set. All in lossless, single threaded mode so far (just like all the other experiments before). zfp performance is... "underwhelming" to say the least :( (fpzip is pretty good though, if only a bit slow at decompression). Maybe zfp is much better on 3D or 4D data, and/or in lossy mode. Which might be a topic for a future post.