3 ms·
- Timestamps are very easy to compress down to 0.01% - Floats are hard to compress without quantization or lossy compression. The Gorilla algorithm is not help
by powturbo 7y ago
- Timestamps are very easy to compress down to 0.01%
- Floats are hard to compress without quantization or lossy compression. The Gorilla algorithm is not helping so much.
You can do your own benchmarks with your data with TurboPFor : https://github.com/powturbo/TurboPFor https://github.com/powturbo/TurboPFor
or download icapp including more powerfull algorithms.
- m0zg 7y agoNonrandom time series floats are often significantly helped by delta-compression before applying the actual block compression algorithm. I used to do this quite successfully in late 90s when sending FOREX quotes to clients over analog modem connections.
- powturbo 7y agoYeah, for benchmarking it is important to use real data, because the compression ratio can significantly differ. You'll have better compression with lossy compression and better precision with user controlled and bounded relative error. see for ex. this benchmark for scientific data: https://github.com/powturbo/TurboTranspose https://github.com/powturbo/TurboTranspose