2 ms·

This is great news! We at VictoriaMetrics [1] are happy that TimescaleDB eventually addressed high disk usage issue and implemented type-aware column-based comp

This is great news! We at VictoriaMetrics [1] are happy that TimescaleDB eventually addressed high disk usage issue and implemented type-aware column-based compression. This should improve its' position and performance in TSBS benchmark [2]. One question: why do you use Gorilla compression for floating-point values? It works well for integer values, but is pretty useless for floating-point values [3]. [1] https://github.com/VictoriaMetrics/VictoriaMetrics https://github.com/VictoriaMetrics/VictoriaMetrics [2] https://medium.com/@valyala/measuring-vertical-scalability-for-time-series-databases-in-google-cloud-92550d78d8ae https://medium.com/@valyala/measuring-vertical-scalability-f... [3] https://medium.com/faun/victoriametrics-achieving-better-compression-for-time-series-data-than-gorilla-317bc1f95932 https://medium.com/faun/victoriametrics-achieving-better-com...

mfreed 7y ago
Gorilla is one of the few simple, efficent, _lossless_ algorithms that compress binary data. Tricks such as renormalizing floats in base-10 (which I believe VictoriaMetrics implements) while great when they work, potentially truncates lower-order digits. Lossiness is not a decision we're willing to make for our users unilaterally. And, as Gorilla is one the default compression algorithms we employ, it needs to be one that is correct in all cases. Incidentally, we experimented with the binary equivalent of the renormalization algorithm, and it performed on-par with Gorilla in our tests.