3 ms·
am interested in the `dummyBytes` generated by flog[2], in the linked benchmark result[1]. it's random words from `/usr/share/dict/words`, which may not be high
by thewisenerd 4y ago
am interested in the `dummyBytes` generated by flog[2], in the linked benchmark result[1]. it's random words from `/usr/share/dict/words`, which may not be highly compressible..
i.e., 500 GB -> 207 GB (with zstd data compression + indexes) seems like a worst case scenario. with "real" logs, I am expecting this to be much better (for logs at least)..
does anyone have a similar size comparison with real life examples? (similar data size, interested in compressed logs size and indexes size with clickhouse)
[1] https://signoz.io/blog/logs-performance-benchmark https://signoz.io/blog/logs-performance-benchmark
[2] https://github.com/signoz/flog https://github.com/signoz/flog
[3] https://github.com/tjarratt/babble/blob/cbca2a4833c1dd0e0287d50982e6dda554e95c9d/babble_unix.go#L11 https://github.com/tjarratt/babble/blob/cbca2a4833c1dd0e0287...
- pranay01 4y agoYeah, agreed. This is a worst case scenario. We also expect compression to be better in real life scenario data