5 ms·
Those are interesting results! I'm not surprised it works a lot better for metrics than logs and traces. Something I'd really love to have for logs/traces proce
by jcgrillo 2y ago
Those are interesting results! I'm not surprised it works a lot better for metrics than logs and traces. Something I'd really love to have for logs/traces processing is the ability to query clp[1][2] with a dataframe interface (e.g. datafusion [3]). While I'm on that subject, I'd also prefer that interface for metrics processing. I don't need real-time streaming metrics graphs, it's perfectly fine to compute one on-demand.
I suspect something like clp is the way to go for logs-like data, that is, low entropy text with a lot of numerical content.
[1] https://www.uber.com/blog/reducing-logging-cost-by-two-orders-of-magnitude-using-clp/ https://www.uber.com/blog/reducing-logging-cost-by-two-order...
[2] https://www.uber.com/blog/modernizing-logging-with-clp-ii/ https://www.uber.com/blog/modernizing-logging-with-clp-ii/
[3] https://github.com/apache/datafusion https://github.com/apache/datafusion