2 ms·
Parsing but also casting to string efficiently is still an open problem with regular breakthrough. Ryu or Dragonbox for float to string come to mind. Parsing w
by di4na 3y ago
Parsing but also casting to string efficiently is still an open problem with regular breakthrough. Ryu or Dragonbox for float to string come to mind.
Parsing with errors that are useful for the human is definitely an open research topic.
Capabilities have seen interesting work, in particular in relationships to effect handlers.
Hyperloglog and other datasketch keep seeing incremental progress. I have not had time to read it all but Omar Ertl has a massive reduction in memory use in UltraLogLog that i expect a paper on soon. The code is already up.
- hinkley 3y agoI've been poking at prometheus internals lately and I am starting to wonder if histograms need some sort of hyperloglog treatment to improve on buckets. They don't seem like a very accurate solution.
- di4na 3y agoYou may appreciate ddsketch https://www.datadoghq.com/blog/engineering/computing-accurate-percentiles-with-ddsketch/ https://www.datadoghq.com/blog/engineering/computing-accurat... Or tdigest. For the prom use case, i usually prefer ddsketch. Easier to integrate. Tdigest is really complicated. That said there are definitely interesting work that need to happen on compressing time series histograms for heatmaps. Iirc influx had some interesting work there.
- drewbug01 3y agoHave you seen the Prometheus “native”/“sparse” histograms that are being worked on: https://promcon.io/2022-munich/talks/native-histograms-in-prometheus/ https://promcon.io/2022-munich/talks/native-histograms-in-pr... They’ve released it as experimental recently, and we’ve found it to be a massive improvement over the original histograms. It’s still bucket-based at the end of the day, but the exponential bucket layout and sparse representation seems to work really well in practice.