3 ms·
I'm not sure I understand. Google, Circonus, etc. all store distributions as scalar values. A histogram is merely a collection of scalar values, where the colle
by otterley 6y ago
I'm not sure I understand. Google, Circonus, etc. all store distributions as scalar values. A histogram is merely a collection of scalar values, where the collection is a set of bounded buckets, where the boundaries are typically adjacent to one another (e.g., 0.1 <= x < 0.2, 0.2 <= x < 0.3, etc.). You can do this with any timeseries DB.
The aggregation functionality is best done in a separate layer so that you have the flexibility to change how you create your distributions without having to change your storage backend.
- kasey_junk 6y agoThey store scalar values in that they are binary encoded data structures. But the data structures they actually encode at the cell levels they histogram bins in a datastructure optimized for histogram calculations (in the Circonus case log linear histograms encoded as 3 values[0]). These histogram structures are optimized for fast re-aggregation and quantization across time/dimension. By optimizing for histograms first and plugging things like counts and gauges into them (rather than the reverse for something like Prometheus) they get around a lot of the bad statistics and expensive query calculations that other metrics systems have. This is different than say how Prometheus stores histograms where it treats the bucket boundaries as a dimension in a fixed bucket histogram and the cell as the count of items in that bucket. [0] https://www.circonus.com/2018/05/effective-management-of-high-volume-numeric-data-with-histograms/ https://www.circonus.com/2018/05/effective-management-of-hig...
- otterley 6y agoThanks for the link! This is some good reading.