3 ms·
The paper on the algorithm is here: http://www.vldb.org/pvldb/vol8/p1816-teller.pdf http://www.vldb.org/pvldb/vol8/p1816-teller.pdf Somebody implemented the al
by storyinmemo 10y ago
The paper on the algorithm is here: http://www.vldb.org/pvldb/vol8/p1816-teller.pdf http://www.vldb.org/pvldb/vol8/p1816-teller.pdf
Somebody implemented the algorithm in go based on the paper here: https://github.com/dgryski/go-tsz https://github.com/dgryski/go-tsz
A short answer that may work for your question: the bits that are set in RAM are xor values relative to previous values. To provide an answer as to what the value is, a series of read|xor operations are performed.
- jdonaldson 10y agoThey refer to it as "delta of delta", which implies the encoding handles things like acceleration and momentum. I'm guessing the healthcare data has much more of this sort of behavior than your typical server event time series.
- Scaevolus 10y agoNot quite. Delta of deltas means that periodic timestamps turn into a long run of zeros (since the difference between each point is constant), which is very easy to compress. In the Gorilla paper, 96% of timestamps compress down to zero, which is encoded by a single zero bit. For series without periodic sampling, delta-of-deltas performs similarly to normal deltas.
- rakoo 10y ago"delta of delta" is only used for timestamps, not for values. In any standard monitoring setup, the acceleration of timestamps will be 0 for a big portion of measurements, and very small for a big portion of the rest, so it makes compression extremely easy and efficient.
- throwaway2016a 10y ago> I'm guessing the healthcare data has much more of this sort of behavior than your typical server event time series. I thought they were talking about healthcare data too when I first read it but seeing as this is Facebook I realized they are talking about server health not human health.
- machineman44 10y agoHaha, I thought jdonaldson was making a joke :P