4 ms·
Bolt author here. I don't think I'd agree that Bolt isn't good for time series. In fact, it has fast appends and fast sequential reads so it works really well f
by benbjohnson 12y ago
Bolt author here. I don't think I'd agree that Bolt isn't good for time series. In fact, it has fast appends and fast sequential reads so it works really well for time sorted data. Having a sequential page ordering (like Catena) will have a huge boost when running on spinning disk but it's much less of an issue on SSDs.
All that being said, I do like the approach of Catena. I think it works well for best case scenarios and you can get some large perf gains from tailoring the format to a specific type of data. However, some of the trade offs such as dropping writes after the in-memory window are not going to be acceptable to many people.
As far as disk usage goes, it sounds like you could improve the utilization by increasing Bucket.FillPercent to 1.0. By default Bolt splits pages in half since it assumes random writes. You can do some optimizations if you're writing in order.
Another option that'd be interesting to look at would be doing Catena on top of Bolt. You can write your large, compressed Catena-ized data as a single value in Bolt and it'll be laid out sequentially on disk. I know you mentioned that you don't need transactions but I find them to be really useful once you start distributing data across a cluster and need some guarantees about writes.