3 ms·
Yet Another Database.... As Shelnutt2 states below, its success will depend on how quickly TileDB can be integrated into other tools. However, I don't see any
by oxfordmale 6y ago
Yet Another Database.... As Shelnutt2 states below, its success will depend on how quickly TileDB can be integrated into other tools.
However, I don't see any benefit in TileDB supporting fast and efficient updates (and duplicates) of time series data. Time series should be immutable, and only in rare occasions require updating.
- Shelnutt2 6y agoTime series data can vary and I'd agree that most time series is immutable. There are however use cases in which updates can happen. For instance, at my previous job before TileDB Inc, we had a case where 99% of our data was immutable and never updated but a very small amount of data could be updated if there were late arriving parts. In order to get near-realtime data we accepted that sometimes some columns might not be available within the window the datapoint represents. In that case that record might reappear at a later time with the complete and correct values. Of course there are trade offs, we could have forced a longer waiting period until we were confident the data was finalized. We also could have ignored the updates. In the end we used a system of staging tables for loading the last 24 hours of data before merging out into a more finalized table. This kept the load of updating records in the database down, and still allowed us to achieve our goals. At the time, several years ago, I was not aware of TileDB else we would have considered it instead of a more traditional database vendor.