3 ms·
The storage system we use only stores the rows that change. We have a blog post we're publishing on Wednesday explaining how.
by timsehn 7y ago
The storage system we use only stores the rows that change. We have a blog post we're publishing on Wednesday explaining how.
- jamesblonde 7y agoThat's nice. Do you have any idea if it is possible to translate those rows into higher-level time-travel queries? Like if you could plugin an adapter to transform the rows into a data structure (parquet, arrow, json, whatever) that could be useful to analytics and ML apps?
- timsehn 7y agoLike "as of" queries or history queries? We have both of those. AS OF: https://www.dolthub.com/blog/2020-03-20-querying-historical-data-with-as-of/ https://www.dolthub.com/blog/2020-03-20-querying-historical-... HISTORY SYSTEM TABLE: https://www.dolthub.com/blog/2020-01-23-access-to-everything-through-sql/ https://www.dolthub.com/blog/2020-01-23-access-to-everything... You can run `dolt q -r csv -q <query>` to output whatever you want to a CSV. We would need to do work to output a hierarchical format. I'm sure it's possible to build whatever time travel operation you want. We can produce an audit log of every cell in the database pretty quickly.