3 ms·
I'm a bit confused by this being an in-memory only database and also using Apache Parquet. Isn't Apache Parquet a file format specification? Whats the point of
by psanford 4y ago
I'm a bit confused by this being an in-memory only database and also using Apache Parquet. Isn't Apache Parquet a file format specification? Whats the point of using it if you aren't serializing data to disk? Is the goal to eventually support durable storage?
- brancz 4y agoYou're absolutely right that that's confusing. Short answer: We just wanted to release this as soon as possible and haven't gotten to it yet. Slightly longer answer: We currently build parquet buffers in-memory which we are soon going to persist. We still want to finish up some details about how we partition and compact data over time before we do that though. We've learned from previous projects that once we write to disk people start to depend on that :)