2 ms·
We are not building it in house right now - we are focusing on OLTP. But have plenty ideas of how to do it in the future. In the first iteration we are thinking
by nikita 4y ago
We are not building it in house right now - we are focusing on OLTP. But have plenty ideas of how to do it in the future. In the first iteration we are thinking about an integration with Singlestore, Snowflake, and BigQuery. Singlestore will be best due to low latency updates, but the other two are very popular so integration with Snowflake and Big Query just makes sense.
In the future we can integrate a columnstore right into the engine to make a smooth on system experience. There are some awesome open source implementations: arrow and duckdb. Updatability is tricky but doable as proven by Singlestore and SQL Server (I'm ex SQL Server and a huge fan of this feature). Not this year.
- nojvek 4y agoMakes sense. I agree. DuckDB is pretty phenomenal. I enjoyed reading its source code and playing around with it. Another open source nice C++ codebase is Typesense for Hyperfast text search (algolia competitor). It’s been on my mind for many months how to build indexing like this as postgres extensions. I love how versatile postgres is with so many indexing datastructures.
- nikita 4y agoAgain it is a big commitment to do a good job on implementing columnstores. If you don't do it all the way it not very usable and you are confusing your users by giving them too many options. The performance expectation is now set by great columstore implementations and you just can't afford a half ass job here. Here is an example from Google AlloyDB: https://twitter.com/mim_djo/status/1527900193626025984 https://twitter.com/mim_djo/status/1527900193626025984. My understanding that DuckDB is even faster on the TPCH benchmark. TPCDS is much harder and I doubt AllowDB can even run it at any reasonable scale.