3 ms·
DuckDB is in-memory. It makes sense as a in-memory columnar cache. We went with DataFusion to benefit from the deep integration it has into the ecosystem with A
by philippemnoel 3y ago
DuckDB is in-memory. It makes sense as a in-memory columnar cache. We went with DataFusion to benefit from the deep integration it has into the ecosystem with Arrow, Parquet, etc.
As for Delta Lake, we would've loved to do Iceberg but the Rust crate for Iceberg is very new, and doesn't have enough functionalities yet. We're tracking the project and planning to contribute, so that we can add support for Iceberg eventually