4 ms·
Yes this has been an up-and-coming theme in the data science world. Arrow for the data format, Ibis for the API, Acero/Velox/DataFusion/DuckDB/Polars for execut
by sakras 3y ago
Yes this has been an up-and-coming theme in the data science world. Arrow for the data format, Ibis for the API, Acero/Velox/DataFusion/DuckDB/Polars for execution, Substrait for the query plan representation, etc.
- soulbadguy 3y agoIsn't spark already providing a bundled and somewhat functional version of all of this ?
- sakras 3y agoRight, so I think the idea is to "unbundle" this so that you can compose your own data analytics engine.