3 ms·
Are you using the iceberg-rust crate for Rust? It's a rather young project, have you found it sufficient for your needs (if using)?
by philippemnoel 2y ago
Are you using the iceberg-rust crate for Rust? It's a rather young project, have you found it sufficient for your needs (if using)?
- sammysidhu 2y agoWe're actually using pyiceberg to retrieve metadata! All our IO and decoding happens in the rust side once the data has been passthrough. We expose something called a ScanOperator which allows integration into various catalogs through a thin layer that exposes ScanTasks. Iceberg's impl: https://github.com/Eventual-Inc/Daft/blob/416009138359a9d410db90819ea708b630aa7a39/daft/iceberg/iceberg_scan.py#L84 https://github.com/Eventual-Inc/Daft/blob/416009138359a9d410...