4 ms·

Delta is nice, but a lot of features are missing from the FOSS version. Hudi is nice, but they are in the middle of a big format change right now. Iceberg is

Delta is nice, but a lot of features are missing from the FOSS version. Hudi is nice, but they are in the middle of a big format change right now. Iceberg is nice, but is the most conservative and slow format out of three.

MrPowers 3y ago
There are a few features missing from the FOSS Scala/Spark implementation of Delta Lake, but I wouldn't say a lot. The FOSS version supports all the table features in the Delta Lake protocol. The Delta Rust implementation is missing more table features, but we're closing the gap fast. We just added support for constraints to Delta Rust and are working on change data feed right now.
- orthoxerox 3y ago
  Delta Live Tables and automatic vacuuming are the two big features I'm missing.
  - MrPowers 3y ago
    Delta Live Tables are a Databricks feature and aren't related to Delta Lake. Can't you just setup a cron job to vacuum periodically?
  - switchbak 3y ago
    DLT is a big feature of a full platform, you can’t really say that’s a missing feature of a delta library.
chimerasaurus 3y ago
I’d take issue with the “Iceberg is slow” theme that Databricks in particular has tried to push. If that were true, Snowflake would not be as fast on Iceberg/Parquet as its native format. The engine makes something fast or slow, not the table format. Disclaimer - am at Snowflake.
- orthoxerox 3y ago
  Back when were choosing between the three formats about 1.5 years ago, Iceberg was definitely the slowest. If the situation has changed since then, I would love to see an updated comparison. We tested all three of them using Spark batches that converted a stream of changes into SCD2.