3 ms·

I agree with all of this. Databricks are also holding back features from open source Delta (like bloom filters), which is their right. But then you can't claim

I agree with all of this. Databricks are also holding back features from open source Delta (like bloom filters), which is their right. But then you can't claim it is a community-driven open format, unless it is an animal farm version of that, where one of the versions is the Pig (some are more equal than others).

hacful-tonteg 3y ago
Databricks has a lot of nice closed-sourced components, e.g., Unity Catalog, Delta Live Tables and Photon (a C++ implementation of Spark). Delta itself seems fairly open-source: https://github.com/orgs/delta-io/projects/10/views/1 https://github.com/orgs/delta-io/projects/10/views/1 and hopefully someone will implement Liquid Clustering!