4 ms·
>the delta also still keeps partitioning information in the hive metastore, while iceberg keeps it in storage, making it a far superior design. Check out https
by saj1th 5y ago
>the delta also still keeps partitioning information in the hive metastore, while iceberg keeps it in storage, making it a far superior design.
Check out https://github.com/delta-io/delta/blob/3ffb30d86c6acda9b59b9b029e6e945992ef4f42/PROTOCOL.md https://github.com/delta-io/delta/blob/3ffb30d86c6acda9b59b9... when you get a chance. You don't need hive metastore to query delta tables since all metadata for a Delta table is stored alongside the data
>they did not include features like optimizing small files
For optimizing small files, you could run https://docs.delta.io/latest/best-practices.html#compact-files https://docs.delta.io/latest/best-practices.html#compact-fil...