3 ms·
It is not a chicken and egg problem, it is just a requirement to have an RDBMS available for systems like DuckLake and Hive to store their catalogs in. Metadata
by matt123456789 11mo ago
It is not a chicken and egg problem, it is just a requirement to have an RDBMS available for systems like DuckLake and Hive to store their catalogs in. Metadata is relatively small and needs to provide ACID r/w => great RDBMS use case.
- dsp_person 11mo agoWhat about file-based catalogs with Iceberg? Found one that puts it in a single json file: https://github.com/boringdata/boring-catalog https://github.com/boringdata/boring-catalog
- saxenaabhi 11mo agoThen concurrency suffers since you have to have locks when you update files. That's also why ducklake performs better than others. For many use cases this trade-off is worth it.