4 ms·
No you can definitely use globs in DuckDB. And no you don’t have to know the exact parquet file. You would treat the Hive partitioned data as a single dataset
by wenc 3y ago
No you can definitely use globs in DuckDB.
And no you don’t have to know the exact parquet file. You would treat the Hive partitioned data as a single dataset and DuckDB will scan it automatically. (Partition elimination, predicate pushdown etc all done automatically)
https://duckdb.org/docs/data/partitioning/hive_partitioning https://duckdb.org/docs/data/partitioning/hive_partitioning
- cmollis 3y agook.. thanks.. I'll try it out. I can think of few use-case that we have where this might be a good alternative to athena.