4 ms·
Is the `weird_rides.db` file python or some sort of duckdb script? I'm quite intrigued about DuckDBs ability to read parquet files off of buckets. How good is
by wodenokoto 2y ago
Is the `weird_rides.db` file python or some sort of duckdb script?
I'm quite intrigued about DuckDBs ability to read parquet files off of buckets. How good is at at simply ignoring files (e.g, filtering based on info in datafiles paths)?
- rcaught 2y agoI've successfully used it's glob support on S3.
- mvc 2y agoIf the objects in S3 are named according to hive conventions, it will do what you would hope and not read any files with no data to match the query. It will even use range requests in S3 to avoid fetching the entire blob. See here for the details https://duckdb.org/2021/06/25/querying-parquet.html#automatic-filter--projection-pushdown https://duckdb.org/2021/06/25/querying-parquet.html#automati...