3 ms·
> You can also use a wildcard in the path (e.g. s3://mybucket/hits/*.parquet) to query a list of files. So is this an FDW that provides a similar function to A
by koolba 3y ago
> You can also use a wildcard in the path (e.g. s3://mybucket/hits/*.parquet) to query a list of files.
So is this an FDW that provides a similar function to Amazon Athena?
Does every query fetch everything again or does it somehow cache the downloaded data files?
Running something like this on EC2 is interesting as you wouldn’t pay the intermediate transfer costs. Only the final results would egress.
- mslot 3y agoIt caches the files on locally attached NVMe drives in the background, and continues to use range requests to S3 for queries until the full download completes.