3 ms·

So it looks like you don't use postgres extensions so you can run this on an EC2 against an Aurora Postgres instance and dump files to S3 Iceberg right? And ca

by gregw2 2y ago

So it looks like you don't use postgres extensions so you can run this on an EC2 against an Aurora Postgres instance and dump files to S3 Iceberg right? And can you then have Glue Catalog auto-crawl them and expose them in Athena? Or are they DuckDB-managed Iceberg tables essentially?

exAspArk 2y ago
Exactly! You can run it on any server connecting to any Postgres, without installing custom extensions (AWS Aurora supports only a limited number of extensions https://docs.aws.amazon.com/AmazonRDS/latest/AuroraPostgreSQLReleaseNotes/AuroraPostgreSQL.Extensions.html https://docs.aws.amazon.com/AmazonRDS/latest/AuroraPostgreSQ...). The Iceberg tables are created separately from the DuckDB query engine. So you should be able to read these Iceberg tables by using any other Iceberg-compatible tools and services like AWS Athena.