4 ms·
Had a similar thought. Azure Postgres has something similar to pg_parquet (pg_azure_storage), but we're looking into replacing it with pg_duckdb assuming the e
by AdamProut 2y ago
Had a similar thought. Azure Postgres has something similar to pg_parquet (pg_azure_storage), but we're looking into replacing it with pg_duckdb assuming the extension continues to mature.
It would be great if the Postgres community could get behind one good opensource extension for the various columnstore data use cases (querying data stored in an open columnstore format - delta, iceberg, etc. being one of them). pg_duckdb seems to have the best chance at being the goto extension for this.
- mslot 2y agoFun fact, I created pg_azure_storage :)
- brinox 2y agoI was just wondering if pg_parquet could be combined with pg_azure_storage to write Parquet files to Azure Storage. I had problems with pg_azure_storage in the past, because the roles pg_read_server_files and pg_write_server_files are unassignable on Azure PostgreSQL databases which makes the use of `COPY {FROM,TO}` impossible.
- mslot 2y agoAzure is not supported as a backend in pg_parquet right now, but shouldn't be hard to add (contributions welcome!) https://github.com/CrunchyData/pg_parquet https://github.com/CrunchyData/pg_parquet It would not be safe to let any user access object storage. Therefore, pg_parquet has two roles called parquet_object_store_read and parquet_object_store_write that give permission to COPY FROM/TO object storage (but not local file system). In pg_azure_storage there is a comparable azure_storage_admin role that needs to be granted to users that need Azure Blob Storage permission.
- dektol 2y agoIs pg_azure_storage available on GitHub?