5 ms·
Wouldn't it be nice if Postgresql could gain a native new storage engine which serializes columnar data as parquet files? Does this even make sense? DuckDB alr
by jhoechtl 4y ago
Wouldn't it be nice if Postgresql could gain a native new storage engine which serializes columnar data as parquet files? Does this even make sense?
DuckDB already has fine Parquet support and I find it really helpful.
- skeletal88 4y agoNo need to change the database itself, these kind of features belong in foreign data wrappers. You can already treat csv files as tables or access tables in other databases(mysql, oracle etc)
- eatonphil 4y agoWith a massive performance impact (for reading csvs anyway). It's a matter of how much you want to keep using postgres versus if you want interactive-level performance. I'm not saying one is better or worse than the other but they are definitely not the same.
- tomnipotent 4y ago> Does this even make sense? Yes, but may not be practical. A non-trivial amount of code makes assumptions about the underlying data layout, so things like the cost-based optimizer and MVCC would basically need to be rewritten.
- masklinn 4y ago> Wouldn't it be nice if Postgresql could gain a native new storage engine which serializes columnar data as parquet files? Does this even make sense? Surely that's exactly what parquet_fdw is?
- pramsey 4y agoThere is the cstore extension, which is pretty much "native" (at some point these terms start to lose meaning... it's running in the same process space and transaction aware, etc.)