4 ms·
For basic reads and writes the FDW APIs are quite suitable and they work well for cstore_fdw. However, to get the most out of columnar storage you also need vec
by mslot 11y ago
For basic reads and writes the FDW APIs are quite suitable and they work well for cstore_fdw. However, to get the most out of columnar storage you also need vectorized execution that can calculate aggregates on blocks of values. The FDW APIs are not well-suited for this purpose, since they pull one row of data at a time. We've done some experiments in which we used the executor hooks to implement a vectorized executor for cstore_fdw [1] and it can give dramatic performance gains. However, the SQL support of this solution is limited as the APIs force you to build a new executor from scratch.
With sharding, the distributed executor can work on the SQL level and translate one SQL query to many SQL queries. In that case your distributed executor can focus on issues such as partitioning and merging data, while the low-level work is still being done by the regular PostgreSQL executor, so you don't have to implement it from scratch.
[1] https://github.com/citusdata/postgres_vectorization_test https://github.com/citusdata/postgres_vectorization_test