3 ms·
There are definitely ways to cleanly make Postgres scale for analytics. We didn't discuss in this blog, but we will be writing about them in the future. For exa
by chuckhend 3y ago
There are definitely ways to cleanly make Postgres scale for analytics. We didn't discuss in this blog, but we will be writing about them in the future. For example, check out what the folks at ParadeDB are doing. https://github.com/paradedb/paradedb https://github.com/paradedb/paradedb. Neon is doing an awesome job separating compute from storage. Supabase contributed foreign data wrappers make it super easy to read from S3 into Postgres. Lots of great work going out there :)
- kdamica 3y agoI totally agree that there are great solutions out there to use Postgres for specific analytics tasks, especially realtime ones, but if you want something that can handle arbitrary aggregations and do ad hoc analytics, nothing comes close to a standard data warehouse. Lots of exciting things happening in the space so that might change!
- philippemnoel 3y agoOne of the authors of ParadeDB here. While ad hoc aggregations in vanilla Postgres are slow due to lack of column-oriented storage, extensions like pg_analytics are addressing that problem. In our view, one of the main use cases of a data warehouse is when you need separation of storage/compute, which enables distributed analytics and scalable storage. With Postgres, query processing + storage are all happening on the same node. That being said we're looking at ways to separate out storage to an external data lake as part of pg_analytics.
- kdamica 3y agoVery cool! I’ll be super interested to see how this develops.