3 ms·
I'm always surprised at how much room there is for big performance improvements, even in a system as mature as PostgreSQL. Evaluating an agg node is a pretty co
by drob 12y ago
I'm always surprised at how much room there is for big performance improvements, even in a system as mature as PostgreSQL. Evaluating an agg node is a pretty core piece of functionality, and there's still room for a pretty big win here.
PostgreSQL is powerful and feature-rich, and there's still so much work to be done. (Consider that index-only scans themselves are new as of 9.2!)
Very cool project.
- jldugger 12y agoWell, you know, until recently, 'my data fits in RAM' just didn't happen at a scale worth optimizing for.
- drob 12y agoSure, but that itself is interesting: context around data systems is changing so fast that even the mature ones have to innovate pretty quickly to keep up. (Or maybe especially the mature ones?) I wonder how long before postgres includes an index type optimized for in-memory workloads, e.g. a skiplist.
- yangyang 12y agoIf you read through it, this is a change to the Citus cstore_fdw extension, which stores data outside PostgreSQL and queries it using the PostgreSQL Foreign Data Wrapper API (see https://wiki.postgresql.org/wiki/Foreign_data_wrappers https://wiki.postgresql.org/wiki/Foreign_data_wrappers). So it's impressive, but it isn't a performance improvement for core PostgreSQL, or data stored natively in PostgreSQL (in particular, cstore_fdw is not very flexible - from the README for that project: "Note. We currently don't support updating table using INSERT, DELETE, and UPDATE commands.")
- drob 12y agoI did see that, although it's plausible that an analogous change would be helpful in core postgres. The readme seemed to suggest that they tried this on cstore_fdw because that was easier on which to develop, not because there was more low-hanging fruit.
- yangyang 12y agoIt sounds like it was precisely because there was more low-hanging fruit. I think column-store databases are far more amenable to vectorised processing. I'm not aware of any row-store databases which do it, and most research into it has been on column-store (or hybrid) systems.
- masklinn 12y agoIt's not so much of a surprise for postgres, after all the team only started working on performance in the 7.x or 8.x cycle, before that Postgres was known for a "correctness first, speed… maybe?" approach.