5 ms·
I'm curious at what threshold you're seeing that kind of query planner disparity. I use postgres professionally at work and when testing a feature I'll load up
by skunkworker 6y ago
I'm curious at what threshold you're seeing that kind of query planner disparity. I use postgres professionally at work and when testing a feature I'll load up a local database with 100k to 500k rows in order to simulate a production environment, normally this is more than enough to get an almost identical query planner result (after running explain (analyze, buffers) etc).
But I haven't seen cases where a 10000x slowdown occurs after deleting data, unless you're hitting data where it spills over an in-memory sort, or is no longer efficient to do a heap/sequential scan.
The hardest thing with postgres and query planning IMO is understanding what kind of index would need to be used, and ordering composite indexes accordingly. I've extensively used a sortable date/id column as the last entry in an index as postgres will end up only doing an index scan.
On the other side there are times where the indexes for a table are greater as whole then the data contained therein.
Though if you're running in the multiple billions of rows in a single table and haven't paritioned/sharded it in some way, YMMV.
- xyzzy_plugh 6y agoYes, I agree with this entirely. While not perfect, if you treat your data as real data, instead of magical database rows, you tend to get predictable results. Pretend the database doesn't exist and these are data structures in your process space -- where do the access patterns degenerate, etc.
- doteka 6y agoThis is our case. A tiny staging environment will cause two tables to have billions of rows. This kills the Postgres. We are looking at moving this data out of the database and query it using Drill or similar. But if anyone has tips for handling huge tables in Postgres (yes, we partition already, this is a singe partition) I’m all ears.
- sojournerc 6y agoIf it's time series data check out the timescaledb extension.