3 ms·
> Stick it in one big 'DELETE WHERE id NOT IN (...)' query. I think that's a terrible idea, because it will scan the entire NOT IN (...) array for each row. Si
by empthought 11y ago
> Stick it in one big 'DELETE WHERE id NOT IN (...)' query.
I think that's a terrible idea, because it will scan the entire NOT IN (...) array for each row. Since it's NOT IN, it will even have to scan the whole thing every time. A temporary table is generally the way to go for unbounded lists of things; there's even support for it in PostgreSQL and the SQL standard.
- orf 11y agoI always assumed pg would be smart and use something with a constant lookup time. This[1] is a query plan 9.5 made with a table containing 10,000,000 sequential rows, executing "DELETE WHERE id IN (select * from generate_series(1, 1000000));". Maybe your right, it spends 5.8 seconds on the nested loop. I've only used this for smaller volumes where an extra table might be overkill, but I guess with loads more rows a temporary table is better. 1. http://tatiyants.com/pev/#/plans/plan_1454709615696 http://tatiyants.com/pev/#/plans/plan_1454709615696
- empthought 11y agoPG is doing the smart thing; it's optimizing for the common case of short IN/NOT IN literals in SQL queries. Simple arrays have much better constant factors. If the clause is a subselect (like yours) and is also an actual table (unlike yours), it will be smart and rewrite it as a JOIN, which will generally have better complexity if the indexes are correct.