3 ms·
Engineer: "HELP, our production DB is frozen on this query that worked fine before!" Infra: "Hmm, let's check... Well would you look at that, it seems like you
by 2001zhaozhao 17d ago
Engineer: "HELP, our production DB is frozen on this query that worked fine before!"
Infra: "Hmm, let's check... Well would you look at that, it seems like your LLM query planner usually works and produces fast queries, but this time when you changed a variable name to trigger query rebuild, it happened to hallucinate and miss an index, would you mind re-running the LLM a few times until you get a faster query?"
- malisper 16d agoFunnily enough, you could replace "LLM query planner" with just "query planner" and this comment would still hold true
- deleted 16d ago[deleted]
- Tanjreeve 16d agoThat bug is fixable and verifiable. The LLM you cross your fingers till the next time the same thing happens.
- egeozcan 16d ago> fixable and verifiable By people with a specific skill set. LLMs generation can also be fixed and verified by people with a certain skill set, and non-deterministic computing doesn't automatically mean unpredictable. When people say that the LLMs are a black box, it means unpredictability in unknown situations. You do structured output, input validation, output validation, lower temperature, limit decisions, RL, etc. to increase predictability to near certainty. It's just statistics after all. Or you can as well generate the code to do the job. It's just that the required skill set is a different one to do those things, and unusual in the context of DB administration.
- Tanjreeve 16d agoOnly if someone is planning on running a pinned self hosted version of an LLM alongside the DB to fix the problem. The developer can change the binary easy enough and test it but the LLM approach just seems either theoretical or bending ourselves in knots to justify using an LLM.
- egeozcan 16d agoI didn't argue that it'd make sense, I just said that it could be reasonably fixable when problems occur and verifyable that the fix works. Even if shipping and RLing an LLM were easy tasks in terms of software distribution (they are not) we'd still hit the skill mismatch, as I said in my previous comment. I just find the "all llms are non dererministic and therefore unreliable" narrative a bit backwards. All software that has more than 0 users needs to deal with non-determinism anyway :)
- tempfile 16d agoNone of the things you mention are guaranteed to increase the probability of correctness. You can run the LLM output through as many deterministic programs as you like, but "the query plan runs in acceptable time" is not something you can verify with such a tool. Nobody knows how the LLM does it, so they cannot know how to make the LLM do it better.
- malisper 16d agoEven if the query plan was not generated by an llm, you can't verify it will run in an acceptable time. This is one of the biggest unsolved problems in databases
- Tanjreeve 16d agoSo what is an LLM bringing to the table if it's not just a fun experiment?
- egeozcan 16d ago> Nobody knows how the LLM does it From a completely technical perspective, we have a rough idea how the LLMs work, and improving a system requires measuring outcomes and you don't necessarily need to understand the mechanism. EXPLAIN ANALYZE against data that's similar in size to prod checks a query written by an LLM as good as anything we can write... but, yes, you're right, we still didn't solve the halting problem - neither the LLMs.
- d0100 16d agoI've reduced queries from minutes long to 2s in Postgres by duplicating a CTE and keeping it unused Query planner feels pretty LLM-esque already
- Tanjreeve 16d agoThis isn't a conversation about the guts of query planners but postgres is known for what can only be described as gremlins in the query planner. But that is not the same thing as being non deterministic.
- malisper 16d ago> That bug is fixable and verifiable A bad query plan is not your typical kind of bug. I would definitely not call it fixable. Query planners are inherently dealing with estimations and approximations. If the query planners estimation is off, you're screwed. Unless you come up with a way to cheaply determine exactly how many rows a query will return, bad query plans will still exist.
- danielbarla 16d agoIndeed. I'm honestly shocked that we're still having to evict bad query plans in 2026. And "you changed a variable", ha, that sounds like an actual reason. How about "data statistics were automatically refreshed and you hit some magical undocumented heuristic threshold, an the query that ran in 35ms yesterday now takes 45 minutes. And we can actually tell you this because we have the data, but decided to let you find out manually, instead." I've literally been saying "I can't believe the date is X and we still have to put up with this" for around 25 years now.
- galkk 16d agoOur database 12.34 was working great, but 12.35 deployment had some optimizer changes that had regression on exact scenario that you have in your statistics. Shit happens, sorry. Use this hint.
- reval 16d agoI’ve seen this happen to SQL Server many times. Every time the solution is a stored proc with the recompile option enabled.
- simondotau 16d agoThe answer to every problem is to rebuild statistics. (And never use stored procedures. It's just a shitty API layer in the worst language imaginable, sitting outside of source control. If you need an API layer, write it in a real language, ideally the one you're already using.)
- mr_toad 16d agoIn my experience with databases the answer is never never, and never always, and it’s almost always sometimes and maybe.
- simondotau 15d agoThat’s mostly occasionally accurate.
- ants_a 15d agoRebuilding statistics will not help if the cause of the bad plan is something that the cost based optimizer is not even trying to model. Stored procedures in this context are just a clumsy workaround to control the planner so your comment about API layer is irrelevant. But if you like, you can write stored procedures in a ton of different languages. And if you do not have source control for your database artifacts, you are doing it wrong. A common reason for stored procedures is to not have a bunch network roundtrips in the middle of your transaction logic while you are holding onto locks / have an open conflict window.
- simondotau 15d agoSQL Server stored procedures don’t provide a separate tier of T-SQL functionality. Anything they do to the data can generally be expressed and executed as a T-SQL batch. If you think you need stored procedures to avoid network roundtrips in the middle of your transaction logic, you’d be wrong. Stored procedures are essentially a crude, database-bound API layer. For serious application development, a proper service layer provides stronger contracts, authentication, testing, versioning, observability and source control in a sane general-purpose language, ideally the same one you’re already manipulating the data with elsewhere.
- deleted 16d ago[deleted]
- ehe78qhe 16d agoThis exact situation has happened to me with Postgres' stock query planner when an automatic analyze got a bad sample of a large table.
- websap 16d agoSounds like a skill issue if your infra can't validate these changes.
- ehe78qhe 16d agoThe issue is that a query plan that works when the data is in a certain state can fail when the data has changed. And the pesky business insists on constantly changing the data for some reason
- wodenokoto 16d agoI’m more worried that the query planner requires more compute than the query.