I agree and I think the reason it's sort of an indictment of LLMs is because there's no real way to guarantee performance over a range of input, which makes it

I agree and I think the reason it's sort of an indictment of LLMs is because there's no real way to guarantee performance over a range of input, which makes it unsuitable for many automation tasks. If you develop an LLM prompt that says "Add <user input> to 5" and prove that it works for a handful of cases, there's no guarantee it works for any other case. Whereas "(lambda (x) (+ x 5))" has some mechanical guarantee based on the implementation. And for the LLM, the negative result becomes more possible the higher the complexity of the prompt. This reduces LLMs to being suitable for tasks where reliability is not particularly important, or a supervisory person or system is checking for consistency and somehow can re-run the LLM to generate new results (which itself is not always reliable).