> What's important is reliability of the output given a distribution of inputs, and that's where LLMs are unreliable. Nailed it! Thank you for stating this bet

> What's important is reliability of the output given a distribution of inputs, and that's where LLMs are unreliable. Nailed it! Thank you for stating this better than I did. Maybe I should have used the word "trustworthy" instead of "predictable". If I'm using an LLM as a calculator to solve math problems for my users, I may be able to say with certainty that my model is 99% accurate, but I'm not able to know ahead of time which questions it's going to miss, and which ones it's going to get correct. It's a near-infinite input space, so it's difficult to prove correctness via induction. The attention mechanism is (thus far) fairly inscrutable, and thus proving the correctness via deduction is (currently) not possible either. It would feel weird to use an LLM for a calculator, just as it feels weird to use an LLM as a compiler. I want something that uses an algorithm that is provably and predictably correct -- not one that is "almost always" correct. That unpredictability of accuracy isn't connected to the temperature used when sampling, so much as the inscrutable nature of the attention mechanism when verifying the trustworthiness over the range of all possible inputs.