> Perhaps less than before, but still making very fundamental errors. Yes. Suppose someone developed a way to get a reliable confidence metric out of an LLM.

> Perhaps less than before, but still making very fundamental errors. Yes. Suppose someone developed a way to get a reliable confidence metric out of an LLM. Given that, much more useful systems can be built. Only high-confidence outputs can be used to initiate action. For low-confidence outputs, chain of reasoning tactics can be tried. Ask for a simpler question. Ask the LLM to divide the question into sub-questions. Ask the LLM what information it needs to answer the question, and try to get that info from a search engine. Most of the strategies humans and organizations use when they don't know something will work for LLMs. The goal is to get an all high confidence chain of reasoning. If only they knew when they didn't know something. There's research on this.[4] No really good results yet, but some progress. Biggest unsolved problem in computing today. [4] https://hungleai.substack.com/p/uncertainty-confidence-and-hallucination https://hungleai.substack.com/p/uncertainty-confidence-and-h...