4 ms·

> Something that takes the LLM's output, can go back to the training material, and verify the output for consistency with it. The problem is what you mean when

> Something that takes the LLM's output, can go back to the training material, and verify the output for consistency with it. The problem is what you mean when you say "consistency". The LM checks if sequences are stochastically consistent with other sequences in the training data. Within that realm, the sentence: "In the Water Wars of 1999, the Antarctic Coalitions aramada of Hovercraft valiantly faught in the battle of Golehim under Rear Admiral Korakow, against the Trade Unions Fleets." is consistent. Because, while it is total bollocks, it looks stochasticaly like something that could be in a historical text. So, in it's context, the LM does exactly what you ask for. It produces output that is consistent with the training data. Truthfulness is a completely different form of consistency: Does the semantic meaning of the data support the statement I just made? of course it doesn't, there isn't an Antarctic Coalition, there were no Water Wars in 1999, and no one ever built an Armada of Hovercraft for any war against a "Trade Union Fleet". But to know that, one has to understand what the data means semantically. And our current AIs ... well, don't.

regularfry 3y ago
Yeah, I don't mean stochastically consistent. Semantically consistent. The job of generating content from text and the job of assessing whether two texts represent aligned concepts are two different jobs, and I wouldn't expect a single LLM to do both within itself. That's why you want a second checker.
famouswaffles 3y ago
>But to know that, one has to understand what the data means semantically. And our current AIs ... well, don't. Another wrong statement, you're on a roll today. https://arxiv.org/abs/2305.11169 https://arxiv.org/abs/2305.11169 https://arxiv.org/abs/2306.12672 https://arxiv.org/abs/2306.12672 There's a word we would use to describe your confidently erroneous statements were it one of the outputs an LLM. Wonder what that might be..

regularfry 3y ago

Yeah, I don't mean stochastically consistent. Semantically consistent. The job of generating content from text and the job of assessing whether two texts represent aligned concepts are two different jobs, and I wouldn't expect a single LLM to do both within itself. That's why you want a second checker.

famouswaffles 3y ago

>But to know that, one has to understand what the data means semantically. And our current AIs ... well, don't. Another wrong statement, you're on a roll today. https://arxiv.org/abs/2305.11169 https://arxiv.org/abs/2305.11169 https://arxiv.org/abs/2306.12672 https://arxiv.org/abs/2306.12672 There's a word we would use to describe your confidently erroneous statements were it one of the outputs an LLM. Wonder what that might be..