4 ms·
The scary thing about this not the mistake itself, but that it learned the mistake from a previous LLM that either hallucinated the answer, or "learned" it from
by LeoPanthera 2y ago
The scary thing about this not the mistake itself, but that it learned the mistake from a previous LLM that either hallucinated the answer, or "learned" it from random Reddit comment.
I suspect we'll quickly see that LLMs need to be trained only on low-background-steel text, with a guarantee that it contains no LLM generated text in it, for it to actually be useful. In practice this probably means no text written after 2021.
- astrange 2y agoRather than "you need to exclude all new text", you want something like "the distribution of outputs including all new text can't disagree too much with the distribution of outputs from only known-good text".
- karmakurtisaani 2y agoThat's going to be really hard to implement though. If I say humans have recently learned to levitate, how do you even check that? It's not like there is a reliable source to say that it's wrong.
- astrange 2y agoPretty hard aside from the lack of other people saying they've done it. It probably works better for keeping the writing style from deteriorating.
- SrslyJosh 2y ago> hallucinated the answer An LLM that's "hallucinating" is doing the exact same thing as an LLM that's generating "correct" output. =)
- LeoPanthera 2y agoThat would seem to be irrelevant to the point.