3 ms·
> A good LLM is still going to be right about most of the things it "improvises", even though it doesn't "know" anything. When I check on the factual details o
by recursive 3y ago
> A good LLM is still going to be right about most of the things it "improvises", even though it doesn't "know" anything.
When I check on the factual details of what ChatGPT tells me, it seems to be wrong a lot of the time. Maybe not most, but I've stopped relying on it for factual information.
- haswell 3y agoI think this is a matter of framing. When compared to a system that does nothing but assembles grammatically correct sentences, the correctness of LLM output is impressively high. But ultimately I’m not saying that the accuracy of LLMs is sufficient, just that thinking about LLMs as improv machines isn’t very helpful.
- seba_dos1 3y agoIt has an encyclopedia around and a bit of human-scale time to consult it before coming up with the next line, but it's still more of an improv actor playing an expert than an actual expert. Sufficiently clever improv actor with sufficiently complete encyclopedia will at some point become indistinguishable from actual experts, but LLMs are still far from that point in general. They can already be great at fooling non-experts though, like an actor that needs to be just factually-correct enough to be believable in their role.