3 ms·
> GPT-3.5 not good at fact checking at all, it will also hallucinate a lot of incorrect facts. Even with access to external information through a very basic Re
by dragonwriter 4y ago
> GPT-3.5 not good at fact checking at all, it will also hallucinate a lot of incorrect facts.
Even with access to external information through a very basic ReAct implementation GPT-3.5 is fairly decent at fact checking. Without external information to check against, whatever GPT-x is doing isn’t fact-checking.
GPT-4 has probably memorized more things because it is a bigger model that has been trained considerably more on a larger training corpus than GPT-3.5, but memorization and fact-checking are different things.
- clnq 4y agoIt depends on the definition of fact-checking. GPT-4 definitely is able to fact-check most statements more accurately than the human brain could in that same time. I would estimate that upon hearing a fact, I will be able to correctly say whether it’s accurate upon 5 seconds of Googling about 60% of the time. GPT-4 would beat me significantly. If we are talking about investigation-level fact-checking only using primary or academic sources, and spending significant time, effort, and resources to check the facts like some journalists do, then GPT-4 will probably do much worse. But just because we can say “in situation X, it will do better; in situation Y, it will do worse”, we can say that this capability exists. It can be fact-checking. Plug-ins help a lot with that as well. I think with plug-ins, it will meet your definition of fact-checking when asked. Unless it’s a very specific definition.