> If you’re looking for factual answers - then you need to verify what comes out of these models. And let’s be honest you should be doing this with any source o

> If you’re looking for factual answers - then you need to verify what comes out of these models. And let’s be honest you should be doing this with any source of information I've seen a lot of statements like this - equating, often flippantly, ChatGPT's (lack of a) relationship with truth to the fact that humans make mistakes, stackoverflow answers are sometimes wrong, etc. It's really not the same thing, though, neither quantitatively (ChatGPT is "wrong" much more often than existing information sources) nor qualitatively (ChatGPT is "wrong" in a different way to existing sources - it is optimised for plausibility and freely mixes correct and incorrect statements in ways highly likely to trip up even the wary). If I google the definition of a word and I see the top result contains a definition visible on the results page and sourced from Merriam-Webster, then no I generally don't need to double check it or even follow the link. I have reasons to believe both that the information did come from that source and that the source is unlikely to be fabricated. If I ask ChatGPT I have no reason to believe anything it says. Here's what I got from ChatGPT for "What is Gell-Mann amnesia?" > Gell-Mann amnesia is a term coined by science writer Michael Crichton to describe the phenomenon where people tend to forget information that contradicts their preconceptions or beliefs. The term is named after physicist Murray Gell-Mann, who won the 1969 Nobel Prize in Physics for his work on the theory of elementary particles. > According to Crichton, Gell-Mann amnesia occurs when people encounter information that contradicts their beliefs, but rather than updating their beliefs to reflect the new information, they simply forget the information and continue to hold onto their preconceptions. This type of cognitive bias can be a significant barrier to learning and can lead to flawed decision-making. The first paragraph of this contains a lot of completely correct factual information: the term was coined by Michael Crichton and named after Murray Gell-Mann, who did win a Nobel prize in that year for that reason. However, the actual statement of what Gell-Mann amnesia means in both paragraphs is completely wrong but utterly plausible. (The real meaning of Gell-Mann amnesia is extremely relevant to the ways some people are currently interacting with ChatGPT.) For this output to be any use to me, I would have to individually Google (or rather Kagi in my case) every individual thing it has said. If I don't I will swallow complete fabrications. Not merely occasionally, but in my experience most times I use it. I've seen people on hn saying they are doing hundreds of ChatGPT "queries" (they are not queries) a day and "learning a lot" from it. I cannot believe these people are spending anywhere near the amount of effort required to properly validate the results, because it would take more effort than just googling the info in the first place. Even Sam Altman says not to use to use ChatGPT as a source of factual information: https://twitter.com/sama/status/1601731295792414720?ref_src=twsrc%5Etfw https://twitter.com/sama/status/1601731295792414720?ref_src=...