4 ms·

How do we know ChatGPT isn't hallucinating parts of its answers? I think this is an interesting service but what are the guardrails so we can trust the answer?

deleted 3y ago
[deleted]
boredemployee 3y ago
That's the problem, we don't. All the tools on top of GPT that I tried has the same problem.
- freedmand 3y ago
  That's a big part of the reason I built the open source tool Semantra[1]. I only really trust the semantic search part of this problem as it is constrained by context, so I tried to make the user experience good on top of simpler semantic search without any chatbot capabilities. [1] https://github.com/freedmand/semantra https://github.com/freedmand/semantra
  - boredemployee 3y ago
    that seems a really good tool, will try it later. thanks for sharing, I hope you get the deserved traction with it!
slashdev 3y ago
The same you get with ChatGPT - no guarantees. It's BYOB (Bring Your Own Brain), so don't trust it more than you would anything else you read on the Internet. I get that people have an objection to how LLMs invent things, but so do people. You should always verify things.
oofbey 3y ago
GPT definitely makes mistakes. GPT4 less than GPT3.5, and way less than Bard. Having spent quite a lot of professional time working through real-world problems with GPT4, I'll say it very rarely does something I'd call hallucination. It almost never invents crazy things out of whole cloth. Sometimes it makes wrong guesses about things. You gotta keep your own head and be skeptical. Often you can tell if it's coming unhinged. If it makes mistakes and you notice, it'll definitely own them unlike many LLMs. But of course it's not perfect.
williamcotton 3y ago
Hallucinations mainly occur when the prompt does not contain the information required for the completion. When you add parts of the content of the PDF to the prompt, which is how these tools work, then it is very likely to produce a completion consistent with the PDF. I say “mainly” and “very likely”, but in my experience I have never had a hallucination when the prompt is augmented with factual data. Hallucinations occur with regularity when the prompt does not contain what is requested to be completed.

deleted 3y ago

[deleted]

boredemployee 3y ago

That's the problem, we don't. All the tools on top of GPT that I tried has the same problem.

freedmand 3y ago
That's a big part of the reason I built the open source tool Semantra[1]. I only really trust the semantic search part of this problem as it is constrained by context, so I tried to make the user experience good on top of simpler semantic search without any chatbot capabilities. [1] https://github.com/freedmand/semantra https://github.com/freedmand/semantra
- boredemployee 3y ago
  that seems a really good tool, will try it later. thanks for sharing, I hope you get the deserved traction with it!

slashdev 3y ago

The same you get with ChatGPT - no guarantees. It's BYOB (Bring Your Own Brain), so don't trust it more than you would anything else you read on the Internet. I get that people have an objection to how LLMs invent things, but so do people. You should always verify things.

oofbey 3y ago

GPT definitely makes mistakes. GPT4 less than GPT3.5, and way less than Bard. Having spent quite a lot of professional time working through real-world problems with GPT4, I'll say it very rarely does something I'd call hallucination. It almost never invents crazy things out of whole cloth. Sometimes it makes wrong guesses about things. You gotta keep your own head and be skeptical. Often you can tell if it's coming unhinged. If it makes mistakes and you notice, it'll definitely own them unlike many LLMs. But of course it's not perfect.

williamcotton 3y ago

Hallucinations mainly occur when the prompt does not contain the information required for the completion. When you add parts of the content of the PDF to the prompt, which is how these tools work, then it is very likely to produce a completion consistent with the PDF. I say “mainly” and “very likely”, but in my experience I have never had a hallucination when the prompt is augmented with factual data. Hallucinations occur with regularity when the prompt does not contain what is requested to be completed.