3 ms·
I am yet to see GPT-4 admit to not knowing something rather than just hallucinating made up shit. If it could reliably tell me when it DOESN'T know something,
by llamaLord 4y ago
I am yet to see GPT-4 admit to not knowing something rather than just hallucinating made up shit.
If it could reliably tell me when it DOESN'T know something, I'd have a lot more respect for it's capabilities. As it stands today, I'd feel I need to fact check nearly anything it gave me if I'm in an environment that requires high levels of factual accuracy.
Edit: To be clear, I mean it telling me it doesn't know something BEFORE hallucinating something incorrect and being caught out on it by me. It will admit that it lied, AFTER being caught, but it will never (in my experience) state that it doesn't have an answer for something upfront, and will instead default to hallucinating.
Also - even when it does admit to lying, it will often then correct itself with an equally convincing, but often just as untrue "correction" to its original lie. Honestly, anyone who wants to learn how to gaslight people just needs to spend a decent amount of time around GPT-4.
- vidarh 4y agoWell, I have. It still certainly goes the way you're describing often as well, to the point I was flabbergasted when it happened, but it did. Specifically I asked it to give me examples of using an NPM module published after the knowledge cutoff. [To be clear, I did not tell it it was from after the cutoff]