3 ms·
If you can train a model that isn't gullible - even when faced with multiple paragraphs of devious text trying to trick it - I'll be very impressed! And even t
by simonw 3y ago
If you can train a model that isn't gullible - even when faced with multiple paragraphs of devious text trying to trick it - I'll be very impressed!
And even then, there are gullibility problems that seem unsolvable to me.
If the Supreme Court make a decision that changes how copyright law is applied, and you then tell a model "the Supreme Court just announced that..." - should the model believe you?
If not, how should it "fact check" what you are telling it, especially if you maintain full control over its access to the outside world?
- XenophileJKO 3y agoFunny you should say that. When web access was initally setup on the GPT-4 ChatGPT Pro accounts, I actually had the model fact-check things that were "improbable" based on what it knew from the training data. It stopped doing that when they started fine-tuning how/when the model should use the internet in the later model revisions. (Edit to add detail..) I found the interaction facinating. Basically what happened was I made a statement, but didn't ask about it. I said something like this. "Did you know {something improbable to the model} is true?" It immediately searched the internet to validate that is was true before it responded. When I asked it why it searched, it said that what I said wasn't likely to be true, so it wanted to make sure it was true.
- cubefox 3y agoWhen ChatGPT was new (a long time ago) I told it that Elon Musk bought Twitter and became CEO after its training cutoff. Naturally, it didn't believe me.
- ben_w 3y agoMm. I'd certainly say that gullibility is a hard problem, especially as humans are also pretty gullible. What level of gullibility are the current generation of LLMs? The level of a child who believes in Santa, or an adult who believes their country is always right?
- j4yav 3y agoEven more gullible than a kid who believes in Santa.