6 ms·

This is funnier than it has any right to be, mostly given the over-abundant "AI will fix everything" narrative currently dominating the hn discourse. LLMs are

This is funnier than it has any right to be, mostly given the over-abundant "AI will fix everything" narrative currently dominating the hn discourse. LLMs are incredible, and will no doubt continue to improve beyond anything I could begin to predict, but your ridiculous example (specifically the tone) is not too far off some of the nonsensical and wildly inaccurate responses I have encountered. I find rhe unfailing confidence rather endearing. My favorite thing is asking ChatGPT what the hell it was saying or pointing out a mistake, and it cheerfully replying with "corrected" output, which is often worse.

NovaDudely 3y ago
There is a kind of charm to AI Hallucinations. To confidently talk nonsense.
startupsfail 3y ago
It’s interesting, I wonder how that style of a cheerful corrected (and wrong) output had emerged. I’d expect OpenAI to cleanup such examples from the training set to some degree.
- wizzwizz4 3y ago
  I assumed OpenAI had added that themselves.
  - startupsfail 3y ago
    They’ve added incorrect answers?
    - wizzwizz4 3y ago
      No; those were there already. I assumed they'd added the "does not double down" behaviour. Bing Chat (also an OpenAI model, but presumably with a different set of fine-tuning / filters to ChatGPT – possibly the bare OpenAI API) doubles down in situations like this: https://nitter.dark.fail/_akhaliq/status/1672267392280571905 https://nitter.dark.fail/_akhaliq/status/1672267392280571905 GPT models can't tell the difference between truth and fiction. All you can choose by fine-tuning is their threshold for "admitting" mistakes.
      - startupsfail 3y ago
        I do think it can tell the difference between truth and fiction. It’s a world model, it includes models of concepts like truth and fiction and can apply these concepts. But that’s beyond the point. The question is, how would you include incorrect responses into the training. In a way that it would not increase the probability of the model to give an incorrect response? I guess you can maybe train with a mix of correct and incorrect responses, hallucinations and nonsense in the conversation, but then make clear that the responses were incorrect, adding context to these. And then fine-tune the AI actor to avoid giving incorrect responses or hallucinations altogether.
IIAOPSW 3y ago
Try pointing out its mistakes when it hasn't made a mistake. You'll usually get the same "I'm sorry, here's the correct answer" output.
- eterm 3y ago
  And getting it to derive it's mistake doesn't always work (NB: This is GPT3 & bard, GPT4 is much better): https://richardcocks.github.io/prime.html https://richardcocks.github.io/prime.html
taylodl 3y ago
LLMs are incredible tools, but they're just that - tools. They're not a replacement for human judgment. Heck, one of the things you have to judge is how accurate the answer is. The LLM can't assess that - as you note it has unfailing confidence in reporting the wrong information. What I have found, and I've been using Bard over ChatGPT because Bard seems to be a bit smarter at first glance, is these tools are powerful but limited. They can augment a workforce but only a fool, soon to go out of business, would use them to replace a workforce.
- NovaDudely 3y ago
  This is just it, I don't fear LLM's - that are a really cool technology that once you understand their utility are really cool. I fear how others will use them not understanding their limitations.
- not2b 3y ago
  Except that search engine providers used to think they were in the business of providing accurate, quality information in exchange for ad revenue (before they took one step after another to degrade this in pursuit of revenue growth). People judge how accurate an answer is by doing searches, hoping to find reputable information to check an assertion. If all they can find is confident-sounding crap, how good is their judgment going to be?
- astrange 3y ago
  Bard is much less capable than GPT4, if you're only using the free tier. What I notice about Bard is that it hallucinates almost anytime I ask it anything. "Compare and contrast <two things that don't exist>" is a good one.
- tyre 3y ago
  > one of the things you have to judge is how accurate the answer is. The LLM can't assess that Not quite true. Some LLMs are surprisingly good at predicting their confidence in answers (where a series of 80% confidence output should end up being right 80% of the time.) “Language Models (Mostly) Know What They Know” by Anthropic: https://www.anthropic.com/index/language-models-mostly-know-what-they-know https://www.anthropic.com/index/language-models-mostly-know-...
a_cardboard_box 3y ago
I wonder if it learned that if someone says something dumb, they'll probably keep saying more dumb things. It's trained to predict what comes next, not what is correct. So what comes next after "the sky is yellow"? "The sky is purple", of course!
- jprete 3y ago
  It learned that if someone on the Internet says X, and someone else says Y, then usually the first person keeps saying X. The first person usually doesn’t admit fault if they discover that they’re wrong, they just go silent.
olalonde 3y ago
> given the over-abundant "AI will fix everything" narrative currently dominating the hn discourse. It seems we are not reading the same HN.
- jahsome 3y ago
  True, it's died down a bit in recent weeks. But it's still taken crypto's place of the mind share.

NovaDudely 3y ago

There is a kind of charm to AI Hallucinations. To confidently talk nonsense.

startupsfail 3y ago

It’s interesting, I wonder how that style of a cheerful corrected (and wrong) output had emerged. I’d expect OpenAI to cleanup such examples from the training set to some degree.

wizzwizz4 3y ago
I assumed OpenAI had added that themselves.
- startupsfail 3y ago
  They’ve added incorrect answers?
  - wizzwizz4 3y ago
    No; those were there already. I assumed they'd added the "does not double down" behaviour. Bing Chat (also an OpenAI model, but presumably with a different set of fine-tuning / filters to ChatGPT – possibly the bare OpenAI API) doubles down in situations like this: https://nitter.dark.fail/_akhaliq/status/1672267392280571905 https://nitter.dark.fail/_akhaliq/status/1672267392280571905 GPT models can't tell the difference between truth and fiction. All you can choose by fine-tuning is their threshold for "admitting" mistakes.
    - startupsfail 3y ago
      I do think it can tell the difference between truth and fiction. It’s a world model, it includes models of concepts like truth and fiction and can apply these concepts. But that’s beyond the point. The question is, how would you include incorrect responses into the training. In a way that it would not increase the probability of the model to give an incorrect response? I guess you can maybe train with a mix of correct and incorrect responses, hallucinations and nonsense in the conversation, but then make clear that the responses were incorrect, adding context to these. And then fine-tune the AI actor to avoid giving incorrect responses or hallucinations altogether.

IIAOPSW 3y ago

Try pointing out its mistakes when it hasn't made a mistake. You'll usually get the same "I'm sorry, here's the correct answer" output.

eterm 3y ago
And getting it to derive it's mistake doesn't always work (NB: This is GPT3 & bard, GPT4 is much better): https://richardcocks.github.io/prime.html https://richardcocks.github.io/prime.html

taylodl 3y ago

LLMs are incredible tools, but they're just that - tools. They're not a replacement for human judgment. Heck, one of the things you have to judge is how accurate the answer is. The LLM can't assess that - as you note it has unfailing confidence in reporting the wrong information. What I have found, and I've been using Bard over ChatGPT because Bard seems to be a bit smarter at first glance, is these tools are powerful but limited. They can augment a workforce but only a fool, soon to go out of business, would use them to replace a workforce.

NovaDudely 3y ago
This is just it, I don't fear LLM's - that are a really cool technology that once you understand their utility are really cool. I fear how others will use them not understanding their limitations.
not2b 3y ago
Except that search engine providers used to think they were in the business of providing accurate, quality information in exchange for ad revenue (before they took one step after another to degrade this in pursuit of revenue growth). People judge how accurate an answer is by doing searches, hoping to find reputable information to check an assertion. If all they can find is confident-sounding crap, how good is their judgment going to be?
astrange 3y ago
Bard is much less capable than GPT4, if you're only using the free tier. What I notice about Bard is that it hallucinates almost anytime I ask it anything. "Compare and contrast <two things that don't exist>" is a good one.
tyre 3y ago
> one of the things you have to judge is how accurate the answer is. The LLM can't assess that Not quite true. Some LLMs are surprisingly good at predicting their confidence in answers (where a series of 80% confidence output should end up being right 80% of the time.) “Language Models (Mostly) Know What They Know” by Anthropic: https://www.anthropic.com/index/language-models-mostly-know-what-they-know https://www.anthropic.com/index/language-models-mostly-know-...

a_cardboard_box 3y ago

I wonder if it learned that if someone says something dumb, they'll probably keep saying more dumb things. It's trained to predict what comes next, not what is correct. So what comes next after "the sky is yellow"? "The sky is purple", of course!

jprete 3y ago
It learned that if someone on the Internet says X, and someone else says Y, then usually the first person keeps saying X. The first person usually doesn’t admit fault if they discover that they’re wrong, they just go silent.

olalonde 3y ago

> given the over-abundant "AI will fix everything" narrative currently dominating the hn discourse. It seems we are not reading the same HN.

jahsome 3y ago
True, it's died down a bit in recent weeks. But it's still taken crypto's place of the mind share.