5 ms·

ChapGPT Hallucinates

11 points by mpweiher 4y ago

femboy 4y ago
You reckon?
_def 4y ago
I keep hearing people who are surprised and upset about this - wasn't this fact clear from the beginning?
calderknight 4y ago
I assume the title of this blog is a typo. Anyway, I'm not sure how interesting a blog post using an out-dated version of the technology, and telling us something we've known from day one is.
brookst 4y ago
I wonder if back in the day there were breathless articles with headlines like “software has bugs!!!”
ec109685 4y ago
Asking it to summarize fake news urls where you give it a made up title as part of the seo url is simplest way to see it hallucinate. It’s unreliableness is exhausting since you are always doubting its output.
MattRix 4y ago
The title has a typo
bsaul 4y ago
it's going to be interesting to see if with all the sudden hype and income around llm people will manage to let them acquire a sense of "facts". From the gpt4 paper it looks like they already managed to get some interesting improvements, but it's quite likely some fundamental mechanism remains to be added..
hdhdhsjsbdh 4y ago
This whole category of blog post where people ask ChatGPT about themselves and then act surprised when it gives inaccurate responses is really uncompelling. The purpose of this technology is to grasp and synthesize information on general topics of use to everyone – science, engineering, literature, language/semantic competency, etc – not to have godlike knowledge of every person who ever existed. As a feature, the only person who would expect or want ChatGPT to dispense accurate information about some random blog poster is said blog poster.
- calderknight 4y ago
  He'd have a lot more luck with Bing Chat.
- deleted 4y ago
  [deleted]
upghost 4y ago
The article makes valid points about the danger of trusting an LLM as gospel. However, I am concerned about the anthropomorphic language. The LLM is only “hallucinating” if we define “hallucinating” to mean generating text that is syntactically correct and has the correct semantic shape but the text as interpreted by the reader is taken to be factually incorrect. Fundamentally this is a reader discernment issue —- which is why, in my opinion, using anthropomorphic language to describe the interaction with LLMs or the interpretation of the generated text is problematic. The imprecise description muddies our understanding of the tool to the point that it implies agency — thereby removing agency in the form of discretion from the user.
- MollyRealized 4y ago
  It's OpenAI's term: https://openai.com/research/webgpt https://openai.com/research/webgpt
elif 4y ago
I saw a similar report that it makes up academic papers, and so since then I have been fact checking when I use it to find research.. and every paper it has cited to me has existed. Perhaps it is a question of how relevant the subject you are asking about is? For instance the example that made me dubious was about eating banana peels for medical purposes, quite esoteric and not able to be confirmed by any outside sources. In this case, it seems GPT is familiar with the blog but has fuzzy details that are not corroborated by any other sources. It is a sort of dunning Kruger effect where the AI does not know the limits of it's limits but acts confidently regardless.
thej 4y ago
Updated the title. Add added the following note. Note: There was a typo in the title - Instead of ChatGPT, it was ChapGPT . I have corrected the title, but the URL remains the same because Cool URIs don't change
SAI_Peregrinus 4y ago
The Onion's recent "ChatGPT Starting To Think Journalist Could One Day Be Capable Of Independent Thought"[1] is a good reaction to these sorts of unsurprising "old news" articles repeating what other journalists have been reporting on repeatedly. [1] https://www.theonion.com/chatgpt-starting-to-think-journalist-could-one-day-be-c-1850233914 https://www.theonion.com/chatgpt-starting-to-think-journalis...
MollyRealized 4y ago
I'll concur with the piece, definitely. It seems to completely hallucinate when giving any reference link, or at least GPT-3 did. I don't know if GPT-4 continues the behavior. GPT-4 is underneath Bing Search, so if it does, that'd be bad news for MS.
seba_dos1 4y ago
Water is wet. That's what these models do. That's how they work. The fact that you can sometimes (often?) end up with factual information is a happy byproduct.

11 points by mpweiher 4y ago

femboy 4y ago

You reckon?

_def 4y ago

I keep hearing people who are surprised and upset about this - wasn't this fact clear from the beginning?

calderknight 4y ago

I assume the title of this blog is a typo. Anyway, I'm not sure how interesting a blog post using an out-dated version of the technology, and telling us something we've known from day one is.

brookst 4y ago

I wonder if back in the day there were breathless articles with headlines like “software has bugs!!!”

ec109685 4y ago

Asking it to summarize fake news urls where you give it a made up title as part of the seo url is simplest way to see it hallucinate. It’s unreliableness is exhausting since you are always doubting its output.

MattRix 4y ago

The title has a typo

bsaul 4y ago

it's going to be interesting to see if with all the sudden hype and income around llm people will manage to let them acquire a sense of "facts". From the gpt4 paper it looks like they already managed to get some interesting improvements, but it's quite likely some fundamental mechanism remains to be added..

hdhdhsjsbdh 4y ago

This whole category of blog post where people ask ChatGPT about themselves and then act surprised when it gives inaccurate responses is really uncompelling. The purpose of this technology is to grasp and synthesize information on general topics of use to everyone – science, engineering, literature, language/semantic competency, etc – not to have godlike knowledge of every person who ever existed. As a feature, the only person who would expect or want ChatGPT to dispense accurate information about some random blog poster is said blog poster.

calderknight 4y ago
He'd have a lot more luck with Bing Chat.
deleted 4y ago
[deleted]

upghost 4y ago

The article makes valid points about the danger of trusting an LLM as gospel. However, I am concerned about the anthropomorphic language. The LLM is only “hallucinating” if we define “hallucinating” to mean generating text that is syntactically correct and has the correct semantic shape but the text as interpreted by the reader is taken to be factually incorrect. Fundamentally this is a reader discernment issue —- which is why, in my opinion, using anthropomorphic language to describe the interaction with LLMs or the interpretation of the generated text is problematic. The imprecise description muddies our understanding of the tool to the point that it implies agency — thereby removing agency in the form of discretion from the user.

MollyRealized 4y ago
It's OpenAI's term: https://openai.com/research/webgpt https://openai.com/research/webgpt

elif 4y ago

I saw a similar report that it makes up academic papers, and so since then I have been fact checking when I use it to find research.. and every paper it has cited to me has existed. Perhaps it is a question of how relevant the subject you are asking about is? For instance the example that made me dubious was about eating banana peels for medical purposes, quite esoteric and not able to be confirmed by any outside sources. In this case, it seems GPT is familiar with the blog but has fuzzy details that are not corroborated by any other sources. It is a sort of dunning Kruger effect where the AI does not know the limits of it's limits but acts confidently regardless.

thej 4y ago

Updated the title. Add added the following note. Note: There was a typo in the title - Instead of ChatGPT, it was ChapGPT . I have corrected the title, but the URL remains the same because Cool URIs don't change

SAI_Peregrinus 4y ago

The Onion's recent "ChatGPT Starting To Think Journalist Could One Day Be Capable Of Independent Thought"[1] is a good reaction to these sorts of unsurprising "old news" articles repeating what other journalists have been reporting on repeatedly. [1] https://www.theonion.com/chatgpt-starting-to-think-journalist-could-one-day-be-c-1850233914 https://www.theonion.com/chatgpt-starting-to-think-journalis...

MollyRealized 4y ago

I'll concur with the piece, definitely. It seems to completely hallucinate when giving any reference link, or at least GPT-3 did. I don't know if GPT-4 continues the behavior. GPT-4 is underneath Bing Search, so if it does, that'd be bad news for MS.

seba_dos1 4y ago

Water is wet. That's what these models do. That's how they work. The fact that you can sometimes (often?) end up with factual information is a happy byproduct.