5 ms·

It's all fun and games until the AI completely hallucinates the answer, while making it sound completely pausible and correct.

photochemsyn 3y ago
Well, sure, but in the context of the previous example I can now go look at poliastro (whatever that is, never heard of it before) documentation and do some more queries and so on. Certainly anyone just running LLM code output without doing a bunch of tests and checks is a lunatic.
paulddraper 3y ago
So....like a person?
- dirkt 3y ago
  The difference to a person is that most (though not all) people actually have an understanding if they know something, if they guess something, if they are making something up, or if they are outright lying. Which is about the first thing you train during a scientific education. And in an honest interaction, they will tell you. ChatGPT etc. does not. So basically it acts like a pathological liar (who happen to be right as long as it has been trained on something that comes close enough).
  - paulddraper 3y ago
    Yes, agreed.
  - deleted 3y ago
    [deleted]
  - cubefox 3y ago
    Animals (including humans) predict experiences, while language models only predict text. Text is not very closely linked to reality, while experiences are. So it is not surprising that we (humans) have a better sense of what we know than language models.
  - TeMPOraL 3y ago
    > The difference to a person is that most (though not all) people actually have an understanding if they know something, if they guess something, if they are making something up, or if they are outright lying. Which is about the first thing you train during a scientific education. Scientist? STEM workers? Maybe. People who value truth / consistent world model for its own sake? Sure. Normies? Not so much. It's not that they can't - I think they never learned to pay enough attention. What I mean is, most people tend to say things in confidence, and maybe even believe them, regardless of how they acquired the information. They don't seem to process the distinction between "I read it in a book", "A colleague told me that their colleague heard on the radio that...", etc. They don't even track provenance of the information, which is a critical skill you need to not end up confidently making things up. It's a learnable skill, and I think it's even quite easy to pick it up from osmosis - but someone has to make the person feel that it's important. > So basically it acts like a pathological liar I think it's more of a bullshitter than a liar, in the sense that it doesn't care about truth value of what it says. Lying involve knowing the truth, or at least knowing that the thing you're saying ain't it. Making a mistake involves you thinking you're saying the truth, but actually being wrong about it. Bullshitting is just saying whatever helps you achieve your goal; the truth value of what you say doesn't even enter the picture.
loandbehold 3y ago
The issue of hallucinations is overblown. I use GPT4 all the time and don't see any hallucinations at all. It's a big problem with Google BARD and GPT3 and earlier models. But GPT4 fixed the issue of hallucinations completely.
- danlugo92 3y ago
  You shouldn't call it overblown given GPT4 was released not even a month ago, but if it's fixed now then great.
- tjr 3y ago
  I wonder if it's "fixed" or if it's just less obvious. It seems that an LLM would "hallucinate" a bogus answer if it didn't actually have a good answer somewhere in its training. Is GPT4 so much more trained that it rarely encounters something it doesn't have a reasonable answer for? In which case, it would still "hallucinate" if cornered on some more obscure matter? I mean, like, what would it mean to actually solve the problem? I would not expect any computer system to know the answer to literally everything, so the fix is not to train it more, but rather, for it to realize and acknowledge if it doesn't have enough data to give a good answer, and tell you that rather than make something up. Does GPT4 do that? (I have yet to use GPT4 myself.)
  - sebzim4500 3y ago
    In my experience, GPT-4 is equally willing to make things up if it doesn't know something but it has so much more knowledge than GPT-3.5 that this happens less often in practice.
    - mustacheemperor 3y ago
      On that note, I’ve found that just including in the prompt a request for GPT4 to consider its confidence level in an answer and inform me of that confidence level, to reconsider its answer if its confidence is low, and that accuracy is critically important for the topic of the conversation, also can result in better steering it. I mean, kind of works with humans too. In a high pressure work or school environment, people can confabulate the answers someone wants to hear to avoid discomfort “oh yes, we did the training exercise, the trucks and tanks are in great shape.” Sometimes people need to know they can admit they are wrong. I wonder if some aspect of current LLM tuning/training could be modified so LLMs are more “comfortable”, for lack of a non-anthropomorphized term coming to mind, with admitting they are unsure.

photochemsyn 3y ago

Well, sure, but in the context of the previous example I can now go look at poliastro (whatever that is, never heard of it before) documentation and do some more queries and so on. Certainly anyone just running LLM code output without doing a bunch of tests and checks is a lunatic.

paulddraper 3y ago

So....like a person?

dirkt 3y ago
The difference to a person is that most (though not all) people actually have an understanding if they know something, if they guess something, if they are making something up, or if they are outright lying. Which is about the first thing you train during a scientific education. And in an honest interaction, they will tell you. ChatGPT etc. does not. So basically it acts like a pathological liar (who happen to be right as long as it has been trained on something that comes close enough).
- paulddraper 3y ago
  Yes, agreed.
- deleted 3y ago
  [deleted]
- cubefox 3y ago
  Animals (including humans) predict experiences, while language models only predict text. Text is not very closely linked to reality, while experiences are. So it is not surprising that we (humans) have a better sense of what we know than language models.
- TeMPOraL 3y ago
  > The difference to a person is that most (though not all) people actually have an understanding if they know something, if they guess something, if they are making something up, or if they are outright lying. Which is about the first thing you train during a scientific education. Scientist? STEM workers? Maybe. People who value truth / consistent world model for its own sake? Sure. Normies? Not so much. It's not that they can't - I think they never learned to pay enough attention. What I mean is, most people tend to say things in confidence, and maybe even believe them, regardless of how they acquired the information. They don't seem to process the distinction between "I read it in a book", "A colleague told me that their colleague heard on the radio that...", etc. They don't even track provenance of the information, which is a critical skill you need to not end up confidently making things up. It's a learnable skill, and I think it's even quite easy to pick it up from osmosis - but someone has to make the person feel that it's important. > So basically it acts like a pathological liar I think it's more of a bullshitter than a liar, in the sense that it doesn't care about truth value of what it says. Lying involve knowing the truth, or at least knowing that the thing you're saying ain't it. Making a mistake involves you thinking you're saying the truth, but actually being wrong about it. Bullshitting is just saying whatever helps you achieve your goal; the truth value of what you say doesn't even enter the picture.

loandbehold 3y ago

The issue of hallucinations is overblown. I use GPT4 all the time and don't see any hallucinations at all. It's a big problem with Google BARD and GPT3 and earlier models. But GPT4 fixed the issue of hallucinations completely.

danlugo92 3y ago
You shouldn't call it overblown given GPT4 was released not even a month ago, but if it's fixed now then great.
tjr 3y ago
I wonder if it's "fixed" or if it's just less obvious. It seems that an LLM would "hallucinate" a bogus answer if it didn't actually have a good answer somewhere in its training. Is GPT4 so much more trained that it rarely encounters something it doesn't have a reasonable answer for? In which case, it would still "hallucinate" if cornered on some more obscure matter? I mean, like, what would it mean to actually solve the problem? I would not expect any computer system to know the answer to literally everything, so the fix is not to train it more, but rather, for it to realize and acknowledge if it doesn't have enough data to give a good answer, and tell you that rather than make something up. Does GPT4 do that? (I have yet to use GPT4 myself.)
- sebzim4500 3y ago
  In my experience, GPT-4 is equally willing to make things up if it doesn't know something but it has so much more knowledge than GPT-3.5 that this happens less often in practice.
  - mustacheemperor 3y ago
    On that note, I’ve found that just including in the prompt a request for GPT4 to consider its confidence level in an answer and inform me of that confidence level, to reconsider its answer if its confidence is low, and that accuracy is critically important for the topic of the conversation, also can result in better steering it. I mean, kind of works with humans too. In a high pressure work or school environment, people can confabulate the answers someone wants to hear to avoid discomfort “oh yes, we did the training exercise, the trucks and tanks are in great shape.” Sometimes people need to know they can admit they are wrong. I wonder if some aspect of current LLM tuning/training could be modified so LLMs are more “comfortable”, for lack of a non-anthropomorphized term coming to mind, with admitting they are unsure.