3 ms·

> It is convinced that it is always factually accurate, even though it is not. I don't think that's true. ChatGPT (or any LLM) isn't convinced much of anything

> It is convinced that it is always factually accurate, even though it is not. I don't think that's true. ChatGPT (or any LLM) isn't convinced much of anything. It might present something confidently (which is what most people want) but that's a side-effect of it's programming, not an indication of how good it feels on the answer. If you reply to anything ChatGPT says with "No, you're wrong." it will try to write a new, confident and satisfying answer that responds to your assertion. LLMs will always be "wrong" because they have no distinction between fiction and fact. Everything it reads is mapped into language, not concept space or an attitude or a worldview.

moonchrome 4y ago
>Everything it reads is mapped into language, not concept space Umm I'm pretty sure it's discovered concepts through compressing text - it seems perfectly capable of generalizing concepts
- groffee 4y ago
  [dead]
- musha68k 4y ago
  Would be interesting to read some unbiased science on that. My hunch would be that any concept it might detect still depends on 2nd order patterns acquired through training? The ultimate “book smart” at best, basically. I’m not a ML scientist though, again would be interesting to read an unbiased evaluation of those aspects. Edit: downvoted, not sure why though. Isn’t this a valid question a lot of non-ML folks would ask themselves?
- smoldesu 4y ago
  Text compression isn't a deterministic process, unfortunately. It's "concept" of compression is clearly derived from token sampling, in the same way it's concept of "math" is based on guessing the number/token that comes next. While I do agree that ChatGPT exhibits pattern-recognizing qualities, that's basically what it was built to do. I'm not arguing against emergent properties, just against emergent intelligence or even the idea of "understanding" in the first place.
- mdp2021 4y ago
  > it seems perfectly capable of generalizing concepts How would you support that perception?
  - firatsarlar 4y ago
    With hope and living? It is a dream come true for people. An abstract perception of a knowledge, is like sniffing a rose. It feels, yes, I get there. This 40.000 pages book, woow, I'll make time to live it or sniff another daisy?!
    - mdp2021 4y ago
      > It is a dream come true for people For as long as they will be an active part of societies, they'll better live in reality. We need to know what we are dealing with factually, because water is not exchangeable with iron in practice. The perception of a machine as being able «to generalize concepts» may be an illusion, so it requires some support: we want to avoid illusions for the contextual purposes.
      - firatsarlar 4y ago
        Let go , make the context further with words , yourself -society- Write books 1000 pages. Force child to read - lucky ones-, make things harder for everyone.
        mdp2021 4y ago
        But you are out of context. A box was created that outputs statements: either it has a «concept space» or just works through constructing plausible language. The poster claims leaning towards the "«concept space»" possibility - but this should not be a poll, and indications of "why" (such idea) are needed not to make the claim intellectually sterile. It is important otherwise the users of the box may cling to an illusion - that is not advisable. Words are there as a commodity to transfer intellectual states, a dynamic we need for practical and social achievements - otherwise, surrounded by indigence and unaware butchers, "«things [will be] harder for everyone»".
rcxdude 4y ago
There's been quite a few different iterations of ChatGPT and bing with different behaviours in this regard: it depends somewhat on the base GPT version, the fine-tuning, and the prompt. Bing very famously at one point was extremely passive aggressive when challenged on basically anything. And while there's nothing intrinsic to the structure and training goals of LLMs which directs them towards more structured reasoning, it does seem that in the process of learning to predict language they pick up some degree of it, however flawed it may be.
- seba_dos1 4y ago
  > Bing very famously at one point was extremely passive aggressive when challenged on basically anything. It still wasn't an indication of how confident it "felt" with its answers. It was just role-playing a more confident and aggressive chat bot than ChatGPT does.
  - int_19h 4y ago
    Outside of the roleplay, there's no "it". The thing that actually interacts with the user is the persona. But for the same reason, it doesn't matter what the underlying LLM "really thinks" - it'll be the persona that'll try to con you, or write broken or malicious code.

moonchrome 4y ago

>Everything it reads is mapped into language, not concept space Umm I'm pretty sure it's discovered concepts through compressing text - it seems perfectly capable of generalizing concepts

groffee 4y ago
[dead]
musha68k 4y ago
Would be interesting to read some unbiased science on that. My hunch would be that any concept it might detect still depends on 2nd order patterns acquired through training? The ultimate “book smart” at best, basically. I’m not a ML scientist though, again would be interesting to read an unbiased evaluation of those aspects. Edit: downvoted, not sure why though. Isn’t this a valid question a lot of non-ML folks would ask themselves?
smoldesu 4y ago
Text compression isn't a deterministic process, unfortunately. It's "concept" of compression is clearly derived from token sampling, in the same way it's concept of "math" is based on guessing the number/token that comes next. While I do agree that ChatGPT exhibits pattern-recognizing qualities, that's basically what it was built to do. I'm not arguing against emergent properties, just against emergent intelligence or even the idea of "understanding" in the first place.
mdp2021 4y ago
> it seems perfectly capable of generalizing concepts How would you support that perception?
- firatsarlar 4y ago
  With hope and living? It is a dream come true for people. An abstract perception of a knowledge, is like sniffing a rose. It feels, yes, I get there. This 40.000 pages book, woow, I'll make time to live it or sniff another daisy?!
  - mdp2021 4y ago
    > It is a dream come true for people For as long as they will be an active part of societies, they'll better live in reality. We need to know what we are dealing with factually, because water is not exchangeable with iron in practice. The perception of a machine as being able «to generalize concepts» may be an illusion, so it requires some support: we want to avoid illusions for the contextual purposes.
    - firatsarlar 4y ago
      Let go , make the context further with words , yourself -society- Write books 1000 pages. Force child to read - lucky ones-, make things harder for everyone.
      - mdp2021 4y ago
        But you are out of context. A box was created that outputs statements: either it has a «concept space» or just works through constructing plausible language. The poster claims leaning towards the "«concept space»" possibility - but this should not be a poll, and indications of "why" (such idea) are needed not to make the claim intellectually sterile. It is important otherwise the users of the box may cling to an illusion - that is not advisable. Words are there as a commodity to transfer intellectual states, a dynamic we need for practical and social achievements - otherwise, surrounded by indigence and unaware butchers, "«things [will be] harder for everyone»".

rcxdude 4y ago

There's been quite a few different iterations of ChatGPT and bing with different behaviours in this regard: it depends somewhat on the base GPT version, the fine-tuning, and the prompt. Bing very famously at one point was extremely passive aggressive when challenged on basically anything. And while there's nothing intrinsic to the structure and training goals of LLMs which directs them towards more structured reasoning, it does seem that in the process of learning to predict language they pick up some degree of it, however flawed it may be.

seba_dos1 4y ago
> Bing very famously at one point was extremely passive aggressive when challenged on basically anything. It still wasn't an indication of how confident it "felt" with its answers. It was just role-playing a more confident and aggressive chat bot than ChatGPT does.
- int_19h 4y ago
  Outside of the roleplay, there's no "it". The thing that actually interacts with the user is the persona. But for the same reason, it doesn't matter what the underlying LLM "really thinks" - it'll be the persona that'll try to con you, or write broken or malicious code.