4 ms·

Of course not. Hallucinations are the only things these models can produce, they have no mechanism to tell if what they're writing is a fact or not. They genera

by __loam 2y ago

Of course not. Hallucinations are the only things these models can produce, they have no mechanism to tell if what they're writing is a fact or not. They generate bullshit in the "on bullshit" by Frankfurt sense of the word.

ein0p 2y ago
Neither do humans. You write what you believe. It is intractable for you to confirm every fact you believe in
- __loam 2y ago
  [flagged]
- recursive 2y ago
  No one is claiming LLMs "believe" things. Well, maybe someone is.
  - ein0p 2y ago
    Belief is a posterior distribution over a combination of priors and inputs. That’s literally the only thing LLMs currently do.
    - recursive 2y ago
      It seems my dictionary must have a different entry for the word.
- seadan83 2y ago
  > It is intractable for you to confirm every fact you believe A 'fact' that is not confirmed is an opinion. A 'opinion' sincerely held to be true without data is called faith. I do think humans are quite bad at recognizing that most of what they think they know is opinion, and/or the 'facts' are often based on personal experience (which is by definition a cherry-picked data set), and thus is also an opinion too. I think as well that those that practice science extensively are more practiced to really identify what is fact vs unsupported opinion. Without that practice, it's a lot easier IMO to then think a person knows a lot more facts than they really do. Which is to say, humans generally know very little, and it's not terribly comfortable to acknowledge and feel that way. Which, leads us to the ultimate reality. Most of what a lay person has to say would be opinion, it's not really worth much - and it's even worse when data no longer matters.
  - ein0p 2y ago
    My point exactly. The Homo Sapiens baseline is laughably poor in this regard.
    - seadan83 2y ago
      We agree on that point! Though, is the AI baseline even worse though? Computers are nothing more than a set of persisted electrical 0/1 signals and a series of logic gates with intricate timing. There is no 'facts' in that world. For example, "Chat GPT" does not understand questions, it does some pattern matching to find responses that probabilistic-ally would follow. Another example, AI does not understand it is drawing "a hand", it's just a probability algorithm that indicates these pixels are likely to be "good" following these other pixels. If someone can build a 'reasoning' step into AI - that would likely be a game changer. An AI that generates an answer, and then compares that answer against an actual fact-database; and then modifies that answer in light of the existing facts. Even better, one that can also challenge when an understood fact is likely to be wrong. To do all that, the AI would need to understand abstract concepts. To give a baseline of where we are on that, my cat is able to do that and AI is currently at zero capability to do it.
    - __loam 2y ago
      I'm not sure what it means but a lot of people on this forum have a strong tendency to denigrate human beings whenever people bring up that LLMs kind of suck. I think this undersells what humans are capable of and distracts from the discussion about what LLMs can do. I'm really tired of this discussion immediately getting sidetracked into some philosophical quagmire about the nature of thinking and the mind when someone is trying to bring up a point about LLMs themselves. When this Apple Intelligence feature comes out, I won't have a tiny human in my phone telling me things, I'll have a ChatGPT api call. LLMs are being integrated into these systems, not human beings. When someone talks about other systems, like search engines, nobody talks about how they compare to humans. It's weird that we're dead set on making the comparison with these systems in particular. Maybe that's down to poorly chosen names for things like artificial intelligence and neural networks inviting the comparison.

by __loam 2y ago

ein0p 2y ago

Neither do humans. You write what you believe. It is intractable for you to confirm every fact you believe in

__loam 2y ago
[flagged]
recursive 2y ago
No one is claiming LLMs "believe" things. Well, maybe someone is.
- ein0p 2y ago
  Belief is a posterior distribution over a combination of priors and inputs. That’s literally the only thing LLMs currently do.
  - recursive 2y ago
    It seems my dictionary must have a different entry for the word.
seadan83 2y ago
> It is intractable for you to confirm every fact you believe A 'fact' that is not confirmed is an opinion. A 'opinion' sincerely held to be true without data is called faith. I do think humans are quite bad at recognizing that most of what they think they know is opinion, and/or the 'facts' are often based on personal experience (which is by definition a cherry-picked data set), and thus is also an opinion too. I think as well that those that practice science extensively are more practiced to really identify what is fact vs unsupported opinion. Without that practice, it's a lot easier IMO to then think a person knows a lot more facts than they really do. Which is to say, humans generally know very little, and it's not terribly comfortable to acknowledge and feel that way. Which, leads us to the ultimate reality. Most of what a lay person has to say would be opinion, it's not really worth much - and it's even worse when data no longer matters.
- ein0p 2y ago
  My point exactly. The Homo Sapiens baseline is laughably poor in this regard.
  - seadan83 2y ago
    We agree on that point! Though, is the AI baseline even worse though? Computers are nothing more than a set of persisted electrical 0/1 signals and a series of logic gates with intricate timing. There is no 'facts' in that world. For example, "Chat GPT" does not understand questions, it does some pattern matching to find responses that probabilistic-ally would follow. Another example, AI does not understand it is drawing "a hand", it's just a probability algorithm that indicates these pixels are likely to be "good" following these other pixels. If someone can build a 'reasoning' step into AI - that would likely be a game changer. An AI that generates an answer, and then compares that answer against an actual fact-database; and then modifies that answer in light of the existing facts. Even better, one that can also challenge when an understood fact is likely to be wrong. To do all that, the AI would need to understand abstract concepts. To give a baseline of where we are on that, my cat is able to do that and AI is currently at zero capability to do it.
  - __loam 2y ago
    I'm not sure what it means but a lot of people on this forum have a strong tendency to denigrate human beings whenever people bring up that LLMs kind of suck. I think this undersells what humans are capable of and distracts from the discussion about what LLMs can do. I'm really tired of this discussion immediately getting sidetracked into some philosophical quagmire about the nature of thinking and the mind when someone is trying to bring up a point about LLMs themselves. When this Apple Intelligence feature comes out, I won't have a tiny human in my phone telling me things, I'll have a ChatGPT api call. LLMs are being integrated into these systems, not human beings. When someone talks about other systems, like search engines, nobody talks about how they compare to humans. It's weird that we're dead set on making the comparison with these systems in particular. Maybe that's down to poorly chosen names for things like artificial intelligence and neural networks inviting the comparison.