5 ms·

« I don't believe we've seen any evidence of this. » I think chatgpt did show proof of what was considered at least until very recently « intelligence ». aka:

by bsaul 3y ago

« I don't believe we've seen any evidence of this. » I think chatgpt did show proof of what was considered at least until very recently « intelligence ». aka: understand enough about context and concepts to provide relevant answers to very complex and open questions.

kumarvvr 3y ago
Is it understanding or, generating closest possible set of tokens that are accepted as a response to an received input of tokens?
- vidarh 3y ago
  How do we know those two are different?
  - kumarvvr 3y ago
    Understanding and assimilation can lead to generating relations between disjoint sets of tokens. For example, "squeeze a tube to cut water flow" and "put pressure on a deep wound to stop blood loss" can only be related, if not already in the training data, if there is understanding and intelligence. The ability to do that is intelligence. Otherwise, it's just a search and optimization problem.
    - vidarh 3y ago
      Firstly, transformer architectures are not "just a search and optimisation problem". They do generalise. Whether that generalisation is sufficient to be structurally equivalent to what we consider intelligence is an open question, but getting them to demonstrate that they generalise is easy (eg. ChatGPT can do math, albeit badly, with numbers large enough that it is infeasible for it to just have occurred in its training set) Secondly, this poses the problem of 1) finding examples like the one you gave that it can't understand (regarding your specific example, see [1]), 2) ruling out that there was something "too close" drawing the equivalence in the training data, 3) ruling out that the failure to draw the equivalence is something structural (it can't have real understanding) rather than qualitative (it has real understanding, but it just isn't smart enough to understand the specific given problem) So I'm back to my original question of how we would know if these are structurally different things in the first place. [1] vidarh: How does squeezing a tube to cut water flow give you a hint as to what to do about a deep wound? ChatGPT (GPT4): Squeezing a tube to cut off water flow demonstrates the basic principle of applying pressure to restrict or stop the flow of a fluid. Similarly, applying pressure to a deep wound can help control bleeding, which is a critical first aid measure when dealing with serious injuries. [followed by a long list of what to do if encountering a deep wound]
riffraff 3y ago
I think this is the culprit: ChatGPT does not understand, as proven by hallucinations, by being wrong on certain questions or puzzle etc.. It just seems to understand. This is useful, and deeply impressive, but it's not the same thing.
- vidarh 3y ago
  Humans are confidently wrong about all kinds of things all the time. We don't call it hallucination other than for very specific, limited subsets. We call each other names over it, call it mistakes, stupidity, or lies. In fact, we structure large parts of society around it (the presence of multiple contradictory world religions means the majority of the worlds population goes through life being confidently wrong). It's just being confidently wrong about some subjects is more socially acceptable than others. ChatGPT clearly needs reinforcement to know its limits the way humans are constantly reinforced from childhood that making confident claims about things we don't actually know well is often going to have negative consequences, but until we've seen the result of doing that, I don't think we have any foundation to say anything about whether ChatGPT's current willingness to make up answers means it's level of understanding is in any way fundamentally different from that of humans or not.
  - pilsetnieks 3y ago
    You seem to propose a "if it quacks like a duck" scenario here (aka the Chinese Room), and applying it to people who are mistaken or acting on incorrect information. Those people are reasoning, however rightly or wrongly, based on that wrong information. LLMs are not.
    - vidarh 3y ago
      We do not have sufficient knowledge of what reasoning entails to be able to tell whether there's any meaningful distinction between human reasoning and LLMs. It's likely there is some difference. It's not clear whether there's any major qualitative difference. Claiming we can tell that there's a distinction that merits saying people are reasoning and LLMs are not, is "hallucination" to me. It's making a claim there is insufficient evidence to make a reasoned statement about. EDIT: Ironically, on feeding ChatGPT (w/GPT4) my comment and your reply and asking it to "compose a reply on behalf of 'vidarh'" it produced a reply that was far more willing to accept your claim that there is a fundamental difference (while otherwise giving a reasonable reaffirmation of my argument that reinforcement of the boundaries of its knowledge would reduce its "hallucinations")

by bsaul 3y ago

kumarvvr 3y ago

Is it understanding or, generating closest possible set of tokens that are accepted as a response to an received input of tokens?

vidarh 3y ago
How do we know those two are different?
- kumarvvr 3y ago
  Understanding and assimilation can lead to generating relations between disjoint sets of tokens. For example, "squeeze a tube to cut water flow" and "put pressure on a deep wound to stop blood loss" can only be related, if not already in the training data, if there is understanding and intelligence. The ability to do that is intelligence. Otherwise, it's just a search and optimization problem.
  - vidarh 3y ago
    Firstly, transformer architectures are not "just a search and optimisation problem". They do generalise. Whether that generalisation is sufficient to be structurally equivalent to what we consider intelligence is an open question, but getting them to demonstrate that they generalise is easy (eg. ChatGPT can do math, albeit badly, with numbers large enough that it is infeasible for it to just have occurred in its training set) Secondly, this poses the problem of 1) finding examples like the one you gave that it can't understand (regarding your specific example, see [1]), 2) ruling out that there was something "too close" drawing the equivalence in the training data, 3) ruling out that the failure to draw the equivalence is something structural (it can't have real understanding) rather than qualitative (it has real understanding, but it just isn't smart enough to understand the specific given problem) So I'm back to my original question of how we would know if these are structurally different things in the first place. [1] vidarh: How does squeezing a tube to cut water flow give you a hint as to what to do about a deep wound? ChatGPT (GPT4): Squeezing a tube to cut off water flow demonstrates the basic principle of applying pressure to restrict or stop the flow of a fluid. Similarly, applying pressure to a deep wound can help control bleeding, which is a critical first aid measure when dealing with serious injuries. [followed by a long list of what to do if encountering a deep wound]

riffraff 3y ago

I think this is the culprit: ChatGPT does not understand, as proven by hallucinations, by being wrong on certain questions or puzzle etc.. It just seems to understand. This is useful, and deeply impressive, but it's not the same thing.

vidarh 3y ago
Humans are confidently wrong about all kinds of things all the time. We don't call it hallucination other than for very specific, limited subsets. We call each other names over it, call it mistakes, stupidity, or lies. In fact, we structure large parts of society around it (the presence of multiple contradictory world religions means the majority of the worlds population goes through life being confidently wrong). It's just being confidently wrong about some subjects is more socially acceptable than others. ChatGPT clearly needs reinforcement to know its limits the way humans are constantly reinforced from childhood that making confident claims about things we don't actually know well is often going to have negative consequences, but until we've seen the result of doing that, I don't think we have any foundation to say anything about whether ChatGPT's current willingness to make up answers means it's level of understanding is in any way fundamentally different from that of humans or not.
- pilsetnieks 3y ago
  You seem to propose a "if it quacks like a duck" scenario here (aka the Chinese Room), and applying it to people who are mistaken or acting on incorrect information. Those people are reasoning, however rightly or wrongly, based on that wrong information. LLMs are not.
  - vidarh 3y ago
    We do not have sufficient knowledge of what reasoning entails to be able to tell whether there's any meaningful distinction between human reasoning and LLMs. It's likely there is some difference. It's not clear whether there's any major qualitative difference. Claiming we can tell that there's a distinction that merits saying people are reasoning and LLMs are not, is "hallucination" to me. It's making a claim there is insufficient evidence to make a reasoned statement about. EDIT: Ironically, on feeding ChatGPT (w/GPT4) my comment and your reply and asking it to "compose a reply on behalf of 'vidarh'" it produced a reply that was far more willing to accept your claim that there is a fundamental difference (while otherwise giving a reasonable reaffirmation of my argument that reinforcement of the boundaries of its knowledge would reduce its "hallucinations")