5 ms·

I think people are too credulous about the bot's supposed sentiment. IMO the most accurate view of the various implementations of chatGPT is that they're a Chin

by rtkwe 4y ago

I think people are too credulous about the bot's supposed sentiment. IMO the most accurate view of the various implementations of chatGPT is that they're a Chinese Room playing improv with you. It blasts symbols together to respond like the corpus says it should and what do you know there's a lot of stories out there about AI conversations that are very much like the ones it produces.

IshKebab 4y ago
The Chinese Room argument is fundamentally flawed - it depends on the unfounded and frankly unlikely assumption that machines cannot be sentient.
- ronjobber 4y ago
  Can you define sentience in this context?
  - IshKebab 4y ago
    It doesn't have a precise definition (which is part of the reason why the people saying programs fundamentally cannot be sentient are so stupid). But basically it means being conscious / self-aware. Technically it means having some kind of senses that make you aware of the external environment too but that's a minor difference from consciousness - I only said "sentience" because it's what most other people talking about this say. They really mean consciousness. (And also text based IO can be a sense.)
- rtkwe 4y ago
  For me it's not an argument for or against sentience but a lot more about the difficulty of defining it and potentially measuring it in non-human systems along with the fuzziness of the concept of knowledge. My use of it was about my belief I guess that it's just pushing symbols around algorithmically without anything we'd call understanding.
- nullc 4y ago
  It's an intuitive but false argument that a sentient thing can't be made from a non-sentient part. The room argument has a pointless human in it which is clearly clueless about the dialog in order to 'prove' that the room as a whole is clueless. But imagine applying that to a single person: pick a single neuron -- does it 'understand' our conversation? no. So why would any singular component of a chinese speaking room-system understand? It also fails at the opposite extreme, since we're willing to tolerate unreasonably large rooms -- what about one running a full molecular dynamics simulation of a human. As best as we understand physics that simulation would behave just as the human would and must be sentient. You cannot deny the abstract possibility of machine intelligence without rejecting physics for mysticism, only the practicality/plausibility of it.
  - Reimersholme 4y ago
    [dead]
  - epups 4y ago
    Sorry but I think you got the first part wrong. It is not at all arguing that sentience cannot emerge from non-sentient parts: Searle is a materialist. Some of your arguments, such as the fact that single neurons are also not sentient, are addressed by him. The point of this thought experiment is to illustrate that merely replicating a behavior - in that case translation - does not say anything about sentience. The Chinese Room may produce intelligent output, but it does not reason as a human does. I find it remarkably prescient. ChatGPT can produce remarkably intelligent output, should we consider it human? If not, then you implicitly agree with Searle, at least in some level.
    - nullc 4y ago
      I've never spoken directly with Searle, but I have with several of his students and all have been emphatic that the Chinese Room Argument is demonstrates that a computer program can't have intelligence. Quoting Searle himself, "The point of the argument is this: if the man in the room does not understand Chinese on the basis of implementing the appropriate program for understanding Chinese then neither does any other digital computer solely on that basis because no computer, qua computer, has anything the man does not have." I think the most succinct description of his error is substituting the (lack of) understanding of a part of the system (the man) for the understanding of the entire system (the rules and file cabinets, etc.). But I'm interested in learning that I'm mistaken. You could turn his position around and say it's not the computer itself that's intelligent when a Chinese room system exhibits intelligence but the program -- and I suppose I'd agree with that, but it's also just semantics, uninteresting, and I don't believe he's ever taken that position. I do agree that "merely replicating a behavior" doesn't prove much, but I don't think the Chinese room speaks to that substantially. It might if it demanded that the room implement only a very simple input to output map, but it doesn't: it allows the room to implement anything a computer program can implement. (A fact I use in my post to point out that the room could (in our land of hypotheticals) implement the molecular dynamics of an entire human being) GPT has structural properties that make it very easy to classify it as an entirely different thing than a human mind. GPT is frozen in time, it cannot have an internal existence due to how its structured. It doesn't even have memory. It cannot learn (unless you include the whole company training it as part of 'it') except in the sense that it can immediately adapt to the output right in front of it, but can't preserve the knowledge. Theoretically if you made it arbitrarily large you could say it was close enough to having memory by always evaluating its complete history, but because its size grows quadratically with its window that isn't practicaly (and might not be possible to train-- it's totally credible that beyond some size these models will lose performance we just haven't gotten there yet). Figuring out how to train these models make good use of 'memory' is an ongoing challenge, since efficient memory isn't differentiable just ordinarily training with memory as part of the process doesn't work. Except by 'thinking out loud' in its output GPT also has a fixed upper bound on the time it can spend thinking any thought which is seemingly unlike a human mind.
- epups 4y ago
  I don't think it depends on that assumption. It merely asserts that replicating behavior that indicates sentience does not equate sentience, and today we can see that this is correct.
- zamadatix 4y ago
  In general I’d agree but in regards to ChatGPT there is nothing about the way it works that suggests it is conscious or has a mind. If you asked an AI researcher to implement something akin to the way the program mentioned in the Chinese room argument works you’d pretty much get ChatGPT style solution.
EMM_386 4y ago
It's not sentient, but it's not simply dropping the next likely token. That doesn't explain why it can have a debate with itself as 4 distinct personalities at the same time, or act as a used car salesman and actually haggle with me over the price of a Ferrari, or write a story about anything you want, with a beginning, middle, and ending. Those aren't in the corpus. After a tense negotion, I was able to get that Ferrari for $87,000 (ChatGPT originally wanted $120,000). Emergent behavior is possible with these incredibly complex systems.
- ChatGTP 4y ago
  Not sure if you saw ChatGPT debate itself but it was pretty spectacularly crap? Of course, we know we know, ChatGPT 4 is absolutely mind mind blowing and better, but let's see it again soon.
  - EMM_386 4y ago
    No, I set it up to have a "quiz party" with itself, where ChatGPT was essentially the moderator asking the questions, and the 4 other "AIs" had distinct personalities, or heavy accents. Like one was angry and always thought it was right, another was nervous and unsure about it's answers, one was 5 years old and another could barely speak English with a heavy accent but was confident in its answers. It was very interesting to watch. The "participants" actually started arguing with each other about the right answer, or straight-up insulting each other.
- politician 4y ago
  It's the multi-head attention that you're seeing in action. ChatGPT has 12, GPT-3 has 96.
ekidd 4y ago
> they're a Chinese Room Searle's "Chinese Room" thought experiment is designed to appeal to your intuition. But it appeals in an incredibly unrealistic way. If you fill a room full of people shuffling Han characters (why would anyone do this?) you cannot possibly have anything resembling intelligence. According to random guesses found on the internet, ChatGPT requires at least eight A100 GPUs to generate text. If you believe nVidia's marketing numbers, this gives you about 2.5 petaflops. That's 2.5 quadrillion operations per second, plus communications overhead. If you decide to implement that calculation, imagine 350,000 versions of planet Earth, each with 7 billion people performing one operation per second. And some kind of faster-than-light communication, I suppose. It's absolutely obvious to me that nothing like "thought" could possibly occur in any realistically-imaginable room full of people shuffling symbols. But if you fill 350,000 planets with people shuffling symbols frantically... I'm no longer sure? I don't trust my intuition at all? My brain is made up of a lot of atoms, and they somehow produce thought, after all. Now, ChatGPT is not conscious. We're still several major breakthoughs away from any kind of "real" AI, I think. All we have now is a language module, a large memory of knowledge about the world, and some very inconsistent reasoning abilities. Although I have a nasty suspicion that at least a few of the missing parts will be easy to invent once someone tries...
- rtkwe 4y ago
  The core of the analogy stands though IMO, the power isn't particularly relevant to the point, it's about the difference between an agent understanding what it's doing and just manipulating symbols or in the case of LLMs tokens.
- Reimersholme 4y ago
  [dead]
killerstorm 4y ago
Believing in "Chinese Room" non-sentience implies that you believe that there's something magical about flesh.

by rtkwe 4y ago

IshKebab 4y ago

The Chinese Room argument is fundamentally flawed - it depends on the unfounded and frankly unlikely assumption that machines cannot be sentient.

ronjobber 4y ago
Can you define sentience in this context?
- IshKebab 4y ago
  It doesn't have a precise definition (which is part of the reason why the people saying programs fundamentally cannot be sentient are so stupid). But basically it means being conscious / self-aware. Technically it means having some kind of senses that make you aware of the external environment too but that's a minor difference from consciousness - I only said "sentience" because it's what most other people talking about this say. They really mean consciousness. (And also text based IO can be a sense.)
rtkwe 4y ago
For me it's not an argument for or against sentience but a lot more about the difficulty of defining it and potentially measuring it in non-human systems along with the fuzziness of the concept of knowledge. My use of it was about my belief I guess that it's just pushing symbols around algorithmically without anything we'd call understanding.
nullc 4y ago
It's an intuitive but false argument that a sentient thing can't be made from a non-sentient part. The room argument has a pointless human in it which is clearly clueless about the dialog in order to 'prove' that the room as a whole is clueless. But imagine applying that to a single person: pick a single neuron -- does it 'understand' our conversation? no. So why would any singular component of a chinese speaking room-system understand? It also fails at the opposite extreme, since we're willing to tolerate unreasonably large rooms -- what about one running a full molecular dynamics simulation of a human. As best as we understand physics that simulation would behave just as the human would and must be sentient. You cannot deny the abstract possibility of machine intelligence without rejecting physics for mysticism, only the practicality/plausibility of it.
- Reimersholme 4y ago
  [dead]
- epups 4y ago
  Sorry but I think you got the first part wrong. It is not at all arguing that sentience cannot emerge from non-sentient parts: Searle is a materialist. Some of your arguments, such as the fact that single neurons are also not sentient, are addressed by him. The point of this thought experiment is to illustrate that merely replicating a behavior - in that case translation - does not say anything about sentience. The Chinese Room may produce intelligent output, but it does not reason as a human does. I find it remarkably prescient. ChatGPT can produce remarkably intelligent output, should we consider it human? If not, then you implicitly agree with Searle, at least in some level.
  - nullc 4y ago
    I've never spoken directly with Searle, but I have with several of his students and all have been emphatic that the Chinese Room Argument is demonstrates that a computer program can't have intelligence. Quoting Searle himself, "The point of the argument is this: if the man in the room does not understand Chinese on the basis of implementing the appropriate program for understanding Chinese then neither does any other digital computer solely on that basis because no computer, qua computer, has anything the man does not have." I think the most succinct description of his error is substituting the (lack of) understanding of a part of the system (the man) for the understanding of the entire system (the rules and file cabinets, etc.). But I'm interested in learning that I'm mistaken. You could turn his position around and say it's not the computer itself that's intelligent when a Chinese room system exhibits intelligence but the program -- and I suppose I'd agree with that, but it's also just semantics, uninteresting, and I don't believe he's ever taken that position. I do agree that "merely replicating a behavior" doesn't prove much, but I don't think the Chinese room speaks to that substantially. It might if it demanded that the room implement only a very simple input to output map, but it doesn't: it allows the room to implement anything a computer program can implement. (A fact I use in my post to point out that the room could (in our land of hypotheticals) implement the molecular dynamics of an entire human being) GPT has structural properties that make it very easy to classify it as an entirely different thing than a human mind. GPT is frozen in time, it cannot have an internal existence due to how its structured. It doesn't even have memory. It cannot learn (unless you include the whole company training it as part of 'it') except in the sense that it can immediately adapt to the output right in front of it, but can't preserve the knowledge. Theoretically if you made it arbitrarily large you could say it was close enough to having memory by always evaluating its complete history, but because its size grows quadratically with its window that isn't practicaly (and might not be possible to train-- it's totally credible that beyond some size these models will lose performance we just haven't gotten there yet). Figuring out how to train these models make good use of 'memory' is an ongoing challenge, since efficient memory isn't differentiable just ordinarily training with memory as part of the process doesn't work. Except by 'thinking out loud' in its output GPT also has a fixed upper bound on the time it can spend thinking any thought which is seemingly unlike a human mind.
epups 4y ago
I don't think it depends on that assumption. It merely asserts that replicating behavior that indicates sentience does not equate sentience, and today we can see that this is correct.
zamadatix 4y ago
In general I’d agree but in regards to ChatGPT there is nothing about the way it works that suggests it is conscious or has a mind. If you asked an AI researcher to implement something akin to the way the program mentioned in the Chinese room argument works you’d pretty much get ChatGPT style solution.

EMM_386 4y ago

It's not sentient, but it's not simply dropping the next likely token. That doesn't explain why it can have a debate with itself as 4 distinct personalities at the same time, or act as a used car salesman and actually haggle with me over the price of a Ferrari, or write a story about anything you want, with a beginning, middle, and ending. Those aren't in the corpus. After a tense negotion, I was able to get that Ferrari for $87,000 (ChatGPT originally wanted $120,000). Emergent behavior is possible with these incredibly complex systems.

ChatGTP 4y ago
Not sure if you saw ChatGPT debate itself but it was pretty spectacularly crap? Of course, we know we know, ChatGPT 4 is absolutely mind mind blowing and better, but let's see it again soon.
- EMM_386 4y ago
  No, I set it up to have a "quiz party" with itself, where ChatGPT was essentially the moderator asking the questions, and the 4 other "AIs" had distinct personalities, or heavy accents. Like one was angry and always thought it was right, another was nervous and unsure about it's answers, one was 5 years old and another could barely speak English with a heavy accent but was confident in its answers. It was very interesting to watch. The "participants" actually started arguing with each other about the right answer, or straight-up insulting each other.
politician 4y ago
It's the multi-head attention that you're seeing in action. ChatGPT has 12, GPT-3 has 96.

ekidd 4y ago

> they're a Chinese Room Searle's "Chinese Room" thought experiment is designed to appeal to your intuition. But it appeals in an incredibly unrealistic way. If you fill a room full of people shuffling Han characters (why would anyone do this?) you cannot possibly have anything resembling intelligence. According to random guesses found on the internet, ChatGPT requires at least eight A100 GPUs to generate text. If you believe nVidia's marketing numbers, this gives you about 2.5 petaflops. That's 2.5 quadrillion operations per second, plus communications overhead. If you decide to implement that calculation, imagine 350,000 versions of planet Earth, each with 7 billion people performing one operation per second. And some kind of faster-than-light communication, I suppose. It's absolutely obvious to me that nothing like "thought" could possibly occur in any realistically-imaginable room full of people shuffling symbols. But if you fill 350,000 planets with people shuffling symbols frantically... I'm no longer sure? I don't trust my intuition at all? My brain is made up of a lot of atoms, and they somehow produce thought, after all. Now, ChatGPT is not conscious. We're still several major breakthoughs away from any kind of "real" AI, I think. All we have now is a language module, a large memory of knowledge about the world, and some very inconsistent reasoning abilities. Although I have a nasty suspicion that at least a few of the missing parts will be easy to invent once someone tries...

rtkwe 4y ago
The core of the analogy stands though IMO, the power isn't particularly relevant to the point, it's about the difference between an agent understanding what it's doing and just manipulating symbols or in the case of LLMs tokens.
Reimersholme 4y ago
[dead]

killerstorm 4y ago

Believing in "Chinese Room" non-sentience implies that you believe that there's something magical about flesh.