4 ms·

Searle’s room is less a moved goalpost and more a claim that the question is ill-posed. From very early days, the question of “what does it mean for a computer

Searle’s room is less a moved goalpost and more a claim that the question is ill-posed. From very early days, the question of “what does it mean for a computer program to have agency” has been asked. Usually poorly. Never with a satisfactory answer, at least to date. Perlisism 63: When we write programs that "learn", it turns out that we do and they don't.

FartyMcFarter 6y ago
> From very early days, the question of “what does it mean for a computer program to have agency” has been asked. Usually poorly. Never with a satisfactory answer, at least to date. If one asks this question, mustn't one also define what agency is? That seems like a hard problem in itself. Personally I don't see the problem with considering the computer as a black box and evaluating its intelligence (or lack of it) from that standpoint. After all, that is what us humans do with each other on a regular basis; for example, we evaluate humans as a black box in job interviews.
- hprotagonist 6y ago
  yes, agency is also a real pain to define. what does it mean for a thing to “want” to do something? https://www.cs.utexas.edu/users/EWD/transcriptions/EWD09xx/EWD936.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD09xx/E... is also an ever-present problem: when anthroomorphic language is appropriate and when it is not is something of an open question, but the prior should be towards avoiding it. Possibly particularly in systems we’ve built. You don’t have to go full-on radical behaviorist to be wary of this trend, (and tangentially i think behaviorism isn’t the be all and end all of explanations by a long shot) but once you’re aware of it you see it cropping up everywhere. Viruses don’t “want” to infect you, protein binding sites don’t “want” to bind amino acids, REST endpoints don’t “expect” a json datagram, GPT-3 doesn’t “know” anything and hasn’t “learned” anything, and so on. But we regularly speak of them as if that were the case. What category errors are we missing when we do this that could open new doors of understanding?
  - carapace 6y ago
    The solution has been present since forever in the old AI joke: 'AI is when the machine wakes up and asks, "What's in it for me?"'
JKCalhoun 6y ago
Searle's "Chinese Room" supposes that somehow what humans do is always something more than what an algorithm can do. Either the mythical Chinese book he describes is "intelligent" or perhaps humans are not as magical as we imagine we are. At least that was my take away. If he is suggesting somehow that we are not, at our core, machines then you might as well talk about the "soul".
- dleslie 6y ago
  I like to think of it as a question that marks a distinguishing level of intelligence: to have the capacity to formulate and pose the question within Searle's thought exercise is a benchmark itself; one that many humans are not able to achieve. Cutting edge AI is roughly infantile in its human-like ability, and rapidly entering toddlerhood. Given the tools and processes we have, can we expect it to progress to childhood soon?
- kbelder 6y ago
  Right. I always feel like, if it was a valid proof against AI, it also served as a proof against human intelligence. It just seems to take for granted that there is a fundamental (near-mystical) difference between the atoms in a brain and the atoms in a computer, even though that's a very extraordinary claim.
  - Barrin92 6y ago
    it's not an extraordinary claim at all. Digital computers and biological brains are not the same thing. Just like you can't build the Golden Gate Bridge out of jell-O, it's perfectly possible that you cannot replicate human intelligence in a completely different substrate. Physical properties matter is what he is saying. He's not denying that humans are intelligent, he's saying that biological intelligence is (likely) unique because of its configuration. The extraordinary claim is actually the almost dualistic view taken by AI people. That intelligence exists in some sort of cloud, distinct from matter or physical properties. It's almost a Descartes like, religious fantasy. Searle isn't denying that you can build AI, but that you can build any AI out of anything. He's saying that if you want to build human like intelligence, it is very likely that you need human like material. Just like when you build a house it actually matters what the house is structurally made of, it matters to an AI what it is made of. You can't make a house just out of some platonic mathematical idea of a house.

FartyMcFarter 6y ago

> From very early days, the question of “what does it mean for a computer program to have agency” has been asked. Usually poorly. Never with a satisfactory answer, at least to date. If one asks this question, mustn't one also define what agency is? That seems like a hard problem in itself. Personally I don't see the problem with considering the computer as a black box and evaluating its intelligence (or lack of it) from that standpoint. After all, that is what us humans do with each other on a regular basis; for example, we evaluate humans as a black box in job interviews.

hprotagonist 6y ago
yes, agency is also a real pain to define. what does it mean for a thing to “want” to do something? https://www.cs.utexas.edu/users/EWD/transcriptions/EWD09xx/EWD936.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD09xx/E... is also an ever-present problem: when anthroomorphic language is appropriate and when it is not is something of an open question, but the prior should be towards avoiding it. Possibly particularly in systems we’ve built. You don’t have to go full-on radical behaviorist to be wary of this trend, (and tangentially i think behaviorism isn’t the be all and end all of explanations by a long shot) but once you’re aware of it you see it cropping up everywhere. Viruses don’t “want” to infect you, protein binding sites don’t “want” to bind amino acids, REST endpoints don’t “expect” a json datagram, GPT-3 doesn’t “know” anything and hasn’t “learned” anything, and so on. But we regularly speak of them as if that were the case. What category errors are we missing when we do this that could open new doors of understanding?
- carapace 6y ago
  The solution has been present since forever in the old AI joke: 'AI is when the machine wakes up and asks, "What's in it for me?"'

JKCalhoun 6y ago

Searle's "Chinese Room" supposes that somehow what humans do is always something more than what an algorithm can do. Either the mythical Chinese book he describes is "intelligent" or perhaps humans are not as magical as we imagine we are. At least that was my take away. If he is suggesting somehow that we are not, at our core, machines then you might as well talk about the "soul".

dleslie 6y ago
I like to think of it as a question that marks a distinguishing level of intelligence: to have the capacity to formulate and pose the question within Searle's thought exercise is a benchmark itself; one that many humans are not able to achieve. Cutting edge AI is roughly infantile in its human-like ability, and rapidly entering toddlerhood. Given the tools and processes we have, can we expect it to progress to childhood soon?
kbelder 6y ago
Right. I always feel like, if it was a valid proof against AI, it also served as a proof against human intelligence. It just seems to take for granted that there is a fundamental (near-mystical) difference between the atoms in a brain and the atoms in a computer, even though that's a very extraordinary claim.
- Barrin92 6y ago
  it's not an extraordinary claim at all. Digital computers and biological brains are not the same thing. Just like you can't build the Golden Gate Bridge out of jell-O, it's perfectly possible that you cannot replicate human intelligence in a completely different substrate. Physical properties matter is what he is saying. He's not denying that humans are intelligent, he's saying that biological intelligence is (likely) unique because of its configuration. The extraordinary claim is actually the almost dualistic view taken by AI people. That intelligence exists in some sort of cloud, distinct from matter or physical properties. It's almost a Descartes like, religious fantasy. Searle isn't denying that you can build AI, but that you can build any AI out of anything. He's saying that if you want to build human like intelligence, it is very likely that you need human like material. Just like when you build a house it actually matters what the house is structurally made of, it matters to an AI what it is made of. You can't make a house just out of some platonic mathematical idea of a house.