3 ms·

I'm guessing whether you believe it possesses intelligence or not depends on your answer to Searle's Chinese room thought experiment[0]. I'd also recommend chec

I'm guessing whether you believe it possesses intelligence or not depends on your answer to Searle's Chinese room thought experiment[0]. I'd also recommend checking out the Peter Watts' book, Blindsight. [0] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room

staticman2 1mo ago
This comment thread was started with discussions of AI doing a bad job at a task (communication). Doesn't the Chinese Room posit an AI good at the task of communication?
- rcxdude 1mo ago
  The Chinese Room mainly just posits a room that passes the Turing Test, which LLMs do pretty well outside of outright adversarial situations.
  - codethief 1mo ago
    Do they? https://longbets.org/1/ https://longbets.org/1/ has yet to be settled. Either way, I doubt an LLM could fool anyone here who who knows how LLMs work into thinking it is human, at least not for an extended period of time (think about context length/compression, prompt injections, …).
    - idiotsecant 1mo ago
      You'll notice those goalposts are substantially stretched from the original test.
      - codethief 1mo ago
        How so?
- jay_kyburz 1mo ago
  AIs are better communicators that most of people I have worked with in my life. They are infinitely patient, don't mind going into more detail if I ask, not too bad at summary, have no ego and don't boast. They are also not too afraid of hurting my feelings, they will tell me my code sux if it does. I'd don't care if they fit a definition intelligent, they are good colleagues. They have strengths and weaknesses sure, but so do people.
  - tripzilch 1mo ago
    > have no ego and don't boast Yes they do, and they famously do it quite a lot. > they will tell me my code sux if it does If they knew when code sux, someone should write an agentic loop around that.
rcxdude 1mo ago
The Chinese room is a good Rorschach test for this kind of thing (but not a good thought experiment, IMO, because it's obviously correct or obviously wrong depending on where you're already coming from), but also it's not really about intelligence per se, but more abstractly awareness and more adjacent to consciousness than intelligence, and these are not the same thing (though it does seem like a lot of people have conflated them somehow, from the conversations around AI).
- prophesi 1mo ago
  It all comes down to semantics. And yeah, with the thought experiment, Searle presents three axioms of what could constitute intelligence. Then proposes the chinese room thought experiment as an approval of his third rule, "Syntax by itself is neither constitutive of nor sufficient for semantics." Which only makes sense when you take the other two rules together. Oddly enough, defining intelligence with our capability to derive semantics. There are a lot of creative counter-arguments to look into on the thought experiment though.
  - rcxdude 1mo ago
    The thing is it doesn't really need creative counter-arguments, it's basically just assuming its conclusion. If you disagree with the conclusion, the argument is nonsense because Searle is just saying 'well the man doesn't understand, so nothing does', and if you agree with the conclusion you don't really need to jump through any hoops to get to it, the third rule is just obviously following from the first and second. And again it's not really talking about intelligence at all, Searle allows that the machine is intelligent from the start, he just rejects that it has any understanding of meaning.
    - prophesi 1mo ago
      Is that not one of the counter-arguments? You're disagreeing of what Searle determines to be "intelligence." We don't understand how consciousness works, what it's like to be a bat, or how this emergent property came to be. It's why it's necessary to define the terms, then prove them wrong or right. If you disagree with the definitions, that's fine. There is no agreement on what constitutes intelligence.

staticman2 1mo ago

This comment thread was started with discussions of AI doing a bad job at a task (communication). Doesn't the Chinese Room posit an AI good at the task of communication?

rcxdude 1mo ago
The Chinese Room mainly just posits a room that passes the Turing Test, which LLMs do pretty well outside of outright adversarial situations.
- codethief 1mo ago
  Do they? https://longbets.org/1/ https://longbets.org/1/ has yet to be settled. Either way, I doubt an LLM could fool anyone here who who knows how LLMs work into thinking it is human, at least not for an extended period of time (think about context length/compression, prompt injections, …).
  - idiotsecant 1mo ago
    You'll notice those goalposts are substantially stretched from the original test.
    - codethief 1mo ago
      How so?
jay_kyburz 1mo ago
AIs are better communicators that most of people I have worked with in my life. They are infinitely patient, don't mind going into more detail if I ask, not too bad at summary, have no ego and don't boast. They are also not too afraid of hurting my feelings, they will tell me my code sux if it does. I'd don't care if they fit a definition intelligent, they are good colleagues. They have strengths and weaknesses sure, but so do people.
- tripzilch 1mo ago
  > have no ego and don't boast Yes they do, and they famously do it quite a lot. > they will tell me my code sux if it does If they knew when code sux, someone should write an agentic loop around that.

rcxdude 1mo ago

The Chinese room is a good Rorschach test for this kind of thing (but not a good thought experiment, IMO, because it's obviously correct or obviously wrong depending on where you're already coming from), but also it's not really about intelligence per se, but more abstractly awareness and more adjacent to consciousness than intelligence, and these are not the same thing (though it does seem like a lot of people have conflated them somehow, from the conversations around AI).

prophesi 1mo ago
It all comes down to semantics. And yeah, with the thought experiment, Searle presents three axioms of what could constitute intelligence. Then proposes the chinese room thought experiment as an approval of his third rule, "Syntax by itself is neither constitutive of nor sufficient for semantics." Which only makes sense when you take the other two rules together. Oddly enough, defining intelligence with our capability to derive semantics. There are a lot of creative counter-arguments to look into on the thought experiment though.
- rcxdude 1mo ago
  The thing is it doesn't really need creative counter-arguments, it's basically just assuming its conclusion. If you disagree with the conclusion, the argument is nonsense because Searle is just saying 'well the man doesn't understand, so nothing does', and if you agree with the conclusion you don't really need to jump through any hoops to get to it, the third rule is just obviously following from the first and second. And again it's not really talking about intelligence at all, Searle allows that the machine is intelligent from the start, he just rejects that it has any understanding of meaning.
  - prophesi 1mo ago
    Is that not one of the counter-arguments? You're disagreeing of what Searle determines to be "intelligence." We don't understand how consciousness works, what it's like to be a bat, or how this emergent property came to be. It's why it's necessary to define the terms, then prove them wrong or right. If you disagree with the definitions, that's fine. There is no agreement on what constitutes intelligence.