3 ms·

> (insert stochastic parrot/Chinese room nonsense here). Neither of these arguments are nonsense. If you want to argue against these arguments do so, but do so

> (insert stochastic parrot/Chinese room nonsense here). Neither of these arguments are nonsense. If you want to argue against these arguments do so, but do so convincingly without resorting to trivial attacks. The burden of proof is on those with extraordinary claims and the claim that AGI is already evinced by any number of models or that current implementations will lead to AGI are very extraordinary indeed.

madsbuch 3y ago
a proposition does not need to be proved nor falsified. It can exist as a hypothesis. I think that is what the parent commenter points to. I the current environment it is dangerous to be fixed in your opinion, at least until there are proper proofs of anything.
zingelshuher 3y ago
If I may, what is 'AGI' you are talking about? And how can we set up a test which will show that 'thing' isn't GI while humans are. I'm asking because there are different opinions, some believe AGI should reproduce itself, have self-preservation instincts, have a body. To me even long term memory isn't obviously required. This is 'creature' feature. Which is irrelevant to GI. (actually it's more complicated)
IshKebab 3y ago
You've misread. I didn't mention existing models. I agree that it is an extraordinary claim that existing models demonstrate anything like AGI or sentience. I said it was delusional that AI can never be sentient or self-aware. That is also an extraordinary claim that requires extraordinary evidence, and nobody has any. People usually mention stochastic parrots and the Chinese room, but their arguments are very obviously flawed: 1. Current gen AI may only seem like a stochastic parrot, but there's no reason to believe that human intelligence isn't just a really fancy stochastic parrot. 2. The Chinese room experiment just confuses things by anthropomorphizing a single part of the whole intelligent system (i.e. saying there's a man in the room). If you look at the room as a whole, then yes it does understand Chinese. Similarly if you go the other way and imagine each of the neurons in a Chinese person was a tiny man following simple instructions, does that mean the Chinese person no longer understands Chinese? No of course not.
- emptysongglass 3y ago
  > You've misread. I didn't mention existing models. No you didn't, and that's why I didn't limit my rebuttal to existing models alone. > 1. Current gen AI may only seem like a stochastic parrot, but there's no reason to believe that human intelligence isn't just a really fancy stochastic parrot. There is no reason to believe that human intelligence is a stochastic parrot. We have a large body of research to indicate emotion and thought are interdependent [1]. Cognition is not a simple, discrete quantity. > 2. The Chinese room experiment just confuses things by anthropomorphizing a single part of the whole intelligent system (i.e. saying there's a man in the room). If you look at the room as a whole, then yes it does understand Chinese. Similarly if you go the other way and imagine each of the neurons in a Chinese person was a tiny man following simple instructions, does that mean the Chinese person no longer understands Chinese? No of course not. It sounds like you are confusing things. The Chinese room argument is arguing intentionality, that it is a necessary prerequisite to real human cognition. We have zero evidence of any display of intention by these models. The rest of your argument here I honestly can't follow. [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2366118/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2366118/
  - ben_w 3y ago
    > There is no reason to believe that human intelligence is a stochastic parrot. We have a large body of research to indicate emotion and thought are interdependent [1]. Cognition is not a simple, discrete quantity. And? My thoughts can create a mood, my mood can influence my thoughts. Do you seriously believe that this isn't a trivially easy thing to get even just a LLM to do by itself with the right system prompt, let alone more complex models? Sure, my guess is that it would probably just be role-playing rather than having qualia, but so what? > The Chinese room argument is arguing intentionality I agree with IshKebab about the Chinese room. Just because Searle was trying to argue that because the human within the room doesn't know Chinese therefore the system as a whole doesn't, doesn't mean his argument works. I'd say it doesn't prove anything either way, because the human in that room serves the same role as a homunculus in a human brain, leading to an infinite regression problem: https://en.wikipedia.org/wiki/Homunculus_argument https://en.wikipedia.org/wiki/Homunculus_argument
  - 3y ago
OrderlyTiamat 3y ago
I've seen plenty of badly argued Chinese room debates- there is certainly a lot of bad arguments in those styles. You seem to interpret them as saying all of those arguments are bad, which is a fair interpretation, but I think their point stands if it refers to the nonsense versions of those arguments, which are everywhere on social media (hn, reddit, X, etc). By the way, I think referring to the burden of proof in discussions like this is likely never to improve the debate. All sides will simply claim the burden is on others for one reason or another, most of those reasons will be flawed- requiring discussions on those flaws and now we're even further from the actual points we want to discuss. I've just never seen it produce a fruitful discussion.
- emptysongglass 3y ago
  If I say the sky is falling, the burden of proof is surely on me to provide evidence. I don't have time or the energy to make counter-claims to flat-earthers with no evidence. This is why we say the burden of proof is on those making extraordinary claims. I'm happy to debate with those who provide sufficient evidence but it's not fair to expect I put effort into rebuttals of claims of "nonsense" with no evidence.
  - ben_w 3y ago
    I have seen only one falsifiable test for if a mind actually has subjective conscious experience rather than just saying it does, and even that wasn't one which we can apply to humans or other animals, only AI, which means it will still be an open question if that test ever passes. Saying "it is" or "it isn't" at this point is like early sci-fi speculations about the inhabitants of the Moon, Mars, and Venus — we know our own experience of qualia, but we don't understand how and why we got it, so we don't know what to look for in animals, let alone AI.

madsbuch 3y ago

a proposition does not need to be proved nor falsified. It can exist as a hypothesis. I think that is what the parent commenter points to. I the current environment it is dangerous to be fixed in your opinion, at least until there are proper proofs of anything.

zingelshuher 3y ago

If I may, what is 'AGI' you are talking about? And how can we set up a test which will show that 'thing' isn't GI while humans are. I'm asking because there are different opinions, some believe AGI should reproduce itself, have self-preservation instincts, have a body. To me even long term memory isn't obviously required. This is 'creature' feature. Which is irrelevant to GI. (actually it's more complicated)

IshKebab 3y ago

You've misread. I didn't mention existing models. I agree that it is an extraordinary claim that existing models demonstrate anything like AGI or sentience. I said it was delusional that AI can never be sentient or self-aware. That is also an extraordinary claim that requires extraordinary evidence, and nobody has any. People usually mention stochastic parrots and the Chinese room, but their arguments are very obviously flawed: 1. Current gen AI may only seem like a stochastic parrot, but there's no reason to believe that human intelligence isn't just a really fancy stochastic parrot. 2. The Chinese room experiment just confuses things by anthropomorphizing a single part of the whole intelligent system (i.e. saying there's a man in the room). If you look at the room as a whole, then yes it does understand Chinese. Similarly if you go the other way and imagine each of the neurons in a Chinese person was a tiny man following simple instructions, does that mean the Chinese person no longer understands Chinese? No of course not.

emptysongglass 3y ago
> You've misread. I didn't mention existing models. No you didn't, and that's why I didn't limit my rebuttal to existing models alone. > 1. Current gen AI may only seem like a stochastic parrot, but there's no reason to believe that human intelligence isn't just a really fancy stochastic parrot. There is no reason to believe that human intelligence is a stochastic parrot. We have a large body of research to indicate emotion and thought are interdependent [1]. Cognition is not a simple, discrete quantity. > 2. The Chinese room experiment just confuses things by anthropomorphizing a single part of the whole intelligent system (i.e. saying there's a man in the room). If you look at the room as a whole, then yes it does understand Chinese. Similarly if you go the other way and imagine each of the neurons in a Chinese person was a tiny man following simple instructions, does that mean the Chinese person no longer understands Chinese? No of course not. It sounds like you are confusing things. The Chinese room argument is arguing intentionality, that it is a necessary prerequisite to real human cognition. We have zero evidence of any display of intention by these models. The rest of your argument here I honestly can't follow. [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2366118/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2366118/
- ben_w 3y ago
  > There is no reason to believe that human intelligence is a stochastic parrot. We have a large body of research to indicate emotion and thought are interdependent [1]. Cognition is not a simple, discrete quantity. And? My thoughts can create a mood, my mood can influence my thoughts. Do you seriously believe that this isn't a trivially easy thing to get even just a LLM to do by itself with the right system prompt, let alone more complex models? Sure, my guess is that it would probably just be role-playing rather than having qualia, but so what? > The Chinese room argument is arguing intentionality I agree with IshKebab about the Chinese room. Just because Searle was trying to argue that because the human within the room doesn't know Chinese therefore the system as a whole doesn't, doesn't mean his argument works. I'd say it doesn't prove anything either way, because the human in that room serves the same role as a homunculus in a human brain, leading to an infinite regression problem: https://en.wikipedia.org/wiki/Homunculus_argument https://en.wikipedia.org/wiki/Homunculus_argument
- 3y ago

OrderlyTiamat 3y ago

I've seen plenty of badly argued Chinese room debates- there is certainly a lot of bad arguments in those styles. You seem to interpret them as saying all of those arguments are bad, which is a fair interpretation, but I think their point stands if it refers to the nonsense versions of those arguments, which are everywhere on social media (hn, reddit, X, etc). By the way, I think referring to the burden of proof in discussions like this is likely never to improve the debate. All sides will simply claim the burden is on others for one reason or another, most of those reasons will be flawed- requiring discussions on those flaws and now we're even further from the actual points we want to discuss. I've just never seen it produce a fruitful discussion.

emptysongglass 3y ago
If I say the sky is falling, the burden of proof is surely on me to provide evidence. I don't have time or the energy to make counter-claims to flat-earthers with no evidence. This is why we say the burden of proof is on those making extraordinary claims. I'm happy to debate with those who provide sufficient evidence but it's not fair to expect I put effort into rebuttals of claims of "nonsense" with no evidence.
- ben_w 3y ago
  I have seen only one falsifiable test for if a mind actually has subjective conscious experience rather than just saying it does, and even that wasn't one which we can apply to humans or other animals, only AI, which means it will still be an open question if that test ever passes. Saying "it is" or "it isn't" at this point is like early sci-fi speculations about the inhabitants of the Moon, Mars, and Venus — we know our own experience of qualia, but we don't understand how and why we got it, so we don't know what to look for in animals, let alone AI.