4 ms·

Oh, we know it's weighted connections. But there are many, many different ways to arrange those weighted connections. Human brains seem to have structures that

Oh, we know it's weighted connections. But there are many, many different ways to arrange those weighted connections. Human brains seem to have structures that resemble aspects of some, but not all, popular deep learning architectures. They also have many mechanisms that have yet to be replicated in artificial neural networks. For example, I continue to question two propositions that many others seem to take for granted when they try to predict what LLMs can and cannot do well: 1. LLMs can do generalized symbolic reasoning. 2. If a human does it symbolically, that's how it must be done. Over the past couple years I've grown to be much more sympathetic to Searle's Chinese Room argument. LLMs are incredibly good at mimicking human behavior and performing tasks that were previously impossible for machines. But as you examine what they're doing more closely you start to see them failing in all sorts of interesting ways that remind you that they're still very much in an uncanny valley of sorts. Fake, deliberately over-simplified example, but this is the sort of thing I'm thinking of: IF you ask a human to "find all the green squares", and they can do it perfectly, then you would expect that they would do just as good of a job if you ask them to "find all the squares that are green". That sort of expectation does not work with GPT-4. Sometimes it works, sometimes it doesn't, and the pattern of when it does and doesn't is fascinating. I still don't know what to make of it, except to conclude that it's a very strong indication that assuming - explicitly or implicitly - that LLMs internally resemble human cognition is very much in keeping with the spirit (if not the actual letter) of Clarke's Third Law.

thornewolf 2y ago
I always found the Chinese room to be self-evident as intelligent
stickfigure 2y ago
I think you're anthropomorphizing humans too much. Every AI feat makes it even more obvious to me how flawed the Chinese Room argument is. We just need to get past the realization "oh wow, I'm a machine too". Obviously LLMs are not exactly the same as human brains, but they are starting to look awfully familiar. And not all human brains are the same! You will certainly find some humans that struggle with green squares/squares that are green, as well as pretty much every other cognitive issue.
- jrochkind1 2y ago
  I don't even understand what "anthropomorphizing humans" means. "anthro" means human. "Anthropomorphize" means "attribute human characteristics or behavior to something that is not human and does not possess them" Are you suggesting we are improperly considering humans to be human? or was that a joke I missed?
  - danielbln 2y ago
    OP is saying humans are machines, and that we are therefore anthropomorphizing ourselves by attributing human attributes to our machine selves.
    - jrochkind1 2y ago
      I think you might need a new word, I don't think you can anthropomorphize humans.
      - stickfigure 2y ago
        It's humor, along the lines of "Do not fall into the trap of anthropomorphizing Larry Ellison". My point is that humans are not quite as special as we like to think. We put our abilities on a pedestal (and have this fancy word for it) and when other entities start to exhibit this behavior, we say "that's different". The obvious retort to Searle is that "the room understands chinese". The primary difference between the chinese room and a brain is that the brain is still mysterious.
        bunderbunder 2y ago
        What I was getting at, more than anything, is that, like what Searle pointed out, you can't necessarily infer a black box's internal mechanisms from its outwardly observable behavior. Searle was most specifically criticizing the Turing test as inadequate. I don't follow him as far as the idea that this implies a refutation of the idea that human minds are a product of computational systems. To me that idea relies on an untenable definition of "computation". But the weaker conclusion that you can't simply assume that two completely different systems that exhibit the same observable behavior must use the same mechanism to achieve that behavior does strike me as compelling. Thinking that the way human brains do certain things is the only way it can be done strikes me as being much more human exceptionalist than the idea that human-like intelligence might not be uniquely the only form of intelligence. Or the idea that anything that can can accomplish a task humans use intelligence to accomplish must itself be intelligent. Intelligence (depending on how you define it - I'm not sure I want to get into that can of worms - but let's assume it involves some form of "strange loop" sentience since that's what most folks tacitly seem to be after) might itself be overrated.
4gotunameagain 2y ago
>Oh, we know it's weighted connections. I disagree. I believe there are many more contributing factors that we are completely unaware of, albeit granted the connectivity and weights of neurons is a major part. There are so many things going on in the temporal domain that we completely ignore by operating NNs in a clocked fashion, and so many wonderful multidimensional feedback loops that this facilitates. To say we know how brains work, I think is hubris.
fennecfoxy 2y ago
I mean humans are almost the same thing that we shit talk LLMs etc as being. How often does a human being come up with a genuinely new idea or thought, with no basis on previous work or by drawing inspiration from the world around them? Almost everything we do is a riff on what has already been done. Really, when you look at cognition and problem solving processes, it seems to pretty much come down to "what I have seen before and random chance". Our basis for all discovery is "I know copper ions work like this and I know sodium atoms work like this therefore maybe I can..." which in my opinion will be completely reproducible by machines. Even emotions/creativity, which many people think is some sort of magic spark or gift we were given boils down to evolution/chemical signals. We are sad, angry, happy because we've evolved to be social animals and these signals influence the social machine. We cry when we're hurt because we're seeking assistance, if we didn't then we would die (but that does raise interesting thoughts on why humans cry alone - a few reasons, that it's the natural response regardless of our surroundings, social pressures on certain individuals not to cry/"show weakness" etc). Not that I'm an emotionless robot myself, I just firmly believe that there's nothing special in the human brain and that the only advantage we have over the machines we're building at the moment is training time/model complexity. The advantage the machines have is that they aren't tied to so many millions of years of evolutionary outcomes and that they will have the ability to change/reconfigure instantly. ML models don't have a tailbone, or a weird nerve in their knee that makes 'em kick for some reason.

thornewolf 2y ago

I always found the Chinese room to be self-evident as intelligent

stickfigure 2y ago

I think you're anthropomorphizing humans too much. Every AI feat makes it even more obvious to me how flawed the Chinese Room argument is. We just need to get past the realization "oh wow, I'm a machine too". Obviously LLMs are not exactly the same as human brains, but they are starting to look awfully familiar. And not all human brains are the same! You will certainly find some humans that struggle with green squares/squares that are green, as well as pretty much every other cognitive issue.

jrochkind1 2y ago
I don't even understand what "anthropomorphizing humans" means. "anthro" means human. "Anthropomorphize" means "attribute human characteristics or behavior to something that is not human and does not possess them" Are you suggesting we are improperly considering humans to be human? or was that a joke I missed?
- danielbln 2y ago
  OP is saying humans are machines, and that we are therefore anthropomorphizing ourselves by attributing human attributes to our machine selves.
  - jrochkind1 2y ago
    I think you might need a new word, I don't think you can anthropomorphize humans.
    - stickfigure 2y ago
      It's humor, along the lines of "Do not fall into the trap of anthropomorphizing Larry Ellison". My point is that humans are not quite as special as we like to think. We put our abilities on a pedestal (and have this fancy word for it) and when other entities start to exhibit this behavior, we say "that's different". The obvious retort to Searle is that "the room understands chinese". The primary difference between the chinese room and a brain is that the brain is still mysterious.
      - bunderbunder 2y ago
        What I was getting at, more than anything, is that, like what Searle pointed out, you can't necessarily infer a black box's internal mechanisms from its outwardly observable behavior. Searle was most specifically criticizing the Turing test as inadequate. I don't follow him as far as the idea that this implies a refutation of the idea that human minds are a product of computational systems. To me that idea relies on an untenable definition of "computation". But the weaker conclusion that you can't simply assume that two completely different systems that exhibit the same observable behavior must use the same mechanism to achieve that behavior does strike me as compelling. Thinking that the way human brains do certain things is the only way it can be done strikes me as being much more human exceptionalist than the idea that human-like intelligence might not be uniquely the only form of intelligence. Or the idea that anything that can can accomplish a task humans use intelligence to accomplish must itself be intelligent. Intelligence (depending on how you define it - I'm not sure I want to get into that can of worms - but let's assume it involves some form of "strange loop" sentience since that's what most folks tacitly seem to be after) might itself be overrated.

4gotunameagain 2y ago

>Oh, we know it's weighted connections. I disagree. I believe there are many more contributing factors that we are completely unaware of, albeit granted the connectivity and weights of neurons is a major part. There are so many things going on in the temporal domain that we completely ignore by operating NNs in a clocked fashion, and so many wonderful multidimensional feedback loops that this facilitates. To say we know how brains work, I think is hubris.

fennecfoxy 2y ago

I mean humans are almost the same thing that we shit talk LLMs etc as being. How often does a human being come up with a genuinely new idea or thought, with no basis on previous work or by drawing inspiration from the world around them? Almost everything we do is a riff on what has already been done. Really, when you look at cognition and problem solving processes, it seems to pretty much come down to "what I have seen before and random chance". Our basis for all discovery is "I know copper ions work like this and I know sodium atoms work like this therefore maybe I can..." which in my opinion will be completely reproducible by machines. Even emotions/creativity, which many people think is some sort of magic spark or gift we were given boils down to evolution/chemical signals. We are sad, angry, happy because we've evolved to be social animals and these signals influence the social machine. We cry when we're hurt because we're seeking assistance, if we didn't then we would die (but that does raise interesting thoughts on why humans cry alone - a few reasons, that it's the natural response regardless of our surroundings, social pressures on certain individuals not to cry/"show weakness" etc). Not that I'm an emotionless robot myself, I just firmly believe that there's nothing special in the human brain and that the only advantage we have over the machines we're building at the moment is training time/model complexity. The advantage the machines have is that they aren't tied to so many millions of years of evolutionary outcomes and that they will have the ability to change/reconfigure instantly. ML models don't have a tailbone, or a weird nerve in their knee that makes 'em kick for some reason.