5 ms·

> Its still bunch of instructions. So is your brain. That's the problem with this argument.

freehorse 5mo ago
The brain is (part of) an allostatic system and part of a living organism which reproduces itself constantly. Certainly not "a bunch of instructions".
- simianwords 5mo ago
  Distinction without a difference.
  - meroes 5mo ago
    That’s like saying simulating a supernova and a supernova are both extremely hot.
    - simianwords 5mo ago
      Funny you mention simulations. The simulation theory that we all live in simulations doesn’t have any practical consequences for me or any one. If our world behaves exactly the same, what difference does it make?
      - 9dev 5mo ago
        The perspective. From within the simulation there is no point in making the distinction, but from the outside it does. For another example - running a program on a virtual machine or a physical computer is the same to the program, but very different to debug when you see hardware errors.
      - gerardschiere 5mo ago
        [flagged]
    - MrScruff 5mo ago
      The problem with this is that the word 'hot' only has meaning to a conscious being. And while we don't know what conciousness is, it's extremely hard to argue it's not an emergent property of physics. So if your supernova simulation is complex enough to also model emergent properties like conciousness, the simulated conciousness may well regard the supernova as 'hot'.
      - meroes 5mo ago
        Ya and this leads to the point that if the simulation is so complex, which I take to mean is so like the real thing, then the near-perfect simulation and the actual thing must be highly similar physically. And digital circuits are not physically similar to biological brains yet. So we shouldn’t conclude they have the similar consciousness property.
        MrScruff 4mo ago
        That would imply that the biological physical substrate is necessary for conciousness, which I don't think you can say with any degree of certainty. It's not an assumption I would personally make. And while I'm speculating, my own view is that whatever the eventual subjective experience of what it's like to be an AI is, it will be nothing like the experience of what it's like to be a human, regardless of the fact we're training them to interact in human-like ways.
- rough-sea 5mo ago
  Unless you ascribe to some meta-physical soul - you believe human consciousness is encoded in matter - in the interactions between atoms. Actually at a much higher level of abstraction- neurons - but it’s all simulatable in principle. Thus, yes, it could literally be a bunch of x86 instructions.
  - freehorse 5mo ago
    The problem is 1. that artificial neurons are very restrictive model for actual neurons, let alone brain/organism, as there is more going on in neurons than a thresholded 0-1 activation 2. a brain does not function in isolation, but as regulator of the bodily functions (along with even more things). If you look at the brain in isolation to the body you don't really understand fully what it is doing unless you narrow your view a lot. Eg the brain modulates production of hormones, which in turn affects stuff like heart rate which then comes back to the brain as signal, in a feedback loop. Not to mention actual behaviour and interaction with the world. Toy models of organisms are not organisms. 3. "interaction between atoms" (or rather matter in general, as we have to take into account electrons, photons, gravity and a lot of other things that matter) is too general, too big, artificial neurons are a very useful (for applications) abstraction inspired by biological processes, but not modelling said underlying biology fully. Nobody can imo right now know if "computation" is a good model for "atom interactions" as in whether we can adequately enough model "atom interactions" in a computationally tractable way, and surely we do not do that right now except in very narrow scopes, and we have good reasons to believe that the current computational paradigms/turing machines are inadequate in doing that efficiently enough.
    - jaggederest 5mo ago
      I'm going to ask a couple dumb but genuine questions: Does it matter that neurons are more complex than 0-1? Does the fact that transformer layers don't use purely thresholded 0-1 activation invalidate what you're saying? How do you know that artificial neurons are less capable of producing consciousness than biological ones? How can other people independently verify this? How does the embodied nature of human consciousness preclude consciousness emerging from a computational system? What is the definition of consciousness if it is precluded from occurring in a computational system but present in biological systems? Why do you think exactly modeling interaction between atoms matters for consciousness? And where is the fidelity threshold? Is it the planck length? Finally, a dumb question: how do we know humans are actually conscious, and where is the threshold between consciousness and unconsciousness? And do these criteria exclude all other forms, or other animals?
- adampunk 5mo ago
  To take this seriously we must extend consciousness to nearly all of the tree of life.
plmpsu 5mo ago
Maybe OP is a dualist.
meroes 5mo ago
Well if you go that route, a computer simulating digestion has almost no physical features in common with actual digestion of a stomach. The same holds for consciousness and brains and computers. Them saying it’s just instructions is shorthand for pointing out the physical differences of brains and computers. It’s all just particles, but the higher level differences are vast, and only brains are implicated for first person perspectives via science.
j2j8 5mo ago
A very large number of people could do the calculations of an LLM by hand using pen and paper. It would take a long time, but if the result were conscious then were would the consciousness exist? In the humans? Is it consciousness within consciousness?
gazebo2 5mo ago
This sort of tech-centric distillation of visceral human experience into simple analogies is so grating. We can simultaneously acknowledge that consciousness is not well defined and not "testable" with certainty while also acknowledging that there is something different between the conscious experience we are all aware of as humans, and instructions executing on a chip. The only thing that has changed in the discourse about AI and consciousness vs. "is my home desktop conscious in 2004" is the quality of the simulacrum that LLMs produce vs. pre-ML chatbots.
- DiogenesKynikos 5mo ago
  > while also acknowledging that there is something different between the conscious experience we are all aware of as humans, and instructions executing on a chip. You say this based on what? Your brain is executing instructions on wetware. The entire universe is governed by physical laws. At its base, the argument that computers can't be conscious is dualist. It assumes that there's some parallel realm of spirit that the brain is peculiarly able to tap into, but which computer programs that function in very similar ways to the brain don't tap into. > the quality of the simulacrum that LLMs produce vs. pre-ML chatbots. It seems to me that you're denigrating LLMs, or implying that they're only simulating thought, as opposed to actually thinking. But the difference between thinking and an extremely good simulacrum of thought is meaningless. They become the same thing. It's a bit like the Mitchell and Webb skit about faking the moon landings, in which the plotters quickly realize that the only way to convincingly fake landing on the moon is to fly a rocket to the moon and film the "fake" landings on the moon.
  - gazebo2 5mo ago
    >You say this based on what? I say it based on the fact that I experience consciousness and therefore know it is a phenomenon wholly separate from the outward effect I have on the world via speech or any other physical action. >It assumes that there's some parallel realm of spirit that the brain is peculiarly able to tap into, but which computer programs that function in very similar ways to the brain don't tap into. Ruling this out completely implies that modern science has a total and complete understanding of consciousness and the universe in its entirety, which it does not. I don't think it's unreasonable to leave open the possibility that there is an unexplained phenomenon that explains the conscious experience which is still bounded by the laws of physics. >.. the difference between thinking and an extremely good simulacrum of thought is meaningless. They become the same thing. Do they? To me it seems like you equate the process with the result, like saying if I gave you a gift, it doesn't matter whether I bought it or made it by hand, in either case I "made" it because the end result is the same. There is a conscious experience that (hopefully) all humans experience and can testify to, the question is whether or not an LLM predicting tokens based on a giant vector map is experiencing the same thing, which I'd say is obviously not happening. My point about chatbots is about exactly this -- 10 years ago even you would have laughed at somebody that said a hand-rolled chatbot was conscious/thinking, but because the output has improved, now suddenly it's all the same, and your brain is a computer, and Claude has feelings, blah blah.

The brain is (part of) an allostatic system and part of a living organism which reproduces itself constantly. Certainly not "a bunch of instructions".

simianwords 5mo ago
Distinction without a difference.
- meroes 5mo ago
  That’s like saying simulating a supernova and a supernova are both extremely hot.
  - simianwords 5mo ago
    Funny you mention simulations. The simulation theory that we all live in simulations doesn’t have any practical consequences for me or any one. If our world behaves exactly the same, what difference does it make?
    - 9dev 5mo ago
      The perspective. From within the simulation there is no point in making the distinction, but from the outside it does. For another example - running a program on a virtual machine or a physical computer is the same to the program, but very different to debug when you see hardware errors.
    - gerardschiere 5mo ago
      [flagged]
  - MrScruff 5mo ago
    The problem with this is that the word 'hot' only has meaning to a conscious being. And while we don't know what conciousness is, it's extremely hard to argue it's not an emergent property of physics. So if your supernova simulation is complex enough to also model emergent properties like conciousness, the simulated conciousness may well regard the supernova as 'hot'.
    - meroes 5mo ago
      Ya and this leads to the point that if the simulation is so complex, which I take to mean is so like the real thing, then the near-perfect simulation and the actual thing must be highly similar physically. And digital circuits are not physically similar to biological brains yet. So we shouldn’t conclude they have the similar consciousness property.
      - MrScruff 4mo ago
        That would imply that the biological physical substrate is necessary for conciousness, which I don't think you can say with any degree of certainty. It's not an assumption I would personally make. And while I'm speculating, my own view is that whatever the eventual subjective experience of what it's like to be an AI is, it will be nothing like the experience of what it's like to be a human, regardless of the fact we're training them to interact in human-like ways.
rough-sea 5mo ago
Unless you ascribe to some meta-physical soul - you believe human consciousness is encoded in matter - in the interactions between atoms. Actually at a much higher level of abstraction- neurons - but it’s all simulatable in principle. Thus, yes, it could literally be a bunch of x86 instructions.
- freehorse 5mo ago
  The problem is 1. that artificial neurons are very restrictive model for actual neurons, let alone brain/organism, as there is more going on in neurons than a thresholded 0-1 activation 2. a brain does not function in isolation, but as regulator of the bodily functions (along with even more things). If you look at the brain in isolation to the body you don't really understand fully what it is doing unless you narrow your view a lot. Eg the brain modulates production of hormones, which in turn affects stuff like heart rate which then comes back to the brain as signal, in a feedback loop. Not to mention actual behaviour and interaction with the world. Toy models of organisms are not organisms. 3. "interaction between atoms" (or rather matter in general, as we have to take into account electrons, photons, gravity and a lot of other things that matter) is too general, too big, artificial neurons are a very useful (for applications) abstraction inspired by biological processes, but not modelling said underlying biology fully. Nobody can imo right now know if "computation" is a good model for "atom interactions" as in whether we can adequately enough model "atom interactions" in a computationally tractable way, and surely we do not do that right now except in very narrow scopes, and we have good reasons to believe that the current computational paradigms/turing machines are inadequate in doing that efficiently enough.
  - jaggederest 5mo ago
    I'm going to ask a couple dumb but genuine questions: Does it matter that neurons are more complex than 0-1? Does the fact that transformer layers don't use purely thresholded 0-1 activation invalidate what you're saying? How do you know that artificial neurons are less capable of producing consciousness than biological ones? How can other people independently verify this? How does the embodied nature of human consciousness preclude consciousness emerging from a computational system? What is the definition of consciousness if it is precluded from occurring in a computational system but present in biological systems? Why do you think exactly modeling interaction between atoms matters for consciousness? And where is the fidelity threshold? Is it the planck length? Finally, a dumb question: how do we know humans are actually conscious, and where is the threshold between consciousness and unconsciousness? And do these criteria exclude all other forms, or other animals?
adampunk 5mo ago
To take this seriously we must extend consciousness to nearly all of the tree of life.

plmpsu 5mo ago

Maybe OP is a dualist.

meroes 5mo ago

Well if you go that route, a computer simulating digestion has almost no physical features in common with actual digestion of a stomach. The same holds for consciousness and brains and computers. Them saying it’s just instructions is shorthand for pointing out the physical differences of brains and computers. It’s all just particles, but the higher level differences are vast, and only brains are implicated for first person perspectives via science.

j2j8 5mo ago

A very large number of people could do the calculations of an LLM by hand using pen and paper. It would take a long time, but if the result were conscious then were would the consciousness exist? In the humans? Is it consciousness within consciousness?

gazebo2 5mo ago

This sort of tech-centric distillation of visceral human experience into simple analogies is so grating. We can simultaneously acknowledge that consciousness is not well defined and not "testable" with certainty while also acknowledging that there is something different between the conscious experience we are all aware of as humans, and instructions executing on a chip. The only thing that has changed in the discourse about AI and consciousness vs. "is my home desktop conscious in 2004" is the quality of the simulacrum that LLMs produce vs. pre-ML chatbots.

DiogenesKynikos 5mo ago
> while also acknowledging that there is something different between the conscious experience we are all aware of as humans, and instructions executing on a chip. You say this based on what? Your brain is executing instructions on wetware. The entire universe is governed by physical laws. At its base, the argument that computers can't be conscious is dualist. It assumes that there's some parallel realm of spirit that the brain is peculiarly able to tap into, but which computer programs that function in very similar ways to the brain don't tap into. > the quality of the simulacrum that LLMs produce vs. pre-ML chatbots. It seems to me that you're denigrating LLMs, or implying that they're only simulating thought, as opposed to actually thinking. But the difference between thinking and an extremely good simulacrum of thought is meaningless. They become the same thing. It's a bit like the Mitchell and Webb skit about faking the moon landings, in which the plotters quickly realize that the only way to convincingly fake landing on the moon is to fly a rocket to the moon and film the "fake" landings on the moon.
- gazebo2 5mo ago
  >You say this based on what? I say it based on the fact that I experience consciousness and therefore know it is a phenomenon wholly separate from the outward effect I have on the world via speech or any other physical action. >It assumes that there's some parallel realm of spirit that the brain is peculiarly able to tap into, but which computer programs that function in very similar ways to the brain don't tap into. Ruling this out completely implies that modern science has a total and complete understanding of consciousness and the universe in its entirety, which it does not. I don't think it's unreasonable to leave open the possibility that there is an unexplained phenomenon that explains the conscious experience which is still bounded by the laws of physics. >.. the difference between thinking and an extremely good simulacrum of thought is meaningless. They become the same thing. Do they? To me it seems like you equate the process with the result, like saying if I gave you a gift, it doesn't matter whether I bought it or made it by hand, in either case I "made" it because the end result is the same. There is a conscious experience that (hopefully) all humans experience and can testify to, the question is whether or not an LLM predicting tokens based on a giant vector map is experiencing the same thing, which I'd say is obviously not happening. My point about chatbots is about exactly this -- 10 years ago even you would have laughed at somebody that said a hand-rolled chatbot was conscious/thinking, but because the output has improved, now suddenly it's all the same, and your brain is a computer, and Claude has feelings, blah blah.