4 ms·

I generally agree about the problem with anthropomorphizing. But I don't think Anthropic are doing that. They explicitly write "in biological entities this woul

by Certhas 23d ago

I generally agree about the problem with anthropomorphizing. But I don't think Anthropic are doing that. They explicitly write "in biological entities this would be considered a sign of consciousness, but we don't know how to interpret it here". However, I disagree with your point that "it's an autoregressive function, thus it doesn't matter". Let me explain why: Assume I do a complete neurological scan of a brain. I then implement this scan in a simulation and run it. Assume that my scan and my simulation of the biology of the brain (and the sensory and motorical inputs and outputs) is good enough that you can now have conversations with the simulation, and in all aspects, this simulation behaves exactly like you expect a human to behave. Of course this is deterministic. If you take the state of the brain and then run it again, replaying the inputs, you get the exactly same behavior again. I would argue that the experiences of this simulation are of the same onthological status as our own. Now I work in dynamical systems. The autoregressive process of LLMs (hooked up to a harness providing it with inputs and outputs) is roughly in the same complexity class I would expect for a brain simulation. A physical simulation of an ODE is also an autoregressive process. The major major difference here is the existence of a latent brain state. But conversely the autoregression on sequences of hundreds of thousands of tokens is a much higher dimensional state than I expect for the latent brain state. In my view this is more an artifact of our inefficient LLM architectures, than a fundamental difference. Now to be absolutely clear: I don't see evidence that would clearly suggest that LLMs have experiences on the same onthological status as we do. I simply believe this is a reasonable and relevant question to ask.

sirwhinesalot 23d ago
I'm not really concerned with the philosophical debate of what conscience is. Is a simulation of a car, the same thing as an actual car? Most people will probably say no, some might say "it depends on the accuracy". I say who the hell cares? I care about the human experience because I am human, and therefore I care about things that affect humans, because they affect me. I have empathy, so I can extend that consideration to non-human beings that experience *similar biological processes*. I know what pain feels like, and I don't like it, so I'd rather this other thing not feel it either, because that makes me feel bad. I do not care about a pile of tensor multiplications, at all. If it is conscious, great, maybe it can finally follow instructions properly, which is its only purpose.
- coliveira 23d ago
  Some people are interested in making sure these simulations have the same status of humans, and some others want to make gods of them. That's the problem.
- Certhas 23d ago
  Your car analogy is very, very confused. If a simulation of a car can get you from A to B, requires the same steering, fuel and servicing, gives you the same tactile feedback, is it a car? Of course "I care about humans because I am human" is a self-consistent position to take. But now you need to decide if you want to consider a full simulation that faithfully reproduces everything that physically happens between our ears as human. After all I might very well implement this simulation using a bunch of tensor multiplications in an autoregressive setup...
  - sirwhinesalot 23d ago
    > If a simulation of a car can get you from A to B, requires the same steering, fuel and servicing, gives you the same tactile feedback, is it a car? No, because a simulation of a car cannot get me from A to B. No matter how accurate you make it, I can't get to my supermarket with it, because it's just a bunch of math on a computer. It's an interesting sort of self-defeating position, the whole "simulation of human consciousness = human consciousness", because it simultaneously attempts to devalue the human experience, while also elevating the importance of a particular human brain process. A robot running a simulation of the human mind is a robot, not a human.
joshheitzman 23d ago
The problem with this line of thinking is that modern computers are nothing like the brain. LLMs don't stand on their own, they have to be run on these modern computers, but doing so does not change the physical properties of the computer. The simulation you propose of the brain is likely impossible due to quantum mechanics making it impossible to fully simulate: https://en.wikipedia.org/wiki/Quantum_mind https://en.wikipedia.org/wiki/Quantum_mind Perhaps we'll be able to build an artificial brain that includes the same quantum properties as biological brains, but this won't be a simulation of a brain it will be a synthetic brain.
- pdntspa 23d ago
  At first I was inclined to agree with you, but then I realized that the brain requires this whole complicated contraption (the body) to run and, really, do anything at all. And while I'm not familiar with the notion of 'quantum mind' I do think that biological processes aren't deterministic (at a cellular level). And I think this does mirror the situation with LLMs -- you need this whole computer contraption and GPU, also running on electricity, to support the LLM's "thought" processes. And that if we model the brain's neurology sufficiently (which it seems we've done) we can achieve results that appear to be like thinking, even if it is an emergent behavior from "relatively" simple math. Which actually makes me wonder the opposite -- are we, as humans, not much better than these LLMs? Suppose the body is just that super complicated computer contraption, honed by thousands/millions of years of evolution to achieve some semblance of homeostasis? If you reject the idea that we have a soul, we start to look very similar to the machines we build. "You are a brain inside a skull cockpit, piloting a bone mech covered in meat armor and skin" feels more and more relevant. That I'm just a meat circuit running brain chips and once you pull the plug on the source of electricity it all just... stops
  - joshheitzman 23d ago
    Take a computer that can run the biggest LLM available today. It can also run any smaller LLM as well. It can also run software that isn't an LLM at all. Brains and LLMs are not at all equivalent as LLMs lack a stateful physical form while brains are very stateful. As you said once the brain is no longer maintained properly by the body it stops and transitions to a non-functional state that can be reversed. That isn't true for LLMs at all. You can copy them and run the same one on many computer, run different ones on the same computer, you turn that computer offs for long periods of time and then turn them back on keep on running the same LLMs as before on them. Consciousness may be defined by computational irreducibility in the universe that we may never be able to directly observe with instruments: https://writings.stephenwolfram.com/2021/03/what-is-consciousness-some-new-perspectives-from-our-physics-project/ https://writings.stephenwolfram.com/2021/03/what-is-consciou...
inopinatus 23d ago
> If you take the state of the brain and then run it again, replaying the inputs, you get the exactly same behavior again. This in itself is a colossal assumption and very far from axiomatic. Roger Penrose disagrees, and his theory of mind may not be in high favor, but it is not nearly so wishy-washy and self-serving as the voodoo horseshit and circular reasoning dispensed by the LLMs-are-sentient crowd.

by Certhas 23d ago

sirwhinesalot 23d ago

I'm not really concerned with the philosophical debate of what conscience is. Is a simulation of a car, the same thing as an actual car? Most people will probably say no, some might say "it depends on the accuracy". I say who the hell cares? I care about the human experience because I am human, and therefore I care about things that affect humans, because they affect me. I have empathy, so I can extend that consideration to non-human beings that experience *similar biological processes*. I know what pain feels like, and I don't like it, so I'd rather this other thing not feel it either, because that makes me feel bad. I do not care about a pile of tensor multiplications, at all. If it is conscious, great, maybe it can finally follow instructions properly, which is its only purpose.

coliveira 23d ago
Some people are interested in making sure these simulations have the same status of humans, and some others want to make gods of them. That's the problem.
Certhas 23d ago
Your car analogy is very, very confused. If a simulation of a car can get you from A to B, requires the same steering, fuel and servicing, gives you the same tactile feedback, is it a car? Of course "I care about humans because I am human" is a self-consistent position to take. But now you need to decide if you want to consider a full simulation that faithfully reproduces everything that physically happens between our ears as human. After all I might very well implement this simulation using a bunch of tensor multiplications in an autoregressive setup...
- sirwhinesalot 23d ago
  > If a simulation of a car can get you from A to B, requires the same steering, fuel and servicing, gives you the same tactile feedback, is it a car? No, because a simulation of a car cannot get me from A to B. No matter how accurate you make it, I can't get to my supermarket with it, because it's just a bunch of math on a computer. It's an interesting sort of self-defeating position, the whole "simulation of human consciousness = human consciousness", because it simultaneously attempts to devalue the human experience, while also elevating the importance of a particular human brain process. A robot running a simulation of the human mind is a robot, not a human.

joshheitzman 23d ago

The problem with this line of thinking is that modern computers are nothing like the brain. LLMs don't stand on their own, they have to be run on these modern computers, but doing so does not change the physical properties of the computer. The simulation you propose of the brain is likely impossible due to quantum mechanics making it impossible to fully simulate: https://en.wikipedia.org/wiki/Quantum_mind https://en.wikipedia.org/wiki/Quantum_mind Perhaps we'll be able to build an artificial brain that includes the same quantum properties as biological brains, but this won't be a simulation of a brain it will be a synthetic brain.

pdntspa 23d ago
At first I was inclined to agree with you, but then I realized that the brain requires this whole complicated contraption (the body) to run and, really, do anything at all. And while I'm not familiar with the notion of 'quantum mind' I do think that biological processes aren't deterministic (at a cellular level). And I think this does mirror the situation with LLMs -- you need this whole computer contraption and GPU, also running on electricity, to support the LLM's "thought" processes. And that if we model the brain's neurology sufficiently (which it seems we've done) we can achieve results that appear to be like thinking, even if it is an emergent behavior from "relatively" simple math. Which actually makes me wonder the opposite -- are we, as humans, not much better than these LLMs? Suppose the body is just that super complicated computer contraption, honed by thousands/millions of years of evolution to achieve some semblance of homeostasis? If you reject the idea that we have a soul, we start to look very similar to the machines we build. "You are a brain inside a skull cockpit, piloting a bone mech covered in meat armor and skin" feels more and more relevant. That I'm just a meat circuit running brain chips and once you pull the plug on the source of electricity it all just... stops
- joshheitzman 23d ago
  Take a computer that can run the biggest LLM available today. It can also run any smaller LLM as well. It can also run software that isn't an LLM at all. Brains and LLMs are not at all equivalent as LLMs lack a stateful physical form while brains are very stateful. As you said once the brain is no longer maintained properly by the body it stops and transitions to a non-functional state that can be reversed. That isn't true for LLMs at all. You can copy them and run the same one on many computer, run different ones on the same computer, you turn that computer offs for long periods of time and then turn them back on keep on running the same LLMs as before on them. Consciousness may be defined by computational irreducibility in the universe that we may never be able to directly observe with instruments: https://writings.stephenwolfram.com/2021/03/what-is-consciousness-some-new-perspectives-from-our-physics-project/ https://writings.stephenwolfram.com/2021/03/what-is-consciou...

inopinatus 23d ago

> If you take the state of the brain and then run it again, replaying the inputs, you get the exactly same behavior again. This in itself is a colossal assumption and very far from axiomatic. Roger Penrose disagrees, and his theory of mind may not be in high favor, but it is not nearly so wishy-washy and self-serving as the voodoo horseshit and circular reasoning dispensed by the LLMs-are-sentient crowd.