5 ms·

While it is true in general that simulating a thing is not that thing itself, a simulation may be the same as the thing being simulated in all aspects that matt

While it is true in general that simulating a thing is not that thing itself, a simulation may be the same as the thing being simulated in all aspects that matter. In particular, information-manipulating systems are amenable to simulation by other information-manipulating systems: one can, for example, make a simulation of a cryptographic machine that encrypts and decrypts messages. https://www.101computing.net/enigma-machine-emulator/ https://www.101computing.net/enigma-machine-emulator/ I agree that we don't know how to build machines that can feel (and are conscious); all I am saying here is that Koch's argument (at least as presented in this article) against a suitably-programmed transistor-based, von Neumann architecture digital computer becoming a conscious machine does not hold up.

slibhb 2y ago
> While it is true in general that simulating a thing is not that thing itself, a simulation may be the same as the thing being simulated in all aspects that matter. What does matter? Does consciousness? It may matter less than is widely assumed since LLMs can do lots of cool stuff and aren't conscious. > I agree that we don't know how to build machines that can feel (and are conscious); all I am saying here is that Koch's argument (at least as presented in this article) against a suitably-programmed transistor-based, von Neumann architecture digital computer becoming a conscious machine does not hold up. I agree that his argument isn't very good. Yet I'm sure it's the case that there is no possible program that can run on any computer that exists today that can be conscious. Hardware matters. Computers need multiple cores to do parallel computations. Similarly, they'd need some kind of hardware to be conscious, hardware that we don't know how to build. If we want to build it, we'll almost certainly need to understand what makes humans or other animals conscious. The odds of just stumbling across the solution seems very low.
- mannykannot 2y ago
  > What does matter? Does consciousness? Personally, I think it does, and I do not think it can be dismissed as either epiphenomenal or some sort of illusion. I don't think LLMs are evidence against this, as, while they are uncannily like humans in some respects, they seem to lack some pretty important things (such as self-recognition or a theory of mind, or, for that matter, any concept of language as referring to an external world that goes by rules independently of what is said about it.) > Hardware matters. Computers need multiple cores to do parallel computations. Similarly, they'd need some kind of hardware to be conscious, hardware that we don't know how to build. This is not a valid argument - the conclusion ('[computers would] need some kind of hardware to be conscious, hardware that we don't know how to build') does not ineluctably follow from the premise ('Computers need multiple cores to do parallel computations' or even just 'Hardware matters'.)
  - slibhb 2y ago
    I don't think consciousness will end up mattering very much. It matters a lot to us because we're conscious and it's not totally clear that moral considerations apply to things that aren't conscious. But I think it will turn out that non-conscious computer programs will be able to do more or less everything we can do. > This is not a valid argument - the conclusion ('[computers would] need some kind of hardware to be conscious, hardware that we don't know how to build') does not ineluctably follow from the premise ('Computers need multiple cores to do parallel computations' or even just 'Hardware matters'.) I'm not claiming that my argument proves anything. The point I'm making is that simulating something in software isn't magic. You can't write a computer program that executes code in parallel and run it on a single core CPU and observe parallelism. You need to have hardware that supports parallelism. I think it's very likely that this is also the case for consciousness. You can simulate consciousness. But to actually achieve conscious experience, you need a certain kind of hardware. This is just my opinion, there's no proof here.
    - mannykannot 2y ago
      I am curious as to what one would be simulating if one were simulating consciousness without the simulation having achieved conscious experience. By simulating consciousness, do you mean just simulating the behavior of an agent presumed to be conscious, such as a person? We should be careful not to equivocate between the two, and to be clear, wherever I have mentioned or referred to the simulation of a conscious agent, I mean the former, not merely the latter.

slibhb 2y ago

> While it is true in general that simulating a thing is not that thing itself, a simulation may be the same as the thing being simulated in all aspects that matter. What does matter? Does consciousness? It may matter less than is widely assumed since LLMs can do lots of cool stuff and aren't conscious. > I agree that we don't know how to build machines that can feel (and are conscious); all I am saying here is that Koch's argument (at least as presented in this article) against a suitably-programmed transistor-based, von Neumann architecture digital computer becoming a conscious machine does not hold up. I agree that his argument isn't very good. Yet I'm sure it's the case that there is no possible program that can run on any computer that exists today that can be conscious. Hardware matters. Computers need multiple cores to do parallel computations. Similarly, they'd need some kind of hardware to be conscious, hardware that we don't know how to build. If we want to build it, we'll almost certainly need to understand what makes humans or other animals conscious. The odds of just stumbling across the solution seems very low.

mannykannot 2y ago
> What does matter? Does consciousness? Personally, I think it does, and I do not think it can be dismissed as either epiphenomenal or some sort of illusion. I don't think LLMs are evidence against this, as, while they are uncannily like humans in some respects, they seem to lack some pretty important things (such as self-recognition or a theory of mind, or, for that matter, any concept of language as referring to an external world that goes by rules independently of what is said about it.) > Hardware matters. Computers need multiple cores to do parallel computations. Similarly, they'd need some kind of hardware to be conscious, hardware that we don't know how to build. This is not a valid argument - the conclusion ('[computers would] need some kind of hardware to be conscious, hardware that we don't know how to build') does not ineluctably follow from the premise ('Computers need multiple cores to do parallel computations' or even just 'Hardware matters'.)
- slibhb 2y ago
  I don't think consciousness will end up mattering very much. It matters a lot to us because we're conscious and it's not totally clear that moral considerations apply to things that aren't conscious. But I think it will turn out that non-conscious computer programs will be able to do more or less everything we can do. > This is not a valid argument - the conclusion ('[computers would] need some kind of hardware to be conscious, hardware that we don't know how to build') does not ineluctably follow from the premise ('Computers need multiple cores to do parallel computations' or even just 'Hardware matters'.) I'm not claiming that my argument proves anything. The point I'm making is that simulating something in software isn't magic. You can't write a computer program that executes code in parallel and run it on a single core CPU and observe parallelism. You need to have hardware that supports parallelism. I think it's very likely that this is also the case for consciousness. You can simulate consciousness. But to actually achieve conscious experience, you need a certain kind of hardware. This is just my opinion, there's no proof here.
  - mannykannot 2y ago
    I am curious as to what one would be simulating if one were simulating consciousness without the simulation having achieved conscious experience. By simulating consciousness, do you mean just simulating the behavior of an agent presumed to be conscious, such as a person? We should be careful not to equivocate between the two, and to be clear, wherever I have mentioned or referred to the simulation of a conscious agent, I mean the former, not merely the latter.