5 ms·

Does this mean that artificial computer systems, wired appropriately, can be conscious? Not necessarily, Koch says. This might one day be possible with the adve

Does this mean that artificial computer systems, wired appropriately, can be conscious? Not necessarily, Koch says. This might one day be possible with the advent of new technology, but we are not there yet. He writes. “The high connectivity [in a human brain] is very different from that found in the central processing unit of any digital computer, where one transistor typically connects to a handful of other transistors.” For the foreseeable future, AI systems will remain unconscious despite appearances to the contrary. I think one would have to be unaware of the ability of simple Turing-equivalent machines to simulate devices of much greater complexity than themselves in order to find this argument persuasive. We see here that Koch is a fan of IIT; can he show one machine with a high IIT score that cannot be simulated by a simple (though sufficiently-sized) Turing-equivalent computer, possibly abetted with a source of entropy?

slibhb 2y ago
Simulating something isn't the same thing as being that thing. You can write a computer program that simulates pain. Perhaps even to the point that it convinces people that it feels pain. But, actually, it doesn't have a nervous system and can't feel pain. Maybe in the future we'll build machines that can feel (and are conscious) but currently we don't know how. It would be very strange if we build those machines accidentally while training LLMs.
- mannykannot 2y ago
  While it is true in general that simulating a thing is not that thing itself, a simulation may be the same as the thing being simulated in all aspects that matter. In particular, information-manipulating systems are amenable to simulation by other information-manipulating systems: one can, for example, make a simulation of a cryptographic machine that encrypts and decrypts messages. https://www.101computing.net/enigma-machine-emulator/ https://www.101computing.net/enigma-machine-emulator/ I agree that we don't know how to build machines that can feel (and are conscious); all I am saying here is that Koch's argument (at least as presented in this article) against a suitably-programmed transistor-based, von Neumann architecture digital computer becoming a conscious machine does not hold up.
  - slibhb 2y ago
    > While it is true in general that simulating a thing is not that thing itself, a simulation may be the same as the thing being simulated in all aspects that matter. What does matter? Does consciousness? It may matter less than is widely assumed since LLMs can do lots of cool stuff and aren't conscious. > I agree that we don't know how to build machines that can feel (and are conscious); all I am saying here is that Koch's argument (at least as presented in this article) against a suitably-programmed transistor-based, von Neumann architecture digital computer becoming a conscious machine does not hold up. I agree that his argument isn't very good. Yet I'm sure it's the case that there is no possible program that can run on any computer that exists today that can be conscious. Hardware matters. Computers need multiple cores to do parallel computations. Similarly, they'd need some kind of hardware to be conscious, hardware that we don't know how to build. If we want to build it, we'll almost certainly need to understand what makes humans or other animals conscious. The odds of just stumbling across the solution seems very low.
    - mannykannot 2y ago
      > What does matter? Does consciousness? Personally, I think it does, and I do not think it can be dismissed as either epiphenomenal or some sort of illusion. I don't think LLMs are evidence against this, as, while they are uncannily like humans in some respects, they seem to lack some pretty important things (such as self-recognition or a theory of mind, or, for that matter, any concept of language as referring to an external world that goes by rules independently of what is said about it.) > Hardware matters. Computers need multiple cores to do parallel computations. Similarly, they'd need some kind of hardware to be conscious, hardware that we don't know how to build. This is not a valid argument - the conclusion ('[computers would] need some kind of hardware to be conscious, hardware that we don't know how to build') does not ineluctably follow from the premise ('Computers need multiple cores to do parallel computations' or even just 'Hardware matters'.)
      - slibhb 2y ago
        I don't think consciousness will end up mattering very much. It matters a lot to us because we're conscious and it's not totally clear that moral considerations apply to things that aren't conscious. But I think it will turn out that non-conscious computer programs will be able to do more or less everything we can do. > This is not a valid argument - the conclusion ('[computers would] need some kind of hardware to be conscious, hardware that we don't know how to build') does not ineluctably follow from the premise ('Computers need multiple cores to do parallel computations' or even just 'Hardware matters'.) I'm not claiming that my argument proves anything. The point I'm making is that simulating something in software isn't magic. You can't write a computer program that executes code in parallel and run it on a single core CPU and observe parallelism. You need to have hardware that supports parallelism. I think it's very likely that this is also the case for consciousness. You can simulate consciousness. But to actually achieve conscious experience, you need a certain kind of hardware. This is just my opinion, there's no proof here.
- jhickok 2y ago
  >Simulating something isn't the same thing as being that thing. That's a controversial, though probably majority position, among people who study consciousness. Functional equivalence in the minds of many hard-nosed, very important folks in the field *does* entail feature equivalence with respect to consciousness.
  - goatlover 2y ago
    Problem is functional equivalence doesn't deal with the hard problem, since the hard problem isn't functional, it's a felt experiential matter. No degree of functional equivalence can tell us whether a system is experiencing something. It only tells us that it's functioning in an equivalent manner. Fundamentally, this is a dichotomy between objective and subjective. Functional is an objective measurement.
    - mannykannot 2y ago
      > No degree of functional equivalence can tell us whether a system is experiencing something. What makes you sure of that? I have read all the usual arguments, and I don't find them sufficient to justify the certainty that it is beyond the realm of possibility of doing so, at least up to the standard of evidence by which we assume other people are experiencing something. Framing the physical-mental problem in terms of equivalence is essentially begging the question anyway, tacitly presupposing that the mental is not a physical phenomenon. No-one asks what, physically, is equivalent to a hurricane (to use just one example of a complex and what was once a mysterious phenomenon.)
    - jhickok 2y ago
      That view of the hard problem begs the question against functionalist theories of consciousness though. Many physicalists deny that the hard-problem (or backing arguments like the zombie argument) entails that there is such a dichotomy.
  - mannykannot 2y ago
    Why would these viewpoints be regarded as mutually exclusive? What's wrong with X and Y being functionally equivalent in every respect, yet different in implementation? See the last paragraph of my reply to goatlover for why I feel that identity is a red herring.

slibhb 2y ago

Simulating something isn't the same thing as being that thing. You can write a computer program that simulates pain. Perhaps even to the point that it convinces people that it feels pain. But, actually, it doesn't have a nervous system and can't feel pain. Maybe in the future we'll build machines that can feel (and are conscious) but currently we don't know how. It would be very strange if we build those machines accidentally while training LLMs.

mannykannot 2y ago
While it is true in general that simulating a thing is not that thing itself, a simulation may be the same as the thing being simulated in all aspects that matter. In particular, information-manipulating systems are amenable to simulation by other information-manipulating systems: one can, for example, make a simulation of a cryptographic machine that encrypts and decrypts messages. https://www.101computing.net/enigma-machine-emulator/ https://www.101computing.net/enigma-machine-emulator/ I agree that we don't know how to build machines that can feel (and are conscious); all I am saying here is that Koch's argument (at least as presented in this article) against a suitably-programmed transistor-based, von Neumann architecture digital computer becoming a conscious machine does not hold up.
- slibhb 2y ago
  > While it is true in general that simulating a thing is not that thing itself, a simulation may be the same as the thing being simulated in all aspects that matter. What does matter? Does consciousness? It may matter less than is widely assumed since LLMs can do lots of cool stuff and aren't conscious. > I agree that we don't know how to build machines that can feel (and are conscious); all I am saying here is that Koch's argument (at least as presented in this article) against a suitably-programmed transistor-based, von Neumann architecture digital computer becoming a conscious machine does not hold up. I agree that his argument isn't very good. Yet I'm sure it's the case that there is no possible program that can run on any computer that exists today that can be conscious. Hardware matters. Computers need multiple cores to do parallel computations. Similarly, they'd need some kind of hardware to be conscious, hardware that we don't know how to build. If we want to build it, we'll almost certainly need to understand what makes humans or other animals conscious. The odds of just stumbling across the solution seems very low.
  - mannykannot 2y ago
    > What does matter? Does consciousness? Personally, I think it does, and I do not think it can be dismissed as either epiphenomenal or some sort of illusion. I don't think LLMs are evidence against this, as, while they are uncannily like humans in some respects, they seem to lack some pretty important things (such as self-recognition or a theory of mind, or, for that matter, any concept of language as referring to an external world that goes by rules independently of what is said about it.) > Hardware matters. Computers need multiple cores to do parallel computations. Similarly, they'd need some kind of hardware to be conscious, hardware that we don't know how to build. This is not a valid argument - the conclusion ('[computers would] need some kind of hardware to be conscious, hardware that we don't know how to build') does not ineluctably follow from the premise ('Computers need multiple cores to do parallel computations' or even just 'Hardware matters'.)
    - slibhb 2y ago
      I don't think consciousness will end up mattering very much. It matters a lot to us because we're conscious and it's not totally clear that moral considerations apply to things that aren't conscious. But I think it will turn out that non-conscious computer programs will be able to do more or less everything we can do. > This is not a valid argument - the conclusion ('[computers would] need some kind of hardware to be conscious, hardware that we don't know how to build') does not ineluctably follow from the premise ('Computers need multiple cores to do parallel computations' or even just 'Hardware matters'.) I'm not claiming that my argument proves anything. The point I'm making is that simulating something in software isn't magic. You can't write a computer program that executes code in parallel and run it on a single core CPU and observe parallelism. You need to have hardware that supports parallelism. I think it's very likely that this is also the case for consciousness. You can simulate consciousness. But to actually achieve conscious experience, you need a certain kind of hardware. This is just my opinion, there's no proof here.
jhickok 2y ago
>Simulating something isn't the same thing as being that thing. That's a controversial, though probably majority position, among people who study consciousness. Functional equivalence in the minds of many hard-nosed, very important folks in the field *does* entail feature equivalence with respect to consciousness.
- goatlover 2y ago
  Problem is functional equivalence doesn't deal with the hard problem, since the hard problem isn't functional, it's a felt experiential matter. No degree of functional equivalence can tell us whether a system is experiencing something. It only tells us that it's functioning in an equivalent manner. Fundamentally, this is a dichotomy between objective and subjective. Functional is an objective measurement.
  - mannykannot 2y ago
    > No degree of functional equivalence can tell us whether a system is experiencing something. What makes you sure of that? I have read all the usual arguments, and I don't find them sufficient to justify the certainty that it is beyond the realm of possibility of doing so, at least up to the standard of evidence by which we assume other people are experiencing something. Framing the physical-mental problem in terms of equivalence is essentially begging the question anyway, tacitly presupposing that the mental is not a physical phenomenon. No-one asks what, physically, is equivalent to a hurricane (to use just one example of a complex and what was once a mysterious phenomenon.)
  - jhickok 2y ago
    That view of the hard problem begs the question against functionalist theories of consciousness though. Many physicalists deny that the hard-problem (or backing arguments like the zombie argument) entails that there is such a dichotomy.
- mannykannot 2y ago
  Why would these viewpoints be regarded as mutually exclusive? What's wrong with X and Y being functionally equivalent in every respect, yet different in implementation? See the last paragraph of my reply to goatlover for why I feel that identity is a red herring.