3 ms·

To me, LLMs are living (heh) proof of the validity of Searle's Chinese Room argument about weak AI. At the time, I think a lot of the people who disputed that

To me, LLMs are living (heh) proof of the validity of Searle's Chinese Room argument about weak AI. At the time, I think a lot of the people who disputed that had the position, basically, that any program that could pass a Turing test would be so advanced in other ways that we would not be able to confidently declare it to be not conscious. The assumption was that carrying on a conversation is so difficult that only something intelligent or nearly so could accomplish it. It turns out, we can create a program that can do exactly what Searle described, and hooboy is it obviously not conscious, nowhere close. The obvious conclusion to draw is not that LLMs are intelligent, but that natural language processing isn't as difficult as we thought.

jackmott42 4y ago
What sort of features in the design of a system would need to exist for you to no longer assert that is definitely isn't conscious? Say 10 years from now there is an AI that consists of 100 different large neural networks, of which a language model is just one, along with vision/audio networks and stuff all coordinated by a coordination network. would you say "hoooboy obviously not conscious" because it is just doing matrix multiplication?
- ineptech 4y ago
  What makes it obviously not conscious is that it isn't a process. It can't do anything unless you ask it a question. I can certainly imagine a conscious or nearly-conscious AI someday, one that likely uses an LLM to handle language processing in much the same way that a cpu might use a coprocessor to do floating point operations. But just as you wouldn't say a computer is intelligent because it can do math, I think chatpgt has proven that the same can be said of processing natural language.
  - Uehreka 4y ago
    So like, if someone wrote a systemd daemon that continuously fed input to a local ChatGPT instance (passing in context from its previous interactions as well so it could act as a continuous entity) and performed actions based on its responses, you’d consider that conscious? I’m willing to buy “ChatGPT is not conscious” arguments if people can come up with a rigorous definition of what “conscious” means. But so far I don’t see a lot of people doing that, and I don’t know if it’s even possible.
    - jhrmnn 4y ago
      I’d think it’d need to be something like a diffusion model that’s continuously fed noise which it turns into outputs (thoughts). It’d also need to be getting continuous feedback and to have the ability to incorporate that feedback over all scales, not just the most recent window (some hybrid of a state and continual learning).
      - Uehreka 4y ago
        That both doesn’t seem that far off, and doesn’t feel like a very rigorous definition of consciousness. Like, if you’re describing qualities you yourself would find interesting, that’s one thing. But if we’re going to be saying “hooboy, that isn’t consciousness” with our chests, we really need an agreed-upon definition of what consciousness is that both can be met by a sufficiently advanced system and agrees with the way people use the word “consciousness” in everyday speech. Honestly, I don’t think that’s possible, but I’m willing to entertain/critique people’s proposals.
        jhrmnn 4y ago
        Yeah, that was supposed to be just my personal feeling for the thing. And it’s not even a description of consciousness but rather of something that I imagine might have a capacity for consciousness. I don’t think we’ll really know until we get there, and then we’ll know.
    - ineptech 3y ago
      I don't have a rigorous definition, but I don't think we need one at this point, do we? Chatgpt is a search engine with a natural language interface, you can draw a straight line from its inputs to its outputs. I'm not making a scientific claim here, just a subjective observation, but it seems self-evidently too simple to be hiding a self or awareness of some kind, there just isn't enough complexity there.
      - jacquesm 3y ago
        This is a very interesting discussion. One observation: I can draw a 'straight line from inputs to outputs' for almost every human interaction as well, and if you were to observe a human over a lifetime you could say that you have that straight line covered for their entire life. If you then take an LLM that you feed in all the inputs and outputs to mimic the person after how many new 'inputs' would the output of the model start to diverge too much to be able to say it is no longer accurate? And would that mean the model is 'conscious' to some lesser degree or is this a binary choice where you either are or are not conscious?

jackmott42 4y ago

What sort of features in the design of a system would need to exist for you to no longer assert that is definitely isn't conscious? Say 10 years from now there is an AI that consists of 100 different large neural networks, of which a language model is just one, along with vision/audio networks and stuff all coordinated by a coordination network. would you say "hoooboy obviously not conscious" because it is just doing matrix multiplication?

ineptech 4y ago
What makes it obviously not conscious is that it isn't a process. It can't do anything unless you ask it a question. I can certainly imagine a conscious or nearly-conscious AI someday, one that likely uses an LLM to handle language processing in much the same way that a cpu might use a coprocessor to do floating point operations. But just as you wouldn't say a computer is intelligent because it can do math, I think chatpgt has proven that the same can be said of processing natural language.
- Uehreka 4y ago
  So like, if someone wrote a systemd daemon that continuously fed input to a local ChatGPT instance (passing in context from its previous interactions as well so it could act as a continuous entity) and performed actions based on its responses, you’d consider that conscious? I’m willing to buy “ChatGPT is not conscious” arguments if people can come up with a rigorous definition of what “conscious” means. But so far I don’t see a lot of people doing that, and I don’t know if it’s even possible.
  - jhrmnn 4y ago
    I’d think it’d need to be something like a diffusion model that’s continuously fed noise which it turns into outputs (thoughts). It’d also need to be getting continuous feedback and to have the ability to incorporate that feedback over all scales, not just the most recent window (some hybrid of a state and continual learning).
    - Uehreka 4y ago
      That both doesn’t seem that far off, and doesn’t feel like a very rigorous definition of consciousness. Like, if you’re describing qualities you yourself would find interesting, that’s one thing. But if we’re going to be saying “hooboy, that isn’t consciousness” with our chests, we really need an agreed-upon definition of what consciousness is that both can be met by a sufficiently advanced system and agrees with the way people use the word “consciousness” in everyday speech. Honestly, I don’t think that’s possible, but I’m willing to entertain/critique people’s proposals.
      - jhrmnn 4y ago
        Yeah, that was supposed to be just my personal feeling for the thing. And it’s not even a description of consciousness but rather of something that I imagine might have a capacity for consciousness. I don’t think we’ll really know until we get there, and then we’ll know.
  - ineptech 3y ago
    I don't have a rigorous definition, but I don't think we need one at this point, do we? Chatgpt is a search engine with a natural language interface, you can draw a straight line from its inputs to its outputs. I'm not making a scientific claim here, just a subjective observation, but it seems self-evidently too simple to be hiding a self or awareness of some kind, there just isn't enough complexity there.
    - jacquesm 3y ago
      This is a very interesting discussion. One observation: I can draw a 'straight line from inputs to outputs' for almost every human interaction as well, and if you were to observe a human over a lifetime you could say that you have that straight line covered for their entire life. If you then take an LLM that you feed in all the inputs and outputs to mimic the person after how many new 'inputs' would the output of the model start to diverge too much to be able to say it is no longer accurate? And would that mean the model is 'conscious' to some lesser degree or is this a binary choice where you either are or are not conscious?