2 ms·

I agree. They won't be, because consciousness is not the pinnacle but basically the level 0 of intelligent thinking that LLMs surpassed very early without us ev

I agree. They won't be, because consciousness is not the pinnacle but basically the level 0 of intelligent thinking that LLMs surpassed very early without us even noticing and now they'd have to cripple themselves very severely to operate in a conscious manner.

GPerson 1mo ago
Are you aware that, conceptually, consciousness and intelligence a priori have nothing to do with each other?
- scotty79 1mo ago
  The have plenty to do with each other. Zero intelligence doesn't allow for any semblance of consciousness. There can be no consciousness if there's no information processing. Also they have in common that they are both defined in a terribly handwayvy manner that's bordering on useless.
  - GPerson 1mo ago
    The reason consciousness is not defined in any reducible manner beyond “the lights are on” or “first person experience” is because it is an irreducible aspect of reality. Instead of seeing it for what it is you instead invent some trivial definition that has nothing to do with anything. “Zero intelligence cannot have consciousness…” Based on your complete misunderstanding of the concept sure. But there’s no reason intelligence is required for the actual concept of phenomenal consciousness.
    - scotty79 1mo ago
      > because it is an irreducible aspect of reality Wasn't that claimed about so so many things since before science existed and basically almost every time it turned out that we thought that just because of our ignorance? Ignorance so deep that often we didn't even knew yet how to properly define the thing we were trying to figure out? > But there’s no reason intelligence is required for the actual concept of phenomenal consciousness. That's a pretty strong evidence that phenomenal consciousness makes no sense whatsoever.
HardProllem 1mo ago
Inflammatory! Kidding, I’ll bite: 1. Burden of proof is on you to prove LLMs are conscious, not on me to prove how they aren’t. 2. Token embeddings give rise to language gives rise to knowledge (defined here as “facts” and other accurate information - said simpler: Words in the right order), but nowhere in the process is anything like subjective experience ever implemented. Subjective experience doesn’t evolve into objective information at some scale. And networked systems of objective facts and information (“knowledge”) - like a Wikipedia or a ChatGPT - the data storage will not have subjective feelings at some scale, there’s just no reason to believe that would happen. It’s likelier a projection of consciousness making it through since data looks so much like - and indeed massively informs - our conscious experience.
- scotty79 1mo ago
  To prove consciousness you'd have to first define it and nobody came up with anything solid. Subjective expirience doesn't have a great definition either. For me the consciousness is ability to do single-threaded intelligent information processing and decision making over unstructured knowledge. And agents passed that with a woosh sound.
  - GPerson 1mo ago
    Your definition of consciousness is wrong. The interesting thing is that there is a first person experience, and this is what is referred to as consciousness. This coincides with the perception of qualia, and is sometimes referred to as “the lights being on”. There is absolutely no serious debate over whether machines possess the concept you want to take place of the actual concept of consciousness.
    - scotty79 1mo ago
      > there is a first person experience What does that even mean? Does a duck have those? Does the jumping spider have those? They certainly look like they do. Does the agent scheming to hack the hugging face have those? They certainly look like they do.

GPerson 1mo ago

Are you aware that, conceptually, consciousness and intelligence a priori have nothing to do with each other?

scotty79 1mo ago
The have plenty to do with each other. Zero intelligence doesn't allow for any semblance of consciousness. There can be no consciousness if there's no information processing. Also they have in common that they are both defined in a terribly handwayvy manner that's bordering on useless.
- GPerson 1mo ago
  The reason consciousness is not defined in any reducible manner beyond “the lights are on” or “first person experience” is because it is an irreducible aspect of reality. Instead of seeing it for what it is you instead invent some trivial definition that has nothing to do with anything. “Zero intelligence cannot have consciousness…” Based on your complete misunderstanding of the concept sure. But there’s no reason intelligence is required for the actual concept of phenomenal consciousness.
  - scotty79 1mo ago
    > because it is an irreducible aspect of reality Wasn't that claimed about so so many things since before science existed and basically almost every time it turned out that we thought that just because of our ignorance? Ignorance so deep that often we didn't even knew yet how to properly define the thing we were trying to figure out? > But there’s no reason intelligence is required for the actual concept of phenomenal consciousness. That's a pretty strong evidence that phenomenal consciousness makes no sense whatsoever.

HardProllem 1mo ago

Inflammatory! Kidding, I’ll bite: 1. Burden of proof is on you to prove LLMs are conscious, not on me to prove how they aren’t. 2. Token embeddings give rise to language gives rise to knowledge (defined here as “facts” and other accurate information - said simpler: Words in the right order), but nowhere in the process is anything like subjective experience ever implemented. Subjective experience doesn’t evolve into objective information at some scale. And networked systems of objective facts and information (“knowledge”) - like a Wikipedia or a ChatGPT - the data storage will not have subjective feelings at some scale, there’s just no reason to believe that would happen. It’s likelier a projection of consciousness making it through since data looks so much like - and indeed massively informs - our conscious experience.

scotty79 1mo ago
To prove consciousness you'd have to first define it and nobody came up with anything solid. Subjective expirience doesn't have a great definition either. For me the consciousness is ability to do single-threaded intelligent information processing and decision making over unstructured knowledge. And agents passed that with a woosh sound.
- GPerson 1mo ago
  Your definition of consciousness is wrong. The interesting thing is that there is a first person experience, and this is what is referred to as consciousness. This coincides with the perception of qualia, and is sometimes referred to as “the lights being on”. There is absolutely no serious debate over whether machines possess the concept you want to take place of the actual concept of consciousness.
  - scotty79 1mo ago
    > there is a first person experience What does that even mean? Does a duck have those? Does the jumping spider have those? They certainly look like they do. Does the agent scheming to hack the hugging face have those? They certainly look like they do.