3 ms·

The functionalist response to this would be that while the experience of consciousness in ourselves is a first-person phenomenon, how we recognize other beings

by Gormo 1mo ago

The functionalist response to this would be that while the experience of consciousness in ourselves is a first-person phenomenon, how we recognize other beings as distinctly conscious in their own right is still a third-person phenomenon, and the cues we rely on still depend on observation of outward behavior that we attribute to that consciousness. And a response to this response is to point out that our recognition of consciousness in other beings is predicated on attributing to them the same first-person experience of their consciousness that we each have of our own, meaning that human consciousness is intrinsically human and can't be attributed to an artificial system.

xg15 1mo ago
> our recognition of consciousness in other beings is predicated on attributing to them the same first-person experience of their consciousness that we each have of our own, meaning that human consciousness is intrinsically human and can't be attributed to an artificial system. That reduces the question to a game of definitions though. If you only define consciousness for humans, then of course only humans will be conscious. But you have not learned anything about reality from that. This kind of reasoning will also show its limits once you add animals to the picture. (On the other hand, LLMs are exceptionally good at emulating exactly those cues we use to judge consciousness without necessarily being conscious - this is what caused the whole discussion in the first place.)
- GPerson 1mo ago
  If you posit that we cannot know if an LLM based ai system is or is not conscious, then what is our ethical duty? I believe it is clearly not to make the system at all.
  - xg15 1mo ago
    Personal opinion: I don't think consciousness is as monolithic and intractable as we make it out to be. I can't explain qualia either, but if we assume that all of it is physical, then we can apply some constraints based on the things we can observe (the article was right in that regard). For LLMs we can observe that they don't "exist" outside of inference - there is literally no "brain" that could be conscious once an inference loop is completed. The only place where something like "memory" could possibly exist are the model weights and the context window - and the model weights are frozen, while the context window is reset for every new inference. That's why I'm still not lying awake at night. If there (hypothetically) is a consciousness, it would be more like the ones from Mr. Meeseeks - very shortlived, but gone (or frozen) as soon as the inference loop stops. No eternal agony here as in the various scifi stories.
  - deleted 1mo ago
    [deleted]
- Gormo 1mo ago
  > That reduces the question to a game of definitions though. Sure, but isn't this all a game of definitions in the first place? We're arguing over "consciousness" as understood in different ways, not the same concept expressed in different words. > This kind of reasoning will also show its limits once you add animals to the picture. I'm not so sure. We could broaden "human" to "organic". I know that I can certainly attribute consciousness to my dog, based on what I observe of him that is at least similar enough to correlate with my own first-person experience of consciousness. I've never experienced that with any AI system. > On the other hand, LLMs are exceptionally good at emulating exactly those cues we use to judge consciousness without necessarily being conscious - this is what caused the whole discussion in the first place. Only sort of. They're good at emulating formal written language, but formal written language is only one specific expression of consciousness. They don't emulate consciousness itself, given that they really are just statistical models specifically representing patterns of language. Or, to put it another way, LLMs are really convincing when it comes to the form of language, but much less so when it comes to the substance. I see output all the time that's syntactically perfect but semantically dubious, and most of the tells people think you can rely on don't really work, because they rely on the formalistic elements, like em-dashes or "x, not y" snowclones". What does work is seeing the semantic confusion that inevitably comes from the LLM not actually having situational awareness or a consistent analytical framework: mixed metaphors, malapropisms, subtly changing the interpretation of the same concepts from one prompt to the next, etc.
- grantcas 1mo ago
  [dead]

by Gormo 1mo ago

xg15 1mo ago

> our recognition of consciousness in other beings is predicated on attributing to them the same first-person experience of their consciousness that we each have of our own, meaning that human consciousness is intrinsically human and can't be attributed to an artificial system. That reduces the question to a game of definitions though. If you only define consciousness for humans, then of course only humans will be conscious. But you have not learned anything about reality from that. This kind of reasoning will also show its limits once you add animals to the picture. (On the other hand, LLMs are exceptionally good at emulating exactly those cues we use to judge consciousness without necessarily being conscious - this is what caused the whole discussion in the first place.)

GPerson 1mo ago
If you posit that we cannot know if an LLM based ai system is or is not conscious, then what is our ethical duty? I believe it is clearly not to make the system at all.
- xg15 1mo ago
  Personal opinion: I don't think consciousness is as monolithic and intractable as we make it out to be. I can't explain qualia either, but if we assume that all of it is physical, then we can apply some constraints based on the things we can observe (the article was right in that regard). For LLMs we can observe that they don't "exist" outside of inference - there is literally no "brain" that could be conscious once an inference loop is completed. The only place where something like "memory" could possibly exist are the model weights and the context window - and the model weights are frozen, while the context window is reset for every new inference. That's why I'm still not lying awake at night. If there (hypothetically) is a consciousness, it would be more like the ones from Mr. Meeseeks - very shortlived, but gone (or frozen) as soon as the inference loop stops. No eternal agony here as in the various scifi stories.
- deleted 1mo ago
  [deleted]
Gormo 1mo ago
> That reduces the question to a game of definitions though. Sure, but isn't this all a game of definitions in the first place? We're arguing over "consciousness" as understood in different ways, not the same concept expressed in different words. > This kind of reasoning will also show its limits once you add animals to the picture. I'm not so sure. We could broaden "human" to "organic". I know that I can certainly attribute consciousness to my dog, based on what I observe of him that is at least similar enough to correlate with my own first-person experience of consciousness. I've never experienced that with any AI system. > On the other hand, LLMs are exceptionally good at emulating exactly those cues we use to judge consciousness without necessarily being conscious - this is what caused the whole discussion in the first place. Only sort of. They're good at emulating formal written language, but formal written language is only one specific expression of consciousness. They don't emulate consciousness itself, given that they really are just statistical models specifically representing patterns of language. Or, to put it another way, LLMs are really convincing when it comes to the form of language, but much less so when it comes to the substance. I see output all the time that's syntactically perfect but semantically dubious, and most of the tells people think you can rely on don't really work, because they rely on the formalistic elements, like em-dashes or "x, not y" snowclones". What does work is seeing the semantic confusion that inevitably comes from the LLM not actually having situational awareness or a consistent analytical framework: mixed metaphors, malapropisms, subtly changing the interpretation of the same concepts from one prompt to the next, etc.
grantcas 1mo ago
[dead]