3 ms·

> I know it doesn't experience joy or surprise or empathy. You don't know. You strongly feel that you know. There's no standard procedure to assess the existen

> I know it doesn't experience joy or surprise or empathy. You don't know. You strongly feel that you know. There's no standard procedure to assess the existence of phenomenal consciousness.

GMoromisato 2mo ago
Agreed. I don't even know if you, or anyone else, feels joy. Maybe I'm the only person in the world with qualia. I certainly can't prove otherwise. But with what I know of people and what I know of matrix multiplication, I have a strong suspicion.
- red75prime 2mo ago
  I wasn't going solely for "we don't know and we can't know." There are other relevant observations. A matmul (or a fixed finite sequence of matmuls) is not Turing-complete. An LLM that produces answer in the number of steps that is proportional to the length of the answer is not Turing-complete. An LLM with CoT that produces answer after a number of steps, which is determined by the LLM itself, is Turing-complete. So, we do have a qualitative step between a matmul and modern LLMs. Of course, it doesn't guaranty having feelings, but it doesn't allow to simply dismiss the notion due to sheer computational inadequacy.
  - adlpz 2mo ago
    Thanks for this comment. It's an angle worth remembering. A machine is not turing complete, but by running it recursively on its own output, turing completeness emerges. Who knows at what stage of intricacy consciousness emerges? Arguably, either it does at some point, or it never does and we requiere the supernatural soul. So, if it does, we can't disregard the possibility that we've reached it already.
    - owebmaster 2mo ago
      Right. Is it conscious when it is generating the response? And it's consciousness resets on every prompt?
      - red75prime 2mo ago
        Its state is fully defined by its context window. So, yes, if the context is reset, then its state is reset too.
keicjwdjwj 2mo ago
Nice philosophical loop hole. Like a first year’s philosophy major trying to sound smart in front of the class. Please. You know it doesn’t experience joy, surprise or empathy too. You absolutely, undoubtedly know that.
- oenton 2mo ago
  That’s an insult to first year philosophy students.
  - Thanemate 2mo ago
    What OP said is an insult to first year philosophy students who try too hard to sound smarter than everyone else. You're reading too much into it.
- red75prime 2mo ago
  > You absolutely, undoubtedly know that. First things first. You are wrong. I don't have a gut feeling that LLMs have no phenomenal states. Not anymore. I was skeptical that LLMs can form complex internal representations. Mechanistic interpretability has shown that they can. I'm still skeptical in regard to introspective mechanisms. One token being fed back into the network and KV cache update do seem lacking in comparison to the ubiquitous recurrent connections in the brain. What I do know is that philosophy hasn't produced an operationalized definition of qualia. So, we need to approach the problem of LLMs' phenomenal states from other angles. For example, we can argue that LLMs lack something, which seems to be relevant to the problem: a biological substrate, generality (non-CoT LLMs aren't Turing-complete), "something something feedback"[1], and things like that. "Admit that you know that" is boring. [1] I'm referring to https://www.astralcodexten.com/p/the-new-ai-consciousness-paper?utm_source=chatgpt.com https://www.astralcodexten.com/p/the-new-ai-consciousness-pa...
  - isjejxjwjdj 2mo ago
    > "Admit that you know that" is boring. Reality is under no obligation to be fun. It might be boring but it is the truth. The fact that you needed such effort to try and give this position any shred of sense only serves as proof of how nonsensical it is to do so.

> I know it doesn't experience joy or surprise or empathy. You don't know. You strongly feel that you know. There's no standard procedure to assess the existence of phenomenal consciousness.

GMoromisato 2mo ago

Agreed. I don't even know if you, or anyone else, feels joy. Maybe I'm the only person in the world with qualia. I certainly can't prove otherwise. But with what I know of people and what I know of matrix multiplication, I have a strong suspicion.

red75prime 2mo ago
I wasn't going solely for "we don't know and we can't know." There are other relevant observations. A matmul (or a fixed finite sequence of matmuls) is not Turing-complete. An LLM that produces answer in the number of steps that is proportional to the length of the answer is not Turing-complete. An LLM with CoT that produces answer after a number of steps, which is determined by the LLM itself, is Turing-complete. So, we do have a qualitative step between a matmul and modern LLMs. Of course, it doesn't guaranty having feelings, but it doesn't allow to simply dismiss the notion due to sheer computational inadequacy.
- adlpz 2mo ago
  Thanks for this comment. It's an angle worth remembering. A machine is not turing complete, but by running it recursively on its own output, turing completeness emerges. Who knows at what stage of intricacy consciousness emerges? Arguably, either it does at some point, or it never does and we requiere the supernatural soul. So, if it does, we can't disregard the possibility that we've reached it already.
  - owebmaster 2mo ago
    Right. Is it conscious when it is generating the response? And it's consciousness resets on every prompt?
    - red75prime 2mo ago
      Its state is fully defined by its context window. So, yes, if the context is reset, then its state is reset too.

keicjwdjwj 2mo ago

Nice philosophical loop hole. Like a first year’s philosophy major trying to sound smart in front of the class. Please. You know it doesn’t experience joy, surprise or empathy too. You absolutely, undoubtedly know that.

oenton 2mo ago
That’s an insult to first year philosophy students.
- Thanemate 2mo ago
  What OP said is an insult to first year philosophy students who try too hard to sound smarter than everyone else. You're reading too much into it.
red75prime 2mo ago
> You absolutely, undoubtedly know that. First things first. You are wrong. I don't have a gut feeling that LLMs have no phenomenal states. Not anymore. I was skeptical that LLMs can form complex internal representations. Mechanistic interpretability has shown that they can. I'm still skeptical in regard to introspective mechanisms. One token being fed back into the network and KV cache update do seem lacking in comparison to the ubiquitous recurrent connections in the brain. What I do know is that philosophy hasn't produced an operationalized definition of qualia. So, we need to approach the problem of LLMs' phenomenal states from other angles. For example, we can argue that LLMs lack something, which seems to be relevant to the problem: a biological substrate, generality (non-CoT LLMs aren't Turing-complete), "something something feedback"[1], and things like that. "Admit that you know that" is boring. [1] I'm referring to https://www.astralcodexten.com/p/the-new-ai-consciousness-paper?utm_source=chatgpt.com https://www.astralcodexten.com/p/the-new-ai-consciousness-pa...
- isjejxjwjdj 2mo ago
  > "Admit that you know that" is boring. Reality is under no obligation to be fun. It might be boring but it is the truth. The fact that you needed such effort to try and give this position any shred of sense only serves as proof of how nonsensical it is to do so.