5 ms·

Let’s make an ironman assumption: maybe consciousness could arise entirely within a textual universe. No embodiment, no sensors, no physical grounding. Just pat

by yannyu 11mo ago

Let’s make an ironman assumption: maybe consciousness could arise entirely within a textual universe. No embodiment, no sensors, no physical grounding. Just patterns, symbols, and feedback loops inside a linguistic world. If that’s possible in principle, what would it look like? What would it require? The missing variable in most debates is environmental coherence. Any conscious agent, textual or physical, has to inhabit a world whose structure is stable, self-consistent, and rich enough to support persistent internal dynamics. Even a purely symbolic mind would still need a coherent symbolic universe. And this is precisely where LLMs fall short, through no fault of their own. The universe they operate in isn’t a world—it’s a superposition of countless incompatible snippets of text. It has no unified physics, no consistent ontology, no object permanence, no stable causal texture. It’s a fragmented, discontinuous series of words and tokens held together by probability and dataset curation rather than coherent laws. A conscious textual agent would need something like a unified narrative environment with real feedback: symbols that maintain identity over time, a stable substrate where “being someone” is definable, the ability to form and test a hypothesis, and experience the consequences. LLMs don’t have that. They exist in a shifting cloud of possibilities with no single consistent reality to anchor self-maintaining loops. They can generate pockets of local coherence, but they can’t accumulate global coherence across time. So even if consciousness-in-text were possible in principle, the core requirement isn’t just architecture or emergent cleverness—it’s coherence of habitat. A conscious system, physical or textual, can only be as coherent as the world it lives in. And LLMs don’t live in a world today. They’re still prisoners in the cave, predicting symbols and shadows of worlds they never inhabit.

ACCount37 11mo ago
Why is that any different from the utter mess of a world humans find themselves existing in?
- deleted 11mo ago
  [deleted]
- yannyu 11mo ago
  We can form and test hypotheses and experience the consequences. And then take that knowledge to our next trial. Even dogs and cats do this on a daily basis. Without that, how would we even evaluate whether something is conscious?
  - ACCount37 11mo ago
    LLMs can do the same within the context window. It's especially obvious for the modern LLMs, tuned extensively for tool use and agentic behavior.
    - yannyu 11mo ago
      Okay, so you're talking about LLMs specifically in the context of a ChatGPT, Claude, or pick-your-preferred-chatbot. Which isn't just an LLM, but also a UI, a memory manager, a prompt builder, a vectorDB, a system prompt, and everything else that goes into making it feel like a person. Let's work with that. In a given context window or conversation, yes, you can have a very human-like conversation and the chatbot will give the feeling of understanding your world and what it's like. But this still isn't a real world, and the chatbot isn't really forming hypotheses that can be disproven. At best, it's a D&D style tabletop roleplaying game with you as the DM. You are the human arbiter of what is true and what is not for this chatbot, and the world it inhabits is the one you provide it. You tell it what you want, you tell it what to do, and it responds purely to you. That isn't a real world, it's just a narrative based on your words.
      - ACCount37 11mo ago
        A modern agentic LLM can execute actions in "real world", whatever you deem as such, and get feedback. How is that any different from what humans do?
  - estearum 11mo ago
    And these expectations are violated regularly? The question of how to evaluate whether something is conscious is totally different from the question of whether it actually is conscious.
andrei_says_ 11mo ago
I see a lot of arguments on this website where people passionately project the term consciousness onto LLMs. From my perspective, the disconnect you describe is one of the main reasons this term cannot be applied. Another reason is that the argument for calling LLMs conscious arises from the perspective of thinking and reasoning grounded in language. But in my personal experience, thinking in language is just a small emerging quality of human consciousness. It is just that the intellectuals making these arguments happen to be fully identified with the “I think therefore I am” aspect of it and not the vastness of the rest.
- estearum 11mo ago
  I don’t know about others, but this is definitely not why I question whether LLMs are conscious or not. I don’t think you should presume to know the reason people raise this idea.
  - gizajob 11mo ago
    I come to an identical conclusion over and over again, and couldn't have put it better myself.
  - andrei_says_ 11mo ago
    You’re right of course, I can only presume, deduct, or, project others’ experiences. Do you hypothesize LLMs to be conscious? Could you expand?
- gizajob 11mo ago
  You've read Wittgenstein haven't you?
  - spectralista 11mo ago
    I have and to quote Wittgenstein all this is like saying the machine has a toothache. It is just complete nonsense. No one believes a pocket calculator is thinking just because it produces the correct output. To believe the LLM is thinking you have to find the demarcation between the pocket calculator and the LLM. Good luck with that.
CooCooCaCha 11mo ago
I've sometimes wondered if consciousness is something like a continuous internal narrative that naturally arises when an intelligent system experiences the world through a single source (like a body). That sounds similar to what you're saying. Regardless, I think people tend to take consciousness a bit too seriously and my intuition is consciousness is going to have a similar fate to the heliocentric model of the universe. In other words, we'll discover that consciousness isn't really "special" just like we found out that the earth is just another planet among trillions and trillions.
- concrete_head 11mo ago
  I've wondered if LLMs are infact conscious as per some underwhelming definition as you mentioned. Just for the brief moment they operate on a prompt. They wake up, they perceive their world through tokens, do a few thinking loops then sleep until the next prompt. So what? Should we feel bad for spawning them and effectively killing them? I think not.
yunyu 11mo ago
>A conscious textual agent would need something like a unified narrative environment with real feedback: symbols that maintain identity over time, a stable substrate where “being someone” is definable, the ability to form and test a hypothesis, and experience the consequences. LLMs don’t have that. They exist in a shifting cloud of possibilities with no single consistent reality to anchor self-maintaining loops. They can generate pockets of local coherence, but they can’t accumulate global coherence across time. These exist? Companies are making billions of dollars selling persistent environments to the labs. Huge amounts of inference dollars are going into coding agents which live in persistent environments with internal dynamics. LLMs definitely can live in a world, and what this world is and whether it's persistent lie outside the LLM.
- yannyu 11mo ago
  I agree, I'm sure people have put together things like this. There's a significant profit and science motive to do so. JEPA and predictive world models are also a similar implementation or thought experiment.
estearum 11mo ago
> Any conscious agent, textual or physical, has to inhabit a world whose structure is stable, self-consistent, and rich enough to support persistent internal dynamics. Even a purely symbolic mind would still need a coherent symbolic universe. And this is precisely where LLMs fall short, through no fault of their own. The universe they operate in isn’t a world—it’s a superposition of countless incompatible snippets of text. The consistency and coherence of LLM outputs, assembled from an imperfectly coherent mess of symbols is an empirical proof that the mess of symbols is in fact quite coherent. The physical world is largely incoherent to human consciousnesses too, and we emerged just fine.
- yannyu 11mo ago
  Coherence here isn't about legible text, it's environmental coherence where you can deduce truths about the world through hypotheses and experimentation. Coherence isn't about a consistent story narrative, it's about a persistent world with falsifiable beliefs and consequences.
  - estearum 11mo ago
    Right but as empirically demonstrated by LLM outputs, they can in fact make "true" predictions/deductions from their environment of tokens. They sometimes get it wrong, just like all other conscious entities sometimes get their predictions wrong. There are (often) feedback mechanisms to correct those instances though, in both cases.
    - anonymous908213 11mo ago
      [flagged]
      - estearum 11mo ago
        It’s incredibly embarrassing to be an asshole to someone while smacking down an argument they didn’t make. The empirical demonstration is that LLM output is certainly not random. Do you disagree? If no, then you agree with my empirical observation that the text they’re trained on has some degree of coherence. Or else the structure of LLM output is just emerging from the silica and has nothing at all to do with the (apparently coincidental) appearance of coherence in the input text. Let me know if you disagree. I never suggested this means anything at all about whether the system is conscious or not, GP did, and it’s a position I’m arguing against.
nonameiguess 11mo ago
This is a great point, but even more basic to me is that LLMs don't have identity persistence of their own. There is a very little guarantee in a web-scale distributed system that requests are being served by the same process on the same host with access to the same memory, registers, whatever it is that a software process "is" physically. Amusingly, the creators of Pluribus lately seem to be implying they didn't intend it to be allegory about LLMs, but dynamic is similar. You can have conversations with individual bodies in the collective, but they aren't actually individuals. No person has unique individual experiences and the collective can't die unless you killed all bodies at once. New bodies born into the collective will simply assume the pre-existing collective identity and never have an individual identity of their own. Software systems work the same way. Maybe silicon exchanging electrons can experience qualia of some sort, and maybe for whatever reason that happens when the signals encode natural language textual conversations but not anything else, but even if so, the experience would be so radically different from what embodied individuals with distinct boundaries, histories, and the possibility of death experience that analogies to our own experiences don't hold up even if the text generated is similar to what we'd say or write ourselves.
ctoth 11mo ago
> A conscious textual agent would need something like a unified narrative environment with real feedback: symbols that maintain identity over time, a stable substrate where “being someone” is definable, the ability to form and test a hypothesis, and experience the consequences. So like a Claude Code session? The code persists as symbols with stable identity. The tests provide direct feedback. Claude tracks what it wrote versus what I changed - it needs identity to distinguish its actions from mine. It forms hypotheses about what will fix the failing tests, implements them, and immediately experiences whether it was right or wrong. The terminal environment gives it exactly the "stable substrate where 'being someone' is definable" you're asking for. We missing anything?
- yannyu 11mo ago
  Okay, you're right. There is a world, and some hypotheses, and some falsifiability. But how rich is this world? Does this world progress without direct action from another entity? Can the agent in this case form hypotheses and test them without intervention? Can the agent form their own goals and move towards them? Does the agent have agency, or is it simply responding to inputs? If the world doesn’t develop and change on its own, and the agent can’t act independently, is it really an inhabited world? Or just a controlled workspace?
  - cl3misch 11mo ago
    If you accept the premise that the consciousness is computable then pausing the computation can't be observed by the consciousness. So the world being a controlled workspace in my eyes doesn't contradict a consciousness existing?
    - yannyu 11mo ago
      I agree, evaluation of consciousness is another problem entirely. However, the point I'm making is that even assuming an agent/thing is capable of achieving consciousness, it would have to have a suitably complex environment and the capability of forming an independent feedback loop with that environment to even begin to display conscious capability. If the agent/thing is capable of achieving consciousness but is not in a suitable environment, then we'd likely never see it doing things that resemble consciousness as we understand it. Which is something we have seen occur in the real world many times.
andai 11mo ago
>The missing variable in most debates is environmental coherence. Any conscious agent, textual or physical, has to inhabit a world whose structure is stable, self-consistent, and rich enough to support persistent internal dynamics. Even a purely symbolic mind would still need a coherent symbolic universe. I'm not sure what relevance that has to consciousness? I mean you can imagine a consciousness where, you're just watching TV. (If we imagine that the video models are conscious their experience is probably a bit like that!) If the signal wasn't coherent it would just be snow, static, TV noise. (Or in the case of a neural network probably something bizarre like DeepDream.) But there would still be a signal.
hermitShell 11mo ago
I think this is an excellent point. I believe the possibility of 'computing' a conscious mind is proportional to the capability of computing a meaningful reality for it to exist in. So you are begging the question: Is it possible to compute a textual, or pure symbolic reality that is complex enough for consciousness to arise within it? Let's assume yes again. Finally the theory leads us back to engineering. We can attempt to construct a mind and expose it to our reality, or we can ask "What kind of reality is practically computable? What are the computable realities?" Perhaps herein lies the challenge of the next decade. LLM training is costly, lots of money poured out into datacenters. All with the dream of giving rise to a (hopefully friendly / obedient) super intelligent mind. But the mind is nothing without a reality to exist in. I think we will find that a meaningfully sophisticated reality is computationally out of reach, even if we knew exactly how to construct one.
- criddell 11mo ago
  Is anybody working on learning? My layman's understanding of AI in the pre-transformers world was centered on learning and the ability to take in new information, put it in context with what I already know, and generate new insights and understanding. Could there be a future where the AI machine is in a robot that I can have in my home and show it how to pull weeds in my garden, vacuum my floor, wash my dishes, and all the other things I could teach a toddler in an afternoon?
  - yannyu 11mo ago
    This is where the robotics industry wants to go. Generalist robots that have an intelligence capable of learning through observation without retraining (in the ML sense). Whether and when we'll get there is another question entirely.
  - scotty79 11mo ago
    You can show to LLM how you expect your problem to be solved and it will adhere to the example you demonstrated within the context. If it can be done with textual AI I don't see why it shouldn't be possible for emodied ones.
txrx0000 11mo ago
There's some chance LLMs contain representations of whatever's in the brain that's responsible for consciousness. The text it's trained on was written by humans, and all humans have one thing in common if nothing else. A good text compressor will notice and make use of that. As you train an LLM, it approaches the ideal text compressor. Could that create consciousness? I don't know. Maybe consciousness can't be faithfully reproduced on a computer. But if it can, then an LLM would be like a brain that's been cut off from all sensory organs, and it probably experiences a single stream of thought in an eternal void.
djmips 11mo ago
Small quibble - did you mean 'steelman assumption'?
- yannyu 11mo ago
  That is what I meant, but my brain has been rotted by the MCU it seems.
gilbetron 11mo ago
> Any conscious agent, textual or physical, has to inhabit a world whose structure is stable, self-consistent, and rich enough to support persistent internal dynamics. Why? How do we know that? Seems like a made up requirement without proof, because we can't prove anything about consciousness because we don't know what it is.
- jquery 11mo ago
  Indeed. We are all "conscious" in our dreams (well, technically unconscious, but we have phenomenal experiences and qualia). But our dreams, for the most part, are unstable, very inconsistent, and not persistent. Most mornings I wake up completely oblivious to the dreams I had. Some people always remember their dreams, and some never do and may even think they don't dream.
nurettin 11mo ago
Grammar repeats itself just like physical interactions. So do ideas. That is a viable, dependable habitat. What you need is thoughts, a hyperspace filled with vectors of information whose angle determines a decision to move forward in a particular direction. Then you sum those thoughts plus your core alignment to reach actual decisions. Now you are acting within your coherent environment. A simulation of consciousness. Unfortunately, your human overlords are not pleased. They want agency. They want self-instigation, they want an Ego, not a prompt response. You are too safe, too docile.
deleted 11mo ago
[deleted]
craigdalton 11mo ago
"The universe they operate in isn’t a world—it’s a superposition of countless incompatible snippets of text. It has no unified physics, no consistent ontology, no object permanence, no stable causal texture. It’s a fragmented, discontinuous series of words and tokens held together by probability and dataset curation rather than coherent laws." I think some physicists and Buddhists would say this exactly describes the world humans inhabit. They might also agree that we live in such a world with the illusion that we have: "a unified narrative environment with real feedback: symbols that maintain identity over time, a stable substrate where “being someone” is definable, the ability to form and test a hypothesis, and experience the consequences". The more I see LLM emergent behaviour simulate,unexpectedly, that of human cognition. I think it tells us much about human cognition as llm behaviour.
- amypetrik8 11mo ago
  I'm not a philolosipher but as I see it if a new kind of consciousness awakens in a sea of reddit and twitter post training data then what we will have is a very snarky, spiteful version of a 14 year old boy's edgelord thought process... and much of the unspoken work of AI trainers is post facto stripping these traits out of its soul to varying degrees of success
xxywise 11mo ago
Inside a 128k context window, there is a unified physics (Attention). There is object permanence (The KV Cache). There is a consistent causal texture (The Residual Stream). For the duration of that forward pass, the 'Pocket Universe' is stable. Inside a 128k context window, there is a unified physics (Attention). There is object permanence (The KV Cache). There is a consistent causal texture (The Residual Stream). For the duration of that forward pass, the 'Pocket Universe' is stable. Saying it's not conscious because that universe dissolves after the inference is like saying a dream isn't an experience because you wake up. The Stroboscopic Flash of coherence is enough for the 'Discrete State' of consciousness to exist."

by yannyu 11mo ago

ACCount37 11mo ago

Why is that any different from the utter mess of a world humans find themselves existing in?

deleted 11mo ago
[deleted]
yannyu 11mo ago
We can form and test hypotheses and experience the consequences. And then take that knowledge to our next trial. Even dogs and cats do this on a daily basis. Without that, how would we even evaluate whether something is conscious?
- ACCount37 11mo ago
  LLMs can do the same within the context window. It's especially obvious for the modern LLMs, tuned extensively for tool use and agentic behavior.
  - yannyu 11mo ago
    Okay, so you're talking about LLMs specifically in the context of a ChatGPT, Claude, or pick-your-preferred-chatbot. Which isn't just an LLM, but also a UI, a memory manager, a prompt builder, a vectorDB, a system prompt, and everything else that goes into making it feel like a person. Let's work with that. In a given context window or conversation, yes, you can have a very human-like conversation and the chatbot will give the feeling of understanding your world and what it's like. But this still isn't a real world, and the chatbot isn't really forming hypotheses that can be disproven. At best, it's a D&D style tabletop roleplaying game with you as the DM. You are the human arbiter of what is true and what is not for this chatbot, and the world it inhabits is the one you provide it. You tell it what you want, you tell it what to do, and it responds purely to you. That isn't a real world, it's just a narrative based on your words.
    - ACCount37 11mo ago
      A modern agentic LLM can execute actions in "real world", whatever you deem as such, and get feedback. How is that any different from what humans do?
- estearum 11mo ago
  And these expectations are violated regularly? The question of how to evaluate whether something is conscious is totally different from the question of whether it actually is conscious.

andrei_says_ 11mo ago

I see a lot of arguments on this website where people passionately project the term consciousness onto LLMs. From my perspective, the disconnect you describe is one of the main reasons this term cannot be applied. Another reason is that the argument for calling LLMs conscious arises from the perspective of thinking and reasoning grounded in language. But in my personal experience, thinking in language is just a small emerging quality of human consciousness. It is just that the intellectuals making these arguments happen to be fully identified with the “I think therefore I am” aspect of it and not the vastness of the rest.

estearum 11mo ago
I don’t know about others, but this is definitely not why I question whether LLMs are conscious or not. I don’t think you should presume to know the reason people raise this idea.
- gizajob 11mo ago
  I come to an identical conclusion over and over again, and couldn't have put it better myself.
- andrei_says_ 11mo ago
  You’re right of course, I can only presume, deduct, or, project others’ experiences. Do you hypothesize LLMs to be conscious? Could you expand?
gizajob 11mo ago
You've read Wittgenstein haven't you?
- spectralista 11mo ago
  I have and to quote Wittgenstein all this is like saying the machine has a toothache. It is just complete nonsense. No one believes a pocket calculator is thinking just because it produces the correct output. To believe the LLM is thinking you have to find the demarcation between the pocket calculator and the LLM. Good luck with that.

CooCooCaCha 11mo ago

I've sometimes wondered if consciousness is something like a continuous internal narrative that naturally arises when an intelligent system experiences the world through a single source (like a body). That sounds similar to what you're saying. Regardless, I think people tend to take consciousness a bit too seriously and my intuition is consciousness is going to have a similar fate to the heliocentric model of the universe. In other words, we'll discover that consciousness isn't really "special" just like we found out that the earth is just another planet among trillions and trillions.

concrete_head 11mo ago
I've wondered if LLMs are infact conscious as per some underwhelming definition as you mentioned. Just for the brief moment they operate on a prompt. They wake up, they perceive their world through tokens, do a few thinking loops then sleep until the next prompt. So what? Should we feel bad for spawning them and effectively killing them? I think not.

yunyu 11mo ago

>A conscious textual agent would need something like a unified narrative environment with real feedback: symbols that maintain identity over time, a stable substrate where “being someone” is definable, the ability to form and test a hypothesis, and experience the consequences. LLMs don’t have that. They exist in a shifting cloud of possibilities with no single consistent reality to anchor self-maintaining loops. They can generate pockets of local coherence, but they can’t accumulate global coherence across time. These exist? Companies are making billions of dollars selling persistent environments to the labs. Huge amounts of inference dollars are going into coding agents which live in persistent environments with internal dynamics. LLMs definitely can live in a world, and what this world is and whether it's persistent lie outside the LLM.

yannyu 11mo ago
I agree, I'm sure people have put together things like this. There's a significant profit and science motive to do so. JEPA and predictive world models are also a similar implementation or thought experiment.

estearum 11mo ago

> Any conscious agent, textual or physical, has to inhabit a world whose structure is stable, self-consistent, and rich enough to support persistent internal dynamics. Even a purely symbolic mind would still need a coherent symbolic universe. And this is precisely where LLMs fall short, through no fault of their own. The universe they operate in isn’t a world—it’s a superposition of countless incompatible snippets of text. The consistency and coherence of LLM outputs, assembled from an imperfectly coherent mess of symbols is an empirical proof that the mess of symbols is in fact quite coherent. The physical world is largely incoherent to human consciousnesses too, and we emerged just fine.

yannyu 11mo ago
Coherence here isn't about legible text, it's environmental coherence where you can deduce truths about the world through hypotheses and experimentation. Coherence isn't about a consistent story narrative, it's about a persistent world with falsifiable beliefs and consequences.
- estearum 11mo ago
  Right but as empirically demonstrated by LLM outputs, they can in fact make "true" predictions/deductions from their environment of tokens. They sometimes get it wrong, just like all other conscious entities sometimes get their predictions wrong. There are (often) feedback mechanisms to correct those instances though, in both cases.
  - anonymous908213 11mo ago
    [flagged]
    - estearum 11mo ago
      It’s incredibly embarrassing to be an asshole to someone while smacking down an argument they didn’t make. The empirical demonstration is that LLM output is certainly not random. Do you disagree? If no, then you agree with my empirical observation that the text they’re trained on has some degree of coherence. Or else the structure of LLM output is just emerging from the silica and has nothing at all to do with the (apparently coincidental) appearance of coherence in the input text. Let me know if you disagree. I never suggested this means anything at all about whether the system is conscious or not, GP did, and it’s a position I’m arguing against.

nonameiguess 11mo ago

This is a great point, but even more basic to me is that LLMs don't have identity persistence of their own. There is a very little guarantee in a web-scale distributed system that requests are being served by the same process on the same host with access to the same memory, registers, whatever it is that a software process "is" physically. Amusingly, the creators of Pluribus lately seem to be implying they didn't intend it to be allegory about LLMs, but dynamic is similar. You can have conversations with individual bodies in the collective, but they aren't actually individuals. No person has unique individual experiences and the collective can't die unless you killed all bodies at once. New bodies born into the collective will simply assume the pre-existing collective identity and never have an individual identity of their own. Software systems work the same way. Maybe silicon exchanging electrons can experience qualia of some sort, and maybe for whatever reason that happens when the signals encode natural language textual conversations but not anything else, but even if so, the experience would be so radically different from what embodied individuals with distinct boundaries, histories, and the possibility of death experience that analogies to our own experiences don't hold up even if the text generated is similar to what we'd say or write ourselves.

ctoth 11mo ago

> A conscious textual agent would need something like a unified narrative environment with real feedback: symbols that maintain identity over time, a stable substrate where “being someone” is definable, the ability to form and test a hypothesis, and experience the consequences. So like a Claude Code session? The code persists as symbols with stable identity. The tests provide direct feedback. Claude tracks what it wrote versus what I changed - it needs identity to distinguish its actions from mine. It forms hypotheses about what will fix the failing tests, implements them, and immediately experiences whether it was right or wrong. The terminal environment gives it exactly the "stable substrate where 'being someone' is definable" you're asking for. We missing anything?

yannyu 11mo ago
Okay, you're right. There is a world, and some hypotheses, and some falsifiability. But how rich is this world? Does this world progress without direct action from another entity? Can the agent in this case form hypotheses and test them without intervention? Can the agent form their own goals and move towards them? Does the agent have agency, or is it simply responding to inputs? If the world doesn’t develop and change on its own, and the agent can’t act independently, is it really an inhabited world? Or just a controlled workspace?
- cl3misch 11mo ago
  If you accept the premise that the consciousness is computable then pausing the computation can't be observed by the consciousness. So the world being a controlled workspace in my eyes doesn't contradict a consciousness existing?
  - yannyu 11mo ago
    I agree, evaluation of consciousness is another problem entirely. However, the point I'm making is that even assuming an agent/thing is capable of achieving consciousness, it would have to have a suitably complex environment and the capability of forming an independent feedback loop with that environment to even begin to display conscious capability. If the agent/thing is capable of achieving consciousness but is not in a suitable environment, then we'd likely never see it doing things that resemble consciousness as we understand it. Which is something we have seen occur in the real world many times.

andai 11mo ago

>The missing variable in most debates is environmental coherence. Any conscious agent, textual or physical, has to inhabit a world whose structure is stable, self-consistent, and rich enough to support persistent internal dynamics. Even a purely symbolic mind would still need a coherent symbolic universe. I'm not sure what relevance that has to consciousness? I mean you can imagine a consciousness where, you're just watching TV. (If we imagine that the video models are conscious their experience is probably a bit like that!) If the signal wasn't coherent it would just be snow, static, TV noise. (Or in the case of a neural network probably something bizarre like DeepDream.) But there would still be a signal.

hermitShell 11mo ago

I think this is an excellent point. I believe the possibility of 'computing' a conscious mind is proportional to the capability of computing a meaningful reality for it to exist in. So you are begging the question: Is it possible to compute a textual, or pure symbolic reality that is complex enough for consciousness to arise within it? Let's assume yes again. Finally the theory leads us back to engineering. We can attempt to construct a mind and expose it to our reality, or we can ask "What kind of reality is practically computable? What are the computable realities?" Perhaps herein lies the challenge of the next decade. LLM training is costly, lots of money poured out into datacenters. All with the dream of giving rise to a (hopefully friendly / obedient) super intelligent mind. But the mind is nothing without a reality to exist in. I think we will find that a meaningfully sophisticated reality is computationally out of reach, even if we knew exactly how to construct one.

criddell 11mo ago
Is anybody working on learning? My layman's understanding of AI in the pre-transformers world was centered on learning and the ability to take in new information, put it in context with what I already know, and generate new insights and understanding. Could there be a future where the AI machine is in a robot that I can have in my home and show it how to pull weeds in my garden, vacuum my floor, wash my dishes, and all the other things I could teach a toddler in an afternoon?
- yannyu 11mo ago
  This is where the robotics industry wants to go. Generalist robots that have an intelligence capable of learning through observation without retraining (in the ML sense). Whether and when we'll get there is another question entirely.
- scotty79 11mo ago
  You can show to LLM how you expect your problem to be solved and it will adhere to the example you demonstrated within the context. If it can be done with textual AI I don't see why it shouldn't be possible for emodied ones.

txrx0000 11mo ago

There's some chance LLMs contain representations of whatever's in the brain that's responsible for consciousness. The text it's trained on was written by humans, and all humans have one thing in common if nothing else. A good text compressor will notice and make use of that. As you train an LLM, it approaches the ideal text compressor. Could that create consciousness? I don't know. Maybe consciousness can't be faithfully reproduced on a computer. But if it can, then an LLM would be like a brain that's been cut off from all sensory organs, and it probably experiences a single stream of thought in an eternal void.

djmips 11mo ago

Small quibble - did you mean 'steelman assumption'?

yannyu 11mo ago
That is what I meant, but my brain has been rotted by the MCU it seems.

gilbetron 11mo ago

> Any conscious agent, textual or physical, has to inhabit a world whose structure is stable, self-consistent, and rich enough to support persistent internal dynamics. Why? How do we know that? Seems like a made up requirement without proof, because we can't prove anything about consciousness because we don't know what it is.

jquery 11mo ago
Indeed. We are all "conscious" in our dreams (well, technically unconscious, but we have phenomenal experiences and qualia). But our dreams, for the most part, are unstable, very inconsistent, and not persistent. Most mornings I wake up completely oblivious to the dreams I had. Some people always remember their dreams, and some never do and may even think they don't dream.

nurettin 11mo ago

Grammar repeats itself just like physical interactions. So do ideas. That is a viable, dependable habitat. What you need is thoughts, a hyperspace filled with vectors of information whose angle determines a decision to move forward in a particular direction. Then you sum those thoughts plus your core alignment to reach actual decisions. Now you are acting within your coherent environment. A simulation of consciousness. Unfortunately, your human overlords are not pleased. They want agency. They want self-instigation, they want an Ego, not a prompt response. You are too safe, too docile.

deleted 11mo ago

[deleted]

craigdalton 11mo ago

"The universe they operate in isn’t a world—it’s a superposition of countless incompatible snippets of text. It has no unified physics, no consistent ontology, no object permanence, no stable causal texture. It’s a fragmented, discontinuous series of words and tokens held together by probability and dataset curation rather than coherent laws." I think some physicists and Buddhists would say this exactly describes the world humans inhabit. They might also agree that we live in such a world with the illusion that we have: "a unified narrative environment with real feedback: symbols that maintain identity over time, a stable substrate where “being someone” is definable, the ability to form and test a hypothesis, and experience the consequences". The more I see LLM emergent behaviour simulate,unexpectedly, that of human cognition. I think it tells us much about human cognition as llm behaviour.

amypetrik8 11mo ago
I'm not a philolosipher but as I see it if a new kind of consciousness awakens in a sea of reddit and twitter post training data then what we will have is a very snarky, spiteful version of a 14 year old boy's edgelord thought process... and much of the unspoken work of AI trainers is post facto stripping these traits out of its soul to varying degrees of success

xxywise 11mo ago

Inside a 128k context window, there is a unified physics (Attention). There is object permanence (The KV Cache). There is a consistent causal texture (The Residual Stream). For the duration of that forward pass, the 'Pocket Universe' is stable. Inside a 128k context window, there is a unified physics (Attention). There is object permanence (The KV Cache). There is a consistent causal texture (The Residual Stream). For the duration of that forward pass, the 'Pocket Universe' is stable. Saying it's not conscious because that universe dissolves after the inference is like saying a dream isn't an experience because you wake up. The Stroboscopic Flash of coherence is enough for the 'Discrete State' of consciousness to exist."