4 ms·

> Any conscious agent, textual or physical, has to inhabit a world whose structure is stable, self-consistent, and rich enough to support persistent internal dy

> Any conscious agent, textual or physical, has to inhabit a world whose structure is stable, self-consistent, and rich enough to support persistent internal dynamics. Even a purely symbolic mind would still need a coherent symbolic universe. And this is precisely where LLMs fall short, through no fault of their own. The universe they operate in isn’t a world—it’s a superposition of countless incompatible snippets of text. The consistency and coherence of LLM outputs, assembled from an imperfectly coherent mess of symbols is an empirical proof that the mess of symbols is in fact quite coherent. The physical world is largely incoherent to human consciousnesses too, and we emerged just fine.

yannyu 1y ago
Coherence here isn't about legible text, it's environmental coherence where you can deduce truths about the world through hypotheses and experimentation. Coherence isn't about a consistent story narrative, it's about a persistent world with falsifiable beliefs and consequences.
- estearum 1y ago
  Right but as empirically demonstrated by LLM outputs, they can in fact make "true" predictions/deductions from their environment of tokens. They sometimes get it wrong, just like all other conscious entities sometimes get their predictions wrong. There are (often) feedback mechanisms to correct those instances though, in both cases.
  - anonymous908213 1y ago
    [flagged]
    - estearum 1y ago
      It’s incredibly embarrassing to be an asshole to someone while smacking down an argument they didn’t make. The empirical demonstration is that LLM output is certainly not random. Do you disagree? If no, then you agree with my empirical observation that the text they’re trained on has some degree of coherence. Or else the structure of LLM output is just emerging from the silica and has nothing at all to do with the (apparently coincidental) appearance of coherence in the input text. Let me know if you disagree. I never suggested this means anything at all about whether the system is conscious or not, GP did, and it’s a position I’m arguing against.
      - anonymous908213 1y ago
        > I never suggested this means anything at all about whether the system is conscious or not, GP did, and it’s a position I’m arguing against. That's certainly not how I perceived anything you said. Let me get this straight. u:yannyu's original comment (ironically, clearly LLM-generated) supposes that the barrier to LLM consciousness is the absence of environmental coherence. In other words, their stance is that LLMs are not currently conscious. u:estearum, that is, you, reply: > The consistency and coherence of LLM outputs, assembled from an imperfectly coherent mess of symbols is an empirical proof that the mess of symbols is in fact quite coherent. > The physical world is largely incoherent to human consciousnesses too, and we emerged just fine. In reply to a comment about world incoherence as a barrier to consciousness, you suggest that humans became conscious despite incoherence, which by extension implies that you do not think incoherence is a barrier to LLM consciousness either. u:yannyu replies to you with another probably LLM-generated comment pointing out the difference between textual coherence and environmental coherence. u:estearum, that is, you, make a reply including: > They [LLMs] sometimes get it wrong, just like all other conscious entities sometimes get their predictions wrong. "All other conscious entities" is your choice of wording here. "All other" is definitionally inclusive of the thing being other-ed. There is no possible way to correctly interpret this sentence other than suggesting that LLMs are included as a conscious entity. If you did not mean that, you misspoke, but from my perspective the only conclusion I can come to from the words you wrote is that clearly you believe in LLM consciousness and were actively arguing in favor of it, not against it. Incidentally, LLM output is not uniformly random, but rather purely probabilistic. It operates on the exact same premise as the toy script I posted, only with a massive dataset and significantly more advanced math and programming techniques. That anybody even entertains the idea of LLM consciousness is completely ridiculous to me. Regardless of whatever philosophical debates you'd like to have about whether a machine can reach consciousness, the idea that LLMs are a mechanism by which that's possible is fundamentally preposterous.
        estearum 1y ago
        > but rather purely probabilistic. In other words, indicative of a coherence - some type of structure – in the "world" of inputs into an LLM, which was the only point I was making. No, I did not intend to suggest that LLMs are conscious with my "all other conscious entities". I intended to suggest that we don't know if they're conscious, and the provided reason to "prove" that they're not is nonsense. Well, if you have a theory that cleanly proves your intuition that it's "preposterous" that an LLM could be conscious, you should go submit it and win a Nobel! Otherwise you're just sharing your unfounded and unduly confident assumptions which isn't terribly interesting.
- empath75 1y ago
  Peoples interior model of the world is very tenuously related to reality. We don't have a direct experience of waves, quantum mechanics, the vast majority of the electromagnetic spectrum, etc. The whole thing is a bunch of shortcuts and hacks that allow people to survive, the brain isn't really setup to probe reality and produce true beliefs, and the extent to which our internal models of reality naturally match actual reality is related to how much that mattered to our personal survival before the advent of civilization and writing, etc. It's really only been a very brief amount of time in human history where we had a deliberate method for trying to probe reality and create true beliefs, and I am fairly sure that if consciousness existed in humanity, it existed before the advent of the scientific method.
  - yannyu 1y ago
    I don't think it's brief at all. Animals do this experimentation as well, but clearly in different ways. The scientific method is a formalized version of this idea, but even the first human who cooked meat or used a stick as a weapon had a falsifiable hypothesis, even if it wasn't something they could express or explain. And the consequences of testing the hypothesis were something that affected the way they acted from there on out.
  - deleted 1y ago
    [deleted]

yannyu 1y ago

Coherence here isn't about legible text, it's environmental coherence where you can deduce truths about the world through hypotheses and experimentation. Coherence isn't about a consistent story narrative, it's about a persistent world with falsifiable beliefs and consequences.

estearum 1y ago
Right but as empirically demonstrated by LLM outputs, they can in fact make "true" predictions/deductions from their environment of tokens. They sometimes get it wrong, just like all other conscious entities sometimes get their predictions wrong. There are (often) feedback mechanisms to correct those instances though, in both cases.
- anonymous908213 1y ago
  [flagged]
  - estearum 1y ago
    It’s incredibly embarrassing to be an asshole to someone while smacking down an argument they didn’t make. The empirical demonstration is that LLM output is certainly not random. Do you disagree? If no, then you agree with my empirical observation that the text they’re trained on has some degree of coherence. Or else the structure of LLM output is just emerging from the silica and has nothing at all to do with the (apparently coincidental) appearance of coherence in the input text. Let me know if you disagree. I never suggested this means anything at all about whether the system is conscious or not, GP did, and it’s a position I’m arguing against.
    - anonymous908213 1y ago
      > I never suggested this means anything at all about whether the system is conscious or not, GP did, and it’s a position I’m arguing against. That's certainly not how I perceived anything you said. Let me get this straight. u:yannyu's original comment (ironically, clearly LLM-generated) supposes that the barrier to LLM consciousness is the absence of environmental coherence. In other words, their stance is that LLMs are not currently conscious. u:estearum, that is, you, reply: > The consistency and coherence of LLM outputs, assembled from an imperfectly coherent mess of symbols is an empirical proof that the mess of symbols is in fact quite coherent. > The physical world is largely incoherent to human consciousnesses too, and we emerged just fine. In reply to a comment about world incoherence as a barrier to consciousness, you suggest that humans became conscious despite incoherence, which by extension implies that you do not think incoherence is a barrier to LLM consciousness either. u:yannyu replies to you with another probably LLM-generated comment pointing out the difference between textual coherence and environmental coherence. u:estearum, that is, you, make a reply including: > They [LLMs] sometimes get it wrong, just like all other conscious entities sometimes get their predictions wrong. "All other conscious entities" is your choice of wording here. "All other" is definitionally inclusive of the thing being other-ed. There is no possible way to correctly interpret this sentence other than suggesting that LLMs are included as a conscious entity. If you did not mean that, you misspoke, but from my perspective the only conclusion I can come to from the words you wrote is that clearly you believe in LLM consciousness and were actively arguing in favor of it, not against it. Incidentally, LLM output is not uniformly random, but rather purely probabilistic. It operates on the exact same premise as the toy script I posted, only with a massive dataset and significantly more advanced math and programming techniques. That anybody even entertains the idea of LLM consciousness is completely ridiculous to me. Regardless of whatever philosophical debates you'd like to have about whether a machine can reach consciousness, the idea that LLMs are a mechanism by which that's possible is fundamentally preposterous.
      - estearum 1y ago
        > but rather purely probabilistic. In other words, indicative of a coherence - some type of structure – in the "world" of inputs into an LLM, which was the only point I was making. No, I did not intend to suggest that LLMs are conscious with my "all other conscious entities". I intended to suggest that we don't know if they're conscious, and the provided reason to "prove" that they're not is nonsense. Well, if you have a theory that cleanly proves your intuition that it's "preposterous" that an LLM could be conscious, you should go submit it and win a Nobel! Otherwise you're just sharing your unfounded and unduly confident assumptions which isn't terribly interesting.
empath75 1y ago
Peoples interior model of the world is very tenuously related to reality. We don't have a direct experience of waves, quantum mechanics, the vast majority of the electromagnetic spectrum, etc. The whole thing is a bunch of shortcuts and hacks that allow people to survive, the brain isn't really setup to probe reality and produce true beliefs, and the extent to which our internal models of reality naturally match actual reality is related to how much that mattered to our personal survival before the advent of civilization and writing, etc. It's really only been a very brief amount of time in human history where we had a deliberate method for trying to probe reality and create true beliefs, and I am fairly sure that if consciousness existed in humanity, it existed before the advent of the scientific method.
- yannyu 1y ago
  I don't think it's brief at all. Animals do this experimentation as well, but clearly in different ways. The scientific method is a formalized version of this idea, but even the first human who cooked meat or used a stick as a weapon had a falsifiable hypothesis, even if it wasn't something they could express or explain. And the consequences of testing the hypothesis were something that affected the way they acted from there on out.
- deleted 1y ago
  [deleted]