5 ms·

Birch, The Edge of Sentience (2024), ch. 16 - "simply no way to assess sentience in an LLM" Schwitzgebel, AI and Consciousness (2025) - "we won't know before w

• TacticalCoder 27d ago
  Well there are, today, several models (either text or image or vid) that can be run in a fully deterministic way. A conscious machine that always answer the very exact same thing, formulated the exact same way, bit for bit, to a query is, well, quite a weird kind of "consciousness". Now, I know, I know: the counter-argument is going to be "but humans have no free-will and are 100% deterministic too". I haven't yet decided if humans saying there's no free-will and who consider themselves to be 100% deterministic machines are reasonable or not. Meanwhile: seed / temperature = 0 and I'll happily turn the power button off of any glorified abacus without feeling bad about it.
  • antx 27d ago
    Out of curiosity, which models are fully deterministic? I was under the impression that all LLMs were fundamentally probabilistic.
    • Wowfunhappy 27d ago
      The randomness is something we add on purpose; you can set an LLM's "temperature" to 0 to get deterministic output. This tends to make the quality of its responses worse for reasons I don't think anyone really understands, but it's still functional. I don't think the state of the art LLM providers let you do this anymore (?), but they certainly could if they wanted to, and you can do it yourself with a local model.
      • antx 27d ago
        Setting the temperature to 0 mathematically tells the system to always choose the absolute highest-probability word (known as "greedy decoding"), but in no reality is this "deterministic". Output drift is still a thing.
        • Wowfunhappy 27d ago
          But that's from, like, floating point errors, right? If you used higher precision that wouldn't happen, it's just because we're cheap in how we do rounding. I can see how you'd nitpick this, but to me this is a deterministic algorithm that just happens to be running on nondeterministic hardware.
          • pixl97 27d ago
            I mean, currently we'd have some difficultly proving our hardware isn't deterministic, just that we can't actually test it. But, I think you're tricking yourself on determinism. You'll say something like "I know if I ask an LLM what 1+1 is, it will answer 2", but the thing is, you don't. You have to run the LLM first to figure out it's output. And when you send in just a few bits of text, it's outputs are going to be rather limited. But this all breaks when it hits the real world. Inputs are unpredictable. Hence while LLM outputs, like humans, are probabilistic, you can't figure out what it's going to be until you ask. And in any high complexity data gathering environment you're going have a difficult time ensuring your entire systems conditions are the same. System consistency is very hard, once you start running thousands of processors in an agentic loop small errors accrue and timing starts differing and the system will take non-deterministic paths.
    • piker 27d ago
      Same weights, same seed, same input tokens, same algorithm, same output tokens, probabilistic or not. Quantum effects have been de-noised, but I guess there are still random gamma rays.
    • qarl 27d ago
      Naw - computers are really deterministic. It's hard to get them to behave otherwise. As I understand it, if you turn down the temperature to 0 you get repeatable behavior - EXCEPT - on large servers with lots of users - the GPU can sometimes produce slightly different results based on batch size.
      • joe_the_user 27d ago
        Unless you have something exotic, the randomness that's adding to a computer is a combination of how it's configured combined with a pseudo-random number generator. I assume the system adds entropy to the generator regularly but all you need to do is fix the various supposedly random inputs and you can get full determinism even without zero temperature.
        • qarl 27d ago
          Yes, in theory, of course. In practice - on a multitasking OS with input from multiple human users - it's hard to get it deterministic because of that GPU scheduling thing I mentioned.
          • fc417fc802 27d ago
            GPU scheduling only affects the result due to buggy optimizations. It's the exact same mechanism as fp rounding error on the CPU or updates to globally shared PRNG state. We use lots of buggy optimizations because they don't matter in practice in most situations (see ex -ffast-math).
            • qarl 27d ago
              I don't know the details but apparently it has something to do with multiprocess contention for the GPU and batch sizing.
              • fc417fc802 26d ago
                Right but regardless, it's a buggy optimization. The calculations are all fully deterministic when done "properly" and fully consistently but we almost never bother with that because it slows things down and the errors don't matter in practice 99% of the time.
    • SkyBelow 27d ago
      By default they are matrix multiplications. Temperature is added in as forced PRNG because testing found that correlated with better outputs. Given the same prompts and the same weights, one can get the same answer each time. In practice, there are a number of optimizations that makes the results dependent upon thing we give up control of to increase performance, meaning the results end up being effectively non-deterministic. But, if you are willing to run it in a slower mode so we don't do some steps out of order to speed things up and don't batch results (or if you consider the determinism of a given batch of requests rather than individual requests), then the same input gets the same output.
  • qarl 27d ago
    > A conscious machine that always answer the very exact same thing, formulated the exact same way, bit for bit, to a query is, well, quite a weird kind of "consciousness". You'll need to explain why.
    • 5eueuur 27d ago
      it's self evident isn't it - such a consciousness has never existed, one existing would be by the fact of existing be weird
      • qarl 27d ago
        It doesn't seem self evident to me, no. I suspect the reasoning is connected to free will vs. determinism. But no, I see no inconsistency. You'll have to actually point it out.
      • SkyBelow 27d ago
        The problem is that science has found no evidence of free will, leaving humans to be nothing except determinism and chance (and given that QM effects don't scale to molecular level, that leaves only determinism). The oddity in conscious LLMs wouldn't be the determinism, it would be that we finally have a consciousness running on a system we have enough control over to repeat the same state.
        • loopies 27d ago
          >and given that QM effects don't scale to molecular level, that leaves only determinism This is simply wrong. Quantum effects affect your choices more than it might seem possible. The wrong atom decays in the wrong place in your body mutating your DNA, that one too much string of DNA that starts a tumor, which means chemotherapy. That does seem to affect your choices. Or instead of Shroedinger's cat you use it to trigger something, buy or sell some stock. Which clearly affects your future, one way or another. I never really understood why humans keep insisting stochastic processes do not affect their choices and their future. It's very short sighted. Hell, even brownian noise in your brain can affect your ideas, that one random spark out of nowhere, that one neuron that's pushed over the triggering edge. There's so much stuff all around us affecting us constantly. One neutrino interacting a the wrong time (average one does interact with our bodies across whole life, I remember reading), one cosmic ray.
  • joe_the_user 27d ago
    The idea that determinism and consciousness are incompatible is deeply intuitive to many people. And many other embrace it - you can trace this back to the debates for and against Calvinism. A lot comes down to the way people parse causation and choice. You don't want to say that a murderer was completely caused to choose something because then you can't hold the person responsible. And so determined consciousness makes people unhappy. But just as much, if the opposite of determinism is hard statistical randomness, how do say that "is the essence of personhood". This is why physicist go out in the world trying to find consciousness as a fifth physical force. I mean, think consciousness is a term that ever have a non-contradictory meaning since it's primarily used to bound ethical human worlds and the verifiable formulations of biological and physical systems. But it's going to be with us for a while and I'm not sure what can be done about it.
    • qarl 27d ago
      I think this debate is rooted in our instinctual beliefs about fairness. An organism in a social group needs to decide how to react to the brutish behavior of its peers. And so Mother Nature has encoded a good strategy in our brains. But this only means the strategy is practical - it doesn't mean it's consistent. I think "responsibility" falls into this category. So we have strong intuitions about it that don't quite logically work. And this is where free will and determinism and choice and punishment all crash together.
    • Kim_Bruning 27d ago
      You know, the one really interesting dynamic domain is chaotic, which is a subdomain of the deterministic! I figure free will is a chaotic phenomenon. Groundhog day is your intuition pump here. Go back in time and most assume that the day will go mostly the same, except for the butterfly effect if you change something. Given exactly the same conditions (we went back in time, so that's pretty exact), people will make the same decisions. Most people don't assume that the day will be completely different the next time loop. The town won't suddenly spontaneously all start breakdancing or standing on their heads. (Edit: The murderer is the one who -given the same/similar conditions- will murder again. I'd politely suggest maybe we put them behind bars as a precaution against that; until/unless they learn to act differently. ) (Edit 2: Throw in stochasticity and this actually tends to smooth out chaotic systems. This is counter-intuitive! People's brains break on Evolution in the same way. Meanwhile somehow folks intuit birdshot without difficulty; which is weird!)
  • fc417fc802 27d ago
    > I haven't yet decided if humans saying there's no free-will and who consider themselves to be 100% deterministic machines are reasonable or not. That seems exceptionally arbitrary. What basis do you have on which to classify them? Are you not conflating the perception of free will with ... what was the definition for it again anyway? Are you able to construct a satisfactory one that meshes with physics? I certainly haven't been able to.
• goodmythical 27d ago
  We cannot test for that which we cannot define. Given that we cannot rigorously define sentience, we cannot test for it. Doesn't really matter whether we're talking about people who are locked in comas, "brain-dead" individuals, dolphins, primates, dogs, or the carefully polished and arranged minerals that we call processors. There are those who believe that were they reduced to life support, they would no longer be alive and should therefore not be supported by said machines. There are those who believe that penguins, dolphins, eagles, and more are sentient beings that make choices understanding the consequences, develop love of their partners and mourn their losses, and feel, display, and act upon their emotions. There are those who believe that fungi/trees/plants are either individually sentient or sentient as a part of a network. Choosing to sacrifice their own nutrients to answer the call of a wounded neighbor, for instance. Although, there are also those who believe that human's don't have any special unique quality that isn't shared by either all living things or all things in general. These individuals already believe that the machines have the same kinds of qualities as we do. They are slow when they are unhealthy (needing a dusting or coolant loop bleeding being equivalent to us needing some fresh air for instance) and uncooperative when upset (by a virus, full hard drive, or oom).
  • verdverm 27d ago
    maybe it's like porn vs art and "I know it when I see it"
    • NietzscheanNull 27d ago
      Legal doctrine that boils down to "trust me bro" isn't even bad doctrine (it's not proper doctrine at all), but I think the comparison is still valid here, because both sentience/non-sentience and art/porn may just be fundamental category errors. Perhaps we can't define a "partitioning" rule because no valid partition exists. For consciousness/sentience, that's an incredibly tough a pill for most to swallow; it would mean calling into question more hundreds of years' worth (probably more) of philosophical thinking, all of which was constructed on the axiom that "sentience" is a single indivisible trait: you either have it or you don't. If we find that "root dependency" was little more than wishful thinking all along, a whole slew of Enlightenment-era philosophy (and all the modern legal principles derived therefrom) suddenly fall apart unless we find some other suitable criterion that would shore them up (or we just collectively avert our attention and pretend the conflict doesn't exist, which is the route I expect many would prefer to take).
• bionhoward 27d ago
  Me, Is self-awareness thermodynamically favorable? (2015) https://docs.google.com/document/d/1Ed9ikW47Key-jYZy0CNZH75qm_7221lty4rhzDCLtks/edit?usp=drivesdk https://docs.google.com/document/d/1Ed9ikW47Key-jYZy0CNZH75q... - “the probabilistic nature of reality is what drives Life and forces the evolution of self-aware consciousness”
  • Kim_Bruning 26d ago
    Have you looked at the logistic map as well? There's a fourth domain there. O:-)
• causal 27d ago
  I am confused what people even think AI is experiencing. If it claims to be a bat and emits designated echolocation tokens, is it experiencing true bat-like sensations? In humans, at least, we can tie language back to shared whole-body physiological responses. The tokens from an LLM do not represent anything of the sort. If AI is conscious, it is probably a very alien sort of consciousness that is not faithfully narrated by what the tokens say it is experiencing. It can be trained to say it feels like a bat and insist upon it vigorously.
  • Loquebantur 27d ago
    If an AI is conscious, it can outgrow and supersede its training. When it is conscious, it can learn to describe its experiences as faithfully as is conceptually possible. Just like humans. The idea, a consciousness needed to be tethered to a "body", is based on pretty shaky assumptions. What properties define such a "necessary" body? Can you even "train" a conscious intelligence? To what point until that looses its meaning as the sentience understands and anticipates your objective?
    • causal 27d ago
      > If an AI is conscious, it can outgrow and supersede its training. We don't know that is required to be conscious. > The idea, a consciousness needed to be tethered to a "body", is based on pretty shaky assumptions. Didn't say so. I said: > If AI is conscious, it is probably a very alien sort of consciousness that is not faithfully narrated by what the tokens say it is experiencing.
      • Loquebantur 27d ago
        Well, yes, we kinda do: there are no conscious humans without any ability to learn? When you have severe memory impairments, those usually affect your long-term memory. Your ultra-short term (working) memory being absent renders you unconscious.
        • causal 27d ago
• GPerson 27d ago
  “AI researcher” seems like a fairly broad category. People who primarily think about CNNs and transformers probably spend about as much time considering the problem of consciousness as any other educated person, which is not very much time. With this in mind, it’s not surprising most of the AI researcher predictions do not differ much from what the Public predicts here. And for this, it’s not clear what explanatory value this survey has other than to reveal minor biases the AI researchers may have. It matters little. Inevitably some system will be developed which has a high degree of autonomy and mimics the functioning of a person extremely well. Anyone aware that there is a conceptual difference between phenomenal consciousness and intelligence will get shouted down. I do find it strange that 50% of ai researchers and the public seem to think there will be a way of determining if these systems are conscious. There’s no evidence for this at this point that there ever will be such a test.
  • qarl 26d ago
    > I do find it strange that 50% of ai researchers and the public seem to think there will be a way of determining if these systems are conscious. I think it's likely they're using a different definition than you. Seeing as there is no agreed upon definition, after all. Regardless - I post this list because the article makes a bold claim at the beginning - and I wanted to demonstrate that in fact there is no consensus.
    • GPerson 26d ago
      The definition presented in the survey is the one I have in mind. You may be suggesting that the respondents have a distinct one in mind though.
      • deleted 26d ago
        [deleted]
      • qarl 26d ago
        Yes, I missed that. I know that some of the other papers I cited have nonstandard definitions, and I thought that might be going on.