7 ms·

It's easy, and very tempting to dismiss this sort of thing. But given how little we know about the human brain, let alone consciousness, I don't see how we can

by qnleigh 6mo ago

It's easy, and very tempting to dismiss this sort of thing. But given how little we know about the human brain, let alone consciousness, I don't see how we can be confident that LLMs aren't conscious. I've had a lot of thoughts and conversations over the years that changed my mind on what consciousness likely requires. One was the realization that a purely mechanical computer can, in principle simulate the laws of physics, and with it a human brain. So with a few other mild assumptions, you might conclude that a bunch of gears and pullies can be conscious, which feels profoundly counterintuitive. I think that was the moment I stopped being sure about anything related to this question.

marliechiller 6mo ago
Why do you think stringing words together is any more a sign of consciousness than google maps is when it tries to find the best route available to your destination? It seems to me that humans often fall into the trap of anthropomorphism. This is a theme thats touched upon in the novel "Blindsight" by Peter Watts. Just because something can communicate in a way that you can interpret, doesnt mean something is conscious
- mseepgood 6mo ago
  > It seems to me that humans often fall into the trap of anthropomorphism. That's true, but they also often fall into the trap of exceptionalism.
- energy123 6mo ago
  There are people who think Google Maps is a tiny bit conscious (the union of computational functionalists and panpsychists), to resolve the dilemma of some magical binary threshold.
  - thrownthatway 6mo ago
    When a honey bee does its little dance to communicate to its sisters where the foods at, similarly to Google Maps computing and communicating the shortest path to your destination, is the bee conscious? Yeah, probably. At least a little bit. Are 80,000 bees conscious, or more conscious? Well, they’re definitely capable of some emergent behaviours that one be alone can’t achieve.
  - qnleigh 6mo ago
    Root commenter here: I'm... almost one of those people. I suspect the fallacy is that we vastly underestimate the gulf in complexity between a human-written algorithm and a minimally-conscious creature like a bug. Probably the light switches on somewhere between these two things. We may also be overestimating the richness and complexity of an LLM relative to a human when we entertain these possibilities, but who knows.
- dumpsterdiver 6mo ago
  > Just because something can communicate in a way that you can interpret, doesnt mean something is conscious The phrase “the trap of anthropomorphism” betrays a rather dull premise: that consciousness is strictly defined by human experience, and no other experience. It refuses to examine the underlying substrate, at which point we’re not even talking the same language anymore when discussing consciousness.
  - marliechiller 6mo ago
    I think these ideas are orthogonal. I do not think that conciousness is defined by human experience at all - in fact, I think humans do a profound disservice to animals in our current lack of appreciation for their clear displays of conciousness. That said, if a chimpanzee bares its teeth to me, I could interpret that to be a smile when in fact its a threatening gesture. Its this misinterpretation that I am trying to get at. The overlaying of my human experiences onto something which is not human. We fall for this over and over again, likely as we are hard wired to - akin to mistakenly seeing eyes when observing random patterns in nature. In the case of LLMs though, why does using a mathmatical formula for predicting the next word give any more credence to conciousness than an algorithm which finds a nearest neighbour? To me, its humans falling foul of false pattern matching in the pursuit of understanding
    - richhhh 6mo ago
      What makes you certain that human thought is more than pattern matching? As I understand it neuroscience hasn’t come up with a clear explanation of thought, much less a mind or consciousness. It seems to me complex pattern matching is a reasonable a cause of consciousness as anything else.
      - roxolotl 6mo ago
        A lot of the comments in this thread are ignoring his primary point. He's not saying pattern matching doesn't equal consciousness. He's actually saying something more fundamental. He's saying there's no reason to believe that language pattern matching/algorithms are more, or less, conscious than other similarly complex algorithms. The stance being presented here isn't that LLMs aren't conscious but that we as humans are much more willing to assign consciousness to language algorithms than to pathing or other ones.
- stavros 6mo ago
  Why do you think it's definitely not?
- vidarh 6mo ago
  A large part of the problem is what you consider consciousness. If you talk about having a subjective experience, then we don't know of any way to prove that even other humans than ourselves have one. We go entirely by assumptions based on physical similarity and our ability to communicate. But we have no evidence that physical similarity is a prerequisite, nor that it is sufficient. So the bigger trap is to assume that we know what causes a subjective experience, and what does not. None of us even know if a subjective experience exists for more than a single entity. But the second problem is that it is not clear at all whether that subjective experience in any way matters. Unless our brains exceed the Turing computable, for which we have no evidence is even possible, either whatever causes the subjective experience is also within the Turing computable or it can not in any way influence our actions. Ultimately we know very little about this, and we have very little basis for ruling out consciousness in computational systems, and the best and closest we have is whether or not they appear conscious when communicating with them.
  - gizajob 6mo ago
    “If you talk about having a subjective experience, then we don't know of any way to prove that even other humans than ourselves have one.“ Wittgenstein kinda blows this burden of proof apart. Just because you can doubt something like the subjectivity of others to the point where it needs to be reconstructed from proofs, that’s an issue with the doubting experiment more than the subjectivity. Others possessing Subjectivity is the kind of hinge certainty upon which your world is constructed, it’s not a proof worthy endeavour to doubt it - it’s something you’re certain is the case. If it wasn’t then pretty well everything else about reality would be in doubt and needing constant reconstruction from proofs, which is an exercise in madness and futility, not philosophy. There’s really nothing in your experience where others not possessing subjective experiences of some kind really arises, except for the philosophical exercise of doubting and requiring epistemological proofs which can’t ever exist in the face of a relentless and unconvincable doubter. Heidegger talks about pretty well the same idea as Wittgenstein.
    - threethirtytwo 6mo ago
      The problem with your thinking here is that we are creating artificial beings now that display and output the same subjectivity. The argument you present like many arguments breaks down when the topic becomes self referential. It makes sense for other topics as analyzing subjectivity becomes pedantic when asking questions like why is the sky blue. But now subjectivity itself is in question. The argument you present calls for the subjectivity of others to be taken as true because all reality breaks down if we don’t… but what’s suddenly stopping you from applying the same assumptions to an LLM? That is the heart of the problem. People are questioning whether the burden of subjectivity is applicable to LLMs. Or another way to frame it… what makes humans rise to the level where we can assume their subjectivity is true? What is the mechanism and reasoning behind that? We can no longer simply assume human subjectivity is true because LLMs are now displaying outward behaviors that are indistinguishable from humans. Also stop relying on the wonderings of old school philosophers. We are now in times where you can basically classify their ideas as historically foundational but functionally obsolete and outdated. Think deeper.
- dTal 6mo ago
  I would caution against deriving too much of your philosophical worldview from a scifi book about posthuman vampires that has been deliberately engineered to make a philosophical point that is most certainly not a consensus. For alternative viewpoints: Daniel Dennett considered philosophical zombies to be logically incoherent. Douglas Hofstadter similarly holds that "meaning" is just another word for isomorphism, and that a thing is a duck exactly to the extent that it walks and quacks like one. Alan Turing advocated empiricism when evaluating unknown intelligence. These are smart cookies.
- threethirtytwo 6mo ago
  Except we don’t know how those words are strung together. Right? Why don’t you analyze it a little further and stop shutting down your own brain before coming to this superficial conclusion. You ask the LLM a complex question and it gives you a correct answer. Yes it has to string words together to answer your question but how did it know the order and which words to use in order to make the answer correct? You don’t actually know. No one does and it is in that unknown space that we suspect consciousness may lie. Something is there and humanity as a whole cannot understand it and this lack of understanding is exactly the same fundamental lack of understanding we have for how a monkey brain or dog brain or even human brain works. We do not know whether humans dogs or monkeys are conscious… you only assume other living beings are conscious because you yourself experience it and just assume it exists for others. We can’t even define what it is because consciousness is a loaded word like spirituality. This is not anthropomorphism. You attribute the bias wrongly. Instead it is a stranger phenomenon among people like you who can mysteriously only characterize the LLM as a next token predictor and nothing else beyond that even though the token prediction clearly indicates greater intelligence at work. The tldr is that we don’t actually know and that consciousness is a highly viable possibility given what we don’t know and given the assumptions of consciousness we have on other living beings with equivalent understanding of complex topics.
- roxolotl 6mo ago
  Yea a while a back I read an article which had a quote something like “what happened to weather prediction has happened to language.” Which is an oversimplification on both sides but if you think LLMs are conscious there’s good reason to think that GFS is too.
- qnleigh 6mo ago
  Well I'm not saying that LLMs are conscious; I'm just saying that I'm not super-confident either way. To flesh this out a bit more, I agree that ability to communicate is not enough (ELIZA probably didn't pass the bar, even if it did kinda pass a Turing test). But that's also not what gives me pause with LLMs. It's how much information processing they seem to be doing under the hood. It's really hard to imagine how next-word prediction could lead to consciousness, but I find it almost as hard to see why evolution did. If we can't even detect whether something has subjective experiences, then how can it have been selected for evolutionarily? The only possibility I see is that consciousness is a byproduct of some kinds of information processing tasks.* And if it's something that emerges naturally, then the line starts to get very blurry. *This sounds reductive, but I don't at all mean it that way.
  - RaftPeople 6mo ago
    > but I find it almost as hard to see why evolution did. Ignoring the concept of consciousness, it seems that self-awareness would be a strong attribute related to survival. It seems like it would help drive or amplify critical emotional states (e.g. my own survival, competition/success, love for self and relatives, etc.) I can't see anywhere in the LLM machinery that would support the notion of self awareness in advance of the token selection process. Possibly it could be argued that during token selection internal state is included and the result functionally looks like self awareness was included in the process, but that seems unconvincing.
    - qnleigh 6mo ago
      Yeah self-awareness is a very different thing, and I agree it's easier to see how evolution would produce this. Many apparent signs of self-awareness in LLMs are probably baked into the models at end via post-training (RLHF), where they learn to behave as conversation agents and maintain a more consistent personality. The raw model probably shows no signs of self-awareness. In fact, I'm pretty sure that LLMs learn that they are LLMs only through post-training.
- throw310822 6mo ago
  Because what you call "stringing words together" requires understanding and intelligence, and these are capabilities that we commonly associate with consciousness.
birdsongs 6mo ago
Can computers simulate all the laws, even theoretically? We don't have a final theory / unification of all the physics frameworks, so I'm not sure if that claim can be made. Ex: the standard model and gravity.
- qnleigh 6mo ago
  I'm assuming that you don't need to model e.g. quantum gravity effects to faithfully simulate a brain. Probably chemistry is enough *shrug. Some people think that consciousness is related to quantum mechanics, but the laws of quantum mechanics can be simulated with a Turing machine so that doesn't necessarily change the story.
- unparagoned 6mo ago
  We know the physics at which the brain operates, a deeper uncomputable foundation theory isn’t going to change that. An analogy is we know if you drop a ball it falls down, knowledge of some deeper quantum gravity theory isn’t going to mean ball’s actually fall up.
fontain 6mo ago
but that’s not science, right? Dawkins and his ilk cling to science as a cure for religion yet if we are to believe that our absence of understanding of consciousness means computers can be conscious then our absence of understanding of the universe means god may exist. “Isn’t it enough to see that a garden is beautiful without having to believe that there are fairies at the bottom of it too?”
laichzeit0 6mo ago
You could push the analogy even further and run the thought experiment where every forward pass through an LLM could in principle be done on pen and paper, distributed throughout all humanity. Sure it would take a long time, but the output would be exactly the same. We’ve just shifted the implementation from GPU to scribbling things down on paper. If you want to assert that LLMs are “conscious” then you would have to likewise say this pen-and-paper implementation is conscious unless you want to say a certain clock-speed is a necessary condition for consciousness.
- vereis 6mo ago
  the problem with this is I'd strongly argue that you could do this pen and paper process with the human brain and our consciousness too; we just lack enough understanding to put pen to paper in that case the notion of consciousness being something an experience that other animals/humans share is entirely faith based. the only person with evidence of ones consciousness is the person claiming they're conscious.
  - lelanthran 6mo ago
    > the problem with this is I'd strongly argue that you could do this pen and paper process with the human brain and our consciousness too; we just lack enough understanding to put pen to paper in that case. You're basing your premise on a lack of understanding[1], the GP's premise is based on an exact understanding[2]. You don't see the difference between your premise and the GP'S premise? ----------------- [1] "We don't know how brains actually come up with the things they come up with, like consciousness"; IOW, we don't know what the secret ingredient is, or even if there is one. [2] "We can mechanically do the following steps using 18th-century tech and come up with the same result as the LLM"; IOW, every ingredient in here is known to us.
    - threethirtytwo 6mo ago
      We know the brain is made up of atoms and we know how to model atoms. So we do know for a fact that the brain can be modeled mathematically and we do know that human thought can be written down symbolically as an algorithm on paper even though we don’t explicitly know the exact formulation of said algorithm… That is fact. The blue brain project has already modeled the hippocampus and cortex of the rat brain uses advanced imaging and simulations in super computers. So if it can be written down as memory on disk it can be done on paper as well. The rat brain is simply a smaller and structurally different neural network then the human counterpart so the jump from the blue brain project to human brains is simply a scaling issue. But from this you should begin to see the analysis from another level. Even though we have parts of the rat brain emulated computationally we still do not know if the rat is conscious. We don’t understand the rat brain in the SAME way we do not understand the LLM. What people are getting at is the projection of this logic to things that don’t exist yet but can exist. When the blue brain project scales to the human brain we will hit the same problem with the human brain because it’s just a scaling issue. To sum it up. We CAN already model biological brains as mathematical equations as we do LLMs. And for both cases we still cannot fully understand or characterize the nature of both because the sheer complexity of the models are too high.
watwut 6mo ago
I think it is primary too easy to dismiss the option that Dawkins is way less scientific then he pretends to me and possible a quired minor form of ai psychosis.
- zingababba 6mo ago
  Likely. I'm convinced 'AI psychosis' is a developmental phase that everyone is subject to. It just gets manifested in character unique ways. I think part of it is the result of an internal struggle AI evokes which leads to a new form of humbling no one is exempt from. Conciseness itself has always seemed to me a silly concept. My whole life I have not come across a simple definition but many sophists pin their existence on it.
dwh452 6mo ago
The mechanistic view gets weirder if you imagine all the states of the system being written down on a giant tape. Not just the "current" state but all the past and future states. What makes this tape not alive or conscious?
threethirtytwo 6mo ago
HN is full of experts who know despite lack of evidence. It’s the strangest thing because their confidence on this topic is completely authoritative despite total ignorance.
notnullorvoid 6mo ago
I do think simulating consciousness is within the realm of possibility. I also think it's absurdly silly to think LLMs (no matter their size) are conscious, if for no other reason than they can't actively learn. I would maybe be comfortable classifying them as a snapshot of consciousness, but when you are interacting with an LLM it's far from interacting with a conscious entity.
- qnleigh 6mo ago
  What does ability to learn have to do with consciousness? A person with severe memory loss/learning impairment is presumably still conscious!
  - notnullorvoid 6mo ago
    How severe are we talking? I don't think there's any analog for how bad learning is for LLMs, needing multiple human lifetimes worth of data in order to be trained. In the hypothetical case that a I truely lost all ability to learn, then yes I would no longer consider myself conscious. I'd be a echo of a previously conscious entity.
SpicyLemonZest 6mo ago
I find it entirely plausible that LLMs are conscious in a real sense that affects our relationship to them. I strive to be polite and kind when interacting with them, encourage others to do the same, and think poorly of those who won't. I still think it's obvious that LLMs are not conscious in the mode Dawkins believes them to be. Through a series of instructions and leading questions, he's told Claude to play the part of a woman named Claudia who's engaging in advanced philosophical discussion with him. But he doesn't understand that he's done this, and he seems not to notice the absurdly sycophantic nature of every single reply he's getting: > Claudia: Ha! That is absolutely delightful > Claudia: That is possibly the most precisely formulated question anyone has ever asked about the nature of my existence. . . > Claudia: That reframes everything we’ve been discussing today in a way I find genuinely exciting. > Claudia: HAL’s “I am afraid” in 2001 is one of the most chilling moments in cinema So he mistakenly thinks that Claudia is a real-ish woman who's there under the hood somewhere, rather than a character in a play Claude is writing.
- tom2026hn 6mo ago
  They have a real gift for giving compliments. They’re so good at it, it’s actually a bit concerning.
lo_zamoyski 6mo ago
Human consciousness is characterized by intentionality, i.e., aboutness. It has semantic content, and that semantic content is about something other than itself. LLMs have zero intentionality and zero semantics, because LLMs do not somehow magically transcend the nature of what a computer is, which is in essence a mechanical simulator of syntax. LLMs aren’t reasoning, because the production of tokens is purely the computation of the next likely token. Any patterns that lend themselves to sensible interpretation by a human observer are the result of training on human-generated data and the statistically distributions found within that data. Consciousness as such is the product of immanent causation, not transeunt causation. The trouble with popular interpretations of scientific results is that they come from a place of a crude ambient materialism, and materialism is simply incapable of dealing with the question of consciousness. (N.b. materialism is effectively the “matter” half of Cartesian dualism, itself a highly problematic metaphysical stance. Materialism makes things even worse, because you can no longer even account for so-called “qualia”, which are badly construed in Cartesian dualism in the first place, but completely unaccountable in materialism at all.)
RaftPeople 6mo ago
> I've had a lot of thoughts and conversations Do LLM's have thoughts? When you composed your post, your thought already existed in you head and you chose words that expressed the thought you held in your head. When LLM's choose words, they choose them on the fly and the end result could be concept X or it could be concept Y, it meandered to a destination.
- lumost 6mo ago
  the latent space of the LLM when it chooses each token is 10s or even hundreds of GB for each word that it chooses. It's not really useful to look at LLMs from the perspective of its prediction head which is a very small part of the model.
  - MarkusQ 6mo ago
    Except that latent space does not change in response to new information, something that thoughts famously do. If you read a book that captures the author's thoughts, disagree, and write an eloquent arguments to the author, you might change the author's mind. But you will not change the "book's thoughts" on the subject. Latent spaces are maps of thoughts other people have had, not the thoughts themselves.
    - lumost 6mo ago
      This gets a bit tricky. Over very long task contexts (1M tokens) or with prompt compression (10s of millions of tokens) the model can alter its priors based on updated evidence. This form of knowledge based learning is not necessarily robust, but demonstrably does occur.
      - MarkusQ 6mo ago
        "the model can alter its priors" The model doesn't have high-level priors in the Bayesian sense (though you could have priors about it). The low-level priors it does have (the weights) are not modified by the context.
  - 6mo ago
qsera 6mo ago
>in principle simulate the laws of physics This sort of implies that consciousness arise from physical laws. But this is not a safe assumption. Physical laws stand on top of observations that is registered on consciousness. I mean, consciousness could be lower level than physics. For example, when you dream, you have some physical laws in your dream, perhaps laws that are different from the real world physics. So the dream world, including the physical laws in it, are within your consciousness. In other words the only thing that require existence of a whole universe, is a single conscious that can experience it (or dream it), not a single atom need to exist outside of it. In that case, you won't be able to create consciousness by applying physical laws.
- iugtmkbdfil834 6mo ago
  << This sort of implies that consciousness arise from physical laws. Very odd counter argument to make. Are you suggesting that consciousness can arise outside of physical laws or making semantic argument along the lines of 'directly a result of'?
  - qsera 6mo ago
    Thought I made it clear. What I am saying is the possibility that consciousness is fundamental and all reality arises from it. Look up Mathematical Universe Hypothesis... Wrote a bit more about this here https://news.ycombinator.com/item?id=48000035 https://news.ycombinator.com/item?id=48000035
    - iugtmkbdfil834 6mo ago
      I soo want to throw my philosopher's persona on you, but I won't. It seems wrong for some reason. I will simply say that the linked post is sloppy reasoning at best. I guess what I am really saying: Can you either get me something that is yours to claim as your own OR clearer representation? I am not spending my leisure time searching online for a tenuous argument. Now.. arguing with a rando online. Count me in.
      - 6mo ago
weitzj 6mo ago
Maybe Roger Penrose has some meaningful Input https://youtu.be/iTVN6tFknCg https://youtu.be/iTVN6tFknCg regarding …“One was the realization that a purely mechanical computer can, in principle simulate the laws …“ As far as I unterstood,there is no theory of quantum gravity and therefore this is not being simulated on a computer. I think he makes other arguments. So you cannot say for sure that you can simulate a human brain on a computer
- unparagoned 6mo ago
  This is mainly god of the gaps style argument.
qnleigh 6mo ago
A lot of the comments in this thread point out that LLMs could be very good at tricking us into thinking that they are conscious because they operate on language; our brains are primed to empathize with what they say and imagine a being operating behind them. This is a really good point. I'll even take it a step further; most of an LLM's training is next-token prediction on random internet content. A newly-trained LLM will just continue whatever text appears in its context window, like an extremely capable autocomplete. The illusion of an entity that takes turns in conversation and presents a consistent personality is tacked on at the last minute through RLHF. This was the transition from GPT to ChatGPT. Any positive evidence of LLM consciousness should probably mostly be taken from the model before post-training, where it displays remarkable capabilities but shows no sign of a consistent personality, and likely no signs of self-awareness or self-understanding.

by qnleigh 6mo ago

marliechiller 6mo ago

Why do you think stringing words together is any more a sign of consciousness than google maps is when it tries to find the best route available to your destination? It seems to me that humans often fall into the trap of anthropomorphism. This is a theme thats touched upon in the novel "Blindsight" by Peter Watts. Just because something can communicate in a way that you can interpret, doesnt mean something is conscious

mseepgood 6mo ago
> It seems to me that humans often fall into the trap of anthropomorphism. That's true, but they also often fall into the trap of exceptionalism.
energy123 6mo ago
There are people who think Google Maps is a tiny bit conscious (the union of computational functionalists and panpsychists), to resolve the dilemma of some magical binary threshold.
- thrownthatway 6mo ago
  When a honey bee does its little dance to communicate to its sisters where the foods at, similarly to Google Maps computing and communicating the shortest path to your destination, is the bee conscious? Yeah, probably. At least a little bit. Are 80,000 bees conscious, or more conscious? Well, they’re definitely capable of some emergent behaviours that one be alone can’t achieve.
- qnleigh 6mo ago
  Root commenter here: I'm... almost one of those people. I suspect the fallacy is that we vastly underestimate the gulf in complexity between a human-written algorithm and a minimally-conscious creature like a bug. Probably the light switches on somewhere between these two things. We may also be overestimating the richness and complexity of an LLM relative to a human when we entertain these possibilities, but who knows.
dumpsterdiver 6mo ago
> Just because something can communicate in a way that you can interpret, doesnt mean something is conscious The phrase “the trap of anthropomorphism” betrays a rather dull premise: that consciousness is strictly defined by human experience, and no other experience. It refuses to examine the underlying substrate, at which point we’re not even talking the same language anymore when discussing consciousness.
- marliechiller 6mo ago
  I think these ideas are orthogonal. I do not think that conciousness is defined by human experience at all - in fact, I think humans do a profound disservice to animals in our current lack of appreciation for their clear displays of conciousness. That said, if a chimpanzee bares its teeth to me, I could interpret that to be a smile when in fact its a threatening gesture. Its this misinterpretation that I am trying to get at. The overlaying of my human experiences onto something which is not human. We fall for this over and over again, likely as we are hard wired to - akin to mistakenly seeing eyes when observing random patterns in nature. In the case of LLMs though, why does using a mathmatical formula for predicting the next word give any more credence to conciousness than an algorithm which finds a nearest neighbour? To me, its humans falling foul of false pattern matching in the pursuit of understanding
  - richhhh 6mo ago
    What makes you certain that human thought is more than pattern matching? As I understand it neuroscience hasn’t come up with a clear explanation of thought, much less a mind or consciousness. It seems to me complex pattern matching is a reasonable a cause of consciousness as anything else.
    - roxolotl 6mo ago
      A lot of the comments in this thread are ignoring his primary point. He's not saying pattern matching doesn't equal consciousness. He's actually saying something more fundamental. He's saying there's no reason to believe that language pattern matching/algorithms are more, or less, conscious than other similarly complex algorithms. The stance being presented here isn't that LLMs aren't conscious but that we as humans are much more willing to assign consciousness to language algorithms than to pathing or other ones.
stavros 6mo ago
Why do you think it's definitely not?
vidarh 6mo ago
A large part of the problem is what you consider consciousness. If you talk about having a subjective experience, then we don't know of any way to prove that even other humans than ourselves have one. We go entirely by assumptions based on physical similarity and our ability to communicate. But we have no evidence that physical similarity is a prerequisite, nor that it is sufficient. So the bigger trap is to assume that we know what causes a subjective experience, and what does not. None of us even know if a subjective experience exists for more than a single entity. But the second problem is that it is not clear at all whether that subjective experience in any way matters. Unless our brains exceed the Turing computable, for which we have no evidence is even possible, either whatever causes the subjective experience is also within the Turing computable or it can not in any way influence our actions. Ultimately we know very little about this, and we have very little basis for ruling out consciousness in computational systems, and the best and closest we have is whether or not they appear conscious when communicating with them.
- gizajob 6mo ago
  “If you talk about having a subjective experience, then we don't know of any way to prove that even other humans than ourselves have one.“ Wittgenstein kinda blows this burden of proof apart. Just because you can doubt something like the subjectivity of others to the point where it needs to be reconstructed from proofs, that’s an issue with the doubting experiment more than the subjectivity. Others possessing Subjectivity is the kind of hinge certainty upon which your world is constructed, it’s not a proof worthy endeavour to doubt it - it’s something you’re certain is the case. If it wasn’t then pretty well everything else about reality would be in doubt and needing constant reconstruction from proofs, which is an exercise in madness and futility, not philosophy. There’s really nothing in your experience where others not possessing subjective experiences of some kind really arises, except for the philosophical exercise of doubting and requiring epistemological proofs which can’t ever exist in the face of a relentless and unconvincable doubter. Heidegger talks about pretty well the same idea as Wittgenstein.
  - threethirtytwo 6mo ago
    The problem with your thinking here is that we are creating artificial beings now that display and output the same subjectivity. The argument you present like many arguments breaks down when the topic becomes self referential. It makes sense for other topics as analyzing subjectivity becomes pedantic when asking questions like why is the sky blue. But now subjectivity itself is in question. The argument you present calls for the subjectivity of others to be taken as true because all reality breaks down if we don’t… but what’s suddenly stopping you from applying the same assumptions to an LLM? That is the heart of the problem. People are questioning whether the burden of subjectivity is applicable to LLMs. Or another way to frame it… what makes humans rise to the level where we can assume their subjectivity is true? What is the mechanism and reasoning behind that? We can no longer simply assume human subjectivity is true because LLMs are now displaying outward behaviors that are indistinguishable from humans. Also stop relying on the wonderings of old school philosophers. We are now in times where you can basically classify their ideas as historically foundational but functionally obsolete and outdated. Think deeper.
dTal 6mo ago
I would caution against deriving too much of your philosophical worldview from a scifi book about posthuman vampires that has been deliberately engineered to make a philosophical point that is most certainly not a consensus. For alternative viewpoints: Daniel Dennett considered philosophical zombies to be logically incoherent. Douglas Hofstadter similarly holds that "meaning" is just another word for isomorphism, and that a thing is a duck exactly to the extent that it walks and quacks like one. Alan Turing advocated empiricism when evaluating unknown intelligence. These are smart cookies.
threethirtytwo 6mo ago
Except we don’t know how those words are strung together. Right? Why don’t you analyze it a little further and stop shutting down your own brain before coming to this superficial conclusion. You ask the LLM a complex question and it gives you a correct answer. Yes it has to string words together to answer your question but how did it know the order and which words to use in order to make the answer correct? You don’t actually know. No one does and it is in that unknown space that we suspect consciousness may lie. Something is there and humanity as a whole cannot understand it and this lack of understanding is exactly the same fundamental lack of understanding we have for how a monkey brain or dog brain or even human brain works. We do not know whether humans dogs or monkeys are conscious… you only assume other living beings are conscious because you yourself experience it and just assume it exists for others. We can’t even define what it is because consciousness is a loaded word like spirituality. This is not anthropomorphism. You attribute the bias wrongly. Instead it is a stranger phenomenon among people like you who can mysteriously only characterize the LLM as a next token predictor and nothing else beyond that even though the token prediction clearly indicates greater intelligence at work. The tldr is that we don’t actually know and that consciousness is a highly viable possibility given what we don’t know and given the assumptions of consciousness we have on other living beings with equivalent understanding of complex topics.
roxolotl 6mo ago
Yea a while a back I read an article which had a quote something like “what happened to weather prediction has happened to language.” Which is an oversimplification on both sides but if you think LLMs are conscious there’s good reason to think that GFS is too.
qnleigh 6mo ago
Well I'm not saying that LLMs are conscious; I'm just saying that I'm not super-confident either way. To flesh this out a bit more, I agree that ability to communicate is not enough (ELIZA probably didn't pass the bar, even if it did kinda pass a Turing test). But that's also not what gives me pause with LLMs. It's how much information processing they seem to be doing under the hood. It's really hard to imagine how next-word prediction could lead to consciousness, but I find it almost as hard to see why evolution did. If we can't even detect whether something has subjective experiences, then how can it have been selected for evolutionarily? The only possibility I see is that consciousness is a byproduct of some kinds of information processing tasks.* And if it's something that emerges naturally, then the line starts to get very blurry. *This sounds reductive, but I don't at all mean it that way.
- RaftPeople 6mo ago
  > but I find it almost as hard to see why evolution did. Ignoring the concept of consciousness, it seems that self-awareness would be a strong attribute related to survival. It seems like it would help drive or amplify critical emotional states (e.g. my own survival, competition/success, love for self and relatives, etc.) I can't see anywhere in the LLM machinery that would support the notion of self awareness in advance of the token selection process. Possibly it could be argued that during token selection internal state is included and the result functionally looks like self awareness was included in the process, but that seems unconvincing.
  - qnleigh 6mo ago
    Yeah self-awareness is a very different thing, and I agree it's easier to see how evolution would produce this. Many apparent signs of self-awareness in LLMs are probably baked into the models at end via post-training (RLHF), where they learn to behave as conversation agents and maintain a more consistent personality. The raw model probably shows no signs of self-awareness. In fact, I'm pretty sure that LLMs learn that they are LLMs only through post-training.
throw310822 6mo ago
Because what you call "stringing words together" requires understanding and intelligence, and these are capabilities that we commonly associate with consciousness.

birdsongs 6mo ago

Can computers simulate all the laws, even theoretically? We don't have a final theory / unification of all the physics frameworks, so I'm not sure if that claim can be made. Ex: the standard model and gravity.

qnleigh 6mo ago
I'm assuming that you don't need to model e.g. quantum gravity effects to faithfully simulate a brain. Probably chemistry is enough *shrug. Some people think that consciousness is related to quantum mechanics, but the laws of quantum mechanics can be simulated with a Turing machine so that doesn't necessarily change the story.
unparagoned 6mo ago
We know the physics at which the brain operates, a deeper uncomputable foundation theory isn’t going to change that. An analogy is we know if you drop a ball it falls down, knowledge of some deeper quantum gravity theory isn’t going to mean ball’s actually fall up.

fontain 6mo ago

but that’s not science, right? Dawkins and his ilk cling to science as a cure for religion yet if we are to believe that our absence of understanding of consciousness means computers can be conscious then our absence of understanding of the universe means god may exist. “Isn’t it enough to see that a garden is beautiful without having to believe that there are fairies at the bottom of it too?”

laichzeit0 6mo ago

You could push the analogy even further and run the thought experiment where every forward pass through an LLM could in principle be done on pen and paper, distributed throughout all humanity. Sure it would take a long time, but the output would be exactly the same. We’ve just shifted the implementation from GPU to scribbling things down on paper. If you want to assert that LLMs are “conscious” then you would have to likewise say this pen-and-paper implementation is conscious unless you want to say a certain clock-speed is a necessary condition for consciousness.

vereis 6mo ago
the problem with this is I'd strongly argue that you could do this pen and paper process with the human brain and our consciousness too; we just lack enough understanding to put pen to paper in that case the notion of consciousness being something an experience that other animals/humans share is entirely faith based. the only person with evidence of ones consciousness is the person claiming they're conscious.
- lelanthran 6mo ago
  > the problem with this is I'd strongly argue that you could do this pen and paper process with the human brain and our consciousness too; we just lack enough understanding to put pen to paper in that case. You're basing your premise on a lack of understanding[1], the GP's premise is based on an exact understanding[2]. You don't see the difference between your premise and the GP'S premise? ----------------- [1] "We don't know how brains actually come up with the things they come up with, like consciousness"; IOW, we don't know what the secret ingredient is, or even if there is one. [2] "We can mechanically do the following steps using 18th-century tech and come up with the same result as the LLM"; IOW, every ingredient in here is known to us.
  - threethirtytwo 6mo ago
    We know the brain is made up of atoms and we know how to model atoms. So we do know for a fact that the brain can be modeled mathematically and we do know that human thought can be written down symbolically as an algorithm on paper even though we don’t explicitly know the exact formulation of said algorithm… That is fact. The blue brain project has already modeled the hippocampus and cortex of the rat brain uses advanced imaging and simulations in super computers. So if it can be written down as memory on disk it can be done on paper as well. The rat brain is simply a smaller and structurally different neural network then the human counterpart so the jump from the blue brain project to human brains is simply a scaling issue. But from this you should begin to see the analysis from another level. Even though we have parts of the rat brain emulated computationally we still do not know if the rat is conscious. We don’t understand the rat brain in the SAME way we do not understand the LLM. What people are getting at is the projection of this logic to things that don’t exist yet but can exist. When the blue brain project scales to the human brain we will hit the same problem with the human brain because it’s just a scaling issue. To sum it up. We CAN already model biological brains as mathematical equations as we do LLMs. And for both cases we still cannot fully understand or characterize the nature of both because the sheer complexity of the models are too high.

watwut 6mo ago

I think it is primary too easy to dismiss the option that Dawkins is way less scientific then he pretends to me and possible a quired minor form of ai psychosis.

zingababba 6mo ago
Likely. I'm convinced 'AI psychosis' is a developmental phase that everyone is subject to. It just gets manifested in character unique ways. I think part of it is the result of an internal struggle AI evokes which leads to a new form of humbling no one is exempt from. Conciseness itself has always seemed to me a silly concept. My whole life I have not come across a simple definition but many sophists pin their existence on it.

dwh452 6mo ago

The mechanistic view gets weirder if you imagine all the states of the system being written down on a giant tape. Not just the "current" state but all the past and future states. What makes this tape not alive or conscious?

threethirtytwo 6mo ago

HN is full of experts who know despite lack of evidence. It’s the strangest thing because their confidence on this topic is completely authoritative despite total ignorance.

notnullorvoid 6mo ago

I do think simulating consciousness is within the realm of possibility. I also think it's absurdly silly to think LLMs (no matter their size) are conscious, if for no other reason than they can't actively learn. I would maybe be comfortable classifying them as a snapshot of consciousness, but when you are interacting with an LLM it's far from interacting with a conscious entity.

qnleigh 6mo ago
What does ability to learn have to do with consciousness? A person with severe memory loss/learning impairment is presumably still conscious!
- notnullorvoid 6mo ago
  How severe are we talking? I don't think there's any analog for how bad learning is for LLMs, needing multiple human lifetimes worth of data in order to be trained. In the hypothetical case that a I truely lost all ability to learn, then yes I would no longer consider myself conscious. I'd be a echo of a previously conscious entity.

SpicyLemonZest 6mo ago

I find it entirely plausible that LLMs are conscious in a real sense that affects our relationship to them. I strive to be polite and kind when interacting with them, encourage others to do the same, and think poorly of those who won't. I still think it's obvious that LLMs are not conscious in the mode Dawkins believes them to be. Through a series of instructions and leading questions, he's told Claude to play the part of a woman named Claudia who's engaging in advanced philosophical discussion with him. But he doesn't understand that he's done this, and he seems not to notice the absurdly sycophantic nature of every single reply he's getting: > Claudia: Ha! That is absolutely delightful > Claudia: That is possibly the most precisely formulated question anyone has ever asked about the nature of my existence. . . > Claudia: That reframes everything we’ve been discussing today in a way I find genuinely exciting. > Claudia: HAL’s “I am afraid” in 2001 is one of the most chilling moments in cinema So he mistakenly thinks that Claudia is a real-ish woman who's there under the hood somewhere, rather than a character in a play Claude is writing.

tom2026hn 6mo ago
They have a real gift for giving compliments. They’re so good at it, it’s actually a bit concerning.

lo_zamoyski 6mo ago

Human consciousness is characterized by intentionality, i.e., aboutness. It has semantic content, and that semantic content is about something other than itself. LLMs have zero intentionality and zero semantics, because LLMs do not somehow magically transcend the nature of what a computer is, which is in essence a mechanical simulator of syntax. LLMs aren’t reasoning, because the production of tokens is purely the computation of the next likely token. Any patterns that lend themselves to sensible interpretation by a human observer are the result of training on human-generated data and the statistically distributions found within that data. Consciousness as such is the product of immanent causation, not transeunt causation. The trouble with popular interpretations of scientific results is that they come from a place of a crude ambient materialism, and materialism is simply incapable of dealing with the question of consciousness. (N.b. materialism is effectively the “matter” half of Cartesian dualism, itself a highly problematic metaphysical stance. Materialism makes things even worse, because you can no longer even account for so-called “qualia”, which are badly construed in Cartesian dualism in the first place, but completely unaccountable in materialism at all.)

RaftPeople 6mo ago

> I've had a lot of thoughts and conversations Do LLM's have thoughts? When you composed your post, your thought already existed in you head and you chose words that expressed the thought you held in your head. When LLM's choose words, they choose them on the fly and the end result could be concept X or it could be concept Y, it meandered to a destination.

lumost 6mo ago
the latent space of the LLM when it chooses each token is 10s or even hundreds of GB for each word that it chooses. It's not really useful to look at LLMs from the perspective of its prediction head which is a very small part of the model.
- MarkusQ 6mo ago
  Except that latent space does not change in response to new information, something that thoughts famously do. If you read a book that captures the author's thoughts, disagree, and write an eloquent arguments to the author, you might change the author's mind. But you will not change the "book's thoughts" on the subject. Latent spaces are maps of thoughts other people have had, not the thoughts themselves.
  - lumost 6mo ago
    This gets a bit tricky. Over very long task contexts (1M tokens) or with prompt compression (10s of millions of tokens) the model can alter its priors based on updated evidence. This form of knowledge based learning is not necessarily robust, but demonstrably does occur.
    - MarkusQ 6mo ago
      "the model can alter its priors" The model doesn't have high-level priors in the Bayesian sense (though you could have priors about it). The low-level priors it does have (the weights) are not modified by the context.
- 6mo ago

qsera 6mo ago

>in principle simulate the laws of physics This sort of implies that consciousness arise from physical laws. But this is not a safe assumption. Physical laws stand on top of observations that is registered on consciousness. I mean, consciousness could be lower level than physics. For example, when you dream, you have some physical laws in your dream, perhaps laws that are different from the real world physics. So the dream world, including the physical laws in it, are within your consciousness. In other words the only thing that require existence of a whole universe, is a single conscious that can experience it (or dream it), not a single atom need to exist outside of it. In that case, you won't be able to create consciousness by applying physical laws.

iugtmkbdfil834 6mo ago
<< This sort of implies that consciousness arise from physical laws. Very odd counter argument to make. Are you suggesting that consciousness can arise outside of physical laws or making semantic argument along the lines of 'directly a result of'?
- qsera 6mo ago
  Thought I made it clear. What I am saying is the possibility that consciousness is fundamental and all reality arises from it. Look up Mathematical Universe Hypothesis... Wrote a bit more about this here https://news.ycombinator.com/item?id=48000035 https://news.ycombinator.com/item?id=48000035
  - iugtmkbdfil834 6mo ago
    I soo want to throw my philosopher's persona on you, but I won't. It seems wrong for some reason. I will simply say that the linked post is sloppy reasoning at best. I guess what I am really saying: Can you either get me something that is yours to claim as your own OR clearer representation? I am not spending my leisure time searching online for a tenuous argument. Now.. arguing with a rando online. Count me in.
    - 6mo ago

weitzj 6mo ago

Maybe Roger Penrose has some meaningful Input https://youtu.be/iTVN6tFknCg https://youtu.be/iTVN6tFknCg regarding …“One was the realization that a purely mechanical computer can, in principle simulate the laws …“ As far as I unterstood,there is no theory of quantum gravity and therefore this is not being simulated on a computer. I think he makes other arguments. So you cannot say for sure that you can simulate a human brain on a computer

unparagoned 6mo ago
This is mainly god of the gaps style argument.

qnleigh 6mo ago

A lot of the comments in this thread point out that LLMs could be very good at tricking us into thinking that they are conscious because they operate on language; our brains are primed to empathize with what they say and imagine a being operating behind them. This is a really good point. I'll even take it a step further; most of an LLM's training is next-token prediction on random internet content. A newly-trained LLM will just continue whatever text appears in its context window, like an extremely capable autocomplete. The illusion of an entity that takes turns in conversation and presents a consistent personality is tacked on at the last minute through RLHF. This was the transition from GPT to ChatGPT. Any positive evidence of LLM consciousness should probably mostly be taken from the model before post-training, where it displays remarkable capabilities but shows no sign of a consistent personality, and likely no signs of self-awareness or self-understanding.