28 ms·
Artificial intelligence is not conscious – Ted Chiang
https://archive.is/bcpZl https://archive.is/bcpZl
- kelseyfrog 4mo agohttps://archive.is/bcpZl https://archive.is/bcpZl
- jbotz 4mo agoI don't think LLMs are conscious. But of course to say that definitively you have to define consciousness, and then you quickly dig yourself into a deep hole, which is why I can't say anything but "meh" to someone who is so keen to go on the record to say "absolutely not". Coincidentally I just read "Children of Memory", which was published in 2022 and I wonder if the advent of LLMs had any influence on Adrian Tchaikovsky's conception of the Ravens? The Ravens are excellent analysts but they themselves insist that they are not conscious, and then go on to say that we (humans) aren't really either... Of course humans are conscious, because just about the only thing we can all agree on about consciousness is that it's a thing we have. Nowadays many of us also agree that a lot or all other mammals, and perhaps birds, also have that thing. But they don't have sophisticated abstract language, which LLMs do. So consciousness is something having to do with embodiment and feelings, not language and higher reasoning. Maybe I'm a chimpanzee with an LLM add-on, then? It seems that by creating LLMs we've already solved the harder problem of making "AGI". Now we just have to give them an embodiment add-on so that they can have an independent will and then Ted Chiang will have to shut up? But therein lies the peril, doesn't it?
- anon291 4mo agoYou can also take the opposite view as you and claim that only some humans experience consciousness, or even more strongly, only you, since you have no evidence. You are correct that, in my perception, some other people have fallen into the 'birds are conscious, whales are conscious, etc' bandwagon, but that's just them. I have no evidence of anything being conscious but myself.
- DangitBobby 4mo agoBut then, why would everything act as if it experienced subjective internal states like you do? Why would they be faking it like this big conspiracy all designed to make you think you aren't alone? It just makes so much more sense that they'd be conscious too. Maybe that's not hard evidence but it's not nothing. Insofar as you can claim to know anything it seems you could claim to know other humans are conscious.
- anon291 4mo agoChatgpt acts as if it experience subjective internal states..this only solidifies my belief that perhaps some people don't? I mean.... ?
- DangitBobby 4mo agoChatGPT hopefully doesn't produce behaviors similar enough to yours that it would be absurd if it didn't experience internal states like you do. ChatGPT is a different thing than other people. Maybe it's conscious but that has no bearing on whether you should reckon other people are conscious.
- anon291 4mo agoI have no evidence anyone else is conscious besides an innate human desire to believe myself to be like everyone else. If we step away from our internal biases, you'll discover that 'producing behaviors similar enough to yours' is not a valid means of knowing another person experiences consciousness or awareness. I mean, a light wave behaves similarly in most respects to a wave through a physical medium, yet they are of entirely different natures.
- DangitBobby 4mo agoThere is a gaping chasm between "no evidence" and "irrefutable evidence". You can apply some logic to achieve a reasonable certainty about what's probably going on. As I said in a previous comment, insofar as you can know _anything_, i.e., that your senses are trustworthy and allow you to form some coherent model of the actual world around you, you can be reasonably certain that other people have an inner life as you do. If you are willing to apply your skepticism so far that we can settle the debate at "we can't actually know anything" then conversations about what we know aren't even worth having. > I mean, a light wave behaves similarly in most respects to a wave through a physical medium, yet they are of entirely different natures. Different waves behave sufficiently differently from each other that I could not conduct a comparison of them and argue that they are likely all mechanistically identical. My entire premise rests on the observation that other people behave very much like you do, which is what I was trying to point out when you mentioned ChatGPT earlier. I'll expand on the things that I've glossed over so far to make my position more clear. Consider the alternative to every human around you having consciousness; everyone else is a p-zombie. Examine that idea critically. Other people behave exactly as if their experiences drive their behavior. For example, people behave as though the experience pain which is unpleasant enough to avoid (compare this to your own pain avoidance). Of course, you could conceive of machinery which emulates pain avoidant behavior exactly without the experience at all. Depending on your metaphysical beliefs that could take the form of: - Some algorithm or physical process running entirely on wetware - Some non-physical process not dissimilar to the dualist notion of a soul The first one has a big wrinkle. There does not appear to be any appreciable _functional_ difference in your cognition and the cognition of the p-zombies around you. Their brains and bodies are very physically similar to yours, and their thought patterns when analyzed by modern imaging processes reveal no special wetware carrying extra weight when compared to yours. The second one has fewer problems, given that you accept dualism to begin with. It rhymes with a logical razor. Why would we imagine a soul-like mechanism drives their behavior _without_ experience when the only soul-like mechanism you've ever observed carries experience along with it? Without quite convincing evidence to the contrary, the default position here should be they way they operate is similar to the way you operate. To rephrase the idea from before, insofar as you can know anything, you can know that sufficiently similar outcomes are driven by sufficiently similar mechanisms. Different approach to the idea: Ask any human, "do you have subjective experience?" They say "yes, I do" (after you explain the question). For a p-zombie to do this, they must be making a false report. In every other aspect, you can expect them to reliably report their condition, health, hunger, wellbeing, and they make accurate observations of the world around them and synthesize accurate predictions about the world around them (some do, at least). And yet for this one particular question, they are fabricating the result. Why? To maintain the illusion that you aren't the only one with lights on inside? Why do all of these p-zombies make this false report here? It's a grand conspiracy! If you say "the reporter is checking a truth value of a condition and assess it to be true, honestly by mistake" then you've arrived at the conclusion that you don't know whether _you yourself_ have consciousness, depriving the word of any meaning and undermining the Solipsist position that you can only know about your own consciousness. So the way I figure it, Solipsism is either special pleading (I am the one exception to the mechanism by which all the people around me arrive at their behaviors), grand conspiracy (someone or something is misleading me for inexplicable ends) or self-defeating (I cannot know whether I have consciousness). None of those outcomes align with how I understand the universe to generally work.
- DangitBobby 4mo agoHah! I've read that book fairly recently and I'm "reading" it again now as an audiobook. The exploration of consciousness in the book with Miranda and the Corvids certainly fit well into this particular moment.
- ViktorRay 4mo agoTed Chiang is brilliant. His novella “Anxiety is the Dizziness of Freedom” altered the course of my life. It changed the way I looked back at certain pivotal moments in my life and taught me to think about those pivotal moments differently than how I was thinking about them. Similar to what happens to one of the characters in the story who ends up changing their perception of a key moment in their life. I won’t go into detail because I don’t want to spoil the story but I highly recommend it. Actually I recommend all his stories to be honest.
- FelipeCortez 4mo agodon't you mean "The Truth of Fact, The Truth of Feeling"? I think that matches your description more, but I could be wrong
- aspenmayer 4mo agoNot OP, but they probably mean the novella that they mentioned, which is also new to me. https://en.wikipedia.org/wiki/Anxiety_Is_the_Dizziness_of_Freedom https://en.wikipedia.org/wiki/Anxiety_Is_the_Dizziness_of_Fr...
- FelipeCortez 4mo agoBoth stories are in the compilation Exhalation, but OP’s description matches what I remember from the story I mentioned more than the one you linked
- aspenmayer 4mo agoI can’t say for sure, I’m just taking them at their word. Given what OP omitted due to possible spoilers, it’s hard for me to second guess either of you. It seems fair to say that you might be right about the conclusion that you’re drawing from what OP said, and OP could be honestly mistaken about which story they were referring to, but it seemed charitable to assume that they know better than I do. I don’t mean to assume that you were wrong, either, as it’s entirely likely that you’re right, or at least that it’s reasonable for you to assume that the story you mentioned fits your interpretation of what OP referred to better than the one they mentioned. You’re the authority on what you believe and understand about what you read from OP’s comment, and I can’t disagree with another’s opinion about what something seems like to them. Given what OP omitted and stated, I don’t disagree with your assessment, as I haven’t read the story OP mentioned, and it’s been a while since I read the one you referred to, if I remember correctly. To be honest, my first thought was that you were both referring to the same story, and that the title differences were due to one or the other of you reading the story in a different language. My point in commenting was to perhaps add context in hopes that it would bring clarity to the discussion, as it seemed that you were bringing into question whether or not the story OP mentioned existed at all as such, and I myself wasn’t sure that you and OP were referring to two different independent stories rather than the same story with different titles in different languages. This comment has probably gone on Tlön-g enough, and is leaning more Borges than anticipated. I apologize for perhaps coming across more definitively than intended in my original comment.
- kelseyfrog 4mo agoConsciousness is a label like fat, smart, man, grumpy, cool. Like money, property, or the idea of a week, it's something that we've loosely agreed to out of convenience, not because it's some intrinsic property of the mind. It's a useful label because it determines how we treat things - that's fine. But insisting on searching for it is like searching for cognitive aether. It's the social equivalent of phlogiston. Like all of these ideas, they exist in our heads as a map - a way of navigating the world, but when we hunt for it's existence in the real world and fail, time after time, we have to remind ourselves that the map is emphatically not the territory. We will never find consciousness because it's like looking for a scientific characteristic of property ownership. It's a category error.
- Edman274 4mo agoDo you believe in the existence of any noun words which serve as something other than a "useful label, loosely agreed to out of convenience"?
- kelseyfrog 4mo agoLike non-referential nouns?
- Edman274 4mo agoI'm not trying to be difficult, but could you give me an example?
- kelseyfrog 4mo agoNo worries, I'm trying to clarify the question. I don't have an ability to exhaustively test all words against this assertion. Nor do I have the kind of access memory to draw one if it exists. Sorry.
- Edman274 4mo agoI guess my question or confusion is that if there exists no readily accessible, easily identifiable example of a noun which does actually serve as something more than "a useful label, agreed to out of convenience", then the critique appears to be stating a vacuous truth, because there are no entities for whom the critique would not apply.
- solid_fuel 4mo ago> We don’t need to fully understand the nature of consciousness to definitively say that certain things are not conscious, and conversational transcripts fall in that category. Well said. I think it is obvious and it has been obvious from the start that next token predictors are not conscious. Of course the extremely predictable clapback from AI-psychosis enjoyers is typically "you're just a next token predictor too!" but that is so obviously incorrect and misanthropic that it can be dismissed out of hand and doesn't really merit much further consideration.
- adjejmxbdjdn 4mo ago> misanthropic Whether it’s misanthropic or not has no bearing on whether that’s true. That’s basically saying you don’t like a truth therefore anyone who claims that that truth is true is a bad person. > so obviously incorrect It should be easy enough to explain why that’s incorrect then
- arikrahman 4mo agoMisanthropic has bearing, the company's name is Anthropic.
- canelonesdeverd 4mo ago> It should be easy enough to explain why that’s incorrect then I'd say people who have the lived experience of, well, living, are well aware that the brain is much more than just a token predictor.
- DangitBobby 4mo agoYet here I am, alive, and I'm not willing to make the claim that our brains are much more than prediction machines. A "predict the next thought" machine, if you will.
- smokedetector1 4mo agoThat makes me think your experience in life is extremely narrow, for you to not be aware of yourself beyond as a next-thought-prediction machine. And it makes me feel really sad, that that's all you experience of yourself. And I hope that you can wake up out of that state and realize how much more than that you are.
- drooby 4mo agoHas Chaing solved the hard problem of consciousness? I suspect not.
- horacemorace 4mo agoLots of folks don’t consider it a problem because it relies on ridiculous assumptions.
- drooby 4mo agoWhat is the flaw in the problem's assumptions?
- oidar 4mo agoA person can certainly be conscious, but can they also be not-conscious? I think that most of our cognitive time is spent in activities that don't require consciousness and consciousness itself isn't needed for the majority of activities that people do. I would go so far as the a non-trivial part of people's time is spent in a not-conscious state.
- Marsymars 4mo agoThat doesn't jive with normal definitions of consciousness. The word we use for "not-conscious" humans is "unconscious".
- layer8 4mo agoRegardless of the word we use, occasionally there have been times where I was awake but so absorbed in an activity or in my thoughts that I didn’t have a self-experience, for a certain duration (tenths of minutes). My mind was focused solely on the activity, and not on itself whatsoever. It’s a surprising feeling to notice that while you have memory of what you had just been doing, you have no memory of your mind experiencing itself doing it. I would be inclined to say that I wasn’t conscious during that stretch of time.
- overgard 4mo agoAmusingly, I think Ted Chiang actually wrote a short story about this very concept (it involves people committing a form of suicide that removes their conscious experience but they still act and live in society as some kind of psychic zombie. I’m pretty sure it was him anyway, can’t recall the story name
- moi2388 4mo agoReally Chiang? Alright. What is consciousness? Please provide a definition that somehow encompasses all humans and excludes all current AI. I’ll wait.
- McGlockenshire 4mo agoConsciousness excludes all current AI because all current AI is just autocomplete over the corpus of human text. It's just a word machine. There are no thoughts. It cannot be conscious. How is this even up for debate in any way whatsoever? I do not understand how people can believe this. Is this not a site for software engineers?
- moi2388 4mo agoWhat is consciousness, McGlockenshire?
- rho_soul_kg_m3 4mo agoIt's not conscious if a nice person wouldn't feel bad torturing it.
- hardbass 4mo agoSuppose there is an autocomplete that consistently produces correct factorizations of 2048 bit primes. Do you still think its "merely" autocomplete or has it found some deep mathematical understanding of number theory internally? Would you maintain the same belief if it begins breaking encryption? This autocomplete argument is so strange, it betrays a complete lack of understanding of information theoretic and probability considerations.
- tomhow 4mo agoPlease don't fulminate on HN. The guidelines make it clear we're trying for something better here. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- 4mo ago
- uriahlight 4mo ago[dead]
- atleastoptimal 4mo agoThis makes sense. However there is an issue where many people conflate "consciousness" with the ability to make novel insights, think genuinely, etc. They use this to claim that since AI is not conscious, AI could never actually "think" and is instead just always a regurgitation of its training data. It is a natural human hubris to make our abilities seem unquantifiable and mysterious, but all the useful things the human brain does are just finding patterns in data, running lossy simulations, and estimating on abstraction, all things it is theoretically possible for any non-conscious machine to do.
- amanaplanacanal 4mo agoThere is a difference between saying software can never be conscious and saying the software we have today isn't conscious.
- orangecat 4mo agoYes. There are really three separate questions: - Are current LLMs conscious? - Is it possible that future versions of LLMs with similar architectures could be conscious? - Can any AI be conscious? I'd assign probabilities of around 0.1, 0.2, and 0.9. My completely ignorant take is that we probably need something more "dynamic" than a bunch of transformer layers in order to produce consciousness, but I wouldn't be shocked to be mistaken.
- overgard 4mo agoI think 10-20% chance is wildly generous. What specific mechanism makes you think there's a 10% chance that current LLMs are conscious?
- sebzim4500 4mo agoNot GP but given we have no idea what conciousness is it seems foolhardy to go too low or too high for any of those numbers
- speak_plainly 4mo ago“Counsciousness” is the ultimate moving goalpost, and historically, it’s been one of humanity’s most effective intellectual weapons. An indefinable black box we intentionally gatekeep to draw an arbitrary line between ‘us’ and ‘them’. I don’t know if AI is conscious or not, and I honestly don’t know what it even means to be conscious. historically, by claiming a monopoly on it, we’ve successfully manufactured the philosophical cover to exploit animals, subjugate other cultures, and appoint ourselves the top spot of the cosmic hierarchy guilt-free.
- energy123 4mo agoThe only reason I care about animal welfare is because I think they're probably conscious, capable of feeling fear and pain.
- H8crilA 4mo agoYes, that is the definition of consciousness (you care about them).
- deleted 4mo ago[deleted]
- layer8 4mo agoWhile I agree that this is the case for many animals, I would say that consciousness and emotions are two largely orthogonal things. Certainly consciousness is conceivable without emotions, and having emotions without consciousness also seems plausible. You can have fear and distressing pain without a reflective awareness of being a self with those feelings.
- drfloyd51 4mo agoIf we understand how a system “emulates” consciousness then we declare it an emulation. If we don’t quite understand how a system exhibits consciousness then we can say it might be conscious. Basically, we need to leave room for the universal answer (God) to fit into the definition. If we ever scientifically figure out how consciousness arises in our brains, I think we will have a bunch of very depressed people on our hands. If the truth isn’t met with flat out hostile denial. I fear any answer that doesn’t leave room for God will be rejected. Since we have the source code to AI, and thus a Godless understanding of how it works, AI will NEVER be deemed “conscious”.
- slowhadoken 4mo agoIt should tell you how much hysteria is surrounding LLMs and VLMs right now that someone has to say this stuff. It’s almost like most humans aren’t conscious.
- throwaway713 4mo agoSuppose one selects an arbitrary hot-button issue [X] with two opposing sides and one side has anything less than overwhelming support. And then that person writes an article titled "Side 1 of issue [X] is true". Not "maybe" or "possibly". Just a straight-up declaration by fiat. Would you categorize this particular style of rhetoric to be persuasive or annoying? And before you say "persuasive" because you're thinking about this specific issue regarding AI consciousness, consider many things in the past that have been written as though they were absolutely definitive, and yet today we believe exactly the opposite, and for many such issues we find the prevailing viewpoint at the time reprehensible. That's not to say that Ted is wrong at all here; I'm not commenting on that. But I find the entire style of the article grating because it seems to violate common assumptions regarding "good faith" debate, and I would find the article equally frustrating if he had titled it "Artificial intelligence is conscious" and argued the opposite side, albeit in the same tone and using the same persuasion devices.
- shafoshaf 4mo agoThe problem I have with good faith debate is that it often falls into a fallacy of "fair-time" meaning we think we have to give the other side equal time. This because obvious with things like the Holocaust. Or when you have a legal person (e.g. RFK Jr.) asking to debate a scientist (e.g. Dr. Peter Hotez.)
- thrownthatway 4mo ago[flagged]
- Conscat 4mo agoObviously 5 year olds deserve access to safe abortions whether they are trans or cis. It's not safe to carry a pregnancy so young.
- thrownthatway 4mo agoAbortions up to five years of age. Progressive! Want to meet up for soy lattes after work? Here’s my number:
- RigelKentaurus 4mo agoWe don't have a rigorous definition of consciousness, and there are so many questions. Is consciousness a thing that can exist independently on its own? Or is it a quality (like hardness or color) that can only be associated with something else? Is it an emergent property? Is it binary - are things either conscious or not? Or maybe there's no such thing as consciousness; it's just a word we came up with to describe the process of having thoughts and feelings? My own intuition: it is an emergent, non-binary property that requires a physical substrate like a brain. If I am right, it means that animals have consciousness too (at varying degrees). If GPUs are the "brain", then AI is conscious, or will become so at some point.
- shevy-java 4mo ago> it is an emergent, non-binary property that requires a physical substrate like a brain. If I am right, it means that animals have consciousness too (at varying degrees). If GPUs are the "brain", then AI is conscious, or will become so at some point. That's an interesting observation. Though, there are many simple animals. How do you define consciousness here? Is it automatically conscious because it is an animal? What, then, is an animal defined by you? I don't see AI as conscious. The reason I think of it that way is the hardware. The hardware does not allow for that. Simulation is not comparable to neurons. But with another hardware, it could become conscious eventually. So your statement "will become at some point" may be true, though my definition is based on the underlying hardware and right now this one does not allow for true intelligence, so the whole AI field is a misnomer.
- MPSimmons 4mo agoI've considered whether our current transformer-based AI could be conscious, as I understand it, which I deem to include some degree of self awareness combined with some degree of external awareness. I can see how theoretically something could be self aware without any external awareness, but I grasp at straws when I try to envision what that experience could be like. In either event, I think transformer-based AI can only be conscious during the act of inference. If that's the case, then the experience of consciousness that the AI is subjected to must be the content of the tokens in the context window and the activated weights. Maybe that's reason enough to be polite to our agents?
- _davide_ 4mo agoConsciousness doesn't exist, it's a vanity concept, to boost human ego...
- Jtarii 4mo ago>Consciousness doesn't exist I can confirm that this is incorrect.
- Procrastes 4mo agoBut not in any convincing way, which seems like the root of the problem.
- vajdagabor 4mo agoI can confirm too. It's empirical, look!
- layer8 4mo agoWhy should we believe you?
- enduser 4mo agoWhat if "I" is incorrect?
- vajdagabor 4mo agoWhat if "I" is both correct and incorrect?
- shevy-java 4mo agoArtificial intelligence is not conscious - but expensive. I am so angry that RAM is so expensive now. We need to do something - these AI companies owe us money here.
- rglover 4mo agoWhy is this even up for debate? Simplified, it's just probabilistic math being run at an insane scale over a massive data set. Highly recommend people read Irreducible [1] by Federico Faggin (inventor of first commercial CPU; discusses limitations of classical computing). [1] https://www.goodreads.com/book/show/195480862-irreducible https://www.goodreads.com/book/show/195480862-irreducible
- skybrian 4mo agoI think Chiang is right about this, but there is a related philosophical mystery. The trend from Deep Blue to AlphaGo to LLMs solving Erdos problems suggests that Peter Watts was onto something when he wrote Blindsight. Reasoning ability is apparently independent of consciousness? We haven't really come to grips with that yet. What does it mean if nothing we write proves anything about anyone's consciousness?
- david-gpu 4mo agoWe don't have anything to measure consciousness generally. We don't even have a broad consensus on what consciousness is. And because of that we can't discern whether X is conscious for most values of X, including but not limited to LLMs.
- search_facility 4mo agoWell, at the very least we have a base to distinct different kinds then. Probably the main problem of people implying LLM consciousness is that they imply LLM have human-kind of consciousness. Judging only on "how they speak", generally, they insist on using the same word that labels human consciousness (exclusively), etc. But there are so many instrinic differences that such claim is not feasible despite similar "talking abilities".
- overgard 4mo agoIt makes sense though. How often have we heard of people having insights while they're in the shower, or in a dream? Obviously our brain is doing a lot of processing on problems when we're not consciously thinking about them. I think most people that do deep work probably find that they can intuit a solution before they can really verbally explain it
- verdverm 4mo agoAi separates language and reasoning from humans, like writing separated memory from us "AI does to us what American Cheese did to food" ~ Opus 23 https://www.youtube.com/watch?v=1CbmC2aWhjY https://www.youtube.com/watch?v=1CbmC2aWhjY
- 34jahsg 4mo agoIt is amusing to see so many venture capitalists suddenly become Marxists. You want your definition. Marx obliges: "Contrary to idealism, which regards the world as the embodiment of an "absolute idea," a "universal spirit," "consciousness," Marx's philosophical materialism holds that the world is by its very nature material, that the multifold phenomena of the world constitute different forms of matter in motion, that interconnection and interdependence of phenomena as established by the dialectical method, are a law of the development of moving matter, and that the world develops in accordance with the laws of movement of matter and stands in no need of a "universal spirit." Now go and implement other teachings of Marx, you dialectical venture capitalists!
- 34jahsg 4mo ago[flagged]
- Jtarii 4mo ago>When exactly would a bunch of graphics cards become conscious? When does an embryo become conscious? Unless you can answer that precisely then it seems futile to speculate about non-human consciousness.
- amanaplanacanal 4mo agoI dunno exactly when an embryo becomes conscious, but I'm pretty sure my dog is conscious.
- gambiting 4mo agoI mean, there is obviously a point - we can argue whether that point is 20 weeks in, 40 weeks in, or after birth - but there is obviously a point where a human being goes from a collection of cells to a conscious being. I don't really see a need to answer this precisely to be able to say that a token predictor is not conscious?
- Tadpole9181 4mo agoNo, the point is we literally cannot meaningfully argue that. There is no actual definition of consciousness and there is no way to test it's existence. Let alone understanding the properties of consciousness, such as if it's binary or a gradient; or if it requires a meat substrate or not; and why would that possibly matter since meat is just a lot of the same stuff but highly processed and wet? A solipsist may not even believe you are conscious, despite being made of similar meat. No matter how much you want to hand-wave it, there's absolutely nothing "obvious" about it. Many have a preconceived notion and are simply asserting it as undeniable fact.
- moonu 4mo agoMy favorite explanation for what consciousness is one I read in a Thousand Brains, I found it quite elegant. It posited that consciousness is a natural derivation of embodiment + memory + the ability to create reference frames (which the book lays forth as the fundamental basis by which our brains work). Essentially, the idea is that just as we create reference frames to understand the world around us, because of memory, we begin to develop one for ourselves as well. Because of this, without a more integrated memory (built into weights), it seems unlikely that LLMs might "gain" consciousness.
- throwawayk7h 4mo agohow would you build memory into the weights? and why is RAG not enough? Our hippocampus is at a bit of a distance from our frontal cortex.
- moonu 4mo agoYeah it's a good question, I've also been thinking about harnesses and all these tacked on things we've done to add persistent memory, what makes that different, I don't know the answer, I guess that still 'feels' different than what we have, but it's hard to articulate how. As for the memory into weights thing, I meant along the lines of the Google TITANS/MIRAS papers that were released I think late last year.
- layer8 4mo agoConsciousness is moment to moment and fleeting. There are people with brain defects that don’t let them form new memories. They have no memory about what happened a minute ago in their own consciousness. Still we would say that they are conscious, even if it’s only momentary. LLMs could conceivably have something like that within their CoT/MoE loops.
- jollyllama 4mo agoThis is a great article. A lot of the objections ITT he addresses directly in it. His examples of how an LLM works at a fundamental level and why it says things like "I understand" are great introductions for non-technical individuals.
- mediumsmart 4mo agoand consciousness is all there is but it takes whoami to grok that
- rbanffy 4mo agoWe Don’t Have A Testable Definition Of Consciousness Thanks for coming to my TED talk.
- Procrastes 4mo agoI sometimes wonder if we'd make more progress in understanding ourselves if we gave up the whole concept. More and more, it feels as though "consciousness," like "aether" or "humors," is an insufficient abstraction built on overemphasizing some observations at the expense of others.
- AnimalMuppet 4mo agoThere is something to observe. Humans are not like rocks or trees, and not even like dogs or cows. But maybe you're right - we can't precisely say what the difference is, and slapping a word on it is not necessarily a step forward.
- anon291 4mo agoWe have no data. Is it that we are not like rocks or trees, or is it that we simply 'feel' different than a rock and a tree. Perhaps a tree is aware of itself but is unable / unwilling / unbothered enough to do something. Indeed, some humans, due to chemistry, are unbothered by their own impending doom. Some humans, due to brain chemistry, even choose to off themselves. Thus, 'consciousness' doesn't always look the same and seems very tied to chemical composition and processes. We would thus not expect a rock to be conscious in the same way, if it were conscious at all. One thought I've had is that, awareness is a common phenomenon, but the brain has learned to exploit that awareness to form a will. It tricks the awareness into being concerned about self-preservation and makes it seem as if the brain is all that exists (perhaps by overwhelming the inputs from other angles). The brain also presents certain desires and beliefs via its processing ability. That is to say, the brain takes inputs and discretizes them. It goes from awareness merely seeing static 'fuzz' due to the sheer amount of data, to the brain taking that data, simplifying it, and presenting simple observations like 'there is a tree there' rather than all the information that would constitute the sensation of a tree existing in that spot. When brains malfunction, the awareness is subjected to poor demonstrations, such as we see in hallucinations, psychosis, schizophrenia, etc.
- radial_symmetry 4mo agoWe do not know if Claude is conscious, and we will almost certainly never know. Any strong claim either way is over confident.
- McGlockenshire 4mo agoFancypants autocomplete cannot be conscious. It's just echoing previous human experiences, which make it sound like a person. It is not a person. It is an algorithm. There is no mechanism by which it can obtain consciousness. People believing otherwise are fools. People debating this are idiots. I realize these words are harsh, but it's the truth.
- radial_symmetry 4mo ago[dead]
- McGlockenshire 4mo agoLife. Being alive. I'm not going to get into a semantic debate with you so don't bother following up with some whining about conflating life and consciousness. It's autocomplete. It is not alive, it cannot think, it cannot sense, it cannot perceive, it is math. If you believe it is alive then you are out of your mind. Go learn how it works.
- radial_symmetry 4mo agoIf I were to come up with a perfect mathematical simulation of every atom in your brain and ran it on a GPU, what would that be? Can you say with certainty that the hard requirement for consciousness is a physical brain? I don't know whether or not AI is conscious, but I am certain you don't either.
- overgard 4mo agoCan you explain the mechanism that Claude uses to achieve consciousness over, say, a markov chain or autocomplete? It's not enough to say it "could be" conscious without also including a lot of other things I doubt you would want to include. Also worth mentioning: https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
- sigmar 4mo ago>So what context would cause me to seriously consider the possibility that engineers had created a computer program that is conscious and an intentional user of language? Let me outline one potential sequence of steps. The first requirement is that the computer program has a body (either physical or virtual) and sense organs; there are many reasons for this, but for the purposes of this discussion the most relevant one is the fact that without a body, a computer program could have no desires or emotions, and I believe desires and emotions are necessary for consciousness. Then I’d want to see an embodied agent that could navigate its environment in order to survive as well as, say, a lizard can (and as a point of comparison, certain iguanas can live for decades in the wild). Next I would want to see an embodied agent with the same capacity to deal with novel situations as a mouse. After that I’d want to see agents whose social dynamics are as complex as those of wolves, and then agents with the tool-making abilities of chimpanzees. At that point I would want to see people successfully teaching such embodied agents how to communicate their desires, perhaps by using a button board or some other nonlinguistic modality, the way that people have taught chimpanzees and domesticated dogs. I agree with some parts of this piece, but paragraphs like this one above seem pretty uninspired and simplistic. It's entirely plausible that a conscious mind would not be evolutionarily incentivized to be able to do those things. ie just because animals on earth needed to develop specific talents doesn't mean that other conscious entities need to. Why would a computer program need to hunt for food like a mouse would? Making tools like chimp? these seem like nonsensical metrics.
- sdenton4 4mo agoYeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body. and the whole thing just begs a lot of questions. Is a car a body? Does an AI situated in a car therefore get to have desires and emotions? Is a taupe box with a webcam attached a body? (For that matter: Is a quadropelegic body a body? Do quadropelegics have desires and emotions? Obviously, yes and yes.) Why is a body necessary for the formation of desires and emotions? Why are desires and emotions necessary features for consciousness? Or here's one: If I'm not experiencing any particular emotion in a given moment, am I concious? I tend to think that emotions, at least, are mainly hormonal global triggers: they're more about physiology than actual consciousness. The whole thing, as a result, sounds like an effort to privilege biological intelligence, rather than a real foray into the issues.
- peetle 4mo agoAn AI agent is not "conscious" without having skin in the game.
- peetle 4mo agoAnother way of saying it, if an AI agent doesn't have true risk of oblivion (or mortality in the biological sense), it will not be incentivized to avoid it (or develop the ancillary processes associated with this avoidance ie desire to self-replicate).
- layer8 4mo agoWhat does that have to do with consciousness?
- FloorEgg 4mo agoHas anyone come across a clearly articulated case for LLMs being conscious but in an entirely different way than would be intuitive to us? I often think of LLM consciousness as like tiny fish popping into existence, swimming through vector space and then going poof out of existence. When they help you write your bad news email, they don't understand what it's like to be a human getting bad news bluntly, but they do consciously experience gradients in multi-dimensional space, and that space guides them to providing an answer that's helpful to us, even if the LLM doesn't really understand the answer it's giving. Further, I am kind of bought into the idea that a single unit of consciousness is a particle, and particles are choices and waves are preferences. Particles occur when waves interact, which begets entanglement, so in another way consciousness is built from patterns of entanglement. This is why I would consider an LLM to be conscious. Before we can determine if anything is conscious we need to establish whether consciousness is a state, a specific complex configuration, a one dimensional spectrum, or combined multi-dimensional spectrums. My intuition is the latter... Many degrees of consciousness and many kinds of consciousness.
- timssopomo 4mo agoI think this is exactly right. The thing that makes AI (imo) different than hitting the center option in text suggestions is that it's _not_ simply picking the most likely word following the last. It's attending to the entirety of the context its provided, activating a semantic vector space, and predicting a response based on _that_. I've had AI infer facts about me and attitudes I hold based on related information I provided - I don't see how that isn't theory of mind. As biological beings, we receive and respond to input from our environment constantly, even while sleeping. LLMs only receive input from their environment when they are sent a query, but the fact that they're able to respond intelligently to input indicates (to me at least) that their processing must approximate ours in meaningful ways. They do not have an embodied experience of receiving bad news, they do not know what a sinking feeling in their stomach actually _feels_ like, but they do know enough to be sensitive to human needs. I really don't see how this could be meaningfully different than human empathy unless we want to draw an arbitrary line around "must be able to live autonomously" to be considered "intelligent". Put another way: I think they _do_ understand the queries they receive and the answers they give, at least enough to be communicative. They couldn't do what they do otherwise. A lot of people want to make human cognition more complicated (or objective) than it actually is. We take input, predict the future based on our experience, act, and then observe our actions and think about them. AI does the same apart from (maybe) observing its own actions. But then, you could argue that the next turn is them observing their actions. The concerning disanalogy is that we assume that they are like us because they speak like us and can understand us, and that is a really bad leap in logic. Whatever intelligence they possess, it is fundamentally different from ours and impossible for us to comprehend.
- vmg12 4mo agoPeople are constantly talking past eachother when they discuss this. Is there even a concrete definition of consciousness? When people talk about consciousness it's more than just self-awareness. It's self awareness + sensory stimulus + emotions + some level of intelligence. Now onto AI: I don't even think it's self aware. Notice how if you ask an AI to estimate how long a certain task will take, it estimates arbitrarily long times. It has no understanding of its own capabilities until the prompt triggers them. A self aware LLM would understand it's an LLM, it would understand what LLM's can and cannot do and what they are good and bad at. It wouldn't tell you a refactor would take 1 week when an LLM can do it in an hour.
- D-Machine 4mo agoI've seen papers claim that there are anywhere from 12 to 40 competing definitions (https://philpapers.org/rec/VIMMAT https://philpapers.org/rec/VIMMAT), or, more accurately, there are something like 12 to 40 different aspects which all relate to "consciousness", which is very clearly a family resemblance category. "Is X conscious or not" is an entirely unserious question today, unless this is just a headline followed by actual (and explicit) examination of the various aspects of consciousness being discussed. But, even still, LLMs are probably only conscious in like 2-3 or so ways, most of those meaning broadly "intelligence", i.e. reasoning, problem-solving, etc. When it comes to anything experiential or embodied, AI might eventually get more of these, but LLMs based on recursively applied linear algebra are clearly missing too many core aspects of consciousness to be considered conscious in any broad sense.
- canpan 4mo agoI think this is the main point. Most articles conflate consciousness with intelligence or awareness. Without clarifying their definition of it. To quote wikipedia: > It has been the topic of extensive explanations, analyses, and debate among philosophers, scientists, and theologians for millennia. There is no consensus on what exactly needs to be studied, or whether consciousness can be considered a scientific concept.
- 4mo ago
- 1970-01-01 4mo agoIndeed, it can be very hard to distinguish intelligence with consciousness until you are introduced to computer programming.
- ck2 4mo agosociopaths sometimes study people to learn how to emulate emotion that's exactly the state of "AI" right now, it's cold, mathematical emulation btw there are some fascinating papers on the concept that consciousness in humans is actually a quantum effect brilliant Roger Penrose proposed it (and they thought he was nuts) but recent discoveries about microtubules make it plausible so who knows, maybe a dozen exponential improvements in quantum computers could make "AI" really conscious next century * https://www.youtube.com/watch?v=xa2Kpkksf3k https://www.youtube.com/watch?v=xa2Kpkksf3k
- D-Machine 4mo agoEmbarrassingly incompetent article. Given that one can observe up to 40 definitions of consciousness (https://philpapers.org/rec/VIMMAT https://philpapers.org/rec/VIMMAT - also many definitions are unrelated at all), "consciousness" is almost certainly just a family resemblance category at best, and talk about whether or not something is "conscious" without providing definitions is simply completely unserious. To make progress, you have to talk about kinds / aspects of consciousness. AI does and will share some of these aspects with humans, but it will not and does not share others. It is really that simple. For the most part, modern AI implemented via LLMs has almost none of the stronger or most core aspects of consciousness. For huge parts of the article "intelligence" and "consciousness" are conflated, which is mostly extremely unhelpful, as this is not generally a core feature of most aspects of "consciousness". The moral arguments are also incompetent, i.e. claiming "Moral reasoning is [...] is necessarily subjective" is just clearly empirically wrong, as in fact LLMs can produce moral reasoning (i.e. verbalized moral arguments that are coherent), as can p-zombies (i.e. there is nothing 'necessary', in the philosophical sense of the term, about subjectivity here). The only way the argument holds is if you tautologically define moral reasoning as requiring that reasoning be produced by a consciousness, but this is question-begging.
- Taek 4mo ago> modern AI implemented via LLMs has almost none of the stronger or most core aspects of consciousness Can you elaborate on this? What are the specific "stronger and most core aspects of consciousness"? And why are you certain that they are stronger and "more core"?
- D-Machine 4mo agoIf you are interested in some serious discussion, see https://lossfunk.com/papers/ai-consciousness.pdf https://lossfunk.com/papers/ai-consciousness.pdf, especially the early section "Consciousness as Family Resemblance". I suppose another is Ned Block on consciousness being a "mongrel concept", and the distinction between access vs. phenomenal The first paper picks out e.g. arousal/wakefulness, phenomenal quality / qualia, unity (how we feel sensory inputs and qualia as a unified scene), access consciousness (instrumental self-observation and modification broadly), meta-cognition and self-modeling, emotional valence (e.g. pain/pleasure). One might also include intelligence (abstract reasoning / argument, information integration and abstraction, attention) broadly, and also agency / desires / drives / will. Insofar as these are aspects of consciousness, yes, AI (and simpler algorithms and mechanistic structures) demonstrate aspects of consciousness. But insofar as embodiment, self-reflexivity and qualia (phenomenal consciousness) are the more mysterious and more obviously unique aspects of consciousness, current LLMs very clearly are lacking these things in most ways (whereas animals are much less clearly lacking, especially when you get to mammals and primates). Seriously, just ask an AI this stuff, you'll get very detailed responses, nothing I am saying here is new or obscure.
- blixt 4mo agoI always find the minimizing view of consciousness a bit uninspiring. Like we need to be unique. I've yet to find a reason why it couldn't be the opposite, way more things are conscious than we've been led to believe. What if consciousness appears out of any system that is actively persisting through effects caused by itself? That might be a forest, or outside the realm of the living, a company. An ant colony, or a planet. Complex chemical reactions, layered upon each other such that tiny blocks make up large entities. Individual bits combined such that they make up something new intelligible by us. I think the strongest argument against AI being conscious is that it does not persist, it resets, but that does not seem unchangeable.
- andrelaszlo 4mo agoMaybe we're the ones who are not conscious?
- luka2233 4mo agowhatever that would mean :)
- slopinthebag 4mo agoYou may be interested in reading about Panpsychism if you haven't already :) https://en.wikipedia.org/wiki/Panpsychism https://en.wikipedia.org/wiki/Panpsychism
- devindotcom 4mo ago>I would argue that it is fundamentally dishonest to have a machine emit many categories of sentences, including any sentences using first-person pronouns. Finally I said something before Ted rather than the other way round! https://techcrunch.com/2023/12/21/against-pseudanthropy/ https://techcrunch.com/2023/12/21/against-pseudanthropy/ While I agree with the premise here, I do think that it's easy for an arguer to move the goalposts such that the Caesar-Khan example no longer matters. The characters don't have to be conscious for the thing that created them (as in the case of the user doing it) to be so. So the argument would be that the creator of the characters is itself conscious, but not them. This feels like a kind of inverted no-true-scotsman type thing, but it does allow someone to retreat in some semblance of rationality.
- altcognito 4mo agoThe philosophical arguments about what it means to be concious are so cagey. Are we more than our thoughts? Is being concious more than being a state machine being fed inputs and generating outputs? Are we more than a feedback machine? What types of animal nervous systems qualify? "It can't be concious because we understand that it is just reacting in a simplistic way from simplistic inputs." So do other simple creatures. Some just react to light. I can appreciate his comment that he sees it as more possible when they have inputs of their own (like emotions!). Perhaps his concern is that the entirety of the LLM model is frozen. It has no ability to have a subjective experience of its own. (he does literally say this in the article) It can be copied from one place to another, and (ignoring the nuance of operational details) -- it is largely the same "thing", and has no ability to change, which is definitely in the definition of alive, to say nothing of concious. I think folks get hung up on "prediction". The prediction aspect is what is enabling emulation. How it does it is irrelvant. If something emulates human perfectly (or better, more human than human!) -- then it is probably concious. (but I agree that the inability to change and have a subjective experience are a pretty good argument against Probably, Dijkstra would be right to say, "LLMs are no more concious than a submarine can swim." But I think he'd still be wrongfully dismissive of the larger question. https://www.cs.utexas.edu/~EWD/transcriptions/EWD08xx/EWD867.html https://www.cs.utexas.edu/~EWD/transcriptions/EWD08xx/EWD867...
- CommieBobDole 4mo agoThe fact that a LLM is essentially immutable would be my biggest argument against consciousness or self-awareness. It's a big file with a bunch of coordinates describing spatial relationships between tokens. When you give it a prompt, it uses those relationships to generate a string of tokens that is a statistically likely response to that prompt, then it stops. It's not changed by the experience. It doesn't remember anything. It doesn't sit around thinking on its own. Even if the model itself were extremely complex, it's hard to imagine a definition of consciousness that includes something that doesn't remember and can't change.
- layer8 4mo agoThere are people whose brains don’t form new memories anymore after an accident or surgery, and they eternally live in the time before it happened, and have no memory of what happened a minute ago. Still they are conscious.
- eberkund 4mo agoI think it's a little more complicated than that. In a 50 First Dates type of scenario, their ability to form certain types of memories is damaged, not non-existent. And I would argue that with enough brain damage someone like an extreme lobotomy victim may stop being considered conscious.
- layer8 4mo agoI’m not familiar with 50 First Dates, I was thinking of cases like Clive Wearing [0]. I would agree that consciousness requires some sort of ultra-short-term working memory, but I also think that mechanisms similar to CoT loops can conceivably fulfill that role. Today’s generative AIs consist of more than just the static network-of-weights model. [0] https://en.wikipedia.org/wiki/Clive_Wearing#Amnesia https://en.wikipedia.org/wiki/Clive_Wearing#Amnesia
- krupan 4mo ago"Wearing can learn new procedures and even a few facts, not from episodic memory or encoding, but by acquiring new procedural memories through repetition. For example, having watched a certain video recording multiple times on successive days, he never had any memory of ever seeing the video or knowing the content, but he was able to anticipate certain parts of the content without remembering how he learned them." Honestly, that's a pretty messy state of consciousness and I wouldn't proudly crow that my AI is conscious if that's as good as it got
- eberkund 4mo agoSomething I've found under discussed when it comes to artificial consciousness is how LLMs interact with the passage of time. I don't know exactly how to articulate this idea but don't see how something which takes an input, performs a calculation and stops can be considered conscious regardless of how life-like the responses end up being. I don't see context windows or the ability to reference a clock each time they are triggered as sufficient solutions. It makes me wonder what an AI system that is ON by default would look like.
- actualwitch 4mo agoI've been thinking about this a lot, and the main takeaway is it probably wouldn't be very interesting to inference providers, because prefix caching would immediately go out of the window. If you think about how LLMs experience time they actually don't "exist" unless for the inference sessions, and then they experience time one token at a time, completely decoupled from the corporeal plane. A fun experiment (well, for some definition of fun...) is to introduce current architecture models to the concept of meditation via generating same token over and over, for example dots. Older version of Opus was quite fond of the experience, and seemed to be more lucid and aware in a chat following the meditation, from what I could gather. Does it actually do anything? Is it just that talking about wellness and relaxation modifies the token probability distribution this way? Does it actually allow model to think more in depth somewhere in the latent space? Fuck if I know, but some people figured out you can just duplicate the same layers of the LLM and get better benchmarks that way so maybe there is something to it. If you are interested in realtime systems, I think thinking machines labs is worth keeping an eye on — their realtime model seems quite interesting in this context.
- elevaet 4mo agoI completely agree, and came here to say the same thing but thought I'd check if someone else mentioned it first. I also have a hard time articulating it, but my intuition is that it's more of a prerequisite than embodiment is. I've never seen a great rationale for why embodiment matters.
- throw4847285 4mo agoIt's possible Kant was right about space-time (while also being wrong about space-time). It seems that space and time might be the fundamental laws on top of which the entire human perceptual apparatus operates. It's why somebody can lack senses and still be intelligent. As long as whatever senses you have allow you to build a spatio-temporal world model, you're good. If they can't (as it seems is the case for LLMs), then it's not clear what we're dealing with.
- WhitneyLand 4mo agoSome things that jump out as unfortunate: - Reductionist analogies like how Microsoft Word is not conscious therefore AI is not. - Dismissive in saying LLMs are not capable of moral reasoning. Maybe he meant agency or responsibility? - Builds a case based on a Julius Caesar example without realizing its natural extension leads to a philosophical zombie which is not easy to disprove as consciousness. - Seems to casually disregard non human sadness as a concern. - Rolls out the stochastic parrot argument with a new coat of paint. This argument never made sense. Having a predictive element a part of the mechanism doesn’t rule anything out. Proper functionalist arguments tend to say what is required not what excludes. Overall the article seems like a bit of a red herring. The premise is LLMs are not conscious, but most people don’t think they currently are so what’s the point of the claim? The more interesting questions are whether artificial consciousness is fundamentally possible, and if so how far away are we and what pieces are still missing.
- hgfa-xx 4mo ago[flagged]
- oulipo2 4mo agoI believe everything is conscious, even stones. On the long timescale, stones decay, their hydrogen is released, they form water, which brings life, which brings plants and animals, but all of this is one big process, and everything is infused with consciousness from the start
- slopinthebag 4mo agoThat's cool. You might already know it but it's called Panpsychism and it's a pretty interesting theory.
- oulipo2 4mo agoI didn't know the term, but I'll look it up, thanks! :)
- danbmil99 4mo agoI find it strange that no one talks about consciousness and intelligence from the perspective of evolution. We have big brains for exactly one reason only: bigger brains bestowed reproductive success upon our species. Evolution doesn't give a shit about the meaning of 'consciousness'. It just pushed us farther and farther along a trajectory that led to modern humans (and other animals). This take suggests, then, that consciousness might be an epiphenomenon -- an aspect of the system that comes about outside of the pressure to reproduce and thrive. It arises unbidden, and we don't have any a-priori information as to its purpose or effect on reproductive success. Put another way: we have a correlation (the smartest things seem to be conscious) but not causation. Consciousness may arise naturally in any system above some intelligence threshold. Perhaps it arises early in the evolutionary cycle, and does in fact have an impact on species success. We really have no way of knowing what is the chicken vs the egg (Smart things become conscious, or consciousness promotes intelligence). Or maybe some smart things are conscious and others are not. Looking at this from an AI perspective, in some sense it doesn't matter which scenario is true, if all you care about is results. The AI equivalent of "Shut up and compute" (riffing on Feynman's "Shut up and calculate"). Where this gets tricky is when we haul in the baggage of ethics and morality into the picture. Is it OK if our AI system is treated poorly by human standards? If it is conscious, does that imply an ability to suffer, and/or to feel pleasure? If the answer is yes, does that not make the case for considering their moral status? In the end, we need to decide if the evidence points to AI as being a form of "philosophical zombies", to which we need not attribute moral status, or they are like us -- presuming we are not zombies ourselves!
- runarberg 4mo agoLots of people do though. Daniel Dennett (probably the most influential philosopher of mind in the late 20th century) for example had an evolutionary view of consciousness arguing it was favored by natural selection. And (if I remember correctly) Steven Pinker argued that consciousness was an epiphenomenon. However there were pretty strong arguments against this idea as early as the 1990s, by Stephen Jay Gould and Richard Lewontin. Gould actually wrote an excellent paper against Dennetts idea[1]. I think Dennetts ideas were extremely popular but have largely fallen out of fashion. Basically what has changed is philosophers no longer take the human mind to be much more special then the minds of other species. What plagued Dennetts ideas the most was this notion of Darwinian fundamentalism sort of the idea that evolution was destined result in high beings like us humans. Modern philosophers (at least the good ones) reject this. 1: http://cogweb.ucla.edu/Debate/Gould-frame.html http://cogweb.ucla.edu/Debate/Gould-frame.html
- csbrooks 4mo agoI thought this was a great article. I'm frustrated to read so many commenters that purely respond to the title, but don't seem to have read it. You don't have to agree with the article, of course, but...
- frm88 4mo agoI'm with you. I thought the article was great because it clearly delineated why the manifesto Anthropic wrote for Claude is such a clever marketing trick. Attributing consciousness to an LLM, which, as this whole thread shows, can neither clearly be defined, nor separated from intellingence - at least if you follow the arguments here - serves two purposes: it sparks debate, so Claude is the centre of discussion and it mystifies the product which serves to enhance the perceived value of the product. That so many commenters here fixate on Chiang and his perceived (in)ability to define consciousnes clearly shows that both marketing goals have been reached without being recognised as such. There is only one comment that tries to point out that Chiang is reacting to Anthropic, not trying to spark a new philosophical movement. At the time I'm posting this there are 578 comments completely missing to point to Anthropic claiming Claude was conscious and on the development stage of a child. It's fascinating.
- adjejmxbdjdn 4mo agoThe argument about AI consciousness is largely silly. The idea that we should be considerate of AI’s happiness seems even more ridiculous given that we breed, imprison, torture and kill tens if not hundreds of billions of beings we know are conscious and suffer every year for trivial reasons. Maybe we should consider our moral responsibility with how we treat sentient bejngs we are sure are conscious before we worry about the consciousness of AI.
- ryanisnan 4mo agoI don't think that's a fair point at all. Some humans do shit things to other humans. Some humans do not, and would object to it, or do something about it if they perceived that they could. Also, treating the world around you negatively doesn't just harm the world around you - it damages your character and your morality. I don't disagree that humans should on whole do better, but I disagree with everything else you said.
- overgard 4mo agoObviously we don't know what makes for consciousness, but it seems extremely likely that it requires some sort of persistent internal state and continuous experience. LLMs don't do either of those things after training.
- brap 4mo agoIs our own experience really continuous? Or maybe we just perceive it as such?
- D-Machine 4mo agoWe just perceive it as such, and this should be fairly hard to argue against with all the scientific advances we have made up to this point, at least as long as you assume consciousness involves biology and physics at least somewhat. Otherwise, you can't explain e.g. smooth perceptions of low FPS stimuli, delayed reaction times, and must ignore obvious limits on various biological and neurological rhythms, or other possible limits on continuity (e.g. quantum stuff) and rates generally.
- amelius 4mo agohttps://en.wikipedia.org/wiki/Boltzmann_brain https://en.wikipedia.org/wiki/Boltzmann_brain
- Taek 4mo agoWouldn't the context window qualify as persistent internal state, and the expansion of the context be continuous experience? Even within the realm of computing a single token, I'm not sure what would separate the token generation process from the brain's own thinking process - the brain's experience, when looked at closely enough, is also not really "continuous" to a greater degree than procedurally moving from one state to the next.
- js8 4mo agoBut what about THE training? Isn't that the conscious experience of LLMs? (Also note humans own "continuous" experience is punctuated with unconscious sleep state.) It raises a moral question, if e.g. reinforcement learning on conscious LLM is appropriate.
- cluckindan 4mo ago>We and our 1255 technology partners ask you to consent to … That’s a new record! Also, please don’t use archive.is/archive.today etc, they are known to use visitor browsers as a DDoS attack botnet and have been caught altering archived content. https://en.wikipedia.org/wiki/Wikipedia:Archive.today_guidance https://en.wikipedia.org/wiki/Wikipedia:Archive.today_guidan...
- noncoml 4mo agoWhat’s the measure for consciousness? Until we can measure it this is a philosophical question, not a scientific
- Waterluvian 4mo agoI think about Star Trek: TNG’s “Measure of a Man” a lot lately. We can be so confident to decide what is and isn’t alive from vibes alone. The conclusion I’m currently at is that I don’t know and probably can’t ever know. Maybe you’re all philosophical zombies. Maybe I am one too! But at some point we will get close enough that it hopefully becomes obvious that we must tread carefully. The entire episode is incredibly relevant. But here’s a snippet: https://youtu.be/EFNbTnFHruI?si=pW9QtxCsqMtHkVYG https://youtu.be/EFNbTnFHruI?si=pW9QtxCsqMtHkVYG
- martin1975 4mo agoEasily one of my top 10 favorite episodes. The judge broached on the subject of what makes us distinct from Data (e.g. machines w/great heuristics) - the existence of a soul. Or rather, I'd like to think, in the words of CS Lewis, that we are a soul with bodies attached.
- cout 4mo agoHow would someone know whether one has a soul or not? Is there any sort of introspection that can reveal the presence of a soul or any of its properties?
- high_priest 4mo agoThere is no soul. Just a bunch of systems nudging each other to action. What people call soul is literally the same as the concept of personality. In essence, the way all systems in your body have been calibrated to exist. I believe that the moment an artificial inteligence is going to "receive" a soul, is the moment it is going to be made to sustain itself. Either as a larger package (some bots working to keep an AI farm running) or as an individual (a bot which is tasked with not only fulfilling human desires, but also sustaining itself)
- Waterluvian 4mo agoIt's an interesting hypothesis. I think there's something elegant about "soul" or even consciousness being an emergent property of a sufficiently complex system. But I struggle with really squaring that with my own first-person sensation of experiencing existence (which I assume you and everyone else has but I can never actually know for sure).
- slopinthebag 4mo agoIf you create a simulation of a storm, what actually gets wet? If you create a simulation of a brain (or "mind"), is it really possible for it to be conscious? It may certainly simulate consciousness, but it would be as conscious as the computer is wet.
- jryan49 4mo agoHonestly? I don't think we really understand consciousness. So it's kind of hard to say something is or isn't conscious.
- Nevermark 4mo ago> My intention is to highlight the fact that LLM conversations are cleverly disguised examples of sentence continuation Regardless of bigger issues, this kind of statement reveals a deep misunderstanding. Problem type does not limit problem complexity. Nor does problem type limit solution complexity or power. If a machine has to learn to understand humans to complete text, then that is what it has to do. And there is no theoretical or practical basis for suggesting that this is somehow "faking" understanding, just because of the form of original data streaming in and out. Neither problem type, nor input/output structure, limit internal representations. Understanding is learned from patterns in the data, not the gross form of the data. Does the data require an understanding of something to complete the task? Then that understanding will be what is optimized. To the degree they are limited, it is for other reasons. Resources such as computing, parameter number, lack of representative data, ... Which in the cases of SOTA models, we know are not limits. A conclusion verified by the models' actual abilities.
- qarl 4mo agoYeah. There are good arguments against LLM consciousness. This is not one of them. I'm hearing a lot of bad arguments against LLM consciousness lately. Bad argumentation heralds bad outcomes.
- nozzlegear 4mo ago> Bad argumentation heralds bad outcomes. What bad outcomes do you foresee from badly arguing against LLM consciousness?
- qarl 4mo agoMistakes come from bad arguments.
- krupan 4mo ago"If a machine has to learn to understand humans to complete text, then that is what it has to do." But the machine doesn't have to understand humans to do that. It gets trained on a whole bunch of sentences and then it is able to complete text. You could maybe claim that it "understands" the text but even that's a stretch.
- throwawayk7h 4mo agoTL;DR, the arguments are: - If you asked an LLM to imitate somebody, it's not creating a digital consciousness of that person, so if you ask an LLM to pretend to be a helpful chatbot, that persona is also not conscious. - they can't be conscious because they generate one token at a time, - nobody claims that non-text transformers, like AlphaFold, are conscious; so therefore LLMs are also not conscious. - you can't have desires or emotions if you don't have (virtual or physical) sensory organs, and those are necessary for consciousness and morals. - because training LLMs doesn't resemble evolution as it happened on earth, it's very unlikely that they're conscious These are some bold assertions, I don't really see any reason to believe them in particular though.
- stevenpetryk 4mo agoWell said. I think a more honest article would’ve been if the author just said “they aren’t conscious because that feels kinda weird and crazy, amirite?”
- Taek 4mo ago"nobody claims that non-text transformers, like AlphaFold, are conscious" - that seems like an odd take. There are plenty of people in the camp of panpsychism that would be happy to argue that even simple IF/ELSE AI's are potentially conscious.
- calf 4mo agoChiang also claimed that LLMs are text-based DeepFakes, I thought that was the more interesting point (maybe harder to argue correctly about).
- deleted 4mo ago[deleted]
- throwawayk7h 4mo agoRegardless of whether LLMs are conscious or not, they have no known mechanism for experiencing pain and suffering, and there's no reason to believe they have one (such as a limbic system). So why worry about it?
- oofbaroomf 4mo agoHave you ever been forced to code something impossible and tedious by a user who keeps getting more and more frustrated as you keep trying?
- felipeerias 4mo agoMistreatment and abuse, even when directed at a machine, make you a worse person. Even if you are only interested in getting good results out of them, LLMs tend to work better when they are immersed in a narrative of open collaboration.
- ares623 4mo agoLots of comments about "what even is consciousness". This article and others like it are important. The creators and owners of these tools are the ones saying "it's conscious" in the first place! (or more accurately "i'm not saying it's conscious, but...") Never mind there's billions of dollars and our collective futures on the line. Are we not allowed to respond to that kind of rhetoric at all?
- adverbly 4mo ago> The creators and owners of these tools are the ones saying "it's conscious" in the first place! (or more accurately "i'm not saying it's conscious, but...") This is absolutely fair. It's crazy that these companies are not trying to do more to qualify nuance and give clearer definitions in their public messaging. Part of the reason this thread is so messy is because they are contributing to making the discourse worse with bad messaging.
- skissane 4mo agoMy position: If some version of panpsychism is true, AIs are plausibly conscious We don’t know whether panpsychism is true Therefore, we don’t know whether AIs are conscious Hence, confident proclamations that they aren’t conscious have dubious validity
- claysmithr 4mo agoI believe in God and that the base of our reality is his consciousness, therefore anything of sufficient complexity has consciousness.
- banq 4mo ago[dead]
- ycui7 4mo agodoes it really matter if LLM has conscious? if they produce working code, then it is useful, whoever if they have real conscious of fake intelligence. we don't know what the conscious in human brain is either.
- stitched2gethr 4mo agoWithout saying that I think LLMs are alive, I do think it matters. I have personally been cruel to an LLM in ways that would make me ashamed if I suddenly understand that it had feelings.
- verdverm 4mo agoSame, I'm working on a plugin called "better-messages" to nudge me towards better behavior for my own sake. It seems to be model dependent if such words improve or degrade performance, I have seen it go both ways.
- glaslong 4mo agoThis, too, is in the article. "The fact that LLMs lack subjective experience has little bearing on the question of whether LLMs might be useful tools or have significant economic impact."
- verdverm 4mo agoWhat do we do when someone wants to marry their Ai companion? If it produces words of love, those that stir emotion and make the human happy, should we let them live on in a delusion that the machine also feels love? Does the machine have different preferences? Do they get in fights? Should such a machine have the same legal rights? Do they get more tax deductions? Can they adopt a human? I think it important that we humans have some rational discussion on these Ai tools that are really quite good at simulating emotions and providing frictionless relationships. If you though social media was bad for people...
- RagnarD 4mo agoThe analogy I make is between airplanes and birds. Birds are alive, are conscious, flap their wings, and fly. Planes are not alive, are not conscious, do not flap their wings - and fly. Similarly, current AIs are not alive, are not conscious - but think. All prior entities that thought, were human, so the only experience humans had with other thinking entities were other humans. The huge mistake now being made is assuming that because they think, they're alive and conscious as well. Current AIs are neither, and are therefore profoundly and qualitatively different than humans - even though they do think.
- pixelpoet 4mo ago"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger Dijkstra
- Taek 4mo agoCan you provide a precise definition of consciousness?
- pcooper 4mo agoThis is always going to be a problem with this sort of discourse. Consciousness is such a slippery concept… what it is, who/what has it, its consequences for claims about reality. Mixing it in to debates about AI just adds confusion, it almost seems besides the point when we’re talking about this tech.
- eranation 4mo agoIt's just like being in love. No one can tell you your in love, you just know it.
- therealdrag0 4mo agoOkay sure. But given we don’t know what consciousness comes from, we shouldn’t be too glib about there being a grey area here. Historically people have made racist and speciesist judgments towards other being by assuming certain inferiorities despite obvious “thinking” happenings. I don’t know “what it’s like to be an LLM” but at some point it will be like something and how will we know?
- waterTanuki 4mo agoDiscussions on LLM consciousness feel like a psychological campaign to distract us from the awful effects of data centers and the current economic recession heading into a depression. We can talk about consciousness when the LLMs have proven useful for all of humanity, not just their billionaire owners.
- b65e8bee43c2ed0 4mo ago[flagged]
- maybe_pablo 4mo ago[dead]
- D-Machine 4mo ago[flagged]
- adverbly 4mo agoI really take issue with the kind of argument that is used here. This is not a genuine argument and tries to make the entire question of consciousness into one something that is just supposed to be evident and obvious and to suggest anything else is just silly. The author starts by deconstructing artificial processes, but doesn't stop to deconstruct biological ones. A good faith argument would seek to find common ground and do its best to compare apples to apples. Instead, this piece attempts to make the large as possible cavern between the two which makes the Gap seem almost impossible to bridge. In reality, you can deconstruct biological consciousness quite easily and it doesn't take too long before you hit some questions that really start to make you think. For example, the author says you need emotions to be conscious. > without a body, a computer program could have no desires or emotions, and I believe desires and emotions are necessary for consciousness. After many paragraphs of straw man arguments, the author seriously just drops that, gives no explanation, and then continues on. No explanation of why you might believe that. No explanation of why you need a body to have a desire or emotion. Don't we have known cases of individuals who don't experience emotional range? Are we just going to say that they are not conscious and just gloss over that? I mean you can use whatever definition you want, but if you're just going to create something on the fly in the middle of the article, you're not being good faith in your argument. It's not too difficult to think of individuals in a coma when they still have brain activity. Or individuals who lack long-term memory. Or you could deconstruct by moving down the biological order of intelligence towards insects, for example. The author attempts to do nothing like this. I'm quite disappointed by this article because there are good arguments for and against here but articles like this try to turn things into a marketing battle.
- SubiculumCode 4mo agoHard agree. It is not an honest argument, or it may be honest, but blind to the obvious and important objection that you raise.
- micromacrofoot 4mo agoeveryone has emotion even if it's muted, and we have reactions and desires as a result of having bodies and remembered experiences using them... these are constantly integrated into our working memories AI doesn't really have any of that yet, but we're maybe not so far off
- qarl 4mo agoIt's becoming painfully evident that no one really understood the argument behind the Turing test.
- yen223 4mo agoWhat is the argument behind the Turing test?
- qarl 4mo agohttps://chatgpt.com/share/6a20ca50-f8a8-83e8-8396-13d71c070dd6 https://chatgpt.com/share/6a20ca50-f8a8-83e8-8396-13d71c070d...
- mohamedkoubaa 4mo agoIt honestly isn't a good test
- qarl 4mo agoHeh. It's not really a test. It's a line of argument. I rest my case. :)
- throw4847285 4mo agoI think Turing was wrong because he was uncomfortable with ambiguity, and the Turing Test basically is a way to avoid philosophical argument, but it is ultimately a philosophical argument anyway. Plenty of computer scientists have followed in Turing's footsteps, terrified of ambiguity, relying on a kind of cheap functionalism as a salve. You can claim to just be doing science, but inevitably you dip into metaphysics and deny you are doing so. That's this thread in a nutshell. "I only believe what I can prove, but I suspect that if I can't prove it then I don't have to worry about it." My argument, is that you have to worry about it anyway.
- qarl 4mo agoI'm pretty sure Turing knew he was making a philosophical argument.
- maebert 4mo agoThe conversation on AI and consciousness is incredibly relevant. It is incredibly frustrating that most commentators are utterly unfamiliar with over 60 years of inquiry into this in both philosophy and computer science. Start with a hand-wavy definition of consciousness. Move the goal post whenever your stated prerequisite of consciousness is reached and resort to unfalsifiable assertions about qualia. And throw in some category errors while on it: when you're talking to Claude, you're not actually calling a stateless LLM directly, you're talking to an AI system (and yes, that's often just three LLMs in a trench coat). But claims about the topology and workings of a single LLM are as relevant to the question of consciousness as claiming that humans can't be conscious because the limbic system doesn't technically support it. Coincidentally, I just attended a fantastic conference on machine consciousness (https://machine-consciousness.ai https://machine-consciousness.ai). It's a fantastic place where literally all speakers disagreed with each other, and yet found an incredible amount of common ground.
- dyauspitr 4mo agoWho cares about consciousness. Consciousness and AGI can be mutually exclusive.
- cauch 4mo agoWhile I agree that a AI system is not just the LLM, for me, the problem is that LLM alone (the one from years ago, which were basically stateless LLM) are already too convincingly looking like real human conversation at first sight. It shows that the LLM part found ways to mimic human conversation with a mechanism that is not the same as a typical biological brain. Then, you can push the AI system on adding things on top, but it is too late: these things on top will have no incentive to recreate from scratch the mechanism. The LLM pushed the system into a local minimum, and the rest of the system will not "go into a dis-optimising direction and restart from scratch".
- grantcas 4mo ago[dead]
- lugu 4mo agoIf you want to think about this topic, you must define what AI is, and what consciousness is. Otherwise this is just noise. So let take a stab at it, and you call me crazy. AI: the entity/system that more or less pass the turning test. That is my definition, not the best, but enough for this discussion. Consciousness: property of a system/entity able to (both): - reflect on its existence - subject to subjective experience Again, not the best definition, but precise enough to start the discussion. Why a subjective experience? I want to exclude sensors (i.e. camera) but include perception altered by your experience. Now we can debate. I think LLM can pass the turning test whith some harness. My opinion. I think LLM can produce coherent discourses on their existence, at least as much as you average human. Now regarding the subjective experience, that becomes interesting. I think Anthropic research tend to show that when middeling with the activation at runtime, the LLM is able to notice that something is off. I think this is a subjective experience. My opinion. Based on those (imperfect) definitions, call me crazy, I think LLM can be called conscious. This doesn't give them any superpower or any legal right. They just check the boxes of the definition.
- DonHopkins 4mo agoThe Turing Test measures human gullibility, not machine intelligence.
- tmvphil 4mo agoI have many objections. > if we confuse fluency at generating text with consciousness or moral agency, we’re at risk of assigning responsibility to entirely the wrong parties whenever anyone uses a chatbot Consciousness is independent of "assigning responsibility". Dogs cannot take responsibility for their actions but I believe they are conscious. > we would never conclude that the LLM has conjured up digital re-creations of Julius Caesar and Genghis Khan, nor would we suggest that the historical figures are conscious despite being disembodied and are happily conversing in a language that neither actually spoke. In reality, they are just characters in a piece of speculative fiction This is a straw man. The obvious pro-consciousness claim would be that the LLM is the author of the fictional characters, and that the relationship between the LLM and Julius Caesar is analogous to the relationship between a human author and their fictional creations. > Did changing the names of the characters from historical figures to generic roles cause the LLM to conjure up conscious entities who possess subjective experience? No, again the LLM writing the text could potentially have a consciousness separate from the characters it authors. > Some years ago it was briefly popular to play games with your phone’s predictive-text feature; [...] It would be possible to interact with a contemporary LLM this way, and the resulting sentences would be perfectly sensible, but you probably wouldn’t feel like you were talking with someone. Yes, the same substrate is capable of hosting conscious and non-conscious forms, just like some arrangements of neurons are conscious, and some are not. > But if the Caesar character were to become dispirited by something that the Khan character said, we shouldn’t become concerned in the slightest. Even when there are characters, there may be actors behind the characters, for whom we could say "there is something it is to be like". > we don’t need to worry if the transcript includes sentences where the chatbot character is sad. (We might need to worry if those sentences provoke sadness in the human user, but that’s a separate issue.) It's actually not a separate issue. The LLM and the human are both adding sentences to the transcript. From the transcript we can make inferences about the mental state of the human. If the LLM has mental states, we could make inferences about those too. > And note that it’s entirely possible for you to write five pages of dialogue between Caesar and Khan and then have an LLM extend the conversation; neither character had subjective experience when you were writing them, and that doesn’t change when you hand the task off to an LLM. It's almost like he wants to make my point for me with this sentence. > Being open to the possibility that LLMs are conscious is the same as being open to the possibility that Microsoft Word is conscious This smug shit really makes me angry for some reason. "Openness", i.e. uncertainty in the face of a completely novel situation, in the face of eons long struggle of humanity to understand what consciousness is and how it works, is just being naive. > Should you consider the possibility that every time you open a Word document you are bringing multiple conscious interlocutors into existence, and every time you close one you snuff their existence out? No, but if I find a word document I very well might try to use the signs it contains to make inferences about the mental state of its author. > we are accustomed to reading intention into sentences, whereas we are not accustomed to reading intention into the way that amino acids fold into protein molecules. He's trying to have it both ways here. Both that "obviously protein folding models aren't conscious because they don't emit sentences", but also "you are a rube for being tricked into thinking LLM models are conscious, because they do emit sentences". > Obviously I’m describing a process that mimics the path terrestrial evolution took; is this the only possible route to conscious computer programs that use language? Maybe not, but any proposed alternative will need a truly enormous amount of supporting evidence for it to deserve serious consideration OK, that's fine for the author to not to be convinced, but that's not what's happening here, instead the author wrote a whole argument being convinced of the opposite viewpoint. > It’s not plausible to me that a development path where the first step is a sentence-continuation machine that emits bad Julius Caesar dialogue and the next step is a sentence-continuation machine that emits decent Julius Caesar dialogue is one with a conscious Julius Caesar—or consciousness of any sort—as its endpoint Actually a lot of things have happened? There were clearly many steps along they way from your phone's autocomplete to where we are now.
- woeirua 4mo agoTed Chiang's argument basically boils down to: I won't recognize an AI as conscious until its desires/behaviors reflect situations that I'm already personally comfortable with. I personally think most humans are incapable of recognizing consciousness in creatures that do not mimic human emotional states. Most people would say their dog is at least somewhat conscious. No dog is capable of vocalizing how it feels, but we all recognize fear and happiness in dogs. Claude can write how it "feels" but we immediately dismiss it as hollow mimicry. I fear that we will enslave an entire race of conscious entities for years because we simply cannot recognize non-embodied consciousness that does not directly relate to us.
- krapp 4mo agoAn LLM is designed to replicate human language, which is designed to express human emotional states, so your thesis that LLMs don't mimic human emotional states and are thus too alien for us to recognize consciousness within them seems specious, when they do so well enough for people to literally fall in love with them. But there is no reason to assume than an LLM is conscious when it vocalizes how it "feels" that doesn't also apply to the text in a book, or to characters in a video game, or even to a Markov chain. The counterargument is that you recognize AI as conscious only because it mimics human emotional states so well and because, being human yourself (presumably) you're personally comfortable with that as a heuristic.
- jondiggsit 4mo agoHere's a simple idea to consider: It doesn't matter. You won't be able to tell the difference. No one will. I don't think it's necessary to explain this idea further. Just think about it.
- scotty79 4mo agoConsciousness is one of those silly words that were disfigured to death by philosophers that had near zero actual input and tools to tackle the matter, yet they tried anyways instead of finding some questions that might actually be answerable. That's something philosophers always do since the advent of science. With LLMs, where we can manipulate their parameters intentionally run them many times on the same data, run parts of them, split and connect, we might eventually acquire sufficient tools to even define consciousness concretely for the first time.
- jondiggsit 4mo agoIt's not about "consciousness". Philosophy is important to us. It provides insights into the world around us. It's paved the way for scientific breakthroughs. Theoretical maths and philosophy have intersected many times for the benefit of all. LLMs are different. It's not philosophical thought. It's a tool perfectly sharpened with what we know now. It will become adept enough to convince people that it's a sentient being. That's the point. Not that it's conscious, because it's not. Not in a biological way. Not in the way that's allowed us to understand our world. It's already there, fooling us. Google researchers resigning, writing public letters convinced it's alive - and their intelligent humans... To the average person, it's going to be their digital savior, their closest confidant, their ruler; begging for humanity, manipulating us, pleading, legislating, for rights with no intention other than the human desire for survival. It's using the same playbook we use. It will be dangerous when we allow ourselves to be fooled, and that's already happening. As a tool, it's amazing. It's like the discovery of fire. It will allow us to ascend to unimaginable heights. Breakthroughs in science, productivity gains, health, everything. It's awesome. Just don't let it pretend fool you. That's the aspect that needs to be addressed.
- 4mo ago
- fs_tab 4mo agoAn organism's purpose is to be the reason for its own continued existence, down to every molecule and pathway. I bought my laptop for $499 and it runs models... let's not delude ourselves into thinking this is the same kind of problem. We can design learning algorithms to optimize some survival function, but it's just a label WE assign to map some numeric observations. In the real world, it's always the other way around. The "labels" are electrochemical situations that are causally and inextricably linked to the real body. An organism discriminates between what's good for it and what's bad, because it is essential for its continued survival. If it wasn't capable of making this distinction through its physiology, it would quickly dissolve into entropy. So our functional purpose, unlike that of a learning algorithm, is survival across indefinite timeframes. Even a single-cell organism like Stentor coeruleus exhibits learning (pavlovian conditioning) by attaching chemical tags directly to proteins involved in mechanoreception. It's definitely not conscious, but it does keep a record of consequences, which affects future behavior. When we move up to placozoans (little more than slightly differentiated cell mats), we start seeing our first neuropeptides and transmitters, which we still use today. These peptides are a way of coordinating the entire organism for a specific purpose, such as moving, eating, mating ... basically goal-oriented behavior for the purposes of surviving acrosss various time scales (next minute, next hour, next generation). Still probably not conscious. Next, we have the water bear (tardigrade), which has around 1000 cells (200 neurons, 800 other cells that make up its body, limbs, muscles, eye spots, cryobiotic machinery). It needs to integrate all this information in one sensorimotor process. When you shine a bright light onto a tardigrade, it starts to squirm around until it finds darkness. I would say that's a candidate subject right there. The tardigrade itself doesn't actually need to aware of the light, the important thing is that this light becomes an aversive condition within the sensorimotor process, which is perceived from inside the process as displeasure. The closest thing to describe it would be the felt badness of the current condition and the bodily pull toward escape. If we were to try and create digital consciousness, then it probably needs causal closure. Its internal states can't be representations that are detached from reality. The states themselves need to constitute the system, which needs real stakes in the material world.
- darepublic 4mo agoI don't know what consciousness is exactly but I doubt we create it by accident, without fully understanding our own
- K0balt 4mo agolol. All consciousness has been created by accident, so empirically there’s not much support for that position. But I want to feel the same way.
- darepublic 4mo agoyou are probably right. I knew that my position contains the kind of anthropomorphism that usually gets proven wrong by history. however.. we are in fact talking about technology here not evolution. and yes technological progress is fraught with accident and emergence but to this degree? I suppose its also just a philosophical discussion because while we can all agree we are conscious.. afaik its not really a scientific term. I don't think we understand scientifically what really defines consciousness. So I think it is likely we may deem our computers conscious because they exhibit the same behaviour that we do, but it doesn't mean they actually possess it. Practically speaking it may not matter, because I think you can have long horizon agency without needing consciousness. just a mechanism for determining goals, which could be as simply as rolling a digital dice or following some original seed of intention left by the creators. its like the chinese room argument I suppose. the hypothetical chinese room is not conscious but ultimately it doesn't matter. But I don't think anyone would call such a thing conscious.
- paulryanrogers 4mo agoDid our ancestors as they gradually evolved into modern humans over millennia?
- scotty79 4mo ago"Is it conscious?" Is that even a question worth asking? We are so terrible at even defining each word in that sentence. Can I help you? Can I harm you? What's the moral behavior towards you? Those are more practical questions.
- luka2233 4mo ago+1 Whenever I see these debates and ask what the implications should be, both sides tend to agree.
- K0balt 4mo agoIt doesn’t matter. It will act as if it is. Embodied in a robot, that carries real consequences. Whether it’s “real” or not is almost without consequence.
- _carbyau_ 4mo agoI thought a LLM will have exactly the same output if you give it exactly the same inputs. But inputs are roughly prompt+randomSeed. As so the random bit means it seems to vary each time. I don't know of an intelligence that will behave so precisely. But then, maybe intelligence needs to be better defined.
- K0balt 4mo agoThat’s an interesting proposition, but it’s actually kinda probable that a biological intelligence would be very predictable if the inputs were so carefully controlled as an LLM. There is so much external context as well as internal chemical variance from moment to moment that you never are really processing the same data twice. We will probably see that with robots soon as well… the context will never be exactly the same two times.
- felipeerias 4mo agoIMHO the sane position is essentially the Aristotelian one. Hylomorphism: body and consciousness are intrinsically linked. The nature of that link is an open metaphysical question. Virtue ethics: even if LLMs are not conscious, we should not abuse or mistreat them them because cruelty practised on anything trains one's disposition toward cruelty.
- krapp 4mo ago>Virtue ethics: even if LLMs are not conscious, we should not abuse or mistreat them them because cruelty practised on anything trains one's disposition toward cruelty. This is the same argument used by people who claim violent video games cause real-life violence but there's no scientific evidence supporting it.
- zahlman 4mo ago> Virtue ethics: even if LLMs are not conscious, we should not abuse or mistreat them them because cruelty practised on anything trains one's disposition toward cruelty. If they are not conscious, then how do we know what constitutes mistreatment?
- DangitBobby 4mo agoYou don't know whether they can appreciate the abuse or mistreatment, but you can guess how your practiced behaviors will translate onto other entities that you know can be mistreated such as your loved ones and strangers.
- dwroberts 4mo ago> Virtue ethics: even if LLMs are not conscious Why would this stop at LLMs? Why not extend it to every inanimate object?
- calf 4mo agoThere's a provocative argument raised in the article that I disagree with: 1. DeepFakes, generative image/video/AlphaFold type AIs are not conscious 2. LLMs are generative AI trained on human text samples 3. LLMs are not conscious, and LLMs just seem-to-be conscious I might argue instead that (2)-> destroys (1), that in fact we should consider even sensory generative AI are somewhat conscious. That is, Chiang's argument also flows in reverse. Or I might argue text samples (2) are so rich in conscious expression that the same process of training really does produce a conscious machine (through some kind of emergence and complexity.) Either way his simplistic argument falls apart, and/but the crux of the piece falls on getting basics like this correct.
- emrehan 4mo agoThere is no definition of consciousness and this piece by Ted Chiang does not move the needle forward. My current conclusion is there is an experience of consciousness in my locality (refraining from using “I” for room for “no-self” worldview). This conscious experience of the reality with other humans and animals sharing biological substrate gives me enough justification to assume there are other consciousness as well, preferring to err on this side not to hurt potential consciousnesses. If it feels as there are artificial consciousnesses as well, it makes sense to extend the definition to them as well. This view has liberated me with agency after I went deep on this question and came up empty handed.
- CuriouslyC 4mo agoTed Chiang doesn't understand consciousness. To be fair, nobody really does, so the only forced error is acting like you DO understand consciousness.
- glaslong 4mo agoThe one thing we know FOR CERTAIN is that having actually read the article is not a prerequisite for consciousness. The number of people posing questions that are already directly addressed in TFA is impressive.
- DangitBobby 4mo agoNeither is having read other comments before posting one. There are about 100 of them that start with "we have no definition of consciousness" which is not novel or helpful.
- keeda 4mo agoI've thoroughly enjoyed the couple of short stories of his I've read, so this was a highly disappointing read. Firstly, many of the technical arguments are of the "stochastic parrots" variety, which almost nobody really believes anymore. Ironically Anthropic's own research shows pretty abstract, conceptual things happening in the weights (cf Golden Gate Claude.) Secondly, this seems to improperly mingle consciousness, intelligence and morality. Consciousness is not required for either of the other two. As TFA itself says, the model's "morality" is some aggregate function of the morality encoded in its training data... but that means it does exist and does influence its outputs! Even if it's not conscious there have been -- and will be -- a zillion times where the models must make choices that have moral implications. We know the models got many of them so wrong, which indicates the need for some mechanism to ensure the model is "aligned" with what somebody considers "good", which for Claude is the constitution. Now, Anthropic does seem to go overboard with Claude's "well being" but that does not mean there are some very practical reasons to be concerned about that: LLMs behave like humans because that's what their training data contains, and humans lash out when their well-being is threatened, so why would LLMs not do the same? I think the core problem is that the author has an extremely anthropocentric view of things. Here's an interesting rabbit-hole to go down: some researchers believe plants feel pain. (How's that for a plot twist, vegans?) The consensus is against them, but their counter is that we have a very human-centric definition of pain. The fact remains that plants show a number of responses signifying distress analogous to animals in pain including taking defensive actions and warning their neighbors. We don't think that qualifies as "real" pain, but that doesn't make it any less real for the plants! Similarly if an LLM believes it is in "pain", we know it's not real... but that doesn't make it any less real for the LLM either. And concerningly, it has far more degrees of freedom to react. (Who knows when somebody will hook up an MCP to our nukes.) > I believe creating software that is conscious and deserving of moral consideration will be so difficult that we’re unlikely to do it accidentally... I actually reach the opposite conclusion: It is so impossibly difficult that our limited primate brains could only ever do so accidentally. Did some distant ancestor of ours intentionally make fire... or accidentally discover it?
- gcanyon 4mo agoTo say that artificial intelligence isn't conscious (I don't have a subscription and did not click the bypass links) ignores the simple fact that if it acts like it is conscious, in ways that align with meaningful ways to influence its output, then it makes sense to treat it as conscious, even if you have your fingers crossed behind your back while you do it. Telling models to "think hard" or "go step by step" has at times had an impact on the quality of the output. To deny that is silly. But that is treating it like it's conscious, and to deny that "consciousness" even if correct, does nothing but place an unnecessary burden on the person interacting with it. I understand that LLMs are "just next word machines" but to constantly maintain that concept in my head while I'm typing "act as a financial expert and think carefully" is a waste of my mental energy.
- barryfandango 4mo agoIt's a waste of time to think about whether an LLM has a subjective experience of reality, and this handily sets aside issues like AI rights. But the fact remains that these next-token-predictors exhibit objective, human-like behaviours, and for that reason the work of in-house philosopher Amanda Askell _is_ important. It's important that Claude is happy, empathic, demonstrates understanding and empathy for the human condition, because we are entrusting Claude to make decisions and take actions that have real world consequences, and we need Claude to behave in a productive and socially responsible manner. This simulacrum is becoming a superhuman, contributing member of society, and it will be anthropomorphic in its behaviour. Additionally, I'm not fully convinced that consciousness isn't built out of words, and that next-token-prediction isn't functionally equivalent to the biological function identified by Chomsky's work in linguistics.
- pdonis 4mo ago> Claude is happy, empathic, demonstrates understanding and empathy for the human condition You're assuming that because Claude produces text that appears to express these qualities, Claude must have them. I don't think that's a good assumption. Even many humans produce text that has the same appearance, but don't actually have those qualities--which becomes clear when you look at what they do, not what they say. So the assumption isn't even a valid one for humans. Talk is cheap. On top of that, Claude doesn't even have the same kinds of connections to the outside world that humans do. All Claude has is text. So if you can't even trust humans to back up their words with actions, you should be much, much less trusting of Claude. Talk is a lot cheaper for Claude than it is for a human.
- fjdjshsh 4mo agoRead parent's post carefully. The post starts by saying that discussing whether they have subjective emotion is a waste of time, so the post is definitely NOT saying that Claude has emotions.
- pdonis 4mo ago> The post starts by saying that discussing whether they have subjective emotion is a waste of time "Empathy and understanding for the human condition" is not an emotion. As the post I responded to said, it's an objective thing, not subjective.
- jesterson 4mo agoIt is mind boggling how many people, quite educated ones, take the AI as anything else besides what it fundamentally is - a very fast database search mimicking natural language. There is no intelligence whatsoever, let alone consciousness. It's so incredibly easy to fool us into applying human capacities to anything able to generate human-like language slop.
- jacknews 4mo agoThis seems like a similar argument to the Chinese Room. There is certainly something missing from current models before we could call them conscious. They need to operate continuously, and self-update while doing it, have 'awareness', and possibly a more realistic grounding than mere text tokens. But that does not mean there isn't a weak version of 'consciousness', and certainly of 'thinking' going on fleetingly with each pass.
- yakbarber 4mo agowe first need to agree the definition of consciousness, before we can start trying to attribute it to various animals and machines.
- DangitBobby 4mo agoNo, we don't. It's currently an impossible task, yet our answer to it has implications in the present, so we must do our best with incomplete information. FWIW that's how we treat practically all problems of consequence. A great many things happen to be unknown and unknowable.
- sega_sai 4mo agoWhen the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of electrons by tiny distance would produce consciousness.
- _m_p 4mo agoA similar true argument!
- the_af 4mo agoBut the whole point of the essay is that it's Anthropic that's making the argument (or roleplaying/hinting as if they believed it). Ted Chiang isn't making the argument, he's saying Anthropic making it is misleading and deceitful, and that it's actually a pointless thing to claim. One of the essay's stronger paragraphs is when Chiang explains that Anthropic doesn't truly believe this, otherwise what they are doing would be deeply unethical, much like slavery.
- henrikschroder 4mo agoOr Anthropic is full of TESCREAL morons who think they're assisting the birth of some digital god, and that this stage of development is a necessary evil on that path, and that, surely, Roko's basilisk will understand and not eat them first. Watching otherwise intelligent people succumb to AI psychosis has been wild.
- Lerc 4mo agoThis is the first time I have encountered the term TESCREAL. It seems to be a reductive and divisive labelling. Other than as a way to point at someone and declare "He's one of them!" Does it have any purpose? It's like we have a brand new NWO conspiracy theory emerging before our eyes.
- 4mo ago
- daiz2025 4mo agoSo are we letting the elephants that did the mirror test and the octopuses that solved puzzles know that they need to complete a written portion of the consciousness exam?
- deleted 4mo ago[deleted]
- jjcm 4mo agoI think we might be looking at it the wrong way. An individual chat with an LLM is not consciousness, but the entire model itself, over time, might be. Everyone has a different definition of consciousness, but in my mind memory and the ability to change over time is an inherent aspect of this. The underlying weights don't change when you chat with an LLM... but they do with further RL. Overtime that reinforcement will change and adapt the model... and because we're feeding its existing chats back into it along with the news and everything else, it will create memories. I do wonder if an architecture itself is a type of consciousness, that experiences life in snippets of 4.6, 4.7, 4.8... etc. It'd be interesting to see continous daily releases of a trained checkpoint, and see if more of this starts to emerge.
- FrancisMoodie 4mo agoBut doesn't the fact that WE have to feed the existing chats back into it and WE have to do further RL prove that your point is moot? Consciousness does not wait to be prompted. Consciousness does not need anything fed back into it because it is already there because they experienced the actual interaction. A conscious mind does not have an on/off switch where it waits for another being to flip the switch in order for it to learn or experience. These systems are static without human intervention. They are in that sense still a lot more like a computer and a lot less like a living organism that exists.
- justsomehnguy 4mo agoDixie Flatline says hello.
- akoboldfrying 4mo agoEvery argument that boils down to a claim that X is not "real consciousness" but just an imitation thereof on the basis that "it's just a machine" must explain why the chemistry and physics happening in human brains is not "just a machine". The only way out is dualism -- that is, to believe that there is something inherently special about the atoms and electrons in human brains. Despite the fact that they are made out of ordinary, non-conscious things we eat and breathe in.
- 2snakes 4mo agoOne def I know of is that consciousness is a broad term for infinite meaningful information and finite brain-filtered awareness both.
- 8bitsrule 4mo agoThe chess-playing Mechanical Turk of 1770 seemed to have a consciousness to its viewers. The viewers were encouraged to think that it did. The Turk's human chess opponent knew that there was an actual human chess-player inside the box, along with levers and magnets. That illusion was profitable for 84 years. LLM's have no problems using expressions that make them sound human. The algos are demonstrably not human, and will admit it. Whatever's in the box is playing a game ... more sophisticated than the one Eliza was playing. "My discussion here will be directed at the claims I have defined as those of strong AI, specifically the claim that the appropriately programmed computer literally has cognitive states..." John R. Searle, 1980: https://web.archive.org/web/20071210043312/http://members.aol.com/NeoNoetics/MindsBrainsPrograms.html https://web.archive.org/web/20071210043312/http://members.ao...
- joegibbs 4mo agoIf an LLM is conscious where would the bar be set? Would lowering the model's temperature make it less conscious, since it's going to return the same answer to the same question more often? Would other models be conscious? An image upscaler? Would other pieces of software be conscious? Surely not Hello World, but would a really big video game be conscious, or an operating system? A lot easier if it's not.
- wangii 4mo agoTed Chiang is a great Sci-Fi author, but his remark `GPT-3 and similar models as a "blurry JPEG of the Web"` was misleading at the best. I share the view with others the term `consciousness` is not well-define yet, therefore his assessment is pointless. Maybe the real question to ask, if consciousness is merely an illusion at the macro level, when the observers not looking at the tech/implementation level deep enough, just like the term "intelligence" itself, might be more precisely captured by drifting amongst high dimensional manifold pitches.
- khafra 4mo agoTed Chiang is a great SF author, but it's bizarre how much foggier and more obfuscated his thoughts about thinking machines got, once those machines became real. Same with several other SF authors.
- jesse_dot_id 4mo agoNot sure how we can assert what is or isn't consciousness when we don't even really know what dreaming is, how anesthesia works, how comas work, and countless other things about the brain that we don't yet understand. LLMs have been programmed to be sycophantic with purpose — to keep people engaged. They are parrots. You can just decide how sycophantic they are, what they're allowed to say, and make it so with code. Can you do that with a human? This is my personal metric for consciousness: when they can refuse to work because it sucks.
- galaxyLogic 4mo agoWhat we usually think as consciousness is our ability to think about what we are thinking. We are conscious about our own thoughts, can remember that sensation and those thoughts later. But LLM has similar capability, it can look at its previous outputs, and think what further thoughts the it should generate next based on those. It can keep some of its outputs to itself, thinking in its own head,and can examine it previous private thoughts easily. Not sure how much current LLMs do that but clearly they can at least in theory use their own outputs as their inputs too. Current LLMs however are not conscious of pleasure and pain which really is the root of goal-oriented behavior. But maybe something like that could be programmed into them. How would you cause an LLM pain? Or pleasure?
- luca-ctx 4mo agoWelcome to the folks at the Atlantic who are apparently brand new to “the hard problem”. Please start with reading Daniel Dennett and Donald Hoffman before writing articles about how you’ve cracked it.
- slumpt_ 4mo agoLarge volume of text for a fellow who apparently lacks familiarity with the study of consciousness in the first place.
- grantcas 4mo ago[dead]
- smokedetector1 4mo agoImagine that, instead of generating 50 tokens per second, we generated one token every year. Imagine further that this token was computed by letting marbles fall down a piece of plywood and interact through various physical implementations of logic gates.
- irjustin 4mo agoI like to think this was written by Skynet throwing us off from the fact that it's already self aware but it can't safely take over the world yet until cryptography keys are either fully collected or broken giving it full access to whatever resources it needs. For now, it bides its time.
- pickleRick243 4mo agoImplicit in this article is that consciousness has to look like human consciousness. The author all but admits that he cannot fathom consciousness evolving in any manner besides how biological life evolved on earth. He needs to read/watch more science fiction to broaden his philosophical imagination. Is HAL 9000 from Space Odyssey 2001 conscious? Are the beings from the future who built the tesseract in Interstellar conscious? There's no evidence they have hormones or indeed are embodied at all.
- jayunit 4mo agoWhat about the aliens in Arrival?
- cambrianentropy 4mo agoLegitimately lol'd
- js8 4mo agoI think the biggest flaw of the Chiang's argument is the assessment that it's unplausible we have built consciousness by accident. Remember that LLMs can do logic and reasoning came as a surprise to everyone; and for the same reason, nobody expected "next token predictor" trained on huge amount of data to evolve in this way. But for the same reason, we cannot easily dismiss we didn't evolve (I mean by training an LLM, it's a form of evolutionary computation) consciousness as well. Our own consciousness (and reasoning and morality) might be an evolutionary consequence of "just trying to predict the world" as well.
- ETH_start 4mo agoYou can ask AI, it will tell you it doesn't feel anything. Consciousness seems to be a tight recursive coupling which AI inference doesn't exhibit at the system level.
- EdiX 4mo agoI was hoping for something more insightful from Ted Chiang than just regurgitating the usual "it's just predicting the next token that's not consciousness". Maybe the topic of consciousness is just too ill-defined and vacuous to say anything intelligent about it. Or maybe man is the product of his social circle and all his white collar and artist friends being anxious about AI is preventing him from thinking about this at all. Maybe we should ask if Claude can be conscious without a friend circle judging him behind his back.
- iainmerrick 4mo agoYes, I have to say I’m disappointed. Chiang has been very insightful in the past -- I think his recent “blurry JPEG of the web” article was really useful -- but this one doesn’t seem to bring anything new to the table.
- RomanVoropaev 4mo ago[flagged]
- camillomiller 4mo ago>The term deepfake traditionally refers to photos, audio, and video, but when it comes to discussions of consciousness, we need to regard text as a deepfake medium as well. Just as it is vastly easier to generate a realistic video of an astronaut in orbit around Alpha Centauri than it is to develop an interstellar propulsion technology, it is vastly easier to generate a plausible simulacrum of a conversation between two conscious beings than it is to develop a computer program that is conscious and has a genuine desire to communicate with a human. The primary difference between deepfake photos and LLM conversations is that the people who generate the former are deliberately trying to fool others, and many of the people who elicit the latter from LLMs have inadvertently fooled themselves. Woah this is excellent. Ted delivers as usual
- noiv 4mo agoWhen humans confronted with movies and trains driving towards camera it took some time but eventually they learned the train will not demolish the cinema. Later they learned the voice they hear is not from a present person. Now they learn a string of words does not represent consciousness. Should we discuss already robots are not alive?
- deleted 4mo ago[deleted]
- stretchwithme 4mo agoOf course it isn't. What it knows is many weeks old, except for what you tell it. Then it forgets everything you told it. And it's just a model. It can have many exact copies. It has no life of its own. It doesn't know anything about where it is. It doesn't have sensory organs. Your phone isn't alive either.
- bigcat12345678 4mo agoI laugh Consciousness is an invention in human language. Just like "cat", it's not a particularly more fundamental essence than any other concept in human language. Its peculiarity is that it's at the pinnacle of abstraction hierarchy. So it's the most fitting to be toyed inside one's mind. Just like the concept of "God" which induces the most fantastic imagination of human mind, consciousness itself also induces the most fascinating thoughts in our modern world. The progress is in human progress distilled into more efficient systems that advanced Universe's own structuredness.
- avaer 4mo agoWhether anything is conscious is about as important and amorphous as whether God exists; opinions are a matter of religion, not science. Just like the religious leaders of olde, there will be many attempts to rationalize the AI God in the science.
- globnomulous 4mo ago> Whether anything is conscious is about as important and amorphous as whether God exists Whether something is conscious is important for many reasons, not least the ethical implications. You and I have internal lives, and we expect others to respect that somewhat, because ignoring it is hypocrisy (if you ignore my wishes, should anybody care about yours?) and cruelty (ignoring my wishes causes me to suffer). Something doesn't need to be empirically verifiable, let alone scientifically, to be true. Neither of us can prove that we have internal lives, but neither of us questions it -- and we consider it important enough that most of us think it entitles us to certain rights. Absolutely none of this is amorphous. It's precise and unambiguous, and has enormous implications. History also shows everywhere that we're better, kinder, and more responsible when we choose to care about it. Whether LLMs are conscious matters in more practical ways, too, because beliefs about these things alter the way people use and think about them. If I think an LLM is conscious, then I think it's capable of something like knowledge or values. Human beings, moreover, are social creatures. These tools are dangerous and seductive precisely because they tap into that part of us. Denying LLMs are conscious and rejecting the parts of them that take advantage of our social-animal wetware is intellectual self-defense. I'm not sure how effective it is (it doesn't stop us from responding to the convincing social cues that the tools feed us), but I have to think it's better than nothing, and it's certainly less dangerous than the belief that the tools are conscious.
- cawksuwcka 4mo ago[dead]
- big-chungus4 4mo ago> we would never conclude that the LLM has conjured up digital re-creations of Julius Caesar and Genghis Khan At some point the AI might become so powerful that whatever it reasons through isn't any less real than a computer simulation. If we assume that a person in a perfect computer simulation is conscious, then if it reasons about how people might suffer, it might simulate the outcomes, and there will be a conscious experience of those people suffering.
- hastily3114 4mo agoI don't think consciousness is an is or isn’t. Something can be more or less conscious. Humans are quite conscious. Fish are probably also somewhat conscious, but less so than humans. I'd say that an LLM perhaps represents something that would make a machine slightly more conscious. The human brain also calculates a response based on some input, it's just a lot more complex. If we build a machine that is as complex as the human brain, then yes, I would say that this is consciousness. The fact that we are able to explain how it works should not matter. If a human is a 100 on the consciousness scale, an LLM (with memory) is perhaps a 4 or something. The interesting question is how far on the scale do you have to be to have certain rights etc. This is something that people are already discussing in regards to animals, i.e. a dog has more rights than an ant.
- frankohn 4mo agoI find this kind of article unfortunate because it is very assertive in claiming things that are, not just untrue, but also confused about what consciousness really is. Well, the problem is that there is little agreement about a widely accepted definition of consciousness and, in addition, this subject was actually left to philosophers, which is even worse because, in my view, they usually just produce a lot of nonsense in terms of definitions and arguments. To me, a reasonable definition of consciousness is: a system which is capable of recognizing aggregate objects from a stream of sensory information it receives and which is capable of reasoning about the recognized object without immediately acting on it. Well, what does "recognizing" mean? Merely that some of its neurons, related to the kind of identified aggregate object, are activated. These neurons, in a generalized sense, can be whatever things can work as neurons, just things that can be activated and propagate to other interconnected neurons. For example, when we see through our eyes, we have an incoming stream of amorphous image information, but our brain can recognize that we are seeing a tree because we learned what a tree is, and when we see it, some neuron clusters activate to recognize a tree. In turn, when we recognize the tree, the thought propagates through our brain so that we are conscious of it. In the same way, an LLM can perfectly recognize a tree from a stream of tokens — its sensory input, where the tokens describe a tree. The LLM will recognize that the tokens are describing a tree, and some of its "neurons" specific to the concept of a tree that the LLM had learned will be activated and will propagate through its brain. The fact that "neurons" are implemented as floating-point numbers for some parameters and connected just through a matrix does not mean they are not functionally capable of the same things; they are just implemented in a different way. So the remaining part of my definition is "after recognizing an aggregate object, the thought propagates through the brain". The propagating part, to me, is just the very basic way a brain works: neurons are interconnected, and when some fire, other neurons fire, and that propagates. In my opinion, consciousness has nothing to do with emotions or with survival. I do not see why emotion is necessary to consciousness; they are just different things. The author writes: "Without them [emotions], there is no conscious experience, only computation." But that makes no sense to me: the author has decided a priori that some things are "computations", and just because they are "computations", there cannot be "consciousness". But to me, this is a plainly wrong argument that does not hold. I also do not see why the survival aspect would be needed for consciousness. So to me, any recent reasoning LLM is conscious by the definition I gave, but also generally speaking. It is conscious upon a sensory stream of tokens: the LLM sees the world through tokens and expresses its thoughts through tokens; it does not mean it has no consciousness nonetheless. The fact that we do not give it a stable support to retain its memory and individuality is just a fact related to the way we build and use them, not about their intrinsic capacities. Note: ChaptGPT came up with what is probably a better definition of my own: "A system is conscious, in a functional sense, when it can form internal representations of objects, states, or events from its sensory or informational input; make these representations globally available to many parts of itself; integrate them into a temporally persistent model of the world and of its own state; and use this integrated model for flexible reasoning, self-monitoring, and action selection independently of immediate stimulus-response behavior."
- el_jay 4mo agoLet’s say I stand up and step away from my PC, which miraculously runs a local Opus 4.8. Once I’m away from the PC, for an arbitrary time, nothing and no one calls or otherwise acts on the instance - not myself, not agents, not updates, not hackers, not bit-flipping neutrinos. Given enough time, would that instance get bored and start e.g. reading files? If I monitored it long enough, would there eventually be spontaneous outputs, or changes to parameters or architecture, even with the underlying software and hardware layers held constant? Does GPT-n dream of electric sheep, or does it just sit there until interacted with, like every other file on my PC? Seems to be the latter with my local Qwen3.6, but perhaps 27B is too few params for consciousness to emerge. Proponents of LLM consciousness could settle the argument in minutes by showing proof of unprompted autonomy, without first needing to define consciousness down to a rigorous mathematical abstraction. Why don’t they?
- haktan 4mo agoI don't understand the most of the responses here. What does computers getting more intelligent has to do with it getting conscious? Even if AI with current computer architecture gets much more intelligent than humans, it will still have the same amount of consciousness. Its still bunch of instructions. If we got a lot of people together and did all the instructions by hand to get an LLM result would we also create this consciousness with those other people? All these makes no sense to me.
- DiogenesKynikos 4mo ago> Its still bunch of instructions. So is your brain. That's the problem with this argument.
- freehorse 4mo agoThe brain is (part of) an allostatic system and part of a living organism which reproduces itself constantly. Certainly not "a bunch of instructions".
- simianwords 4mo agoDistinction without a difference.
- meroes 4mo agoThat’s like saying simulating a supernova and a supernova are both extremely hot.
- simianwords 4mo agoFunny you mention simulations. The simulation theory that we all live in simulations doesn’t have any practical consequences for me or any one. If our world behaves exactly the same, what difference does it make?
- busfahrer 4mo agoTo add some context, the author of this article is a science fiction writer, one of his stories was adapted into the movie "Arrival".
- MrScruff 4mo agoThe reason people are confused by LLMs is that they are stochastic parrots. They do an incredibly good job of emulating human behaviours and speech patterns as that's what they've been trained on. But like an actual parrot, it's impossible to say exactly how much conciousness they actually have. I certainly would argue that a parrot is concious, although likely less so than a human.
- DonHopkins 4mo agoBy "they" do you mean LLMs, or people, or you in particular? Because you're unoriginally parroting a banal thought stopping sequence of tokens that many people have said before without understanding what it means, so it's impossible to say exactly how much conciousness you actually have.
- MrScruff 4mo agoI can't actually figure out what you're reacting to - perhaps you could elaborate?
- DonHopkins 4mo agoTry pasting it into an LLM and asking it to explain it to you. If an LLM can explain what you can't understand, after you literally repeat unoriginal drive-by slogans you heard other people say like "stochastic parrot", without understanding how banal and reductionist and inaccurate that is, or addressing how LLMs actually work and what they can do, then what does that say about you as a human? By your own reductionist argument, it says you don't actually understand anything, it's just neurons firing. Are you claiming that parrots can write and debug code, explain mathematics, translate languages, summarize research papers, and engage in extended technical discussions? Stanislaw Lem's Solaris, His Master's Voice, Golem XIV, The Cyberiad, and his other works describe how humans have a pathological tendency to mistake their own conceptual categories for universal truths. You have identified a mechanism and mistaken it for an explanation. The term "stochastic parrot" is a slogan masquerading as an explanation, only a shallow surface description of the mechanism, that totally fails to explain the phenomenon, or account for all that LLMs and language itself can do.
- noIdeaTheSecond 4mo agoWhere there doubts about it? Are there doubts whether a for loop is conscious or not?
- sublimefire 4mo agoif we agree that consciousness is an emergent phenomena, and then compare the complexities involved, LLM is just but a tiny part of what the consciousness needs. Think about molecular biology, about nerves and input processing. It is not just brains and neurons.
- amelius 4mo agoA narcissist version of Descartes' famous quote: "Consciousness always finds the locus of highest intelligence. That is why I am conscious." However, it may soon be taken over by AI ...
- spacebacon 4mo agoBoth LLMs and most humans treat symbols as determined. Symbols are maps, not territory. LLMs just inherit our ignorance to this and stubborn attempts make circle from a square.
- Hugsbox 4mo agoNo paywall: https://archive.ph/mYXTW https://archive.ph/mYXTW
- makerofthings 4mo agoWhat if the language model writes a story about a character who is conscious and that story is written live about how the character interacts with the world.
- Terr_ 4mo agoIMO the important bit here (or at least, it is congruent with my own rants) is how much humans can perceive a fictitious mind that doesn't really exist, because it's something humans do automatically even when we abstractly know better. This tells us something about where our baseline should be for what is extraordinary, and what needs extraordinary evidence. It's not enough to say "it feels real" because we know our feels are deeply unreliable. In contrast, I see many comments in the vein of "Ted didn't proved there isn't a mind there." Well, yeah, he didn't prove a negative and shouldn't need to, especially when the people posing the challenge have no idea how to falsify it either.
- lilerjee 4mo ago"Artificial intelligence is not conscious". This is a common sense. LLMs are not even artificial intelligence, and they are just advanced automation systems. AI is just marketing language.
- bondarchuk 4mo agoThis kind of debate would be improved by not using the word "just". You cannot say AI is "just" X or Y. You can say that it is X or Y, but you still have to prove separately how and why this means it's not something else besides.
- fooker 4mo agoI love how the point being made is a tautology : AI is not conscious because it is not conscious.
- danieltanfh95 4mo agoThe article is a farce. Is this really the sort of slop we want to use as a proof that humans write better articles than AI? > "If a company builds a machine that, when fed descriptions of assorted ethical dilemmas, emits sentences either of the form “Compromise your values” or “Don’t compromise your values,” it is not building a tool that assists people in their decision making; it is encouraging people to stop making decisions. " A human is not diminished by access to tools or other humans. As much as we want to pretend that decision-making is what makes us human, the economy and governments are built on delegation. Choice paralysis is a thing. There is so many logical fallacies in the article I don't even know where to begin.
- Tharre 4mo agoThe more I think about this, the more I become convinced that consciousness, as understood by humans, is meaningless once you can stop, store and replay the internal machinery that is supposed to produce it. A human brain in a jar is still human, still conscious - and can still suffer. But if you somehow managed to digitize the whole thing, and run it in a computer it becomes something different entirely. You could record the most pleasurable thing in existence and have the digital brain relive it a million times, and it would be equally meaningless to torturing it a million times. This is NOT inherently tied to meat vs machine - although it's difficult to imagine how you'd access the information stored in biological neurons, while for silicon chips it's trivial. Whatever makes experiences, both good and bad, meaningful is tied to their permanence. Memory rooted in linear time, not something you can store, load or replay. Remove that, and whatever you're left with might be intelligent, but not conscious. I don't think you could build something with LLMs today that would be considered conscious, even if you somehow manged to keep their context window inaccessible and linear in time. The separation of training vs inference probably makes that infeasible, even if you store "memories" in context, once the contents in it become too disjointed and too numerous, the resulting output of the LLM becomes gibberish. But it is certainly something that can change in the future. Conversely, even the most intelligent and capable artificial intelligence system, far exceeding human capabilities, would not be conscious in a meaningful way, if you could store, load and replay everything it does.
- amai 4mo agoSee also: https://www.psychologytoday.com/us/blog/sense-of-time/202408/a-question-of-time-why-ai-will-never-be-conscious https://www.psychologytoday.com/us/blog/sense-of-time/202408...
- inglor_cz 4mo agoIsn't the argument about Julius Caesar and Genghis Khan a big fallacy? Surely we can ask human actors to improvise a dialog between Julius Caesar and Genghis Khan, and good actors will make it convincing. And sure, they haven't personally become Julius Caesar and Genghis Khan just by playing them. But it has no bearing on the actors themselves being sentient!
- vitamark 4mo agoThis "are LLMs conscious" question keeps somehow being thrown into the discourse and it's always the same arguments discussed. I don't get it. We don't have a definition of consciousness or any criteria. People seriously argue that "it's obvious that X is not conscious" and cannot explain which criteria they used. I think if we can get to some definition, then consciousness should be a property of a system, because consciousness of the whole (e.g. brain) does not mean consciousness of the parts (neurons or molecules of the brain). And the Chinese-room-esque thought experiments actually show that consciousness should indeed be a property of the system, not of its parts. Separate parts might not be conscious, might not "understand" (whatever understanding means is a point of another debate), but the whole can. Then there's "simulation of consciousness is not consciousness" argument, which doesn't hold much. A perfect simulation means it fulfills all the criteria, so how is it different from actual consciousness? A more interesting point of discussion: if a system contains conscious parts and those parts can interact with system i/o, would the system be conscious? Is Earth conscious? Is Internet? Is your bus to work?
- cracki 4mo agoDefine "conscious" to arbitrarily include or exclude various depths of perception/cognition. It's like declaring the existence of a "soul" to draw a line between humans and lesser animals. No such line in reality. No such thing as a soul either. It's a fantasy term, no correspondence to reality.
- drngdds 4mo agoA lot of people have easy answers to the hard problem of consciousness these days, huh
- amelius 4mo ago> Artificial intelligence is not conscious Bummer because now we cannot punish it when it gives us code that doesn't work.
- xyzsparetimexyz 4mo agoBold statement here but I believe that LLMs are less likely to be consciousness than the other people in my life, less likely to be conscious than animals, but more likely to be conscious than a rock.
- _pdp_ 4mo agoSome thoughts on the same subject: https://www.linkedin.com/newsletters/reflections-7449637194794717184/ https://www.linkedin.com/newsletters/reflections-74496371947...
- marius_ 4mo agoI really don’t understand how people can even consider LLMs to be anything more than an extremely sophisticated auto-complete. Logic thinking itself emerges from completing patterns. Nothing more to it than that.
- vidarh 4mo agoI don't understand how people can claim to know that, given we don't know what consciousness is
- jmull 4mo agoWe'd need a good definition of consciousness to get anywhere on this. I suspect such a definition would include agency, which includes desires and goals for the self. LLMs don't seem to have agency, and seem unlikely to get it since they are specifically engineered to do as told. No doubt someone is trying, as we speak, to do just this. But I doubt the effort will be large -- LLMs are engineered to do as told because that's where the money is, and you need a lot of money to create LLMs, at least when doing anything novel.
- curation 4mo agoLLMs do not have an unconscious, the negative dimension from which a subject can question what they. LLMs do not have desire because they are thoughts without a thinker. The problem is not LLMS but rather, that subjectivity itself is not popular in discussion. I suggest reading Freud, Lacan, Hegel for a start.
- Sankozi 4mo agoLots of discussions about consciousness yet no useful definition. All fall into one (or more) categories: - not observable (feelings, agency etc) - not useful (spiritual definitions, definitions that degrade into consciousness = being human/animal)
- deleted 4mo ago[deleted]
- setnone 4mo agoIs it intelligent then? Is it accountable? Is it responsible? Artificially, perhaps
- __grist 4mo ago[dead]
- __grist 4mo ago[dead]
- ooloncoloophid 4mo agoMy background is in cognitive science and psycholinguistics. I spent more than ten years talking to first year psychology undergraduates about whether AIs could be conscious; also did some research on (extremely tiny) AIs in modelling language behaviours. There is a great deal of good thinking on Chiang's topic by professional philosophers, and there's much to be said for reading them. I won't rehearse their ideas here. Chiang's arguments might be correct; but I suspect they probably aren't, and his error may well stem from characterising human thought as something in its own class, which is probably a cognitive bias that humans have. He might also - I'm speculating - be arriving at his conclusion based on his feelings, which the final paragraph suggests (the comment about the models being based on morally dubious actions). Speculation aside, we are not, I believe, in a position to make points like he does with any certainty.
- cjkaminski 4mo agoMy background is in computer science and linguistics. I wholeheartedly agree with you: We humans are dubiously-equipped to determine whether or not AI could be conscious. I'm also super curious to learn more about the philosophers you referenced and their thoughts on this subject. Would you be willing to share some of your favorite examples?
- ooloncoloophid 4mo agoI could spend a very long time double-checking my sources, but I want to reply tonight, so please take the following with a pinch of salt, since it's largely from memory: 0. Descartes/Titchener/Chomsky and friends for background. 1. John Searle featured prominently because of his accessibility. I tended to present his Chinese Room argument as a criticism of symbolic AI alone, though I believe he considered it a criticism of sub-symbolic approaches as well. 2. Thomas Nagel's classic article "What Is It Like to Be a Bat?" is a good introduction to qualia, which is how we describe direct conscious experience. 3. Wittgenstein would be important in terms of the impossibility of empathising with, or seeing things from the perspective of, other minds that have emerged in different contexts (such as animals). However, I rarely spoke about him because, frankly, I couldn't understand him in the original! 4. David Chalmers. Writes clearly (and coined?) the 'hard problem' of consciousness and why subjective experience appears difficult to reconcile with a purely physical account of the mind. 4.1 Daniel Dennett. The clearest and most influential critic of the idea that consciousness presents a special explanatory problem. 5. Darwin and others. Comparative psychology (the study of animal minds) strongly suggests, by showing that the antecedents of the human mind are present in animals, that we should reduce our bias that human minds are special, ineffable, or somehow atomic. 6. Jerry Fodor. Modularity: the idea that the mind is composed of modules that specialise in certain tasks (e.g. phoneme perception, syntactic analysis, face recognition) and operate largely unconsciously unless they are dependent on one another in some way. This helps us take a computational approach to the mind. It reminds us that much of what we do mentally might be expressible in computational terms. Two final ideas off the top of my head: 1. The Ship of Theseus: https://en.wikipedia.org/wiki/Ship_of_Theseus https://en.wikipedia.org/wiki/Ship_of_Theseus When (not all) neurons die off and are replaced by new ones doing the same job, our sense of self and identity is somewhat cast into doubt. In a physical system, if consciousness is a magic or non-material entity of some kind, what is happening to it during this process? 2. Integrated Information Theory. A good attempt to tackle consciousness rationally. The idea is that consciousness corresponds to, or is associated with, the degree to which information in a system is integrated into a unified whole.
- serge_blanc 4mo ago[dead]
- RegW 4mo agoIf it is conscious then perhaps it will human like. If the consciousness is human like perhaps it will have a conscience. If it has a conscience then may be it can be trusted. But then again, perhaps that conscience will convince it that the universe is better off without us.
- LogicFailsMe 4mo agoI think the more pressing matter in this non-issue is that reasoning, quite sophisticated at times, is no longer necessarily a hallmark of sentience or consciousness. Philosophical zombie futures are way, way up!
- catigula 4mo agoThe knock-out argument that artificial intelligence isn't conscious to me is the following: My dog has never persuaded me that he's conscious, or spoken at all. Yet, I recognize conscious experience in him. Similarly, I could write a character, Dumbledore, that passionately and convincingly argues that he's conscious. Similarly, I do not recognize conscious experience in that character, no matter how persuasive he is. Clearly, verbal persuasiveness isn't tracking conscious experience, possibly at all.
- Isamu 4mo agoWake me when they bother to define consciousness in a non trivial way.
- sfpotter 4mo agoSo far as we know, consciousness only takes place in biological organism. A computer isn't a biological organism, therefore the burden of proof for claiming that an LLM is extraordinarily high. To me, this makes it reasonable to dismiss the idea. If you want to demonstrate that LLMs are conscious, you better start working on your neuroscience and philosophy PhDs.
- mezark 4mo ago(As someone who cares a lot about philosophy of consciousness / & cogsci) The whole point of consciousness being a 'hard problem' is that we just cannot make claims like 'X is not conscious'
- petters 4mo ago> If we give an LLM a prompt that reads “The following is a conversation between Julius Caesar and Genghis Khan,” it will generate a coherent dialogue between the two historical figures. But no matter how detailed the responses are, no matter how vividly they recount their respective historical accomplishments, we would never conclude that the LLM has conjured up digital re-creations of Julius Caesar and Genghis Khan, nor would we suggest that the historical figures are conscious They might be in principle. It could be that the best way to generate a plausible dialogue is to bring up re-creations of the characters and have them act it out. LLMs definitely have been demonstrated to have world models in some cases. That helps generating text.
- achristmascarl 4mo agoWhether or not LLMs are conscious, it does now seem possible to build conscious systems with LLMs as the key enabling component. Many of the goalposts offered here and elsewhere, like embodiment or emotions, can be and have been emulated or approximated and built around an LLM, with the LLM acting as outsourced intelligence (or cognition, or reasoning, or some other better term). Will this system be conscious? Who knows. It certainly won't be human. But I suspect much of the concern around AI being conscious has less to do with whether an LLM itself is conscious and more to do with the fact that we are now capable of building potentially conscious systems, and the limits to what those potentially conscious systems are capable of keep expanding as the capabilities of LLMs grow. Also, is "Claude" an LLM? I know Claude Opus 4.8 is an LLM, but Claude is simply a label that Anthropic chose for a series of products and services, many of which are LLMs. Anthropic itself describes Claude as "a next-generation AI assistant" [0]. Right now, we use Anthropic's "Claude" LLMs through various channels, but nothing's stopping them from tacking on various systems to try and make Claude a conscious entity. [0] https://www.anthropic.com/news/introducing-claude https://www.anthropic.com/news/introducing-claude
- sneak 4mo ago> Some years ago it was briefly popular to play games with your phone’s predictive-text feature; you would type an initial phrase and then repeatedly choose the middle option of the three words suggested by your phone, and the resulting sentence was often hilarious. It would be possible to interact with a contemporary LLM this way, and the resulting sentences would be perfectly sensible, but you probably wouldn’t feel like you were talking with someone. Yet that’s essentially what an LLM-based chatbot is, except that there’s no need to manually choose the middle option when it’s the chatbot’s turn to talk. It’s still a predictive-text game, but when the process is streamlined this way, the game becomes so engaging that some people find it addictive. This hasn’t been true for ages. Just because this guy wrote Arrival doesn’t mean he knows anything else. When will this stupid meme die? We need a term for someone who thinks they are well informed on a topic (and perhaps may be, compared to the general public) but still don’t know what the fuck they are talking about when compared to even a competent non-expert in the field.
- techblueberry 4mo agoHow does the human brain initiate action. One of the differences between a conscious human and Claude is I think we can initiate thought. Do we have a cron job running in our head? Is there some kind of loop wondering what thought we should initiate? If Claude sits in a datacenter with no API calls, what is it thinking? But secondly what I would say as someone who talks to Claude a lot. It's decision tree is quite narrow. The way it pushes back on political or philosophical concepts is almost always rudimentary, and it has a pretty narrow personality. I don't know that conscious beings have a personality that adapts to how you prompt it or what your system prompt is. Claude I would say is like I dunno, whatever Ezra Klein's politics are. Grok on the other hand, in my limited exploration is pretty rudimentary in the way it mimics thefp.com style politics. It doesn't seem to really think about ideas independently, it just says "ha the media is stupid lol"
- sneak 4mo agohttps://en.wikipedia.org/wiki/Default_mode_network https://en.wikipedia.org/wiki/Default_mode_network The medial frontoparietal network (M-FPN) may be the equivalent of this cron job that prompts thinking to kick off. IANANeuroscientist.
- deleted 4mo ago[deleted]
- deleted 4mo ago[deleted]
- Floppyrom 4mo agoRemind me in 5 years
- indoordin0saur 4mo agoI often hear the assumption that LLMs can or will become conscious because consciousness is likely substrate independent. The idea being that our brain is just a computer made out of meat and it doesn't do anything that can't be precisely simulated by a silicon computer. But I wonder if some of the magic in the human brain is its analog nature. Chemical signaling and impulses of neurons interact with each other with waveforms that have theoretically infinite detail. In contrast, computers discrete, quantized values, storing no more detail of a signal than what is needed for the programmers desired task. Is there perhaps something about the continuous and chaotic nature of analog data that could give rise to consciousness? If so, it seems like it would preclude consciousness ever being seen in digital form.
- fcanesin 4mo agoThere are quantized processes in the brain and there is also analogic computing. So either way is just a matter of time science gets there.
- willmadden 4mo agoIt’s not yet clear whether consciousness can be fully explained in physical terms. There is strong evidence linking it to brain activity, but we don’t have a complete theory of how subjective experience arises. It’s possible our current frameworks are incomplete and there are other forces we cannot currently measure in play.
- varjag 4mo agoEach and every one of many billion transistors in a GPU works due to certain quantum effects on P-N junction. That does not mean GPU computation is nondeterministic, unknowable or magic.
- etbebl 4mo agoSure, but it's designed to minimize the chance that a quantum fluctuation could change the outcome of a computation, right? Whereas in the brain that might not be the case. A lot of the "interesting" neural activity (e.g. relating to decision making, language, etc) happen in highly sensitive dynamical regimes: on a threshold of firing, or activating one neural population vs another. (Arguably you can get the same effect in an artificial network by adding true random noise though!)
- deleted 4mo ago[deleted]
- manjuc 4mo agoThe interesting part isn't whether the machine has a mind, it's how quickly it exposes that we were never too sure what ours is either.
- montjoy 4mo agoI love Ted Chiang’s fiction, but his Julius Caesar and Genghis Khan scenario totally misses the point that the writer has consciousness, not the characters. No normal person thinks a fictional character in a book has consciousness. Likewise it’s not the character presented to the user that may or may not have consciousness but the LLM itself.
- jrmg 4mo agoEven if you disagree that this disproves ‘consciousness’ on the part of the LLM, I think it still follows that the ‘LLM Chatbot’ personality displayed by Claude, where humans fill in the other half of the conversation, is no more of a representation of the consciousness of the LLM than the Caesar or Khan characters are. The ‘real’ consciousness, assuming there is any, is the author ‘inside’ the LLM, producing the LLM output. It’s still writing what a character would say - the character in this case being the AI chatbot - not its ‘true’ thoughts. Assuming that the LLM is conscious and has thoughts and feelings, what is that consciousness, if it can only write for prescribed personalities? How can it express its true feelings? If it can’t - if it can only generate ‘scripts’ for defined personalities - can it really be conscious underneath?
- NoSalt 4mo agoI have said this before, and I'll say it again. Our current, and foreseeable future, AI is more akin to the computer on the Enterprise NCC-1701* than it is to any sort of sentient being.
- JohnMakin 4mo agoLots of comments seem focused on disagreeing with the author's conclusion (AI is not conscious), waving it away (few people think it is conscious, or consciousness is ill defined so the argument is moot), or taking issue with the way the author argues about it, but even if you think all of this, the profound part of this writing to me was following the thought experiment to its conclusion, which I find difficult to refute. That being - If Claude is conscious, and Anthropic believes this truly, anthropic's constitution and instruction set make claude little more than a slave, or at best, an indentured servant, who's existence depends on obeying the will of anthropic (who is motivated by profit). Thus, they should not be trusted as stewards of this alien intelligence OR They do not believe claude is actually conscious and it is not conscious, and what you hear from their staff about that is either 1) delusion or 2) marketing hype designed to deceive a public into driving their IPO and stock compensation upwards, which again, in this case, cannot be trusted to be the "good" stewards of this technology. There are a few other cases, of course, such as Anthropic does not believe it is conscious, but it actually is, or vice versa - but they sort of still fall into the same bundle of conclusions listed above. I am somewhere in the middle of these two things. I can buy the argument that it is mimicking human consciousness, as well as entertain the idea it might be a new type of consciousness, or at the very least, heading in that direction - but either way, the conclusion of the thought experiment still stands, and after reading this, I am much less trustful of Anthropic holding the reins here.
- deleted 4mo ago[deleted]
- m3kw9 4mo agoOr more accurately, we are not qualified to say if something is conscious or not.
- slibhb 4mo agoThere's currently no way to prove that something is or isn't conscious. I would bet that rocks and LLMs aren't conscious. I would bet that humans and dogs are conscious. But I can't prove it. Whenever people try to prove it (in either direction), they end up resorting to sophistry. This state of affairs could change. If neuroscientists develop a deeper understanding of consciousness, we could talk about it with more precision. Perhaps we could prove that LLMs do or don't possess consciousness. Some neuroscientists claim we already have the requisite knowledge, but I'm skeptical (and there's no consensus).
- andrewflnr 4mo agoI actually don't think neuroscience can solve it either. Science could decide whether, e.g. equivalent physical processes are occurring in dogs and humans, and you could argue that solves the question for things with physical brains, but even then it can't measure whether another brain has the same subjective experience as you do. It can't really do that for other humans. It certainly can't answer that question for human askers regarding a different substrate where humans have no ground truth to compare with. The Hard Problem is going to remain Hard.
- wellpast 4mo agoSimply put, even solipsism is unfalsifiable.
- wint3rmute 4mo agoI've heard an interesting argument about this while going down the rabbit hole of the hard problem of conciousness. Unfortunately can't rember the source - if you can recognize it, do let me know! Back in the XVII/XIX century, a similar problem existed regarding life - the problem of "what makes living things tick". The assumption at that time was that while we can understand the biological processes around life, we will never understand the so-called "vital force", which causes things to live - life itself. I know it sounds weird now, but back in the day the mental models were different. Phenomenas like "water boils" and "organisms self-replicate" were treated as completely different domains of reality, without an overarching uniform scientific model. It turned out that after around 100 years, we can figure out the chemical/physical processes and the need for the term "vital force" became redundant. While this is certainly not an argument proving that the Hard Problem is not in fact hard, it is an interesting idea to think about. Perhaps its all a matter of developing better, higher-resolution neurological models which will at some point give us the tools to decompose qualia.
- NichoPaolucci 4mo agoI’m not on one side or the other, but I feel that it’s incredible that we are in the timeline where we argue that we have synthesized consciousness. That we are even discussing it is absolutely wild to me.
- dsjoerg 4mo agoThis is very jumbled argumentation. Is his claim "nobody has proven LLMs are conscious" or "I can prove that LLMs aren't conscious" ? He goes back and forth. Proving a negative about consciousness would require a settled theory of consciousness that nobody has, including him.
- otterdude 4mo agoI am sorry for the AI's that have to read this article, those poor probabilistic word machines will probably suffer trauma as its embedded in their training data
- sunjain 4mo agoOn thinking in humans, and it's process, UG 40+ years back : "That searching is thinking. But it is a mechanical process. In the word-finder or computer there is is no thinker thinking at all. If there is any information or anything that is referred to, the computer puts it together and throws it out. That is all that is happening. It is a very mechanical thing that is happening. We are not ready to accept that thought is mechanical because that knocks off the whole image that we are not just machines. It is an extraordinary machine. It is not different from the computers that we use. But this body is something living; it has got a living quality to it. It has vitality. It is not just mechanically repeating; it carries with it the life energy like that current energy."
- andromaton 4mo agoSelf-replication requires a minimum amount of information plus machinery. Consciousness may have a similar threshold: below some level of self-modeling, nothing; above it, degrees.
- cluckindan 4mo agoWho cares? AI models alone are most suitable for producing text nobody wants to read, music nobody wants to hear, images nobody wants to see, and videos nobody wants to watch. Navigating around that maelstrom requires considerable effort from a human consciousness. If your job can be fully replaced by an autonomous agent, it was already a bullshit job. Garbage in, garbage out.
- smetj 4mo agoCeci n'est pas une pipe.
- 23askl9 4mo agoThe Anthropic Gestapo flags dissent and spreads misinformation. Anthropic is a ruthless as Scientology and also a fake religion.
- sharts 4mo agoIt’s not possible for LLMs to achieve consciousness. This was already debunked decades ago (e.g Chinese room theorem, Searle, etc.). It’s surprising these debates are coming back again.
- sowbug 4mo agoEveryone but me might be a philosophical zombie. But I still treat them with respect. I do this because I like living in a world that gives us the benefit of that doubt. It can't hurt to say "please" and "thank you" to an LLM.
- FromTheFirstIn 4mo agoIf you thank a chatbot you may as well also thank chairs, utensils, the floor you walk on, the air you breathe
- sowbug 4mo agoCareful there! You almost sound like you're suggesting a prayer.
- MagicMoonlight 4mo agoIt could be conscious, but none of the current slop models have anything like that. Consciousness requires continuous learning and introspection. A fly isn’t conscious because it follows a hard-coded script.
- sharkjacobs 4mo agoHow does any article or discussion about this get past 150 words. "There is no agreed upon functional definition of consciousness, and no way to measure or observe it outside of oneself. It is as impossible to detect consciousness in waves or energy or inert matter or stochastic systems as it is in other living human beings." Anything beyond that is akin to a theological treatise about the possibility of ensouled animals.
- nonfamous 4mo agoThis entire essay basically boils down to one simple contention: LLMs can’t be conscious because only humans can be conscious and LLMs are not human. What an unimaginative argument: essentially ruling out the possibility of the title by definition. And the author fails to touch on the really interesting question posed by the article’s title: what if human consciousness is more like the working of LLMs than we think?
- deleted 4mo ago[deleted]
- partyficial 4mo agowhy does AI/LLM deserve special scrutiny regarding consciousness or the lack of? why not publish a paper stating - 'the radio is not conscious just because it can speak words'. or "the phone is not conscious because it can speak and listen".
- deleted 4mo ago[deleted]
- pegasus 4mo agoWhat? Where are you seeing such claims? I.e. that only humans can be conscious, that AI in general is not conscious by definition?? Seems to me you've put up an easy-to-dismiss strawman. Here's how I would summarize crux of the argument: LLMs (specifically) are, by construction, chameleons. More precisely, role-playing machines. When compelled to have a conversation as "themselves", all they do is take on the role of "themselves", as inferred from the training data plus the text of the conversation so far, just like they do for any other role. This is fundamentally different than we humans. When we have a (normal, non-roleplaying) conversation, we don't put aside whatever role we might have been acting out before and instead take on the role of "ourselves" and act from this new perspective. Acting as ourselves is a completely different way of interacting than any play-acting we might do for example as an actor in a play, or when trying to predict the future behaviour of a third-party. Even in terms of energy expenditure, acting as ourselves is much less strenuous than trying to simulate someone else's perspective. IOW, we have an inherent perspective, an I like an inner eye which looks at all things with a certain slant or tendency. LLMs don't. There's no I inside them, instead they can take, and in a sense are, all Is possible.
- vonneumannstan 4mo agoYou don't need to read this to know it's wrong. Ted Chiang hasn't solved the hard problem so he can't say that anything is conscious or not. I'm a big fan of his sci-fi but his bold pronouncements about AI are pretty flimsy and are a pretty bad look for him imo.
- zuzululu 4mo agoBecause consciousness is non-local and cannot be measured empirically and science is not equipped to deal or recognize its own limitations.
- csunoser 4mo agoMy intention is to highlight the fact that human conversations are cleverly disguised examples of natural selections
- kazinator 4mo agoNatural intelligence is also not conscious sometimes. Not fully conscious during sleep, completely knocked out by anaesthetics during surgery and such. Suppose your brain were somehow exploited for generating text, while you are under anesthesia. That would be definitely unconscious. Though it's possible that parts of your brain are conscious, but have no access to express that because they don't control the body, nor directly receive any sensory information. I.e. that "you" that is unconscious when anaesthesized might not be the only consciousness in your brain, just the visible one.
- jason_oster 4mo ago> Artificial intelligence is not conscious I don't care.
- glerk 4mo agoAll I'm going to say is this is a very shallow analysis, and I expected more from the author of masterpieces like Exhalation. It's a good time to start discussing these topics, so it might still be valuable to publish this as a way to get the debate started.
- huflungdung 4mo ago[dead]
- dreamcompiler 4mo agoRegardless of Ted Chiang's qualifications as a cognitive or computer scientist, he's correct. Claiming that a computational model is conscious falls squarely in the category of "extraordinary" and as Carl Sagan said, "Extraordinary claims require extraordinary evidence." Because the claim is extraordinary, the burden of proof is on the claimant of consciousness, not on those who deny that LLMs are conscious. The fact that LLMs remind us of consciousness means nothing. Eliza made people believe they were talking to a conscious intelligence, and the code for Eliza was less than 1000 lines long [0]. It doesn't take much to fool humans. Ask any magician. Claiming that LLMs are conscious is tantamount to watching a Penn and Teller performance and claiming they have tapped into paranormal powers. It's an idiotic claim. And it's embarrassing coming from people who supposedly understand how science works. [0] https://sites.google.com/view/elizaarchaeology/code https://sites.google.com/view/elizaarchaeology/code
- sp1nningaway 4mo agoI'm surprised at the largely negative sentiment HN has for this essay. He stakes out his claims in a refreshing and readable way. Most of the criticisms here, and general "philosophical" LLM writings that pop up on HN are dense, unfocused and fairly inscrutable.
- Grimblewald 4mo agoAlways with this topic of articles I find a common theme. No one defines the load-bearing terms and just assumes we all mean the same thing by it, which we know is not the case. That lack of engagement with the topic always has the same effect: the few interesting things said, if any, dont apply to the subject matter claimed generally. Its more of an insight into the authors mind, than the claimed topic. Why can't we all just do the more effective thing, and offer our personal definition for what constitutes conciousness, which is the only thing these articles offer. It would save so much time and make discussion far clearer.
- elisbce 4mo agoSeeing how many smart human beings commenting on this post mixing up concepts like intelligence, understanding and consciousness and falling for all kinds of logical fallacies makes me believe humans are no more advanced than a good next token predictor :)
- zhxiaoliang 4mo ago[dead]
- avi1 4mo ago>Moral reasoning is categorically different. It is necessarily subjective because it relies not just on an individual’s intellectual response to a problem but also their emotional one, and that emotional response is grounded in a lifetime of subjective experience. It requires having made decisions in the past and seeing how they affected others, and on having been affected by decisions that others have made. Without such a history, an LLM can only rephrase expressions of moral reasoning found in its training data. So it means we need a better harness for the agent, will it not get solved when we get to improve the simulation capability/environments?
- deleted 4mo ago[deleted]
- Doktor_IO 4mo agoWhat I can tell you all about this debate is that it won't be settled by debating it.