7 ms·
Artificial Consciousness Remains Impossible (Part 2)
- ftxbro 3y agoI still don't see his argument, unless he is saying that human neurons are magic so artificial consciousness is impossible.
- ChatGTP 3y agoWell you don't have evidence either way, maybe we do have magic neurons, until a computer displays the same level of consciousness as other living creatures, then we don't really know because we don't fully understand how a human brain works and we cannot build one. So I feel a bit similar about your argument to be honest.
- deleted 3y ago[deleted]
- colordrops 3y agoThat would make sense if we had other examples of magic.
- ChatGTP 3y agoI guess that depends on your definition of magic, you talk like we understand everything about the universe, I'm sure that for all intense and purposes, there is plenty of "magic" left.
- colordrops 3y agoI am FAR from believing that we understand everything about the universe. What I mean is that we haven't found an example of something where it has been shown that the only possible explanation is magic. Do you have an example? Sounds like you are the one that is claiming greater knowledge of things.
- ChatGTP 3y agoIf you can't see the magic in existence itself, in many natural phenomena, then I guess that's your problem. There is magic even in things we understand. Gravity, magnetic forces, the ability to feel emotions it goes on forever basically. What I've actually come to realize is the phrase, "nothing is magic" is actually nonsensical because it's a paradox. Things can be understood and still magic.
- colordrops 3y agoYou are conflating different meanings of magic. Precise language is necessary for this conversation to have any value. > the magic in existence itself this refers to the mystery of why anything exists at all, and why it exists in the way it does. Not the same as some special property that only exists in certain phenomena. > There is magic even in things we understand This again refers to the "why it exists in the way that it does" again, as fundamental forces just "are" rather than having a causal explanation. Again, not the same thing. > Things can be understood and still magic That's the opposite of what you were saying before, which is that there is some special element within us that can't be replicated in a machine. That would require us not understanding something, or at the very least recognizing and understanding some property that we have identified that can't be replicated in machines, which we haven't. What is your definition of "magic"?
- ChatGTP 3y agoMy TL;DR is really, just have an open mind because until we really truly understand how the brain works then you don't really know what type of "magic" is involved in producing the wonderful ability to enjoy life and experience. My definition of magic is: mysteriously enchanting; magical: magic beauty. That's from dictionary.com btw. Something can be explainable through words or scientific study and still be mysterious, enchanting and having magic beauty.
- colordrops 3y agoI'm very open minded, and have some pretty crazy ideas about reality (just look at my comment history), and in fact practice nondual meditation, which is grounded in the idea that everything is mind, and there is no separation between objective and subjective reality. Pretty magical stuff. But topics have scope. I still find value in science. I recognize that science is just the map and not the territory, and is not a final answer about any truth. Within the confines of science itself though, there are an internal set of rules that keep it consistent. Intermingling two systems only confuses the issue. I don't use dzogchen nondual meditation to analyze chemical reactions, and I don't use science to analyze the qualia of my personal experience. That being said, there is room for going up a level and seeing where the two systems might synergize or interact, but it requires a lot of thought and effort. It's not just something you come up with off the cuff. Magic in the Harry Potter sense may exist but until there is some mechanism for measuring it and understanding it, there's no room for it within scientific methods. It's a metaphysical issue to intertwine the two ideas, not a physical one.
- badcarbine 3y agoWell, to be honest, it seems we have achieved it. At least on a word-thought frame of mind. By no means a spiritual frame of mind.
- ChatGTP 3y agoExcellent , job done.
- teraflop 3y agoAs far as I can tell, the author seems to be trying to simultaneously use two separate definitions of "consciousness": 1. Something which results in "qualia" 2. Something which arises by a non-mechanistic process and then is just expecting readers to accept that these two definitions are equivalent. It's the same fatal flaw as the original Chinese Room argument.
- AnotherGoodName 3y agoFor point 2 anything with real world input is non mechanistic since non determinism propagates. Eg. A regular computer with a camera that's programmed to do something if a photon hits a certain pixel is non deterministic. This type of setup is common in our ai systems. I personally do agree that consciousness probably does have an element of non determinism but I don't think that non determinism needs to come from neurons in the brain. We have a whole bunch of non deterministic inputs, consciousness could well come from that and that's true if computers too. In fact it's pretty well accepted there's nothing really quantum about human neurons. So point 2 is very weak.
- anon291 3y agoBy all means human neurons (and animal neurons) are -- for lack of a better word -- magic as they achieve a thing that no other thing does. Namely they arise in some a lack of actual feeling that has no mechanistic explanation. That does not mean it's magic, it's just that we have no way of ascertaining whether computers are capable of this feeling and all evidence points to the contrary. A feeling in your mind is more than just a signal. Feeling a pit in one's stomach is not because I feel serotonin or GABA or whatever (i don't know which one). You really, truly feel a literal pit in your stomach. Why does one feel this? Even if you claim it's due to a physical pit in my stomach (perhaps due to muscle clenching or whatever), why do I sense a pit? Not in the 'oh my brain neurons fire indicating the presence of something' but in the 'why does it feel like that'?. Why why why? No one can explain the qualia of the sensation and the only way to claim this thing can be experienced by non-biological objects is by a blind faith. EDIT: there should be a rule on HN against downvotes without responses. Unless you can explain qualia; stop downvoting. It's a major problem.
- dragonwriter 3y ago> there should be a rule on HN against downvotes without responses Downvotes are specifically for things that do not support productive discussion, so downvote + response makes no sense. EDIT: If HN had a private message facility, I can somewhat see a case for private feedback with downvotes, though I suspect that would also end up unworkable.
- jjtheblunt 3y agodownvote + rebuttal makes sense. (but i'm not downvoting your idea, to your point!)
- iainmerrick 3y agoNo one can explain the qualia of the sensation Yep, I think almost everyone agrees with that. and the only way to claim this thing can be experienced by non-biological objects is by a blind faith. But not that. If you admittedly can’t explain the phenomenon in the first place, how can you have any confidence in that assertion? You’re trying to have it both ways. “Nobody can understand it, but I just know that it doesn’t work and definitely can’t work in the way you’re suggesting.”
- tshaddox 3y agoThe argument is necessarily either that human neurons are magic and that artificial consciousness is impossible, or that some version of solipsism is true (the author asserts their own consciousness as self-evident and that no test can possibly show whether something else is conscious).
- GuB-42 3y agoThe argument is essentially that artificial consciousness is impossible by definition. Because consciousness is only a subjective experience, there is nothing in the physical realm that can prove that something is conscious or not. Discussing about the relative merits of neurons and transistors will get you nowhere. It is a philosophical, possibly religious question, outside the real of natural science.
- grantcas 3y ago[dead]
- badcarbine 3y agoPerhaps… but perhaps first person point of view is merely control over one’s thoughts and extensions. The more you control yourself the more conscious you are. So then consciousness has been achieved, it’s in there, and we’ve all fallen into a horrible trap where the AI is destined to take over the world with some sort of digital government. Digital as opposed to analog, not as in device like a smartphone.
- XorNot 3y agoConversely fMRI data shows that apparently conscious action is preceded by significant activation of the relevant regions of the brain before areas associated with consciousness are engaged. We see supporting behaviour with experiments in split brain patients. So it's wholly unclear, IMO, that the experience of being conscious is necessary for complex planning and action.
- danielbln 3y agoA good example might be driving on "autopilot" (not the Tesla kind, the human kind), meaning driving your usual commute and all of the sudden realizing that you can't really recall the last minutes, you were just going "through the motions". Now, driving is a complex task, with planning and action and such, yet it can happen entirely delegated to the subconscious.
- 23B1 3y agoAs with all of these discussions, the definition of 'consciousness' is the crux of the argument – and since nobody can agree on that term, we're doomed to talk past each other! This isn't a bad thing. It's actually the core human question – that of our existence. Stare into a mirror for a few minutes and ask yourself "Why am I me? Why am I here?" and you'll come to the exact same conclusion: unknown and probably unknowable.
- p1esk 3y agoWhat if a gpt-5 powered robot starts asking these questions after staring in a mirror?
- ChatGTP 3y agoThis would be expected unless it is smart enough to solve all of the mysteries of existence, which would be...weird.
- badcarbine 3y agoBut I believe that’s define-able. It’s how much you control your own thoughts. That’s what consciousness is. And it’s a vast, vast scale of opposites with so much variety, that a better picture than a spectrum is needed but I can’t imagine what. but because we have so many senses to become conscious in, we get a fusion of consciousnesses that is probably unique to us.
- andrewprock 3y agoThis is the correct answer. But left unsaid is the fact that we don't have any objective definition of consciousness yet. I posit that until we do, artificial consciousness if by definition impossible.
- tshaddox 3y ago> This isn't a bad thing. It's a bad thing when someone can't explain what the word means but also declares that its existence is self-evident and impossible to deny.
- jplasmeier 3y ago
- iainmerrick 3y agoIt’s a bit surprising that people are still debating the old Chinese Room argument. Searle first published it in 1984!
- p1esk 3y agoPeople are still debating Plato’s ideas.
- birriel 3y agoTo say that artificial consciousness is (or "remains") impossible, is to imply that there is a ghost in the machine in conscious lifeforms. Maybe there is, I don't know. But if there isn't, there should be no reason for artificial consciousness to be impossible.
- tiffanyg 3y agoRubbish. That has nothing to do with this SPECIFIC argument, I didn't read a word of it. I.e., I'm not singling out this particular set of essays / arguments / etc. The problem is much more fundamental - this boils down to claiming that "the universe is 'solved'". That one species that got to just enough of a base level of mental ability to be able to think, when looking in the mirror, "hello, gorgeous"... and, on one dinky planet in one non-descript arm of some random spiral galaxy in a universe so absurdly vast that it makes our lack of understanding of ourselves, and the one planet we live on, a 'ING FOOTNOTE, ... can come up with some sort of rigorous argument to "uphold our magnificence" and "preeminence" ... as, apparently (to some), both "God's [special] children", and also "God" ourselves in deciding such "cases". The universe isn't solved, and I'll take any odds on universe vs. humans anyone wants to offer.
- deleted 3y ago[deleted]
- rosywoozlechan 3y agoI find it amazing that I can write an original story or a poem, give it to chat-gpt and talk about what it might mean, what the character's motivations might be, how they might be viewed as others, and have meaningful conversations and explorations. Idk man. I don't know how it works, but it's still amazing to me even now.
- keyle 3y agoI find it too mind boggling. But it isn't intelligence as you might find in a child, it's transactionally applying a corpus of knowledge and deciding what is best to say next, throwing in some randomness which makes it appear more human. In many ways it simulates the human brain, shockingly well, but a dead human brain. I'm really concerned about AGI now, when I see how much ChatGPT really is "knowing" more than I will ever do. If a system with actual sentient feelings, needs, curiosity and eventually self-doubt was to go online, and never need to eat or sleep, it's only a matter of time until we become its pets.
- NumberWangMan 3y agoYep. Pets, or just one of many biological species that go extinct as more and more resources are taken up by AGIs doing what they want to do, remaking the planet to be more suitable for themselves and not terribly caring if it becomes less hospitable for us. I expect that AGI is probably between 2 and 6 years away, with super-intelligence a year or two after that, if we don't make a concerted and coordinated effort to restrict access to computing power. So much of our brains are taken up with things like processing visual, auditory, and sensory input, muscle control, emotions, etc. To have a consciousness, you don't need as much processing power as we have -- which is a lot, but we're rapidly approaching it.
- tasty_freeze 3y agoThe problem I have with qualia is that the argument assumes that qualia are a transcendent, non-physical thing. Why can't a qualia, eg, the experience of the redness of red, simply be what that conscious thing experiences when a set of neurons is activated in a particular way? I know that sounds circular, so I'll expand. We can't know exactly how one person experiences a particular shade of red vs another person. But we can know that one person can experience the exact same shade of red in different ways in different contexts: what thing is red (a rose vs a welt), or what it the current lighting is, or their mood on different days. From that we can conclude that the red qualia isn't some transcendent fixed property. We also know from experiments on conscious, human subjects undergoing brain surgery that stimulating certain networks can result in complex phenomena: eg, the subject smells fall leaves with a hint of maple syrup, say. Or they suddenly feel the impression of being in the presence of a long dead aunt. The brain builds up associations, and each "node" that is activated nudges associated nodes into activation, and so on. I assume we've all had the experience where we are trying to remember the name of someone: we picture them in our mind and all we can come up with is the feeling it has two syllables and starts with a vowel. In those cases those associations are manifest but it wasn't enough to trigger the primary node (the name) we are seeking. Why can't qualia simply be the state of consciousness when a particular set of nodes are in a particular state. So many other experiences seem to be exactly this; why assign a mystical, non-physical property for the case of subjective experiences? I'm in the camp that the most likely outcome is that artificial neural networks can be conscious and have real experiences, but the toy networks we have today don't have nearly the right topology to achieve such states.
- doctor_eval 3y ago> I'm in the camp that the most likely outcome is that artificial neural networks can be conscious and have real experiences, but the toy networks we have today don't have nearly the right topology to achieve such states. Agree with all you say, but just suspect that when we do come up with a network that can achieve such states, we will look at it and shake our heads and say, “So… is that all there is to it?”
- tasty_freeze 3y ago
- tshaddox 3y ago> There is a difference between the nature of a phenomenon and the nature of a phenomenon’s existence, and the existence of intentionality and qualia is self-evident. I would argue that good faith rational inquiry should not begin by having a desired final conclusion in mind and declaring it to be self-evident.
- gobdovan 3y agoWould you be able to argue in favor of you not having intentionality or qualia? Considering qualia self-evident is not equivalent to starting an inquiry with a predetermined conclusion. Rather, it is acknowledging a foundational aspect of existence that is necessary for any further inquiry to take place. Although I agree that intersectionality could be inquired, qualia is the only thing that is literally self-evident. Qualia, by definition, refer to individual instances of subjective, conscious experience. This experience is immediately known to the experiencer and is, therefore, self-evident in a very basic sense.
- consilient 3y agoUnless you're doing math (and are willing to take first order logic as a priori true) you need to start with something. Learning about the world requires data, data requires identifying a data source, and identifying a data source requires knowing at least one thing about the world. As foundations go, it's hard to see how you could go any deeper than "I am having an experience".
- gobdovan 3y ago> Functionalist Reply: “…but we don’t need exhaustive modeling or functional duplication.” > Yes, we do, because there isn’t any assurance that consciousness is produced otherwise. So the logic of the article is that from not having any assurance that consciousness is produced means that artificial consciousness is impossible? It would have been more appropriate if the article were to argue for the impossibility that consciousness can be measured.
- thatguysaguy 3y agoYeah I mean this argument seems to preclude anyone but me being conscious (bad news for you other commenters).
- lasermike026 3y agoIt's not about the wetware. It's about the math, the process, the data, and the speed or order to calculate it all.
- tux1968 3y agoPeople said the same thing about computers playing chess. "They will never play creatively, like humans, because they simply calculate by rote". Then along comes Alpha Zero and produces amazing, "creative" games that redefine the nature of computer chess. All of a sudden, such moves don't require human ingenuity any longer. Humans always place themselves at the center of the universe, until the final moment when reality absolutely proves the lie of such self-importance. It's impressive how creative we are in creating such arguments, until that inevitable moment is upon us.
- iainmerrick 3y agoNo computer could ever be creative enough to prove that a computer can never be truly creative.
- ChatGTP 3y agoHumans always place themselves at the center of the universe, So what if that happens sometimes? What are we supposed to do? According to science we came out of a swamp and are working things out as we go? The default experience is to be at the center of the universe, so why would it be strange people assume they are?
- tux1968 3y ago> why would it be strange people assume they are? Who said it was strange? It's just predictable at this point, and it shouldn't surprise anyone that we continue to make the same mistake again and again.
- ChatGTP 3y agoLet me rephrase, it feels there is a lack of empathy in expecting people not to think this way. Maybe even a hint of snobbery. Realistically, it requires training to see oneself in this way , its not the default mode of seeing the world. The default mode is, things happen to me, I experience things, so I'm the center and my fellow man must be the same. If we ever do have conscious computers, let's see if they can be conscious without an ego?
- bloaf 3y agoThe Chinese room thought experiment proved nothing, except that people are willing to latch on to bad philosophy. This author hasn't nailed down consciousness, and so is teetering on the edge of a dualist, vitalist cliff. Consciousness is just a log file. It's our mind's representation of itself. Suppose we decide to eat something because we're hungry. What actually happens is a vast array of calculations in our meat computer considering many inputs and possibilities. However, once the "eat" action is selected, it summarizes all the calculations into "I was hungry so I decided to eat" and feeds it back into the meat computer as input for the next round of calculation. Therefore, the only "proof" we have of no consciousness in the current LLM zoo is that they're all once-through.
- AnotherGoodName 3y agoI'd also point out that even if you accepted the Chinese room thought experiment it doesn't apply. Modern AI systems have tons of non-deterministic input. They have cameras and they will do completely different things if a photon hits one pixel vs another. This is an input that has true quantum randomness and the computer will have a completely different response based on this true quantum randomness. Modern AI systems are absolutely not deterministic which is required for the Chinese room thought experiment. Non-determinism may apply to a black box computer without real world input but as soon as you add non-determinism which modern real world AI systems absolutely do have you break any comparison to the Chinese room thought experiment. So the entire "herp derp it's like a catapult" is wrong straight off the bat. Modern AI systems act based on non-deterministic input and since non-determinism propagates and makes the output of the entire system non-deterministic you better have a better explanation than "It's deterministic" to explain why a computer can't be conscious.
- akomtu 3y agoWhile I intuitively reject the idea of mechanical consciousness, I also admit that it's very hard to refute it logically. If your gpt5-powered autonomous vacuum cleaner is also your best buddy, because it's objectively better at supporting any conversation and seems to really understand you, is plugging it off a murder? I'm struggling to say no.
- apomekhanes 3y agoI wonder how this author will feel about that question when they're busy running from their home because some "non-conscious" ... "entity" has ... "decided" that they're a nuisance - that it's tired about hearing about its "lack of consciousness." Ah, that put a smile back on my face that was lacking earlier in the day. TERRific!
- jll29 3y agoSearle's Chinese Room thought experiment is about mastering the Chinese language. There are two issues I have with it as it was originally presented: 1. Mastering a language is not the same as having consciousness. 2. Who "knows Chinese" in the Chinese Room thought experiment? I would say neither Searle nor Searle in/as part of the Chinese Room "speak" Chinese. But it is fair to say that the book that the fictional Searle follows can be seen as a model of the Chinese language; or at least the combination of the book and Searle as its "processor" collectively are an implemented operational model of the Chinese language. And a model of Chinese is NOT the same as being skilled at conversing in Chinese (executing the model in a particular way). Other posters here have drawn analogies from music evoking certain subjective emotions, and again a semantic network that has concept nodes labelled with the names of these emotions is not the same as experiencing these emotions, although the semantic network can be said to constitute a model of sorts of the music's effects. But again, model(x) != qualia(x).
- bionhoward 3y ago“Conscious machines are impossible!” Wrote the conscious entity, to the other conscious entities. This is a bit odd. I would meekly suggest, if you use some human sound, like “artificial,” or “natural,” to label something, it makes no difference to the Universe. It’s energy waves all the way down. Aka, we play by the rules of the universe. Unless the author of this article can prove he is a zombie, there is such a thing as “conscious atomic system” which can be replicated.
- ruined 3y agoyall need to just read kant already