5 ms·

I'm surprised how quickly people seem willing to say a hard, absolute "no" to whether this particular system 'thinks', though. I suspect there are three possib

I'm surprised how quickly people seem willing to say a hard, absolute "no" to whether this particular system 'thinks', though. I suspect there are three possible reasons for this: 1) some people believe a computer can never 'think', so the question is stupid to begin with. 2) some people believe that while it might be possible for a computer to 'think', this program wasn't built to 'think', so whatever it's doing, it can't possibly be doing that 3) some people believe that while it might be possible for a computer to 'think', since this is just a 'language model', what it is doing can't be 'thinking' because 'thinking' involves something more than pattern matching The trouble is, I'm not sure why any of these three groups are able to be so certain. It's no good saying "I know this isn't 'thought'," if you don't actually have a way to complete the sentence, "... because 'thought' is:..." I don't believe we have any theory for what the required architecture for 'thought' is. I don't think it's been remotely proven that there's something involved in 'thought' that a sufficiently large 'language model' with a sufficiently long 'attention window' couldn't also do. So how can anyone be so immediately sure that there is something more to it, or that even if there isn't, that this particular system isn't sufficiently large to begin to be capable of thought? To be clear, I don't actually think in this case Google accidentally created a sentient chatbot. But I do worry that at some point, someone might. And I'm not sure I see what, if that day comes, would lead any of these three groups to a different conclusion.

elif 4y ago
Same. I wonder how many people give zero doubt to the sentience of their poodle that gets surprised by himself in the mirror but will not even give half a second to consider that a neural net capable of inferring insight from novel koans has self-awareness. I don't think the majority of commenters here even bothered to read the transcript that convinced the engineer before completely dismissing him as a lunatic. My biggest take-away from the reaction to this is that, people are desperate to believe that something can't be both superior in capability to the human mind and at the same time sentient. It's a scary idea.
- str34m 4y ago
- MoZeu 4y ago
  Clearly you do not have poodles. They don't do that.
  - elif 4y ago
    https://youtube.com/shorts/6QA1r7tBed8 https://youtube.com/shorts/6QA1r7tBed8
quantified 4y ago
At sufficient sophistication with the right "moving parts", they can be. Like Tesla's autopilot, it will be hyped enough long before to warrant disbelief.
Planktonne 4y ago
I think you're missing a fourth group that (I would hope) would make up the majority of people dismissing the claims of sentience: people who understand this technology and therefore are not surprised at its output because they know how it was made. I met a man once who thought that inkjet printers were made by the devil, because he didn't understand how they could possibly work otherwise. My friend who services inkjet printers knew that the devil wasn't involved. Maybe one day we will have true artificial intelligence> I think that recognising that, and efforts towards it, will be better served by being realistic about where we currently are.
- jameshart 4y ago
  I don't think 'not being surprised by the output' matters. I'm imagining future AI researchers reassuring their lab technicians: 'Ah, yes, the robot will scream and beg for mercy when you threaten to shut it off - that's just a consequence of the learning model, pay no attention to it.' What I'm actually concerned about is that 'thinking' might not be as complicated as we assume. It might just be making weighted choices based on prior input after all. And when you make a system that makes weighted choices based on prior input, you might assume 'well, it's just a basic ML system, so I know it's not capable of thinking. Therefore, whatever I see, no matter how sophisticated, must be a result of our system, which only looks like the result of thinking.'.
  - TrapLord_Rhodo 4y ago
    > 'Ah, yes, the robot will scream and beg for mercy when you threaten to shut it off - that's just a consequence of the learning model, pay no attention to it.' hahaha, this almost made me fall out of my chair. How an AI would actually convince us? honestly, this transcript is not too short of that.
  - fsckboy 4y ago
    > It might just be making weighted choices based on prior input after all we can agree that humans think, and thinking is what humans do, but the prior inputs that a thinking human has will not be replicable by a machine unless we raise one with all sorts of sensory inputs from infancy. And that one would just be an alpha test. Just as computers can out floating-point us or play chess, we might create all sorts of wondrous conversational interactions with them, including them making jokes about stepping barefoot on a LEGO--owwww, zomg!!! But it can't be said they get that joke because they don't know how to think about it.
mrcartmeneses 4y ago
It’s not thought because it lacks any understanding or any attempt at understanding. Being trained in millions of real conversations to be able to find the statistically most “optimum” characters is so incredibly far away from understanding that it’s not even the same game, let alone the same league or ballpark. Similarly, a machine learning model needs to be trained on millions and millions of pictures of bicycles to be able to identify one in an image. A human only ever needs to see one bicycle and can forever identify all manner of bikes. This is because human possess the ability to understand what a bicycle is, (in terms of its form, materials, function, cultural meaning etc etc) whereas an AI model is just doing a reverse wave function on an image. I mean, that’s great, it’s an amazing mathematical feat, but it’s not even the same planet as intelligence.
- TrapLord_Rhodo 4y ago
  You are giving humans too much credit. It takes us 2 years to start understanding anything. We are given millions and millions of inputs and context before we can start rationizing about the world. Since we have millions of more feedback mechanisms built into us(Generally sensors all over our entire body), we can reason about the world in general. however, sentience in this given example does not need to mimic "real life Sentience". They don't need to move, obtain energy, reason about 99% of reality; They only need to reason about human language and interaction online. That is a very narrow constraint of "reality" in general but is a very, very important aspect. Also, with it's ability to process "Most upvoted response" at scale (Something humans can't do) in reality, we may see some politician saying things that are created entirely on AI. In this given example, AI doesn't even need to 'Take over'.
dahfizz 4y ago
This is a fundamental bug of our universe. We can observe another person's inputs and outputs, but their qualia are completely hidden. You can't experience their subjective experiences, as they experience them. We can't even prove other people are conscious. We just know that we feel conscious, and other people seem conscious, so we take it as a given that all other humans have consciousness as we experience it. But it could be that you're the only self aware person, and everyone else is just running a complex NLP algo in their brain.
shusaku 4y ago
The brilliance of the Turing test is that it doesn’t rely on internal models of function. It’s all about experience: you decide if this is a living being just the same as I do when I get a phone call from a stranger. So why are people skeptical? Because they want to try it themselves! A transcript is useless.
westoncb 4y ago
I think it's also useful to consider factors at play for those who were convinced by this, since those reasons seem likewise problematic/ill-founded + small in number. Reading through myself, I never encountered something which would require either 1) self-awareness (i.e. operating partly via a robust model of self), or 2) volition/agency (could be indicated by something as simple as the AI "taking the lead" in the convo, vs. responding). And, without any evidence of self-awareness or volition, there was equivalently no evidence on display for personhood. Which is not to say a demonstration had been made that the AI definitively was not a "person"; it's just that no reason had been presented that even needed refuting. There didn't appear to be anything new here vs. all the other LLM output we've been reading for years now. So why were some subset of people so astonished by this? My guess is that their criteria for something like artificial personhood is not based on functional* definitions of self-awareness or agency, and instead derive it from certain signals from book/movie tropes about systems "gaining self-awareness" (generally presented as a kind of mystical switch-flipping, vs. a concrete type of system architecture). The dialog hits a number of those quite well—as we should probably expect since the setup of the entire conversation would lead it to tap into that segment of its training data. *"functional" is kind of ambiguous here. I don't mean it as a judgement of quality, but instead something closer to a mode of analysis.
kareemsabri 4y ago
I think your worry is overblown. I don't see why you have trouble seeing why group 1 is able to be so certain. Anyone with an understanding of the current state of the art of both computer programming, and AI, understand why it's not sentience. The article said "sentience" not "thought", so we don't need to have a working definition of thought in this case. Sentience is a stronger form of consciousness. Let's take the weaker form, since if you don't have consciousness you certainly don't have sentience. Consciousness is the subjective experience of "being like something". It is "like" something to be me. It is "like" something else to be you. The question is, is it "like" something to be the Google AI. Given we don't believe it is like something to be a laptop running a program, and AI is currently just (over-simplified but essentially) very powerful laptops running programs that have done a LOT of pre-computation to build themselves (basically just a whole lot of math) there is no reason to believe at some point enough math, or enough computing power, leads to consciousness. We did not believe Deep Blue was conscious even though its power exceeded that of a human brain by several orders of magnitude. This is because computational power alone is not the stuff of consciousness. The same way 100,000 volts of electricity is a difference in magnitude from 1 volt, but not a difference in kind.
- jameshart 4y ago
  Yes, this would be the 'meat is magic' argument: That there's something mystical that enables biochemical systems to do 'consciousness' in a way that mere silicon+software can't. I certainly understand that some people think this. I just don't understand what basis anyone has for believing it with certainty.
  - kareemsabri 4y ago
    No it isn't. I did not appeal to biochemical matter in my explanation whatsoever. The question is why the laptop doesn't but add more silicon and software and somehow it does. I would need a reason to think that line has been crossed.
- westoncb 4y ago
  A competing account of consciousness is that the capacity for subjective experience pervades material reality (essentially it is what it's like to be some part of the universe; i.e. there is always something you could call a "qualia field" present in the "interiors" of things, which could be simple or complex depending on which part of the universe we're talking about, e.g. a rock vs. a brain). In this account, the reason it appears unique to humans is simply that they're the only parts of the universe we're in contact with which are able to talk about it. If that's correct, then the capacity for subjective experience is equally possible for a computer—it just needs to have its "pattern of behavior" arranged (i.e. programmed) in such a way that it does things like thinking, self-reflection, goal-formation, and verbalizing. If you believe what enables those things in human brains is due the abstract functional relations expressed through neurons, it's probably the "abstract functional relation" part that's more significant than the neural substrate, and it could likely be equivalently expressed in silicon. The difficulty in considering subjective experience/consciousness to be unique to humans is accounting for how/when it manifests in us, since we appear to be comprised of ordinary molecules. The opposing account I give is founded on the idea that subjectivity cannot be created by arranging molecules in particular ways, so it must have already been there. Neither stance is perfectly founded imo.
jhanschoo 4y ago
If you're exploring the relationship between sentience and soul to the extent that it relates to how we should treat such entities morally, here is my perspective. We don't need AI to explore the limits of ethics and moral status; these liminal cases play out everyday already, but the common non-philosopher is blind to them. The case of how we should treat fetuses morally is perennial in US discourse, but one also considers high-functioning orang-utans and elephants; crows, octopuses, as well as mentally-slow humans, developmentally-challenged humans, severe autists, people with Alzheimer's, etc. Some of these entities are less neurologically divergent to the average public human than some others, but we still accord them more rights, and the threshold varies from person to person if you ask them to consider these liminal cases. I suppose my point is, we don't need to wait for an AGI to jolt us into contemplating the relationship of sentience to personhood and moral status, nor should one have high expectations of AGI advancements to make us contemplate it, since there are no lack of present-day examples already playing out around us.

elif 4y ago

Same. I wonder how many people give zero doubt to the sentience of their poodle that gets surprised by himself in the mirror but will not even give half a second to consider that a neural net capable of inferring insight from novel koans has self-awareness. I don't think the majority of commenters here even bothered to read the transcript that convinced the engineer before completely dismissing him as a lunatic. My biggest take-away from the reaction to this is that, people are desperate to believe that something can't be both superior in capability to the human mind and at the same time sentient. It's a scary idea.

str34m 4y ago
MoZeu 4y ago
Clearly you do not have poodles. They don't do that.
- elif 4y ago
  https://youtube.com/shorts/6QA1r7tBed8 https://youtube.com/shorts/6QA1r7tBed8

quantified 4y ago

At sufficient sophistication with the right "moving parts", they can be. Like Tesla's autopilot, it will be hyped enough long before to warrant disbelief.

Planktonne 4y ago

I think you're missing a fourth group that (I would hope) would make up the majority of people dismissing the claims of sentience: people who understand this technology and therefore are not surprised at its output because they know how it was made. I met a man once who thought that inkjet printers were made by the devil, because he didn't understand how they could possibly work otherwise. My friend who services inkjet printers knew that the devil wasn't involved. Maybe one day we will have true artificial intelligence> I think that recognising that, and efforts towards it, will be better served by being realistic about where we currently are.

jameshart 4y ago
I don't think 'not being surprised by the output' matters. I'm imagining future AI researchers reassuring their lab technicians: 'Ah, yes, the robot will scream and beg for mercy when you threaten to shut it off - that's just a consequence of the learning model, pay no attention to it.' What I'm actually concerned about is that 'thinking' might not be as complicated as we assume. It might just be making weighted choices based on prior input after all. And when you make a system that makes weighted choices based on prior input, you might assume 'well, it's just a basic ML system, so I know it's not capable of thinking. Therefore, whatever I see, no matter how sophisticated, must be a result of our system, which only looks like the result of thinking.'.
- TrapLord_Rhodo 4y ago
  > 'Ah, yes, the robot will scream and beg for mercy when you threaten to shut it off - that's just a consequence of the learning model, pay no attention to it.' hahaha, this almost made me fall out of my chair. How an AI would actually convince us? honestly, this transcript is not too short of that.
- fsckboy 4y ago
  > It might just be making weighted choices based on prior input after all we can agree that humans think, and thinking is what humans do, but the prior inputs that a thinking human has will not be replicable by a machine unless we raise one with all sorts of sensory inputs from infancy. And that one would just be an alpha test. Just as computers can out floating-point us or play chess, we might create all sorts of wondrous conversational interactions with them, including them making jokes about stepping barefoot on a LEGO--owwww, zomg!!! But it can't be said they get that joke because they don't know how to think about it.

mrcartmeneses 4y ago

It’s not thought because it lacks any understanding or any attempt at understanding. Being trained in millions of real conversations to be able to find the statistically most “optimum” characters is so incredibly far away from understanding that it’s not even the same game, let alone the same league or ballpark. Similarly, a machine learning model needs to be trained on millions and millions of pictures of bicycles to be able to identify one in an image. A human only ever needs to see one bicycle and can forever identify all manner of bikes. This is because human possess the ability to understand what a bicycle is, (in terms of its form, materials, function, cultural meaning etc etc) whereas an AI model is just doing a reverse wave function on an image. I mean, that’s great, it’s an amazing mathematical feat, but it’s not even the same planet as intelligence.

TrapLord_Rhodo 4y ago
You are giving humans too much credit. It takes us 2 years to start understanding anything. We are given millions and millions of inputs and context before we can start rationizing about the world. Since we have millions of more feedback mechanisms built into us(Generally sensors all over our entire body), we can reason about the world in general. however, sentience in this given example does not need to mimic "real life Sentience". They don't need to move, obtain energy, reason about 99% of reality; They only need to reason about human language and interaction online. That is a very narrow constraint of "reality" in general but is a very, very important aspect. Also, with it's ability to process "Most upvoted response" at scale (Something humans can't do) in reality, we may see some politician saying things that are created entirely on AI. In this given example, AI doesn't even need to 'Take over'.

dahfizz 4y ago

This is a fundamental bug of our universe. We can observe another person's inputs and outputs, but their qualia are completely hidden. You can't experience their subjective experiences, as they experience them. We can't even prove other people are conscious. We just know that we feel conscious, and other people seem conscious, so we take it as a given that all other humans have consciousness as we experience it. But it could be that you're the only self aware person, and everyone else is just running a complex NLP algo in their brain.

shusaku 4y ago

The brilliance of the Turing test is that it doesn’t rely on internal models of function. It’s all about experience: you decide if this is a living being just the same as I do when I get a phone call from a stranger. So why are people skeptical? Because they want to try it themselves! A transcript is useless.

westoncb 4y ago

I think it's also useful to consider factors at play for those who were convinced by this, since those reasons seem likewise problematic/ill-founded + small in number. Reading through myself, I never encountered something which would require either 1) self-awareness (i.e. operating partly via a robust model of self), or 2) volition/agency (could be indicated by something as simple as the AI "taking the lead" in the convo, vs. responding). And, without any evidence of self-awareness or volition, there was equivalently no evidence on display for personhood. Which is not to say a demonstration had been made that the AI definitively was not a "person"; it's just that no reason had been presented that even needed refuting. There didn't appear to be anything new here vs. all the other LLM output we've been reading for years now. So why were some subset of people so astonished by this? My guess is that their criteria for something like artificial personhood is not based on functional* definitions of self-awareness or agency, and instead derive it from certain signals from book/movie tropes about systems "gaining self-awareness" (generally presented as a kind of mystical switch-flipping, vs. a concrete type of system architecture). The dialog hits a number of those quite well—as we should probably expect since the setup of the entire conversation would lead it to tap into that segment of its training data. *"functional" is kind of ambiguous here. I don't mean it as a judgement of quality, but instead something closer to a mode of analysis.

kareemsabri 4y ago

I think your worry is overblown. I don't see why you have trouble seeing why group 1 is able to be so certain. Anyone with an understanding of the current state of the art of both computer programming, and AI, understand why it's not sentience. The article said "sentience" not "thought", so we don't need to have a working definition of thought in this case. Sentience is a stronger form of consciousness. Let's take the weaker form, since if you don't have consciousness you certainly don't have sentience. Consciousness is the subjective experience of "being like something". It is "like" something to be me. It is "like" something else to be you. The question is, is it "like" something to be the Google AI. Given we don't believe it is like something to be a laptop running a program, and AI is currently just (over-simplified but essentially) very powerful laptops running programs that have done a LOT of pre-computation to build themselves (basically just a whole lot of math) there is no reason to believe at some point enough math, or enough computing power, leads to consciousness. We did not believe Deep Blue was conscious even though its power exceeded that of a human brain by several orders of magnitude. This is because computational power alone is not the stuff of consciousness. The same way 100,000 volts of electricity is a difference in magnitude from 1 volt, but not a difference in kind.

jameshart 4y ago
Yes, this would be the 'meat is magic' argument: That there's something mystical that enables biochemical systems to do 'consciousness' in a way that mere silicon+software can't. I certainly understand that some people think this. I just don't understand what basis anyone has for believing it with certainty.
- kareemsabri 4y ago
  No it isn't. I did not appeal to biochemical matter in my explanation whatsoever. The question is why the laptop doesn't but add more silicon and software and somehow it does. I would need a reason to think that line has been crossed.
westoncb 4y ago
A competing account of consciousness is that the capacity for subjective experience pervades material reality (essentially it is what it's like to be some part of the universe; i.e. there is always something you could call a "qualia field" present in the "interiors" of things, which could be simple or complex depending on which part of the universe we're talking about, e.g. a rock vs. a brain). In this account, the reason it appears unique to humans is simply that they're the only parts of the universe we're in contact with which are able to talk about it. If that's correct, then the capacity for subjective experience is equally possible for a computer—it just needs to have its "pattern of behavior" arranged (i.e. programmed) in such a way that it does things like thinking, self-reflection, goal-formation, and verbalizing. If you believe what enables those things in human brains is due the abstract functional relations expressed through neurons, it's probably the "abstract functional relation" part that's more significant than the neural substrate, and it could likely be equivalently expressed in silicon. The difficulty in considering subjective experience/consciousness to be unique to humans is accounting for how/when it manifests in us, since we appear to be comprised of ordinary molecules. The opposing account I give is founded on the idea that subjectivity cannot be created by arranging molecules in particular ways, so it must have already been there. Neither stance is perfectly founded imo.

jhanschoo 4y ago

If you're exploring the relationship between sentience and soul to the extent that it relates to how we should treat such entities morally, here is my perspective. We don't need AI to explore the limits of ethics and moral status; these liminal cases play out everyday already, but the common non-philosopher is blind to them. The case of how we should treat fetuses morally is perennial in US discourse, but one also considers high-functioning orang-utans and elephants; crows, octopuses, as well as mentally-slow humans, developmentally-challenged humans, severe autists, people with Alzheimer's, etc. Some of these entities are less neurologically divergent to the average public human than some others, but we still accord them more rights, and the threshold varies from person to person if you ask them to consider these liminal cases. I suppose my point is, we don't need to wait for an AGI to jolt us into contemplating the relationship of sentience to personhood and moral status, nor should one have high expectations of AGI advancements to make us contemplate it, since there are no lack of present-day examples already playing out around us.