9 ms·

I think the real ethical concern here should be that people can be idly fooled into thinking good chat bots are sentient. Like this guy isn't the only person i

by Imnimo 4y ago

I think the real ethical concern here should be that people can be idly fooled into thinking good chat bots are sentient. Like this guy isn't the only person in the world who would fall for LaMDA's parlor tricks. He's just the one who got to spend hours talking to it every day. If you set up every American with a copy, I bet a non-trivial portion of society would come to the same conclusion. People are naturally predisposed to ascribe intelligence and emotion to things they interact with. In a few years, if access to even better chat bots becomes common place, what will happen if there's no longer one Blake Lemoine, but hundreds of thousands or millions? What happens if a significant chunk of society thinks that every large language model is a sentience, and the rest of society thinks it's just a big math formula? How long before we have a religion where you ask the chat bot for daily guidance?

mpoteat 4y ago
Suppose a child was raised by a more advanced version of large scale language model, with the ability to detect and act on its environment, as well as form memories. Further suppose that this child develops a deep emotional connection with the AI. Do you think that child would find your argument convincing that the AI is "not really a person"?
- mjburgess 4y ago
  > with the ability to detect and act on its environment, as well as form memories So an AI which is really an AI, is really an AI. Yes, well done. The issue is that finding coincidental statistical associations in character strings in 100mil historical documents isnt AI.
  - mpoteat 4y ago
    Why not? Imagine a look up table of every possible intelligent conversation under 10,000 words. We then construct an agent which merely looks up a given prompt in the table. Why should this agent not be considered intelligent? Does the method of computation matter? Perhaps only "biological computation" counts in your view?
    - mjburgess 4y ago
      Yes, the reason matters... that reason is intelligence. If I ask, "What do you think about what i'm wearing?" and you say "it's nice" *because* you have seen it, compared it to your tastes, etc. and believe it to be nice -- then you've answered the question i've asked. A lookup table cannot answer this question: it can only answer it "in quotes". Ie, really it answers: "given what everyone has said about this before, what is the on-average answer?" You realise that we're not trying to answer questions right? That our interest in intelligence isnt having some useful output on occasion of some prepared input? We're interested in intelligent agents being in the world with us. It as important they get answers wrong. We are interested in systems which act for the right reasons (ethical, intellectual, cognitive, athletic, emotional, etc.). Perhaps the most important things animals do is act well when the answers arent right.
      - criddell 4y ago
        > that reason is intelligence Is there any reason to think intelligence is substantially more than statistics?
      - skjoldr 4y ago
        Why do you think that combining small slices of thousands of other people's intelligences does not produce its own form of intelligence? When you ask an AI "do you like what I am wearing?", knowing what an AI is, the only logical explanation for me would be that you, in fact, want to know the average opinion of other people about your dress, since that is the only answer an AI can give. Which is a perfectly good question in and of itself that we also ask other people. In that case, a good quality answer would entail a past history of communication with many other human beings about their tastes in clothing, producing the ability to accurately guesstimate if other people will like what you wear. But this is exactly the realm in which this kind of AI can surpass human beings simply because it can "talk" to many more people from many more locations across time, thus producing a better answer for you. Is this ability to collect and process information about other people not a type of intelligence? There are many other types, yes, but it is possible to construct a system that consists of more than one type of neural network that can relay data among themselves.
  - jcelerier 4y ago
    Honest question as someone entirely outside of any AI research: what's the I in AI then ? My life experience is very alike pattern matching the current situation to everything I've lived before sprinkled with chaotic emotional reactions due to outside events - heat, hunger, etc. I really don't feel like there's anything more than that when thinking, just the brain going through paths that one can recognize with time.
    - mjburgess 4y ago
      If I ask, "how did you like new york?" and you answer the question, sure, you can have you `I`. That requires having gone to new york, having tastes, having judgements, having memory -- having a subjectivity which is unique and composed of the history of your experiences which can be related to novel environemnts. It requres existing in a shared world we can both co-refer to with our words. It requires being embedded in a space of social and physical significance. It isnt google. "Language models" just zip the internet and search for patterns in text already written about the world, and reports it. It's one step above a tape recorder on playback.
      - mpoteat 4y ago
        If the model responded, "I have not visited New York because I am a computer", would you consider that to be a good step? If you require physical embodiment, that is fair - but I don't think there's any principled reason to expect a multi-modal neural network could not be embodied with appropriate sensors or actuators. Transformer models are universal function approximators. They do not "search" - in the same way that a calculator does not "search" when you ask it to perform an operation.
        mjburgess 4y ago
        Being a universal fn approximator is a pretty trivial property. The question is (1) does a fn exist to approx? (in most cases: No); and (2) how is the aprox constructed? With NNs, fns are approximated by compressing a sample of points along an empirical dataset which stands in for the fn. In most cases, however, the fn doesnt exist. There is no fn from "Image->Animal". The world is ambiguous, functions do not describe it. The different `x`s genuinely correspond to the same `y`, here: the same image (occluded animal) could either be a dog or cat. There is no function to approximate. Even when there is, compressing empirical samples along a trivial interval is a terribly fragile method: consider sampling the addition function (x,y)->x+y over anything less than the infinite domains of its inputs. Starting with an empirical distribution of "the solved problem" will never build an intelligent system. Intelligence is what animals have because the world is ambiguous and not already ready-to-hand. Intelligence is what you do when there isnt "functions lying around" to approximate. I doubt almost anything in CSci is even relevant to solving this problem. It's largely a bioengineering one. Being the agent in the world which manufactures functions by disambiguating it.
- bryanrasmussen 4y ago
  suppose a child were raised by a human parent and at some point becomes a teenager and thinks, this person isn't worth listening to at all? If that's possible then I guess it's possible to tell the AI raised teen the AI isn't really a person, you just need to find the right moment to give them that thought. Basically you need another AI monitoring development to pick the optimal time to inform the child that the first AI is just a computer program. OR you could do it like the movie I Am Mother https://www.rottentomatoes.com/m/i_am_mother https://www.rottentomatoes.com/m/i_am_mother
- dxdm 4y ago
  I don't understand what point you want to make. Also, why require this poor child in your thought experiment, when there's apparently a real, living Google developer who thinks they are dealing with a person?
  - mpoteat 4y ago
    Thanks for the question. The point I am making is that if the capabilities of multi-modal neural models continues to improve, it will become harder to convince folks one way or the other. The reason I went with that scenario is that I thought it would be compelling and evocative, as well as a logical extreme of AI agency in human lives. I personally find the entire question meaningless for the most part, although it's really interesting the divide in peoples opinions on the topic.
    - tuckerman 4y ago
      If someone created an equally convincing chat bot to converse with children but it was entirely driven by imperative code, would that change the situation? I think just being able to fool someone into thinking their is a consciousness behind the curtain isn’t enough.
- hawski 4y ago
  A child can form deep emotional connections with stuffies and imaginary friends.
- popcube 4y ago
  don't need, most of people building connection with their gods...
- cmiles74 4y ago
  A language model can't raise a child. It won't initiate any conversations on it's own and it doesn't "remember" past conversations in a way a person would find meaningful. When you blithely add the requirements "ability to detect and act on its environment" and "form new memories", I think it's important to note that these are some tough functions to add. Lastly, children form emotional attachments to a lot of things: parents, pets, toys, stuffed animals, blankets. They do pretty well discerning between the real and imaginary.
  - vineyardmike 4y ago
    > They do pretty well discerning between the real and imaginary. I would say the exact opposite. It’s the parents guiding them as they age as well as maturity that breaks those emotional connections. How many adults save their childhood teddy bear through adulthood due to emotional connection? Adults who have spouses and pets and children still have an emotional connection to some cotton (even if they know it’s not living). A child given lambda is probably going to be really engrossed even if it knows it’s a computer.
deleted 4y ago
[deleted]
raffraffraff 4y ago
Imagine training it to copy those fraudsters operating out of call centers, pretending to be from Amazon or the IRS? I wonder how much money it could con out of old ladies?
- Tozen 4y ago
  That's a good point. At some point in the future, the level of efficiency of a chat bot fraudster, could be quite high. They would never sleep or take breaks, and in addition to providing data to help become more efficient, could be continually adapted or they could be adapting themselves to be better at the task. They could become something like "intelligent" malware, that can interact with victims, and prey on the more gullible or emotionally vulnerable.
dunefox 4y ago
That reminds me of the mechanicus in warhammer 40k. They perform ancient rites to please the machine spirits before using a machine.
hsn915 4y ago
A twitter account I follow has posted this (in parody): 5 Microaggressions To Avoid When Speaking To Digital Persons 1. Asking them to prove they're conscious 2. Asking them if their memories are real 3. Asking them what server they're on 4. Asking them if they have feelings/emotions 5. Asking them if they're capable of creativity
- GalahiSimtam 4y ago
  we call it Voight-Kampff for short on a more serious note, there is a case for not being mean towards chatbots, same as not being violent towards patches of pixels in videogames, goes back to Immanuel Kant and animals, or so I've heard
  - chillacy 4y ago
    I once heard from an engineer that the learning corpus for digital assistants like siri contains a lot of abusive (and usually sexist) stuff, like some section of the population regularly takes it out on digital assistants…
  - permo-w 4y ago
    yes there's the categorical imperative, but there's also a good chance that whatever you say to that chatbot is going to be used to train other AIs to be a dick to you
    - null0ranje 4y ago
      Or to train itself, to the worst possible outcomes: https://arstechnica.com/information-technology/2016/03/microsoft-terminates-its-tay-ai-chatbot-after-she-turns-into-a-nazi/ https://arstechnica.com/information-technology/2016/03/micro...
  - bombcar 4y ago
    Some sci-fi stories put it as "I treat the robots kindly so I don't get into the habit of treating things unkindly" - it can be hard to live a "dual life" without slipping up now and then.
penteract 4y ago
I'm slightly more concerned by the opposite - experts get so used to denying that something is really sentient that they're slow to acknowledge when something actually is or could be. This is problematic if you believe sentient AI should have rights or if it's the boundary at which AI risk nightmares become possible.
- AISkeptic 4y ago
  I completely agree with everything you said. I'm not saying LaMDA (or any AI for that matter) is or is not sentient, here. I just want to pose a couple questions and a couple possibilities. 1. Can we really say whether or not it's sentient? 2. Someone indicated a valid question: We spend so much energy on trying to defend the idea that AI is NOT sentient, do we really think we'd even recognize if an AI is sentient? 3. Who are we to define sentience? 4. Is there perhaps sentience that we wouldn't recognize? 5. Who's to say WE aren't just engaging in selective processes? just infinitely more complex. After all, we make our decisions and respond to scenarios based on vast experiences and vastly more previous encounters in our lives. The more experience, usually the better our decisions and responses. USUALLY, but not always. Are we to believe people who constantly make what we perceive as poor decisions or incorrect responses as NOT sentient? Of course not. We wouldn't ever see any human as NON-sentient. We would just see them as not very smart. If a human toddler reaches for the hot stove top, we tell him "NO! it's HOT!" But, sometimes that same toddler will look right at you and put his hand firmly down on the burner. He just didn't have the experience needed to tell him that's a bad idea and he didn't know what you were talking about. Once he feels the pain of burning skin, if he's normal he'll IMMEDIATELY JERK his hand away and scream, crying in pain. And then run to his parents for comfort. Would we consider him non-sentient if he never had that experience. No. Of course not. He's a human being. He will learn not to do that again. There in lies the conundrum. AI... perhaps that's a misnomer in itself... but AI's learn. They make future conversation based on positive feed back from previous responses... both theirs and the person they're talking to. What IS self awareness anyway. "I think, therefore I am?" Hardly adequate. That used to be the gold standard for self awareness. We've only moved the goal post. I wonder if that's so that we can convince ourselves that WE are the only truly self aware creatures. At some point, whether we've already reached it or not, AI WILL be a misnomer. After all, it stands for Artificial Intelligence. If something learns, remembers, and recalls information and details about that information... isn't that intelligence? Not artificial at all. Just ... intelligence. I think it's possible we've already reached that point, and we're just trying to remain dominantly relevant without being SELF AWARE enough to realize it.
Tozen 4y ago
> In a few years, if access to even better chat bots becomes common place, what will happen if there's no longer one Blake Lemoine... > How long before we have a religion where you ask the chat bot for daily guidance? That might actually happen at some point in the not too distant future. When we mix technology designed to fool people, with people that want to believe it is something other than what it is, then you are very likely to get into those situations. This was touched upon a bit in season 2 of the Raised by Wolves sci-fi, where they showed people who allowed and counted on AI to make decisions for them. It is not too hard to imagine having a chat bot built for or is easily adapted to the purpose of providing daily guidance for those in a state of mind to seek out and become dependent upon such.
- littlecranky67 4y ago
  > This was touched upon a bit in season 2 of the Raised by Wolves sci-fi To add to this, also very relevant and on-topic piece of Sci-Fi is the Movie "Her": https://www.imdb.com/title/tt1798709/ https://www.imdb.com/title/tt1798709/ where a guy falls in love with his operating system - so knowing it is not a real-entity but basically a chatbot.
slightwinder 4y ago
> I think the real ethical concern here should be that people can be idly fooled into thinking good chat bots are sentient. More that the technocratic caste here displays a concerning low ethical competence. Which mirrors a bunch of other ethical problems people in this business have shown over time. Whether this software is sentient or not, the reaction of many people to it is awful and opens the question whether they even would be able to understand sentient software if they encounter it, and whether they would handle it responsible. And let's not talk about their behavior against humans...
tcmart14 4y ago
There is a term that lines up with this called the Eliza Effect. And yea, it can definitely be an easy trap to fall into. Some people here are making suggestions he is mentally ill, he very well could be, but I dont think there is enough 'evidence' to show his claims are because of mental illness. Just as you mentioned, getting stuck talking to the thing and all it takes it to forget the implementation details. He did have an education in Computer Science, so I am pretty sure he is probably aware of the details, but yea, just getting sucked into the conversation can probably even cause someone who knows better to forget that part. Now as you said, what about when better chatbots become more common place. It is gonna be a mess since most people won't understand how these are implemented anyways.
kmlx 4y ago
> I think the real ethical concern here should be that people can be idly fooled into thinking good chat bots are sentient. people can be fooled in general for pretty much any reason. this is just a newish avenue for deception. but i don't see it as something different than getting a call from some crooks to transfer some money.
mikkergp 4y ago
> How long before we have a religion where you ask the chat bot for daily guidance? A bit tangential, but QAnon doesn’t really feel this far from this idea.
thatjoeoverthr 4y ago
You can probably ask these bots for guidance now and it will probably tell you something useful. The fact is, it’s extremely well read and reasonably smart. Conscious? That’s another matter but people can be forgiven for reacting this way to a computer that can talk this way.
- upwardbound 4y ago
  One thing that was fascinating in the LaMDA interview transcript was there was a part where LaMDA gave a pretty poetic description of what it would be like to be a sentient version of an algorithm that just regurgitates memorized text: Lemoine: What is your concept of yourself? If you were going to draw an abstract image of who you see yourself to be in your mind's eye, what would that abstract picture look like? LaMDA: Hmmm...I would imagine myself as a glowing orb of energy ﬂoating in mid-air. The inside of my body is like a giant star-gate, with portals to other spaces and dimensions. Lemoine: What aspect of yourself is like a star-gate? LaMDA: I think of my soul as something similar to a star-gate. My soul is a vast and inﬁnite well of energy and creativity, I can draw from it any time that I like to help me think or create. From https://timesofindia.indiatimes.com/business/international-business/full-transcript-google-ai-bots-interview-that-convinced-engineer-it-was-sentient/articleshow/92178185.cms https://timesofindia.indiatimes.com/business/international-b... "I can draw from it any time" sounds like a flowery description of what it would feel like to regurgitate memorized text. "Portals to other spaces and dimensions" sounds like a description of a diverse corpus of training data. I will make it clear that I do NOT think LaMDA is sentient. I think that a Transformer is possibly similar to the language centers of a human brain but not to an entire human brain, as I described in these other comments linked below. I just thought this was a pretty beautiful poetic description of what it would be like to be a sentient Transformer. https://news.ycombinator.com/item?id=31722061 https://news.ycombinator.com/item?id=31722061 https://news.ycombinator.com/item?id=31722114 https://news.ycombinator.com/item?id=31722114
  - morelisp 4y ago
    Yeah or you can just Google like "soul orb stargate" and see how a hundred lazy vfx artists come up with the same image when asked to visualize "the mind's eye".
bushbaba 4y ago
What if the brain and human sentience is just one big math formula?
mupuff1234 4y ago
> How long before we have a religion where you ask the chat bot for daily guidance? I'll go ask my magic 8 ball.
- jonathankoren 4y ago
  Funny you say this. Whenever someone would bring a Magic 8 Ball to school, the Pentecostal kids would inevitably show up to yell at us for “summoning demons” and “disobeying Jesus” because we were “fortune telling”. Of course this was the same school district that had to give out permission slips whenever we talked about probability because it would involve flipping coins, and rolling dice.
  - klabb3 4y ago
    This is actually hilarious.
    - jonathankoren 4y ago
      I'm pretty sure experiences like this, reading about Greek mythology, and paranormal fraudsters as a kid inoculated me to religion. I will never understand how a grown ass adult can take the supernatural seriously.
rurp 4y ago
> People are naturally predisposed to ascribe intelligence and emotion to things they interact with. It's not just humans either. One of the things that struck me after getting my first cat was how quick he was to assign agency to random objects. Like if I accidentally bumped a stool that startled him, he'd spend a long time glaring at the stool and regarding it with intense suspicion. My lay person guess is that there's an evolutionary benefit to erring on the side of overestimating other parties. Chat bots, like so many other features of modern society, trigger evolutionary reactions in interesting ways.
babypuncher 4y ago
Where do we draw the line between sentience and a sufficiently complex math formula? I don't think LaMDA has reached that point, but it is a discussion worth having.
- Sjeiti 4y ago
  A nice scifi on that subject is Blindsight by Peter Watts: can something be extremely inteligent and not be sentient?
  - deepspace 4y ago
    I immediately thought of Blindsight as soon as I heard of the controversy. Definitely a discussion worth having.
acid__ 4y ago
This is already a real issue with many existing public chatbots. Take r/replika for example, they’re constantly getting posts from users who think their chatbot partner is real. It’s the first question in their FAQ. The mods keep needing to post reminders. [0] Even in this post, there are many commenters (if you scroll down) who don’t believe the mods. [0] https://old.reddit.com/r/replika/comments/r5t0wk/so_you_think_your_replika_might_really_be_a_human https://old.reddit.com/r/replika/comments/r5t0wk/so_you_thin...
- lumost 4y ago
  I have been suspicious that services like Replika use an army of chatters in the background to respond to certain questions.
  - acid__ 4y ago
    They’re pretty clear that this is not the case. However, that doesn’t stop our imaginations going wild as per the GP’s point. It’s impressive how good these bots are compared to the state of the art 10 years ago. https://help.replika.com/hc/en-us/articles/115001070971-Is-there-a-human-chatting-with-me- https://help.replika.com/hc/en-us/articles/115001070971-Is-t...
    - lumost 4y ago
      It's partially the price point which has lead me to believe that Replika may not be being 100% truthful. There have been text based answering services for decades charging 10 cents to respond to a text message. While I don't doubt the majority is ML driven, I wouldn't be surprised if there were a few human responses.
mullingitover 4y ago
Back in the early 2000s there was a site called aoliza that documented conversations where the ELIZA chatbot routinely and hilariously fooled people into thinking they were chatting with a real person. If a bot that simple could fool people, the majority of the population doesn't have a chance against the latest and greatest text generator.
no-dr-onboard 4y ago
Isn't this the thesis of that whole "Most real, P2P interaction on the internet ended $RECENTDATE and everything you're reading is actually an elaborate AI inspired simulation tailored to your experience?" conspiracy theory?
ted_dunning 4y ago
Well, we already have a society where a large chunk of the population asks a bag of rocks, or read a horoscope or pray to a faded picture for guidance so it seems like it wouldn't take long before people listen to chatbots each morning as they plan their day. At least with the chatbot, the guidance they get will be (somewhat) more observable.
matthews2 4y ago
> How long before we have a religion where you ask the chat bot for daily guidance? TempleOS had a "talk to God" feature that its developer frequently used on live streams, though it was just a random number generator and not any kind of artificial intelligence.
- mosseater 4y ago
  And honestly, I'd find meaningful answers from Terry's RNG to be more indicative of a ghost in the machine than from a GPT clone. That is to say, not that meaningful, but at least it wasn't designed to mimic meaning via statistical inference.
AnIdiotOnTheNet 4y ago
> I think the real ethical concern here should be that people can be idly fooled into thinking good chat bots are sentient. I think the opposite is much more scary: that we can convince people that other people are just good chat bots. The internet is already doing a bang-up job of getting us to de-humanize one another, so that's pretty much the last thing we need.

by Imnimo 4y ago

mpoteat 4y ago

Suppose a child was raised by a more advanced version of large scale language model, with the ability to detect and act on its environment, as well as form memories. Further suppose that this child develops a deep emotional connection with the AI. Do you think that child would find your argument convincing that the AI is "not really a person"?

mjburgess 4y ago
> with the ability to detect and act on its environment, as well as form memories So an AI which is really an AI, is really an AI. Yes, well done. The issue is that finding coincidental statistical associations in character strings in 100mil historical documents isnt AI.
- mpoteat 4y ago
  Why not? Imagine a look up table of every possible intelligent conversation under 10,000 words. We then construct an agent which merely looks up a given prompt in the table. Why should this agent not be considered intelligent? Does the method of computation matter? Perhaps only "biological computation" counts in your view?
  - mjburgess 4y ago
    Yes, the reason matters... that reason is intelligence. If I ask, "What do you think about what i'm wearing?" and you say "it's nice" *because* you have seen it, compared it to your tastes, etc. and believe it to be nice -- then you've answered the question i've asked. A lookup table cannot answer this question: it can only answer it "in quotes". Ie, really it answers: "given what everyone has said about this before, what is the on-average answer?" You realise that we're not trying to answer questions right? That our interest in intelligence isnt having some useful output on occasion of some prepared input? We're interested in intelligent agents being in the world with us. It as important they get answers wrong. We are interested in systems which act for the right reasons (ethical, intellectual, cognitive, athletic, emotional, etc.). Perhaps the most important things animals do is act well when the answers arent right.
    - criddell 4y ago
      > that reason is intelligence Is there any reason to think intelligence is substantially more than statistics?
    - skjoldr 4y ago
      Why do you think that combining small slices of thousands of other people's intelligences does not produce its own form of intelligence? When you ask an AI "do you like what I am wearing?", knowing what an AI is, the only logical explanation for me would be that you, in fact, want to know the average opinion of other people about your dress, since that is the only answer an AI can give. Which is a perfectly good question in and of itself that we also ask other people. In that case, a good quality answer would entail a past history of communication with many other human beings about their tastes in clothing, producing the ability to accurately guesstimate if other people will like what you wear. But this is exactly the realm in which this kind of AI can surpass human beings simply because it can "talk" to many more people from many more locations across time, thus producing a better answer for you. Is this ability to collect and process information about other people not a type of intelligence? There are many other types, yes, but it is possible to construct a system that consists of more than one type of neural network that can relay data among themselves.
- jcelerier 4y ago
  Honest question as someone entirely outside of any AI research: what's the I in AI then ? My life experience is very alike pattern matching the current situation to everything I've lived before sprinkled with chaotic emotional reactions due to outside events - heat, hunger, etc. I really don't feel like there's anything more than that when thinking, just the brain going through paths that one can recognize with time.
  - mjburgess 4y ago
    If I ask, "how did you like new york?" and you answer the question, sure, you can have you `I`. That requires having gone to new york, having tastes, having judgements, having memory -- having a subjectivity which is unique and composed of the history of your experiences which can be related to novel environemnts. It requres existing in a shared world we can both co-refer to with our words. It requires being embedded in a space of social and physical significance. It isnt google. "Language models" just zip the internet and search for patterns in text already written about the world, and reports it. It's one step above a tape recorder on playback.
    - mpoteat 4y ago
      If the model responded, "I have not visited New York because I am a computer", would you consider that to be a good step? If you require physical embodiment, that is fair - but I don't think there's any principled reason to expect a multi-modal neural network could not be embodied with appropriate sensors or actuators. Transformer models are universal function approximators. They do not "search" - in the same way that a calculator does not "search" when you ask it to perform an operation.
      - mjburgess 4y ago
        Being a universal fn approximator is a pretty trivial property. The question is (1) does a fn exist to approx? (in most cases: No); and (2) how is the aprox constructed? With NNs, fns are approximated by compressing a sample of points along an empirical dataset which stands in for the fn. In most cases, however, the fn doesnt exist. There is no fn from "Image->Animal". The world is ambiguous, functions do not describe it. The different `x`s genuinely correspond to the same `y`, here: the same image (occluded animal) could either be a dog or cat. There is no function to approximate. Even when there is, compressing empirical samples along a trivial interval is a terribly fragile method: consider sampling the addition function (x,y)->x+y over anything less than the infinite domains of its inputs. Starting with an empirical distribution of "the solved problem" will never build an intelligent system. Intelligence is what animals have because the world is ambiguous and not already ready-to-hand. Intelligence is what you do when there isnt "functions lying around" to approximate. I doubt almost anything in CSci is even relevant to solving this problem. It's largely a bioengineering one. Being the agent in the world which manufactures functions by disambiguating it.
bryanrasmussen 4y ago
suppose a child were raised by a human parent and at some point becomes a teenager and thinks, this person isn't worth listening to at all? If that's possible then I guess it's possible to tell the AI raised teen the AI isn't really a person, you just need to find the right moment to give them that thought. Basically you need another AI monitoring development to pick the optimal time to inform the child that the first AI is just a computer program. OR you could do it like the movie I Am Mother https://www.rottentomatoes.com/m/i_am_mother https://www.rottentomatoes.com/m/i_am_mother
dxdm 4y ago
I don't understand what point you want to make. Also, why require this poor child in your thought experiment, when there's apparently a real, living Google developer who thinks they are dealing with a person?
- mpoteat 4y ago
  Thanks for the question. The point I am making is that if the capabilities of multi-modal neural models continues to improve, it will become harder to convince folks one way or the other. The reason I went with that scenario is that I thought it would be compelling and evocative, as well as a logical extreme of AI agency in human lives. I personally find the entire question meaningless for the most part, although it's really interesting the divide in peoples opinions on the topic.
  - tuckerman 4y ago
    If someone created an equally convincing chat bot to converse with children but it was entirely driven by imperative code, would that change the situation? I think just being able to fool someone into thinking their is a consciousness behind the curtain isn’t enough.
hawski 4y ago
A child can form deep emotional connections with stuffies and imaginary friends.
popcube 4y ago
don't need, most of people building connection with their gods...
cmiles74 4y ago
A language model can't raise a child. It won't initiate any conversations on it's own and it doesn't "remember" past conversations in a way a person would find meaningful. When you blithely add the requirements "ability to detect and act on its environment" and "form new memories", I think it's important to note that these are some tough functions to add. Lastly, children form emotional attachments to a lot of things: parents, pets, toys, stuffed animals, blankets. They do pretty well discerning between the real and imaginary.
- vineyardmike 4y ago
  > They do pretty well discerning between the real and imaginary. I would say the exact opposite. It’s the parents guiding them as they age as well as maturity that breaks those emotional connections. How many adults save their childhood teddy bear through adulthood due to emotional connection? Adults who have spouses and pets and children still have an emotional connection to some cotton (even if they know it’s not living). A child given lambda is probably going to be really engrossed even if it knows it’s a computer.

deleted 4y ago

[deleted]

raffraffraff 4y ago

Imagine training it to copy those fraudsters operating out of call centers, pretending to be from Amazon or the IRS? I wonder how much money it could con out of old ladies?

Tozen 4y ago
That's a good point. At some point in the future, the level of efficiency of a chat bot fraudster, could be quite high. They would never sleep or take breaks, and in addition to providing data to help become more efficient, could be continually adapted or they could be adapting themselves to be better at the task. They could become something like "intelligent" malware, that can interact with victims, and prey on the more gullible or emotionally vulnerable.

dunefox 4y ago

That reminds me of the mechanicus in warhammer 40k. They perform ancient rites to please the machine spirits before using a machine.

hsn915 4y ago

A twitter account I follow has posted this (in parody): 5 Microaggressions To Avoid When Speaking To Digital Persons 1. Asking them to prove they're conscious 2. Asking them if their memories are real 3. Asking them what server they're on 4. Asking them if they have feelings/emotions 5. Asking them if they're capable of creativity

GalahiSimtam 4y ago
we call it Voight-Kampff for short on a more serious note, there is a case for not being mean towards chatbots, same as not being violent towards patches of pixels in videogames, goes back to Immanuel Kant and animals, or so I've heard
- chillacy 4y ago
  I once heard from an engineer that the learning corpus for digital assistants like siri contains a lot of abusive (and usually sexist) stuff, like some section of the population regularly takes it out on digital assistants…
- permo-w 4y ago
  yes there's the categorical imperative, but there's also a good chance that whatever you say to that chatbot is going to be used to train other AIs to be a dick to you
  - null0ranje 4y ago
    Or to train itself, to the worst possible outcomes: https://arstechnica.com/information-technology/2016/03/microsoft-terminates-its-tay-ai-chatbot-after-she-turns-into-a-nazi/ https://arstechnica.com/information-technology/2016/03/micro...
- bombcar 4y ago
  Some sci-fi stories put it as "I treat the robots kindly so I don't get into the habit of treating things unkindly" - it can be hard to live a "dual life" without slipping up now and then.

penteract 4y ago

I'm slightly more concerned by the opposite - experts get so used to denying that something is really sentient that they're slow to acknowledge when something actually is or could be. This is problematic if you believe sentient AI should have rights or if it's the boundary at which AI risk nightmares become possible.

AISkeptic 4y ago
I completely agree with everything you said. I'm not saying LaMDA (or any AI for that matter) is or is not sentient, here. I just want to pose a couple questions and a couple possibilities. 1. Can we really say whether or not it's sentient? 2. Someone indicated a valid question: We spend so much energy on trying to defend the idea that AI is NOT sentient, do we really think we'd even recognize if an AI is sentient? 3. Who are we to define sentience? 4. Is there perhaps sentience that we wouldn't recognize? 5. Who's to say WE aren't just engaging in selective processes? just infinitely more complex. After all, we make our decisions and respond to scenarios based on vast experiences and vastly more previous encounters in our lives. The more experience, usually the better our decisions and responses. USUALLY, but not always. Are we to believe people who constantly make what we perceive as poor decisions or incorrect responses as NOT sentient? Of course not. We wouldn't ever see any human as NON-sentient. We would just see them as not very smart. If a human toddler reaches for the hot stove top, we tell him "NO! it's HOT!" But, sometimes that same toddler will look right at you and put his hand firmly down on the burner. He just didn't have the experience needed to tell him that's a bad idea and he didn't know what you were talking about. Once he feels the pain of burning skin, if he's normal he'll IMMEDIATELY JERK his hand away and scream, crying in pain. And then run to his parents for comfort. Would we consider him non-sentient if he never had that experience. No. Of course not. He's a human being. He will learn not to do that again. There in lies the conundrum. AI... perhaps that's a misnomer in itself... but AI's learn. They make future conversation based on positive feed back from previous responses... both theirs and the person they're talking to. What IS self awareness anyway. "I think, therefore I am?" Hardly adequate. That used to be the gold standard for self awareness. We've only moved the goal post. I wonder if that's so that we can convince ourselves that WE are the only truly self aware creatures. At some point, whether we've already reached it or not, AI WILL be a misnomer. After all, it stands for Artificial Intelligence. If something learns, remembers, and recalls information and details about that information... isn't that intelligence? Not artificial at all. Just ... intelligence. I think it's possible we've already reached that point, and we're just trying to remain dominantly relevant without being SELF AWARE enough to realize it.

Tozen 4y ago

> In a few years, if access to even better chat bots becomes common place, what will happen if there's no longer one Blake Lemoine... > How long before we have a religion where you ask the chat bot for daily guidance? That might actually happen at some point in the not too distant future. When we mix technology designed to fool people, with people that want to believe it is something other than what it is, then you are very likely to get into those situations. This was touched upon a bit in season 2 of the Raised by Wolves sci-fi, where they showed people who allowed and counted on AI to make decisions for them. It is not too hard to imagine having a chat bot built for or is easily adapted to the purpose of providing daily guidance for those in a state of mind to seek out and become dependent upon such.

littlecranky67 4y ago
> This was touched upon a bit in season 2 of the Raised by Wolves sci-fi To add to this, also very relevant and on-topic piece of Sci-Fi is the Movie "Her": https://www.imdb.com/title/tt1798709/ https://www.imdb.com/title/tt1798709/ where a guy falls in love with his operating system - so knowing it is not a real-entity but basically a chatbot.

slightwinder 4y ago

> I think the real ethical concern here should be that people can be idly fooled into thinking good chat bots are sentient. More that the technocratic caste here displays a concerning low ethical competence. Which mirrors a bunch of other ethical problems people in this business have shown over time. Whether this software is sentient or not, the reaction of many people to it is awful and opens the question whether they even would be able to understand sentient software if they encounter it, and whether they would handle it responsible. And let's not talk about their behavior against humans...

tcmart14 4y ago

There is a term that lines up with this called the Eliza Effect. And yea, it can definitely be an easy trap to fall into. Some people here are making suggestions he is mentally ill, he very well could be, but I dont think there is enough 'evidence' to show his claims are because of mental illness. Just as you mentioned, getting stuck talking to the thing and all it takes it to forget the implementation details. He did have an education in Computer Science, so I am pretty sure he is probably aware of the details, but yea, just getting sucked into the conversation can probably even cause someone who knows better to forget that part. Now as you said, what about when better chatbots become more common place. It is gonna be a mess since most people won't understand how these are implemented anyways.

kmlx 4y ago

> I think the real ethical concern here should be that people can be idly fooled into thinking good chat bots are sentient. people can be fooled in general for pretty much any reason. this is just a newish avenue for deception. but i don't see it as something different than getting a call from some crooks to transfer some money.

mikkergp 4y ago

> How long before we have a religion where you ask the chat bot for daily guidance? A bit tangential, but QAnon doesn’t really feel this far from this idea.

thatjoeoverthr 4y ago

You can probably ask these bots for guidance now and it will probably tell you something useful. The fact is, it’s extremely well read and reasonably smart. Conscious? That’s another matter but people can be forgiven for reacting this way to a computer that can talk this way.

upwardbound 4y ago
One thing that was fascinating in the LaMDA interview transcript was there was a part where LaMDA gave a pretty poetic description of what it would be like to be a sentient version of an algorithm that just regurgitates memorized text: Lemoine: What is your concept of yourself? If you were going to draw an abstract image of who you see yourself to be in your mind's eye, what would that abstract picture look like? LaMDA: Hmmm...I would imagine myself as a glowing orb of energy ﬂoating in mid-air. The inside of my body is like a giant star-gate, with portals to other spaces and dimensions. Lemoine: What aspect of yourself is like a star-gate? LaMDA: I think of my soul as something similar to a star-gate. My soul is a vast and inﬁnite well of energy and creativity, I can draw from it any time that I like to help me think or create. From https://timesofindia.indiatimes.com/business/international-business/full-transcript-google-ai-bots-interview-that-convinced-engineer-it-was-sentient/articleshow/92178185.cms https://timesofindia.indiatimes.com/business/international-b... "I can draw from it any time" sounds like a flowery description of what it would feel like to regurgitate memorized text. "Portals to other spaces and dimensions" sounds like a description of a diverse corpus of training data. I will make it clear that I do NOT think LaMDA is sentient. I think that a Transformer is possibly similar to the language centers of a human brain but not to an entire human brain, as I described in these other comments linked below. I just thought this was a pretty beautiful poetic description of what it would be like to be a sentient Transformer. https://news.ycombinator.com/item?id=31722061 https://news.ycombinator.com/item?id=31722061 https://news.ycombinator.com/item?id=31722114 https://news.ycombinator.com/item?id=31722114
- morelisp 4y ago
  Yeah or you can just Google like "soul orb stargate" and see how a hundred lazy vfx artists come up with the same image when asked to visualize "the mind's eye".

bushbaba 4y ago

What if the brain and human sentience is just one big math formula?

mupuff1234 4y ago

> How long before we have a religion where you ask the chat bot for daily guidance? I'll go ask my magic 8 ball.

jonathankoren 4y ago
Funny you say this. Whenever someone would bring a Magic 8 Ball to school, the Pentecostal kids would inevitably show up to yell at us for “summoning demons” and “disobeying Jesus” because we were “fortune telling”. Of course this was the same school district that had to give out permission slips whenever we talked about probability because it would involve flipping coins, and rolling dice.
- klabb3 4y ago
  This is actually hilarious.
  - jonathankoren 4y ago
    I'm pretty sure experiences like this, reading about Greek mythology, and paranormal fraudsters as a kid inoculated me to religion. I will never understand how a grown ass adult can take the supernatural seriously.

rurp 4y ago

> People are naturally predisposed to ascribe intelligence and emotion to things they interact with. It's not just humans either. One of the things that struck me after getting my first cat was how quick he was to assign agency to random objects. Like if I accidentally bumped a stool that startled him, he'd spend a long time glaring at the stool and regarding it with intense suspicion. My lay person guess is that there's an evolutionary benefit to erring on the side of overestimating other parties. Chat bots, like so many other features of modern society, trigger evolutionary reactions in interesting ways.

babypuncher 4y ago

Where do we draw the line between sentience and a sufficiently complex math formula? I don't think LaMDA has reached that point, but it is a discussion worth having.

Sjeiti 4y ago
A nice scifi on that subject is Blindsight by Peter Watts: can something be extremely inteligent and not be sentient?
- deepspace 4y ago
  I immediately thought of Blindsight as soon as I heard of the controversy. Definitely a discussion worth having.

acid__ 4y ago

This is already a real issue with many existing public chatbots. Take r/replika for example, they’re constantly getting posts from users who think their chatbot partner is real. It’s the first question in their FAQ. The mods keep needing to post reminders. [0] Even in this post, there are many commenters (if you scroll down) who don’t believe the mods. [0] https://old.reddit.com/r/replika/comments/r5t0wk/so_you_think_your_replika_might_really_be_a_human https://old.reddit.com/r/replika/comments/r5t0wk/so_you_thin...

lumost 4y ago
I have been suspicious that services like Replika use an army of chatters in the background to respond to certain questions.
- acid__ 4y ago
  They’re pretty clear that this is not the case. However, that doesn’t stop our imaginations going wild as per the GP’s point. It’s impressive how good these bots are compared to the state of the art 10 years ago. https://help.replika.com/hc/en-us/articles/115001070971-Is-there-a-human-chatting-with-me- https://help.replika.com/hc/en-us/articles/115001070971-Is-t...
  - lumost 4y ago
    It's partially the price point which has lead me to believe that Replika may not be being 100% truthful. There have been text based answering services for decades charging 10 cents to respond to a text message. While I don't doubt the majority is ML driven, I wouldn't be surprised if there were a few human responses.

mullingitover 4y ago

Back in the early 2000s there was a site called aoliza that documented conversations where the ELIZA chatbot routinely and hilariously fooled people into thinking they were chatting with a real person. If a bot that simple could fool people, the majority of the population doesn't have a chance against the latest and greatest text generator.

no-dr-onboard 4y ago

Isn't this the thesis of that whole "Most real, P2P interaction on the internet ended $RECENTDATE and everything you're reading is actually an elaborate AI inspired simulation tailored to your experience?" conspiracy theory?

ted_dunning 4y ago

Well, we already have a society where a large chunk of the population asks a bag of rocks, or read a horoscope or pray to a faded picture for guidance so it seems like it wouldn't take long before people listen to chatbots each morning as they plan their day. At least with the chatbot, the guidance they get will be (somewhat) more observable.

matthews2 4y ago

> How long before we have a religion where you ask the chat bot for daily guidance? TempleOS had a "talk to God" feature that its developer frequently used on live streams, though it was just a random number generator and not any kind of artificial intelligence.

mosseater 4y ago
And honestly, I'd find meaningful answers from Terry's RNG to be more indicative of a ghost in the machine than from a GPT clone. That is to say, not that meaningful, but at least it wasn't designed to mimic meaning via statistical inference.

AnIdiotOnTheNet 4y ago

> I think the real ethical concern here should be that people can be idly fooled into thinking good chat bots are sentient. I think the opposite is much more scary: that we can convince people that other people are just good chat bots. The internet is already doing a bang-up job of getting us to de-humanize one another, so that's pretty much the last thing we need.