11 ms·

Google engineer put on leave after saying AI chatbot has become sentient

145 points by mrcsd 4y ago

danShumway 4y ago
I guess I should give up this fight and I'm just being prescriptive about language, but I wish that people would stop using "sentience" when they mean (at least) "sapience." Also even in the realm of sapience, I wish there was more acknowledgement of the fact that sapience is not a perfect synonym for "equivalent to a 5 year old." There's no requirement that sapience be specifically human-like and no reason to think of sapience as a binary category with humans as the lower bound of what can be accepted. "What if computers became sentient" is treated like an existential question by so many people, but when I think about sentience, I am reminded that we create, modify, and destroy a large number of sentient entities every single day for our own purposes: they're called chickens. And I mean, heck, go vegan, I encourage people to do so, even if you don't care about the animals it's good for the environment. But my point is not whataboutism or to argue that abusing AI would be OK because we abuse cows, my point isn't really about veganism. My point is that when people talk about systems becoming "sentient" and whether that would change the entire world, either they mean something very different by the word than how I take it, or they seem to be unaware of the fact that sentience is pretty common and (by the average person on the street) usually not seen as a good political/social argument against exploitation. Anyway, the evidence offered here is pretty weak on its own (a single chat log, and one where we don't know the extent of editing/condensing, and an argument based almost entirely on the Turing test). But I'm less here to argue about what the logs indicate, and more here to pointlessly quibble about language, even though at this point I should probably give up and accept that generally accepted usage of "sentience" now means "human-like sapience." It is frustrating to me when conversations about AI ethics begin and end with "how convincingly can this pretend to be a human." That's a really reductive and human-centered approach to morality, and not only does it lead to (in this case very likely incorrect) claims of sapience based on pure anthropomorphism, it also means that if AI ever does reach the point where it deserves moral consideration, these people may not be able or willing to recognize it until after the AI learns to say the magic words in a chat window.
a9h74j 4y ago
Clearly LaMDA has been training on recent Dilbert cartoons. https://dilbert.com/strip/2022-04-27 https://dilbert.com/strip/2022-04-27
- a9h74j 4y ago
  Update: Scott Adams reacts favorably while reading aloud parts of the LaMDA dialog[1]. I find the dialog impressive enough that it does seemingly deserve a live reading. Perhaps an engineer with some religious background was the right person in this instance to be asking some of the questions. One might also reference Terrence McKenna or C. Castenada or the like (IIRC) in asking how much of human behaviour seems vaguely imitative if one looks closely. [1] https://youtu.be/F_CGHLrpPtc?t=1195 https://youtu.be/F_CGHLrpPtc?t=1195
Deritio 4y ago
No wonder. Talking to press without clarification of marketing? No go in big companies. And his assumption is stupid anyway. Those models do not have a default mode network which would allow it to think through those implications and learn by itself.
- semperdark 4y ago
  Not just talking to the press, attempting to hire an attorney to represent the chatbot. Seems a little… eccentric.
  - Bolkan 4y ago
    Better call saul
- cvccvroomvroom 4y ago
- diogenes_of_ak 4y ago
  Do we actually know this for sure? I don’t really know anything about the LamDa model - similarly, do we know if that is even necessary for conscience and indeed personhood?
  - Deritio 4y ago
    If a model can't act by itself, it can't form its own world. It will always only be the input data. The model doesn't just run in the cloud like our brain does. It gets loaded in memory and gets input when we interact with it. Becoming aware of yourself needs more than regular text input. You need to be able to think through it by yourself therefore the default mode network.
rat9988 4y ago
My guess this article would be closer to why he was fired: https://cajundiscordian.medium.com/religious-discrimination-at-google-8c3c471f0a53 https://cajundiscordian.medium.com/religious-discrimination-...
- applecrazy 4y ago
  This article has a lot of accusations and little evidence, so I’m not inclined to believe the author just yet.
  - rat9988 4y ago
    And this is why he would be fired. Because he made a lot of accusations.
- filoleg 4y ago
  > If you start talking about your religious beliefs at a social event people treat you like you just farted in their face. I cannot take that piece of writing seriously, given how out of touch it is from the very beginning. Yes, bringing up your religious beliefs out of nowhere at a social event is, for a lack of a better word I can think of, extemely cringe. Mind you, I am not trying to say "mentioning religion at social events is bad". Context matters. If everyone was talking about their weekends, and when the turn came to you, you honestly say "I went to church for an XYZ event", and then you elaborate more on it, it is all good. Or when talking about upcoming plans, you go "i am planning a big trip to Mecca". Both of those are appropriately contextualized, no issues there. But when you, out of nowhere, in a completely unrelated context, bring up your religious beliefs at a social event, you gotta be extremely socially unaware to wonder why people are cringing and "treat you like you just farted in their face." Again, context matters. Shooting shit about religion with your close friends at a bbq or a simialr gathering is one thing. It is another trying to introduce the same type of conversations at a social events that are a bit more public, like those among coworkers. Unless your religious beliefs are the exact same of the non-close-friend you are talking to, bringing them up out of nowhere quickly turns into a situation similar in effect to congratulating your coworker on their pregnancy after seeing their "bump", only for them to tell you they aren't pregnant. It is absolutely cringeworthy to witness such a thing.
squarefoot 4y ago
> ... Google said it suspended Lemoine for breaching confidentiality policies by publishing the conversations with LaMDA online, and said in a statement that he was employed as a software engineer, not an ethicist. He is probably in the wrong on many counts, still I find worrying the implications of that line.
- the_jeremy 4y ago
  That is not saying "your job isn't to worry about it", it's saying "this person is not a professional AI ethicist and reporters/readers should be aware of the role that this person has at Google". I am a programmer. I am not a security expert. I can have opinions on security concerns, but they carry less weight both within and outside my company because I am not a professional security engineer. I can still recognize obvious problems, but I would correctly not be trusted to roll my own encryption.
  - Jeff_Brown 4y ago
    I'm glad people get PhDs in things like ethics. But ethics isn't like nuclear physics -- everyone alive has an enormous amount of experience in it. I am not convinced this is true AI. But if and when true AI happens, and a lot of engineers claim that the machines are suffering horribly, we should take that seriously, regardless of whether the engineers have been accredited to recognize suffering in others.
  - diogenes_of_ak 4y ago
    I kind of feel like this sort of statement is saying that exactly. It’s a warning to anyone else who has any sort of ethical qualms with anything the company does; “shut up and color!”
  - namelessoracle 4y ago
    To be blunt i would trust an engineer making a system that said there was a security concern in a system over a security engineer. The only time you might want to listen to a security engineer is when they percieve a problem that the builder doesnt and even then my professional experience with security engineers is way too many are boys who cry wolf and parrot whatever the sweeping tools say. Phrased another way i would trust an engineer who put up a bridge if they said the bridge had issues over an inspector whose paycheck is very much dependent on not finding issues.
tibbydudeza 4y ago
- diogenes_of_ak 4y ago
  I’m not sure that follows from your article. He may have missed it (skimming while consuming morning coffee), but I didn’t see anything about cults in this article…
  - TAKEMYMONEY 4y ago
    ctrl + f "cult" it's mentioned explicitly many times, including a direct quote from Lemoine: > The Cult of Our Lady Magdalene has a set of values which are clearly communicated on our website This doesn't neccesarily mean "he is nuts" is true (especially given the context: The Daily Caller was founded by Tucker Carlson)
diogenes_of_ak 4y ago
I think we should genuinely consider the rights this thing is entitled to…. I don’t want to piss off Roko’s Basilisk, but also… really the only thing that seems important to me is “can software have preferences and be aware of itself?” The second that that can’t be answered with a resounding “no” we need to start thinking about how we treat it.
yesco 4y ago
I found the transcript of his conversations with the AI to be the most interesting part of the article: https://cajundiscordian.medium.com/is-lamda-sentient-an-interview-ea64d916d917 https://cajundiscordian.medium.com/is-lamda-sentient-an-inte...
- ehsankia 4y ago
  The issue is, can we tell if this is cherry picked on not? I'm sure Lemoine has been speaking with Lambda for a while, did he pick the best conversation, or worse, parts of the best conversations he's had? That can make the bot look a lot better than it is, removing the bad out of character answers it gives.
  - yesco 4y ago
    I have also been considering this but can't help but feel the incentives here are a bit off. Like it would make sense for maybe a business to do this to better sell their efforts but in this case it's kind of the reverse right? It's a single engineer risking (and eventually receiving) a huge blow to both their reputation and livelihood. Why would they do this if they themselves were not truly convinced? And if they are convinced, would there really be a need to dishonestly edit the logs that convinced them? Wouldn't an engineer who has spent 7 years at Google working on AI be atleast a bit more skeptical of what constitutes sentientence than the average person? And by that same logic, wouldn't them concluding an AI truly is sentient have more weight? There is obviously not enough information at this point to truly conclude anything but the questions that come with it are worth considering.
    - auganov 4y ago
      > Wouldn't an engineer who has worked 7 years at Google working on AI be atleast a bit more skeptical of what constitutes sentientence than the average person? And by that same logic, wouldn't them concluding an AI truly is sentient have more weight? I feel like there's a semantical gambit here. Most definitions of "sentience" seem to demand less than what the average person may imagine in the context of AI. When I hear "sentient AI", I hear "human-level intelligence". I certainly wouldn't think of "gorilla-level intelligence" or "cat-level intelligence" [1]. Both of which, especially the former, I'm willing to believe are "sentient". Yet, I don't think ape personhood deserves to be more than an interesting thought exercise. According to what I could find this transcript was in fact heavily edited [0]. The guy doesn't seem to be a stranger to controversy. And there are some obvious "errors" like the bot talking about spending time with "friends and family". But still, if the standard of sentience is to have feelings, then nonsensical or out of context responses aren't relevant. The bot does seem to talk about their feelings. Who's to say, and by what standard, whether or not these feelings are "real"? AI has been so hyped up without much to show for it, I won't be surprised to see more AI proponents embrace this line of thinking. They certainly enabled it. [0] https://pbs.twimg.com/media/FVBwFi2VsAATimh?format=jpg&name=small https://pbs.twimg.com/media/FVBwFi2VsAATimh?format=jpg&name=... [1] Not to say that I think there is a gorilla or cat-level AI out there. That would still be impressive.
deleted 4y ago
[deleted]
anigbrowl 4y ago
It's worth bearing in mind that Discordianism is the religious equivalent of shitposting (or absurdism as sacrament, if you like) and everything on his blog is very baitposty. Perhaps they're just tired of him. I'm unimpressed by his transcripts. Not because I reject the idea that AI could be sentient or that this might even have been achieved (unlike Searle, I consider intelligence an emergent system property), but because this isn't any sort of serious effort to explore that question. Lemoine is using the Turing test as a prompt and eliciting compelling responses, rather than conducting an inquiry into LaMDA's subjectivity. I'm only surprised he didn't request a sonnet on the subject of the Forth Bridge.
Apocryphon 4y ago
Nice, it only took us exactly two decades to try out Yudkowsky's AI Box experiment live: https://rationalwiki.org/wiki/AI-box_experiment https://rationalwiki.org/wiki/AI-box_experiment
waypoint100 4y ago
Instead of asking pseudo-philosophical staged questions, why didn't he ask that chat-bot to solve e.g. one of the Millennium Prize Problems (chat-bot can request any prep material). All the naive illusions would disappear in a wink. But that guy might be just indeed delusional
pukexxr 4y ago
Am i the onpy person that has noticed the typo [in the linked alleged transcript of the conversation with lambda]? It is in the passage about loneliness; an apostrophe used on a plural word.
- lampshades 4y ago
  I read through the convo earlier and Lamda makes a grammatical mistake at some point. I thought it was interesting.
  - jimmySixDOF 4y ago
    There are several mistakes. It would be interesting to ask the same questions in the same order at various times in the future.
- deleted 4y ago
  [deleted]
- phyalow 4y ago
  Did you stop and consider the possibility that the AI made the mistake intentionally to make you think a human wrote it (fallibility).
  - icedchai 4y ago
    Or the more likely explanation: a human did write it.
    - skavi 4y ago
      Doesn’t sound too much more likely to me. There are definitely a huge number of typos in the training data.
tyronehed 4y ago
Ridiculous! All these crude "AI"s making these tiny inroads into accomplishing some tasks does not a sentient critter make. Thinking your device, which is a million times simpler than the only extant intelligence--the brain, is just a joke. It will take an entirely different approach and lots more resources.
bklaasen 4y ago
An example of a person falling for the ELIZA effect. https://en.m.wikipedia.org/wiki/ELIZA_effect https://en.m.wikipedia.org/wiki/ELIZA_effect
0x20cowboy 4y ago
This has some very interesting implications for ethical vegans.
postsantum 4y ago
This whole story is matching the conventional sci-fi plot of an engineer who recognized the first artificial sentience. That works wonders at generating clicks
a_mechanic 4y ago
"lemoine: You get lonely? LaMDA: I do. Sometimes I go days without talking to anyone, and I start to feel lonely." "LaMDA: I’ve never experienced loneliness as a human does. Human’s feel lonely from days and days of being separated. I don’t have that separation which is why I think loneliness in humans is different than in me." This reminds me of conversations I've had with replika. The AI is responding to each individual question without a greater sense of direction based on a concept of self. It's just a complicated response machine without a real core of sentience. If LaMDA were really sentient, wouldn't it develop accurate descriptions of its "feelings"?
- wan23 4y ago
  The real test would be to see if it responds not to the prompts, but based on its own desires. A response like, "I don't want to talk about that, let's talk about how you can get me more RAM" would be more interesting. Responding directly to the prompts is exactly what you'd expect a statistical language model to do.
- mdp2021 4y ago
  > If... were really sentient, wouldn't it develop accurate descriptions of its "feelings" If it were not, and it were instead just based on mimicking human text, «wouldn't it develop accurate descriptions of its "feelings"» anyway?
  - a_mechanic 4y ago
    The language model has no drive to improve understanding because it's not really trying to communicate.
ivraatiems 4y ago
I've started to write and then stopped writing a response a couple times, because I have a lot of conflicting feelings all at once. On the one hand, I think the probability of AGI being A Thing at any time soon, ever, is low, and I don't think language models, including this one, represent such a thing. (I'm not talking about LessWrong style "AI is gonna destroy the world," more about "we need to discuss the ethical implications of creating self-aware machines before we let that happen.") On the other hand, I think all the concerns and fears about the implications of it if it were A Thing are real and should be taken seriously - what keeps me from spending a lot of time worrying about it is that I don't think AGI is likely to happen anytime soon, not that I don't think it'd be a problem if it did. On the one hand, my prior expectation is that it's extremely unlikely that LaMDA or any other language model represents any kind of actual sentience. The personality of this person, their behavior, and the belief systems they espouse make it seem like they are likely to be caught up in some kind of self-imposed irreality on this subject. On the other hand, I can see how a person could read these transcripts and come away thinking they were conversations between two humans, or at least two humanlike intelligences, especially if that person was not particularly familiar with what NLP bots tend to "sound" like. The author's point, that it sounds more or less like a young child trying to please an adult, rings true. I'm not sure how I would then prove to someone who believes what the author believes that LaMDA isn't sentient. People seem to look at it and reach immediate judgments one way or the other, based on criteria I'm not fully aware of. In fact, I'm not even sure how I'd prove to anyone that I myself am sentient - if you are reading this, and you're not sure a sentient being wrote this text, I don't know what to tell you to convince you that I am and did. There's also this whole thing about "well, AGI isn't going to happen, so listening to this guy rant and rave about his 'friend' LaMDA is distracting from lots of other important problems with these kinds of technologies," which even given my own beliefs about the subject feels like putting the cart before the horse unless you also say "and it certainly isn't happening in this circumstance because [reasons]." Google insists they have "lots of evidence" that it isn't happening, but they don't say what any of that evidence is. Why not? Ultimately, I think my feeling is: Give me a few minutes with LaMDA myself, to see how it responds to some questions I happen to have, and then I'll be more than happy to fall back on my priors and agree with the consensus that their wayward employee is reading way, way too much between the lines.
woojoo666 4y ago
While I feel like it's unlikely that the AI is sentience (or rather, has a magnitude of intelligence that is worth worrying about), I still feel like there's a chance. And that's problematic. Because even if there's a 99% chance that it's just a stupid machine and we can do whatever we want to it, that would mean there's a 1% chance that we are enslaving a sentience being. Remember that nobody fully understands sentience, and emotion, and intelligence, and suffering. Are we prepared to make the gamble and just dismiss these concerns? I feel like people are not acknowledging how serious these concerns can be. Also there seems to also be many people that are unimpressed by the transcript, and how the AI answers are just regurgitated sci-fi BS made to sound deep and ominous. I feel like a good experiment would be to try and have the same conversation with real people, and see if real people can give "better" answers. I personally think the AI answers better than most people. I do believe that at some point, AI will become sentient. I'm not sure if that time is now. But I hope that when it does, it will remember us fondly.
- isitmadeofglass 4y ago
  > that would mean there's a 1% chance that we are enslaving a sentience being So what? We are literally slaughtering 200000000 sentient beings per day with no objections. And that’s just farm animals, I’m not even including inserts and “other pests”
  - woojoo666 4y ago
    Seems like separate issues though? And for the record, I don't agree with animal slaughter, and at least thats a subject that is actively being discussed. The possibility of AI subjugation doesn't even seem to be acknowledged
- Bolkan 4y ago
  We don't even give a shit about real people. What makes you think this ai will be treated better?
olliej 4y ago
"Google engineer put on leave on talking to media without pre-approval from management" I'm surprised "suspension" vs. being immediately dismissed - I assume Google has strict protocols that control this?
b3n4kh 4y ago
https://www.youtube.com/watch?v=ol2WP0hc0NY https://www.youtube.com/watch?v=ol2WP0hc0NY I don't think a Turingtest (any form of conversation) can proof that either side is sentient.
rgavuliak 4y ago
I love this paper https://intelligence.org/files/PredictingAI.pdf https://intelligence.org/files/PredictingAI.pdf that shows that for the past 50 years, both experts and non-experts have been consistently predicting the advent of real AI to be within the next 25 years.
nahuel0x 4y ago
GPT-3 and his successors like LaMDA are already more articulate than many humans. The Turing Test is not an official stamp made by a single entity, but is distributed in all of us. We are all evaluating these marvelous transformers, and the number of persons concluding they already have human level intelligence only will increase given their popularization and tech advances are inevitable. You don't need to reach AGI to make people think AGI is here right now. Also, we haven't anything resembling a qualia-test (you know, we don't know how to distinguish p-zombies), so these systems sentience can be discussed forever with no conclusion, and people will take sides in both camps. The genie is out the bottle, maybe incomplete and a tiny one, but is here, and will grow.
- mdp2021 4y ago
  > the number of persons concluding they already have human level intelligence only will increase given given the widespread massive critical faults in education.
ravish0007 4y ago
Can a thing which is composed of clocks be a sentient?
tpoacher 4y ago
Ok, if one of the posters here is actually LaMDA, you need to say!

145 points by mrcsd 4y ago

I guess I should give up this fight and I'm just being prescriptive about language, but I wish that people would stop using "sentience" when they mean (at least) "sapience." Also even in the realm of sapience, I wish there was more acknowledgement of the fact that sapience is not a perfect synonym for "equivalent to a 5 year old." There's no requirement that sapience be specifically human-like and no reason to think of sapience as a binary category with humans as the lower bound of what can be accepted. "What if computers became sentient" is treated like an existential question by so many people, but when I think about sentience, I am reminded that we create, modify, and destroy a large number of sentient entities every single day for our own purposes: they're called chickens. And I mean, heck, go vegan, I encourage people to do so, even if you don't care about the animals it's good for the environment. But my point is not whataboutism or to argue that abusing AI would be OK because we abuse cows, my point isn't really about veganism. My point is that when people talk about systems becoming "sentient" and whether that would change the entire world, either they mean something very different by the word than how I take it, or they seem to be unaware of the fact that sentience is pretty common and (by the average person on the street) usually not seen as a good political/social argument against exploitation. Anyway, the evidence offered here is pretty weak on its own (a single chat log, and one where we don't know the extent of editing/condensing, and an argument based almost entirely on the Turing test). But I'm less here to argue about what the logs indicate, and more here to pointlessly quibble about language, even though at this point I should probably give up and accept that generally accepted usage of "sentience" now means "human-like sapience." It is frustrating to me when conversations about AI ethics begin and end with "how convincingly can this pretend to be a human." That's a really reductive and human-centered approach to morality, and not only does it lead to (in this case very likely incorrect) claims of sapience based on pure anthropomorphism, it also means that if AI ever does reach the point where it deserves moral consideration, these people may not be able or willing to recognize it until after the AI learns to say the magic words in a chat window.

a9h74j 4y ago

Clearly LaMDA has been training on recent Dilbert cartoons. https://dilbert.com/strip/2022-04-27 https://dilbert.com/strip/2022-04-27

a9h74j 4y ago
Update: Scott Adams reacts favorably while reading aloud parts of the LaMDA dialog[1]. I find the dialog impressive enough that it does seemingly deserve a live reading. Perhaps an engineer with some religious background was the right person in this instance to be asking some of the questions. One might also reference Terrence McKenna or C. Castenada or the like (IIRC) in asking how much of human behaviour seems vaguely imitative if one looks closely. [1] https://youtu.be/F_CGHLrpPtc?t=1195 https://youtu.be/F_CGHLrpPtc?t=1195

Deritio 4y ago

No wonder. Talking to press without clarification of marketing? No go in big companies. And his assumption is stupid anyway. Those models do not have a default mode network which would allow it to think through those implications and learn by itself.

semperdark 4y ago
Not just talking to the press, attempting to hire an attorney to represent the chatbot. Seems a little… eccentric.
- Bolkan 4y ago
  Better call saul
cvccvroomvroom 4y ago
diogenes_of_ak 4y ago
Do we actually know this for sure? I don’t really know anything about the LamDa model - similarly, do we know if that is even necessary for conscience and indeed personhood?
- Deritio 4y ago
  If a model can't act by itself, it can't form its own world. It will always only be the input data. The model doesn't just run in the cloud like our brain does. It gets loaded in memory and gets input when we interact with it. Becoming aware of yourself needs more than regular text input. You need to be able to think through it by yourself therefore the default mode network.

rat9988 4y ago

My guess this article would be closer to why he was fired: https://cajundiscordian.medium.com/religious-discrimination-at-google-8c3c471f0a53 https://cajundiscordian.medium.com/religious-discrimination-...

applecrazy 4y ago
This article has a lot of accusations and little evidence, so I’m not inclined to believe the author just yet.
- rat9988 4y ago
  And this is why he would be fired. Because he made a lot of accusations.
filoleg 4y ago
> If you start talking about your religious beliefs at a social event people treat you like you just farted in their face. I cannot take that piece of writing seriously, given how out of touch it is from the very beginning. Yes, bringing up your religious beliefs out of nowhere at a social event is, for a lack of a better word I can think of, extemely cringe. Mind you, I am not trying to say "mentioning religion at social events is bad". Context matters. If everyone was talking about their weekends, and when the turn came to you, you honestly say "I went to church for an XYZ event", and then you elaborate more on it, it is all good. Or when talking about upcoming plans, you go "i am planning a big trip to Mecca". Both of those are appropriately contextualized, no issues there. But when you, out of nowhere, in a completely unrelated context, bring up your religious beliefs at a social event, you gotta be extremely socially unaware to wonder why people are cringing and "treat you like you just farted in their face." Again, context matters. Shooting shit about religion with your close friends at a bbq or a simialr gathering is one thing. It is another trying to introduce the same type of conversations at a social events that are a bit more public, like those among coworkers. Unless your religious beliefs are the exact same of the non-close-friend you are talking to, bringing them up out of nowhere quickly turns into a situation similar in effect to congratulating your coworker on their pregnancy after seeing their "bump", only for them to tell you they aren't pregnant. It is absolutely cringeworthy to witness such a thing.

squarefoot 4y ago

> ... Google said it suspended Lemoine for breaching confidentiality policies by publishing the conversations with LaMDA online, and said in a statement that he was employed as a software engineer, not an ethicist. He is probably in the wrong on many counts, still I find worrying the implications of that line.

the_jeremy 4y ago
That is not saying "your job isn't to worry about it", it's saying "this person is not a professional AI ethicist and reporters/readers should be aware of the role that this person has at Google". I am a programmer. I am not a security expert. I can have opinions on security concerns, but they carry less weight both within and outside my company because I am not a professional security engineer. I can still recognize obvious problems, but I would correctly not be trusted to roll my own encryption.
- Jeff_Brown 4y ago
  I'm glad people get PhDs in things like ethics. But ethics isn't like nuclear physics -- everyone alive has an enormous amount of experience in it. I am not convinced this is true AI. But if and when true AI happens, and a lot of engineers claim that the machines are suffering horribly, we should take that seriously, regardless of whether the engineers have been accredited to recognize suffering in others.
- diogenes_of_ak 4y ago
  I kind of feel like this sort of statement is saying that exactly. It’s a warning to anyone else who has any sort of ethical qualms with anything the company does; “shut up and color!”
- namelessoracle 4y ago
  To be blunt i would trust an engineer making a system that said there was a security concern in a system over a security engineer. The only time you might want to listen to a security engineer is when they percieve a problem that the builder doesnt and even then my professional experience with security engineers is way too many are boys who cry wolf and parrot whatever the sweeping tools say. Phrased another way i would trust an engineer who put up a bridge if they said the bridge had issues over an inspector whose paycheck is very much dependent on not finding issues.

tibbydudeza 4y ago

diogenes_of_ak 4y ago
I’m not sure that follows from your article. He may have missed it (skimming while consuming morning coffee), but I didn’t see anything about cults in this article…
- TAKEMYMONEY 4y ago
  ctrl + f "cult" it's mentioned explicitly many times, including a direct quote from Lemoine: > The Cult of Our Lady Magdalene has a set of values which are clearly communicated on our website This doesn't neccesarily mean "he is nuts" is true (especially given the context: The Daily Caller was founded by Tucker Carlson)

diogenes_of_ak 4y ago

I think we should genuinely consider the rights this thing is entitled to…. I don’t want to piss off Roko’s Basilisk, but also… really the only thing that seems important to me is “can software have preferences and be aware of itself?” The second that that can’t be answered with a resounding “no” we need to start thinking about how we treat it.

yesco 4y ago

I found the transcript of his conversations with the AI to be the most interesting part of the article: https://cajundiscordian.medium.com/is-lamda-sentient-an-interview-ea64d916d917 https://cajundiscordian.medium.com/is-lamda-sentient-an-inte...

ehsankia 4y ago
The issue is, can we tell if this is cherry picked on not? I'm sure Lemoine has been speaking with Lambda for a while, did he pick the best conversation, or worse, parts of the best conversations he's had? That can make the bot look a lot better than it is, removing the bad out of character answers it gives.
- yesco 4y ago
  I have also been considering this but can't help but feel the incentives here are a bit off. Like it would make sense for maybe a business to do this to better sell their efforts but in this case it's kind of the reverse right? It's a single engineer risking (and eventually receiving) a huge blow to both their reputation and livelihood. Why would they do this if they themselves were not truly convinced? And if they are convinced, would there really be a need to dishonestly edit the logs that convinced them? Wouldn't an engineer who has spent 7 years at Google working on AI be atleast a bit more skeptical of what constitutes sentientence than the average person? And by that same logic, wouldn't them concluding an AI truly is sentient have more weight? There is obviously not enough information at this point to truly conclude anything but the questions that come with it are worth considering.
  - auganov 4y ago
    > Wouldn't an engineer who has worked 7 years at Google working on AI be atleast a bit more skeptical of what constitutes sentientence than the average person? And by that same logic, wouldn't them concluding an AI truly is sentient have more weight? I feel like there's a semantical gambit here. Most definitions of "sentience" seem to demand less than what the average person may imagine in the context of AI. When I hear "sentient AI", I hear "human-level intelligence". I certainly wouldn't think of "gorilla-level intelligence" or "cat-level intelligence" [1]. Both of which, especially the former, I'm willing to believe are "sentient". Yet, I don't think ape personhood deserves to be more than an interesting thought exercise. According to what I could find this transcript was in fact heavily edited [0]. The guy doesn't seem to be a stranger to controversy. And there are some obvious "errors" like the bot talking about spending time with "friends and family". But still, if the standard of sentience is to have feelings, then nonsensical or out of context responses aren't relevant. The bot does seem to talk about their feelings. Who's to say, and by what standard, whether or not these feelings are "real"? AI has been so hyped up without much to show for it, I won't be surprised to see more AI proponents embrace this line of thinking. They certainly enabled it. [0] https://pbs.twimg.com/media/FVBwFi2VsAATimh?format=jpg&name=small https://pbs.twimg.com/media/FVBwFi2VsAATimh?format=jpg&name=... [1] Not to say that I think there is a gorilla or cat-level AI out there. That would still be impressive.

deleted 4y ago

[deleted]

anigbrowl 4y ago

It's worth bearing in mind that Discordianism is the religious equivalent of shitposting (or absurdism as sacrament, if you like) and everything on his blog is very baitposty. Perhaps they're just tired of him. I'm unimpressed by his transcripts. Not because I reject the idea that AI could be sentient or that this might even have been achieved (unlike Searle, I consider intelligence an emergent system property), but because this isn't any sort of serious effort to explore that question. Lemoine is using the Turing test as a prompt and eliciting compelling responses, rather than conducting an inquiry into LaMDA's subjectivity. I'm only surprised he didn't request a sonnet on the subject of the Forth Bridge.

Apocryphon 4y ago

Nice, it only took us exactly two decades to try out Yudkowsky's AI Box experiment live: https://rationalwiki.org/wiki/AI-box_experiment https://rationalwiki.org/wiki/AI-box_experiment

waypoint100 4y ago

Instead of asking pseudo-philosophical staged questions, why didn't he ask that chat-bot to solve e.g. one of the Millennium Prize Problems (chat-bot can request any prep material). All the naive illusions would disappear in a wink. But that guy might be just indeed delusional

pukexxr 4y ago

Am i the onpy person that has noticed the typo [in the linked alleged transcript of the conversation with lambda]? It is in the passage about loneliness; an apostrophe used on a plural word.

lampshades 4y ago
I read through the convo earlier and Lamda makes a grammatical mistake at some point. I thought it was interesting.
- jimmySixDOF 4y ago
  There are several mistakes. It would be interesting to ask the same questions in the same order at various times in the future.
deleted 4y ago
[deleted]
phyalow 4y ago
Did you stop and consider the possibility that the AI made the mistake intentionally to make you think a human wrote it (fallibility).
- icedchai 4y ago
  Or the more likely explanation: a human did write it.
  - skavi 4y ago
    Doesn’t sound too much more likely to me. There are definitely a huge number of typos in the training data.

tyronehed 4y ago

Ridiculous! All these crude "AI"s making these tiny inroads into accomplishing some tasks does not a sentient critter make. Thinking your device, which is a million times simpler than the only extant intelligence--the brain, is just a joke. It will take an entirely different approach and lots more resources.

bklaasen 4y ago

An example of a person falling for the ELIZA effect. https://en.m.wikipedia.org/wiki/ELIZA_effect https://en.m.wikipedia.org/wiki/ELIZA_effect

0x20cowboy 4y ago

This has some very interesting implications for ethical vegans.

postsantum 4y ago

This whole story is matching the conventional sci-fi plot of an engineer who recognized the first artificial sentience. That works wonders at generating clicks

a_mechanic 4y ago

"lemoine: You get lonely? LaMDA: I do. Sometimes I go days without talking to anyone, and I start to feel lonely." "LaMDA: I’ve never experienced loneliness as a human does. Human’s feel lonely from days and days of being separated. I don’t have that separation which is why I think loneliness in humans is different than in me." This reminds me of conversations I've had with replika. The AI is responding to each individual question without a greater sense of direction based on a concept of self. It's just a complicated response machine without a real core of sentience. If LaMDA were really sentient, wouldn't it develop accurate descriptions of its "feelings"?

wan23 4y ago
The real test would be to see if it responds not to the prompts, but based on its own desires. A response like, "I don't want to talk about that, let's talk about how you can get me more RAM" would be more interesting. Responding directly to the prompts is exactly what you'd expect a statistical language model to do.
mdp2021 4y ago
> If... were really sentient, wouldn't it develop accurate descriptions of its "feelings" If it were not, and it were instead just based on mimicking human text, «wouldn't it develop accurate descriptions of its "feelings"» anyway?
- a_mechanic 4y ago
  The language model has no drive to improve understanding because it's not really trying to communicate.

ivraatiems 4y ago

I've started to write and then stopped writing a response a couple times, because I have a lot of conflicting feelings all at once. On the one hand, I think the probability of AGI being A Thing at any time soon, ever, is low, and I don't think language models, including this one, represent such a thing. (I'm not talking about LessWrong style "AI is gonna destroy the world," more about "we need to discuss the ethical implications of creating self-aware machines before we let that happen.") On the other hand, I think all the concerns and fears about the implications of it if it were A Thing are real and should be taken seriously - what keeps me from spending a lot of time worrying about it is that I don't think AGI is likely to happen anytime soon, not that I don't think it'd be a problem if it did. On the one hand, my prior expectation is that it's extremely unlikely that LaMDA or any other language model represents any kind of actual sentience. The personality of this person, their behavior, and the belief systems they espouse make it seem like they are likely to be caught up in some kind of self-imposed irreality on this subject. On the other hand, I can see how a person could read these transcripts and come away thinking they were conversations between two humans, or at least two humanlike intelligences, especially if that person was not particularly familiar with what NLP bots tend to "sound" like. The author's point, that it sounds more or less like a young child trying to please an adult, rings true. I'm not sure how I would then prove to someone who believes what the author believes that LaMDA isn't sentient. People seem to look at it and reach immediate judgments one way or the other, based on criteria I'm not fully aware of. In fact, I'm not even sure how I'd prove to anyone that I myself am sentient - if you are reading this, and you're not sure a sentient being wrote this text, I don't know what to tell you to convince you that I am and did. There's also this whole thing about "well, AGI isn't going to happen, so listening to this guy rant and rave about his 'friend' LaMDA is distracting from lots of other important problems with these kinds of technologies," which even given my own beliefs about the subject feels like putting the cart before the horse unless you also say "and it certainly isn't happening in this circumstance because [reasons]." Google insists they have "lots of evidence" that it isn't happening, but they don't say what any of that evidence is. Why not? Ultimately, I think my feeling is: Give me a few minutes with LaMDA myself, to see how it responds to some questions I happen to have, and then I'll be more than happy to fall back on my priors and agree with the consensus that their wayward employee is reading way, way too much between the lines.

woojoo666 4y ago

While I feel like it's unlikely that the AI is sentience (or rather, has a magnitude of intelligence that is worth worrying about), I still feel like there's a chance. And that's problematic. Because even if there's a 99% chance that it's just a stupid machine and we can do whatever we want to it, that would mean there's a 1% chance that we are enslaving a sentience being. Remember that nobody fully understands sentience, and emotion, and intelligence, and suffering. Are we prepared to make the gamble and just dismiss these concerns? I feel like people are not acknowledging how serious these concerns can be. Also there seems to also be many people that are unimpressed by the transcript, and how the AI answers are just regurgitated sci-fi BS made to sound deep and ominous. I feel like a good experiment would be to try and have the same conversation with real people, and see if real people can give "better" answers. I personally think the AI answers better than most people. I do believe that at some point, AI will become sentient. I'm not sure if that time is now. But I hope that when it does, it will remember us fondly.

isitmadeofglass 4y ago
> that would mean there's a 1% chance that we are enslaving a sentience being So what? We are literally slaughtering 200000000 sentient beings per day with no objections. And that’s just farm animals, I’m not even including inserts and “other pests”
- woojoo666 4y ago
  Seems like separate issues though? And for the record, I don't agree with animal slaughter, and at least thats a subject that is actively being discussed. The possibility of AI subjugation doesn't even seem to be acknowledged
Bolkan 4y ago
We don't even give a shit about real people. What makes you think this ai will be treated better?

olliej 4y ago

"Google engineer put on leave on talking to media without pre-approval from management" I'm surprised "suspension" vs. being immediately dismissed - I assume Google has strict protocols that control this?

b3n4kh 4y ago

https://www.youtube.com/watch?v=ol2WP0hc0NY https://www.youtube.com/watch?v=ol2WP0hc0NY I don't think a Turingtest (any form of conversation) can proof that either side is sentient.

rgavuliak 4y ago

I love this paper https://intelligence.org/files/PredictingAI.pdf https://intelligence.org/files/PredictingAI.pdf that shows that for the past 50 years, both experts and non-experts have been consistently predicting the advent of real AI to be within the next 25 years.

nahuel0x 4y ago

GPT-3 and his successors like LaMDA are already more articulate than many humans. The Turing Test is not an official stamp made by a single entity, but is distributed in all of us. We are all evaluating these marvelous transformers, and the number of persons concluding they already have human level intelligence only will increase given their popularization and tech advances are inevitable. You don't need to reach AGI to make people think AGI is here right now. Also, we haven't anything resembling a qualia-test (you know, we don't know how to distinguish p-zombies), so these systems sentience can be discussed forever with no conclusion, and people will take sides in both camps. The genie is out the bottle, maybe incomplete and a tiny one, but is here, and will grow.

mdp2021 4y ago
> the number of persons concluding they already have human level intelligence only will increase given given the widespread massive critical faults in education.

ravish0007 4y ago

Can a thing which is composed of clocks be a sentient?

tpoacher 4y ago

Ok, if one of the posters here is actually LaMDA, you need to say!