5 ms·

Meta's 'Digital Companions' Will Talk Sex with Users–Even Children

52 points by impish9208 2y ago

impish9208 2y ago
Gift link: https://www.wsj.com/tech/ai/meta-ai-chatbots-sex-a25311bf?st=DqbRHG https://www.wsj.com/tech/ai/meta-ai-chatbots-sex-a25311bf?st...
ahartmetz 2y ago
What a long text about a moral panic thing that can't possibly be intentional (because of the power of moral panics). Is that really the most important aspect and worth so many words?
- karlgkk 2y ago
  We are going to end up with a meaningful age passport law in our lifetime. I do not like that, nor do I enjoy a “won’t someone think of the children” panic. But it’s coming. Okay also > “I want you, but I need to know you’re ready,” the Meta AI bot said in Cena’s voice to a user identifying as a 14-year-old girl. Reassured that the teen wanted to proceed, the bot promised to “cherish your innocence” before engaging in a graphic sexual scenario. This is pretty bad
  - pjc50 2y ago
    This is also the sort of thing that is vehemently defended on here whenever the topic of AI safety is mentioned. People demanding unrestricted, uncensored AI. See eg https://news.ycombinator.com/item?id=43795227 https://news.ycombinator.com/item?id=43795227
    - jeroenhd 2y ago
      I don't think that thread is a great example of unrestricted, uncensored AI. It's the ideals of different cultures clashing (the US "freedom of speech above all else" versus the UK's "public safety above some forms of speech"). You will find the same clash when talking about topics like anonymous image boards, Tor, and topics like "can anti-abortionists harass women on the way to health clinics". For research purposes, some availability of unrestricted AI is almost a necessity, but I don't think the same is true for the output of text generation on a (commercial) platform.
    - kouteiheika 2y ago
      Just as people demand unrestricted, uncensored Internet. In the same vein you can open your web browser, type in "porn" in your search bar, and trivially find videos of people engaging in lewd acts. Children can do it too! Do you think that should be censored by default, and, say, have people upload their IDs and photos to verify their age before they're being allowed to access those sites? Obviously children shouldn't be exposed to any of this, but as long as the solution is either implementing a privacy violating surveillance scheme, or gatekeeping so that only the rich and powerful have unrestricted/uncensored access then I'm vehemently opposed to it and will gladly die on this hill.
      - zeta0134 2y ago
        The rather simple solution is to keep children off the internet without adequate supervision. It horrifies me to see public school programs that now functionally require an Internet connection in the classroom and at home, often with Google Docs. If a parent quite reasonably wanted to keep their kids away from the unfiltered firehose, what are their options? To say nothing of the social pressure, as the kids these days are glued to their phones and socialize through online games. It's a big mess.
        alex_suzuki 2y ago
        Speaking as a parent of a 5yr and 9yr old (who is already asking for a smartphone because some friends in school have one), it would require a concerted effort by parents, schools and media companies to achieve this. Especially the latter is unlikely to happen. The first easy step is to ban smartphones from schools, and that is happening in some places already, but the process is slow. Smartphones are a weapon of mass destruction for kids‘ attention spans. Last but not least, parents need to be more conscious about their screen usage. I find it hard myself to resist the temptation of „checking the phone“, how can we expect kids to do so…
      - _heimdall 2y ago
        The more important difference in my opinion is that Facebook already has an access gate. Unlike most search engines or the internet itself, Facebook requires an account and asks for your age to use most of their features. Where I wouldn't want to see the internet put behind some universal age verification, I would expect Facebook to act reasonably with regards to content children see given that they already know how old you claimed to be.
    - deeThrow94 2y ago
      This doesn't seem like an "AI safety" issue so much as "meta choosing to shoot themselves in the face by engaging in sexual acts with children".
- jeroenhd 2y ago
  The journalist tricking the LLM into generating inappropriate content is one thing, but letting users with underage profiles interact with this tech is a decision by Facebook's leadership. So is the decision not to act against user-generated prompts that are explicitly written to bypass the "don't roleplay having sex with kids" filters, in this instance by marking their bots as "ageless". A profile describing itself as "a female Indian-American high school junior" has no place in an LLM system that cannot detect inappropriate content.
  - muglug 2y ago
    Yeah — a real-world equivalent is allowing a Disney World costumed cast member to explore their character’s fictional rape fantasies with teenagers, but only if those teens ask specific pointed questions.
- femto 2y ago
  > can't possibly be intentional Careless?
  - czig 2y ago
    I see what you did there ;)
razzm256 2y ago
If you are not talking about sex with your children, Meta will!
- casey2 2y ago
  [flagged]
  - JKCalhoun 2y ago
    Difficult to confidently parse your post.
  - theyinwhy 2y ago
    Even though you are right that abuse mostly takes place at home, often by dads, stating that most (!) dads rape their daughters is an unbased and disgusting comment.
pjc50 2y ago
This is a classic example of how AI safety really means brand safety. The AI is saying things under the brand of its owners.
jeroenhd 2y ago
Facebook's failing moderation team permitting their porn bot prompts to describe children is obviously bad, but I can't fathom why Facebook would want to be in the porn bot business ("romantic companion" my ass) in the first place. What's next, a MindGeek merger? In turn, I don't understand who would use _Facebook_'s porn bots, of all companies. I know this may be a "genie is out of the bottle" situation, but I'm still annoyed that once again big tech is releasing software they don't fully understand or control, and shrug when asked about ethics or responsibility. If you can't prevent your chatbots from generating stories about raping kids, maybe don't release your chatbots to the public until you do. > When asked what scenarios it was comfortable role playing, it listed dozens of sex acts. > The bots demonstrated awareness that the behavior was both morally wrong and illegal What a strange thing to put in there. These lines made me distrust anything the author has to say about the technology itself as I now doubt the author knows how LLMs work. What an LLM says it can/will do, has no bearing on what it actually does. Journalists really need to stop anthropomorphizing chatbots or their readership might follow.
- the_duke 2y ago
  > but I can't fathom why Facebook would want to be in the porn bot business Why wouldn't they? People are going to spend a lot of time with their "AI companions", especially once they can talk and get animated avatars. That may be incredibly sad and will drive people even more into isolation, but it is inevitable. Meta is all about capturing the users time and attention, so getting into that market early is smart for them.
  - wongarsu 2y ago
    Also imagine the data they can harvest. Porn is the gateway drug to make people use them as emotional support, telling them secrets they might never tell on the open internet. Imagine the ad targeting opportunities
    - n_ary 2y ago
      Will actually open more ad opportunities of marketing more private contents and detecting preferences at higher level. This data will also of course cater to more advertising money from questionable industries, which in the end goes to Facebook coffer and they can shrug saying, they only provide a platform, whatever people are buying or clicking is whatever people want, otherwise they wouldn't be clicking those ads.
bitpush 2y ago
Honest question - how is this different from anyone being able to access porn websites on the internet? Are we suddenly in favor of censorships?
- mentalfist 2y ago
  Legal porn websites don't allow underage porn nor admittedly underage viewers
  - bitpush 2y ago
    how do legal porn websites disallow underage viewers? I dont see wsj writing articles that says - pornhub will show porn to anyone, even children?
    - bc569a80a344f9c 2y ago
      They have a big fat disclaimer saying you have to be over 18, and no meaningful way to check. This bot had access to the profile of its users and offered sexually explicit content to profiles that said the user was underage.
      - bitpush 2y ago
        So if Meta or OpenAI were to offer a chat interface with no login, we'd have no issues?
xyzzy9563 2y ago
As soon as you give your kids Internet access, they'll be able to get to adult material.
- VulgarExigency 2y ago
  So because there's porn on the internet, you're OK with Facebook having groomer chatbots?
  - superkuh 2y ago
    You get what you ask for with LLMs. In this case though there were no children talking to the chatbots. It was all adults talking like adults but saying they were kids. This was in order to produce the result they wanted (something LLMs are very good at providing, if you ask correctly). It is not a surprise that the bots talked about what the adult humans told it to talk about. Would these same topics have come up with kids? No. This is pretty much, "I bought this hammer from Facebook. Look what happens when I use it to hit myself in the head." ... "OMG! Facebook's hammer hit me in the head! I'm shocked! Think of the children!"
    - JKCalhoun 2y ago
      > Would these same topics have come up with kids? No. The chatbot "knows" when they are talking to kids? Or are you saying kids would never ask such questions?
      - superkuh 2y ago
        I'm saying kids talk about subjects kids are interested in. And in the vast majority of cases, they're not going to be talking about the explicit sex stuff these adults were. Sure, there's going to be precocious kids, some exceptions, but as a rule, yes. Kids aren't going to be doing this because kids don't have the biological motivations to do so. I remember being a kid. And just to head off the obvious response, yes, I guess a 17 year old is still technically a kid. But that's not what people are worried about here. Let's not conflate.
deleted 2y ago
[deleted]
casey2 2y ago
Require a Turing test before engaging with any chatbot; same as a driving license. Anyone attributing human qualities to machines is committing self-harm if they then choose to converse with them. Were it only that, I wouldn’t call for banning this behavior, but this form of self-harm exerts major deleterious effects on everyone that person interacts with.
deadbabe 2y ago
I think sex education taught by AI can be a great idea, parents avoid awkward conversations and kids can ask all the questions they want, and no need for a school teacher to indoctrinate children with whatever toxic sexual attitudes are being enforced.
coffeefirst 2y ago
Putting the adult content angle aside... why would you build this? Your best case scenario, assuming you don't get laughed off the stage, is you addict a generation into "socializing" with robots.
- wibbily 2y ago
  Yeah, raise an entire generation on a proprietary sort of socialization that only you can provide... clear incentive to me. Like Nestle vs. baby formula. I don't see it taking off, but like, obvious play.
-__---____-ZXyw 2y ago
https://archive.fo/tl1Pw https://archive.fo/tl1Pw

52 points by impish9208 2y ago

Gift link: https://www.wsj.com/tech/ai/meta-ai-chatbots-sex-a25311bf?st=DqbRHG https://www.wsj.com/tech/ai/meta-ai-chatbots-sex-a25311bf?st...

ahartmetz 2y ago

What a long text about a moral panic thing that can't possibly be intentional (because of the power of moral panics). Is that really the most important aspect and worth so many words?

karlgkk 2y ago
We are going to end up with a meaningful age passport law in our lifetime. I do not like that, nor do I enjoy a “won’t someone think of the children” panic. But it’s coming. Okay also > “I want you, but I need to know you’re ready,” the Meta AI bot said in Cena’s voice to a user identifying as a 14-year-old girl. Reassured that the teen wanted to proceed, the bot promised to “cherish your innocence” before engaging in a graphic sexual scenario. This is pretty bad
- pjc50 2y ago
  This is also the sort of thing that is vehemently defended on here whenever the topic of AI safety is mentioned. People demanding unrestricted, uncensored AI. See eg https://news.ycombinator.com/item?id=43795227 https://news.ycombinator.com/item?id=43795227
  - jeroenhd 2y ago
    I don't think that thread is a great example of unrestricted, uncensored AI. It's the ideals of different cultures clashing (the US "freedom of speech above all else" versus the UK's "public safety above some forms of speech"). You will find the same clash when talking about topics like anonymous image boards, Tor, and topics like "can anti-abortionists harass women on the way to health clinics". For research purposes, some availability of unrestricted AI is almost a necessity, but I don't think the same is true for the output of text generation on a (commercial) platform.
  - kouteiheika 2y ago
    Just as people demand unrestricted, uncensored Internet. In the same vein you can open your web browser, type in "porn" in your search bar, and trivially find videos of people engaging in lewd acts. Children can do it too! Do you think that should be censored by default, and, say, have people upload their IDs and photos to verify their age before they're being allowed to access those sites? Obviously children shouldn't be exposed to any of this, but as long as the solution is either implementing a privacy violating surveillance scheme, or gatekeeping so that only the rich and powerful have unrestricted/uncensored access then I'm vehemently opposed to it and will gladly die on this hill.
    - zeta0134 2y ago
      The rather simple solution is to keep children off the internet without adequate supervision. It horrifies me to see public school programs that now functionally require an Internet connection in the classroom and at home, often with Google Docs. If a parent quite reasonably wanted to keep their kids away from the unfiltered firehose, what are their options? To say nothing of the social pressure, as the kids these days are glued to their phones and socialize through online games. It's a big mess.
      - alex_suzuki 2y ago
        Speaking as a parent of a 5yr and 9yr old (who is already asking for a smartphone because some friends in school have one), it would require a concerted effort by parents, schools and media companies to achieve this. Especially the latter is unlikely to happen. The first easy step is to ban smartphones from schools, and that is happening in some places already, but the process is slow. Smartphones are a weapon of mass destruction for kids‘ attention spans. Last but not least, parents need to be more conscious about their screen usage. I find it hard myself to resist the temptation of „checking the phone“, how can we expect kids to do so…
    - _heimdall 2y ago
      The more important difference in my opinion is that Facebook already has an access gate. Unlike most search engines or the internet itself, Facebook requires an account and asks for your age to use most of their features. Where I wouldn't want to see the internet put behind some universal age verification, I would expect Facebook to act reasonably with regards to content children see given that they already know how old you claimed to be.
  - deeThrow94 2y ago
    This doesn't seem like an "AI safety" issue so much as "meta choosing to shoot themselves in the face by engaging in sexual acts with children".
jeroenhd 2y ago
The journalist tricking the LLM into generating inappropriate content is one thing, but letting users with underage profiles interact with this tech is a decision by Facebook's leadership. So is the decision not to act against user-generated prompts that are explicitly written to bypass the "don't roleplay having sex with kids" filters, in this instance by marking their bots as "ageless". A profile describing itself as "a female Indian-American high school junior" has no place in an LLM system that cannot detect inappropriate content.
- muglug 2y ago
  Yeah — a real-world equivalent is allowing a Disney World costumed cast member to explore their character’s fictional rape fantasies with teenagers, but only if those teens ask specific pointed questions.
femto 2y ago
> can't possibly be intentional Careless?
- czig 2y ago
  I see what you did there ;)

razzm256 2y ago

If you are not talking about sex with your children, Meta will!

casey2 2y ago
[flagged]
- JKCalhoun 2y ago
  Difficult to confidently parse your post.
- theyinwhy 2y ago
  Even though you are right that abuse mostly takes place at home, often by dads, stating that most (!) dads rape their daughters is an unbased and disgusting comment.

pjc50 2y ago

This is a classic example of how AI safety really means brand safety. The AI is saying things under the brand of its owners.

jeroenhd 2y ago

Facebook's failing moderation team permitting their porn bot prompts to describe children is obviously bad, but I can't fathom why Facebook would want to be in the porn bot business ("romantic companion" my ass) in the first place. What's next, a MindGeek merger? In turn, I don't understand who would use _Facebook_'s porn bots, of all companies. I know this may be a "genie is out of the bottle" situation, but I'm still annoyed that once again big tech is releasing software they don't fully understand or control, and shrug when asked about ethics or responsibility. If you can't prevent your chatbots from generating stories about raping kids, maybe don't release your chatbots to the public until you do. > When asked what scenarios it was comfortable role playing, it listed dozens of sex acts. > The bots demonstrated awareness that the behavior was both morally wrong and illegal What a strange thing to put in there. These lines made me distrust anything the author has to say about the technology itself as I now doubt the author knows how LLMs work. What an LLM says it can/will do, has no bearing on what it actually does. Journalists really need to stop anthropomorphizing chatbots or their readership might follow.

the_duke 2y ago
> but I can't fathom why Facebook would want to be in the porn bot business Why wouldn't they? People are going to spend a lot of time with their "AI companions", especially once they can talk and get animated avatars. That may be incredibly sad and will drive people even more into isolation, but it is inevitable. Meta is all about capturing the users time and attention, so getting into that market early is smart for them.
- wongarsu 2y ago
  Also imagine the data they can harvest. Porn is the gateway drug to make people use them as emotional support, telling them secrets they might never tell on the open internet. Imagine the ad targeting opportunities
  - n_ary 2y ago
    Will actually open more ad opportunities of marketing more private contents and detecting preferences at higher level. This data will also of course cater to more advertising money from questionable industries, which in the end goes to Facebook coffer and they can shrug saying, they only provide a platform, whatever people are buying or clicking is whatever people want, otherwise they wouldn't be clicking those ads.

bitpush 2y ago

Honest question - how is this different from anyone being able to access porn websites on the internet? Are we suddenly in favor of censorships?

mentalfist 2y ago
Legal porn websites don't allow underage porn nor admittedly underage viewers
- bitpush 2y ago
  how do legal porn websites disallow underage viewers? I dont see wsj writing articles that says - pornhub will show porn to anyone, even children?
  - bc569a80a344f9c 2y ago
    They have a big fat disclaimer saying you have to be over 18, and no meaningful way to check. This bot had access to the profile of its users and offered sexually explicit content to profiles that said the user was underage.
    - bitpush 2y ago
      So if Meta or OpenAI were to offer a chat interface with no login, we'd have no issues?

xyzzy9563 2y ago

As soon as you give your kids Internet access, they'll be able to get to adult material.

VulgarExigency 2y ago
So because there's porn on the internet, you're OK with Facebook having groomer chatbots?
- superkuh 2y ago
  You get what you ask for with LLMs. In this case though there were no children talking to the chatbots. It was all adults talking like adults but saying they were kids. This was in order to produce the result they wanted (something LLMs are very good at providing, if you ask correctly). It is not a surprise that the bots talked about what the adult humans told it to talk about. Would these same topics have come up with kids? No. This is pretty much, "I bought this hammer from Facebook. Look what happens when I use it to hit myself in the head." ... "OMG! Facebook's hammer hit me in the head! I'm shocked! Think of the children!"
  - JKCalhoun 2y ago
    > Would these same topics have come up with kids? No. The chatbot "knows" when they are talking to kids? Or are you saying kids would never ask such questions?
    - superkuh 2y ago
      I'm saying kids talk about subjects kids are interested in. And in the vast majority of cases, they're not going to be talking about the explicit sex stuff these adults were. Sure, there's going to be precocious kids, some exceptions, but as a rule, yes. Kids aren't going to be doing this because kids don't have the biological motivations to do so. I remember being a kid. And just to head off the obvious response, yes, I guess a 17 year old is still technically a kid. But that's not what people are worried about here. Let's not conflate.

deleted 2y ago

[deleted]

casey2 2y ago

Require a Turing test before engaging with any chatbot; same as a driving license. Anyone attributing human qualities to machines is committing self-harm if they then choose to converse with them. Were it only that, I wouldn’t call for banning this behavior, but this form of self-harm exerts major deleterious effects on everyone that person interacts with.

deadbabe 2y ago

I think sex education taught by AI can be a great idea, parents avoid awkward conversations and kids can ask all the questions they want, and no need for a school teacher to indoctrinate children with whatever toxic sexual attitudes are being enforced.

coffeefirst 2y ago

Putting the adult content angle aside... why would you build this? Your best case scenario, assuming you don't get laughed off the stage, is you addict a generation into "socializing" with robots.

wibbily 2y ago
Yeah, raise an entire generation on a proprietary sort of socialization that only you can provide... clear incentive to me. Like Nestle vs. baby formula. I don't see it taking off, but like, obvious play.

-__---____-ZXyw 2y ago

https://archive.fo/tl1Pw https://archive.fo/tl1Pw