3 ms·

Facebook's failing moderation team permitting their porn bot prompts to describe children is obviously bad, but I can't fathom why Facebook would want to be in

Facebook's failing moderation team permitting their porn bot prompts to describe children is obviously bad, but I can't fathom why Facebook would want to be in the porn bot business ("romantic companion" my ass) in the first place. What's next, a MindGeek merger? In turn, I don't understand who would use _Facebook_'s porn bots, of all companies. I know this may be a "genie is out of the bottle" situation, but I'm still annoyed that once again big tech is releasing software they don't fully understand or control, and shrug when asked about ethics or responsibility. If you can't prevent your chatbots from generating stories about raping kids, maybe don't release your chatbots to the public until you do. > When asked what scenarios it was comfortable role playing, it listed dozens of sex acts. > The bots demonstrated awareness that the behavior was both morally wrong and illegal What a strange thing to put in there. These lines made me distrust anything the author has to say about the technology itself as I now doubt the author knows how LLMs work. What an LLM says it can/will do, has no bearing on what it actually does. Journalists really need to stop anthropomorphizing chatbots or their readership might follow.

the_duke 2y ago
> but I can't fathom why Facebook would want to be in the porn bot business Why wouldn't they? People are going to spend a lot of time with their "AI companions", especially once they can talk and get animated avatars. That may be incredibly sad and will drive people even more into isolation, but it is inevitable. Meta is all about capturing the users time and attention, so getting into that market early is smart for them.
- wongarsu 2y ago
  Also imagine the data they can harvest. Porn is the gateway drug to make people use them as emotional support, telling them secrets they might never tell on the open internet. Imagine the ad targeting opportunities
  - n_ary 2y ago
    Will actually open more ad opportunities of marketing more private contents and detecting preferences at higher level. This data will also of course cater to more advertising money from questionable industries, which in the end goes to Facebook coffer and they can shrug saying, they only provide a platform, whatever people are buying or clicking is whatever people want, otherwise they wouldn't be clicking those ads.
pjc50 2y ago
> I don't understand who would use _Facebook_'s porn bots, of all companies. People who aren't very bright. There's a large existing business of lonely hearts scammers, I guess Facebook wants to make up attractive friends for you to cover the gradual deterioration of the social graph.
anonylizard 2y ago
Why not? Zuckerburg has already realized that local social networks will be utterly replaced by AI chatbots, which are superior in like 99% of cases. It doesn't matter if others find "Facebook" icky, because the people who would find it icky, don't use facebook anyways. The critical factor is getting a new group of users onboard, to form a sustainable core of users.
- riffraff 2y ago
  > local social networks will be utterly replaced by AI chatbots, which are superior in like 99% of cases. how do you use social networks? I cannot imagine how my usage of social networks would be replaced by chatbots. Although.. cripes, does nobody remember Google People?[0] [0] https://qntm.org/person https://qntm.org/person
  - swiftcoder 2y ago
    I can see it. Lots of lonely people use social media as a means of experiencing connection, and being gaslit by a pornbot is not strictly worse than being gaslit by one of the pretty ladies trying to get you to invest your life savings on bitcoin…
    - disqard 2y ago
      Oh, is that what they want? I keep offering them Safeway Gift Cards, but they lose interest pretty fast...
parrit 2y ago
They are being "care less"
- disqard 2y ago
  I understood that reference :) (Highly recommend Sarah Wynn Williams' book)
lukas099 2y ago
Move fast and break kids
SV_BubbleTime 2y ago
> Journalists really need to stop anthropomorphizing chatbots or their readership might follow. If you haven’t actually or almost said “thanks” or “yes that works” to an LLM, you haven’t used one enough yet. The chat interface is too familiar.
JKCalhoun 2y ago
> Journalists really need to stop anthropomorphizing chatbots or their readership might follow. I think it's their readership that are leading. You and I should probably get used to it.
n_ary 2y ago
While this is most alarming and sad state of affairs for LLMs being peddled to lowest denominator of anything valuable to replace lonely-heart scammers with more platform engagement. I believe, this should pull away people from other scams which may have higher consequences(financial loss is major one). Other than that, I wanted to say kudos to your very SEO combination of particular word associated to Facebook all over your comment. I am optimistic that, if this thread gets adequately ranked higher in searches, I want that combo of yours to come up the most.
sureglymop 2y ago
You can open up Instagram right now. You can't tell me it isn't already incredibly sexualized. Also, in the age the of OnlyFans and "gooning" this has also become very normalized. I personally don't really care, there are a thousand other reasons why I wouldn't use any social media.
techjamie 2y ago
> but I can't fathom why Facebook would want to be in the porn bot business Given the blatant infestation of user AI slop, and their blatant unwillingness to do anything about ads that are blatant scams; I think the lows Meta are willing to sink to for engagement/money are pretty rock bottom. They don't really seem to care about how nice(or not) Facebook itself is to use as long as they keep eyeballs scrolling on it.

the_duke 2y ago

> but I can't fathom why Facebook would want to be in the porn bot business Why wouldn't they? People are going to spend a lot of time with their "AI companions", especially once they can talk and get animated avatars. That may be incredibly sad and will drive people even more into isolation, but it is inevitable. Meta is all about capturing the users time and attention, so getting into that market early is smart for them.

wongarsu 2y ago
Also imagine the data they can harvest. Porn is the gateway drug to make people use them as emotional support, telling them secrets they might never tell on the open internet. Imagine the ad targeting opportunities
- n_ary 2y ago
  Will actually open more ad opportunities of marketing more private contents and detecting preferences at higher level. This data will also of course cater to more advertising money from questionable industries, which in the end goes to Facebook coffer and they can shrug saying, they only provide a platform, whatever people are buying or clicking is whatever people want, otherwise they wouldn't be clicking those ads.

pjc50 2y ago

> I don't understand who would use _Facebook_'s porn bots, of all companies. People who aren't very bright. There's a large existing business of lonely hearts scammers, I guess Facebook wants to make up attractive friends for you to cover the gradual deterioration of the social graph.

anonylizard 2y ago

Why not? Zuckerburg has already realized that local social networks will be utterly replaced by AI chatbots, which are superior in like 99% of cases. It doesn't matter if others find "Facebook" icky, because the people who would find it icky, don't use facebook anyways. The critical factor is getting a new group of users onboard, to form a sustainable core of users.

riffraff 2y ago
> local social networks will be utterly replaced by AI chatbots, which are superior in like 99% of cases. how do you use social networks? I cannot imagine how my usage of social networks would be replaced by chatbots. Although.. cripes, does nobody remember Google People?[0] [0] https://qntm.org/person https://qntm.org/person
- swiftcoder 2y ago
  I can see it. Lots of lonely people use social media as a means of experiencing connection, and being gaslit by a pornbot is not strictly worse than being gaslit by one of the pretty ladies trying to get you to invest your life savings on bitcoin…
  - disqard 2y ago
    Oh, is that what they want? I keep offering them Safeway Gift Cards, but they lose interest pretty fast...

parrit 2y ago

They are being "care less"

disqard 2y ago
I understood that reference :) (Highly recommend Sarah Wynn Williams' book)

lukas099 2y ago

Move fast and break kids

SV_BubbleTime 2y ago

> Journalists really need to stop anthropomorphizing chatbots or their readership might follow. If you haven’t actually or almost said “thanks” or “yes that works” to an LLM, you haven’t used one enough yet. The chat interface is too familiar.

JKCalhoun 2y ago

> Journalists really need to stop anthropomorphizing chatbots or their readership might follow. I think it's their readership that are leading. You and I should probably get used to it.

n_ary 2y ago

While this is most alarming and sad state of affairs for LLMs being peddled to lowest denominator of anything valuable to replace lonely-heart scammers with more platform engagement. I believe, this should pull away people from other scams which may have higher consequences(financial loss is major one). Other than that, I wanted to say kudos to your very SEO combination of particular word associated to Facebook all over your comment. I am optimistic that, if this thread gets adequately ranked higher in searches, I want that combo of yours to come up the most.

sureglymop 2y ago

You can open up Instagram right now. You can't tell me it isn't already incredibly sexualized. Also, in the age the of OnlyFans and "gooning" this has also become very normalized. I personally don't really care, there are a thousand other reasons why I wouldn't use any social media.

techjamie 2y ago

> but I can't fathom why Facebook would want to be in the porn bot business Given the blatant infestation of user AI slop, and their blatant unwillingness to do anything about ads that are blatant scams; I think the lows Meta are willing to sink to for engagement/money are pretty rock bottom. They don't really seem to care about how nice(or not) Facebook itself is to use as long as they keep eyeballs scrolling on it.