5 ms·

Real photos of young girls were in nudify-app ads on Facebook, Instagram

75 points by Bender 27d ago

cactacea 27d ago
> In captions, ads enticed Meta users to click with claims like “I can show you more” or promises that “our AI transforms your hidden desires into hypnotic videos. Dark, seductive, unforgettable.” Another boasted, “there are no restrictions. All the characters are real people and there are many options. This is the AI men actually use.” > Even more troublingly, TTP emphasized in a press release that these findings came after the group teamed up with Wired and flagged an earlier batch of 53 ads in August. At that time, Meta claimed that it had “recently rolled out new systems to better detect child exploitation.” But TTP “found hundreds of new CSAM ads ran after Meta boasted about its new technology.” Prison is too good for the people responsible for this.
- FireBeyond 27d ago
  Garbage, too. I've reported at least 10 of these things. Ads, reels. Specifically as child abuse. "After review, we determined this content does not breach our policies."
  - mitxela 27d ago
    Of course. No large company cares about child abuse. They've all been to Epstein island, or one of the other unknown islands.
    - FireBeyond 27d ago
      Hah, I don't know if Zuck has visited, but if he or any of the exec team has, you can bet that filter is finely finely tuned.
  - peterashford 27d ago
    Expecting a corporate to police itself is, obviously, a joke
bsoqk 27d ago
>the Tech Transparency Project (TTP) reported that Meta failed to detect 332 ads containing CSAM this year Considering the amount of ads they handle that seems... reasonable? Not to mention the stupidity of calling the result of nudify apps "CSAM" when no minors in situation of sexual abuse are used in any way to generate those photos.
- washadjeffmad 27d ago
  This is one of those topics that gets radioactive quickly, so I'm trying to parse this comment under the recent ruling by a judge that certain obscene products of generative AI are protected under the First Amendment. "We now live in an age where GenAI [generative AI] models can render images depicting the abuse of virtual children that are virtually indistinguishable from those depicting the abuse of actual children. This case illuminates how this evolving technology complicates the lines drawn by the Supreme Court in Stanley, Osborne, and Free Speech Coalition. Indeed, in Free Speech Coalition, the Supreme Court addressed the scope of First Amendment protections for virtual CSAM, but that was nearly twenty-five years ago, and the image-generation technology available today was likely unimaginable back then. Given the relentless advancement in artificial intelligence models, we have some concerns about the lines these cases draw, but we are not free to redraw them ourselves." https://www.snopes.com/fact-check/judge-ai-child-sex-abuse-ruling/ https://www.snopes.com/fact-check/judge-ai-child-sex-abuse-r... Despite the source images not being CSAM, the products of nudify services may be CSAM but not child pornography. Because nudifying minors involves generation of material intended to depict real people who cannot give consent on computers outside the home, I'd consider them not to be protected speech.
  - deleted 27d ago
    [deleted]
- steve_adams_86 27d ago
  > Considering the amount of ads they handle that seems... reasonable? Why reasonable? Don't these ads pass through pre-publication review, and Meta makes money off of it? Any number above zero is a serious problem. If they can't run an ad business that doesn't publish and profit off of CSAM, they shouldn't run it. I don't think we should distinguish between 'fake' and real CSAM based on if kids are hurt; this ad machine shouldn't be able to publish it, regardless. The moral question of if it's real or kids were directly harmed seems separate to me.
hollow-moe 27d ago
"Not sorry, it will happen again." If only they were as liable for this kind of stuff as they are for DMCA'ble content lmao
beart 27d ago
Presumably these ads are targeted using the algorithm. Thought crime is scary and all that, but it's an interesting part of this system to consider. And then there's the question of these images living in your cache if you happened to be shown one. "Welcome to the internet.."
- mhitza 27d ago
  The "Kompromat" angle should not be ignored in this day and age. "In cases of kompromat during the early 21st century, Russian operatives have been suspected or accused of placing child pornography on the personal computers of individuals they were attempting to discredit." https://en.wikipedia.org/wiki/Kompromat https://en.wikipedia.org/wiki/Kompromat The only real question is how easy it would be to target the undesired via the ad placement targetting options on social media platforms. On the one on which I've run ads (LinkedIn), pretty hard. At least on a technical level, because if your audience size is too small it asks you to expand your criteria. Maybe other platforms are less strict? For sure would be a more comfortable approach for state operatives than physical access. -- Aside. Watch the movie with the same name. Use an ad blocker, religiously.
  - gfrecvh 27d ago
    Do you mean this? https://imdb.com/title/tt13349892/ https://imdb.com/title/tt13349892/
    - mhitza 27d ago
      Yes that is the one. I'm surprised by the low rating on IMDb. Well produced movie, with an interesting real-life inspired premise, a bit of thriller.
- FireBeyond 27d ago
  The only things I can think of that have pushed these into my viewport are that I follow multiple AI news pages, and multiple photography pages, same with my IG.
- bigfatkitten 27d ago
FireBeyond 27d ago
To be really, really fucking clear, it's not just "failed to catch" it's, "even when reported, failed to do anything about". I got ads, reels that were not-at-all veiled hints at CSAM. "Report: CSAM" "We reviewed your report and determined that the content does not breach our policy."
- vlachen 27d ago
  That's the rub. And I think that they've found my "well, fuck this" point. I already don't frequent their products, but I think it's time to disentangle what I do have on there from other services so I can download and delete.

75 points by Bender 27d ago

> In captions, ads enticed Meta users to click with claims like “I can show you more” or promises that “our AI transforms your hidden desires into hypnotic videos. Dark, seductive, unforgettable.” Another boasted, “there are no restrictions. All the characters are real people and there are many options. This is the AI men actually use.” > Even more troublingly, TTP emphasized in a press release that these findings came after the group teamed up with Wired and flagged an earlier batch of 53 ads in August. At that time, Meta claimed that it had “recently rolled out new systems to better detect child exploitation.” But TTP “found hundreds of new CSAM ads ran after Meta boasted about its new technology.” Prison is too good for the people responsible for this.

FireBeyond 27d ago
Garbage, too. I've reported at least 10 of these things. Ads, reels. Specifically as child abuse. "After review, we determined this content does not breach our policies."
- mitxela 27d ago
  Of course. No large company cares about child abuse. They've all been to Epstein island, or one of the other unknown islands.
  - FireBeyond 27d ago
    Hah, I don't know if Zuck has visited, but if he or any of the exec team has, you can bet that filter is finely finely tuned.
- peterashford 27d ago
  Expecting a corporate to police itself is, obviously, a joke

bsoqk 27d ago

>the Tech Transparency Project (TTP) reported that Meta failed to detect 332 ads containing CSAM this year Considering the amount of ads they handle that seems... reasonable? Not to mention the stupidity of calling the result of nudify apps "CSAM" when no minors in situation of sexual abuse are used in any way to generate those photos.

washadjeffmad 27d ago
This is one of those topics that gets radioactive quickly, so I'm trying to parse this comment under the recent ruling by a judge that certain obscene products of generative AI are protected under the First Amendment. "We now live in an age where GenAI [generative AI] models can render images depicting the abuse of virtual children that are virtually indistinguishable from those depicting the abuse of actual children. This case illuminates how this evolving technology complicates the lines drawn by the Supreme Court in Stanley, Osborne, and Free Speech Coalition. Indeed, in Free Speech Coalition, the Supreme Court addressed the scope of First Amendment protections for virtual CSAM, but that was nearly twenty-five years ago, and the image-generation technology available today was likely unimaginable back then. Given the relentless advancement in artificial intelligence models, we have some concerns about the lines these cases draw, but we are not free to redraw them ourselves." https://www.snopes.com/fact-check/judge-ai-child-sex-abuse-ruling/ https://www.snopes.com/fact-check/judge-ai-child-sex-abuse-r... Despite the source images not being CSAM, the products of nudify services may be CSAM but not child pornography. Because nudifying minors involves generation of material intended to depict real people who cannot give consent on computers outside the home, I'd consider them not to be protected speech.
- deleted 27d ago
  [deleted]
steve_adams_86 27d ago
> Considering the amount of ads they handle that seems... reasonable? Why reasonable? Don't these ads pass through pre-publication review, and Meta makes money off of it? Any number above zero is a serious problem. If they can't run an ad business that doesn't publish and profit off of CSAM, they shouldn't run it. I don't think we should distinguish between 'fake' and real CSAM based on if kids are hurt; this ad machine shouldn't be able to publish it, regardless. The moral question of if it's real or kids were directly harmed seems separate to me.

hollow-moe 27d ago

"Not sorry, it will happen again." If only they were as liable for this kind of stuff as they are for DMCA'ble content lmao

beart 27d ago

Presumably these ads are targeted using the algorithm. Thought crime is scary and all that, but it's an interesting part of this system to consider. And then there's the question of these images living in your cache if you happened to be shown one. "Welcome to the internet.."

mhitza 27d ago
The "Kompromat" angle should not be ignored in this day and age. "In cases of kompromat during the early 21st century, Russian operatives have been suspected or accused of placing child pornography on the personal computers of individuals they were attempting to discredit." https://en.wikipedia.org/wiki/Kompromat https://en.wikipedia.org/wiki/Kompromat The only real question is how easy it would be to target the undesired via the ad placement targetting options on social media platforms. On the one on which I've run ads (LinkedIn), pretty hard. At least on a technical level, because if your audience size is too small it asks you to expand your criteria. Maybe other platforms are less strict? For sure would be a more comfortable approach for state operatives than physical access. -- Aside. Watch the movie with the same name. Use an ad blocker, religiously.
- gfrecvh 27d ago
  Do you mean this? https://imdb.com/title/tt13349892/ https://imdb.com/title/tt13349892/
  - mhitza 27d ago
    Yes that is the one. I'm surprised by the low rating on IMDb. Well produced movie, with an interesting real-life inspired premise, a bit of thriller.
FireBeyond 27d ago
The only things I can think of that have pushed these into my viewport are that I follow multiple AI news pages, and multiple photography pages, same with my IG.
bigfatkitten 27d ago

FireBeyond 27d ago

To be really, really fucking clear, it's not just "failed to catch" it's, "even when reported, failed to do anything about". I got ads, reels that were not-at-all veiled hints at CSAM. "Report: CSAM" "We reviewed your report and determined that the content does not breach our policy."

vlachen 27d ago
That's the rub. And I think that they've found my "well, fuck this" point. I already don't frequent their products, but I think it's time to disentangle what I do have on there from other services so I can download and delete.