7 ms·

Meta Ran Ads That Contained AI-Generated Child Sexual Abuse Imagery

327 points by malshe 2mo ago

malixp 2mo ago
[dead]
stephenhuey 2mo ago
Non-paywall alternative: https://www.engadget.com/2231100/meta-apps-displayed-ads-that-contained-ai-generated-csam/ https://www.engadget.com/2231100/meta-apps-displayed-ads-tha...
snitzr 2mo ago
Shut it all down.
steveBK123 2mo ago
[flagged]
- antonvs 2mo ago
  [flagged]
ToucanLoucan 2mo ago
> but as Wired notes, it relies on "automated tools" as part of the process, which may have been how the more than 50 ads TTP discovered made it through. Again, and again, and again, these technologies have proven ineffective to moderate content. User posted and otherwise. Until we have a legal mechanism in place to hold these companies meaningfully liable for this, it will not stop. Every single one of these has the same damn list of excuses about cost of hiring human moderators, but I promise everyone reading this if you make the executives of a social network legally liable for distributing CSAM on their platforms, they will find all kinds of money to get human moderators really fucking quick.
- duped 2mo ago
  But they couldn't possibly do it at that the scale they're operating at. They couldn't hire enough people in the countries they operate, or know the language, or trust that who they do hire aren't corrupt or working against their interest. And they can't manually approve every ad, or make sure targeting isn't illegal or otherwise harmful. --- This is sarcasm, by the way. Most decent people make sure what they're about to do won't harm other people and if would cause harm they wouldn't do it. But social media companies and ad networks aren't run by decent people. They're run by people who believe their business is entitled to infinite scale regardless of how many people are harmed along the way.
  - john_strinlai 2mo ago
    >But they couldn't possibly do it at that the scale they're operating at. They couldn't hire enough people in the countries they operate, or know the language, or trust that who they do hire aren't corrupt or working against their interest. And they can't manually approve every ad, or make sure targeting isn't illegal or otherwise harmful. you say sarcasm, but i think this is literally true. this isn't a defense of facebook, i think that their scale makes it actually impossible (while remaining a going concern). the result should be that they arent allowed to operate at a scale that they cant handle, instead of being able to brush it under the rug every ~6 months when an incident hits the news cycle.
    - lokar 2mo ago
      Don’t they have like an 80% gross margin?
      - john_strinlai 2mo ago
        the amount of content posted per second to facebook is unfathomable. a quick google (just for a rough order of magnitude) says ~65,000 posts per second. add in instagram (which is worse for CSAM) and its gotta be at least double that. it would require millions on millions of employees, spread across ~every country. between the payroll, buildings and property, equipment, etc. i just dont think its possible at that scale. (again, emphasizing that im not saying this as an excuse for facebook, but that i dont really think facebook/instagram should exist at all)
picofarad 2mo ago
36 million CSAM media removed in 2025? Why aren't millions of people in jail? Wtf?
- ceejayoz 2mo ago
  VPNs, hacked accounts, no jurisdiction, etc., I'd presume.
  - catlikesshrimp 2mo ago
    [flagged]
    - ceejayoz 2mo ago
      No, it's the "local law enforcement struggles to handle largely anonymous international crimes" note. I'm not arguing anything; I'm just describing the reality.
- john_strinlai 2mo ago
  it can take awhile for things to work their way from NCMEC (national center for missing & exploited children) to the appropriate local agency, then have that agency do their investigation, get a warrant (if they can), etc. and that is assuming that there is even a cooperative jurisdiction for NCMEC to send the initial report to. but, plenty of people do end up in prison, especially if its facebook/dropbox/gmail/etc... they just dont make the headlines (especially nationally) unless its especially heinous.
  - zimpenfish 2mo ago
    I think there's also the "roll up a whole network at once" approach they prefer to take for things like this - hence nothing will happen for months/years whilst investigations happen in secret and then there'll be a day where you get 40 coordinated raids across 10 countries which lead to tens of people being arrested. Then of course there's another period of months/years where they have to work on the new stuff they've got from that raid and also preparing for trials, etc.
alex1138 2mo ago
(This reminds me of, and not to be political. This is a technological, rather than political angle) There's a TED talk by Carole Cadwalladr about "Facebook's role in Brexit" in which she talks about the ads the Leave campaign ran. This is what is meant by Facebook's "breaking democracy", where campaigns don't have to go through the usual disclosure avenues Are there reasons why Brexit might have been popular? Yes, in the sense I've heard about them. Should we then be blaming Facebook? The problem is consolidated power. Carole herself said in her talk that it's not about Trump vs no Trump (sorry. It's been a bit since I've seen the talk. I shouldn't just paraphrase), it's about citizens' right to meaningfully choose in an information landscape This is the problem with Facebook. It becomes an Everything App. Who cares about fraud (say, on marketplace, needless to say the site itself, the way it's screwed over businesses) on the platform when you can grow big? Let's say in this case, the actual case I'm commenting on - "Facebook didn't know". Well then that's the problem, isn't it? It's a company set up by someone of low moral reasoning (https://news.ycombinator.com/item?id=1692122 https://news.ycombinator.com/item?id=1692122) that doesn't set out to make a good product, something people will actually like. It's about dominating an industry and ads can be totally unvetted or scummy or scammy if it enables them to do this
CrzyLngPwd 2mo ago
All their fines are merely a cost of doing business, and they won't change their ways until the fines hurt.
- burningChrome 2mo ago
  This is so true. When you have a billion users, you can literally lose hundreds of millions of users and it wouldn't make a dent. The only way is to get the government involved and regulate this and just keep hammering them with massive fines that really make a dent in their bottom line. For a lot of people, government regulation of this sort is not desirable, but at what point does someone need to drop the hammer to stop this?
  - ASalazarMX 2mo ago
    I guess you're used to standard billions (a million millions). A US billion would certainly suffer from losing hundreds of millions, you're talking about a double-digit percent.
    - anamexis 2mo ago
      I doubt they are saying any services out there have 1,000,000,000,000 users.
    - yetanotherjosh 2mo ago
      10^9 is the standard definition of a billion, it's very wrong to call 10^12 a "standard billion." You can say "long scale billion" if you want to refer to the definitively non-standard 10^12 billion.
      - Terr_ 2mo ago
        To quibble/add-nuance: The words have diverged into a false-friend [0] situation. It looks like the same word, but the correct/standard meaning depends on which language is being used at the time. If someone is writing English and puts "billion" for 10^12, that's a translation error, much like writing "my grandfather became pregnant when he realized he'd locked the keys inside the car." (Instead of "embarrassed.") Conversely, if I wrote a French or German sentence and put "billion" for 10^9, then I'd be the one making the translation mistake. [0] https://en.wikipedia.org/wiki/False_friend https://en.wikipedia.org/wiki/False_friend
daun_gee 2mo ago
[flagged]
- giancarlostoro 2mo ago
  So according to you 60% of Americans are all pedophiles and “retarded” because they use AI. Glad to have such constructive discourse with you bud.
  - Retric 2mo ago
    Saying X if for A and B, implies a non exclusive logical or. Income is for daily necessities and long term investment.
josefritzishere 2mo ago
[flagged]
- dylan604 2mo ago
  So you throw theZuck in the hoosegow. Now what? FB will continue printing money and doing the things it always has even if under new management. Why do you think it'll change with just throw theZuck in jail?
  - bastardoperator 2mo ago
    Court monitor that ensures compliance. It's not that hard.
  - mywittyname 2mo ago
    Presumably the new CEO would like to stay out of jail and will fix things?
  - lenerdenator 2mo ago
    Because at that point, the last guy to run a company that displayed gross negligence to CSAM distribution is sitting in prison. That serves as an example, a deterrent. "Clean up the act, or you too can go from being a hyper-wealthy executive to the biohazard cleanup detail in a cell block at Leavenworth."
malshe 2mo ago
https://archive.ph/4LhoU https://archive.ph/4LhoU I didn't realize the original was behind a paywall.
lp4v4n 2mo ago
I'd be interested to know if the same amount of leniency shown here would be applied if we were talking about ads depicting violence against elected politicians.
- shawn_w 2mo ago
  Depends on the politician.
- danudey 2mo ago
  [flagged]
- inigyou 2mo ago
  Only if they're LW.
type0 2mo ago
So that's why they are lobbying for age verification, i.e ID verification?
- dathinab 2mo ago
  no age verification is not ID verification through meta might have lobbied on both but they lobby for specific (often pretty bad) forms of age verification to be able to - spy on people better - have a regulatory moat where due to beady designed laws new alternatives have a much higher hurdle/initial investments and in turn never get anywhere
- mplewis 2mo ago
  Meta is lobbying for age verification because it helps provide an institutional moat for them against the open web.
  - barbazoo 2mo ago
    Could you elaborate? I thought Meta wants to put the burden onto the platform, i.e. Apple and Google, is what they are lobbying for. https://pluribusnews.com/news-and-events/meta-lobbies-for-app-store-age-verification-laws/ https://pluribusnews.com/news-and-events/meta-lobbies-for-ap... What is the institutional moat?
dotcoma 2mo ago
Classy as usual.
alex1138 2mo ago
> and ran across Facebook, Threads, Messenger and Instagram. So it's the SAME AD PLATFORM? Even in MESSENGER? My god. Not only is this whole thing horrible but it's doubly so
Kalanos 2mo ago
What's crazy is how long it takes to report something like this to a big company and actually have them take action on it. I remember 10 years ago I reported a Giphy ad for the same thing, and it took them months to take action.
- bombcar 2mo ago
  Meta doesn't have to give a shit, because they're huge. Tiny providers have to scramble, because they'll be destroyed by a single failure to handle it fast enough.
- jhartikainen 2mo ago
  Out of curiosity, I've ran some experiments on Facebook for stuff like this. Results: - Reporting a message does nothing no matter how much the content breaks their terms of use. 100% of messages I reported bounced back in a matter of minutes with "no action" - However, using the "request another review" function, which is available after the initial "no action" response, actually seems to work. How quickly this works seems to vary, I've had actions taken as result of this from 1 day to multiple weeks.
  - gs17 2mo ago
    Interesting, I've kind of made a hobby of reporting scams and hate speech and watching it take a week to come back with "it's no violation", requesting another and getting the same response in another week. I've never seen them do anything no matter how obvious or awful it is.
    - inigyou 2mo ago
      You have to actually, honestly, fall for the scam and lose all your money and only then will you have standing to sue Meta for making you lose all your money. You won't be able to afford to sue them, of course.
      - cindyllm 2mo ago
deleted 2mo ago
[deleted]
deleted 2mo ago
[deleted]
godwinson__4-8 2mo ago
Somewhat related: open YouTube in an incognito window. Search something rather innocuous like "news". Click the first thing then go back to the main page. You have to watch through the ad. The amount of slop you will get suggested as soon as the algorithm warms up is nuts. The ad you are first served might also be slop. I mean AI generated video advertising overt bogus cures (a memorable one recently - "did you know the Bible has the answer to Alzheimer's?") over AI slop footage. When I compare to my normal/auth'd Youtube experience, it feels like a totally different app. I have never really experienced such a cheap illustration of the "Youtube rabbit hole" effect. The degree to which AI authored advertising is participating in this is imo pretty concerning if you have, in particular, older parents or relatives.
- fragmede 2mo ago
  By the by, if you watch a video while logged in that someone sent you as being batshit crazy, and it's affecting your recommendations, you can go into your watch history and delete it from there so you don't get recommended similar videos.
- VCFundedGenYer 2mo ago
  Youtube is pushing a lot of AI features without creators' consent. Some of it can be turned off but others cannot. It's becoming a major problem. Can't wait till they lose so much money on the infrastructure that they have to remove it all.
  - bilegeek 2mo ago
    The auto-translation especially bothers me. Kinda cool for casual viewing... but the inability to disable it gives me an immediate skeptical "this entire channel is probably AI slop" response before I recognize what's happened.
- carljungslabtek 2mo ago
  Part of me would like to stop paying for a youtube premium family subscription because they keep raising the price, but my mom watches it a lot and I don’t want her to see (or have to see) those ads. They’re so unbelievably bad. It can’t be good for your brain to watch, the few times I’ve seen them on my work computer I can’t help but watch part of them.
lenerdenator 2mo ago
This is what happens when you let a company be controlled by one person who has no reason to be accountable to anyone. If Zuckerberg wants total voting control, then he can also deal with the consequences of being caught hosting CSAM.
- olelele 2mo ago
  Digital feudalism. Musk is building a city where he can prohibit worker organizing. Joe Hill died for nothing.
froobius 2mo ago
I'm starting to wonder if any of this is better than local newspapers, where there's an editor with oversight over all content...
meric_ 2mo ago
I'm wondering what people expect these companies to do? Human moderators? Sure, but Meta products have approximately half the worlds population using them. At this scale it's pretty mind bogglingly difficult to moderate down everything. The ads/posts served per second is an astronomical number. This truly is a case where AI is the solution. There's no way to automate this at scale with humans. As AI gets better hopefully detection and moderation will catch up
- deleted 2mo ago
  [deleted]
- spacecadet404 2mo ago
  AI is generating ads that will be reviewed by AI that will then be pushed to humans largely to ignore. Ad pipeline is malware at this point.
  - techjamie 2mo ago
    > Ad pipeline is malware at this point. Always has been. Occasionally my girlfriend will comment on something she saw in an ad, even buy things off them. Thing is though that ad platforms don't seem to be doing anything meaningful to stop bad actors. So really, clicking any ad and interacting with it amounts to placing your trust in a total stranger with the only criteria being they paid a little bit of money for you to see them. To which you will then download their software, give them your credit card, or whatever the end goal is. It's far easier to assume anything in an advertisement is a scam and block them accordingly.
- dijksterhuis 2mo ago
  It's not everyone else's fault that executives etc. decided to grow their userbase/business/profits at all costs, i.e. with no concerns for the harms they've been causing. It's similar to how adding more and more code, more and more features, more and more stuff into software increases technical debt. Apparently "growth at all costs" for a business equates to ethical/moral debt, quite obviously in this case. Which shouldn't be surprising to anyone who understands/recognises the mechanisms of greed. For software, sometimes paying off that debt means getting rid of some of your features/code/stuff. i.e. there's another alternative to "let's AI the AI". scale the business down. shrink the number of users. make less money.
xgulfie 2mo ago
Weird how they never thought to require ID verification for advertisers hmmm
- woodrowbarlow 2mo ago
  for real. if you want to make the web safer for children, start by holding ad networks to a higher accountability. it's painfully obvious "protect the children" is not the real goal.
bigbuppo 2mo ago
As if you didn't need another reason to use an ad blocker.
- qwerpy 2mo ago
  Meta is better than anyone at anti-ad-blocking. Even the best ad blockers struggle to keep ahead of the ever-changing structure served up by their websites, and the websites are intentionally bad to make you use the apps instead. But the apps are nearly immune to ad-blocking. “Just stop using meta properties then”. Sure but some of us have wives, friends, etc
craftkiller 2mo ago
I've noticed on YouTube I see a lot of adult sexual ads. Things like home cleaning ads that have an unmistakable wood-grain image of fellatio. I've always wondered "how are these getting past the moderators?". I guess the answer is the obvious one: no one is moderating anything.
- ihuk 2mo ago
  Reported an obvious sexual ad the other day and got a very AI generated response that started with "Dear Username" stating that they reviewed the ad and found nothing wrong with it.
  - p0w3n3d 2mo ago
    The Dead Internet Theory is real!
  - unsnap_biceps 2mo ago
    Before I gave up on Facebook entirely, I reported an ad that was an explicit blow job and received a similar response and this was before AI was huge. I really don't think they've ever cared.
    - mullingitover 2mo ago
      Facebook literally trolls you when you report criminal activity, rampant well-documented scams, on their site. They write back and tell you it's not violating their policy and ask if you want to support for your personal problem.
      - Spooky23 2mo ago
        Absolutely. My uncle got taken by an “investment club” supposedly endorsed by Warren Buffet and run day to day by some real financial person. All communications were via WhatsApp. They started with real stock tips (which were pretty good actually!) and then did a pump and dump. See: https://www.cleuremissionfund.com/ https://www.cleuremissionfund.com/ I received a similar ad (mine was from the shark tank people), reported it and got a message similar to what you described. They were obviously stealing Mark Cuban’s likeness and peddling an obvious scam.
chrisjj 2mo ago
> AI-generated How on earth can they know?
- SV_BubbleTime 2mo ago
  Read the article. Between the lines it doesn’t seem like the real issue is any specific image, it’s the ”nudify” AI apps. This is why they pushed for the change from “child porn” to “child sexual assault material” because LOTS OF THINGS can be “materials”.
  - chrisjj 2mo ago
    I did read the article. It does not answer the question.
andy_ppp 2mo ago
If any of us did that clearly we’d be in jail for several years and put on the sex offenders register but when Meta appear to be hugely responsible for increases in teen suicide or crimes like these we get nothing. If corporations want the rights of individuals they equally should shoulder our shared societal responsibilities as well. I honestly don’t get how they can just pretend they aren’t responsible somehow because it’s slightly expensive to not commit crimes or something… If you are downvoting this please explain why, it’s important that we come to a shared understanding how these things are ever okay for corporations and not individuals…
- archildress 2mo ago
  Our regulatory framework is so asleep at the wheel. It is as if there is this unpierceable veil if you operate as a corporate entity, as if multiple people's combined transgressions are less than a single person doing it.
  - olelele 2mo ago
    Move fast and break literally everything. I am so depressed by what the internet has become. Such potential only to become monopolist behemoth bullshit.
  - carljungslabtek 2mo ago
    Is it really asleep at the wheel or is it that decades of intense lobbying have crippled it? I mean of course it’s both, but I think it’s important to point out that it’s not just negligence, it’s more nefarious and selfish than that.
    - Telaneo 2mo ago
      It's in an induced coma at the wheel.
- cucumber3732842 2mo ago
  >If any of us did that... You don't have Metas legal team nor does the power structure find your valuable to have helping them out nor can you harm them if you don't like them. There was, quite famously, a man who got away with these sorts of things for quite some time. He owned an island. I can't recall his name....
shitloadofbooks 2mo ago
Meta will do whatever it can to run any and all ads. Meta knowingly runs scam ads, they just charge more to run them: https://www.reuters.com/legal/litigation/consumer-watchdog-group-files-suit-alleging-meta-profited-ads-scams-2026-04-21/ https://www.reuters.com/legal/litigation/consumer-watchdog-g... > Reuters also reported that Meta internally projected in 2024 that it would earn about 10% of its overall annual revenue – or $16 billion – from running ads for scams and banned goods, internal company documents show. At $employer we're CONSTANTLY fighting fake ads cloning our brand. Their various brand protection features seem deliberately inept and ineffective.
disillusioned 2mo ago
Beyond these obviously deplorable ads, my wife got sniped by an ecommerce lookalike ad on Google the other day: a near-identical match of the vendor she was looking for, with a paid for sponsored ad (that was clearly linked to some chinese firm upon further review). Site had prices that were just a bit too good to be true, and a slightly different TLD that _I_ would've caught, but she missed because the price was a bit too good to be true. Only caught it when Citi asked me if I was spending 9000EGD on some random website. And my point here is, we're in this great revolution of AI empowered everything and... these companies can't be bothered to scan this volume of AI ad slop to make sure we're not being actively scammed? Or having porn or CSAM spread? I understand it's a "perverse incentive" in the truest sense: they make money by taking the scammers money, but I was scammed once by an insanely clever lookalike Facebook ad, and had trusted their verified checkmark (about a decade ago)... with AI, these things should be a thing of the past.
onlyrealcuzzo 2mo ago
Guys, it's fine. Relax. It's only a crime if someone else does it or is not sufficiently wealthy, otherwise it's a mistake.
- _carbyau_ 2mo ago
  I believe that is referred to as: organised crime.
  - tremon 2mo ago
    Rather, this is institutional crime. Organised crime is when crime operates its own control structures in defiance of government enforcement; this is the government actively allowing the crime to proliferate.
    - _carbyau_ 2mo ago
      Organised crime involves the corruption of institutions too - specifically to allow getting away with it. But I like your take on it as institutionalised crime because it is now publicly overt. The corruption is complete.
yalogin 2mo ago
Even if they are caught and it’s illegal, as long as the fine for it is less than the roi of running them the companies will not stop.
callamdelaney 2mo ago
Meta is still running ads that suggest violence against politicians, when reported they say it does not contravene their policies.
- wredcoll 2mo ago
  Politicians don't count.
  - inigyou 2mo ago
    LW politicians don't count. If there was an ad calling for violence against an RW politician it'd be taken down and the poster would go to jail.
    - callamdelaney 2mo ago
      Actually it’s for a right wing politician and there are many variations.
cdrnsf 2mo ago
We were better off with landlines and local journalism.
randyrand 2mo ago
Non-specific terms are a pet peeve of mine in accusations. Is this new term broader than the old term of CP or is it the same? Why did we even switch?
- tbrownaw 2mo ago
  > Is this new term broader than the old term of CP or is it the same? Why did we even switch? The purpose of the new term was to emphasize that producing it involves the actual abuse of an actual child. And then somehow it got applied to drawings and lately computer-generated stuff.
  - inigyou 2mo ago
    The purpose of the new term was virtue signalling by suggesting that it's somehow outside of the definition of the word "porn".
- bluefirebrand 2mo ago
  > Why did we even switch? If I had to guess, this is to try and distance child sexual assault from the concept of "pornography", and maybe make it more abundantly clear that "child porn" is always, 100% of the time, sexual abuse. I dunno, language evolves like this sometimes. Like "smoking" which is now called "tobacco use" or "cannabis use" to clarify more what is being smoked
crmd 2mo ago
I asked my grandpa about the business idea of self-serve magazine advertising. He said if you do that wait till you see the weird shit people will try to publish. I wonder if we could learn anything from these cranky old timers.
eboy 2mo ago
[dead]
throwatdem12311 2mo ago
Just shut the company down, throw the executives in a deep dark hole and forget they exist. The world will be better off.
2OEH8eoCRo0 2mo ago
Repeal Section 230
thi2 2mo ago
Metas ads are all over the place. Thomas Kehl, one of the founders of "Finanzfluss" (a german site/content creation company about finance) is currently suing meta because scammers keep using his picture. There really needs to be some harder punishments for this kind of behavior, after all meta profits by selling scammy ads.
xhevahir 2mo ago
If stuff like this shows up in GAI I wonder what that means for how common such material is in the wild. Frightening.
ticulatedspline 2mo ago
Personally I find these kinds of things fascinating. Partially for the practical problem it represents and it's facets but also the one sided backlash of it all. Moderation is an interesting conglomerate of so many factors. First of all I don't think Meta is a good or honest company, but that doesn't mean they're not trying at all. In reality even if they were good and honest I genuinely believe their task is probably nearly impossible. Humans just suck too much. It certainly doesn't help that Meta has at least some perverse incentive to be permissive with ads, as they get money from them. Though Meta execs aren't stupid. Inflammatory political ads are one thing, kiddie porn is another I suspect they lose more money over CSAM than all the ad revenue from them. I believe them when they say they removed over 36 million pieces of child sexual exploitation content last year. Which is a number that I should let sink in. Basically more than one piece of CSAM shows up in their ad stream every second of every day all year. If their moderation techniques are 99% effective that results in 360,000 CSAM ads leaking to the public annually. Another interesting thought is how much would it cost to actually get that rate to 100%? let's assume human moderators aren't paid much but are infallible. That all-in a single moderator employee costs $65K , that's salary, benefits, HR ratio, psychological therapy, equipment, the works, all factored in. let's say on average you get about 1 ad review ~every 2 minutes, so bout 250 ads per-day per-employee factoring in some time off you actually get about 65,000 reviews a year. So bout $1 per review. So the question begs how many distinct ads enter their system each year? which I haven't found a stat for. But assuming only 1 in 1000 ads are CSAM, which still feels high but is probably a reasonable ratio. we get 36 billion ads needing review annually. At a dollar each 23% of their gross annual ad revenue each year would be just moderation overhead and they would need to hire over 550,000 moderators. Evil or not I think they organically built a thing that operates at an unimaginable scale, that even moderation with failures in the parts-per-million would result in tens of thousands of unsavory or unscrupulous ads reaching users and they have neither the incentive nor the capacity to reach even that level of accuracy.
- happymellon 2mo ago
  Disagree with essentially all of this. To get a Facebook account they require all sorts of PII. Why not have people give their PII to set up an advertising account and if they distribute CSAM then they are banned. Removing access to WhatsApp will stop a large portion of the world from participating. And the parts that can't? Well, they can't advertise on Facebook/Instagram anymore.
inigyou 2mo ago
Fork found in kitchen.
blks 2mo ago
Good example why not to post your children faces online
eayomi 2mo ago
[flagged]
pickleglitch 2mo ago
Zuckerberg should thrown in prison.
ajdude 2mo ago
I reported a blatantly sexual adult ad on Facebook the other day, they got back to me and said that their moderators reviewed the ad and that it was appropriate.

327 points by malshe 2mo ago

[dead]

Non-paywall alternative: https://www.engadget.com/2231100/meta-apps-displayed-ads-that-contained-ai-generated-csam/ https://www.engadget.com/2231100/meta-apps-displayed-ads-tha...

snitzr 2mo ago

Shut it all down.

steveBK123 2mo ago

[flagged]

antonvs 2mo ago
[flagged]

ToucanLoucan 2mo ago

> but as Wired notes, it relies on "automated tools" as part of the process, which may have been how the more than 50 ads TTP discovered made it through. Again, and again, and again, these technologies have proven ineffective to moderate content. User posted and otherwise. Until we have a legal mechanism in place to hold these companies meaningfully liable for this, it will not stop. Every single one of these has the same damn list of excuses about cost of hiring human moderators, but I promise everyone reading this if you make the executives of a social network legally liable for distributing CSAM on their platforms, they will find all kinds of money to get human moderators really fucking quick.

duped 2mo ago
But they couldn't possibly do it at that the scale they're operating at. They couldn't hire enough people in the countries they operate, or know the language, or trust that who they do hire aren't corrupt or working against their interest. And they can't manually approve every ad, or make sure targeting isn't illegal or otherwise harmful. --- This is sarcasm, by the way. Most decent people make sure what they're about to do won't harm other people and if would cause harm they wouldn't do it. But social media companies and ad networks aren't run by decent people. They're run by people who believe their business is entitled to infinite scale regardless of how many people are harmed along the way.
- john_strinlai 2mo ago
  >But they couldn't possibly do it at that the scale they're operating at. They couldn't hire enough people in the countries they operate, or know the language, or trust that who they do hire aren't corrupt or working against their interest. And they can't manually approve every ad, or make sure targeting isn't illegal or otherwise harmful. you say sarcasm, but i think this is literally true. this isn't a defense of facebook, i think that their scale makes it actually impossible (while remaining a going concern). the result should be that they arent allowed to operate at a scale that they cant handle, instead of being able to brush it under the rug every ~6 months when an incident hits the news cycle.
  - lokar 2mo ago
    Don’t they have like an 80% gross margin?
    - john_strinlai 2mo ago
      the amount of content posted per second to facebook is unfathomable. a quick google (just for a rough order of magnitude) says ~65,000 posts per second. add in instagram (which is worse for CSAM) and its gotta be at least double that. it would require millions on millions of employees, spread across ~every country. between the payroll, buildings and property, equipment, etc. i just dont think its possible at that scale. (again, emphasizing that im not saying this as an excuse for facebook, but that i dont really think facebook/instagram should exist at all)

picofarad 2mo ago

36 million CSAM media removed in 2025? Why aren't millions of people in jail? Wtf?

ceejayoz 2mo ago
VPNs, hacked accounts, no jurisdiction, etc., I'd presume.
- catlikesshrimp 2mo ago
  [flagged]
  - ceejayoz 2mo ago
    No, it's the "local law enforcement struggles to handle largely anonymous international crimes" note. I'm not arguing anything; I'm just describing the reality.
john_strinlai 2mo ago
it can take awhile for things to work their way from NCMEC (national center for missing & exploited children) to the appropriate local agency, then have that agency do their investigation, get a warrant (if they can), etc. and that is assuming that there is even a cooperative jurisdiction for NCMEC to send the initial report to. but, plenty of people do end up in prison, especially if its facebook/dropbox/gmail/etc... they just dont make the headlines (especially nationally) unless its especially heinous.
- zimpenfish 2mo ago
  I think there's also the "roll up a whole network at once" approach they prefer to take for things like this - hence nothing will happen for months/years whilst investigations happen in secret and then there'll be a day where you get 40 coordinated raids across 10 countries which lead to tens of people being arrested. Then of course there's another period of months/years where they have to work on the new stuff they've got from that raid and also preparing for trials, etc.

alex1138 2mo ago

(This reminds me of, and not to be political. This is a technological, rather than political angle) There's a TED talk by Carole Cadwalladr about "Facebook's role in Brexit" in which she talks about the ads the Leave campaign ran. This is what is meant by Facebook's "breaking democracy", where campaigns don't have to go through the usual disclosure avenues Are there reasons why Brexit might have been popular? Yes, in the sense I've heard about them. Should we then be blaming Facebook? The problem is consolidated power. Carole herself said in her talk that it's not about Trump vs no Trump (sorry. It's been a bit since I've seen the talk. I shouldn't just paraphrase), it's about citizens' right to meaningfully choose in an information landscape This is the problem with Facebook. It becomes an Everything App. Who cares about fraud (say, on marketplace, needless to say the site itself, the way it's screwed over businesses) on the platform when you can grow big? Let's say in this case, the actual case I'm commenting on - "Facebook didn't know". Well then that's the problem, isn't it? It's a company set up by someone of low moral reasoning (https://news.ycombinator.com/item?id=1692122 https://news.ycombinator.com/item?id=1692122) that doesn't set out to make a good product, something people will actually like. It's about dominating an industry and ads can be totally unvetted or scummy or scammy if it enables them to do this

CrzyLngPwd 2mo ago

All their fines are merely a cost of doing business, and they won't change their ways until the fines hurt.

burningChrome 2mo ago
This is so true. When you have a billion users, you can literally lose hundreds of millions of users and it wouldn't make a dent. The only way is to get the government involved and regulate this and just keep hammering them with massive fines that really make a dent in their bottom line. For a lot of people, government regulation of this sort is not desirable, but at what point does someone need to drop the hammer to stop this?
- ASalazarMX 2mo ago
  I guess you're used to standard billions (a million millions). A US billion would certainly suffer from losing hundreds of millions, you're talking about a double-digit percent.
  - anamexis 2mo ago
    I doubt they are saying any services out there have 1,000,000,000,000 users.
  - yetanotherjosh 2mo ago
    10^9 is the standard definition of a billion, it's very wrong to call 10^12 a "standard billion." You can say "long scale billion" if you want to refer to the definitively non-standard 10^12 billion.
    - Terr_ 2mo ago
      To quibble/add-nuance: The words have diverged into a false-friend [0] situation. It looks like the same word, but the correct/standard meaning depends on which language is being used at the time. If someone is writing English and puts "billion" for 10^12, that's a translation error, much like writing "my grandfather became pregnant when he realized he'd locked the keys inside the car." (Instead of "embarrassed.") Conversely, if I wrote a French or German sentence and put "billion" for 10^9, then I'd be the one making the translation mistake. [0] https://en.wikipedia.org/wiki/False_friend https://en.wikipedia.org/wiki/False_friend

daun_gee 2mo ago

[flagged]

giancarlostoro 2mo ago
So according to you 60% of Americans are all pedophiles and “retarded” because they use AI. Glad to have such constructive discourse with you bud.
- Retric 2mo ago
  Saying X if for A and B, implies a non exclusive logical or. Income is for daily necessities and long term investment.

josefritzishere 2mo ago

[flagged]

dylan604 2mo ago
So you throw theZuck in the hoosegow. Now what? FB will continue printing money and doing the things it always has even if under new management. Why do you think it'll change with just throw theZuck in jail?
- bastardoperator 2mo ago
  Court monitor that ensures compliance. It's not that hard.
- mywittyname 2mo ago
  Presumably the new CEO would like to stay out of jail and will fix things?
- lenerdenator 2mo ago
  Because at that point, the last guy to run a company that displayed gross negligence to CSAM distribution is sitting in prison. That serves as an example, a deterrent. "Clean up the act, or you too can go from being a hyper-wealthy executive to the biohazard cleanup detail in a cell block at Leavenworth."

malshe 2mo ago

https://archive.ph/4LhoU https://archive.ph/4LhoU I didn't realize the original was behind a paywall.

lp4v4n 2mo ago

I'd be interested to know if the same amount of leniency shown here would be applied if we were talking about ads depicting violence against elected politicians.

shawn_w 2mo ago
Depends on the politician.
danudey 2mo ago
[flagged]
inigyou 2mo ago
Only if they're LW.

type0 2mo ago

So that's why they are lobbying for age verification, i.e ID verification?

dathinab 2mo ago
no age verification is not ID verification through meta might have lobbied on both but they lobby for specific (often pretty bad) forms of age verification to be able to - spy on people better - have a regulatory moat where due to beady designed laws new alternatives have a much higher hurdle/initial investments and in turn never get anywhere
mplewis 2mo ago
Meta is lobbying for age verification because it helps provide an institutional moat for them against the open web.
- barbazoo 2mo ago
  Could you elaborate? I thought Meta wants to put the burden onto the platform, i.e. Apple and Google, is what they are lobbying for. https://pluribusnews.com/news-and-events/meta-lobbies-for-app-store-age-verification-laws/ https://pluribusnews.com/news-and-events/meta-lobbies-for-ap... What is the institutional moat?

dotcoma 2mo ago

Classy as usual.

alex1138 2mo ago

> and ran across Facebook, Threads, Messenger and Instagram. So it's the SAME AD PLATFORM? Even in MESSENGER? My god. Not only is this whole thing horrible but it's doubly so

Kalanos 2mo ago

What's crazy is how long it takes to report something like this to a big company and actually have them take action on it. I remember 10 years ago I reported a Giphy ad for the same thing, and it took them months to take action.

bombcar 2mo ago
Meta doesn't have to give a shit, because they're huge. Tiny providers have to scramble, because they'll be destroyed by a single failure to handle it fast enough.
jhartikainen 2mo ago
Out of curiosity, I've ran some experiments on Facebook for stuff like this. Results: - Reporting a message does nothing no matter how much the content breaks their terms of use. 100% of messages I reported bounced back in a matter of minutes with "no action" - However, using the "request another review" function, which is available after the initial "no action" response, actually seems to work. How quickly this works seems to vary, I've had actions taken as result of this from 1 day to multiple weeks.
- gs17 2mo ago
  Interesting, I've kind of made a hobby of reporting scams and hate speech and watching it take a week to come back with "it's no violation", requesting another and getting the same response in another week. I've never seen them do anything no matter how obvious or awful it is.
  - inigyou 2mo ago
    You have to actually, honestly, fall for the scam and lose all your money and only then will you have standing to sue Meta for making you lose all your money. You won't be able to afford to sue them, of course.
    - cindyllm 2mo ago

deleted 2mo ago

[deleted]

deleted 2mo ago

[deleted]

godwinson__4-8 2mo ago

Somewhat related: open YouTube in an incognito window. Search something rather innocuous like "news". Click the first thing then go back to the main page. You have to watch through the ad. The amount of slop you will get suggested as soon as the algorithm warms up is nuts. The ad you are first served might also be slop. I mean AI generated video advertising overt bogus cures (a memorable one recently - "did you know the Bible has the answer to Alzheimer's?") over AI slop footage. When I compare to my normal/auth'd Youtube experience, it feels like a totally different app. I have never really experienced such a cheap illustration of the "Youtube rabbit hole" effect. The degree to which AI authored advertising is participating in this is imo pretty concerning if you have, in particular, older parents or relatives.

fragmede 2mo ago
By the by, if you watch a video while logged in that someone sent you as being batshit crazy, and it's affecting your recommendations, you can go into your watch history and delete it from there so you don't get recommended similar videos.
VCFundedGenYer 2mo ago
Youtube is pushing a lot of AI features without creators' consent. Some of it can be turned off but others cannot. It's becoming a major problem. Can't wait till they lose so much money on the infrastructure that they have to remove it all.
- bilegeek 2mo ago
  The auto-translation especially bothers me. Kinda cool for casual viewing... but the inability to disable it gives me an immediate skeptical "this entire channel is probably AI slop" response before I recognize what's happened.
carljungslabtek 2mo ago
Part of me would like to stop paying for a youtube premium family subscription because they keep raising the price, but my mom watches it a lot and I don’t want her to see (or have to see) those ads. They’re so unbelievably bad. It can’t be good for your brain to watch, the few times I’ve seen them on my work computer I can’t help but watch part of them.

lenerdenator 2mo ago

This is what happens when you let a company be controlled by one person who has no reason to be accountable to anyone. If Zuckerberg wants total voting control, then he can also deal with the consequences of being caught hosting CSAM.

olelele 2mo ago
Digital feudalism. Musk is building a city where he can prohibit worker organizing. Joe Hill died for nothing.

froobius 2mo ago

I'm starting to wonder if any of this is better than local newspapers, where there's an editor with oversight over all content...

meric_ 2mo ago

I'm wondering what people expect these companies to do? Human moderators? Sure, but Meta products have approximately half the worlds population using them. At this scale it's pretty mind bogglingly difficult to moderate down everything. The ads/posts served per second is an astronomical number. This truly is a case where AI is the solution. There's no way to automate this at scale with humans. As AI gets better hopefully detection and moderation will catch up

deleted 2mo ago
[deleted]
spacecadet404 2mo ago
AI is generating ads that will be reviewed by AI that will then be pushed to humans largely to ignore. Ad pipeline is malware at this point.
- techjamie 2mo ago
  > Ad pipeline is malware at this point. Always has been. Occasionally my girlfriend will comment on something she saw in an ad, even buy things off them. Thing is though that ad platforms don't seem to be doing anything meaningful to stop bad actors. So really, clicking any ad and interacting with it amounts to placing your trust in a total stranger with the only criteria being they paid a little bit of money for you to see them. To which you will then download their software, give them your credit card, or whatever the end goal is. It's far easier to assume anything in an advertisement is a scam and block them accordingly.
dijksterhuis 2mo ago
It's not everyone else's fault that executives etc. decided to grow their userbase/business/profits at all costs, i.e. with no concerns for the harms they've been causing. It's similar to how adding more and more code, more and more features, more and more stuff into software increases technical debt. Apparently "growth at all costs" for a business equates to ethical/moral debt, quite obviously in this case. Which shouldn't be surprising to anyone who understands/recognises the mechanisms of greed. For software, sometimes paying off that debt means getting rid of some of your features/code/stuff. i.e. there's another alternative to "let's AI the AI". scale the business down. shrink the number of users. make less money.

xgulfie 2mo ago

Weird how they never thought to require ID verification for advertisers hmmm

woodrowbarlow 2mo ago
for real. if you want to make the web safer for children, start by holding ad networks to a higher accountability. it's painfully obvious "protect the children" is not the real goal.

bigbuppo 2mo ago

As if you didn't need another reason to use an ad blocker.

qwerpy 2mo ago
Meta is better than anyone at anti-ad-blocking. Even the best ad blockers struggle to keep ahead of the ever-changing structure served up by their websites, and the websites are intentionally bad to make you use the apps instead. But the apps are nearly immune to ad-blocking. “Just stop using meta properties then”. Sure but some of us have wives, friends, etc

craftkiller 2mo ago

I've noticed on YouTube I see a lot of adult sexual ads. Things like home cleaning ads that have an unmistakable wood-grain image of fellatio. I've always wondered "how are these getting past the moderators?". I guess the answer is the obvious one: no one is moderating anything.

ihuk 2mo ago
Reported an obvious sexual ad the other day and got a very AI generated response that started with "Dear Username" stating that they reviewed the ad and found nothing wrong with it.
- p0w3n3d 2mo ago
  The Dead Internet Theory is real!
- unsnap_biceps 2mo ago
  Before I gave up on Facebook entirely, I reported an ad that was an explicit blow job and received a similar response and this was before AI was huge. I really don't think they've ever cared.
  - mullingitover 2mo ago
    Facebook literally trolls you when you report criminal activity, rampant well-documented scams, on their site. They write back and tell you it's not violating their policy and ask if you want to support for your personal problem.
    - Spooky23 2mo ago
      Absolutely. My uncle got taken by an “investment club” supposedly endorsed by Warren Buffet and run day to day by some real financial person. All communications were via WhatsApp. They started with real stock tips (which were pretty good actually!) and then did a pump and dump. See: https://www.cleuremissionfund.com/ https://www.cleuremissionfund.com/ I received a similar ad (mine was from the shark tank people), reported it and got a message similar to what you described. They were obviously stealing Mark Cuban’s likeness and peddling an obvious scam.

chrisjj 2mo ago

> AI-generated How on earth can they know?

SV_BubbleTime 2mo ago
Read the article. Between the lines it doesn’t seem like the real issue is any specific image, it’s the ”nudify” AI apps. This is why they pushed for the change from “child porn” to “child sexual assault material” because LOTS OF THINGS can be “materials”.
- chrisjj 2mo ago
  I did read the article. It does not answer the question.

andy_ppp 2mo ago

If any of us did that clearly we’d be in jail for several years and put on the sex offenders register but when Meta appear to be hugely responsible for increases in teen suicide or crimes like these we get nothing. If corporations want the rights of individuals they equally should shoulder our shared societal responsibilities as well. I honestly don’t get how they can just pretend they aren’t responsible somehow because it’s slightly expensive to not commit crimes or something… If you are downvoting this please explain why, it’s important that we come to a shared understanding how these things are ever okay for corporations and not individuals…

archildress 2mo ago
Our regulatory framework is so asleep at the wheel. It is as if there is this unpierceable veil if you operate as a corporate entity, as if multiple people's combined transgressions are less than a single person doing it.
- olelele 2mo ago
  Move fast and break literally everything. I am so depressed by what the internet has become. Such potential only to become monopolist behemoth bullshit.
- carljungslabtek 2mo ago
  Is it really asleep at the wheel or is it that decades of intense lobbying have crippled it? I mean of course it’s both, but I think it’s important to point out that it’s not just negligence, it’s more nefarious and selfish than that.
  - Telaneo 2mo ago
    It's in an induced coma at the wheel.
cucumber3732842 2mo ago
>If any of us did that... You don't have Metas legal team nor does the power structure find your valuable to have helping them out nor can you harm them if you don't like them. There was, quite famously, a man who got away with these sorts of things for quite some time. He owned an island. I can't recall his name....

shitloadofbooks 2mo ago

Meta will do whatever it can to run any and all ads. Meta knowingly runs scam ads, they just charge more to run them: https://www.reuters.com/legal/litigation/consumer-watchdog-group-files-suit-alleging-meta-profited-ads-scams-2026-04-21/ https://www.reuters.com/legal/litigation/consumer-watchdog-g... > Reuters also reported that Meta internally projected in 2024 that it would earn about 10% of its overall annual revenue – or $16 billion – from running ads for scams and banned goods, internal company documents show. At $employer we're CONSTANTLY fighting fake ads cloning our brand. Their various brand protection features seem deliberately inept and ineffective.

disillusioned 2mo ago

Beyond these obviously deplorable ads, my wife got sniped by an ecommerce lookalike ad on Google the other day: a near-identical match of the vendor she was looking for, with a paid for sponsored ad (that was clearly linked to some chinese firm upon further review). Site had prices that were just a bit too good to be true, and a slightly different TLD that _I_ would've caught, but she missed because the price was a bit too good to be true. Only caught it when Citi asked me if I was spending 9000EGD on some random website. And my point here is, we're in this great revolution of AI empowered everything and... these companies can't be bothered to scan this volume of AI ad slop to make sure we're not being actively scammed? Or having porn or CSAM spread? I understand it's a "perverse incentive" in the truest sense: they make money by taking the scammers money, but I was scammed once by an insanely clever lookalike Facebook ad, and had trusted their verified checkmark (about a decade ago)... with AI, these things should be a thing of the past.

onlyrealcuzzo 2mo ago

Guys, it's fine. Relax. It's only a crime if someone else does it or is not sufficiently wealthy, otherwise it's a mistake.

_carbyau_ 2mo ago
I believe that is referred to as: organised crime.
- tremon 2mo ago
  Rather, this is institutional crime. Organised crime is when crime operates its own control structures in defiance of government enforcement; this is the government actively allowing the crime to proliferate.
  - _carbyau_ 2mo ago
    Organised crime involves the corruption of institutions too - specifically to allow getting away with it. But I like your take on it as institutionalised crime because it is now publicly overt. The corruption is complete.

yalogin 2mo ago

Even if they are caught and it’s illegal, as long as the fine for it is less than the roi of running them the companies will not stop.

callamdelaney 2mo ago

Meta is still running ads that suggest violence against politicians, when reported they say it does not contravene their policies.

wredcoll 2mo ago
Politicians don't count.
- inigyou 2mo ago
  LW politicians don't count. If there was an ad calling for violence against an RW politician it'd be taken down and the poster would go to jail.
  - callamdelaney 2mo ago
    Actually it’s for a right wing politician and there are many variations.

cdrnsf 2mo ago

We were better off with landlines and local journalism.

randyrand 2mo ago

Non-specific terms are a pet peeve of mine in accusations. Is this new term broader than the old term of CP or is it the same? Why did we even switch?

tbrownaw 2mo ago
> Is this new term broader than the old term of CP or is it the same? Why did we even switch? The purpose of the new term was to emphasize that producing it involves the actual abuse of an actual child. And then somehow it got applied to drawings and lately computer-generated stuff.
- inigyou 2mo ago
  The purpose of the new term was virtue signalling by suggesting that it's somehow outside of the definition of the word "porn".
bluefirebrand 2mo ago
> Why did we even switch? If I had to guess, this is to try and distance child sexual assault from the concept of "pornography", and maybe make it more abundantly clear that "child porn" is always, 100% of the time, sexual abuse. I dunno, language evolves like this sometimes. Like "smoking" which is now called "tobacco use" or "cannabis use" to clarify more what is being smoked

crmd 2mo ago

I asked my grandpa about the business idea of self-serve magazine advertising. He said if you do that wait till you see the weird shit people will try to publish. I wonder if we could learn anything from these cranky old timers.

eboy 2mo ago

[dead]

throwatdem12311 2mo ago

Just shut the company down, throw the executives in a deep dark hole and forget they exist. The world will be better off.

2OEH8eoCRo0 2mo ago

Repeal Section 230

thi2 2mo ago

Metas ads are all over the place. Thomas Kehl, one of the founders of "Finanzfluss" (a german site/content creation company about finance) is currently suing meta because scammers keep using his picture. There really needs to be some harder punishments for this kind of behavior, after all meta profits by selling scammy ads.

xhevahir 2mo ago

If stuff like this shows up in GAI I wonder what that means for how common such material is in the wild. Frightening.

ticulatedspline 2mo ago

Personally I find these kinds of things fascinating. Partially for the practical problem it represents and it's facets but also the one sided backlash of it all. Moderation is an interesting conglomerate of so many factors. First of all I don't think Meta is a good or honest company, but that doesn't mean they're not trying at all. In reality even if they were good and honest I genuinely believe their task is probably nearly impossible. Humans just suck too much. It certainly doesn't help that Meta has at least some perverse incentive to be permissive with ads, as they get money from them. Though Meta execs aren't stupid. Inflammatory political ads are one thing, kiddie porn is another I suspect they lose more money over CSAM than all the ad revenue from them. I believe them when they say they removed over 36 million pieces of child sexual exploitation content last year. Which is a number that I should let sink in. Basically more than one piece of CSAM shows up in their ad stream every second of every day all year. If their moderation techniques are 99% effective that results in 360,000 CSAM ads leaking to the public annually. Another interesting thought is how much would it cost to actually get that rate to 100%? let's assume human moderators aren't paid much but are infallible. That all-in a single moderator employee costs $65K , that's salary, benefits, HR ratio, psychological therapy, equipment, the works, all factored in. let's say on average you get about 1 ad review ~every 2 minutes, so bout 250 ads per-day per-employee factoring in some time off you actually get about 65,000 reviews a year. So bout $1 per review. So the question begs how many distinct ads enter their system each year? which I haven't found a stat for. But assuming only 1 in 1000 ads are CSAM, which still feels high but is probably a reasonable ratio. we get 36 billion ads needing review annually. At a dollar each 23% of their gross annual ad revenue each year would be just moderation overhead and they would need to hire over 550,000 moderators. Evil or not I think they organically built a thing that operates at an unimaginable scale, that even moderation with failures in the parts-per-million would result in tens of thousands of unsavory or unscrupulous ads reaching users and they have neither the incentive nor the capacity to reach even that level of accuracy.

happymellon 2mo ago
Disagree with essentially all of this. To get a Facebook account they require all sorts of PII. Why not have people give their PII to set up an advertising account and if they distribute CSAM then they are banned. Removing access to WhatsApp will stop a large portion of the world from participating. And the parts that can't? Well, they can't advertise on Facebook/Instagram anymore.

inigyou 2mo ago

Fork found in kitchen.

blks 2mo ago

Good example why not to post your children faces online

eayomi 2mo ago

[flagged]

pickleglitch 2mo ago

Zuckerberg should thrown in prison.

ajdude 2mo ago

I reported a blatantly sexual adult ad on Facebook the other day, they got back to me and said that their moderators reviewed the ad and that it was appropriate.