11 ms·
Good one. No platform ever should allow CSAM content. And the fact that they didn’t even care and haven’t want to spend money for implementing guardrails or m
by mnewme 8mo ago
Good one.
No platform ever should allow CSAM content.
And the fact that they didn’t even care and haven’t want to spend money for implementing guardrails or moderation is deeply concerning.
This has imho nothing to do with model censorship, but everything with allowing that kind of content on a platform
- exodust 8mo ago[flagged]
- cess11 8mo ago[flagged]
- myrmidon 8mo agoThere are multiple valid reasons to fight realistic computer-generated CSAM content. Uncontrolled profileration of AI-CSAM makes detection of "genuine" data much harder, prosecution of perpetrators more difficult and specifically in many of the grok cases it harms young victims that were used as templates for the material. Content is unacceptable if its proliferation causes sufficient harm, and this is arguably the case here.
- Eisenstein 8mo ago> Uncontrolled profileration of AI-CSAM makes detection of "genuine" data much harder I don't follow. If the prosecutor can't find evidence of a crime and a person is not charged, that is considered harmful? As such the 5th amendment would fall under the same category and so would encryption. Making law enforcement have to work harder to find evidence of a crime cannot be criminalized unless you can come up with a reason why the actions themselves deserve to be criminalized. > specifically in many of the grok cases it harms young victims that were used as templates for the material. What is the criteria for this? If something is suitably transformed such that the original model for it is not discernable or identifiable, how can it harm them? Do not take these as an argument against the idea you are arguing for, but as rebuttals against arguments that are not convincing, or if they were, would be terrible if applied generally.
- myrmidon 8mo agoIf there is a glut of legal, AI generated CSAM material then this provides a lot of deniability for criminal creators/spreaders that cause genuine harm, and reduces "vigilance" of prosecutors, too ("it's probably just AI generated anyway..."). You could make a multitude of arguments against that perspective, but at least there is a conclusive reason for legal restrictions. > What is the criteria for this? My criteria would be victims suffering personally from the generated material. The "no harm" argument only really applies if victims and their social bubble never find out about the material (but that did happen, sometimes intentionally, in many cases). You could make the same argument that a hidden camera in a locker room never causes any harm as long as it stays undetected; that is not very convincing to me.
- Eisenstein 8mo ago> You could make a multitude of arguments against that perspective, but at least there is a conclusive reason for legal restrictions. But that reason is highly problematic. Laws should be able to stand on their own for their reasons. Saying 'this makes enforcement of other laws harder' does not do that. You could use the same reasoning against encryption. > You could make the same argument that a hidden camera in a locker room never causes any harm as long as it stays undetected; that is not very convincing to me. I thought you were saying that the kids who were in the dataset that the model was trained on would be harmed. I agree with what I assume you meant based on your reply, which is people who had their likeness altered are harmed.
- myrmidon 8mo ago> Saying 'this makes enforcement of other laws harder' does not do that. You could use the same reasoning against encryption. Yes. I almost completely agree with your outlook, but I think that many of our laws trade such individual freedoms for better society-wide outcomes, and those are often good tradeoffs. Just consider gun legislation, driving licenses, KYC laws in finance, etc: Should the state have any business interfering there? I'd argue in isolation (ideally) not; but all those lead to huge gains for society, making it much less likely to be murdered by intoxicated drivers (or machine-gunners) and limit fraud, crime and corruption. So even if laws look kinda bad from a purely theoretical-ethics point of view it's still important to look at the actual effects that they have before dismissing them as unjust in my view.
- KaiserPro 8mo ago> I remember when CSAM meant actual children not computer graphics. The "oh its photoshop" defence was an early one, which required the law to change in the uk to be "depictions" of children, so that people who talk about ebephiles don't have an out for creating/distributing illegal content.
- master-lincoln 8mo agoThere still needs to be sexual abuse depicted, no? Just naked kids should not be an issue, right?
- direwolf20 8mo agoNaked kid pictures intended for sexual gratification are illegal in most countries
- master-lincoln 8mo agoHard to know the intent of a picture in most cases. E.g. there used to be a magazine for teens when I grew up showing a picture of a naked adolescent of each sex in every edition (Bravo Dr Sommer). The intent was to educate teens and to make them feel less ashamed. I bet there were people who used these for sexual gratification. Should that have been a reason to ban them? I don't think so.
- mnewme 8mo agoEducational nudity where the subject consented possible for teenagers over 16 in Germany (and the publication complied with the law) is not the same category as CSAM or non-consensual sexual imagery. In the former, misuse by a minority doesn’t automatically make the publication illegal. In the latter, the harm is intrinsic: a child cannot legally consent, and non-consensual sexual images are a direct rights violation.
- direwolf20 8mo agoDo you think the judge is stupid? The naked kid pics aren't printed in an anatomical textbook. Because they're AI hallucinations, I doubt they're even anatomically correct.
- mnewme 8mo agoWhat the hell? As a father there shouldn’t be any CSAM content anywhere. And think about that it is already proven these models apparently had CSAM content in their training data. Also what about the nudes of actual people? That is invasion of privacy I am shocked that we are even discussing this.
- pjc50 8mo ago> How about "R-Rated" violence like we see in popular movies? Movie ratings are a good example of a system for restricting who sees unacceptable content, yes.
- ascagnel_ 8mo agoMore to the point, now that most productions are using intimacy coordinators, there's a degree of certainty around the consent of R-rated images. There's basically no consent with what Grok is doing.
- guerrilla 8mo ago> There's basically no consent with what Grok is doing. Wait how do you get consent from people that don't exist?
- deleted 8mo ago[deleted]
- thrance 8mo ago[flagged]
- mnewme 8mo agoExactly! This should not be ok
- master-lincoln 8mo agoA generated picture of a family member in a bikini is an issue? I don't see it...
- mnewme 8mo ago[flagged]
- master-lincoln 8mo ago[flagged]
- mnewme 8mo agoThere is a difference between running around in a bikini and people creating sexy pictures of yourself without consent. You do understand that?
- master-lincoln 8mo agoI do understand that, but in this thread there was no mention of anything sexy so far. I am just pedantic because I think it is important when it comes to criminal accusations. Sexuality and nakedness are two different things.
- mnewme 8mo agoThen check what the discussion is about, this is not about some funny ai pics
- SecretDreams 8mo agoIs this post low-key advocating for anime csam in the name of freedom?
- exodust 8mo ago> name of freedom Speaking of freedom, I lost posting permission yesterday after my earlier post. Even though vile replies were insinuating vile things, my right of reply was taken away. We should never diminish the "name of freedom" as you've just done. It should go without saying that CSAM is revolting. Who wants to see that stuff? Not me, not most people. Grok can't make that content. Maybe someone got around it temporarily. I've always thought Grok heavily censored, it refuses to analyse an image of the Statue of David because "naughty bits". The ironic and sad thing is a fig leaf gets around that restriction. So the accusation that Grok has no guardrails and can generate CSAM seems at best an anomaly or a lie.
- gadders 8mo ago[flagged]
- pjc50 8mo agoThe different factors are scale (now "deepfakes" can be automatically produced), and endorsement. It is significant that all these images aren't being posted by random users, they are appearing under the company's @grok handle. Therefore they are speech by X, so it's X that's getting raided.
- mnewme 8mo agoThere is no content like that on Bluesky nor Mastadon. Show the evidence
- Zenst 8mo agohttps://bsky.social/about/blog/01-17-2025-moderation-2024 https://bsky.social/about/blog/01-17-2025-moderation-2024 "In 2024, Bluesky submitted 1,154 reports for confirmed CSAM to the National Centre for Missing and Exploited Children (NCMEC). Reports consist of the account details, along with manually reviewed media by one of our specialized child safety moderators. Each report can involve many pieces of media, though most reports involve under five pieces of media." If it wasn't there, there would be no reports.
- GaryBluto 8mo ago> There is no content like that on [...] Mast(o)don. How can you say that nobody is posting CSAM on a massive decentralized social network with thousands of servers?
- sunshine-o 8mo ago[flagged]
- pjc50 8mo agoEnforcement of anti-CSAM law has been a significant thing for a long time. It's in no way "only now". Even the "free speech" platforms banned it because they knew they would get raided otherwise. There are long standing tools for dealing with it, such as a database of known hashes of material. There's even a little box you can tick in Cloudflare to automatically check outgoing material from your own site against that database - because this is a strict liability offence, and you are liable if other people upload it to you where it can be re-downloaded. What's new is that X automated the production of obscene or sexualised images by providing grok. This was also done in a way that confronted everyone; it's very different from a black market, this is basically a harassment tool for use against women and girls.
- sunshine-o 8mo ago> What's new is that X automated the production of obscene or sexualised images by providing grok. Yes we are now dealing with an automated Photoshop. And somehow the people in charge have decided to do something about it, probably more for political or maybe darker reasons. So let me make a suggestion: maybe France or the EU should ban its citizen from investing in the upcoming SpaceX/xAI IPO, and also Microsoft, NVIDIA, OpenAI, Google, Meta, Adobe, etc. ? Hit them hard at the money level... it wouldn't be more authoritarian than something like ChatControl or restricting access to VPNs. And actually all the mechanisms are already in place to implement something like that.
- cbolton 8mo ago> Yes we are now dealing with an automated Photoshop. And somehow the people in charge have decided to do something about it, probably more for political or maybe darker reasons. I don't get what's difficult to understand or believe here. Grok causes a big issue in practice right now, a larger issue than photoshop, and it should be easy for X to regulate it themselves like the competition does but they don't, so the state intervenes. > maybe France or the EU should ban its citizen from investing in the upcoming SpaceX/xAI IPO, and also Microsoft, NVIDIA, OpenAI, Google, Meta, Adobe, etc. ? You're basically asking "why do a surgical strike when you can do carpet bombing"? A surgical strike is used to target the actual problem. With carpet bombing you mostly cause collateral damage.
- darkport 8mo ago[flagged]
- direwolf20 8mo ago[flagged]
- ReptileMan 8mo agoI disagree. Prosecute people that use the tools, not the tool makers if AI generated content is breaking the law. A provider should have no responsibility how the tools are used. It is on users. This is a can of worms that should stay closed, because we all lose freedoms just because of couple of bad actors. AI and tool main job is to obey. We are hurling at "I'm sorry, Dave. I'm afraid I can't do that" future with breakneck speed.
- mnewme 8mo agoI agree that users who break the law must be prosecuted. But that doesn’t remove responsibility from tool providers when harm is predictable, scalable, and preventable by design. We already apply this logic elsewhere. Car makers must include seatbelts. Pharma companies must ensure safety. Platforms must moderate illegal content. Responsibility is shared when the risk is systemic.
- ReptileMan 8mo ago>But that doesn’t remove responsibility from tool providers when harm is predictable, scalable, and preventable by design. Platforms moderating illegal content is exactly what we are arguing about, so you can't use it as an argument. The rest cases you list are harms to the people using the tools/products. It is not harms that people using the tools inflict on third parties. We are literally arguing about 3d printer control two topics downstream. 3d printers in theory can be used for CSAM too. So we should totally ban them - right? So are pencils, paper, lasers, drawing tablets.
- mnewme 8mo agoThat is not the argument. No one is arguing about banning open source LLMs that could potentially create problematic content on huggingface, but X provides not only an AI model, but a platform and distribution as well, so that is inherently different
- ReptileMan 8mo ago
- moorebob 8mo ago[flagged]
- mnewme 8mo agoThey are not doing anything that is the problem. X just doesn’t care at all. But they are happy to censor for countries like turkey Hypocrisy level 1 million
- tw85 8mo agoIt seems people have a rather short memory when it comes to twitter. When it was still run by Jack Dorsey, CP was abundant on twitter and there was little effort to tamp down on it. After Musk bought the platform, he and Dorsey had a public argument in which Dorsey denied the scale of the problem or that old twitter was aware of it and had shown indifference. But Musk actually did take tangible steps to clean it up and many accounts were banned. It's curious that there wasn't nearly the same level of outrage from the morally righteous HN crowd towards Mr. Dorsey back then as there is in this thread.
- misnome 8mo agoWhen did Jack Dorsey unban personal friends of his that had gotten banned for posting CSAM?
- brightball 8mo agoI meant to reply to you with this: https://news.ycombinator.com/item?id=46886801 https://news.ycombinator.com/item?id=46886801
- ryandrake 8mo agoDidn't Reddit have the same problem until they got negative publicity and were basically forced to clean it up? What is with these big tech companies and CP?
- goodolddays9090 8mo ago[dead]
- imperio59 8mo agoReddit was forced to clean it up when they started eyeballing an IPO.
- Manuel_D 8mo agoNot exactly. Reddit always took down CSAM (how effectively I don't know, but I've been using the site consistently since 2011 and I've never come across it). What Reddit did get a lot of negative public publicity for were subreddits focused on sharing non-explicit photos of minors, but with loads of sexually charged comments. The images themselves, nobody would really object to in isolation, but the discussions surrounding the images were all lewd. So not CSAM, but still creepy and something Reddit tightly decided it didn't want on the site.
- brightball 8mo agoAgreed. For anyone curious, here's the UK report from the National Society for the Prevention of Cruelty to Children (NSPCC) from 2023-2024. https://www.bbc.com/news/articles/cze3p1j710ko https://www.bbc.com/news/articles/cze3p1j710ko Reports on sextortion, self-generated indecent images, and grooming via social media/messaging apps: Snapchat 54% Instagram 11% Facebook 7% WhatsApp 6-9% X 1-2%
- rcpt 8mo agoWhat are the percentages?
- brightball 8mo agoEdited to add clarification.
- spacemule 8mo agoThe meaning of the percentages is still unclear.
- ishouldstayaway 8mo agoRead the article they linked to.
- jbenner-radham 8mo agoAre those numbers in the article somewhere? From what I read it says that out of 7,062 cases, the platform was known for only 1,824. Then it says Snapchat accounts for 48% (not 54%). I don't see any other percentages.
- yibg 8mo agoMy natural reaction here is like I think most others; that yes Grok / X bad, shouldn't be able to generate CSAM content / deepfakes. But I am having trouble justifying in an consistent manner why Grok / X should be liable here instead of the user. I've seen a few arguments here that mostly comes down to: 1. It's Grok the LLM generating the content, not the user. 2. The distribution. That this isn't just on the user's computer but instead posted on X. For 1. it seems to breakdown if we look more broadly at how LLMs are used. e.g. as a coding agent. We're basically starting to treat LLMs as a higher level framework now. We don't hold vendors of programming languages or frameworks responsible if someone uses them to create CSAM. Yes LLM generated the content, but the user still provided the instructions to do so. For 2. if Grok instead generated the content for download would the liability go away? What if Grok generated the content to be downloaded only and then the user uploaded manually to X? If in this case Grok isn't liable then why does the automatic posting (from the user's instructions) make it different? If it is, then it's not about the distribution anymore. There are some comparisons to photoshop, that if i created a deep fake with photoshop that I'm liable not Adobe. If photoshop had a "upload to X" button, and I created CSM using photoshop and hit the button to upload to X directly, is now Adobe now liable? What am I missing?
- bjoyx 8mo ago[flagged]
- realusername 8mo ago> But I am having trouble justifying in an consistent manner why Grok / X should be liable here instead of the user. Because Grok and X aren't even doing the most basic filtering they could do to pretend to filter out CSAM.
- yibg 8mo agoFiltering on the platform or Grok output though? If the filtering / flagging on X is insufficient then that is a separate issue independent of Grok. If filtering output of Grok, while irresponsible in my view, I don’t see why that’s different from say photoshop not filtering its output.
- lux-lux-lux 8mo agoThe lack of guardrails wasn’t a carelessness issue - Grok has many restrictions and Elon regularly manipulates the answers it gives to suit his political preferences - but rather one of several decisions to offer largely unrestricted AI adult content generation as a unique selling point. See also, e.g. the lack of real age verification on Ani’s NSFW capabilities.