13 ms·
X blames users for Grok-generated CSAM; no fixes announced
- pastureofplenty 10mo ago[flagged]
- jshier 10mo ago"Stop asking the guy we hired to draw CSAM, we're not going to tell him to stop."
- chinathrow 10mo agoAssholes all the way down. "“We take action against illegal content on X, including Child Sexual Abuse Material (CSAM), by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary,” X Safety said. “Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content.” How about not enabling generating such content, at all?
- Forgeties79 10mo agoGotta pass the buck somewhere and it sure as hell isn’t going to Musk. It’s always the user’s fault.
- renewiltord 10mo agoI don’t think anyone can claim that it’s not the user’s fault. The question is whether it’s the machine’s fault (and the creator and administrator - though not operator) as well.
- codyb 10mo agoThe article claims Grok was generating nude images of Taylor Swift without being prompted and that there was no way for the user to take those images down I don't know how common this is, or what the prompt was that inadvertently generated nudes. But it's at least an example where you might not blame the user
- renewiltord 10mo agoYeah but “without being asked” here means the user has to confirm they are 18+, choose to enable NSFW video, select “spicy” in Grok’s video generation settings and then prompt “Taylor Swift celebrating Coachella with the boys”. The prompt seems fine but the rest of it is clearly “enable adult content generation”. I know they said “without being prompted” here but if you click through you’ll see what the person actually selected (“spicy” is not default and is age-gated and opt-in via the nsfw wall).
- codyb 10mo agoNice, thanks for the details! Very weird for Taylor Swift...
- renewiltord 10mo agoYes, the reporter should not be generating porn of her. Pretty unethical.
- Forgeties79 10mo agoLet’s not lose sight of the real issue here: Grok is a mess from top to bottom run by an unethical, fickle Musk. It is the least reliable LLM of the major players and musk’s constant fiddling with it so it doesn’t stray too far from his worldview invalidates the whole project as far as I’m concerned.
- SAI_Peregrinus 10mo agoIsn't it a strict liability crime to posses it in the US? So if AI-generated apparent CSAM counts as CSAM legally (not sure on that) then merely storing it on their servers would make X liable.
- renewiltord 10mo agoI think platforms that host user-generated content are (rightly) treated differently. If I posted a base64 of CSAM in this comment it would be unreasonable to shut down HN. The questions then, for me, are: * Is Grok considered a tool for the user to generate content for X or is Grok/X considered similar to a vendor relationship * Is X more like Backpage (not protective enough) than other platforms I’m sure this is going to court, at least for revenge porn stuff. But why would anyone do this to their platform? Crazy. X/Twitter is full of this stuff now.
- NekkoDroid 10mo agoI don't think you can argue yourself out of "The Grok account is owned and operated by Twitter". In no planet is what it outputs user generated content since the content does not originate from the user, at most they requested some content from Twitter and Twitter provided it.
- bluGill 10mo agoYou are only liable if you know - or should know - that you possess it. You can help someone out by mailing their sealed letter containing CSAM and be fine since you have no reason to suspect the sealed letter isn't legal. X can store CSAM so long as they have reason to think it is legal. Note that things change. In the early days of twitter (pre X) they could get away with not thinking of the issue at all. As technology to detect CSAM marches on they need to use it (or justify why it shouldn't be used - too many false positives???). As a large platform for such content they need to push the state of the art in such detection.. At no point do they need perfection - but they need to show they are doing their reasonable best to stop this. The above is of course my opinion. I think the courts will go a similar direction, but time will tell...
- queenkjuul 10mo agoGrok loves to make things lewd without asking first.
- Forgeties79 10mo agoMusk pretends he made Vision but what he made was Great Value Ultron
- renewiltord 10mo ago[flagged]
- thorum 10mo agoThey don’t seem to have taken even the most basic step of telling Grok not to do it via system prompt.
- notahacker 10mo agothey use that for more important stuff like ensuring that it predicts Elon Musk would beat Usain Bolt in a race...
- DANmode 10mo agoThat would admit legal liability for the capabilities of their model - no?
- shimman 10mo agoDirected negligence isn't much better, especially morally.
- bluGill 10mo agoYou always have liability. If you put something there you tell the court that you see the problem and are trying to prevent it. It often becomes easier to get out of liability if you can show the courts you did your best to prevent this. Courts don't like it when someone is blatantly unaware of things - ignorance is not a defense if "a reasonable person" would be aware of it. If this was the first AI in 2022 you could say "we never thought about that" and maybe get by, but by 2025 you need to tell the court "we are aware of the issue, and here is why we think we had reasonable protections that the user got around". See a lawyer for legal details of course.
- praptak 10mo agoThey already censor Grok when it suits them.
- UncleMeat 10mo agoYep. "Oh grok is being too woke" gets musk to comment that they'll fix it right away. But turn every woman on the platform into a sex object to be the target of humiliation? That's just good fun apparently.
- wtfwhateven 10mo ago"permanently suspending accounts" Surprising, usually the system automatically bans people who post CSAM and elon personally intervenes to unban then. https://mashable.com/article/x-twitter-ces-suspension-right-wing-account-reinstated https://mashable.com/article/x-twitter-ces-suspension-right-...
- subscribed 10mo agoThis is probably harder because it's synthetic and doesn't exist in PhotoDNA database. Also, since Grok is really good in getting the context, something akin to "remove their T-shirt" would be enough to generate a picture someone wanted, but very hard to find using keywords. IMO they should mass hide ALL the images created since then specific moment, and use some sort of the AI classifier to flag/ban the accounts.
- 9x39 10mo agoI use AI, but I don't get such content. I understand everyone pouncing when X won't own Grok's output, but output is directly connected to its input and blame can be proportionally shared. Isn't this a problem for any public tool? Adversarial use is possible on any platform, and consistent law is far behind tech in this space today.
- wmf 10mo agoUnfortunately society seems to have decided that moderation is a complete replacement for personal accountability.
- skybrian 10mo agoAn analogy: if you're running the zoo, the public's safety is your job for anyone who visits. It's of course also true that sometimes visitors act like idiots (and maybe should be prosecuted), and also that wild animals are not entirely predictable, but if the leopards are escaping, you're going to be judged for that.
- wmf 10mo agoBut the visitors are almost never prosecuted, even when something is obviously their fault.
- skybrian 10mo agoMaybe because sometimes they're kids? You gotta kid-proof stuff in a zoo. Also, punishment is a rather inefficient way to teach the public anything. The people who come through the gate tomorrow probably won't know about the punishment. It will often be easier to fix the environment. Removing troublemakers probably does help in the short term and is a lot easier than punishing.
- wmf 10mo agoSocial media is mostly not removing troublemakers though.
- consonaut 10mo agoHow about policing CSAM at all? I can still vividly remember firehose API access and all the horrible stuff you would see on there. And if you look at sites like tk2dl you can still see most of the horrible stuff that does not get taken down. Do yourself a favor and not Google that.
- ebbi 10mo agoIt's on X, not some fringe website that many people in the world don't access. Regardless of how fringe, I feel like it should be in everyones best interests to stop/limit CSAM as much as they reasonably can without getting into semantics of who requested/generated/shared it.
- consonaut 10mo agoWell, then you might want to look up tk2dl, because it just links to Twitter content. It gets disgusting fairly quickly though.
- esalman 10mo agoWilling to bet that X premium signups have shot up because of this feature. Currently this is the most convenient tool to generate porn of anything and everything.
- foxglacier 10mo agoBecause synthetic CSAM is a victimless puritanical crime and only some countries countries criminalize it.
- ux266478 10mo agoGetting off to images of child abuse (simulated or not) is a deep violation of social mores. This itself does indeed constitute a type of crime, and the victim is taken to be society itself. If it seems unjust, it's because you have a narrow view of the justice system and what its job actually is (hint: it's not about exacting controlled vengeance) It may shock you to learn that bigamy and sky-burials are also quite illegal.
- queenkjuul 10mo ago"we take action... Including permanently suspending accounts" unless, of course, the account is Elon's pet
- JumpCrisscross 10mo ago> How about not enabling generating such content, at all? Or, if they’re being serious about the user-generated content argument, criminally referring the users asking for CSAM. This is hard-liability content. Also, where are all the state attorneys general?
- tastyface 10mo agoMusk has literally unbanned users who posted CSAM: https://www.theguardian.com/technology/2023/aug/10/twitter-x-restores-unbans-account-shared-child-abuse-material https://www.theguardian.com/technology/2023/aug/10/twitter-x...
- jmuguy 10mo agoShocked that the same people that were using Lolicon as their avatars on Twitter won't do anything about the CSAM on X.
- ranyume 10mo agoHow do you know they're not doing anything? Do they have the power to do anything at all beyond virtue signaling?
- Dweller1622 10mo agoWhat in the world is this referring to?
- matsemann 10mo agoI'm not much on X anymore due to the vitriol, and visiting now kinda proved it. Beneath almost every trending post made by a female is someone using grok to sexualize a picture of them. (And whatever my timeline has become now is why I don't visit more often, wtf, used to only be cycling related) Edit: just to bring receipts, 3 instances in a few scrolls: https://x.com/i/status/2007949859362672673 https://x.com/i/status/2007949859362672673 https://x.com/i/status/2007945902799941994 https://x.com/i/status/2007945902799941994 https://x.com/i/status/2008134466926150003 https://x.com/i/status/2008134466926150003
- QuercusMax 10mo agoI left when they started putting verified (paid) comments at the top of every conversation. Having the worst nazi views front and center on every comment isn't really a great experience.
- wahnfrieden 10mo agoThey also don’t take down overt Nazi content anymore. Accounts with all the standard unambiguous Nazi symbologies and hate content about their typical targets with associated slurs. With imagery of Hitler and praises of his policies. And calls for exterminating their perceived enemies and dehumanizing them as subhuman vermin. I’ve tried reporting many accounts and posts. It’s all protected now and boosted via payment.
- bakies 10mo agoWell the owner supports neo-nazi political parties, so that tracks.
- QuercusMax 10mo agoand of course he gave a very public Nazi salute
- zamadatix 10mo ago
- dannyfreeman 10mo agoAt what point will payment processors step in and stop processing blue check mark subscriptions?
- xemoka 10mo agoMaybe this is the right angle to crush it... two wrongs make a right? Hmm.
- focusgroup0 10mo ago[flagged]
- TimorousBestie 10mo agoIt’s been reported Grok has generated CSAM by editing photos of real children, so there’s the real victim you shouldn’t need to find this situation abominable.
- crtified 10mo agoHumanity?
- cephei 10mo agoI think primarily this victimizes all those all ready victimized by the CSAM in the training material and also generally offends the collective sense of morality our society has.
- warmedcookie 10mo agoSimplistically and ignorantly speaking, if a diffusion model knows what a child looks like and also knows what an adult woman in a bikini looks like, couldn't it just merge the two together to create a child in a bikini? It seems to do that with other things (ex. Pelican riding a bicycle)
- wizzwizz4 10mo agoIn principle yes, but in practice no: the models don't just learn the abstract space, but also memorise individual people's likenesses. The "child" concept contains little clusters for each actual child who appeared enough times in the dataset. If you tried to do this, the model would produce sexualised imagery of those specific children with distressing regularity.
- warmedcookie 10mo agoI appreciate the info, thanks.
- TimorousBestie 10mo agoTruly insane as a legal position. I assume the courts will uphold this anyway because Musk rich and cannot be held accountable for his actions.
- breakingrules3 10mo ago[dead]
- croes 10mo ago> That’s like blaming a pen for writing something bad,” DogeDesigner opined. So from technical wonder to just like a pen in one easy step. Wouldn’t it be great if you could tell the AI what not to output?
- dmitrygr 10mo ago> Wouldn’t it be great if you could tell the AI what not to output? This has been tried extensively and has not yet fully worked. Google "ai jailbreaks".
- femiagbabiaka 10mo agoAny lawyers around? I would assume (IANAL) that Section 230 does not apply to content created by an agent owned by the platform, as opposed to user-uploaded content. Also it seems like their failure to create safeguards opens up the possibility of liability. And of course all of this is narrowly focused on CSAM (not that it should be minimized) and not on the fact that every person on X, the everything app, has been opened up to the possibility of non-consensual sexual material being generated of them by Grok.
- consp 10mo ago> non-consensual sexual material being generated of them by Grok They should disable it in the Netherlands in this case since it really sounds like a textbook slander case and the spreader can also be held liable. note: it's not the same as in the US despite using the same word, deepfakes have been proven as slander and this is no different. Especially if you know it's fake by using "AI". There have been several cases of pornographic deep fakes, all of which were taken down quickly, in which the poster/creator was sentenced. The unfortunate issue even of taking posts down quickly is unfortunately the rule which states that if something is on the internet it stays on the internet. The publisher always went free due to acting quickly and not creating it. I would like to see where it goes when both publisher and creator are the same entity, and they do nothing to prevent it.
- TitaRusell 10mo agoYeah this is pretty funny. Seeing all these discussions about section 230 and the American constitution... Nobody in the Netherlands gives one flying fuck about American laws what GROK is doing violates many Dutch laws. Our parliament actually did it's job and wrote some stuff about revenge porn, deep fakes and artificial CP.
- zerocrates 10mo agoThe CSAM aspects aren't necessarily as affected by 230: to the extent that you're talking about it being criminal, 230 doesn't apply at all there. For civil liability, 230 really shouldn't apply; as you say, 230's shield is about avoiding vicarious liability for things other people post. This principle stretches further than you might expect in some ways but here Grok just is X (or xAI). Nothing's set in stone much at all with how the law treats LLMs but an attempt to say that Grok is an independent entity sufficient to trigger 230 but incapable of being sued itself, I don't see that flying. On the other hand the big AI companies wield massive economic and political power, so I wouldn't be surprised to see them push for and get explicit liability carveouts that they claim are necessary for America to maintain its lead in innovation etc. etc., whether those come through legislation or court decisions.
- danbrooks 10mo agoCould there be a criminal investigation? CSAM is a serious issue.
- idontwantthis 10mo agoNormally yes, but literally and seriously the US government is run by pedophiles now so we will have to see.
- paxys 10mo agoThere should be, but there won't be.
- pastureofplenty 10mo agoI can see this becoming a culture war thing like vaccines. Conservatives will become pro-CSAM because it triggers the overly sensitive crybaby Liberals.
- QuercusMax 10mo ago[flagged]
- lynndotpy 10mo agoThis has already been a culture war thing, and it's why X.com is able to continue to provide users with CSAM with impunity. The site is still up after everything, and the app is still on the app store everywhere. When the far-right paints trans people as pedophiles, it's not an accident that also provides cover for pedophiles. The age of consent between 16 and 18 is a relatively high born from progressive feminist wins. In the United States, the lowest AOC was 14 until the 1990s, and the AOC in the US ranged from _7 to 12_ for most of our existence. To be clear, I'm in defense of a high age of consent. But it's something that had to be fought for, and it's not something that can be assumed to be safe in our culture (like the rejection of nazis and white supremacists, or valuing womens rights including voting and abortion). Influential politicians like Tom Hofeller were advocates for pedophilia and nobody cares at all. Trump is still in power despite the Epstein controversy, Matt Gaetz still hasn't been punished for paying for sex with an underage girl in 2017. The Hitler apologia in the far-right spaces even explicitly acknowledge he was a pedophile. Etc. In a different era, X would have been removed from Apple and Google's app stores for the CEO doing nazi salutes and the chatbot that promoting Hitler. But even now that X is a CSAM app, as of 3PM ET, I can still download X on both of their app stores. That would not have been normal just two years ago. This has already been a culture war issue for awhile, there is a pro-pedophilia side, and this is just another victory for them.
- kergonath 10mo ago> When the far-right paints trans people as pedophiles, it's not an accident that also provides cover for pedophiles. Projection. It’s always projection…
- mrtksn 10mo agoThat’s the right way, companies should not do the police job. Lock down those who make the machine generate the illegal content.
- monkaiju 10mo agoRemove the machine and charge it's creators for generation and distribution of CSAM
- mrtksn 10mo agowhy? how many Kodak executives were jailed? How many Nikon or Canon?
- bakies 10mo agoHow many of those execs generated CSAM?
- leosanchez 10mo agoWhat an obnoxious anology dude. Taking creepy pictures and asking a machine to create create creapy pictures for the world to see are not the same.
- mrtksn 10mo agoThe one with taking creepy pictures has real victims, the one with making the machine generate the picture doesn’t but it tells something about the character of the person who makes it generate so I’m fine with them punished. Either way making the machine provider do the policing is ridiculous.
- epistasis 10mo agoWhat ever happened to all that talk about Section 230 protections for platforms? It used to get a ton of discussion in the past, did something change?
- b40d-48b2-979e 10mo agoSection 230 only covers user-generated content. I imagine this gets dicey considering Grok is platform-owned and generating the content.
- mingus88 10mo agoThis will be interesting to see how it plays out. When do we cross the line of culpability with tool-assisted content? If I have a typo in my prompt and the result is illegal content, am I responsible for an honest mistake or should the tool have refused to generate illegal content in the first place? Do we need to treat genAI like a handgun that is always loaded?
- KaiserPro 10mo agoand in good faith knowingly allowing it is not in good faith.
- advisedwang 10mo agoI guess people draw a difference between a platform generating illegal content vs merely hosting illegal content uploaded by users.
- burkaman 10mo ago> No provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider. That's what section 230 says. The content in question here is not provided by "another information content provider", it is provided by X itself.
- staticassertion 10mo agoEven ignoring that Grok is generating the content, not users, I think you can still hold to Section 230 protections while thinking that companies should take more significant moderation actions with regards to issues like this. For example, if someone posted CSAM on HN and Dang deleted it, I think that it would be wrong to go after HN for hosting the content temporarily. But if HN hosted a service that actively facilitated, trivialized, and generated CSAM on behalf of users, with no or virtually no attempt to prevent that, then I think that mere deletion after the fact would be insufficient. But again, you can just use "Grok is generating the content" to differentiate if that doesn't compel you.
- mgdev 10mo agoIt's the perfect honeypot.
- _fzslm 10mo ago> That’s like blaming a pen for writing something bad,” DogeDesigner opined. Genuinely terrifying how Elon has a cadre of unpaid yes-men ready to justify his every action. DogeDesigner regularly sub tweets Elon agreeing to his latest dumb take of the day, and even seems to have based his entire identity on Elon's doge obsession. I can't imagine how terrible that self imposed delusion feels deep down for either of them.
- QuercusMax 10mo agoI thought DogeDesigner is Elon. That account has certainly posted things in the past that make it look like one of Elon's sockpuppets.
- ryandrake 10mo ago> Genuinely terrifying how Elon has a cadre of unpaid yes-men ready to justify his every action. A similar article[1] briefly made it to the HN front page the other day, for a few minutes before Elon's army of unpaid yes-men flag-nuked it out of existence. 1: https://news.ycombinator.com/item?id=46468414 https://news.ycombinator.com/item?id=46468414
- rsynnott 10mo agoI always assumed that one was a Musk sockpuppet. It's either that or the world's saddest person.
- pelorat 10mo agoJokes on xAI. Europe doesn't have a Section 230 and the responsibility fall squarely on the platform and its owners. In Europe, AI generated or photoshopped CSAM is treated the same as actual abuse-backed CSAM if the depiction is realistic. Possession and distribution are both serious crimes. The person(s) ultimately in charge of removing (or preventing the implementation of) Grok guardrails might find themselves being criminally indicted in multiple European countries once investigations have concluded.
- lovich 10mo agoIs Europe actually going to do anything? They currently appear to be puckering their assholes and cowering in the face of Trump, and his admin are already yelling about how the EU is "illegally" regulating American companies. They might just let this slide to not rock the boat, either out of fear and they will do nothing, or to buy time if they are actually divesting from the alliance with and economic dependence on the US
- torlok 10mo agoThere's so many of these nonsense views of the EU here. Not being vocal about a mental case president doesn't mean politicians are "puckering their assholes". The EU is not affraid to moderate and fine tech companies. These things take time.
- lovich 10mo agoUnder previous US admins and the relationship the EU had, yeah. The asshole puckering is from how Trump has completely flipped the table, everything is hyper transactional now, and as we’ve seen military action against leaders personally is also on the table. I’m saying I could see the EU let this slide now because it’s not worth it politically to regulate US companies for shit like this anymore. Whether that would be out of fear or out of trying to buy time to reorganize would probably end up in future getting the same kind of historical analysis that Chamberlain’s policy of appeasement to Germany gets nowadays
- roywiggins 10mo ago
- schnebbau 10mo agoWhat would happen if someone used photoshop to create CSAM? Should Adobe be held responsible because they didn't prevent it? Grok is just another tool, and IMO it shouldn't have guard rails. The user is responsible for their prompts and what they create with it.
- jeffwask 10mo agoSomeone spending 40 hours drawing a nude is not equivalent to someone saying take this photo and make them naked and having a naked photo in 4 seconds. Only one of these is easily preventable with guardrails.
- schnebbau 10mo ago[flagged]
- jeffwask 10mo agoNo, silly billy but the responsibility for when your SAAS platform is generating it falls on you as a developer. The user in not creating it you are based on a prompt you could easily say no to.
- theLegionWithin 10mo agobet I can guess which of those two is more profitable
- MallocVoidstar 10mo ago> Grok is just another tool, and IMO it shouldn't have guard rails. How is the world improved by an AI tool that will generate sexual deepfake images of children?
- deleted 10mo ago[deleted]
- SirMaster 10mo ago
- jeffwask 10mo agoNo one could have predicted this is how users would behave. /s
- praptak 10mo agoI don't buy the "I only provide the tool" cop out. Musk does control what Grok spews out and just chooses not to act in this case. When Grok stated that Israel was committing genocide, it was temporarily suspended and fixed[0]. If you censor some things but not others, enabling the others becomes your choice. There is no eating the cookie and having it too - you either take a "common carrier" stance or censor, but also take responsibility for what you don't censor. [0] https://www.france24.com/en/live-news/20250813-chatbot-grok-stirs-confusion-over-suspension-after-gaza-claims https://www.france24.com/en/live-news/20250813-chatbot-grok-...
- bakies 10mo agoHis take over of X has never been about "Free Speech" he just wants control of the speech.
- deleted 10mo ago[deleted]
- ethical_source 10mo agoIf you follow the "tool-maker is responsible for tool-use" thread of thought to its logical conclusion, you have to hold creators of open-weights models responsible for whatever people do with these models. Do you want to live in a world that follows this rule?
- praptak 10mo ago> "tool-maker is responsible for tool-use" You left out "who controls the output of the tool", which makes it a strawman.
- jnovek 10mo agoThe phrase “its logical conclusion” is doing a lot of heavy lifting here. Why on earth would that absurdity be the logical conclusion? To me it looks like a very illogical conclusion.
- UncleMeat 10mo ago
- dleslie 10mo agoAt this point it should be clear that they know that Grok is unsafe to use, and will generate potentially illegal content even without a clear prompt asking it to do so. This is a dangerous product, the manufacturer _knows_ it is dangerous, and yet still they provide the service for use.
- foxglacier 10mo ago[flagged]
- ribosometronome 10mo agoIt's hardly a hunt when there's bunch of warty green women with black tipped hats openly stirring a cauldron going "Double double toil and trouble." The world grew up and learned better. We've seen the effects that bullying, including cyberbullying, have on people. We've seen teenagers (and adults) get harassed with fake revenge porn.
- ebbi 10mo ago"pearl clutching nana" for simply stating CSAM is dangerous? I get the people arguing for free speech (in general) as there are lots of upsides. Where are the upsides in CSAM - whether real or computer generated? How does it benefit society?
- tredre3 10mo ago> Where are the upsides in CSAM - whether real or computer generated? How does it benefit society? In a public forum like X, there probably are no upsides. In general, though, pedophilia exists. This isn't something that is going to change. What is the harm in providing them with a alternative to real CSAM (which actually and actively hurts children)?
- mothballed 10mo agoGiving them no legal avenue allows us to put more of them in jail. Once they are in jail the odds of them molesting children goes from "possible very low but measurably above zero" to "~0." I think you are looking at it from the point of it being a punishment for victimizing someone -- when in fact it's used not to punish crime but to put away people who potentially might victimize someone in the future.
- dmitrygr 10mo ago[flagged]
- antonvs 10mo agoStrong “it’s not pedophilia, it’s ephebophilia” vibes. Certainly if the image started out as one of a real child, then abuse is inherent. See https://rainn.org/get-the-facts-about-csam-child-sexual-abuse-material/what-is-csam/ https://rainn.org/get-the-facts-about-csam-child-sexual-abus... : > CSAM includes both real and synthetic content, such as images created with artificial intelligence tools. A child cannot legally consent to any sexual act, let alone to being recorded in one.
- dmitrygr 10mo agoNo idea what "vibes" are. My question was a very clear one -- definition. If one draws someone in a bikini (the current Twitter craze), it fails the Roth test for unprotected obscene speech on point 2 ("Judged by what the average person in a particular community finds acceptable") -- nobody online today finds bikini photos unacceptable. It is thus constitutionally-protected free speech. If one prompts an ML model to do the drawing, how is it different?
- antonvs 10mo ago> No idea what "vibes" are. If your English is weak, there are dictionaries, translation programs, and LLMs that can help. The first meaning at https://www.merriam-webster.com/dictionary/vibe https://www.merriam-webster.com/dictionary/vibe is “a distinctive feeling or quality capable of being sensed,” which is the relevant one here. The OP article refers to “outputs that sexualized real people without consent.” If any of those real people are minors, that qualifies as CSAM. It’s not complicated.
- dmitrygr 10mo agoGiven that “put X into a bikini” is constitutionally protected speech, and the output fails the Roth test for obscenity and is thus the same, would not every other law be null and void w.r.t. stoping this? And “qualifies as X according to some unelected org with no lawmaking power” is even weaker than a law. You dislike it. I get it. I do too. But this is a discussion of law. Legally, I do not see how any law was broken. I welcome any citation to the contrary. I note, again, that "some unelected org said so" is not a weighty argument when the opposition is the SCOTUS's clear stance on the 1st amendment.
- wateralien 10mo agoElon Musk attends the Donald Trump school of responsibility. Take no blame. Admit no fault. Blame everyone else. Unless it was a good thing, then take all credit and give none away.
- 2OEH8eoCRo0 10mo agoTwitter should be liable no? The user didn't generate it, Grok did. No Section 230 protection
- batveal 10mo ago[flagged]
- ebbi 10mo agoKilling people in a game isn't a crime. Producing and disseminating CSAM (whether it's physical or digital copies) is a crime.
- mirabilis 10mo agoMost of the instances I’m seeing discussed on X are not “fictional depiction of nonexistent child” but instead “minor’s posted photo directly replied to with ‘grok, put her in a bikini covered in ‘donut glaze’’”, which, in my opinion, crosses moral bounds far beyond the scope of your theoretical lab-grown work of fiction
- ux266478 10mo agoMany territories prosecute for simulated sexual abuse of children, including drawn images. Most consider it obscene beyond simulated murder.
- Rzor 10mo agoAnd yet people will remain in that cesspool, hell be damned. Network effects are indeed a powerful force.
- UncleMeat 10mo agoThe number of people saying that it is not worthy of intervention that every single woman who posts on twitter has to worry about somebody saying "hey grok, take her clothes off" and then be made into a public sex object is maybe the most acute example of rape culture that I've seen in decades.
- jeffbee 10mo ago[flagged]
- trueismywork 10mo agoTaliban 2.0
- ux266478 10mo agoThe Taliban would have arrested and executed twitter users, twitter engineers and probably Musk himself for this nonsense.
- cogman10 10mo agoAh no, I don't think you understand how religious fundamentalism works in practice. For most fundamentalist religions, men are almost never penalized for bad behavior. It's nearly impossible to find a man being killed for violating a morality law including selling pornography or engaging in prostitution. But on the other hand, it's very easy to find examples of women getting stoned for dressing improperly.
- ux266478 10mo agoFirstly, what we are talking about here is not "selling pornography" or "engaging in prostitution". It's a public event of widespread obscenity being actively defended by those who made it possible. I can't take your comparison seriously as being good faith. Secondly, you posit it's "nearly impossible to find a man being killed for violating a morality law", which is true because of two factors. The first of which is because it's difficult to find any kind of representative sample of justice being meted out in Afghanistan because it's not freely and actively reported on. The second of which is because the punishment for violating moral laws is usually public flogging. The idea that these laws overwhelmingly target women is false, most of the time the people being punished for breaking morality laws are men: https://amu.tv/137185 https://amu.tv/137185 It is clear to me the only thing you know about Afghanistan is that women unfortunately live as second class citizens. This is clear not only because of the naive things you say, but because you explicitly fall back to painting with ridiculously unspecific brushstrokes. With your knowledge exhausted, you revert to talking about "most fundamentalist religions", despite the domain already being pretty well defined to specifically the Taliban. You shoehorn in the misogyny angle, as though that's relevant to the context and makes your point stronger, but it's just vacuous nonsense. Your entire point seems to be that the justice system in Afghanistan primarily punishes women (which is an silly falsehood in and of itself), and that's why a major public figure enabling mass obscenity would entail no consequences? Are you actually out of your mind? There's simply no way you actually believe this crap. I'm sorry, that kind of naivete is just too ridiculous to buy. The cherry on top is that you lead all of this crypto neo-orientalist shit with "I don't think you understand how religious fundamentalism works in practice". Give me a break.
- ascii0eks84 10mo agoIt's not a bug nor a vulnerability. It's a tool.
- aldousd666 10mo agoseems like blaming the users is the appropriate thing to do. just like blaming users for how they misuse guns and or photoshop.
- femiagbabiaka 10mo agoExcept in this case someone else owns the gun and they allow anyone on the planet (with an X account) to ask for someone else to be shot at will.
- roywiggins 10mo agoWould you blame only the users of a murder-for-hire service? Sure, yes, they are also to blame, but the murder-for-hire service would also seem to be equally culpable.
- ethical_source 10mo agoA prediction market is a murder-for-hire service. See https://www.lesswrong.com/posts/8e4QNySp4LjvdBstx/how-prediction-markets-can-create-harmful-outcomes-a-case https://www.lesswrong.com/posts/8e4QNySp4LjvdBstx/how-predic... Shall we ban prediction markets?
- roywiggins 10mo agoyeah, probably. life was fine without them
- Kreutzer 10mo agoYes, please.
- footy 10mo agoyes. It's obscene that they exist.
- BigTTYGothGF 10mo agoThat link claims to be a "case study" but I don't see any case being discussed?
- jmward01 10mo agoA core issue here is that there isn't a black and white on a subject like this. Yes, it is wrong. Yes they have a responsibility. But at the same time taking that to an extreme leads to heavy censorship. So what is a practical middle ground? Is there something like a 'sexualized validation suite' that could be an industry standard for testing if an LLM needs additional training? If there were then victims could potentially claim negligence if they aren't using best practices and they were harmed because of it right? Are there missing social or legal mechanisms to deal with misuse? One thing I think is missing is a '911' for cyber offenses like this. If someone breaks into my house I can call 911, if someone creates revenge porn who do I call? I don't think there is a simple answer here, but constructive suggestions, suggestions that do balance free speech and being a responsible service provider would be helpful. 'They are wrong' doesn't actually lead to change.
- jmward01 10mo agoLooks like this hit a nerve. Any comments on the practical solutions though? The comment wasn't advocating that they should make CSAM or that they shouldn't face repercussions for enabling it, at least I don't think it reads that way. I honestly think that a core issue here is we are missing practical fixes. Things that make it easier for victims to get relief and things that make it clear that a provider is being irresponsible so that they can face civil or criminal penalties. If there aren't solid industry standards then how can you claim they aren't implementing best practices to hold them accountable? If victims don't have effective means of relief then how will we find and stop this? I'd love to hear actual concrete actions that the industry can put in place. 'Just tell them to stop' doesn't create a framework that leads to change.
- SpicyLemonZest 10mo agoThe reason it hit a nerve is that you're just being extraordinarily credulous about xAI's lies. There are solid industry standards, and we can just tell them to stop; we know this because Grok has a number of competitors which don't generate CSAM. Indeed, they've already implemented the industry standards which prevent CSAM generation; they just added a flag called "spicy mode" to turn them off, because those standards also prevent the generation of pornographic images.
- metalman 10mo agoXchan
- whatever1 10mo ago[flagged]
- deleted 10mo ago[deleted]
- SilentM68 10mo agoWhen I generate content on most AI's including Grok, I ask it to fashion a prompt first of the subject I want and ask it to make sure that it does not violate any TOS or CSAM policies. I also instruct it that the prompt should be usable by most AIs. It fashions the prompt. When I use the prompt, the system complains that the prompt violates the TOS. I then ask the AI to locate the troubling aspect of the prompt. It says that it has and provides an alternative, safer prompt. More often than not, this newer prompt is also flagged as inappropriate. This is very frustrating even when the original intent is not to create content that violates any public AI policy. From my experience, both users and the technology make mistakes.
- simianparrot 10mo agoSo let me get this straight. When people use these tools to steal artist’ styles directly to generate fake Ghibli art, then it’s «just a tool, bro». But when it’s used to create CSAM, then it’s suddenly not just a tool. You _cannot_ stop these tools from generating this kind of stuff. Prompt guards only get you so far. Self-hosted versions don’t have them. The human writing the prompt is at fault. Just like it’s not Adobe’s responsibility if some sick idiot puts bikinis on a child in Photoshop.
- jl6 10mo agoSeems like a new application of 2nd amendment logic (both for and against).
- AndrewGaspar 10mo agoGranting that I think X should have stronger content policies and technological interventions to bad behavior as a matter of business, I do think that the X Safety's team position[0] is the only workable legal standard here. Any sufficiently useful AI product will _inevitably_ be usable, at minimum via subversion of their safety controls, to violate current (or future!) laws, and so I don't see how it's viable to prosecute legal violations at the level of the AI model or tool developers, especially if the platform is itself still moderating the actually illegal content. Obviously X is playing much looser with their safety controls than their competitors, but we're just debating over degrees rather than principles at that point. [0] > Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content.
- paxys 10mo agoI have a very hard time understanding the business case for xAI/Grok. It is supposedly worth $200 billion (at least by Silicon Valley math), putting it in the company of OpenAI and Anthropic, but like...Who is using it? What is it good for? Is it making a single dollar in revenue? Or is the whole thing just "omg Elon!!" hype similar to most of his other endeavors?
- catgirlinspace 10mo agoOnly use I've seen is people replying to basically every post on twitter with "@grok explain this", so likely the latter.
- rsynnott 10mo ago> Or is the whole thing just "omg Elon!!" hype similar to most of his other endeavors? Yes, but combined with "omg AI" (which happened elsewhere; for instance, see the hype over OpenAI Sora, which is clearly useless except as a toy), so extra-hype-y.
- imoverclocked 10mo agoSeems odd that they “fix” grok to say only positive things about its owner but then can’t be bothered to control such a topic.
- lynndotpy 10mo agoTo make this explicit: They are able to change how Grok is prompted to deny certain inputs, or to say certain things. They decided to do so to praise Musk and Hitler. That was intentional. They decided not to do so to prevent it from generating CSAM. X offering CSAM is intentional.
- handoflixue 10mo agoGrok will shit-talk Elon Musk, and it will also put him in a bikini for you. I've always found it a bit surprisingly how little control they seem to have there.
- llmslave2 10mo agoI find it fascinating to read comments from a lot of people who support open models without guardrails, and then to read this thread with seemingly the opposite sentiment in overwhelming majority. Is it just two different sets of users with differing opinions on if models should be open or closed?
- Ajedi32 10mo agoAny mention of Musk on HN seems to cause all rational thought to go out the window, but yeah I wonder in this case how much of this wild deviation from the usual sentiment is attributable to: 1. Hypocrisy (people expressing a different opinion on this subject than they usually would because they hate Musk) vs. 2. Selection bias (article title attracts a higher percentage of people who were already on the more regulation, less freedom side of the debate) vs. 3. Self-censorship (people on the "more freedom, less regulation" side of the debate being silent or not voting on comments because in this case defending their principles would benefit someone they hate) There might be other factors I haven't considered as well.
- llmslave2 10mo ago3 seems likely, especially since defending their views in this particular case opens them up to some pretty nasty insults. Also I think a lot of people simply think models which are published openly shouldn't be held to the same legal standards as proprietary models.
- thrance 10mo agoGee, I wonder why people would take offense at an AI model being used to generate unprecedented amounts of CSAM from real children, or objectify millions of women without their consent. Must be that classic Musk Derangement Syndrome. The real question is how can the pro-Musk guys still find a way to side with him on that. My leading theory is that they're actually pro-pedophilia.
- theLegionWithin 10mo agomaybe they are. what's anyone going to do about it? complaining on forums? best of luck with that
- gbnwl 10mo agolol. Always fun to watch HN remove hightly relevant topics from the top of the front page. To their credit they usually give us about an hour to discuss before doing so. How kind of them.
- zelphirkalt 10mo agoGreat, can we finally get X blocked in the EU then? Far too many people are still hooked to toxic content on that platform, and it is owned by an anti-EU, right-extreme, nazi-salute guy, who would love nothing more than seeing the EU fail.
- softwaredoug 10mo agoI can’t create a Sora video that depicts Elon Musk. But X can’t figure out how to auto-filter CSAM?
- sebow 10mo agoIf you post pictures of yourself on X and don't want grok to "bikini you", block grok. Yes, under the TOS, what grok is doing is not the "fault" of grok(the reason is the causal factor of the post[enabled by 2 humans: the poster and the prompter]; the human intent is what initiates the generated post, not the bot; just like a gun is shot by a human, not by the strong winds). You could argue it's the fault of the "prompter", but we're going to circle back to the cat & mouse censorship issue. And no, I don't want a less censored grok version that's unable to "bikini a NAS"(which is what I've been fortunate to witness) just because "new internet users" don't understand what the Internet is.(Yes, I know you can obviously fine-tune the model to allow funny generations and deny explicit/spicy generations) If X would implement what the so-called "moralists" want, it will just turn into Facebook. And for the "protect the children" folks, it's really disappointing how we're always coming back to this bullsh*t excuse every time a moral issue arises. Blocking grok is a fix both for the person who doesn't want to get edited AND the user who doesn't want to see grok replies(in case the posts don't get the NSFW tag in time). Ironically, a decent amount of people who want to censor grok are bluesky users, where "lolicorn" and similar dubious degenerate content is being posted non-stop AS HUMAN-MADE content. Or what, just because it's an AI it's suddenly a problem? The fact that you can "strip" someone by tweeting a bot? And lastly, sex sells. If people haven't figured out that "bikinis", "boobs", and everything related to sex will be what wins the AI/AGI/etc. race (it actually happens for ANY industry), then it's their problem. Dystopian? Sure, but it's not an issue you can win with moral arguments like "don't strip me". You will get stripped down if it created 1M impressions and drives engagement. You will not convince Musk(or any person who makes such a decision) to stop grok from "stripping you", because the alternative is that other non-grok/xAI/etc. entities/people will make the content, drive the engagement, make the money.
- deleted 10mo ago[deleted]
- KoolKat23 10mo agoI don't know, I feel like I'm taking crazy pills with this whole saga. Perhaps I havent seen the full story. The fact of the matter is they do have a policy and they have removed it, suspended accounts and perhaps even taken it further. As would be the case on other platforms. As far as I understand there is no nudity generated by grok. Should public gpt models be prevented from generating detestable things, yes I can see the case for that. I won't argue there is a line between acceptable and unacceptable, but please remember people perv over less (Rule 34). Are bikinis now taboo attire? What next, ankles, elbows, the entire human body?(Just like the Taliban). (Edit: I'm mentioning this paragraph for my below point.) GPT's are not clever enough to make the distinction by the way either, so there's an unrealistic technical challenge here. I suspect the this saga blowing out of proportion is purely "eLoN BAd".
- KaiserPro 10mo agoNice logical fallacies. > As far as I understand there is no nudity generated by grok. There is nudity, and more importantly there is CSAM material being generated. reference: https://www.reddit.com/r/grok/comments/1pijcgq/unlocking_groks_imagine_for_nudes_and_spicy_shit/ https://www.reddit.com/r/grok/comments/1pijcgq/unlocking_gro... > Are bikinis now taboo attire? generating sexualised pictures of kids is verboten. Thats epstien level of illegality. There is no legitiamte need for the public to hold, make or transmit sexualised images of children. Anyone arguing otherwise has a lot of questions to answer
- KoolKat23 10mo agoYou're the one making the logical fallacies and reacting emotionally. Read what I have said first please. That is a different grok to the one publishing images and discussed in the article. Your link clearly states they are being moderated in the comments and all comments are discussing adults only. The links comments also imply that these folks are jailbreaking nearly, because of guardrails that exist too. As I say read what I said, please don't put words in my mouth. The GPT models wouldn't know what is sexualised. I said there is a line at some point. Non-sexualized bikinis are sold everywhere, do you not use the internet to buy clothes? Your immediate dismissive reaction indicates you are not giving what I'm saying any thought. This is what puritanical thought often looks like. The discourse is so poisoned people can't stop, look at the facts and think rationally.
- bgbntty2 10mo agoIf it's AI-generated, it should be legal - regardless of whether the person consented for their image to be used and regardless of the age of the person. You can't have AI-generated CSAM, as you're not sexually abusing anyone if it's AI-generated. It's better to have AI-generated CP instead of real CSAM because no child would be physically harmed. No one is lying that the photos are real, either. And it's not like you can't generate these pics on free local models, anyway. In this case I don't see an issue with Twitter that should involve lawyers, even though Twitter is pure garbage otherwise. As to whether Twitter should use moderation or not, it's up to them. I wouldn't use a forum where there are irrelevant spam posts.
- CrzyLngPwd 10mo agoWhy does anyone NEED grok to generate images on X? Seems like a toy feature.
- wnevets 10mo agoIts a feature, not a bug. It fits perfectly for the type of users that still use twitter.
- theLegionWithin 10mo agois it working?
- uberdru 10mo agoThen, there's "VC twitter". . . .
- lawlessone 10mo agoTwitter has turned into 4chan with paid accounts.
- wiredpancake 10mo ago[dead]
- pier25 10mo agoMaybe users are responsible for the prompts but X is definitely responsible for hosting that content and the behavior of Grok.
- gigatexal 10mo agoThe irony. Musk fumes about pedo leftist weirdos. And then his own ai bot creates CSAM. The right are full of hypocrites and weirdos compensating so so very hard.
- Hard_Space 10mo agoWhy was this flagged, after so many contributions and so much interest?
- AlecSchueler 10mo agoGrok serves the patriarchy and this is an overwhelmingly male forum.
- rsynnott 10mo agoNaughty Old Mr Car's fans are triggered by any criticism of Dear Leader. This is actually separate to hn's politics-aversion, though I suspect there's a lot of crossover. Any post which criticised Musk has tended to get rapidly flagged for at least the last decade.
- Cub3 10mo agoThe line you walk past is the one you accept, x accepts CSAM
- AlecSchueler 10mo agoOk I understood when stuff related to dodge was consistently flagged for being political and not relevant to hacking but... This is surely relevant to the audience here, no?
- iso1631 10mo agoThe entire VC industry is political and very much on one side. Anything that raises questions about that needs to be buried
- esalman 10mo agoIf "cesspool" was a social media it would look like 2026 twitter.