7 ms·
X Didn't Fix Grok's 'Undressing' Problem. It Just Makes People Pay for It
- SilverElfin 10mo agoIt also made it harder to track. One way to see what Grok is doing is to look at the Grok account’s replies. So you can see the image it generates in response to someone - for example - undressing a woman who appears in a photo. You can then go visit THAT thread to see what the exchange with Grok was, which often would show a long series of lewd images. A few days ago, nearly the ENTIRE stream of the Grok account’s replies at any moment were deepfakes without consent. Mostly in response to women’s posts, but sometimes to generate racist attacks. I’m not against people using AI to generate a fantasy image for their own needs. I guess in a way it’s like what people imagine in their own heads anyways. But I do think it is problematic when you share those publicly because it can damage others’ reputation, and because it makes social media hostile to some groups of people who are targeted with misogynist or racist deepfakes. It may seem like a small problem but the actual final effect is that the digital public square becomes a space only for identity groups that aren’t harassed.
- chrisjj 10mo ago[flagged]
- SilverElfin 10mo agoYea - like if you have someone depicted in some negative way or compromising way.
- chrisjj 10mo agoI think that harms reputation only if it is believed true.
- themafia 10mo agoWell, grok just automates what you can do by hand if you want, and there's not much to stop me just drawing out these same types of images manually if I want. The problem is that doing this would get me banned. Shouldn't using Grok in this way get you banned similarly?
- mullingitover 10mo agoWhy? The people creating and operating the CSAM/revenge porn/depfakes creation and distribution platform are the ones who are culpable. The users who are creating text prompts are just writing words. There’s a frantic effort to claim 230 protection, but this doesn’t protect you from the consequences of posting content all by yourself on the site you own and control.
- themafia 10mo ago> the CSAM/revenge porn/depfakes creation and distribution platform Which, in this case, is Twitter itself, no? > The users who are creating text prompts are just writing words. With highly specific intentions. It's not as if grok is curing cancer. Perhaps it's worth throwing away this minor distinction and considering the problem holistically.
- mullingitover 10mo ago> With highly specific intentions Intentions to pull the CSAM out of the server full of CSAM that twitter is running. Yes, you are making the flailing argument that the operators of the CSAM site desperately want to establish as the false but dominant narrative. If you have a database full of CSAM, and investigators write queries with specific intentions, and results show that there is CSAM in your database: you have a database full of CSAM. Now substitute 'model' for 'database.'
- themafia 10mo agoGrok enables their behavior. An investigator does not _create novel child porn_ in doing a query. You're making a fallacious argument.
- bediger4000 10mo ago[flagged]
- miltonlost 10mo agoHow does a "market-based solution" work with ending grok peddling a child pornography and revenge porn generator?
- i80and 10mo ago[flagged]
- RankingMember 10mo agoI know you're joking but I have seen genuine comments like this!
- mullingitover 10mo agoYou’re going to be excited to learn that there are private for-profit prisons in many of the jurisdictions where X executives and employees are committing the felony of creating and distributing CSAM and explicit deepfakes.
- buellerbueller 10mo ago[flagged]
- nailer 10mo agoFrom Wired's original article, at https://archive.is/https://www.wired.com/story/grok-is-pushing-ai-undressing-mainstream/ https://archive.is/https://www.wired.com/story/grok-is-pushi... > Every few seconds, Grok is continuing to create images of women in bikinis or underwear in response to user prompts on X, according to a WIRED review of the chatbots’ publicly posted live output. On Tuesday, at least 90 images involving women in swimsuits and in various levels of undress were published by Grok in under five minutes, analysis of posts show. ChatGPT and Gemini also do this: https://x.com/Marky146/status/2009743512942579911?s=20 https://x.com/Marky146/status/2009743512942579911?s=20
- samesamebut 10mo ago[flagged]
- hairofadog 10mo agoReal question as I don't use ChatGPT or Gemini: They publish images of women in bikinis or underwear in response to user prompts? Where do they publish them? I'm looking at Gemini and I don't see any sort of social aspect to it. I just tried the prompt "picture of a dog" and I don't see any way that another person could see it unless I decided to publish it myself.
- plagiarist 10mo agoFor this particular one it seems to be that you @grok under a posted image with a request for modifications and that account posts the modified image as a reply.
- hairofadog 10mo agoRight, that seems to me like an important distinction. Other people in this thread have said things like "Well you can draw people in a bikini with a pencil without their permission! Should we ban pencils, too!?" Honestly if someone wants to be weird and draw bikini pictures of journalists they don't like AND KEEP IT TO THEMSELVES, then whatever I guess. That's not what this is. Grok is creating the images. Grok is publishing the images. Grok is harrassing the subjects of the images by posting it in their replies. Neither ChatGPT, Gemini, nor pencils are doing that. (And that doesn't even get into the CSAM aspect.) One of the many reasons I prefer Claude is that it doesn't even generate images.
- kgrizel 10mo agoThe same old misleading headline yet again. Grok continues to be free, whatever that X message said, it had so far no impact on Grok web and mobile app. They just switched off the part of their integration with X that let people generate images. You can of course pay for Grok if you like, but that just buys you bigger quota (up to 50 videos a day is free), not new capabilities or less censorship.
- standardUser 10mo agoHow long can we keep trying to put a finger in this particular dike? In five years, most people will be able to run a local LLM capable of whatever nefarious purposes they choose.
- dmitrygr 10mo ago[flagged]
- lenerdenator 10mo agoThat's their business. If there's a business operating for profit (and Twitter is, ostensibly) and their tool posts pictures of me undressed, then I am going to have a problem with it. And I'm just some dude. It probably means a lot more for women who are celebrities. "It's inevitable" isn't an excuse for bad corporate or personal behavior involving technology. Taken to its logical conclusion, we're all going to die, so it's just executing on the inevitable when someone is murdered.
- standardUser 10mo agoThe only excuses for bad corporate behavior are bad corporate laws and weak enforcement.
- lenerdenator 10mo agoThat's often because of regulatory capture.
- yndoendo 10mo agoHow do you get proper laws passed when the politicians are bought and paid for by the same corporations? In the USA ... a company can declare bankruptcy and shed the debt / liabilities while a person cannot shed most debt after declaring bankruptcy. [0] [1] USA politicians favor companies over the people. I personal support new corporate laws similar to California's three strikes law. Instead of allow companies to budget for fines the CEO and Executives go to jail with the corporation being broken up after habitually breaking same the laws. [0] https://hls.harvard.edu/today/expert-explains-how-companies-are-using-a-controversial-bankruptcy-maneuver-to-handle-mass-tort-claims/ https://hls.harvard.edu/today/expert-explains-how-companies-... [1] https://thenewpress.org/books/unjust-debts/ https://thenewpress.org/books/unjust-debts/
- wnevets 10mo agoSaying cisgender is bad on Twitter but CSAM is not. Very weird.
- Zigurd 10mo agoFits the pattern. One of the two F-words will get your post flagged faster than the other.
- pesus 10mo agoMakes complete sense when you view it through the lens of Musk's opinions.
- ben_w 10mo agoEven given Musk's opinions, he clearly understands that the general public doesn't want kids getting hurt, he demonstrated this by saying Trump was in the Epstein files and by repeatedly saying the UK government isn't doing enough to stop child abuse and opining about a UK civil war. His hypocrisy, his position on the main-character-syndrome-to-narcissism spectrum, him getting a kick out of trolling everyone, or him having straight up psychopathy: whatever it is, I find I no longer care.
- nullocator 10mo ago> he clearly understands that the general public doesn't want kids getting hurt This may be giving him too much credit, the only thing we actually know is he thinks being accused of being a pedophile is bad. We know this because he's done it to several people, and flips his shit when it happens to him or his platform. He doesn't actually seem to care about pedophiles or pedophilia given his on going relationships with people he's accused.
- labrador 10mo ago[flagged]
- bayarearefugee 10mo ago> It's a symptom of rich people thinking the rules don't apply to them. I would argue it's a symptom of rich people knowing the rules don't apply to them. Arguably the rules never really have applied to them, but now they don't even bother to pretend they do.
- labrador 10mo agoYou're right of course. Thanks for the correction.
- nemomarx 10mo agoGrok generates photorealistic imagery of young girls too, so I don't think the anime distinction is the main one here. He might think ai generated photos aren't as real, yeah. But I think mostly musk just acts like all laws don't apply to him - regulations, property lines, fines, responsibility to anyone else.
- porphyra 10mo agoObviously I think that AI generated undressing pictures of people, especially minors, is bad and there should be safeguards against that. But how is it different from other tools like doing it manually with photoshop? Also it has been shown that many other tools like ChatGPT and Gemini/Nanobanana can also do it with sufficiently creative prompting. I also did scroll through the public grok feed and the AI generated bikini pics were mostly Onlyfans creators requesting their own fans to generate these pictures (or sometimes generating them themselves).
- kranke155 10mo agoGrom is much much less censored on purpose. I work in image editing and outside of very few people, hardly anyone uses Grok for professional work. Nano Banana Pro is used for the most part. But for NSFW work it dominates. It’s clearly deliberate.
- etchalon 10mo ago"Technically, anyone can make napalm at home. What's wrong with Walmart selling it?"
- pphysch 10mo agoThe obvious problem is that Grok is also distributing the illegal images. This could be easily fixed by making the generated images sent through private Grok DMs or something, but that would harm the bottom line. Maybe they will do that eventually once they have milked enough subscriptions from the "advertising".
- simianwords 10mo agowhy does it matter if grok is advertising or you are advertising? in reality there's no difference. its just a tool you can invoke.
- kllrnohj 10mo ago> But how is it different from other tools like doing it manually with photoshop? Last I checked Photoshop doesn't have a "undress this person" button? "A person could do bad thing at a very low rate, so what's wrong with automating it so that bad things can be done millions of times faster?" Like seriously? Is that a real question? But also I don't get what your argument is, anyway. A person doing it manually still typically runs into CSAM or revenge porn laws or other similar harassment issues. All of which should be leveraged directly at these AI tools, particularly those that lack even an attempt at safeguards.
- dpc050505 10mo agoAnother question that should be asked is what culturally drives people to want to create lewd content of children and what should we change so that it stops happening? Obviously platforms should have safeguard against child porn and misogynistic defamation, but as a society we also need cultural changes so that people don't become pedophiles and sexists and that the ones that do get help with their glaring issues.
- BrenBarn 10mo agoI don't think we're ever going to be able to eliminate various sorts of harmful desires and urges.
- XorNot 10mo agoWell I'd say one obvious solution would be to punish a company which deploys a tool to enable turn key sexual harassment at scale. Might send a good message about consent ya know?
- rstupek 10mo agoI think that's a naïve idea if you think punishing a company will have any affect on the situation.
- array_key_first 10mo agoIt will tangibly lead to less CSAM on the internet, so yeah it will have an affect. Obviously we can't just - poof - make people not child molesters or not murderers. But that doesn't mean we should sit on our asses and do nothing.
- buellerbueller 10mo agoPunish Musk; charge him with one count of producing CSAM per generated CSAM image. Charge him with one count of sexual harassment per nonconsensually generated and shared image. Do this in state court, so it is not pardonable. He is hosting a rape room. He has the power to stop it, but does not.
- ryandrake 10mo agoThe constant comparisons with Photoshop are so disingenuous. We all know what the difference is. If Adobe had a service where you could E-mail them "Please generate and post CSAM for me" and in response, their backend service did it and posted it, that's a totally different story then the user doing it themself in Photoshop. Come on. We all know about tech products here, and we can all make this distinction. Grok's interface is not "draw this pixel here, draw this pixel there." It's "Draw this child without clothing." Or "Draw this child in a bikini." Totally different.
- deleted 10mo ago[deleted]
- simianwords 10mo agohow is it different? i don't get it.
- nkrisc 10mo agoIf curl is used by hackers in illegal activity, culpability falls on the hackers, not the maintainers of curl. If I ask the maintainers of curl to hack something and they do it, then they are culpable (and possibly me as well). Using Photoshop to do something doesn’t make Adobe complicit because Adobe isn’t involved in what you’re using Photoshop for. I suppose they could involve themselves, if you’d prefer that.
- simianwords 10mo agoso why is the culpability on grok?
- deleted 10mo ago[deleted]
- deleted 10mo ago[deleted]
- 10mo ago
- charcircuit 10mo ago[flagged]
- tantalor 10mo agoBad faith comment, intentionally misunderstanding the scope and nature of the problem.
- int32_64 10mo agoIs there any way to provide a service where an image manipulation bot is mentioned in social media replies and it doesn't lead to total chaos? From what I saw the 'undressing' problem was the tip of the iceberg of crazy things people have asked Grok to do.
- ben_w 10mo ago> Is there any way to provide a service where an image manipulation bot is mentioned in social media replies and it doesn't lead to total chaos? It may be a failure of imagination on my part, but I can't imagine a bot limited to style transfer or replacing faces with corresponding emoji would cause total chaos. Even if someone used that kind of thing with a picture from an open-casket funeral, it would get tuts rather than chaos. > From what I saw the 'undressing' problem was the tip of the iceberg of crazy things people have asked Grok to do. Indeed. I mean, how out of touch does one have to be to look at Twitter and think "yes, this place will benefit from photorealistic image editing driven purely by freeform natural language, nothing could go wrong"?
- tantalor 10mo agoFor anyone unaware/uninformed, take a look at https://www.reddit.com/r/grok https://www.reddit.com/r/grok Warning: it's quite gross
- zimpenfish 10mo agoWorth noting that, in the UK, if there's CSAM on that page, you run a strong risk of prosecution[0] [0] https://www.cps.gov.uk/prosecution-guidance/indecent-and-prohibited-images-children https://www.cps.gov.uk/prosecution-guidance/indecent-and-pro... ("downloading an image from a website onto a computer screen: R v Smith; R v Jayson [2003] 1 Cr. App. R. 13")
- api 10mo agoWhat happens if you ask it to undress Elon Musk? Not saying someone with a Xhitter account to burn should do this, but not not saying they should do it.
- skrebbel 10mo agoI'm only on Twitter once every 2 months but last time I checked the place was absolutely overflowing with images of Musk in a bikini.
- cmxch 10mo agoOr to give Kier Starmer a Borat-esque outfit?
- shepherdjerred 10mo agoElon already did it to himself on his account
- api 10mo agoOh well, hey… at least he did that.
- option 10mo agoThis very clearly violates both Apple's App store and Google's Play store rules. Why is X still on the app stores?
- netsharc 10mo ago"Dear Email Recipient, I am a South African Billionaire with friends in powerful places..."
- bayarearefugee 10mo agoI agree with the sentiment, but nobody even needs to make these sort of threats or asks anymore. It is all a well-defined implicit caste hierarchy at this point and anyone with enough net worth and a willingness to publicly fellate the orange dong gets a protected spot on the 2nd tier of the pyramid.
- netsharc 10mo agoSarah Chayes studies corruption: https://www.newyorker.com/magazine/2015/01/19/corruption-revolt https://www.newyorker.com/magazine/2015/01/19/corruption-rev... In her book she writes how in Egypt, to get a job as police, you need to bribe your boss and pay a monthly tribute. Being a cop was a chance to join the Mafia squeezing the public for bribes. Your boss pays his boss, and so on to the top. So when the public protested against the regime, the police were especially brutal against them, because all the cops were in on the scam and wanted to preserve it/if the regime collapses their necks are also on the line. Looking forward to the ICE thugs beating and shooting Americans who want to bring down the current lawless regime... Or is that already happening? Trump hires fuckwits like Noem or Hagseth who (probably) knows they're dumb fucks who would never otherwise be in their positions, and they would suck any dick, metaphorical or even perhaps real mushroom-sized ones, to remain in power.
- drooby 10mo agoSpecial treatment for big players. I have seen full blown porn on instagram too. Ads. Porn ads. They look exactly like porn ads on porn websites.
- elric 10mo agoWasn't this entirely predictable and inevitable? The genie is out of the bottle. Where can we realistically draw the line? Preventing distribution of this sort of shit is impossible, anyone can run their own generator. CSAM is already banned pretty much everywhere, and making money off it certainly is, but somehow Musk is getting away with distributing it at a massive scale. Is it because it's fake? And can we even tell whether it's still fake? Do we ban profiting from fake porn? Do we ban computing? Do we ban unregulated access to generative AI? X/Grok is an attractive obvious target because it's so heinous and widespread, but putting the axe on them won't make much of a difference.
- Jordan-117 10mo agoHow about we start at "not enabling users to directly generate nonconsensual porn of other users using your platform and then posting it as a reply to their content"?
- nailer 10mo agoDoes any platform prevent that? HN doesn’t.
- roryirvine 10mo agoHN will ban accounts that do that. Grok hasn't been banned from Twitter.
- nailer 10mo agoWould you (and others) be happier if when someone asked Grok to post something, it came from the person's own account? If so we agree.
- Jordan-117 10mo agoI said generating it using the platform. These creeps aren't just freelancing -- the images are being created by Twitter's servers via their native UI and published directly by their official @Grok account. They are utterly complicit in a way they wouldn't be if they were "just" permitting people to post AI revenge porn they created independently.
- kylecordes 10mo agoObviously anybody can post gross things by running an image generation/editing tool locally and publishing the results. People then mostly blame the poster whose name it then appears under. Seems like a pointless and foolish product design error for X/grok to publish arbitrary image generation results under its own name. How could you expect that to go anything but poorly?
- gizmo686 10mo agoIt's not just a matter of publishing it under its own name. It also massively reduced the friction to do so compared to needing to run the image through an external tool and upload the result. That friction would greatly reduce the number of people who do it.
- scoofy 10mo agoIn the US, it used to be that if you made credible threats against people you could/would be prosecuted. Social media made it so common in that no district attorney goes to the trouble of actually finding prosecuting people for doing this. We can expect the same level of institutional breakdown with regards to various types of harassment, misappropriation, libel, and even manufactured revenge porn from AI.
- classified 10mo agoNow the credible threats are being made by Trump fans, and those are protected from criminal prosecution by his long shadow.
- nickthegreek 10mo agoIt’s even worse as the requestor doesn’t vet and approve the image. That seems to have removed editorial control from the requestor. This bot could also mess with user who are not trying to do bad thing X, but the black box bot decides to throw in some offputting stuff and then also associate your name with it. I keep coming to the same conclusion with X as they did in the 80’s masterpiece War Games.
- porridgeraisin 10mo agoThere is a real reason though. Limiting it to verified users is the easiest way to have a KYC on everyone generating images. That way they can respond to legal requests with the KYC of the account that asked grok to undress a minor.
- amai 10mo agoMusk is a business genius. He found the only way to make money with GenAI: Porn!
- datsci_est_2015 10mo agoThe through line of the success many AI ventures seems to be the shifting of liability, sometimes towards the consumer, sometimes to a third party, and sometimes to the nebulous void. With self-driving cars, companies appear to be offloading the liability of their own software to the driver (“it’s not _fully_ self driving, hands on the wheel!”), or even pedestrians (“wasn’t in the crosswalk”). This case appears no different. The thought experiment would be, for me at least, is if Grok was just any other user that was somehow capable of creating these images at scale? Who would be liable, the image prompter or the image generator? Or how about a system that asked the user to approve or disapprove of the image being posted before it’s posted? Or how about a system that posted the generated image under the prompters username, and not Grok’s?
- pickleglitch 10mo agoAnother post about X's child porn bot flagged as soon as it got any traction. See also: https://news.ycombinator.com/item?id=46548451 https://news.ycombinator.com/item?id=46548451 https://news.ycombinator.com/item?id=46503199 https://news.ycombinator.com/item?id=46503199
- IAmGraydon 10mo agoMusk and Trump have clearly talked about this idea that any attention is good attention because they both have started doing it to an extreme in the last year. Its like it doesn't matter how depraved or idiotic an idea is, the more outrage it causes, the better it is. To them, it's the only metric that matters at all.