5 ms·
Sora might have a 'pervert' problem on its hands
- panny 1y agoIsn't generating fetish content a legitimate business model? Doesn't anyone remember the song, >The internet is for porn I actually think this is what's going to happen with AI once the easy money dries up. They'll quickly race to the bottom selling porn generators. AI slop porn already seems like the majority usage after homework generation.
- JKCalhoun 1y agoDon't even bother to look at r/grok. When you have a waifu anime avatar… yeah, it went exactly where you would expect it to. Apparently XAi is reigning in the NSFW and people on the sub are livid.
- redwood 1y agoWell thanks for that rabbit hole... 20 mins I'll never get back :P funny tho
- QuadmasterXLII 1y agoThe article is describing a feedback loop: very few women consent to have their face be usable -> the number of perverts vastly outnumber the number of women so each woman, who has posted nothing nsfw, or even suggestive, gets dedicated attention from many perverts -> other women aren’t comfortable allowing their faces to be used by large numbers of perverts-> very few women consent to have their faces be usable. This is all independent of what is and isn’t a legitimate business model, it’s a social dynamic. It’s also a pretty familiar one: it shows up everywhere from nightclub bouncer policies to the dynamics of early 2000s irc rooms
- panny 1y agoYou can also use photoshop to glue a woman's head on a porn star's body. The results are about the same as AI slop. AI is just faster.
- threatofrain 1y agoThe number of perverts vastly outweigh the number of women? Um?
- frumplestlatz 1y ago> I've allowed anyone to make "cameos" using my face. (You don't have to do this …) … and this is probably where the article should’ve ended. Or in fact, where the author should’ve realized there didn’t need to be an article at all. People are weird and gross. They do weird things that we would often prefer they not. Sora provided a tool to avoid that weirdness. Use it.
- GaryBluto 1y agoAll evidence points to the author trying to make this a moral panic, especially with the emphasis on "real women" being used to generate these (despite the feature meaning they have consented to it)
- JKCalhoun 1y agoWill you be a little outraged though when they use real women without their consent? If we know anything about software in general (and AI specifically) getting around roadblocks is often a fairly simple thing.
- panny 1y ago>Will you be a little outraged though when they use real women without their consent? Probably not. AI slop doesn't really "go viral" except when it is super ridiculous like shrimp Jesus. Most people generating AI slop porn are likely in the 10s of people who will see it. If someone generates porn with my face on it and I never even know, how does this harm me? Why should I care?
- rafterydj 1y agoThis seems like you are not fully thinking this through, either intentionally or not. Benefit of the doubt, I will add to your question about why you should care with more questions: what if it's someone you know? what if they want to then sell the generated content? what if it was your political enemies? what if it was your boss? what if it was someone who was stalking you or made you feel unsafe?
- userbinator 1y agoBut the idea that someone was making a video that had some potential sexual gratification element made me feel fairly icked out. Clearly the author has never visited 4chan... but I think seeing others make such content with your appearance should be taken as flattery. More seriously, I hope this flood of easy video generation will cause people to more easily realise how they can be persuaded, and increase skepticism of evidence in general.
- exasperaited 1y ago> but I think seeing others make such content with your appearance should be taken as flattery. I think maybe you're an adult male. Having actually talked to some female friends about this, I'm pretty sure that women in general don't take so well to the idea of tools that might be used to encourage the fantasies of the men that already have a dangerous interest in harming them sexually. Whether the Valley thinks that's their problem to solve, I doubt. But making a joke out of it is pretty fucked up, dude. ETA: even women who have done some modelling and are a bit more aware of the way those images are used are at least somewhat concerned about content that can make them act and speak like puppets. This is at least as much about consent as it is about content. ETA2: I am rate-limited for being an argumentative sod in the past so I will finally edit this to note that 1) I am replying only to the sentence I quoted which has very troubling connotations, and 2) I really think a lot of people here seem not to have read Julian Dibbell's crucial 1993 article "A Rape In Cyberspace" and it really shows.
- aleph_minus_one 1y ago> tools that might be used to encourage the fantasies of the men that already have a dangerous interest in harming them sexually. This assumption sounds like being taken from some feminist manifesto.
- crooked-v 1y agoOne in five women in the US experience rape or attempted rape during their lifetime, overwhelmingly by someone they alreay know at the time (https://www.nsvrc.org/statistics https://www.nsvrc.org/statistics).
- yesbut 1y agomight? ascii art has a pervert problem.
- appreciatorBus 1y agoYup. Likewise cinematography, photography, fiction writing, spoken language, and cave paintings, all have had a pervert problem since day one.
- yesbut 1y agohumans are perverts. who knew?
- marcellus23 1y ago> And how do you stop people from making fetish content of purely AI-generated characters that aren't cameos of real people? Does OpenAI want to stop that? Maybe OpenAI thinks it's fine for people to make belly-flation or foot-fetish videos as long as they're not of a real person. I can't figure out the tone here. Is the author suggesting we should stop people from creating fetish content of purely AI-generated characters? OpenAI might want to for business reasons, but surely there's nothing inherently wrong with using AI for fetish content. Should we also stop people from drawing fetish content with pencil and paper?
- fwip 1y agoI don't think she's passing any judgment here - she pointed out earlier in the article that she knows people are into weird stuff, and didn't want to "yuck their yum."
- Jackson__ 1y agoI don't think one gets to call these people 'pervert' in the title, then claim that you don't 'yuck their yum'. They yucked their yum before the article even started.
- regularfry 1y agoTitles frequently aren't in the control of the article writer. Don't know if that's the case here or not.
- ivape 1y agoThe problem is adults contribute to turning public platforms lewd. So one lewd person on Instagram leads to many, leading to a lewd platform. This becomes problematic when the children uptake it. It’s not really too different than prostitution appearing near general-purpose places, it turns it into a red light district. A lot of social media is a sex platform, and it got mixed up in this way because there’s no talking adults out of being lewd in public.
- blindriver 1y agoThis is kind of a side issue but I think the time has come where videoing in public and using images of people without explicit consent should be outlawed. It used to be okay when the only medium was TV or movies and it was hard to distribute but now I think it has become a nuisance at best and dangerous at worst. The only thing I would make an exception for is actual journalists with some sort of credentials. But posting videos on social media of unwitting unconsenting people should be outlawed in the age where people with phones can upload videos in second.
- afavour 1y agoIn a world where absolutely everyone has a camera in their pocket I can’t see how you’d ever be able to enforce this.
- lyu07282 1y agoTerrible argument, you should say because of generative AI where anybody can fabricate any image/video/audio of anyone in any context without their consent we urgently need new regulation of social media platforms to accommodate this profound change in our reality. Then make your argument. You have no expectation of privacy in public spaces since forever, that is not a problem. Because nobody can photograph you stabbing someone and uploading it on social media without you actually stabbing someone. This is now different, because anyone now can make that photograph of you stabbing someone and post it. That must be your argument. And it must be on the social media side, because in X months some open model on GitHub is gonna make every watermark or cloud-based safety feature meaningless anyway.
- ungreased0675 1y agoWait a sec, you have to specifically opt-out of this? OpenAI needs to be bankrupted for their massive copyright infringement.
- _345 1y agono she opted in
- vunderba 1y agoSigh, it's in TFA. > I've allowed anyone to make "cameos" using my face. (You don't have to do this: You can choose settings that make your likeness private, or open to just your friends — but I figured, why not? And left my likeness open to everyone, just like Sam Altman.)
- lschueller 1y agoYep, guilty as charged
- Retr0id 1y agoWho could possibly have seen this coming.
- tennisflyi 1y agoWhy are teams always, always so naive?
- busymom0 1y ago> I've allowed anyone to make "cameos" using my face. (You don't have to do this: You can choose settings that make your likeness private, or open to just your friends — but I figured, why not? And left my likeness open to everyone, just like Sam Altman.) So the author specifically allowed their face to be used for content and then is surprised people acted on it? This is silly imo. People should just never even allow their face to be used for this. This reminds me of the "Joanne is awful" episode from black mirror. Exact same story.
- portaouflop 1y agoTbh if you are sufficiently famous or have a bunch of pictures of you available online you don’t need to opt-in. So this is silly yes but people can still easily make deepfake porn of you from any photo available with other oss tools
- Aeolun 1y agoSure, but then they’re not immediately public on Sora xD
- sathackr 1y agoRule 34
- mslt 1y agoYou’re treating this article as an attack when it’s not one. She turned it on to see what would happen, because she’s a technology reporter, and then reported on what happened. And her reporting isn’t even all that negative.
- portaouflop 1y agoMaking fetish porn is the no1 use case for AI since inception.
- deleted 1y ago[deleted]
- cyberax 1y agoJust force Sora to avoid photos of real people. Instead, synthesize a "generic" face. This can be done with training, create a database of photos to avoid and train Sora on that.
- itake 1y agothe whole point of sora is you can generate photos of yourself and friends... The app allows you to control if other people can generate photos of you. If the author doesn't want other people to make these photos, disable public video generation...
- thfuran 1y agoIs it actually easy to train the AI on all the faces so it can make pictures that look like humans while also training it to not make pictures that look like any specific human?
- zb3 1y agoJournalism might have a "bullshit content" problem on its hands.
- ozozozd 1y agoI have no idea how I would feel if someone used my face in fetish or sexual content. Never happened. It’d probably make me uncomfortable. But, I imagine I would be ok if I grew up in a culture where sex wasn’t as much of a taboo. Maybe I would find it flattering. My mental test for deciding whether something should be illegal or unacceptable is questioning if any one would see it the same if religion never existed. During #metoo I remember reading an article where the author was uncomfortable with the “drug fueled sex parties in Silicon Valley.” They basically didn’t want consenting adults to do drugs or engage in group sex. The argument against fetish content with AI generated characters reminded me of the #metoo author’s discomfort with the drug/sex freedom of the Bay Area. The article about Sora sounds like the author is uncomfortable with people generating fetish content, regardless of the content featuring real people or not. It’s sad that the liberals now include the prudes/conservatives.
- frumplestlatz 1y agoI don’t think one is a prude or a conservative for not wanting AI generated porn of themselves to exist. I also don’t think one is a prude or a conservative for thinking there are consent and power issues around anything that commingles sex and the workplace. Things can be unacceptable without being illegal. Things can even be unacceptable without needing to be banned or privately controlled. My bar for what should be unacceptable is a lot lower than my bar for what should be illegal or privately banned. Making weird pregnancy fetish videos of real people without their permission is definitely unacceptable. I have no issue with the idea that anyone doing that should be shamed.
- ozozozd 1y agoI was pretty clear about my agreement with the real people AI fakes issue. And pretty clear on what I considered to be prudish. Would mind reading my comment and editing your response? Or you’re responding to someone else? Edit: the fact that the author is right to be uncomfortable with their face on generated fetish content, doesn’t make their stance on fetish content with generated characters less prudish. They can be right about one thing, and prudish about the other.
- deleted 1y ago[deleted]
- vunderba 1y agoFrom the article > How do you stop people from making fetish content of purely AI-generated characters that aren't cameos of real people? Does OpenAI want to stop that? I'd say a resounding no. Didn't Sam Altman announce a little while back that they're exploring allowing ChatGPT to be used for erotica / NSFW generation?