6 ms·
The Deepfake Nudes Crisis in Schools Is Worse Than You Thought
- colpabar 6mo agoUsually threads like this fill up with comments about how "think of the children" is always a lie used to justify something draconian. I agree with that to an extent, but for those who think that applies here, is there _nothing_ we can do as a society to address this? I'd like a better answer than "let parents deal with it," and the whole "this wouldn't be a problem if america wasn't so puritanical maaaaaan" argument is total bullshit that completely ignores the young girls who get hurt by this.
- alephnerd 6mo ago> Usually threads like this fill up with comments about how "think of the children" is always a lie used to justify something draconian It also highlights HN's demographics. What younger women feel is problematic is viewed as trifling by a number of younger or middle aged men on HN (especially those without kids).
- gwbas1c 6mo agoNo, it shows that people on Hacker News understand two important things: 1: Unintended consequences 2: That power-hungry people latch on to issues like this to further political agendas that have severe negative consequences; mostly by using "think of the children" to stifle important debate and discussion of unintended consequences.
- Taters91 6mo agoSo since there are other consequences, nothing should be done? I think sexually demeaning children is a really big issue, and pretty well-defined and wide spread. We may not need to ban things, but not doing anything is a cop-out.
- 9x39 6mo agoWhat would the effect of hysteria from technologists achieve? If you think it’s a really big issue, why don’t you own the problem? You could just go turn off the trillions in AI spending and destroy computing as we know it. No cop-outs, remember.
- gwbas1c 6mo agoNo one's saying to do nothing. When you debate like that, (specifically twisting words like you did,) you leave yourself open to be a victim of power-hungry people, and trade one problem for another potentially worse problem.
- array_key_first 6mo agoIt depends on what the other consequences are, obviously. I can fix the problem right now. We just throw all the children into the orphan grinder. Can't sexualize ground meat. Is that what you want? Wooooww, so you're telling me you want to grind up children?? That's really messed up dude. It's super easy to twist any argument against this stuff to being "against children", which is extremely annoying and unproductive. That's why you, and others, get a lot of pushback - you're annoying and dishonest. Nobody here, and I do mean nobody, wants children to be hurt. That includes me and you. If you want to fix these issues, you have to answer three questions head-on: 1. What is the actionable, real solution? 2. Will it work? 3. How well will it work? "Durrr sexually demeaning children bad" doesn't answer any of those. We all know sexually demeaning children is bad. Now tell me if, I don't know, banning the Internet or some shit will work
- embedding-shape 6mo agoIf you think the sentiments of comments in one thread (not even specified which) highlights the general HN demographics, I think saying that reveals more about you than HN.
- kerblang 6mo agoAm a middle-aged man, don't have kids, don't see it as a trifling problem, and I don't agree with the libertarian free-speech-at-all-costs angle. Instead I think a) kids shouldn't be on the internet and b) the public school system is a barely supervised dumpster fire.
- lo_zamoyski 6mo agoOnly to certain kinds of defective men/men suffering from ressentiment. No healthy man of good will wishes to see women get hurt or disrespected.
- embedding-shape 6mo ago> is there _nothing_ we can do as a society to address this? Well, I guess the argument goes that regardless of how much you lock down centralized platforms like Grok, these tools can run locally on a PC so as long as people can do local inference with this tooling, it won't fully go away. With that said, limiting the centralized platforms from generating nudes from uploaded images/photos feels like an obvious limitation they should implement, if they haven't already. So if we consider that we could probably limit most of the "off-hand" stuff that happens with the platforms, but we cannot fully limit the offline ones, I'm guessing that the best solution is merely "Education" here, together with laws. AFAIK, it's already illegal to create deepfakes and share those with others, but probably the education around this isn't great, as it's such a new thing hardly adults understand, much less younger folks. It's a societal problem that I'm afraid doesn't have a technological solution, as far as I can tell, because the cat is out of the bag, so whatever "solution" we come up with, have to go beyond just technical capabilities/limitations. One thing people might be missing to carefully consider, is the whole "private" vs "sharing" part of this. People been fantasizing, drawing pictures and similar things about others (even strangers) for as long as humans probably been around. What's new, is that it's effortlessly to share those now, and they spread far, wide and fast. I don't feel like we could possibly stop the whole "I don't like people fantasizing about me" part, it's just too human to get rid of, but what we need to get rid of is the whole sharing part, which is what actively harms people.
- post-it 6mo agoTeenagers aren't running generative models on their home PCs, they're using online tools. Sure, some people will do it offline, but these are by and large crimes of opportunity.
- ryandrake 6mo agoHorny Teenagers Find A Way. If the online tools were all magically shut down today, by tomorrow, they'd all be running local models on the local computer nerd's gaming GPU rig.
- 6mo ago
- b00ty4breakfast 6mo agoI think in this instance the issue is that these tools shouldn't be available at all to anyone, adult or child. There are not many use-cases for generating porn from images of real people who aren't porn stars and most of them are not good.
- post-it 6mo agoYeah, we've pretty successfully gotten rid of online child porn from like front page search results and public social media, there's no reason why we couldn't do the same for this.
- ryandrake 6mo agoThe article isn't about content on the front page of search results. The content in question is being circulated privately through chats. The proposed solutions, so far, have been the broad, nasty, invasive content scanning that nobody on HN wants (for good reason).
- post-it 6mo agoNudify apps are very much present on the front page of search results. Get rid of that and you'll significantly cut down on the problem.
- slashdev 6mo agoHow much violence (government control) do you need to exert to block all of these tools? How successful have we been blocking pirated content? I don't think the genie is easily returned to the bottle and the cure may be worse than the disease.
- phyzix5761 6mo agoI used to think the same about pirated content until I traveled to a country that has been successful in blocking pirated content. Its impossible to even find a free movie on YouTube there. The government really cracked down on it and succeeded.
- dec0dedab0de 6mo agothe whole "this wouldn't be a problem if america wasn't so puritanical maaaaaan" argument is total bullshit that completely ignores the young girls who get hurt by this If society wasn't so puritanical there would be no harm from it.
- ButlerianJihad 6mo ago> If society wasn't so puritanical there would be no harm from it. What is "puritanical" then, in your opinion? Is it being uptight? Do we abandon our efforts to uphold modesty and dignity of the human person? Do we stop teaching our children about boundaries and propriety, and their responsibilities as they mature? How should boys and young men conduct themselves around women? Should they learn about the concept of consent, and respecting others' wishes?
- lo_zamoyski 6mo agoThose who immediately jump to accusations of "puritanical!" at the mere criticism of any sexual indiscretion are usually sexual deviants of some kind. They may not necessarily partake in any actual sexual activity (they probably usually don't, hence the desire to lower the baseline of acceptable behavior), and they aren't fit for relationships as those require respect. Porn brain comes to mind. It's time we got back to sexual ethics, as it is absolutely not the case that anything goes or even that "consent" suffices to make something okay.
- ivell 6mo agoIn this particular type of abuse, the issue arises because humans value hiding reproductive organs. A similar social constraint applies to women's virginity. So if the society was not so focused on hiding reproductive organs, this particular issue would not have arisen. Nobody laughs at naked hands. It is just expected to be bared and if you put on gloves it is because of weather and not usually to hide hands.
- deleted 6mo ago[deleted]
- 6mo ago
- egeozcan 6mo agoI'm a parent, and all I can think as a solution is devalue nudity. I'm not saying we should all walk around naked, but make sure children understand the human body earlier, any why we wear clothes, the values, customs, without the stigma. Makes it also easier to teach what's okay and what's not, and if someone does something "not okay", they are easier to identify as the ones responsible. Fake or real, a nude photo should give no control to anyone. It's extremely easy to do all sorts of weird things with AI, all local. Controlling that means controlling the hardware, something none of us wants, and it will get only easier, I suppose. So, doing what I said earlier becomes even easier if these stuff gets automatically devalued by commoditization. I'm not saying I know better than people who want restrictions, and I'm not trying to offer yet another "ban all bans" opinion, I just don't see any other realistic solution. There are however many other people much more knowledgeable than me in these matters, so, maybe I'll be positively surprised.
- UncleMeat 6mo agoThis isn't just nudity. These fakes are made of young women with what looks like cum all over them or in a pose to give a blowjob or be penetrated. Devaluing nudity does not change how people interpret porn.
- kyleee 6mo agoDemocratize the spank bank
- lostmsu 6mo agoSounds like people here want to outlaw non-physical abuse, but that contradicts the first amendment, doesn't it?
- UncleMeat 6mo agoNo. The first amendment does not provide unlimited protection for all things that resemble speech. Students have further limited speech protections, and sexual speech by students at school has been expressly found to not be protected by the first amendment. Abusive and bullying behavior is similarly unprotected. And while I don't believe that there has been a first amendment challenge to deepfake revenge porn laws, I'd be stunned if such a law didn't survive strict scrutiny.
- Mezzie 6mo agoThere are broadly speaking two options, neither of which will ever be implemented. What will happen is young girls will learn to accept deep fakes are made of them and it's part of life, in the same way explicit sexual commentary starting at the age of ~9 is, or how men follow you when you go somewhere in public, etc. They'll accept nobody cares in the same way nobody cares if you're sexually harassed in other ways. The two options are either the people in power standing up for the girls or giving the girls the power to deal with it themselves. People who have power generally are benefiting from the structure in place and so don't want to change it and/or they don't want to do any more work. Expelling all the kids who create deepfakes would cause a lot of arguing from parents and people who are on the boys' side, and they just don't want to deal with that. It's easier to tell the girls to be quiet. The other option is setting up a system that rebalances the power. For example, if a kid gets caught making deep fakes, give their victim access to every single thing on their devices: Private messages, Discord chats, images, etc. and let the victim decide who and what to release their private information to. Not going to happen. Another reason nothing is going to be done is if we teach 11 year old girls it's not acceptable for people to do this to them, they'll carry that forward through their life and a lot of people who find it gross to create fake porn of children are fine with doing it to older women and they don't want to create women who create a fuss about it. There are a lot of people who think it's disgusting I was sexually propositioned when I was 10/11 but think it's fine I can't go for a walk in my neighborhood now without being bothered since I'm older.
- elictronic 6mo agoNo, you make it a crime and prosecute it. What loony tunes world do you live in where you start giving away peoples other info. This just incentivizes you to abstract your data like businesses abstract their profits. Make it a crime. Prosecute it. This isn’t hard, unless you have a legislature that is incapable of passing laws. People are becoming fed up with this stupidity. Give it a few years and a few congressman kids getting targeted as it becomes more mainstream and things suddenly change. We’re at the leading edge of the stupidity, voters, families, politicians aren’t angry enough yet. It will change, legislation is cyclical.
- 6mo ago
- wolttam 6mo agoI think a pretty good portion of parents would agree with a blanket "your phone shall not be seen during school hours" policy. Something that would probably need to be decided per-district
- negura 6mo agoover the past 20 years technology has become more and more catastrophic for children. i suspect a large number of grownups who oppose child safety legislation don't really grasp the bleakness. they project their idea of a technology that they grew up with. they don't get to see the kind of data that policymakers, healthcare providers and LEAs see. i'm not saying the proposed legislation is justified. but i don't think the people behind it are necessarily draconian
- gicadin 6mo agoThanks for below notice - https://archive.ph/ZSW1a https://archive.ph/ZSW1a Previous link deleted **
- SV_BubbleTime 6mo agoHeads up, for me this wasn’t a working paywall bypass.
- SV_BubbleTime 6mo agoSo I can skip ahead here… 1. This has always been possible but the bar has been lowered to barely above typing “her, but nude!”. Opposed to a talented photoshop or pencil artist doing this previously. Is the issue scale? 2. Lots of things have been illegal and immoral without tools. Assault for example. We don’t really have to deal with those things on a large basis. The difference here is effectively thought crime until distribution takes place and then it’s just another form of assault right? We already have laws on bullying and assault, no? Like I said… skipping ahead, let me guess, For The Children; we must block local models, AI except for The NYSE Chosen Ones, encryption, and must have digital ID to use everything. Tell me this article is in favor of anything else. As soon as you figure out that everything you read at large outlets, esp those owned by Condé Nast is directly written to affect a stock price somewhere it becomes a little exhausting.
- embedding-shape 6mo ago> Tell me this article is in favor of anything else. Someone correct me if I'm wrong, but I don't think the comment section is meant for others to educate you about what the article says, you're supposed to read the article yourself first, then comment. Everything you've said might be true, but are you actually adding insights to the conversation here, when you admit to not even knowing what the article is trying to say in the first place?
- SV_BubbleTime 6mo agoIt’s so tiring. The article has a beyond obvious take even from the headline that “ITS SO BAD, We Must Do Something (tm)”. I read the article first, show where my take is incorrect. Most of us know what this is, and it’s choke full of the usually moral pleas wrapped up in a nice package of “don’t mind the side effects”.
- embedding-shape 6mo ago> show where my take is incorrect > Like I said… skipping ahead, let me guess, For The Children; we must block local models, AI except for The NYSE Chosen Ones, encryption, and must have digital ID to use everything. Tell me this article is in favor of anything else. Please quote the parts of the article where they mention banning/blocking local models, saying that any AI except "The Chosen One" is OK, anything about encryption or even anything about Digital IDs?
- jacknews 6mo agoI don't get it. Everyone knows that a nude is going to be AI. Even actual nudes can be explained away as just AI. Sure it's distasteful, but it's no different to cutting and pasting heads onto porn stars as was done plenty before. I'm not saying there's entirely no harm in that, it's obviously a form of bullying, but AI does not make it novel, or a crisis.
- mywittyname 6mo agoIt's significantly different. It's better quality, more accurate, and more convincing.
- jacknews 6mo agoBut it's still not real.
- phainopepla2 6mo agoYou really don't get how distressing it would be to a teenage girl to have extremely realistic nudes of her spread around publicly, even if everyone knows they're AI generated? Did you try imagining yourself in her place, imagining the social world she inhabits?
- UltraSane 6mo agoIt really is only as distressing as you let it be.
- runjake 6mo agohttps://archive.ph/ZSW1a https://archive.ph/ZSW1a
- BikDk 6mo agoAh, yes - the moral ambiguity of sexualized western culture - somewhere between medieval moral repression and (tech-driven) liberalization.
- avaer 6mo agoAs someone on the spectrum, I really don't understand where society draws this line. Seems self-contradictory to profiteer off of sexualized mass + social media (totally OK) but then get up in arms when that power is given to the masses (deepfake nudes). It seems more about power and money than anything else, and the moral grandstanding/outrage is the manipulative icing on top. EDIT: I am not saying any of this is ok. I am saying it's crazy social media has been destroying young people's lives encouraging them to sexualize for an audience for decades, and nobody is willing to do something that would maybe actually help these kids, like ban social media for under 18. I'm glad the tides are turning though.
- undersuit 6mo agoThese children don't know they are being sexualized. Yeah the adults in the room are exploitive and manipulative but this should literally be a "think of the children" moment.
- avaer 6mo agoThat's what I'm saying. Children are being sexualized and exploited left and right on places like Insta, Youtube, often even encouraged by their parents as it can lucrative. And this is totally okay and encouraged. But when kids start sexualizing each other it's news and a big problem. I'm saying there is a strange standard applied where some forms of exploitation are encouraged. And I don't get it. But I get why it's confusing the kids.
- embedding-shape 6mo ago> Children are being sexualized and exploited left and right on places like Insta, Youtube, often even encouraged by their parents as it can lucrative. And this is totally okay and encouraged. Huh, why would that be OK and encouraged? I haven't heard a single person, either on HN or around me AFK that they're OK with chil;dren being sexualized and exploited on social media, who on earth would even try to make such an argument?
- orwin 6mo agoThat should count as distribution of CP and be punished like it is, each time. I guarantee you the first time a boy will be faced with 15 years, the media will run hard with the story and it will calm everyone down.
- alephnerd 6mo ago> should count as distribution of CP and be punished like it is It should be treated as CP and revenge porn, but the issue is legislation surrounding these in a number of cases doesn't treat digitally altered images as within scope of CP or revenge porn. Additionally, platforms like Grok are taking advantage of this ambiguity by arguing that they do not need to add guardrails. Adding basic guardrails like not generating a sexualized image without identity verification or preemptively blocking questionable prompts would dramatically reduce this problem.
- UltraSane 6mo agoThen any similar regions in latent spaces would also count as CP, right?
- andagar1243 6mo agoMaybe, just maybe, the previous generation (I include myself here) have lost the plot raising/educating children and are breeding just absolute disrespectful, egotistical, attention seeking assholes as younger generations. Teachers in the classrooms have been globally sounding the alarms for decades about the loss of discipline, the loss of basic manners, the loss of respect for authority, the loss of empathy, the attention issues (attention seeking and attention impairment), the increase entitlement, the inability to cope with negatives, the increase in illiteracy, etc.. No one has listened. Then things go wrong and we blame the teachers. It is ok for kids to be mischievous. It is ok for young adults to take the piss out of each other in a healthy way. But this looks to me like an education problem. That kind of value based education where parents used to educate kids to be compassionate to each other, to respect each other and to f**ng understand that if a moment of fun with some friends could ruin someone else's life maybe it is not worth being the cool dude for 5 minutes. We have lost that. Most kids do not have these values these days. But what did we expect? We have been systematically ditching those values. We have an older generation now that is selfish, egotistical, careless and dismissive with anything that it is not them and their belief framework whatever that is. We have polarised to the extent of hatred. And this is showing in society. It is showing in our kids. I don't think tech is to blame. I don't think kids are to blame. This might be our fault.
- alephnerd 6mo agoPotentially, but we can't moan about spilt milk. In the world that we are in today, we should concentrate on what are the tactical wins we can take advantage of to reduce harm. This comes down to adding basic guardrails on the generation of sexual material on foundation models (eg. no sexual imagery can be created without age verification) as well as updating revenge porn and CP laws to remediate the now pertinent loophole around digitally generated imagery. Also, this is a global issue. Similar incidents have happened in South Korea, India, Thailand, Japan, etc.
- andagar1243 6mo agoSpilt milk? I don't think it is all lost. I think the solution is to start reverting it from TODAY. Educating children has always been tough, but it used to be the case that for a parent to fight against the other influences their kids had growing up meant to be against a few friends. And making sure that their kids resonated with the parents values instead of the "bad influences" ones used to be a full time job. It would have been UNTHINKABLE for me and my peers to do some of the shit teenagers find acceptable today to another kid. And that was thanks to the values our parents gave us. We were not special. We were the average kid being raised 30 years ago. In the age of social media, a modern parent is up against hundreds of video hours of the Andrew Tates of the world, the manosphere, the crypto grifters, the gym bros, the makeup divas, the get rich quick scams, etc... All examples of a society compensating rubbish ideas. Hundreds of thousands of examples of people doing the wrong things, and staying on top and being granted lambos, and money and fame. If you are a parent (not you personally, generic you) your job us to scream louder than all that influence. To spend every single minute, every single ounce of energy telling your kids that all that shit is not ok. That doing the wrong thing is not ok even if it nets positive to you. And lead by example. That independently of your political views, the recent behaviour of elected officials is not acceptable. That those influencers are in the wrong. That they are not examples to follow. You need to stay on top of all that. You need to make them clear that all that people they are constantly exposed to are despicable human beings and that this behaviour at scale would be our global doom. If they get all that from 4 hours of instagram and facebook a day, 30 minutes of daily parenting is not going to cut it. You need to be the biggest influence. And if you are not doing that, sorry, you are a shitty parent, and I blame you for the current state of our younger generations' principles. If you are too busy with your life to scream louder than the rest of the influences, you should have thought twice what kind of resources (time in this case) you had before having a kid. This is an education problem. And I think we are beyond tactical wins to reduce harm. Either we start taking responsibility collectively as a society, or this cannot be fixed with laws, and rules, and age checking OSs and porn ID checks. In a society where your average fellow citizen is a Karen, or a gym bro, or political or religious fanatic that cannot name 3 continents, then it follows that their kids are going to have the same level of literacy, critical thinking, empathy and ideas.
- cindyllm 6mo ago[dead]
- smartcc 6mo ago[dead]
- insane_dreamer 6mo agoI'm pretty sure we've given up on morality as a society, in the US at least. In the past 10 years in America, we have twice elected as president a person with the lowest morals, certainly the most vulgar, the most openly racist, sexist and misogynistic, the most blatantly dishonest and untruthful, with the least amount of integrity, the most openly narcissistic, the most entitled, with the least amount of respect and empathy for others, and with no sense of morality or even basic manners, of any president in US history. And that's just him as an individual, not even talking about policies. How are we surprised that kids are acting like assholes when close to half the parents in the country support such an odious character?
- junofan 6mo agoPerhaps a society that shames selects for the shameless.