6 ms·
Deepfakes Tore a High School Apart
https://archive.is/neN8H https://archive.is/neN8H
- ceheaaf 5mo agoDidn't get past the paywall but, recently a friend had their primary school targeted by blackmailers who took public photos of children and made (I'm told) extremely explicit images with them, threatening to release them if not paid. What's the defense? Intelligent screening of incoming messages so that the threat never reaches the blackmail target? I imagine they'll find an unprotected channel. Don't post innocuous images of children ever? Seems like losing.
- eska 5mo agoDon't post innocuous images of children ever? Seems like losing. I do have to say that I find it disturbing how liberally parents seem to post pictures of their children in public though. Respect the privacy of your children, they’re not your pets.
- ceheaaf 5mo agoI'm more thinking, the winning school sports team, a choir performing, etc. Things that would be posted in normal institutional participation and give children positive affirmation and recognition
- defrost 5mo agoCertainly in my lifetime we never posted such things publicly across the globe, neither for myself nor my peers, nor for my children and most of their peers at school. Such things were printed and handed out and rarely made it past that years parents and students and the school archive (physical visit required).
- suddenlybananas 5mo agoGetting in the newspaper or on TV for winning an award or competition or something has been around for a long time.
- defrost 5mo agoFor the very rare few, yes, a fair while. For the Dux's and general high performers, less time than I've been alive. For entire classrooms to be posted up via live streams, Insta, tok's, etc ... barely a decade or so. It's not something that was always normal, and just because it's become "normalised" doesn't mean it must remain that way.
- Cthulhu_ 5mo agoSome schools will (try to) do this, but under European privacy and portrait right laws, they have to get explicit permissions from parents first. I think a school news letter / yearbook is cute, but it should be private and only for the students and parents. A physical version would be best, you can put those in the archive instead of realize 20 years later that the digital versions are gone.
- CalRobert 5mo agoIt's nice to see pictures of people. I like when parents share photos of my kids at events where they're having a great time but I couldn't make it. It's nice when the local paper shows a picture of happy graduating 8th graders. Saying "you shouldn't do this thing that was basically harmless from the birth of photography until the ~2010's" is ignoring a lot of history and context.
- 627467 5mo agoBroadcasting private photos of children to strangers and corporations was common until 2010s? Were you posting them on classified section on newspapers in 1999?
- jvidalv 5mo agoNot caring.
- pjc50 5mo ago> What's the defense? Prosecution of AI operators for making indecent images of children?
- ben_w 5mo agoAlthough I agree this is necessary, I don't think it is sufficient. Hackers and blackmailers on the internet can be anywhere from trivial to impossible to identify. Even harder to prosecute when they're in a different jurisdiction, as not everywhere has extradition treaties to everywhere else. I wouldn't be overly surprised if e.g. the North Korean government runs some schemes like this (though not necessarily actually this) to bring in money. But yes, definitely do the easy things first, like trying to stop e.g. grok from doing it and also going after users of e.g. grok who try to get around such efforts. A lot more crime happens than can be prosecuted, so raising the minimum competence threshold to commit crimes in the first place is very necessary.
- Cthulhu_ 5mo agoSure, but the reality is that if it happens abroad there is no legal recourse. Laws are not global, but the internet is. The only real option there is to close the internet. Or to elect me as god-emperor and unify the world under my rule.
- ben_w 5mo agoI'd already realised by about 2010 that the internet was making a mockery of national sovereignty; I absolutely expect nations to demand some way to effectively close the borders of the internet without breaking everything as it presently would.
- mittensc 5mo ago> What's the defense? Intelligent screening of incoming messages so that the threat never reaches the blackmail target? I imagine they'll find an unprotected channel. Same defenses that are used against fraud and other crime. criminal prosecution of the blackmailers AND the services used to generate the pictures. This is effectively child porn... so penalties would be pretty harsh. There are extradition treaties to most of the world, so unless the blackmailers are in China/Russia they will end up in jail. That same thing played out with piracy with people extradited to the US from various countries
- graemep 5mo agoI just looked it up and according to Wikipedia distributing child porn is a crime in both, possession is a crime in China. Blackmail is a crime in both too. So even if they do not extradite the blackmailers they are likely to face jail under the their own laws.
- Asooka 5mo agoThat is a clear-cut case of extortion. The defence is having the police do their job and apprehend the criminals in question. If you screen incoming messages and stop them from reaching their target, extortionists will switch to publicly releasing one risque image with threats of releasing more explicit ones. There is always an unprotected channel as you noted. As an aside, please do not use the b-mail word. It is insensitive towards BIPOCs.
- account42 5mo ago> As an aside, please do not use the b-mail word. It is insensitive towards BIPOCs. No it isn't. If anything needs to stop its the ever escalating orwellian censorship of words that you're proposing.
- deleted 5mo ago[deleted]
- bandrami 5mo agoIn the US possession of CSAM is a strict-liability crime. If the models contain the data to produce it, the modelmakers should be liable.
- pbhjpbhj 5mo agoModels don't have to contain imagery of a thing to produce imagery of that thing - the canonical example is the pelican riding a bicycle. Model makers, arguably, are like pencil manufacturers but in a World of good artists. I think you can hold people to account for what they help to create, but not what they have potential to do. Now, if models were trained on any csam, of course, the model's owners should be held you account.
- bandrami 5mo agoAnd I think it would be an interesting question of discovery to determine if the models were or weren't trained on CSAM. (Diffusors were trained on both pelicans and bicycles, you know...) I still think there's a lot of legal and IP landmines lying in wait from the Hoover-esque pre-training eras
- pbhjpbhj 5mo agoWell, yes, models might be trained on naked adults and clothed children, incidental to their purpose. Agreed on your second remark. If I'm still around it will be interesting to look back on this period on twenty years and think about how things should have been handled.
- Jordan-117 5mo agoThe vast majority of these offences are from apps or sites that generate and host the images themselves, often by openly promoting their ability to "nudify" people, and payment leaves them a paper trail. That makes them complicit. I agree that local/open source models are harder to police, but most grade schoolers are not going to be able to set up and run something like that.
- hjkl0 5mo ago> Don't post innocuous images of children ever? Don’t raise boys who abuse girls. That’d be a big step in the right direction.
- account42 5mo agoBecause girls can never be abusive?
- teaearlgraycold 5mo agoWell TFA exclusively discusses instances of boys abusing girls.
- croon 5mo agoYes, but statistics are very clear here [0]. Speaking as a dad of both, we have tried to raise our kids to treat everyone as a human first, but there are very obviously ways for boys to fall into different behaviors than girls, both from culture (probably mainly) but also from testosterone when older. Girls can absolutely be cruel in other ways, though, and of course incidents diverging from statistical patterns, as with most things. [0] https://www.abs.gov.au/media-centre/media-releases/97-cent-sexual-assault-offenders-are-male https://www.abs.gov.au/media-centre/media-releases/97-cent-s...
- arkh 5mo ago> What's the defense? Prosecuting every one of those blackmailers. If this kind of crime starts going into the "you're gonna get caught and jailed 100%" category, less people will try it.
- nkrisc 5mo agoWhy a school, in this day and age, is posting pictures of children is beyond me. At my kid’s school we refused to sign the photo release clause, and any time they might be taking pictures they had to email us to ask if it was OK if our child was included. We agreed to it for the yearbook, but refused everything else.
- antran22 5mo agoThat is propagation of CSAM. We already have way of dealing with people transmitting CSAM, that is to prosecute them heavily. You won't prevent people stabbing others with knife by banning knife or asking people to wear knife-proof vest going outside. You deter them by making everybody know that the consequence of harming somebody else is going to be a very unpleasant experience.
- ale42 5mo agohttps://archive.is/neN8H https://archive.is/neN8H
- thunderbong 5mo agohttps://archive.is/neN8H https://archive.is/neN8H
- hjkl0 5mo agoThis has nothing to do with “deepfakes”. These girls could have been attacked by aliens and be received with more credulity. There has to be a point where you understand that this behavior towards girls and women is simply being allowed. One of the mothers says this quite explicitly: > Calling the event rumors and speculation when a crime occurred was almost worse than the crime
- ktallett 5mo agoThis is sadly becoming very common and they are worringly good enough for people to assume they are real. I had a friend who had this issue, she has tried to ride it off as no clue who made them or shared them but I know it can't be easy for her. There needs to be appropriate laws for this the same way there is for revenge porn etc.
- jacknews 5mo agoSurely this comes under defamation law or something similar, as if they photoshopped the girl's heads onto porn photos.
- CalRobert 5mo agoI don't know how society will function when it's as easy to fake a video or image as it is to draw a stick figure. Hopefully it at least means that people view what happened here as akin to scribbling boobs on someone's photo, but we're not there yet apparently.
- Sharlin 5mo agoHopefully we’ll get back to the time back when people didn’t have to worry about someone filming them all the time. Cameras cryptographically signing all photos and videos should become the standard. Media that cannot be traced back to real camera hardware should a priori be considered fake or tampered with. Still, deepfakes are partly a social problem and, as should be well known, social problems can rarely be solved by technological solutions.
- _el1s7 5mo agoThat won't work, a camera can also photograph another screen or a fake printed photo. And also cryptographic modules can be cracked.
- petesergeant 5mo agoI would have thought most iPhones absolutely know if they're photographing a scene or a flat surface
- _el1s7 5mo agoYes, iPhone has some clever depth technologies, but that doesn't mean it can't be tampered with.
- Sharlin 5mo agoNothing is ever foolproof, but also, public-key crypto signatures are as secure as can be. Sure, you might be able to eventually extract a private key from camera hardware, but that's on the level of intelligence agencies, completely beyond the current situation of deepfakes being possible to make by anyone with an internet connection and no skills.
- M4v3R 5mo agoThis kind of behavior should be met with swift and strong reaction from both the school authorities and the law enforcement. Kids responsible should be put on trial and face serious consequences including jail time for most egregious cases. Let the news cover it nationwide. "This is what will happen if you even think about creating and circulating nude pics of anyone" should be the message sent. This is the only way to prevent this behavior - not more censorship, not technical solutions
- noosphr 5mo agoBack in my father's day you'd get executed for drawing a dick on Stalin's forehead. It's nice to see people still care more about symbolic violence than actual violence.
- Rekindle8090 5mo ago[dead]
- numbsafari 5mo agoThese are teenage girls, not adult men. This isn’t graffiti, but highly realistic sexual depictions. There’s a massive difference.
- bondarchuk 5mo agoNevertheless there is almost a moral equivocation between images of sex and actual sex by some people. We are talking about restraining someone physically inside a 2 by 3 meter box for multiple years as a retribution for what is after all just pixels on a screen. (And by the way the implicit comparison here between an image of stalin and an image of a teenage girl is more apt than most would care to admit given the ideological function fulfilled by sexualized images of young women in contemporary American society)
- numbsafari 5mo ago“Just pixels on a screen” What is wrong with you?
- dmos62 5mo ago> Throughout the night, whenever one of the speakers brought up holding Big Tech accountable, and specifically Apple and Google’s app stores, the crowd applauded. The framing bothers me. It's shifting responsibility away from kids and the community. Deepfakes didn't do anything: kids made and used deepfakes in destructive ways. Big tech sucks, you get no argument from me there. But, how do people so quickly get over the fact that a kid is sexually harrassing someone? This is like when a mass shooting happens and everyone only talks about guns. Well, yeah, if you didn't have guns a mass shooting couldn't happen, but isn't the underlying issue that someone decided that hurting a bunch of people is the best thing to do? The more people dislike a crime, the less likely they are to talk about the perpetrator's inner life, but, come on, these are kids. How do people get so side-tracked?
- teaearlgraycold 5mo agoThat's crazy talk. You actually want the village to raise the children and have everyone well integrated and healthy?
- dmos62 5mo agoYou know, that is crazy. Disregard what I said. Down with big tech!
- gorgoiler 5mo agoEven though they fluffed the messaging I think the school was right, in the absence of any actual deepfakes and even after the police department had scraped all the boys phones, to be wary of accusing anyone in particular of doing anything AI related. Kids are weird and one boy saying he “dropped $250” on making deepfakes of “that hoe” sounds as much like a brag about money than it could be about making fake videos. There’s clearly enough of a text message trail between these boys to give them a significant reprimand for misogyny. The PD’s PR team changing their narrative seems to lack competence. The school board focusing on being light touch on a specific case instead of cracking down in general seems like another error of judgment. This whole article says more about the effectiveness and competency of cops and school boards as it does about generative imagery.
- conductr 5mo ago> There’s clearly enough of a text message trail between these boys to give them a significant reprimand for misogyny I skimmed but the worst I saw was one of them referencing her as “that hoe”. Not nice but also punishing kid’s private conversations outside of school this closely seems like an overreach. Worse, is the message it teaches the younger generation that speech should be censored and isn’t as free as they think. I feel that’s a worse outcome on a longer timeline.
- teaearlgraycold 5mo agoI think sharing these images is sexual harassment and it's pretty clearly wrong and deserves punishment. Other than that, I don't think it should ever be possible to commit a crime by providing inputs to an air-gapped "clean" computer and then view the output. But it seems like that's not the case.
- mrvmochi 5mo ago[flagged]
- aarond0623 5mo agoThey chose to do make content about one of their fellow classmates they knew would be harmful and shared it with others. It doesn't matter how aroused they were. If you're aroused, there's petabytes of porn literally all over the Internet.
- croon 5mo agoI have kids. A significant part of raising them is showing/teaching them that feelings are natural, but only some outlets for those feelings are acceptable. My kids are younger than high school. Society is built around not destroying everything when you're angry. Hopefully that is easier when you're adult, but it's certainly the norm to expect it from children as well.
- mrvmochi 5mo agoWhat would you do if your child suffered brain damage from an illness or injury and lost the ability to regulate their emotions? And what if the severity of that impairment was right on the borderline—just enough to be penalized by society, but not enough to qualify for any support? I used to be an instructor at a karate dojo, and several parents brought their kids to me, asking me to "do something" about their violent behavior. One of the children had been diagnosed with autism, but the rest hadn't even been evaluated by a doctor. The parents likely expected me to discipline them through fear and (possibly) corporal punishment, but I chose to instruct them relying solely on words until the very end. However, it was futile. When I taught them that they must not hurt others or lash out, they would reply, "I understand." But the reality was that they had merely learned to say "I understand" whenever spoken to by an intimidating opponent they knew they couldn't beat. Ultimately, every single case ended in disappointment. Consumed by a profound sense of powerlessness with the last child, I gave up teaching kids altogether. You and your children are simply fortunate to possess the cognitive capacity to understand that making deepfakes is wrong.
- bondarchuk 5mo agoThe article intentionally obfuscates the difference between images depicting sexual abuse that did not happen, and actual sexual abuse. Ironically this makes it harder to address the problem on the level where the actual abuse happens i.e. bullying, lack of empathy, teenage body image issues in the properly fucked up american high school environment. For example here the word "abuse" technically refers to the making and spreading of these images and attendant bullying, but obviously has connotations of actual sexual abuse: >Despite all this, Radnor’s administration failed students in the days and weeks after it learned about the abuse, This is maybe subtle but later the article gets more explicit about it: >this technology [...] bears little difference from the non-consensual intimate imagery that’s plagued young girls and teenagers since the invention of the camera. (i.e. a straightforward equation of images of nudity that did not happen, and images produced by actually pointing a camera at someone who is physically present with their nude body) >Because the images aren’t “real,” authorities grapple with how to handle them. Putting "real" in scare-quotes, very nice. Dorfman is also quoted doing some nice sleight of hand, going in one sentence from "It was about the creation of the videos" to "and then share them with others as if the girls were something to be passed around." - i.e. equating the making of videos depicting unreal events with something that almost sounds like gang rape. Anyway enough examples, article is full of them. I'm not saying there's no problem here, just that these people seem to have some kind of ideology of the sanctity of the image, i.e. a realistic image always depicts real events, and should always be interpreted as if it does. If it does not, clearly that's a problem to be corrected by reinforcing the connection between images and reality by making it impossible to produce realistic images of non-real things. It strikes me as exactly the wrong way to go about it, that can only make the bullying and harrassment (the actual problem which should be solved on the social level) worse by making its ammunition stronger. (Slight aside but I do think the closing quote by Woelfel makes a good point)
- IshKebab 5mo agoHow did he not get expelled for that? Jesus.
- cindyllm 5mo ago[dead]