8 ms·
Cops bogged down by flood of fake AI child sex images, report says
- forinti 3y agoI wonder if fake images will mean less attacks on real children. This is a really hard issue to tackle!
- jdiff 3y agoWe've had fake images for decades, with people justifying it the same way. "Drawings of children mean real children aren't being abused." I've yet to see the slightest shreds of evidence supporting that claim.
- forinti 3y agoPlease, don't think I'm trying to justify. This thing is just plain nasty. But these images do bring us some new (terrible) questions to ponder. Especially if the police can't differentiate between what's real and what's not.
- ljf 3y agoThey don't need to differentiate to charge you with possession of child abuse material - they can still do that for drawn images. This just means that efforts to find and protect real children will become harder, as those children will be mixed in with photorealistic 'fake children'. But possession will still (and quite rightly) be illegal.
- sp332 3y agoPhysically maybe, but knowing that someone is getting off to images of you as a young kid is still traumatising, especially if you still are a young kid when you find out.
- 4gotunameagain 3y agoIt will be interesting to see if/how this will modify their argument "for the children". It is a nuanced discussion. Is it a crime ? Why / why not ? Is there a victim, is there harm being done ? Needless to say I personally find it appalling.
- aketchum 3y agothere would have to be illegal images in the training set right? which I would imagine would make the whole model illegal and therefore it outputs?
- sanitycheck 3y agoI have luckily never encountered any of this stuff, real or fake, so I may be hopelessly naive about what's depicted, but... I can ask for a picture of naked Donald Trump wallowing in mud and such a picture will be constructed even though there are no photos of Donald Trump wallowing in mud (naked or otherwise). So I don't think the training set necessarily contains illegal images.
- misnome 3y agoNo _known_ photos of said scenario.
- chefandy 3y agoI reckon this all comes down to a bunch of diligence/negligence judgements that will eventually be ironed out in the courts if necessary after some initial broad legislation, but as someone with no legal expertise, at least, it still seems pretty messy. The inability to extract a training set from a model adds a lot of ethical ambiguity to generative images, as does the ambiguity in who’s responsible for what the models produce. I think it would be utterly ridiculous to say that someone training models with CSAM has no culpability in what it produces— only for possessing the CSAM to begin with— but I also think it would be utterly ridiculous to hold people accountable for everything their models produce, given their flexibility. What about writing a prompt that generates CSAM inadvertently? What if nobody involved intended to make CSAM but through some algorithmic shenanigans the prompt produced it? Should we legally require some amount of model testing before it’s used? Would the tester be violating the law if it failed the test, even if they reviewed every single image in the training set? Who’s responsible for deliberately poisoned models with secret key terms, or malicious data that is not CSAM but can trick the model into creating it? Would some entity like Midjourney that not only provides a model, but a complete appliance for this process be responsible for the images it produced? Does it matter if they authored the models they use? What if users can upload or train their own models? How does automation ethically affect these considerations? Someone with legal expertise obviously would have a better grasp of these situations than I do, but I do know we’ve got a lot of growing pains en route with this technology.
- hiatus 3y agoHow do they determine the images are generated and not real? It's not mentioned in the article.
- danuker 3y agoIf there is no way to tell, then reasonable doubt will become much more prevalent.
- wharvle 3y agoIf the volume of novel images shot up 10x right after these tools started becoming accessible, that’d let you make a good guess that most of it’s not real.
- mdhb 3y agoIt’s very obviously not going to be a meaningful legal distinction in terms of possession nor should it be when you think about it for more than 5 seconds. It’s only particularly relevant in the context of terms of trying to identify the source of abuse and trying to stop or interrupt that.
- nordsieck 3y ago> It’s very obviously not going to be a meaningful legal distinction in terms of possession nor should it be when you think about it for more than 5 seconds. I've thought about it for 5 seconds, and it's not obvious to me. The whole possession of images is a crime is already a stretch, justified by the very real observation that the market for these images necessarily fuels abuse of real children. But AI images cut that link. I'm in complete agreement that the content is odious. But there's plenty of stuff that's odious and legal.
- shaky-carrousel 3y agoIt feels like making a crime of possessing fake ivory items.
- mdhb 3y ago
- kumarvvr 3y agoI am worried about the changes in laws this will bring. As on date, the state cannot force you to testify against your own interests. It is the states responsibility to ensure it finds the evidence to present to courts. With situations like this, the chipping away of rights will begin. How do the cops know which is real and which is fake? Make the convict tell. The problem is a tech problem, which has to be solved by techies. Otherwise, we will land up into a quagmire of privacy and state protections eroding, or the AI technology itself will be heavily regulated / banned.
- ghaff 3y ago>The problem is a tech problem, which has to be solved by techies. When has that ever been the way it worked? Or arguably should have worked?
- loceng 3y agoTL;DR - Could fake AI child abuse content be being pumped out as part of manufacturing consent from the general population to bring in more controlling policy to then implement a totalitarian state?
- shzhdbi09gv8ioi 3y agoThey are already using child protection as an argument to implement totalitarian measures. The most recent example that come to mind is the European Union Chat Control, which were meant to break all e2e encryption, enforcing on-device data scanning and fingerprinting to function.
- thaumasiotes 3y ago> How do the cops know which is real and which is fake? Make the convict tell. What would that accomplish? All you'll learn is that they're all fake.
- londons_explore 3y agoWhen other cultures try to impose their morals on the west, we say it is oppression (eg. compulsary headscarves). When we try to impose our morals on others, that's somehow totally different, and they need to follow our rules even if they're in a totally different country.
- LtWorf 3y agoHehehehehe, my code of conduct starts with "No USA cultural imperialism allowed" (https://codeberg.org/ltworf/international_code_of_conduct/src/branch/master/CODE_OF_CONDUCT.md https://codeberg.org/ltworf/international_code_of_conduct/sr...) Some people loved it, some others told me I'll never find a job because I'm a bad person who can't work with others (I have a job). edit: in case that wasn't clear, it still doesn't allow child porn…
- namaria 3y agoIronically in trying to transcend a conflict you never needed to transcend because you're not even a part of, you just sound like you picked a side.
- LtWorf 3y agoIt is purely based on past experience of who opens bug reports to complain about things.
- namaria 3y agoAnd you chose to react to it by publishing something about it. It wasn't inevitable. You can always ignore irrelevant issues.
- LtWorf 3y agoCode of conduct is compulsory these days, or github nags you about lacking it. And I couldn't pick one of those that say "no skill discrimination", since that is just wrong. So since I had to write my own, might as well address the issues I've encountered.
- djoldman 3y agoIt will be interesting to see how this plays out between two different scenarios: 1. Real-world photographs of people, minors or otherwise, that are altered via photoshop/AI/hand/etc. and then disseminated 2. Images, that appear or seem to be of minors, created entirely through synthetic means (photoshop/AI/hand/etc.) and then disseminated Can drawings be illegal?
- Spivak 3y ago> Can drawings be illegal? Yes, very much so. https://en.m.wikipedia.org/wiki/Legal_status_of_fictional_pornography_depicting_minors https://en.m.wikipedia.org/wiki/Legal_status_of_fictional_po... Wikipedia calls the US a legal grey area but it really isn't. It's just a long road to, "yep still illegal."
- deleted 3y ago[deleted]
- shzhdbi09gv8ioi 3y ago> Can drawings be illegal? For sure. https://en.wikipedia.org/wiki/Legal_status_of_fictional_pornography_depicting_minors https://en.wikipedia.org/wiki/Legal_status_of_fictional_porn...
- djoldman 3y agoThanks for this! If wikipedia is correct, it seems that this is clearly answered. > The PROTECT Act also enacted 18 U.S.C. § 1466A into U.S. obscenity law:[122] > Section 1466A of Title 18, United States Code, makes it illegal for any person to knowingly produce, distribute, receive, or possess with intent to transfer or distribute visual representations, such as drawings, cartoons, or paintings that appear to depict minors engaged in sexually explicit conduct and are deemed obscene. > Thus, virtual and drawn pornographic depictions of minors may still be found illegal under U.S. federal obscenity law. The obscenity law further states in section C "It is not a required element of any offense under this section that the minor depicted actually exist." The difficult part of this is the phrase, "...that appear to depict minors..."
- A4ET8a8uTh0 3y agoIt is a steady drumbeat of 'think of the children'. I have a daughter I would personally love it if politicians and their ilk stopped thinking about children so hard ( and either found a better scapegoat or stop eroding right or neuter new tech ). It is annoying and, frankly, it is keeping us from evolving as a species.
- glimshe 3y agoI find child pornography abhorrent. But AI child pornography could actually help reduce child abuse - some abusers might get their fix from these images instead of needing actual children. This would be similar to the (yet unproven) claim that widespread access to porn has reduced physical sexual activity in general.
- jdiff 3y agoPeople have been saying that for decades before generative AI was a glimmer in anyone's eyes. Even the story we're commenting is of someone who was doing both. It's time to retire that tired theory.
- ben_w 3y agoAs with all interventions, it is a claim that can only be tested by comparing abuse rates in two otherwise very similar jurisdictions, where the only difference is the availability of such material. To think otherwise is to presume that all members of the target group are identical, which is implausible. Even attempting to run such a pure observational study of such a situation would be denounced, even if the researchers had no power to alter the laws. If you did somehow manage to run such a study, I expect you'd find three things: (1) plenty of examples of people who were inspired to act due to the images they saw, (2) plenty of examples of people who used the images as a way out, (3) that there are way more abusers and victims than most people realise.
- oatmeal1 3y agoWhat you're saying is the market-share of the AI version that does not involve additional exploitation of minors in its creation is going up over time. The evidence is increasing that the theory is right. If the AI version gets good enough, the supply of a substitute will real CP will go up, the price of real CP will go down, meaning there is less incentive to exploit real children.
- jdiff 3y agoThat's assuming a lot of things that are shockingly poor choices to assume. Like that the only reason children are exploited is for photographs.
- ajsnigrutin 3y agoThis will be a shitstorm... How do you determine the age of an AI generated personna?
- lm28469 3y agoIf it looks like a 8 years old you won't convince the FBI that it actually is a DnD character that's 106 years old under a spell of rejuvenation, and even if you do it won't matter for your sentencing.
- ajsnigrutin 3y agoI mean sure... but that's because there's an actual person photographed there. What happens when you get into "does (s)he look 17 or 18 in this photo?" and no way to prove anything, since the person does not exist?
- chefandy 3y agoI imagine the same way they do for actual photographs in which they haven’t identified the victim, which I’d guess is a pretty big percentage. Stuff like this is never cut-and-dried, but the law deals with ambiguous situations all the time. The test for what even constitutes pornography compared to, say, medical images involving genitals is a good example.
- ajsnigrutin 3y agoI mean sure... but that's because there's an actual person photographed there. What happens when you get into "does (s)he look 17 or 18 in this photo?" and no way to prove anything, since the person does not exist?
- harimau777 3y agoHow would they differentiate generated images of people who look young from generated images of people who are underage?
- mmvasq 3y agoWho is “law enforcement”. Not “cops” as per YCombinator headline. You can’t get a cop to do a thing about online content. I would really like to know who is feeling bogged down. I have a serious case for them that a person was actually drugged, trafficked and extorted.
- thelastparadise 3y agoYou need to contact FBI child enforcement immediately. FYI you could have some liability of you delay.
- mmvasq 3y agoShe is an adult. Yes adults get trafficked. Especially good looking ones falling into health and economic challenge. She has a daughter. Children are hurt when adults are trafficked- no one gets this. The trafficked party then used to extort me by someone actually pretending to be “cops” with cross platform analytics and data broker access. Craziest thing you ever could have imagined and only an extremely rich tech narcissist can come up with what they did to a woman already being trafficked. They did so to secondarily traumatize me.
- ethanbond 3y agoIf a child, contact NCMEC (National Center for Missing and Exploited Children) https://report.cybertip.org/ https://report.cybertip.org/ 24/7 phone line: +1-800-843-5678 Otherwise/in addition, FBI is a good bet.
- genmud 3y agoIf this is real, please hit up ICE C3 [1]. They specialize in investigations for this type of thing, specifically sexual exploitation and trafficking. [1] - https://www.ice.gov/partnerships-centers/cyber-crimes-center https://www.ice.gov/partnerships-centers/cyber-crimes-center
- explorigin 3y agoSo many commenters didn't read the article. > The troubling case seems to illustrate how AI-generated child sex images can be linked to real criminal activity while also showing how police investigations could be bogged down by attempts to distinguish photos of real victims from AI images that could depict real or fake children.
- cyanydeez 3y agosounds like doomers are still winning