9 ms·
The scary thing about this is that many people have images that look like child porn at home: Family photos of naked grandchildren playing at the sea or in the
by anfelor 5y ago
The scary thing about this is that many people have images that look like child porn at home: Family photos of naked grandchildren playing at the sea or in the mud, as well as sexting between consenting teenagers. There is no way automatic classification can distinguish these from real child porn, because they look almost identical and thus many private pictures will be shared with the government. This not only infringes the privacy of those people in the most drastic way possible, but might also lead to lawsuits that can tear families apart (think your father being accused of having CP only for it to turn out months later it was an old picture of yourself). The article mentions that 40% of CP investigations in Germany are started against minors (implied: through sexting), which shows that this concern is not purely theoretical at all.
This was not an issue with the old approach of comparing checksums against public CP images, but it will increasingly be if we use more AI algorithms for this.
EDIT: Probably Apples neural hashes that are currently in the news won't be vulnerable to this since they trained specifically to only detect changes in color/cropping/rotation. But we can't know for sure since the white paper is not that detailed and "naked kid on beach" photos all look really similar usually.
- aj3 5y agoAFAIK, ML image classification is used for checking user-generated content on social networks, forums, etc. Cloud storage, email and IM providers will continue using hashes (either exact or perceptual) to lower false positives.
- rory 5y agoHey, at least we'll be able to "solve" the "problem" of sexts between youths by having a bunch of middle aged men rifle through them.
- sundvor 5y agoJesus, yep. Big "Brother", always watching. What lives can we destroy today? Privacy is being hit hard now, on multiple fronts - it's very sad, scary.
- mkl95 5y agoNot only that, but child porn seems to mean a different thing in every place. The Netherlands and other European countries define the age of consent at 16, meaning some things that are perfectly legal there will be a ticket to prison in many countries. Now you don't just have an engineering problem, you will also have to spend a bunch of money to have some lawyers review your spec.
- Dma54rhs 5y agoThe age of consent goes low as 14 in Europe but that doesn't change the definition of CP. Probably they won't prosecute when both parties are underage and such so heavily but consent age doesn't change the fact that nudity is 18+.
- lucb1e 5y ago> nudity is 18+. You mean like in movies, those "can't see boobies before you're 18" labels? Because capturing a nude 13-year-olds is perfectly legal, just not in a pornographic manner. E.g. my Dutch biology book had nude ~6 and ~11 year old girls depicted (as well as an adult) for educating different stages of development.
- BiteCode_dev 5y agoNudity is 18+ in extremist countries like the Emirates or the USA. Sorry but in France, 30 years ago before american culture started to dictate where money come from, you would see genitals from all genders and ages in movies. Nobody found it offensive, because we didn't have a catholic puritan morality to impose its sick twisted vision of the human body on us. Meanwhile we sold car by showing cars, not hot girls. But this apparently is ok because money. It's also perfectly alright to show kids image of people killing each others in mass. Because this is sane, unlike nipples. And now this Apple crap. Edit: ok 50 years ago. I'm getting old it seems.
- ceilingcorner 5y ago> catholic puritan morality Might want to check your terms here.
- longtom 5y ago> 40% of CP investigations in Germany are started against minors This in particular seems like such a silly outcome of over-regulation and bureaucracy. Amazing. Research suggests most child sexual abuse does not have much of a long-term effect at all: https://emilkirkegaard.dk/en/wp-content/uploads/A-replication-of-the-meta-analytic-examination-of-child-sexual-abuse-by-Rind-Tromovitch-and-Bauserman.pdf https://emilkirkegaard.dk/en/wp-content/uploads/A-replicatio... This (of course!) does not mean it is good or should be done, but it suggests this is not an urgent issue, certainly not one that can justify abolishing privacy, especially when these measures only catch the small fish, not the large trafficking rings who will probably switch to different means of communication once theirs gets compromised.
- hnbad 5y agoYou're arguing that child sexual abuse is "no that bad" by citing a guy whose "research papers" include gems like * The left-liberal skew of Western media * What Happened to Brussels? The Big Decline and Muslim Immigration * Mental illness and the left * Human Biodiversity for Beginners: A Review of Charles Murray's Human Diversity * Race Differences: A Very Brief Review * Racial and ethnic group differences in the heritability of intelligence: A systematic review and meta-analysis * Global Ancestry and Cognitive Ability * Sex Distribution, Life Expectancy and Educational Attainment of Comedians * Immigrant crime in Germany 2012-2015 * Country of origin and use of social benefits: A large, preregistered study of stereotype accuracy in Denmark * Inequality in the United States: Ethnicity, Racial Admixtureand Environmental Causes * Increasing inequality in general intelligence and socioeconomic status as a result of immigration in Denmark 1980-2014 * Criminality and fertility among Danish immigrant populations He boasts 24 publications in Mankind Quarterly and 20 in OpenPsych, both of which he seems to run himself. Mankind Quarterly according to Wikipedia 'has been described as a "cornerstone of the scientific racism establishment", a "white supremacist journal", an "infamous racist journal", and "scientific racism's keepers of the flame"'. There are excellent cases to be made why privacy and encryption should not be compromised in the name of hot button issues like "protecting children" but citing a study by a "scientific racist" and eugenicist, who is a known advocate for legalizing child pornography, to trivialize child sexual abuse is not it. You don't need to be a "left liberal" not to cite Emil Kirkegaard. Being a decent human being or having any appreciation of actual science would suffice.
- lijogdfljk 5y ago> as well as sexting between consenting teenagers. Good god. I can't imagine the influx of in my view innocent teens suddenly labeled sex offenders because they did what all teens do, only virtually. This is going to be a weird time.
- ryandrake 5y agoIt might also turn into a good teachable moment for teens about privacy, sharing, and technology. I don’t take a picture on a smartphone of anything that I wouldn’t want to make public, and I surely wouldn’t share or post it anywhere. If it leaves your device, it’s effectively public, regardless of whether whoever you sent it to tells you they’ll keep it “private”.
- josephcsible 5y ago> If it leaves your device Even this is naïve with Apple's new plan. How long do you really think they'll wait before a software update makes it apply to even things that don't leave your device?
- angry_octet 5y agoThis would be a much bigger step though, and any brand doing it, even Apple, would face very negative market effect. It couldn't be legislated in the US because it would be unconstitutional to do so, almost impossible in the EU, possible in the UK/AU, even Canada (with the help of some creative rights busting from the Supreme Court, as they recently demonstrated).
- bennyp101 5y agoI mean technically that already happens in the UK - you can have sex legally at 16, but you can't take any pictures of you doing so until you are 18. So 2 consenting 17 year old teenagers cannot send indecent images of themselves to each other. I seem to remember a case from a few years back, about a girl who got herself on the register for sending her boyfriend a picture of her naked or something. Although it seems that the Protection of Children Act [1] I think (I'm not sure, it's hard to actually read with all the brackets ...) let's you do it if you are married (which you can also do at 16) Any sane person would know that teenagers do what teenagers do ... having someone potentially going through those photos is such a strange and wrong approach [1] https://www.legislation.gov.uk/ukpga/1978/37 https://www.legislation.gov.uk/ukpga/1978/37
- camgunz 5y ago"Child porn" is a smokescreen. It's what every privacy advocate saw coming: find the worst, most repugnant thing possible and use it to backdoor encryption. It's not a dream, and it's really happening, right now.
- imglorp 5y agoYes, given all the burgeoning authoritarian states where phones are sold it's clearly population control. Regardless of country, scanning for material critical of PRC, Lukashenko or Erdoğan is the exact same tech you might want to scan for copyright violations, union organizing, political party opposition, cryptocurrency, or any other dissent your local burgeoning authoritarian wants to pay for. Both phone OS vendors have over and over proven eager to satisfy every authoritarian, no matter the human impact. We're screwed.
- matheusmoreira 5y agoYeah. Children are the perfect political weapon. You can use children to justify anything and destroy the reputation of anyone who opposes you. It's reasonable to assume anyone using children as argument is acting in bad faith. I seriously doubt governments actually care about children. They probably care a great deal more about maintaining their own power and control over their subjects. We see countries like Russia and China using the tools of surveillance and law enforcement to silence political opposition and dissent.
- nswest23 5y ago> I seriously doubt governments actually care about children. Goverments don't care about children. Twenty-six first graders were massacred and the US government did nothing.
- deleted 5y ago[deleted]
- trangus_1985 5y agoAnd, they referred to us as the "screeching minority".
- notsureaboutpg 5y agoWhen I was in school, it was mentioned often during the school's programs on why sexting was bad that sexting was technically child pornography distribution. I don't understand what consent has to do with it. Children's consent is not legally valid for anything sexual.
- matheusmoreira 5y ago> The article mentions that 40% of CP investigations in Germany are started against minors (implied: through sexting), which shows that this concern is not purely theoretical at all. I've read news about this. Teenager sends nude photos to her boyfriend, calls the cops after the break up and they proceed to nearly ruin his life by threatening lifetime sex offender registration. The implication that the girl produced and distributed child pornography of herself is never even mentioned. It's completely insane.
- Lammy 5y agoWhy does this need to be gendered?
- Teever 5y agoAsk the real questions Lammy.
- matheusmoreira 5y agoIt's the story I remember reading on the news article. Anything else would be fabrication.
- deleted 5y ago[deleted]
- honkdaddy 5y agoThat's a terrifying story. Do you have a link or any other details?
- matheusmoreira 5y agoI'm searching for it but I can't seem to find the source anymore.
- cvwright 5y agoIn Apple's defense here, there seems to be a reason they're using the longer acronym CSAM. It means not just pictures of people without clothes on, but actual images/video of children being abused. Stuff that would likely provoke you or me or other normal people into acts of extreme violence against the perpetrators if we ever got the chance. The real danger with Apple's system is that there's zero accountability in the list of "bad" hashes. So there's no way to know whether it's really all CSAM, or if it also includes rare Pepes, Bernie memes, or pictures of politicians doing embarrassing things.
- deleted 5y ago[deleted]
- pseudalopex 5y ago> It means not just pictures of people without clothes on, but actual images/video of children being abused. Who said that? They're the same legally. People have said NCMEC's database includes entirely legal images. And groups like NCMEC have tried to rename child pornography for years.
- azinman2 5y agoBut the point is there’s an existing database. Either you have a photo in that database, or you don’t. So your kids naked photos don’t get triggered. And even if that photo is legal, not only is there human verification and minimum quantities required, it would be very strange for you to have in your iCloud photos that were part of CP collection sets regardless of content.
- pseudalopex 5y agoTheir point was CSAM means especially bad CP. Your points were different. It's a perceptual hash. Matches don't have to be exact. And people have engineered collisions for other perceptual hash algorithms. What a computer sees and what a person sees can be very different. People have said the human verification just involves the visual derivative of the suspect image. Apple didn't explain what that is really. And human verification doesn't reassure people who object to anyone viewing their private photos without their consent for any reason. People have said any images collected during an investigation go in the database. Do you think people who collect explicit photos of 17 year olds don't collect explicit photos of 18 year olds?
- dqpb 5y agoChild protective services will have a field day with this. Take away all the children!
- edanm 5y agoNot a lawyer, but my understanding is that in the States, sending naked pictures of teens is illegal, even if it's a picture of yourself. If you send naked selfies to someone, you could be charged with distribution of child pornography if you are a teenager. I'm also fairly sure I heard that even possessing naked selfies is considered illegal, at least in some states, e.g. this article about a teenager facing 10 years in prison for having pictures of himself naked at 16 years old on his phone: https://www.rollingstone.com/culture/culture-news/teenager-prosecuted-for-possessing-own-nude-selfie-66932/ https://www.rollingstone.com/culture/culture-news/teenager-p...
- kook_throwaway 5y agoThe law and morality are often fundamentally opposed. Saudi Arabia punishes homosexuality with the death penalty, and Apple happily removed the encrypted communications from their phones so they could continue selling in that country. Illegal or not, comparing two teens that sent naked selfies to each other is not even remotely similar to involuntary child abuse from an adult. >this article about a teenager facing 10 years in prison for having pictures of himself naked at 16 years old on his phone That anyone would even consider spending taxpayer money to lock up a 16 year old for his own picture shows a colossal failure of government on every level Legislative, judicial, and executive. The DA stacked 5 charges and saddled them with up to a decade so the kids would be afraid to even go to court. In any sane judicial system that case would be laughed out of the courtroom and it's telling that they were presumably advised by a lawyer to take the plea bargain.
- formerly_proven 5y ago> A Fayetteville, North Carolina teenager has reached a plea deal to avoid being charged with multiple sexual exploitation counts after his cell phone was found to contain nude selfies of himself. Seventeen-year-old Cormega Copening, who took the photos of himself when he was 16, agreed to the deal in order to avoid possible jail time and being registered as a sex offender. As part of the plea, the teen agreed to random police searches without warrant for one year as well as other penalties, Fusion reports. The teenager was listed as both the victim and the perpetrator on the sexual exploitation charges. This is quite possibly the most kafkaesque "justice" story I have ever heard coming out of America and that really says something.