9 ms·
UK Cops Want to Use AI to Spot Child Abuse But It Mistakes Desert Pics for Nudes
- demosthenes111 9y agoI suspect part of the problem with this approach relates to data set construction. Even though models these days should theoretically be able to handle this task, there are clear ethical concerns and practical issues with making datasets of illegal imagery large enough for training. It really raises the question - is creating a dataset like that ethical, assuming the intentions are to stop further abuse and dissemination? There might be work arounds (like training one model for nudity and another for age) but such approaches are almost certain to have "suboptimal" performance as compared to a single model trained on a relevant dataset. Maybe something like that is the cause for the performance issues discussed in the article.
- gizmo686 9y agoEthical is a tricky issue here. Not everyone agrees that mere possession of such pictures is unethical. This still leaves the obvious legal problems of having a database of illegal pictures, but we already let law enforcement do otherwise illegal things (including use of illegal pictures in some contexts)
- soundwave106 9y agoHonestly, if they stored evidence collected from previous cases, the data probably already exists in some form. As long as the data was closely guarded internally, I'm not sure if there would be an ethics problem using this for training. The biggest problem with obscenity detection, though, is getting the context right. The AI might be able to get to the point where it can detect "naked human" at a good percentage level. At the moment, however, I doubt it could detect whether the naked human was considered obscene in current culture, eg: the difference between "child pornography" and "famous Vietnam War photo" as alluded to in the Gizmodo article. So no matter how good their model gets, without further refinements in AI it would only be good for a "first pass", I would think.
- jjp 9y agoGizmodo rewrote http://www.telegraph.co.uk/technology/2017/12/18/artificial-intelligence-will-detect-child-abuse-images-save/ http://www.telegraph.co.uk/technology/2017/12/18/artificial-... and didn't include some interesting details.
- Pica_soO 9y agoOh, you dirty little sand dune you. You like that mouse hopping along your edges, dont you? He who controls the spice, controls the universe!
- emmelaich 9y agoNot unexpected. It's surprisingly difficult to tell your arse from your elbow: http://stupidstuff.org/ass_elbow/ http://stupidstuff.org/ass_elbow/
- tritium 9y agoBut... I... I just want to know the answers. That reset quiz button... It’s... Why?
- henpa 9y agolol! The missing "submit" and only a "reset" button also drove me nuts!!! That page actually does some javascript code, which is not currently working. (you can see this when you click on one of the options while in console/devtools).
- tritium 9y agoOh, ha! Awesome! I was killing time on my phone, without full debug options, and from the looks of the site, I figured maybe they were being extra cute.
- dukes_haven 9y agoIts a ruse, they just want you to look at random people's backside and think that it's a elbow. The modern day version of scanning your ass and printing it lol
- lfowles 9y agoFrom the source: false, true, true, false, true, true, false, false, true, true, true, false, true, false (go left to right, top to bottom)
- codefined 9y agoFalse & true aren't great terms to use for an "either or" question.
- raverbashing 9y agoThere are some training techniques that rely on negative examples to improve classification, though I'm not sure they exist for images
- microcolonel 9y agoWell, this can somewhat be accomplished by classifying exactly what an image isn't, but as you can imagine, that is (in the general case) a combinatoric nightmare.
- yorwba 9y agoYou can just do binary classification. That makes the decision boundary a bit more complicated, but if you're using neural networks, that is solved by adding another layer at the end. The "problem" here is probably not the lack of negative training data, but just that they want as few false negatives as possible, so they use a very sensitive model. That naturally increases the number of false positives. It's still a win if you only have to look at some desert pictures instead of literally every picture on someone's computer.
- raverbashing 9y agoYou don't do it for all of them, just for the ones that are getting confused
- nitwit005 9y ago> With the help of Silicon Valley providers, AI could be trained to detect abusive images "within two-three years", Mr Stokes said. If the goal is to get something so high quality that the police won't ever have to look at it, I doubt it. You have Google and Facebook employing people to deal with user reports about this stuff, as well as ad companies manually checking websites.
- IanCal 9y agoThere would be a significant benefit to just being able to reliably get rough numbers plus some examples of the worst pictures for an expert to grade rather than having the task of searching 10000 horrific pictures to see if any fall into certain categories.
- Brotkrumen 9y agoThat's what their story is, but unlikely to be true. All "horrific" pictures will have to be graded by a human, because you have the right to face your accuser, the service providers will not let you near their codebase and AI won't be 100% accurate in the next few years. What they will be able to do with that software is trawl through orders of magnitudes more pictures, filter by probable hits and show those to staff. Right now someone has to be suspected to warrant the expensive attention of an expert. With that software you can fish in everyone's data.
- IanCal 9y agoCases reported in the media usually talk about people having thousands of pictures each, I wasn't talking about thousands of different people.
- Brotkrumen 9y agoThats the point. None of the 90% normal pictures will do any psychological damage, the 10% illegal pictures will but those will have to be looked at anyway. The advantage of an algo detecting illegal content isn't in protecting civil servants, it's scanning everyone without them.
- phillywiggins 9y agoAnyone else have an Arrested Development flashback? :)
- dpiers 9y agoClose up, they always look like landscape...
- thecopy 9y agoA bigger question is why does the police spend time looking for pornography? What type of society is the UK developing into?
- ry_ry 9y agoIsn't this specifically intended to identify images of child abuse though, rather than rifle through the average users pornhub cache?
- beaconstudios 9y agoYes, but we are in fact getting a nation wide porn filter in the near future.
- shocks 9y agoAre we? Source?
- have_faith 9y agoI believe they are referring to the opt-out default filter isp's have been enabling. Not sure on the laws surrounding it.
- edjw 9y agoNo laws requiring ISPs to apply filters. It's just a voluntary agreement between ISPs and the Government. The Digital Economy Act 2017 has provisions in it to allow ISPs to apply default filters if subscribers agree to them in terms of service. There are requirements in the DEA 2017 for 'adult' websites to verify the age of people who visit the site with the threat of their website being blocked if they don't comply. Check openrightsgroup.org for more on this
- noir_lord 9y agoI think they just want to introduce a generation of teenages to VPN's because any sane person can see how this is going to go, that's assuming any site outside the UK even bothers to comply.
- peter303 9y agoThe Silicon Valley television show had the best parody: a deep learning program trained to find hot dogs for a foodie program repurposed for a phallus porn detection program.
- avh02 9y agoHaving grown up in Saudi, I had heard that they had (literally) the same issue when MMS was originally introduced many years ago - but they have a lot more desert and are simply looking for porn, not even anything abusive.
- lostlogin 9y agoGiven that children are forced into marriage in Saudi, it would seem strange for them to care about images of the abuse they condone. A small bit of consistency at least.
- pvaldes 9y agoperl -e 'print "yep, is porn\n" if $filename =~ /(Porn|Sex|XXX)/;' Should spot the 99% of cases.
- guitarbill 9y agoperl -e 'print "yep, is porn\n" if $filename =~ /(.avi|.mp4|.wmv)$/;
- edjw 9y agoThat headline. FFS. It's either child abuse or pornography. It can't be both.
- efdee 9y agoSure it can. One person's abuse can be another person's pornography. Disheartening but no less true.
- bionoid 9y ago> One person's abuse can be another person's pornography. That is accurate for virtually all pornography. If you research how the "industry" works, you will quickly discover that it's built on advanced psychology, specifically hypnosis, torture, culture of fear and other brainwashing techniques. The porn cult is every bit as vicious, evil and destructive to women as any cult you can name.
- zimablue 9y agoI'm not familiar with the wider thesis you seem to be advocating, (could we get a link?) but calling porn a "cult" seems like a stretch, it's not something that has been created by a group and spread through brainwashing, porn is throughout history and across cultures.
- bionoid 9y agoI don't have a link that explains the situation well, which is why I am [slowly] working on a documentary project to bring it to the public. > calling porn a "cult" seems like a stretch That is the general perception, but as I said initially, girls are indoctrinated and trained using advanced psychological manipulation, indistinguishable from what most people think a "cult" is. Everyone is free to believe that massive numbers of young girls just absolutely love the taste of cum, rimming, and so on. Nothing fishy is going on...
- M2Ys4U 9y ago
- wiredfool 9y agowell, they can kiss my shiny metal: https://www.flickr.com/photos/wiredfool/901581193/ https://www.flickr.com/photos/wiredfool/901581193/ And then there's the classic Weston photograph: https://en.m.wikipedia.org/wiki/Pepper_No._30 https://en.m.wikipedia.org/wiki/Pepper_No._30 (Edit, IOW, not a new problem, photographers have been pushing at this sort of issue for a long time)
- gaius 9y agoThe UK police are dealing with a scandal right now in that they tend to suppress electronic evidence proving a suspect is innocent and try to prosecute anyway. The last thing we should give them is another electronic capability. http://www.telegraph.co.uk/news/2017/12/19/met-orders-review-rape-cases-second-trial-collapses/ http://www.telegraph.co.uk/news/2017/12/19/met-orders-review...
- whathaschanged 9y agoThe UK police are dealing with a scandal where they supress evidence physical child rapists are guilty if they happen to be muslim.
- gaius 9y ago"Men alleged to be part of a child-grooming ring in Rochdale were never brought to justice because the evidence of teenage victims was not pursued, according a former police officer" http://www.bbc.co.uk/news/uk-21925205 http://www.bbc.co.uk/news/uk-21925205
- sctb 9y agoWould you mind commenting civilly and substantively, please? https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- YeGoblynQueenne 9y agoSo, the Met wants to train image recognition AI to find child abuse images? I'd be surprised if there aren't some interesting ethical questions around this. To begin with, the way image recognition algorithms work -they learn from examples- to train an image recognition AI to identify images of child abuse you'd have to provide it with a substantial database of such images. I'm pretty sure that's illegal, although of course the Met probably has some sort of exception for the purpose of fighting crime. But, they still have to keep around a big database of child abuse images - and update it regularly, to do this job. It's kind of... icky. Then again, what about victim protection? Besides the fact that keeping such a database is risky because it can always fall into the wrong hands (it only takes one misconfigured database server) it's also a characteristic of the most advanced image recognition algorithms (conv-nets) that their models can be used to generate new images of the kind they've learned to identify. So presumably, if the model itself fell into the wrong hands, someone could start generating new child-abuse images from it (low-fidelity and not really very useful, I imagine, but still). So I wonder if the Met has addressed such ethical issues, anywhere.
- coding123 9y agoLaw enforcement already has tons of child porn lying around used primarily to frame people. Of course similar to drug planting it is extremely hard to defend because no one wants to risk defending someone charged with that. There are exceptions and I'm not saying people like Jared from Subway aren't guilty. There are many many many lesser known individuals that are hushed up using child porn. So I'm basically saying there is no ethical problem (from their perspective) because feds/cops already deal with this all the time and plant this on people like drugs.
- gamblor956 9y agoYou're wrong on literally all points, but why let the truth get in the way? Law enforcement does collect images that have been traded. They do not use these to frame people, unless you are referring to the honeypot sites that they have taken over. They maintain these sites hoping that CP collectors will continue to download images of their own volition and that they can they trace the IPs to a real-world address. There are many many many lesser known individuals that are hushed up using child porn. Citation needed. This would the first I've heard of CP being used to frame people by law enforcement. It happens, though very rarely, that some people try to frame innocent individuals... So I'm basically saying there is no ethical problem (from their perspective) because feds/cops already deal with this all the time and plant this on people like drugs. So I'm basically saying there is no ethical problem because feds/cops already deal with this all the time, but they don't frame people. They let the idiots hang themselves. And by the way, my source is that I used to defend people like this when I was a public defender. What's your source?
- ajmurmann 9y agoI don't get it. Who thinks it's a good idea to fully automate anything like this? Regardless of how accurate the algorithm is a human should look at this before any action is taken. As long as the aren't false negative this is probably a huge win. Even if we have 5% false positives that hugely cuts down at what people need to look at. So desert pictures looking like child porn doesn't matter much. Child porn looking like something else is much worse. Edit: awful typo "lol" -> "look"
- tomcam 9y ago> Stokes told The Telegraph that the department is working with “Silicon Valley providers” to help train the AI to successfully scan for images of child abuse AI requires large image sets to do the training. These images are illegal to possess. I wonder how such training gets done without violating the law. However, I do not wonder enough to do web searches about this matter. I am probably on enough government lists already