4 ms·
If you can get exact collisions, this can be gamed. For example, suppose there are two rival gangsters. One wants to set the police on his rival. He knows that
by ajb 5y ago
If you can get exact collisions, this can be gamed. For example, suppose there are two rival gangsters. One wants to set the police on his rival. He knows that a certain (innocuous) image is on his rival's phone. So he pays someone to generate a fake child-porn image with the same neuralhash, and ensure that it gets into the child porn DB. Then, apple reports the rival to the police, and they come and investigate him. OF course, they may notice that the image isn't the right one, but by that time they may have found other incriminating evidence.
Not sympathetic to a rival gangster? Ok lets find an innocent victim: not a rival criminal, but an innocent witness who our protag wants to intimidate. Gangster wants to intimidate the witness, but can't get at them, so cooks up a scheme to convince the witness that the police are in his pocket. Exactly as above, causing the police to investigate the witnesses phone.
Another one might be, a certain government wants to identify opposition groups using images associated with them . Apple is not keen to be associated with that, but the government can simply generate fake child-porn (remember, programmatically generated CP is just as illegal) for each image of interest.
- monkeynotes 5y agoI think before a criminal investigation, or any investigation at all is pursued, a human verifying the images would dismiss the false positive. I would think surreptitiously placing actual child porn on a rival's phone/computer would be much, much more effective. Cybercriminals could likely do all this remotely. Phish for apple account login, upload images. Done.
- iamtheworstdev 5y agoyou think the US government would respect the law before initiation of actions to take down a criminal? they rarely do that when dealing with people that dont have criminal backgrounds
- KingOfCoders 5y agoSo you think an underpaid worker in India is interested in the upkeeping of innocence of a person they don't know in the US? They'll just flag the images as fast as possible, they are probably paid by pictures-reviewed. "Amazon has people transcribing audio in Costa Rica, India and Romania." according to Engadget. So it would be safe to assume Apple does something along those lines, I can't remember where Siri transcripts have been sent when there was the Alexa controversy. From there it's an entry with law enforcement and you need to find a way to convince them that the images are your children. And if data isn't cleared correctly and you have another run in with law enforcement, there will always be this picture-stuff.
- nullc 5y ago> a human verifying the images would dismiss the false positive How is a human supposed to distinguish that a visual derivative (a low res sobel filtered image, presumably) of ordinary, lawful, adult pornography isn't child porn when the system has already identified it as such? I agree that using real child porn is an attack too, but at least in that case you could say the system was doing as designed (even though what its doing shouldn't be something that we want) ... but it's not even guaranteed to do as designed.
- monkeynotes 5y agoAre you saying humans cannot visually inspect the actual image that the flag is set against? If not, then how on earth do you prosecute someone if you can't demonstrate they had the actual image, and not just the derivation used to flag it? The way I see it working is Apple scans a ton of shit, some of it shows up as possible child porn, human intervenes and looks at the source images, if there is indeed child porn they report to the police.
- nullc 5y agoApple's review has no access to the matched images in the database, just a 'derivative' of your image. Hopefully the prosecutors will check against the real images. But by that point your privacy has been damaged and your reputation might be trashed.
- deleted 5y ago[deleted]
- Someone 5y agoIf your enemy can get an image on your phone and in the “child porn DB”, I think they can easily get you in trouble without Apple’s help. they can either just send the police an anonymous message or set up a child porn web site and have it ‘accidentally’ leak its password database, and make sure your email address is in it.
- ajb 5y agoThe point is that they don't need to get an image on your phone. They can just choose one they know is there.