5 ms·
It might be useful to read the threat model document. Associated data from client neural hash matches are compared with the known CSAM database again on the ser
by shuckles 5y ago
It might be useful to read the threat model document. Associated data from client neural hash matches are compared with the known CSAM database again on the server using a private perceptual hash before being forwarded to human reviewers, so all such an attack would do is expose non-private image derivatives to Apple. It would likely not put an account at risk for referral to NCMEC. In this sense, privacy is indeed preserved versus other server scanning solutions where an adversarial perceptual hash (PhotoDNA is effectively public as well, per an article shared on HN) would trigger human review of all the account’s data.
- nlitened 5y ago> before being forwarded to human reviewers Does that mean that Apple employs people who manually review images known to be child pornography 9-to-5? Is it legal?
- shuckles 5y agoYes, and so does every other major cloud service provider. The working conditions of these people are notoriously difficult and should be the subject of attention.
- deleted 5y ago[deleted]
- kemayo 5y agoI'd imagine the key is that it's "manually review images suspected to be child pornography". The point of the review process is presumably that there are possible hash-collisions / false-positives, so the reviewers are what cause that transition from suspected to known.
- y7 5y agoI assume these "non-private image derivatives" are downscaled versions of the original image. But for downscaling there also are adversarial techniques: perturb an image such that the downscaled version looks like a target image. See https://bdtechtalks.com/2020/08/03/machine-learning-adversarial-image-scaling/ https://bdtechtalks.com/2020/08/03/machine-learning-adversar...
- shuckles 5y agoFor the server side component, the challenge is knowing the target of adversarial generation since, again, the perceptual hash used is secret. At that point, you are reduced to the threat present in every other existing CSAM detection system.
- y7 5y ago> the perceptual hash used is secret Yes, although I'm sure a sufficiently motivated attacker can obtain some CSAM that they are reasonably sure is present in the database, and generate the NeuralHash themselves. > At that point, you are reduced to the threat present in every other existing CSAM detection system. A difference could be that server-side CSAM detection will verify the entire image, and not just the image derivative, before notifying the authorities.
- zepto 5y ago> Yes, although I'm sure a sufficiently motivated attacker can obtain some CSAM that they are reasonably sure is present in the database, and generate the NeuralHash themselves Remind us what the attack is here? The neural hash and the visual derivative both have to match for an image to trigger detection.
- shuckles 5y agoI believe something like: an adversary creates an innocuous photo where: * The photo itself is benign. * The photo’s NeuralHash matches known CSAM. * The photo’s image derivative is not benign. It looks visually like CSAM. * The photo’s image derivative matches known CSAM per a private perceptual hash. The above, combined, could have a victim reported to NCMEC without being aware they were targeted. Since Apple operates on image derivatives, they could be fooled unlike other cloud providers. That is the claim. At that point, the victim could point law enforcement to the original CloudKit asset (safety vouchers include a reference to the associate asset) and clear their name. However, involving law enforcement can always be traumatic.
- TechBro8615 5y agoPrivacy is preserved by a team of humans looking at your private photos. Sounds like a good deal. What are we preserving again?