3 ms·

Perceptual hashing can have collisions, and they will be at a higher rate than completely "identical". The public does not know what the false positive rate is

by cwkoss 5y ago

Perceptual hashing can have collisions, and they will be at a higher rate than completely "identical". The public does not know what the false positive rate is for 'average iphone user pictures'. As engineers we can be certain the false positive rate is not zero. This means that some number of iphone users are going to have non zero "possible child pornographer" scores in the apple database. The false positive rate is crucial to understanding how concerning this should be. If the average iphone user has 1000 photos, and the false positive rate is the claimed 1 in 1 trillion, there is a 1 in a billion chance that you'll be flagged as a potential child pornographer. (~1 in the world will be falsely accused). This seems reasonable enough with the apple-internal screening step. If the chunking and perceptual hashing functionally ends up having a much higher false positive rate for images which have similarities to the dataset (parents' pictures of kids playing shirtless, legal adult porn, etc), the false positive rate could actually be more like 1 in 1 million or worse. In which case there are potentially hundreds of thousands of people who will be falsely accused by this system. How many matches will US judges require before they sign warrants for arrests, search and seizure of digital devices? If they are technically competent it shouldn't only be 1, but I don't trust all judges to understand probability well enough to require multiple matches.

hypothesis 5y ago
Even if we put aside collisions, new system features ‘Synthetic Match Vouchers’, which is seemingly adding noise into actual CP counter. I yet to understand what happens to people who only have those synthetic positives? Regardless of what counter threshold is, can’t those people be hoovered up by a subpoena of counter >0 ?
- cwkoss 5y ago
  Oh yikes, I didn't realize they were handing out fake CP points to preserve privacy. That's really really really user-hostile design.
FireBeyond 5y ago
> How many matches will US judges require before they sign warrants for arrests, search and seizure of digital devices? If they are technically competent it shouldn't only be 1, but I don't trust all judges to understand probability well enough to require multiple matches. Even that doesn't come without issue. How long before '1' becomes the value, because, say for example the number is ten, there's also a horrendous PR spin of "Apple has a high degree of suspicion that you have CSAM on your device, but since there's only 8 images, they won't do anything about it" - "Apple allows up to X non-reported CSAM images on Apple devices" is hard to represent in any positive fashion.

by cwkoss 5y ago

hypothesis 5y ago

Even if we put aside collisions, new system features ‘Synthetic Match Vouchers’, which is seemingly adding noise into actual CP counter. I yet to understand what happens to people who only have those synthetic positives? Regardless of what counter threshold is, can’t those people be hoovered up by a subpoena of counter >0 ?

cwkoss 5y ago
Oh yikes, I didn't realize they were handing out fake CP points to preserve privacy. That's really really really user-hostile design.

FireBeyond 5y ago

> How many matches will US judges require before they sign warrants for arrests, search and seizure of digital devices? If they are technically competent it shouldn't only be 1, but I don't trust all judges to understand probability well enough to require multiple matches. Even that doesn't come without issue. How long before '1' becomes the value, because, say for example the number is ten, there's also a horrendous PR spin of "Apple has a high degree of suspicion that you have CSAM on your device, but since there's only 8 images, they won't do anything about it" - "Apple allows up to X non-reported CSAM images on Apple devices" is hard to represent in any positive fashion.