3 ms·

Nobody's confused about that. > HN is a tech forum and I still saw plenty of people here worried about how they would get in trouble for having innocent photos

Nobody's confused about that. > HN is a tech forum and I still saw plenty of people here worried about how they would get in trouble for having innocent photos of their own children on their phone. They're confused about this. NeuralHash doesn't look for pictures of naked kids. It looks for pictures that are identical to the ones they've put in their signatures list. The problem is that Apple claims that the signatures in their list are all pictures of sexually-abused kids, but we have no way of verifying that. Heck, they don't even have any way of verifying that. Everyone just has to take NCMEC's word for it.

lazide 5y ago
You’re also wrong no? The perceptual hashing doesn’t match EXACTLY the same photos, it intentionally matches photos similar to the same photos, so a minor rotate by a few degrees or crop or whatever also matches
cwkoss 5y ago
Perceptual hashing can have collisions, and they will be at a higher rate than completely "identical". The public does not know what the false positive rate is for 'average iphone user pictures'. As engineers we can be certain the false positive rate is not zero. This means that some number of iphone users are going to have non zero "possible child pornographer" scores in the apple database. The false positive rate is crucial to understanding how concerning this should be. If the average iphone user has 1000 photos, and the false positive rate is the claimed 1 in 1 trillion, there is a 1 in a billion chance that you'll be flagged as a potential child pornographer. (~1 in the world will be falsely accused). This seems reasonable enough with the apple-internal screening step. If the chunking and perceptual hashing functionally ends up having a much higher false positive rate for images which have similarities to the dataset (parents' pictures of kids playing shirtless, legal adult porn, etc), the false positive rate could actually be more like 1 in 1 million or worse. In which case there are potentially hundreds of thousands of people who will be falsely accused by this system. How many matches will US judges require before they sign warrants for arrests, search and seizure of digital devices? If they are technically competent it shouldn't only be 1, but I don't trust all judges to understand probability well enough to require multiple matches.
- hypothesis 5y ago
  Even if we put aside collisions, new system features ‘Synthetic Match Vouchers’, which is seemingly adding noise into actual CP counter. I yet to understand what happens to people who only have those synthetic positives? Regardless of what counter threshold is, can’t those people be hoovered up by a subpoena of counter >0 ?
  - cwkoss 5y ago
    Oh yikes, I didn't realize they were handing out fake CP points to preserve privacy. That's really really really user-hostile design.
- FireBeyond 5y ago
  > How many matches will US judges require before they sign warrants for arrests, search and seizure of digital devices? If they are technically competent it shouldn't only be 1, but I don't trust all judges to understand probability well enough to require multiple matches. Even that doesn't come without issue. How long before '1' becomes the value, because, say for example the number is ten, there's also a horrendous PR spin of "Apple has a high degree of suspicion that you have CSAM on your device, but since there's only 8 images, they won't do anything about it" - "Apple allows up to X non-reported CSAM images on Apple devices" is hard to represent in any positive fashion.

lazide 5y ago

You’re also wrong no? The perceptual hashing doesn’t match EXACTLY the same photos, it intentionally matches photos similar to the same photos, so a minor rotate by a few degrees or crop or whatever also matches

cwkoss 5y ago

Perceptual hashing can have collisions, and they will be at a higher rate than completely "identical". The public does not know what the false positive rate is for 'average iphone user pictures'. As engineers we can be certain the false positive rate is not zero. This means that some number of iphone users are going to have non zero "possible child pornographer" scores in the apple database. The false positive rate is crucial to understanding how concerning this should be. If the average iphone user has 1000 photos, and the false positive rate is the claimed 1 in 1 trillion, there is a 1 in a billion chance that you'll be flagged as a potential child pornographer. (~1 in the world will be falsely accused). This seems reasonable enough with the apple-internal screening step. If the chunking and perceptual hashing functionally ends up having a much higher false positive rate for images which have similarities to the dataset (parents' pictures of kids playing shirtless, legal adult porn, etc), the false positive rate could actually be more like 1 in 1 million or worse. In which case there are potentially hundreds of thousands of people who will be falsely accused by this system. How many matches will US judges require before they sign warrants for arrests, search and seizure of digital devices? If they are technically competent it shouldn't only be 1, but I don't trust all judges to understand probability well enough to require multiple matches.

hypothesis 5y ago
Even if we put aside collisions, new system features ‘Synthetic Match Vouchers’, which is seemingly adding noise into actual CP counter. I yet to understand what happens to people who only have those synthetic positives? Regardless of what counter threshold is, can’t those people be hoovered up by a subpoena of counter >0 ?
- cwkoss 5y ago
  Oh yikes, I didn't realize they were handing out fake CP points to preserve privacy. That's really really really user-hostile design.
FireBeyond 5y ago
> How many matches will US judges require before they sign warrants for arrests, search and seizure of digital devices? If they are technically competent it shouldn't only be 1, but I don't trust all judges to understand probability well enough to require multiple matches. Even that doesn't come without issue. How long before '1' becomes the value, because, say for example the number is ten, there's also a horrendous PR spin of "Apple has a high degree of suspicion that you have CSAM on your device, but since there's only 8 images, they won't do anything about it" - "Apple allows up to X non-reported CSAM images on Apple devices" is hard to represent in any positive fashion.