3 ms·

Clearly this is not a cryptographic hash, and hence it's known hashes are not uniformly distributed. Apple explained in their technical summary [0] that they'l

Clearly this is not a cryptographic hash, and hence it's known hashes are not uniformly distributed. Apple explained in their technical summary [0] that they'll only consider this an offence if a certain number of hashes match. They estimated the likelihood of false positives there (they don't explain which dataset was used, but it was non-CSAM naturally) is 1 out of a trillion [1] In the very unlikely event where that 1 in a trillion occurrence happens, they have manual operators to check each of these photos. They also have a private model (unavailable to the public) to double-check these perceptual hashes which also used before alerting authorities. [0] https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni... [1] https://www.zdnet.com/article/apple-to-tune-csam-system-to-keep-one-in-a-trillion-false-positive-deactivation-threshold/ https://www.zdnet.com/article/apple-to-tune-csam-system-to-k...

tgv 5y ago
> they have manual operators to check each of these photos For now. But what will happen when there are thousands of false positives per day? Will they increase the staff? Or will they add another algorithmic layer? Or just up the threshold a bit? There's no guarantee. The only thing that's certain is that the NeuralHash doesn't inspire confidence.
- snowwrestler 5y ago
  Regardless of what Apple does, law enforcement must always manually review suspected CSAM before requesting a warrant based on it. So the idea that you could SWAT someone with some hash collisions on innocent images is just not possible. At most you could maybe temporarily lock someone’s iCloud account. But again, the collisions would need to be multiple and all look like CSAM at reduced resolution. In general, it seems not correct to think about NeuralHash like SHA or RSA. It’s not a cryptographic system and collisions are not a one-step endgame.
  - bigwavedave 5y ago
    > Regardless of what Apple does, law enforcement must always manually review suspected CSAM before requesting a warrant based on it. So the idea that you could SWAT someone with some hash collisions on innocent images is just not possible. A quick search found four clear cases where law enforcement has favored technological false positives over evidence: Ousmane Bah: https://www.businessinsider.com/teen-sues-apple-1-billion-false-arrest-facial-recognition-2019-4?op=1 https://www.businessinsider.com/teen-sues-apple-1-billion-fa... Robert Williams: https://www.cbsnews.com/news/facial-recognition-60-minutes-2021-05-16/#app https://www.cbsnews.com/news/facial-recognition-60-minutes-2... Nijeer Parks: https://www.cnn.com/2021/04/29/tech/nijeer-parks-facial-recognition-police-arrest/index.html https://www.cnn.com/2021/04/29/tech/nijeer-parks-facial-reco... Michael Oliver: https://www.dailydot.com/debug/detroit-facial-recognition-wrongful-arrest/ https://www.dailydot.com/debug/detroit-facial-recognition-wr... The one on Ousmane Bah really frustrates me- not only was he on a date at prom during the theft, he was in another state! "Nothing to hide" does not mean "nothing to fear" and allegations (even false ones) of possessing CSAM will ruin lives. At the end of the day, what it really comes down to is trust; personally, I do not have enough faith in due process to not ruin innocent people. But I'm just some guy online, I will readily admit I don't know anything about anything.

tgv 5y ago

> they have manual operators to check each of these photos For now. But what will happen when there are thousands of false positives per day? Will they increase the staff? Or will they add another algorithmic layer? Or just up the threshold a bit? There's no guarantee. The only thing that's certain is that the NeuralHash doesn't inspire confidence.

snowwrestler 5y ago
Regardless of what Apple does, law enforcement must always manually review suspected CSAM before requesting a warrant based on it. So the idea that you could SWAT someone with some hash collisions on innocent images is just not possible. At most you could maybe temporarily lock someone’s iCloud account. But again, the collisions would need to be multiple and all look like CSAM at reduced resolution. In general, it seems not correct to think about NeuralHash like SHA or RSA. It’s not a cryptographic system and collisions are not a one-step endgame.
- bigwavedave 5y ago
  > Regardless of what Apple does, law enforcement must always manually review suspected CSAM before requesting a warrant based on it. So the idea that you could SWAT someone with some hash collisions on innocent images is just not possible. A quick search found four clear cases where law enforcement has favored technological false positives over evidence: Ousmane Bah: https://www.businessinsider.com/teen-sues-apple-1-billion-false-arrest-facial-recognition-2019-4?op=1 https://www.businessinsider.com/teen-sues-apple-1-billion-fa... Robert Williams: https://www.cbsnews.com/news/facial-recognition-60-minutes-2021-05-16/#app https://www.cbsnews.com/news/facial-recognition-60-minutes-2... Nijeer Parks: https://www.cnn.com/2021/04/29/tech/nijeer-parks-facial-recognition-police-arrest/index.html https://www.cnn.com/2021/04/29/tech/nijeer-parks-facial-reco... Michael Oliver: https://www.dailydot.com/debug/detroit-facial-recognition-wrongful-arrest/ https://www.dailydot.com/debug/detroit-facial-recognition-wr... The one on Ousmane Bah really frustrates me- not only was he on a date at prom during the theft, he was in another state! "Nothing to hide" does not mean "nothing to fear" and allegations (even false ones) of possessing CSAM will ruin lives. At the end of the day, what it really comes down to is trust; personally, I do not have enough faith in due process to not ruin innocent people. But I'm just some guy online, I will readily admit I don't know anything about anything.