5 ms·

It's hard to reply to your post in the spirit of HN in assuming best intentions. But please suppose most of the readers here do understand how the system works

It's hard to reply to your post in the spirit of HN in assuming best intentions. But please suppose most of the readers here do understand how the system works, and most of them don't approve how it works.

wmf 5y ago
I understand how the system works so I know that hash collisions are not dangerous because they have to be manually reviewed by multiple humans. Since you understand how the system works, please explain the danger of hash collisions instead of meta-commenting.
- smoldesu 5y ago
  I'm not them, but they have a point. This ends up getting re-explained in every single one of these threads, so I'll keep it brief: - There are two main components here: automated flagging and supervised, human confirmation - It has been proven that the automated flagging system can be fooled - The last line of defense is now a human: they have to look at an obfuscated version of your colliding image to determine if it's objectionable The current discussion is mostly around the second portion, moreover what kind of social-engineering attacks would be most effective to overcome a relatively flimsy single-human point of failure.
  - zepto 5y ago
    > - It has been proven that the automated flagging system can be fooled This statement is absolutely false. No such proof has been presented anywhere, including in the linked article. Adversarial Neuralhash collisions are expected and the system is designed avoid false positives if they are encountered. Here is the relevant paragraph from Apple’s documentation: “as an additional safeguard, the visual derivatives themselves are matched to the known CSAM database by a second, independent perceptual hash. This independent hash is chosen to reject the unlikely possibility that the match threshold was exceeded due to non-CSAM images that were adversarially perturbed to cause false NeuralHash matches against the on-device encrypted CSAM database. If the CSAM finding is confirmed by this independent hash, the visual derivatives are provided to Apple human reviewers for final confirmation.” https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model...
  - wmf 5y ago
    There isn't a single-human point of failure because even if Apple thinks your 30+ hash collisions are real CSAM, they'll send it to be examined by NCMEC who may then forward it to the FBI who may then forward it to a prosecutor... all of those people would have to fail in their jobs for hash collisions to matter.
    - zimpenfish 5y ago
      > who may then forward it to the FBI who may then forward it to a prosecutor... To be fair, though, those two steps have failed -in sequence- enough times for it to be statistically significant on the "OH WTF" scale.
  - lifthrasiir 5y ago
    > It has been proven that the automated flagging system can be fooled This is incorrect. You need the actual NeuralHash for the collision attack to begin with, but the device only has blinded hashes so you have no hash to begin with.
    - colinmhayes 5y ago
      You'll probably be able to buy the hashes from someone who distributed a bunch of CSAM for the express purpose of getting it into the database.
      - lifthrasiir 5y ago
        This is a legitimate concern, but you can't be sure because the system doesn't allow you to determine if the attack was successful, at least in principle (I think there is a timing attack possibility).

wmf 5y ago

I understand how the system works so I know that hash collisions are not dangerous because they have to be manually reviewed by multiple humans. Since you understand how the system works, please explain the danger of hash collisions instead of meta-commenting.

smoldesu 5y ago
I'm not them, but they have a point. This ends up getting re-explained in every single one of these threads, so I'll keep it brief: - There are two main components here: automated flagging and supervised, human confirmation - It has been proven that the automated flagging system can be fooled - The last line of defense is now a human: they have to look at an obfuscated version of your colliding image to determine if it's objectionable The current discussion is mostly around the second portion, moreover what kind of social-engineering attacks would be most effective to overcome a relatively flimsy single-human point of failure.
- zepto 5y ago
  > - It has been proven that the automated flagging system can be fooled This statement is absolutely false. No such proof has been presented anywhere, including in the linked article. Adversarial Neuralhash collisions are expected and the system is designed avoid false positives if they are encountered. Here is the relevant paragraph from Apple’s documentation: “as an additional safeguard, the visual derivatives themselves are matched to the known CSAM database by a second, independent perceptual hash. This independent hash is chosen to reject the unlikely possibility that the match threshold was exceeded due to non-CSAM images that were adversarially perturbed to cause false NeuralHash matches against the on-device encrypted CSAM database. If the CSAM finding is confirmed by this independent hash, the visual derivatives are provided to Apple human reviewers for final confirmation.” https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model...
- wmf 5y ago
  There isn't a single-human point of failure because even if Apple thinks your 30+ hash collisions are real CSAM, they'll send it to be examined by NCMEC who may then forward it to the FBI who may then forward it to a prosecutor... all of those people would have to fail in their jobs for hash collisions to matter.
  - zimpenfish 5y ago
    > who may then forward it to the FBI who may then forward it to a prosecutor... To be fair, though, those two steps have failed -in sequence- enough times for it to be statistically significant on the "OH WTF" scale.
- lifthrasiir 5y ago
  > It has been proven that the automated flagging system can be fooled This is incorrect. You need the actual NeuralHash for the collision attack to begin with, but the device only has blinded hashes so you have no hash to begin with.
  - colinmhayes 5y ago
    You'll probably be able to buy the hashes from someone who distributed a bunch of CSAM for the express purpose of getting it into the database.
    - lifthrasiir 5y ago
      This is a legitimate concern, but you can't be sure because the system doesn't allow you to determine if the attack was successful, at least in principle (I think there is a timing attack possibility).