4 ms·

Algorithm may decide picture of building is a child porn. That is problem.

by throw63738 5y ago

imwillofficial 5y ago
The idea you’d accidentally have 30+ hash collisions that also would not pass manual review is a joke.
- throw63738 5y ago
  Why would I even expose myself to such thing?
  - imwillofficial 5y ago
    Then don't
- meowster 5y ago
  The idea that people never accidently click the wrong button is a joke, especially content moderators who are most likely getting PTSD from the job. Do you invite the police into your home to watch what you do to make sure you aren't breaking any laws?
binbag 5y ago
What on earth makes you think this is remotely probable?
- homesickgoose 5y ago
  No matter how smart any "NeuralHash" is, the fact that it IS hash-based means there exists more than one image that maps to the same hash. (Using a opaque NeuralNet-style AI that not even its developers can reason about combined with unknown groud-truths does not help). PoC: https://github.com/anishathalye/neural-hash-collider https://github.com/anishathalye/neural-hash-collider
  - zepto 5y ago
    That isn’t a proof of concept. The system is designed to handle even intentional hash collisions. Here is the relevant paragraph from Apple’s documentation: “as an additional safeguard, the visual derivatives themselves are matched to the known CSAM database by a second, independent perceptual hash. This independent hash is chosen to reject the unlikely possi- bility that the match threshold was exceeded due to non-CSAM images that were ad- versarially perturbed to cause false NeuralHash matches against the on-device en- crypted CSAM database. If the CSAM finding is confirmed by this independent hash, the visual derivatives are provided to Apple human reviewers for final confirmation.” https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model...
    - homesickgoose 5y ago
      Requiring a second key to unlock a lock does not invalidate the fact that the first key can be picked (which the question was about). I had read through the technical whitepaper [1], which does not include this information. Thank you for sharing. Since the second hash only works on pictures that Apple can decrypt within this system ("for an account that exceeded the match threshold"), this merely saves the human reviewers at Apple time. [1] https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
      - zepto 5y ago
        > Requiring a second key to unlock a lock does not invalidate the fact that the first key can be picked (which the question was about). Apple’s CSAM mechanism can’t be ‘picked’ in the way that you claim. The two ‘locks’ are not separate. You can’t pick them one at a time. > I had read through the technical whitepaper [1], which does not include this information. Thank you for sharing. Since the second hash only works on pictures that Apple can decrypt within this system ("for an account that exceeded the match threshold"), this merely saves the human reviewers at Apple time. This is false. The second hash is independent and designed to prevent attacks based on adversarial spoofing of neuralhash. Nobody, not even Apple, denies that someone can generate neuralhash collisions. You can only assert that this is a vulnerability by making the false claim that a neuralhash collision alone will cause the system to flag an image.

Algorithm may decide picture of building is a child porn. That is problem.

imwillofficial 5y ago

The idea you’d accidentally have 30+ hash collisions that also would not pass manual review is a joke.

throw63738 5y ago
Why would I even expose myself to such thing?
- imwillofficial 5y ago
  Then don't
meowster 5y ago
The idea that people never accidently click the wrong button is a joke, especially content moderators who are most likely getting PTSD from the job. Do you invite the police into your home to watch what you do to make sure you aren't breaking any laws?

binbag 5y ago

What on earth makes you think this is remotely probable?

homesickgoose 5y ago
No matter how smart any "NeuralHash" is, the fact that it IS hash-based means there exists more than one image that maps to the same hash. (Using a opaque NeuralNet-style AI that not even its developers can reason about combined with unknown groud-truths does not help). PoC: https://github.com/anishathalye/neural-hash-collider https://github.com/anishathalye/neural-hash-collider
- zepto 5y ago
  That isn’t a proof of concept. The system is designed to handle even intentional hash collisions. Here is the relevant paragraph from Apple’s documentation: “as an additional safeguard, the visual derivatives themselves are matched to the known CSAM database by a second, independent perceptual hash. This independent hash is chosen to reject the unlikely possi- bility that the match threshold was exceeded due to non-CSAM images that were ad- versarially perturbed to cause false NeuralHash matches against the on-device en- crypted CSAM database. If the CSAM finding is confirmed by this independent hash, the visual derivatives are provided to Apple human reviewers for final confirmation.” https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model...
  - homesickgoose 5y ago
    Requiring a second key to unlock a lock does not invalidate the fact that the first key can be picked (which the question was about). I had read through the technical whitepaper [1], which does not include this information. Thank you for sharing. Since the second hash only works on pictures that Apple can decrypt within this system ("for an account that exceeded the match threshold"), this merely saves the human reviewers at Apple time. [1] https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
    - zepto 5y ago
      > Requiring a second key to unlock a lock does not invalidate the fact that the first key can be picked (which the question was about). Apple’s CSAM mechanism can’t be ‘picked’ in the way that you claim. The two ‘locks’ are not separate. You can’t pick them one at a time. > I had read through the technical whitepaper [1], which does not include this information. Thank you for sharing. Since the second hash only works on pictures that Apple can decrypt within this system ("for an account that exceeded the match threshold"), this merely saves the human reviewers at Apple time. This is false. The second hash is independent and designed to prevent attacks based on adversarial spoofing of neuralhash. Nobody, not even Apple, denies that someone can generate neuralhash collisions. You can only assert that this is a vulnerability by making the false claim that a neuralhash collision alone will cause the system to flag an image.