3 ms·

What on earth makes you think this is remotely probable?

by binbag 5y ago

homesickgoose 5y ago
No matter how smart any "NeuralHash" is, the fact that it IS hash-based means there exists more than one image that maps to the same hash. (Using a opaque NeuralNet-style AI that not even its developers can reason about combined with unknown groud-truths does not help). PoC: https://github.com/anishathalye/neural-hash-collider https://github.com/anishathalye/neural-hash-collider
- zepto 5y ago
  That isn’t a proof of concept. The system is designed to handle even intentional hash collisions. Here is the relevant paragraph from Apple’s documentation: “as an additional safeguard, the visual derivatives themselves are matched to the known CSAM database by a second, independent perceptual hash. This independent hash is chosen to reject the unlikely possi- bility that the match threshold was exceeded due to non-CSAM images that were ad- versarially perturbed to cause false NeuralHash matches against the on-device en- crypted CSAM database. If the CSAM finding is confirmed by this independent hash, the visual derivatives are provided to Apple human reviewers for final confirmation.” https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model...
  - homesickgoose 5y ago
    Requiring a second key to unlock a lock does not invalidate the fact that the first key can be picked (which the question was about). I had read through the technical whitepaper [1], which does not include this information. Thank you for sharing. Since the second hash only works on pictures that Apple can decrypt within this system ("for an account that exceeded the match threshold"), this merely saves the human reviewers at Apple time. [1] https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
    - zepto 5y ago
      > Requiring a second key to unlock a lock does not invalidate the fact that the first key can be picked (which the question was about). Apple’s CSAM mechanism can’t be ‘picked’ in the way that you claim. The two ‘locks’ are not separate. You can’t pick them one at a time. > I had read through the technical whitepaper [1], which does not include this information. Thank you for sharing. Since the second hash only works on pictures that Apple can decrypt within this system ("for an account that exceeded the match threshold"), this merely saves the human reviewers at Apple time. This is false. The second hash is independent and designed to prevent attacks based on adversarial spoofing of neuralhash. Nobody, not even Apple, denies that someone can generate neuralhash collisions. You can only assert that this is a vulnerability by making the false claim that a neuralhash collision alone will cause the system to flag an image.
      - homesickgoose 5y ago
        > The two ‘locks’ are not separate. > The second hash is independent lol
        zepto 5y ago
        It seems like there is a misunderstanding on your part. The locks are not separate. You have to open both with the same ‘key’, I.e. the image. From the documentation: “the visual derivatives themselves are matched to the known CSAM database by a second, independent perceptual hash” The “lock” in this case involves one key, and two hashes. If they were separate, you’d be able to ‘pick’ them one at a time with separate keys. You can’t.
    - skinkestek 5y ago
      Thanks for posting. I've come out guns blazing on this issue but I think - as in a few other cases - it is time for me concede a bit: If such a system should be built they surely have gone out of their way to preemptively secure it. I'm still very much of the opinion that we need people both in police, politics and in tech with balls - or whatever the equivalent is for women - to tell people that we cannot just interfere with everyones right just to hopefully maybe catch a few more baddies, but again: If such a system needs to be built, this is an amazingly well thought out system.

What on earth makes you think this is remotely probable?

by binbag 5y ago

What on earth makes you think this is remotely probable?

No matter how smart any "NeuralHash" is, the fact that it IS hash-based means there exists more than one image that maps to the same hash. (Using a opaque NeuralNet-style AI that not even its developers can reason about combined with unknown groud-truths does not help). PoC: https://github.com/anishathalye/neural-hash-collider https://github.com/anishathalye/neural-hash-collider

zepto 5y ago
That isn’t a proof of concept. The system is designed to handle even intentional hash collisions. Here is the relevant paragraph from Apple’s documentation: “as an additional safeguard, the visual derivatives themselves are matched to the known CSAM database by a second, independent perceptual hash. This independent hash is chosen to reject the unlikely possi- bility that the match threshold was exceeded due to non-CSAM images that were ad- versarially perturbed to cause false NeuralHash matches against the on-device en- crypted CSAM database. If the CSAM finding is confirmed by this independent hash, the visual derivatives are provided to Apple human reviewers for final confirmation.” https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model...
- homesickgoose 5y ago
  Requiring a second key to unlock a lock does not invalidate the fact that the first key can be picked (which the question was about). I had read through the technical whitepaper [1], which does not include this information. Thank you for sharing. Since the second hash only works on pictures that Apple can decrypt within this system ("for an account that exceeded the match threshold"), this merely saves the human reviewers at Apple time. [1] https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
  - zepto 5y ago
    > Requiring a second key to unlock a lock does not invalidate the fact that the first key can be picked (which the question was about). Apple’s CSAM mechanism can’t be ‘picked’ in the way that you claim. The two ‘locks’ are not separate. You can’t pick them one at a time. > I had read through the technical whitepaper [1], which does not include this information. Thank you for sharing. Since the second hash only works on pictures that Apple can decrypt within this system ("for an account that exceeded the match threshold"), this merely saves the human reviewers at Apple time. This is false. The second hash is independent and designed to prevent attacks based on adversarial spoofing of neuralhash. Nobody, not even Apple, denies that someone can generate neuralhash collisions. You can only assert that this is a vulnerability by making the false claim that a neuralhash collision alone will cause the system to flag an image.
    - homesickgoose 5y ago
      > The two ‘locks’ are not separate. > The second hash is independent lol
      - zepto 5y ago
        It seems like there is a misunderstanding on your part. The locks are not separate. You have to open both with the same ‘key’, I.e. the image. From the documentation: “the visual derivatives themselves are matched to the known CSAM database by a second, independent perceptual hash” The “lock” in this case involves one key, and two hashes. If they were separate, you’d be able to ‘pick’ them one at a time with separate keys. You can’t.
  - skinkestek 5y ago
    Thanks for posting. I've come out guns blazing on this issue but I think - as in a few other cases - it is time for me concede a bit: If such a system should be built they surely have gone out of their way to preemptively secure it. I'm still very much of the opinion that we need people both in police, politics and in tech with balls - or whatever the equivalent is for women - to tell people that we cannot just interfere with everyones right just to hopefully maybe catch a few more baddies, but again: If such a system needs to be built, this is an amazingly well thought out system.