8 ms·

Notably, none of the the algorithms tested in the cited study are Apple's NeuralHash, or comparable algorithms. They look at aHash, pHash (plus a variant there

Notably, none of the the algorithms tested in the cited study are Apple's NeuralHash, or comparable algorithms. They look at aHash, pHash (plus a variant thereof), dHash, and and PDQ (used at Facebook for similar applications, apparently). The first 4 date to between 2004 and 2010; the last is more recent, but conceptually similar - the citation [0] for PDQ puts it in the same bucket of 'shallower, stricter, cheaper, faster' algorithms as the first four. No one has proposed any of those as 'illegal image detectors'. Apple's NeuralHash may or may not be robust to the same or different perturbations but the cited study provides basically no new information to inform the conversation its press release wants to be a part of. [0]: https://github.com/facebook/ThreatExchange/blob/main/hashing/hashing.pdf https://github.com/facebook/ThreatExchange/blob/main/hashing...

nullc 5y ago
Neuralhash is so easy to trick that not only can I make small changes that change the hash, I can make small (visible but non-annoying) changes to make an image entirely match an essentially arbitrary hash. E.g. here is a Lena I modified to match Barbra: https://user-images.githubusercontent.com/858454/130310372-d6b2c633-c5d3-4b3e-bc12-d3b2f3104ec6.png https://user-images.githubusercontent.com/858454/130310372-d... https://user-images.githubusercontent.com/858454/130310383-9fcc80ba-117e-4383-9177-0df24bc99f9a.png https://user-images.githubusercontent.com/858454/130310383-9... Simply making the hash not match in any single bit is so trivial that it's hardly worth trying. If you give me a 360x360 image of your choice though I'll make one for you if you still have any doubts.
- simondotau 5y ago
  I don't think Apple has ever claimed that NeuralHash was hardened against intentional false positives. And when properly considered as just one component of their CSAM scanning architecture, NeuralHash being hardened against intentional false positives isn't necessary for it to serve its purpose.
  - joe_the_user 5y ago
    Uh, the gp points both intentional false positives and intentional false negatives (IE, anyone intentionally keeping CSAM can easily hide it while anyone can create an image that triggers the hash and gets human to scan your image and etc).
    - simondotau 5y ago
      Intentional false negatives are mostly irrelevant. If you want to store galleries of illegal material on your phone, there are much better ways to do it which don't involve them being visually mangled, even if only slightly. Intentional false positives require knowledge of a hash. And even that isn't enough to get a human to scan your image. You would need to acquire a library of A1 CSAM images/hashes. You would need to generate images which match both NeuralHash and the unspecified secondary hash algorithm which runs on Apple's cloud servers. And then you'd need to repeat this trick a sufficient number of times to trigger human review, at which point a few people (probably some minions in a developing country) get to look at the images for 1 second before pressing the "not CSAM" button and having the images sent to a technical team in Cupertino for analysis.
      - ClumsyPilot 5y ago
        If both false positives and false negatives are irrelevant, then the whole system and argument is irrelevant.
        simondotau 5y ago
        Huh? False negatives and false positives are irrelevant with respect to the utility of the overall system. It means they don't render the system irrelevant. Intentional false negatives are irrelevant because catching 100% of CSAM was never stated as a goal of the system. And because it's just not realistic to expect to see any real-world use by CSAM aficionados. Anyone savvy enough to understand the principles behind hashing (and trust some command line tool downloaded from Github) is probably smart enough to have better places to store their child porn than an iCloud gallery. Intentional false positives are irrelevant because there are, depending how you count them, between two and six other layers which would need to be thwarted or exploited in order for intentional false positives in NeuralHash to result in any adverse outcome.

nullc 5y ago

Neuralhash is so easy to trick that not only can I make small changes that change the hash, I can make small (visible but non-annoying) changes to make an image entirely match an essentially arbitrary hash. E.g. here is a Lena I modified to match Barbra: https://user-images.githubusercontent.com/858454/130310372-d6b2c633-c5d3-4b3e-bc12-d3b2f3104ec6.png https://user-images.githubusercontent.com/858454/130310372-d... https://user-images.githubusercontent.com/858454/130310383-9fcc80ba-117e-4383-9177-0df24bc99f9a.png https://user-images.githubusercontent.com/858454/130310383-9... Simply making the hash not match in any single bit is so trivial that it's hardly worth trying. If you give me a 360x360 image of your choice though I'll make one for you if you still have any doubts.

simondotau 5y ago
I don't think Apple has ever claimed that NeuralHash was hardened against intentional false positives. And when properly considered as just one component of their CSAM scanning architecture, NeuralHash being hardened against intentional false positives isn't necessary for it to serve its purpose.
- joe_the_user 5y ago
  Uh, the gp points both intentional false positives and intentional false negatives (IE, anyone intentionally keeping CSAM can easily hide it while anyone can create an image that triggers the hash and gets human to scan your image and etc).
  - simondotau 5y ago
    Intentional false negatives are mostly irrelevant. If you want to store galleries of illegal material on your phone, there are much better ways to do it which don't involve them being visually mangled, even if only slightly. Intentional false positives require knowledge of a hash. And even that isn't enough to get a human to scan your image. You would need to acquire a library of A1 CSAM images/hashes. You would need to generate images which match both NeuralHash and the unspecified secondary hash algorithm which runs on Apple's cloud servers. And then you'd need to repeat this trick a sufficient number of times to trigger human review, at which point a few people (probably some minions in a developing country) get to look at the images for 1 second before pressing the "not CSAM" button and having the images sent to a technical team in Cupertino for analysis.
    - ClumsyPilot 5y ago
      If both false positives and false negatives are irrelevant, then the whole system and argument is irrelevant.
      - simondotau 5y ago
        Huh? False negatives and false positives are irrelevant with respect to the utility of the overall system. It means they don't render the system irrelevant. Intentional false negatives are irrelevant because catching 100% of CSAM was never stated as a goal of the system. And because it's just not realistic to expect to see any real-world use by CSAM aficionados. Anyone savvy enough to understand the principles behind hashing (and trust some command line tool downloaded from Github) is probably smart enough to have better places to store their child porn than an iCloud gallery. Intentional false positives are irrelevant because there are, depending how you count them, between two and six other layers which would need to be thwarted or exploited in order for intentional false positives in NeuralHash to result in any adverse outcome.