5 ms·

And 5. The “bad” hashes are secret, so it’s not possible to create a false positive that would trigger Apple’s human review.

charcircuit 5y ago
They aren't all secret. Finding a site with pictures in that database is a single google search away.
- maratc 5y ago
  The pictures aren't secret. The hashes are. We don't know what hash algorithm is employed on Apple's server side, so we can't feed that algorithm with these non-secret images and get the hashes.
  - charcircuit 5y ago
    The hash algorithm shouldn't matter if you have something you know it will detect
    - simondotau 5y ago
      Sure, but then you have CSAM and Apple’s system is working as intended.
      - skinkestek 5y ago
        no, that is the entire point: it is trivially simple to make an innocent picture match in neuralhash.
        nullc 5y ago
        You're talking across each other. Apple claims without evidence that there is a second secret hash which is run serverside. Perhaps it's just Microsoft PhotoDNA (which isn't secret anymore), perhaps it's just another copy of neuralhash (which might share the same collisions)-- it isn't currently known to the public. It is, however, known to anyone inside the project at apple and anyone involved in the abuse image databases at a minimum. I'm of the opinion that any security properties of this supposed second hash should be disregarded in public analysis of the system-- for all we know it's just a "return true". Other people make a reasonable argument that this additional function makes some of the attacks people describe harder. In practice it probably does, but security analysis should concern itself with the worst cases admitted by the known system.
        simondotau 5y ago
        By that logic, all cryptography is always broken because anyone could steal anyone's private key at any time. This isn't a reasonable stance, it's just a tactic of poisoning the well such that you can never lose the argument.
        nullc 5y ago
        > This isn't a reasonable stance, it's just a tactic of poisoning the well such that you can never lose the argument. I refrained from outright accusing you of bad faith when you claimed that the example image was obviously manipulated, though I confess that was my instant impression. I'm sad that you haven't extended me the same courtesy. It's considered a bad practice and insecure for systems to keep a cryptographic _algorithm_ secret. In a cryptosystem all that should be secret is the key itself which is small and uniformly randomly generated and by construction cannot contain any weaknesses that could be uncovered by inspection (as it's just uniformly selected out of the space of possible keys). This doesn't at all apply to a perceptual hashing system: the system will certainly contain weaknesses which could be found by chance or by limited black box interrogation, but it could only be cheaply reviewed for flaws with direct access to it. The fact that secure cryptosystems by construction protect the keys from leakage through usage is part of the reason that it's okay to stuff all the security dependency into the key. If you used HMAC with an unknown key and allowed me oracle access to see if inputs I sent in matched, I couldn't learn anything about your hmac key (unless the underlying hash was broken). But if we didn't know the weights of neuralhash and I got match oracle access to it and access to a few matching images (maybe 1 per bit of hash), I could likely the entire set of weights just through interrogating an oracle that gives a pass fail. (Also the papers that successfully design adversarial inputs for unknown networks, simply by designing adversarial inputs for a set of entirely different networks) Moreover, this second perceptual hashing system requires that Apple share the system with third parties-- they must share it with e.g. NCMEC and the agencies of other governments because it is a felony for apple to knowingly handle the unlawful images themselves so they cannot produce the database themselves, it has to be produced by NCMEC that has the special statutory carveout. By contrast, in a standard public key encryption or authentication scheme your private key is shared with no one ever. That makes it much easier to keep secure (though keys often do get leaked and stolen). Finally, if I steal your pgp key or whatever, it effects a single use or a single user. Most cryptosystems use as many keys as reasonably possible so the loss of one has the limited impact-- and they can be replaced with another uniformly generated random key if there is a concern for a key's security. But there is only one second image hash, they likely cost millions to develop, so the leakage of it would be much more damaging to the system compared to leaking a key. I'm perfectly willing say that we have a reasonable disagreement on value of the 'secret' additional hashing scheme. The role of obscurity in security systems has been debated for decades in forums far beyond here-- and we're not likely to add more to it in this discussion. But please don't accuse me of poisoning the well because we have opposing perspectives on this broad subject. I think I've adequately argued why your comparison point isn't a great comparison. We might not agree but we should agree that neither of us is taking a disingenuous position.

charcircuit 5y ago

They aren't all secret. Finding a site with pictures in that database is a single google search away.

maratc 5y ago
The pictures aren't secret. The hashes are. We don't know what hash algorithm is employed on Apple's server side, so we can't feed that algorithm with these non-secret images and get the hashes.
- charcircuit 5y ago
  The hash algorithm shouldn't matter if you have something you know it will detect
  - simondotau 5y ago
    Sure, but then you have CSAM and Apple’s system is working as intended.
    - skinkestek 5y ago
      no, that is the entire point: it is trivially simple to make an innocent picture match in neuralhash.
      - nullc 5y ago
        You're talking across each other. Apple claims without evidence that there is a second secret hash which is run serverside. Perhaps it's just Microsoft PhotoDNA (which isn't secret anymore), perhaps it's just another copy of neuralhash (which might share the same collisions)-- it isn't currently known to the public. It is, however, known to anyone inside the project at apple and anyone involved in the abuse image databases at a minimum. I'm of the opinion that any security properties of this supposed second hash should be disregarded in public analysis of the system-- for all we know it's just a "return true". Other people make a reasonable argument that this additional function makes some of the attacks people describe harder. In practice it probably does, but security analysis should concern itself with the worst cases admitted by the known system.
        simondotau 5y ago
        By that logic, all cryptography is always broken because anyone could steal anyone's private key at any time. This isn't a reasonable stance, it's just a tactic of poisoning the well such that you can never lose the argument.
        nullc 5y ago
        > This isn't a reasonable stance, it's just a tactic of poisoning the well such that you can never lose the argument. I refrained from outright accusing you of bad faith when you claimed that the example image was obviously manipulated, though I confess that was my instant impression. I'm sad that you haven't extended me the same courtesy. It's considered a bad practice and insecure for systems to keep a cryptographic _algorithm_ secret. In a cryptosystem all that should be secret is the key itself which is small and uniformly randomly generated and by construction cannot contain any weaknesses that could be uncovered by inspection (as it's just uniformly selected out of the space of possible keys). This doesn't at all apply to a perceptual hashing system: the system will certainly contain weaknesses which could be found by chance or by limited black box interrogation, but it could only be cheaply reviewed for flaws with direct access to it. The fact that secure cryptosystems by construction protect the keys from leakage through usage is part of the reason that it's okay to stuff all the security dependency into the key. If you used HMAC with an unknown key and allowed me oracle access to see if inputs I sent in matched, I couldn't learn anything about your hmac key (unless the underlying hash was broken). But if we didn't know the weights of neuralhash and I got match oracle access to it and access to a few matching images (maybe 1 per bit of hash), I could likely the entire set of weights just through interrogating an oracle that gives a pass fail. (Also the papers that successfully design adversarial inputs for unknown networks, simply by designing adversarial inputs for a set of entirely different networks) Moreover, this second perceptual hashing system requires that Apple share the system with third parties-- they must share it with e.g. NCMEC and the agencies of other governments because it is a felony for apple to knowingly handle the unlawful images themselves so they cannot produce the database themselves, it has to be produced by NCMEC that has the special statutory carveout. By contrast, in a standard public key encryption or authentication scheme your private key is shared with no one ever. That makes it much easier to keep secure (though keys often do get leaked and stolen). Finally, if I steal your pgp key or whatever, it effects a single use or a single user. Most cryptosystems use as many keys as reasonably possible so the loss of one has the limited impact-- and they can be replaced with another uniformly generated random key if there is a concern for a key's security. But there is only one second image hash, they likely cost millions to develop, so the leakage of it would be much more damaging to the system compared to leaking a key. I'm perfectly willing say that we have a reasonable disagreement on value of the 'secret' additional hashing scheme. The role of obscurity in security systems has been debated for decades in forums far beyond here-- and we're not likely to add more to it in this discussion. But please don't accuse me of poisoning the well because we have opposing perspectives on this broad subject. I think I've adequately argued why your comparison point isn't a great comparison. We might not agree but we should agree that neither of us is taking a disingenuous position.