19 ms·

If the second hash is just another version of neuralhash it's not that unlikely that the same image could match both. It's photodna, people could make images t

by nullc 5y ago

If the second hash is just another version of neuralhash it's not that unlikely that the same image could match both. It's photodna, people could make images that match both. You really can't say anything about "unlikely" when the other algorithm is a secret. Perhaps it's just "return true". Apple failed to disclose[*] that neuralhash could be adversarially matched until the public exploited it, what else are they failing to disclose? [*] I say fail to disclose instead of didn't even know because I believe there is just no way they didn't know this was possible. I generated my first match with almost no experience with the machine learning tools in about an hour, most of which was spent trying to figure out how to use python neural network glue stuff. It should have been expected by design, and other that getting lucky with some parameters basically the dumbest thing possible works (not for every image pair, but enough).

zepto 5y ago
> You really can't say anything about "unlikely" when the other algorithm is a secret. Perhaps it's just "return true". You’re obviously not being honest. You don’t believe it could actually be just ‘return true’. > [*] I say fail to disclose instead of didn't even know because I believe there is just no way they didn't know this was possible. Of course they knew it was possible. That’s why the system doesn’t rely on it not being possible.
- nullc 5y ago
  I believe it is likely that apple will improperly fail to secure the files they can decrypt from insider attack, and I am confident that governments can order them (third party doctrine) to hand over material, bypassing those protections. If you're going to question my honesty, can I ask what financial interest you have in apple or this abuse system, if any? > Of course they knew it was possible. I'm glad we agree. But the fact that their hash is subject to adversarial images, causing totally innocent images to be decryptable by apple, is disclosed nowhere in their documentation or security analysis. I think that is extremely concerning, particularly because it undermines a central plank of their claimed protections: They claim they will protect against state actors including non-child-porn material in their databases by intersecting multiple databases from multiple states. This claimed "protection" is completely obliterated by state actors being able to modify child porn to match the hashes of other images and then submitting this material to each other.
  - zepto 5y ago
    > If you're going to question my honesty… The questioning is a response to your comment. Do you seriously think that Apple’s second hash function could be ‘return true’?
    - nullc 5y ago
      Functionally yes: If apple receives an administrative subpoena to turn over the unfiltered initial matches, then they'll be divulged without further processing. Literally, presumably not, otoh they've outright lied about this system already but I wouldn't think it likely. And when your privacy is compromised your situation will not be improved because the material that mislead you happened to be "technically true".
      - zepto 5y ago
        This is complete and utter bullshit. Firstly it’s a dishonest response since we were talking about how the code worked. But let’s put that aside and consider it anyway. If the code literally returned true, the second hash would have no effect at preventing collisions for any user at any time. As you say, for this to be the case, Apple would need to be lying. A subpoena wouldn’t be issued without cause, would only impact specific individuals, and would have no impact on the general protections provided by the secondary hash at preventing the adversarial attacks proposed. Nothing about this is even close to functionally equivalent to ‘return true’.
kristjansson 5y ago
> not that unlikely that the same image could match both. That seems like a statement that needs to be justified. I suppose one could apply a 50% dropout, construct adversarial pairs on that network, and then run them though the the other half-network to get an idea of the frequency of match on both. My intuition is that the probability of match of an adversarial pair on a second, independent network should be close to the probability of a non-adversarial collision. Robust adversarial perturbations are hard enough when only considering class labels, nevermind hidden state

by nullc 5y ago

zepto 5y ago

> You really can't say anything about "unlikely" when the other algorithm is a secret. Perhaps it's just "return true". You’re obviously not being honest. You don’t believe it could actually be just ‘return true’. > [*] I say fail to disclose instead of didn't even know because I believe there is just no way they didn't know this was possible. Of course they knew it was possible. That’s why the system doesn’t rely on it not being possible.

nullc 5y ago
I believe it is likely that apple will improperly fail to secure the files they can decrypt from insider attack, and I am confident that governments can order them (third party doctrine) to hand over material, bypassing those protections. If you're going to question my honesty, can I ask what financial interest you have in apple or this abuse system, if any? > Of course they knew it was possible. I'm glad we agree. But the fact that their hash is subject to adversarial images, causing totally innocent images to be decryptable by apple, is disclosed nowhere in their documentation or security analysis. I think that is extremely concerning, particularly because it undermines a central plank of their claimed protections: They claim they will protect against state actors including non-child-porn material in their databases by intersecting multiple databases from multiple states. This claimed "protection" is completely obliterated by state actors being able to modify child porn to match the hashes of other images and then submitting this material to each other.
- zepto 5y ago
  > If you're going to question my honesty… The questioning is a response to your comment. Do you seriously think that Apple’s second hash function could be ‘return true’?
  - nullc 5y ago
    Functionally yes: If apple receives an administrative subpoena to turn over the unfiltered initial matches, then they'll be divulged without further processing. Literally, presumably not, otoh they've outright lied about this system already but I wouldn't think it likely. And when your privacy is compromised your situation will not be improved because the material that mislead you happened to be "technically true".
    - zepto 5y ago
      This is complete and utter bullshit. Firstly it’s a dishonest response since we were talking about how the code worked. But let’s put that aside and consider it anyway. If the code literally returned true, the second hash would have no effect at preventing collisions for any user at any time. As you say, for this to be the case, Apple would need to be lying. A subpoena wouldn’t be issued without cause, would only impact specific individuals, and would have no impact on the general protections provided by the secondary hash at preventing the adversarial attacks proposed. Nothing about this is even close to functionally equivalent to ‘return true’.

kristjansson 5y ago

> not that unlikely that the same image could match both. That seems like a statement that needs to be justified. I suppose one could apply a 50% dropout, construct adversarial pairs on that network, and then run them though the the other half-network to get an idea of the frequency of match on both. My intuition is that the probability of match of an adversarial pair on a second, independent network should be close to the probability of a non-adversarial collision. Robust adversarial perturbations are hard enough when only considering class labels, nevermind hidden state