27 ms·

The site shows you could easily receive an innocent image that would collide with child porn which could raise some questions.

zepto 5y ago
No it doesn’t. There is a second hash that has to also be matched even before human review, and this doesn’t demonstrate an image even getting that far. Here is the relevant paragraph from Apple’s documentation: “as an additional safeguard, the visual derivatives themselves are matched to the known CSAM database by a second, independent perceptual hash. This independent hash is chosen to reject the unlikely possibility that the match threshold was exceeded due to non-CSAM images that were adversarially perturbed to cause false NeuralHash matches against the on-device encrypted CSAM database. If the CSAM finding is confirmed by this independent hash, the visual derivatives are provided to Apple human reviewers for final confirmation.” https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model...
- shrimpx 5y ago
  > reject the unlikely possibility "Unlikely possibility" shows either blatant facetiousness or blatant ignorance. Who couldn't predict the explosion of collision-generating projects?
  - nicce 5y ago
    You either have to break their HSM (hardware security module), or their cryptography to access for the known hashes. Or get them from elsewhere. Owning the actual images is illegal. It is hard to see the explosion of these projects. You need 30 matching hashes to their database to trigger anything.
    - belltaco 5y ago
      >Owning the actual images is illegal Yeah it's not like there are a lot of people on the darknet that do illegal things all the time. /s You only need one of them to compute the neural hashes of the images and then upload them. For others, it is not illegal to have or distribute the neural hashes.
      - nicce 5y ago
        > Yeah it's not like there are a lot of people on the darknet that do illegal things all the time. /s How many of them are developers with enough knowledge compared to all developers in world with enough knowledge? And willing to touch CSAM material? I think there is a quite difference. And they have better things to do than pranking some people. Because you are not getting jailed from hash collisions in the country with working judicial system. You have bigger problems if judicial system is not working.
    - zargon 5y ago
      Criminals won't be able to create hash collisions because the images are illegal to possess! Illegal, I say. That is surest way to stop criminals.
      - nicce 5y ago
        Something being very illegal certainly limits it from the masses. I am not saying that it prevents the criminals, but it significantly reduces public projects. And on top of that, even the most of the criminals hate pedophiles, so how many is really willing to access CSAM material?
        zargon 5y ago
        It only takes one to do it and then distribute the hashes.
        zepto 5y ago
        As pointed out elsewhere. It doesn’t matter. NeuralHash collisions don’t break the system.
      - nullc 5y ago
        You don't need to have illegal images to create collisions with them. You just need someone with illegal images to publish hashes.
    - shrimpx 5y ago
      Thanks. I was under the impression that the hashes were easier to get.
      - kristjansson 5y ago
        It's not just that they're hard to get - it's also that no one has demonstrated a preimage attack. So given just an illegal hash one can't construct an innocent image with that hash. These demonstrations show only that a second image can be transformed to have the same hash as a first, given image. As other comments note, that's not a huge increase in difficulty for an adversary willing to deal in actual CSAM.
        nicce 5y ago
        > no one has demonstrated a preimage attack. So given just an illegal hash one can't construct an innocent image with that hash. These demonstrations show only that a second image can be transformed to have the same hash as a first, given image. Isn’t this exactly like that? Or am I missunderstanding something. https://twitter.com/ghidraninja/status/1428269674912002048?s=20 https://twitter.com/ghidraninja/status/1428269674912002048?s...
        kristjansson 5y ago
        No. The author describes their process for generating the image down that very thread[0] Quoting the tweet for those who dislike twitter: > The procedure for this was pretty simple: > - Started with a placeholder image > - Got the hash of it > - Changed text of the image to match the hash > - Then used > @anishathalye great neural-hash-collider which took only 1-2 iterations N.B. it starts with 'started with an image, and the hash of it' [0]: https://twitter.com/ghidraninja/status/1428270675010199554 https://twitter.com/ghidraninja/status/1428270675010199554
        nicce 5y ago
        What is the practical difference? It is forged innocent image which has matching hash. You basically started with only a hash, as image was not related to it.
- sharken 5y ago
  I'll assume you are explaining the concept, but at this point the sum of all the elements sums up to a feature that should be cancelled. The arguments have been stated before, the principal one being that scanning the users device goes against the wish of users and what Apple until now has stood for. I think Apple fails to see the seriousness of this.
  - nicce 5y ago
    The sad truth is, that we won't see more E2EE features by Apple without this. It is hard decision for hard problem. And the problem gets bigger when you are the most valuable company in the world.
    - sharken 5y ago
      Yes that could very well be the dilemma Apple is in. But i don't consider it E2EE if you are scanning content on the device first.
  - colinmhayes 5y ago
    > and what Apple until now has stood for. Apple has a billboard right outside my office. Up until last week it said "privacy" in big letters. Now there's no mention of privacy at all.
  - zepto 5y ago
    > the sum of all the elements The comment chain I was responding to is about the claim that neuralhash is broken so the system is dangerous. This claim has been repeated multiple times in this forum. Indeed it is one of the loudest two arguments against the system and is simply false. The sum of elements which are false is still false now matter how many you have. It’s certainly not universally true that scanning the device goes against the wish of the users. There are many users who are quite happy to have their devices scanned if it is part of making life harder for child abusers. I have spoken to such people. Therein lies the problem. This system really doesn’t do anything harmful, and it really does just make the tool less useful for collecting child pornography. There is no simple argument against it, hence all of the misrepresentations about its purported technical problems. I strongly dislike the feature, and would rather they don’t do it. My arguments are that I don’t want to be, even in principle, suspected of something I haven’t done. And that trust should be mutual. If Apple wants me to trust them, I want them to trust me. I don’t see how we help make the world better by using false narratives about technical issues to get what we want.
- onion2k 5y ago
  Whenever you have a system that relies on probability (eg the extremely low probability that an image might match two independent perceptual hashes) you have to take two things in to account; 1. There is a possibility, however tiny, that an innocent image will still match both hashes. The probability is not zero. It can't be. They're hashes. 2. By telling people that there is no chance of an innocent image matching both hashes you are forcing the burden of proof on to the victim. If someone is unfortunate enough to have an image that matches both hashes they will be dragged into a law enforcement office and told to explain "why the foolproof, perfect, impossible-to-cheat system said there was child porn on their phone". It will be on them to explain why the system isn't correct. The presumption of innocence is lost when too much faith is placed on technology. That is why this is dangerous. Arguably it's a well-designed system that safeguards children and catches despicable criminals, but unless people understand it isn't infallible, and stop arguing that it is, then it could cost an innocent person their freedom. That's a high price to pay.
  - kristjansson 5y ago
    There's also a positive probability that ssh-keygen spits out your ssh key next time I run it.
  - colinmhayes 5y ago
    It's not unlikely that are a grand total of 0 images in the database that have collisions with both hashing algorithms. Of course it's possible, but so is randomly typing in someones pgp key.
- nicce 5y ago
  > by a second, independent perceptual hash Does this mean, that they have another perceptual hashing algorithm, and this is happening purely on the server side? Because that could be quite genius. It would be really hard to adversarially fool two different algorithms.
  - zepto 5y ago
    It does mean exactly that.

zepto 5y ago

No it doesn’t. There is a second hash that has to also be matched even before human review, and this doesn’t demonstrate an image even getting that far. Here is the relevant paragraph from Apple’s documentation: “as an additional safeguard, the visual derivatives themselves are matched to the known CSAM database by a second, independent perceptual hash. This independent hash is chosen to reject the unlikely possibility that the match threshold was exceeded due to non-CSAM images that were adversarially perturbed to cause false NeuralHash matches against the on-device encrypted CSAM database. If the CSAM finding is confirmed by this independent hash, the visual derivatives are provided to Apple human reviewers for final confirmation.” https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model...

shrimpx 5y ago
> reject the unlikely possibility "Unlikely possibility" shows either blatant facetiousness or blatant ignorance. Who couldn't predict the explosion of collision-generating projects?
- nicce 5y ago
  You either have to break their HSM (hardware security module), or their cryptography to access for the known hashes. Or get them from elsewhere. Owning the actual images is illegal. It is hard to see the explosion of these projects. You need 30 matching hashes to their database to trigger anything.
  - belltaco 5y ago
    >Owning the actual images is illegal Yeah it's not like there are a lot of people on the darknet that do illegal things all the time. /s You only need one of them to compute the neural hashes of the images and then upload them. For others, it is not illegal to have or distribute the neural hashes.
    - nicce 5y ago
      > Yeah it's not like there are a lot of people on the darknet that do illegal things all the time. /s How many of them are developers with enough knowledge compared to all developers in world with enough knowledge? And willing to touch CSAM material? I think there is a quite difference. And they have better things to do than pranking some people. Because you are not getting jailed from hash collisions in the country with working judicial system. You have bigger problems if judicial system is not working.
  - zargon 5y ago
    Criminals won't be able to create hash collisions because the images are illegal to possess! Illegal, I say. That is surest way to stop criminals.
    - nicce 5y ago
      Something being very illegal certainly limits it from the masses. I am not saying that it prevents the criminals, but it significantly reduces public projects. And on top of that, even the most of the criminals hate pedophiles, so how many is really willing to access CSAM material?
      - zargon 5y ago
        It only takes one to do it and then distribute the hashes.
        zepto 5y ago
        As pointed out elsewhere. It doesn’t matter. NeuralHash collisions don’t break the system.
    - nullc 5y ago
      You don't need to have illegal images to create collisions with them. You just need someone with illegal images to publish hashes.
  - shrimpx 5y ago
    Thanks. I was under the impression that the hashes were easier to get.
    - kristjansson 5y ago
      It's not just that they're hard to get - it's also that no one has demonstrated a preimage attack. So given just an illegal hash one can't construct an innocent image with that hash. These demonstrations show only that a second image can be transformed to have the same hash as a first, given image. As other comments note, that's not a huge increase in difficulty for an adversary willing to deal in actual CSAM.
      - nicce 5y ago
        > no one has demonstrated a preimage attack. So given just an illegal hash one can't construct an innocent image with that hash. These demonstrations show only that a second image can be transformed to have the same hash as a first, given image. Isn’t this exactly like that? Or am I missunderstanding something. https://twitter.com/ghidraninja/status/1428269674912002048?s=20 https://twitter.com/ghidraninja/status/1428269674912002048?s...
        kristjansson 5y ago
        No. The author describes their process for generating the image down that very thread[0] Quoting the tweet for those who dislike twitter: > The procedure for this was pretty simple: > - Started with a placeholder image > - Got the hash of it > - Changed text of the image to match the hash > - Then used > @anishathalye great neural-hash-collider which took only 1-2 iterations N.B. it starts with 'started with an image, and the hash of it' [0]: https://twitter.com/ghidraninja/status/1428270675010199554 https://twitter.com/ghidraninja/status/1428270675010199554
        nicce 5y ago
        What is the practical difference? It is forged innocent image which has matching hash. You basically started with only a hash, as image was not related to it.
sharken 5y ago
I'll assume you are explaining the concept, but at this point the sum of all the elements sums up to a feature that should be cancelled. The arguments have been stated before, the principal one being that scanning the users device goes against the wish of users and what Apple until now has stood for. I think Apple fails to see the seriousness of this.
- nicce 5y ago
  The sad truth is, that we won't see more E2EE features by Apple without this. It is hard decision for hard problem. And the problem gets bigger when you are the most valuable company in the world.
  - sharken 5y ago
    Yes that could very well be the dilemma Apple is in. But i don't consider it E2EE if you are scanning content on the device first.
- colinmhayes 5y ago
  > and what Apple until now has stood for. Apple has a billboard right outside my office. Up until last week it said "privacy" in big letters. Now there's no mention of privacy at all.
- zepto 5y ago
  > the sum of all the elements The comment chain I was responding to is about the claim that neuralhash is broken so the system is dangerous. This claim has been repeated multiple times in this forum. Indeed it is one of the loudest two arguments against the system and is simply false. The sum of elements which are false is still false now matter how many you have. It’s certainly not universally true that scanning the device goes against the wish of the users. There are many users who are quite happy to have their devices scanned if it is part of making life harder for child abusers. I have spoken to such people. Therein lies the problem. This system really doesn’t do anything harmful, and it really does just make the tool less useful for collecting child pornography. There is no simple argument against it, hence all of the misrepresentations about its purported technical problems. I strongly dislike the feature, and would rather they don’t do it. My arguments are that I don’t want to be, even in principle, suspected of something I haven’t done. And that trust should be mutual. If Apple wants me to trust them, I want them to trust me. I don’t see how we help make the world better by using false narratives about technical issues to get what we want.
onion2k 5y ago
Whenever you have a system that relies on probability (eg the extremely low probability that an image might match two independent perceptual hashes) you have to take two things in to account; 1. There is a possibility, however tiny, that an innocent image will still match both hashes. The probability is not zero. It can't be. They're hashes. 2. By telling people that there is no chance of an innocent image matching both hashes you are forcing the burden of proof on to the victim. If someone is unfortunate enough to have an image that matches both hashes they will be dragged into a law enforcement office and told to explain "why the foolproof, perfect, impossible-to-cheat system said there was child porn on their phone". It will be on them to explain why the system isn't correct. The presumption of innocence is lost when too much faith is placed on technology. That is why this is dangerous. Arguably it's a well-designed system that safeguards children and catches despicable criminals, but unless people understand it isn't infallible, and stop arguing that it is, then it could cost an innocent person their freedom. That's a high price to pay.
- kristjansson 5y ago
  There's also a positive probability that ssh-keygen spits out your ssh key next time I run it.
- colinmhayes 5y ago
  It's not unlikely that are a grand total of 0 images in the database that have collisions with both hashing algorithms. Of course it's possible, but so is randomly typing in someones pgp key.
nicce 5y ago
> by a second, independent perceptual hash Does this mean, that they have another perceptual hashing algorithm, and this is happening purely on the server side? Because that could be quite genius. It would be really hard to adversarially fool two different algorithms.
- zepto 5y ago
  It does mean exactly that.