4 ms·

No, you couldn't. This is only checking images you upload to your iCloud photo library. Why would you save tons of gray blobs to your photo library? Why would

No, you couldn't. This is only checking images you upload to your iCloud photo library. Why would you save tons of gray blobs to your photo library? Why would gray blobs look like child porn? Why would Apple reviewers think gray blobs are child porn? Why would the NCMEC think gray blobs are child porn? Why would law enforcement spend time arresting someone for gray blobs?

deleted 5y ago
[deleted]
nullc 5y ago
The grey blob is a proof of concept. The existence of the original image is proof that not all images which produce the target hash are grey blobs. Since the grey blob exists, I believe it is fully possible to construct natural(-ish) images that have a selected hash. So, you should perhaps instead imagine attackers that modify lawful nude images to have matching hashes with child porn images. With that in mind, most of your questions are answered, except perhaps for "Why would the NCMEC think"-- and the answer would be "because its porn and the computer says its child porn". Of course, an attacker could just as well use REAL child porn. But there are logistic advantages in having to do less handling of unlawful material, and it's less likely that the target will notice. Imagine: if the target already has nude images on their host the attacker might use those as the templates. I doubt most people would notice if their nude image collection was replaced with noisier versions of the same stuff. And even if they did notice, "someone is trying to frame me for child porn" wouldn't be their first guess. :)
- robertoandred 5y ago
  And where would they get the CSAM hash they need to match?
  - nullc 5y ago
    The bulk of the CSAM hash database comes widely circulated images, available on sleeze dark web websites (and historically on sketchy porn sites just on the web). So someone need only find a collection of likely included images and compute their hashes and publish it. The attack works just as well even if the attacker isn't sure that every hash is in the database, they just need to know enough likely-matches such that they get enough hits. Consider it this way, some attacker spends a couple hours searching for child porn and finds some stuff. ... what kind of failure would the NCMEC database be if it didn't include that material?
- fouric 5y ago
  > Since the grey blob exists, I believe it is fully possible to construct natural(-ish) images that have a selected hash. At the very least, there are large classes of images that, while not realistic photographs, people will willingly download, that should be very easy to craft collisions for: Memes. In particular, given the distortions of surreal and deep-fried memes, it should be pretty easy to craft collisions there.
- hannasanarion 5y ago
  The other thread demonstrates a similar attack with pictures of dogs. The same questions still apply. Why would you save tons of pictures of dogs to your photo library? Why would pictures of dogs look like child porn? Why would Apple reviewers think pictures of dogs are child porn? Why would the NCMEC think pictures of dogs are child porn? Why would law enforcement spend time arresting someone for pictures of dogs? What is the scenario where someone can be harmed reputationally or criminally because of a hash collision attack, where the same attack could not be performed more easily and causing more damage by actually using real CSAM images?
  - nullc 5y ago
    I'm really surprised to see a fellow HN poster making such a stark failure to generalize. You see that they can do this with pictures of dogs. What makes you think they can't do exactly the same with pictures of crotches? Presumably you don't believe there is some kind inherent dog-nature that makes dog images more likely to undermine the hash. :) People are using pictures of dogs because they are a tasteful safe-for-work example, not because anyone actually imagines using images of dogs. An actual attack would use ordinary nude images, probably ones selected so that if you were primed to expect child porn you'd believe it if you didn't spend a while studying the image. > where the same attack could not be performed more easily and causing more damage by actually using real CSAM images? Actual child porn images are more likely to get deleted by the target and/or reported by the target. The attacker also takes on some additional risk that their possession of the images is a strict liability crime. This means that if the planting party gets found with the real child porn they'll be in trouble vs with the hash-matching-legal-images they'll only be in trouble if they get caught planting it (and potentially only exposed to a civil lawsuit, rather than a felony, depending on how they were planting it). Personally, I agree that the second-preimage-images are not the most interesting attack! But they are a weakness that makes the system even more dangerous. We could debate how much more dangerous they make it.
    - hannasanarion 5y ago
      What kind of non-child-porn imagery is so similar to child porn that it fools the NCMEC investigator who is looking at it side-by-side with its maliciously-collisioned hash match, but is at the same time so so dissimilar from real child porn that, as you say, the target won't report its sudden appearance on their device and the the sender won't be at legal risk? Your scenario requires the image (not the hash, but the regular human-perceptible image) to be both unremarkably innocent and incontrovertibly damning at the same time.
- nullc 5y ago
  Update: The latest results now have second-preimages that are other photos, not just noise. https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issues/1#issuecomment-901360955 https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue... They still look pretty bad, but attacks only get better!

deleted 5y ago

[deleted]

nullc 5y ago

The grey blob is a proof of concept. The existence of the original image is proof that not all images which produce the target hash are grey blobs. Since the grey blob exists, I believe it is fully possible to construct natural(-ish) images that have a selected hash. So, you should perhaps instead imagine attackers that modify lawful nude images to have matching hashes with child porn images. With that in mind, most of your questions are answered, except perhaps for "Why would the NCMEC think"-- and the answer would be "because its porn and the computer says its child porn". Of course, an attacker could just as well use REAL child porn. But there are logistic advantages in having to do less handling of unlawful material, and it's less likely that the target will notice. Imagine: if the target already has nude images on their host the attacker might use those as the templates. I doubt most people would notice if their nude image collection was replaced with noisier versions of the same stuff. And even if they did notice, "someone is trying to frame me for child porn" wouldn't be their first guess. :)

robertoandred 5y ago
And where would they get the CSAM hash they need to match?
- nullc 5y ago
  The bulk of the CSAM hash database comes widely circulated images, available on sleeze dark web websites (and historically on sketchy porn sites just on the web). So someone need only find a collection of likely included images and compute their hashes and publish it. The attack works just as well even if the attacker isn't sure that every hash is in the database, they just need to know enough likely-matches such that they get enough hits. Consider it this way, some attacker spends a couple hours searching for child porn and finds some stuff. ... what kind of failure would the NCMEC database be if it didn't include that material?
fouric 5y ago
> Since the grey blob exists, I believe it is fully possible to construct natural(-ish) images that have a selected hash. At the very least, there are large classes of images that, while not realistic photographs, people will willingly download, that should be very easy to craft collisions for: Memes. In particular, given the distortions of surreal and deep-fried memes, it should be pretty easy to craft collisions there.
hannasanarion 5y ago
The other thread demonstrates a similar attack with pictures of dogs. The same questions still apply. Why would you save tons of pictures of dogs to your photo library? Why would pictures of dogs look like child porn? Why would Apple reviewers think pictures of dogs are child porn? Why would the NCMEC think pictures of dogs are child porn? Why would law enforcement spend time arresting someone for pictures of dogs? What is the scenario where someone can be harmed reputationally or criminally because of a hash collision attack, where the same attack could not be performed more easily and causing more damage by actually using real CSAM images?
- nullc 5y ago
  I'm really surprised to see a fellow HN poster making such a stark failure to generalize. You see that they can do this with pictures of dogs. What makes you think they can't do exactly the same with pictures of crotches? Presumably you don't believe there is some kind inherent dog-nature that makes dog images more likely to undermine the hash. :) People are using pictures of dogs because they are a tasteful safe-for-work example, not because anyone actually imagines using images of dogs. An actual attack would use ordinary nude images, probably ones selected so that if you were primed to expect child porn you'd believe it if you didn't spend a while studying the image. > where the same attack could not be performed more easily and causing more damage by actually using real CSAM images? Actual child porn images are more likely to get deleted by the target and/or reported by the target. The attacker also takes on some additional risk that their possession of the images is a strict liability crime. This means that if the planting party gets found with the real child porn they'll be in trouble vs with the hash-matching-legal-images they'll only be in trouble if they get caught planting it (and potentially only exposed to a civil lawsuit, rather than a felony, depending on how they were planting it). Personally, I agree that the second-preimage-images are not the most interesting attack! But they are a weakness that makes the system even more dangerous. We could debate how much more dangerous they make it.
  - hannasanarion 5y ago
    What kind of non-child-porn imagery is so similar to child porn that it fools the NCMEC investigator who is looking at it side-by-side with its maliciously-collisioned hash match, but is at the same time so so dissimilar from real child porn that, as you say, the target won't report its sudden appearance on their device and the the sender won't be at legal risk? Your scenario requires the image (not the hash, but the regular human-perceptible image) to be both unremarkably innocent and incontrovertibly damning at the same time.
nullc 5y ago
Update: The latest results now have second-preimages that are other photos, not just noise. https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issues/1#issuecomment-901360955 https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue... They still look pretty bad, but attacks only get better!