5 ms·

Some people seem to be confused why a hash collision of a cat and a dog matters. Here's a potential attack: share (legal) NSFW pictures that are engineered to h

Some people seem to be confused why a hash collision of a cat and a dog matters. Here's a potential attack: share (legal) NSFW pictures that are engineered to have a hash collision with CSAM to get someone else in trouble. The pictures are flagged as CSAM, and they also look suspicious to a human reviewer (maybe not enough context in the image to identify the subject's age). To show that this can be done with real NSFW pictures, here is an example, using an NSFW image from a subreddit's top posts of all time. Here is the image (NSFW!): https://i.ibb.co/Ct64Cnt/nsfw.png https://i.ibb.co/Ct64Cnt/nsfw.png Hash: 59a34eabe31910abfb06f308

robertoandred 5y ago
Does anyone save porn to their personal photo libraries? Especially porn as suspicious as the image you posted?
- throwaway384950 5y ago
  Going by what some people on Reddit say, it seems to be the case. https://old.reddit.com/r/datahoarder/search?q=porn&restrict_sr=on https://old.reddit.com/r/datahoarder/search?q=porn&restrict_... Probably not the weird image I posted, which looks obviously suspicious. But maybe someone will make a program to find "cleaner" hash collisions that don't look suspicious.
  - nullc 5y ago
    https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issues/1#issuecomment-902977931 https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue... I posted some examples that look like totally normal images, they're no harder to produce, you just need to noise-shape the gradient descent so that the introduced noise has a spectrum similar to the image. E.g. just feeding back a gaussian highpassed version of the error signal is sufficient.
- pseudalopex 5y ago
  The CSAM detection system supposes people save actual child porn to their personal photo libraries.
cirrus3 5y ago
You seem to be assuming a human cannot tell the difference from some random NSFW content, and some legit known CSAM, 30 times. Try again.
- least 5y ago
  Roughly thirty images is the threshold for the system to activate, but would they need to review all thirty images to pass it on or would they just need to verify one image looks visually like CSAM in order to pass it along? It seems unlikely in the event that there was anything that they verified as CSAM they wouldn't pass it on just because they found a false positive in those thumbnails.
- marcan_42 5y ago
  Apple's reviewers don't have access to the original CSAM to know if it's a match or not. That stays with NCMEC. If they see some legal porn that looks like it could be illegal, they'd likely flag it as a match.
ec109685 5y ago
Keep in mind that you have to also collide with another perceptual hash function that only Apple has to trigger a match.
- nullc 5y ago
  > Keep in mind that you have to also collide with another perceptual hash function that only Apple has to trigger a match. If it's another neural network I wouldn't be shocked if the adversarial preimages worked across both-- it's not uncommon for blackbox generalization to work for adversarial examples. It would be very likely if someone (maybe the attacker) made their own version of neuralhash and then generated examples that passed both theirs and apple's public one. Privacy wise, if there were two perceptual hash functions Apple should have used the more restrictive one on the devices too -- because even if they decide to not report you, your privacy is still invaded if they inspect at your images at all. The neuralhash function is extremely easy to attack. We should not have any confidence in the competence of its authors, so we shouldn't expect their undisclosed mechanism to provide a great deal of protection. A secret second hash also will not be secret against a state attacker who will have access to this function by virtue of being trusted to create the databases for Apple. There is, however, a very simple technique they could use that would provide almost perfect protection: They could stop invading the privacy of their users and refrain from scanning their private content!
tandav 5y ago
Does Google Chrome scans downloaded images ?

robertoandred 5y ago

Does anyone save porn to their personal photo libraries? Especially porn as suspicious as the image you posted?

throwaway384950 5y ago
Going by what some people on Reddit say, it seems to be the case. https://old.reddit.com/r/datahoarder/search?q=porn&restrict_sr=on https://old.reddit.com/r/datahoarder/search?q=porn&restrict_... Probably not the weird image I posted, which looks obviously suspicious. But maybe someone will make a program to find "cleaner" hash collisions that don't look suspicious.
- nullc 5y ago
  https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issues/1#issuecomment-902977931 https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue... I posted some examples that look like totally normal images, they're no harder to produce, you just need to noise-shape the gradient descent so that the introduced noise has a spectrum similar to the image. E.g. just feeding back a gaussian highpassed version of the error signal is sufficient.
pseudalopex 5y ago
The CSAM detection system supposes people save actual child porn to their personal photo libraries.

cirrus3 5y ago

You seem to be assuming a human cannot tell the difference from some random NSFW content, and some legit known CSAM, 30 times. Try again.

least 5y ago
Roughly thirty images is the threshold for the system to activate, but would they need to review all thirty images to pass it on or would they just need to verify one image looks visually like CSAM in order to pass it along? It seems unlikely in the event that there was anything that they verified as CSAM they wouldn't pass it on just because they found a false positive in those thumbnails.
marcan_42 5y ago
Apple's reviewers don't have access to the original CSAM to know if it's a match or not. That stays with NCMEC. If they see some legal porn that looks like it could be illegal, they'd likely flag it as a match.

ec109685 5y ago

Keep in mind that you have to also collide with another perceptual hash function that only Apple has to trigger a match.

nullc 5y ago
> Keep in mind that you have to also collide with another perceptual hash function that only Apple has to trigger a match. If it's another neural network I wouldn't be shocked if the adversarial preimages worked across both-- it's not uncommon for blackbox generalization to work for adversarial examples. It would be very likely if someone (maybe the attacker) made their own version of neuralhash and then generated examples that passed both theirs and apple's public one. Privacy wise, if there were two perceptual hash functions Apple should have used the more restrictive one on the devices too -- because even if they decide to not report you, your privacy is still invaded if they inspect at your images at all. The neuralhash function is extremely easy to attack. We should not have any confidence in the competence of its authors, so we shouldn't expect their undisclosed mechanism to provide a great deal of protection. A secret second hash also will not be secret against a state attacker who will have access to this function by virtue of being trusted to create the databases for Apple. There is, however, a very simple technique they could use that would provide almost perfect protection: They could stop invading the privacy of their users and refrain from scanning their private content!

tandav 5y ago

Does Google Chrome scans downloaded images ?