3 ms·

I assume these "non-private image derivatives" are downscaled versions of the original image. But for downscaling there also are adversarial techniques: perturb

by y7 5y ago

I assume these "non-private image derivatives" are downscaled versions of the original image. But for downscaling there also are adversarial techniques: perturb an image such that the downscaled version looks like a target image. See https://bdtechtalks.com/2020/08/03/machine-learning-adversarial-image-scaling/ https://bdtechtalks.com/2020/08/03/machine-learning-adversar...

shuckles 5y ago
For the server side component, the challenge is knowing the target of adversarial generation since, again, the perceptual hash used is secret. At that point, you are reduced to the threat present in every other existing CSAM detection system.
- y7 5y ago
  > the perceptual hash used is secret Yes, although I'm sure a sufficiently motivated attacker can obtain some CSAM that they are reasonably sure is present in the database, and generate the NeuralHash themselves. > At that point, you are reduced to the threat present in every other existing CSAM detection system. A difference could be that server-side CSAM detection will verify the entire image, and not just the image derivative, before notifying the authorities.
  - zepto 5y ago
    > Yes, although I'm sure a sufficiently motivated attacker can obtain some CSAM that they are reasonably sure is present in the database, and generate the NeuralHash themselves Remind us what the attack is here? The neural hash and the visual derivative both have to match for an image to trigger detection.
    - shuckles 5y ago
      I believe something like: an adversary creates an innocuous photo where: * The photo itself is benign. * The photo’s NeuralHash matches known CSAM. * The photo’s image derivative is not benign. It looks visually like CSAM. * The photo’s image derivative matches known CSAM per a private perceptual hash. The above, combined, could have a victim reported to NCMEC without being aware they were targeted. Since Apple operates on image derivatives, they could be fooled unlike other cloud providers. That is the claim. At that point, the victim could point law enforcement to the original CloudKit asset (safety vouchers include a reference to the associate asset) and clear their name. However, involving law enforcement can always be traumatic.
      - zepto 5y ago
        Sure, but did I miss something that suggests that all 4 of those conditions can actually be met? That seems like the part that has been made up. Afaik the image derivative isn’t checked for looking like CSAM. It’s checked for looking like the specific CSAM from the database.
        shuckles 5y ago
        CSAM in the database is confidential, so the human reviewer just needs to be plausibly confident that they have CSAM in front of them. However, it’s not clear to me that you can pull off all three simultaneously. Furthermore, the attack doesn’t specify how they would adversarially generate an image derivative for a secret perceptual hash that they can’t run gradient descent on. If someone wanted to plant CSAM and had control of an iCloud account, it seems far easier to send some emails with those images since iCloud Mail is actively scanned and nobody checks their iCloud Mail account, especially not the sent folder.
        zepto 5y ago
        CSAM in the database is confidential. The question is whether the visual derivatives are checked against derivatives from the database or just against abstract criteria. That seems to be an unknown. > However, it’s not clear to me that you can pull off all three simultaneously Agreed. People here seem to keep assuming that you can, but so far nobody has demonstrated that it is possible.

by y7 5y ago

shuckles 5y ago

For the server side component, the challenge is knowing the target of adversarial generation since, again, the perceptual hash used is secret. At that point, you are reduced to the threat present in every other existing CSAM detection system.

y7 5y ago
> the perceptual hash used is secret Yes, although I'm sure a sufficiently motivated attacker can obtain some CSAM that they are reasonably sure is present in the database, and generate the NeuralHash themselves. > At that point, you are reduced to the threat present in every other existing CSAM detection system. A difference could be that server-side CSAM detection will verify the entire image, and not just the image derivative, before notifying the authorities.
- zepto 5y ago
  > Yes, although I'm sure a sufficiently motivated attacker can obtain some CSAM that they are reasonably sure is present in the database, and generate the NeuralHash themselves Remind us what the attack is here? The neural hash and the visual derivative both have to match for an image to trigger detection.
  - shuckles 5y ago
    I believe something like: an adversary creates an innocuous photo where: * The photo itself is benign. * The photo’s NeuralHash matches known CSAM. * The photo’s image derivative is not benign. It looks visually like CSAM. * The photo’s image derivative matches known CSAM per a private perceptual hash. The above, combined, could have a victim reported to NCMEC without being aware they were targeted. Since Apple operates on image derivatives, they could be fooled unlike other cloud providers. That is the claim. At that point, the victim could point law enforcement to the original CloudKit asset (safety vouchers include a reference to the associate asset) and clear their name. However, involving law enforcement can always be traumatic.
    - zepto 5y ago
      Sure, but did I miss something that suggests that all 4 of those conditions can actually be met? That seems like the part that has been made up. Afaik the image derivative isn’t checked for looking like CSAM. It’s checked for looking like the specific CSAM from the database.
      - shuckles 5y ago
        CSAM in the database is confidential, so the human reviewer just needs to be plausibly confident that they have CSAM in front of them. However, it’s not clear to me that you can pull off all three simultaneously. Furthermore, the attack doesn’t specify how they would adversarially generate an image derivative for a secret perceptual hash that they can’t run gradient descent on. If someone wanted to plant CSAM and had control of an iCloud account, it seems far easier to send some emails with those images since iCloud Mail is actively scanned and nobody checks their iCloud Mail account, especially not the sent folder.
        zepto 5y ago
        CSAM in the database is confidential. The question is whether the visual derivatives are checked against derivatives from the database or just against abstract criteria. That seems to be an unknown. > However, it’s not clear to me that you can pull off all three simultaneously Agreed. People here seem to keep assuming that you can, but so far nobody has demonstrated that it is possible.