4 ms·
Reading the imagine scaling attack article, it’s looks like it’s pretty easy to manufacture an image that: 1. Looks like an innocuous image, indeed even an ima
by avianlyric 5y ago
Reading the imagine scaling attack article, it’s looks like it’s pretty easy to manufacture an image that:
1. Looks like an innocuous image, indeed even an image the victim is expecting to receive.
2. Downscales in such a way to produce a CSAM match.
3. Downscales for the derivative image to create actual CSAM for the review process.
Which is a pretty scary attack vector.
- FabHK 5y agoDepends very much on the process Apple uses to make the "visual derivative", though. Also, defence by producing the original innocuous image (and showing that it triggers both parts of Apple's process, NeuralHash and human review of the visual derivative) should be possible, though a lot of damage might've been done by then.
- avianlyric 5y ago> Also, defence by producing the original innocuous image At this point you’re already inside the guts of the justice system, and have been accused of distributing CSAM. Indeed depending on how diligent the prosecutor is, you might need to wait till trial before you can defend yourself. At that point you’re life as you know is already fucked. The only thing proving your innocence (and the need to do so is itself a complete miscarriage of justice) will save you from is a prison sentence.
- ta988 5y agoAnd now you will be accused of trying to hide illegal material in innocuous images.
- zepto 5y agoThis isn’t true at all. If the creation of fakes is as easy as claimed, Neuralhash evidence alone will become inadmissible. There are plenty of lawyers and money waiting to establish this.
- IncRnd 5y ago> This isn’t true at all. > If the creation of fakes is as easy as claimed, Neuralhash evidence alone will become inadmissible. Okay. https://github.com/anishathalye/neural-hash-collider https://github.com/anishathalye/neural-hash-collider
- spurgu 5y agoUh? So his if statement is true?
- IncRnd 5y agoPlease read what is written right before that... You are taking something out of context.
- zepto 5y agoWhy do you keep posting links to this collider as though it means something? As has been already pointed out the system is designed to handle attacks like this. Here is the relevant paragraph from Apple’s documentation: “as an additional safeguard, the visual derivatives themselves are matched to the known CSAM database by a second, independent perceptual hash. This independent hash is chosen to reject the unlikely possi- bility that the match threshold was exceeded due to non-CSAM images that were ad- versarially perturbed to cause false NeuralHash matches against the on-device en- crypted CSAM database. If the CSAM finding is confirmed by this independent hash, the visual derivatives are provided to Apple human reviewers for final confirmation.” https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model...
- zepto 5y agoWhere does it say anything that indicates #1 and #3 are both possible?