3 ms·

And if you make the second image be actual, 18+ porn? Will Apple be able to tell the difference between that and CSAM after the blur is applied? Bonus points i

And if you make the second image be actual, 18+ porn? Will Apple be able to tell the difference between that and CSAM after the blur is applied? Bonus points if you match poses, coloration, background, etc.

floatingatoll 5y ago
The only way to match poses, coloration, background, etc is to possess illegal CSAM content. If you do so successfully, you will result in your crafted image passing the blur check and reaching the agency that possesses the original image for final verification, where it will immediately fail because it is obviously a replica. You will then trigger that agency leading law enforcement to find the creator of the image, so that they can arrest whoever possesses CSAM content in order to model replicas after it. I think that's a perfectly acceptable outcome, since anyone with the hubris to both possess CSAM content and create replicas of it especially deserves to be arrested. Do you see a more problematic outcome here?
- laverya 5y ago
  This assumes that it's impossible to reverse the perceptual hashes used here in such a way that you could determine poses and coloration, for one. And in retrospect, you don't need to match that - you just need it to appear obviously pornographic after the blur is applied in order to get past Apple's reviewers. After that, the lucky individual's life is in the hands of the police/prosecutors. (I have to imagine that both real and faked cases will look pretty much like "Your honor/members of the jury, this person's device contained numerous photos matching known CSAM. No, we won't be showing you the pictures. No, the defence can't see them either." Can you imagine a "tough on crime" prosecutor taking the faked case to trial too? Would the police and prosecutors even know it was faked?)
  - floatingatoll 5y ago
    Apple's reviewers are comparing blurred originals to blurred matches. It needs to look like the blurred original associated with the checksum that matched. It is irrelevant whether the blurred match looks pornographic or not.
    - ultrarunner 5y ago
      This has already been demonstrated. https://twitter.com/SarahJamieLewis/status/1428083744280256512 https://twitter.com/SarahJamieLewis/status/14280837442802565...
      - floatingatoll 5y ago
        At which point the image will be handed to the relevant CSAM group unblurred, who will do a visual comparison and find out immediately that it’s not a match, and then reject it without invoking law enforcement.
- ultrarunner 5y ago
  Cops no-knock raid the wrong houses all the time; what makes you think that the procedure you outlined above will be followed without error? Especially against suspected child abusers, from an anti-child-abuse agency? What if the visual check gets accidentally signed off on, or even gets fraudulently marked as positive by a burned out/lazy/competent-but-distracted employee? There are just so many failure modes for this dragnet that all end with innocent people suffering a very difficult process, even if they don't eventually land in prison. I don't think it's unreasonable to not want to be volunteered to participate.
- SXX 5y ago
  Why would attacker need to create anything from scratch? If you want to build a dataset of images with people with specific skin and hair color, body types, etc in very specific poses it's can be quite hard and expensive to do so. Because even photo stocks have limited number of such photos. Unless... Unless you're looking to build a porn dataset and you're don't care about copyright. Porn is industry where exabytes of material are produced and published on internet almost every week. Who will agency come to? To some OnlyFans creators?

floatingatoll 5y ago

The only way to match poses, coloration, background, etc is to possess illegal CSAM content. If you do so successfully, you will result in your crafted image passing the blur check and reaching the agency that possesses the original image for final verification, where it will immediately fail because it is obviously a replica. You will then trigger that agency leading law enforcement to find the creator of the image, so that they can arrest whoever possesses CSAM content in order to model replicas after it. I think that's a perfectly acceptable outcome, since anyone with the hubris to both possess CSAM content and create replicas of it especially deserves to be arrested. Do you see a more problematic outcome here?

laverya 5y ago
This assumes that it's impossible to reverse the perceptual hashes used here in such a way that you could determine poses and coloration, for one. And in retrospect, you don't need to match that - you just need it to appear obviously pornographic after the blur is applied in order to get past Apple's reviewers. After that, the lucky individual's life is in the hands of the police/prosecutors. (I have to imagine that both real and faked cases will look pretty much like "Your honor/members of the jury, this person's device contained numerous photos matching known CSAM. No, we won't be showing you the pictures. No, the defence can't see them either." Can you imagine a "tough on crime" prosecutor taking the faked case to trial too? Would the police and prosecutors even know it was faked?)
- floatingatoll 5y ago
  Apple's reviewers are comparing blurred originals to blurred matches. It needs to look like the blurred original associated with the checksum that matched. It is irrelevant whether the blurred match looks pornographic or not.
  - ultrarunner 5y ago
    This has already been demonstrated. https://twitter.com/SarahJamieLewis/status/1428083744280256512 https://twitter.com/SarahJamieLewis/status/14280837442802565...
    - floatingatoll 5y ago
      At which point the image will be handed to the relevant CSAM group unblurred, who will do a visual comparison and find out immediately that it’s not a match, and then reject it without invoking law enforcement.
ultrarunner 5y ago
Cops no-knock raid the wrong houses all the time; what makes you think that the procedure you outlined above will be followed without error? Especially against suspected child abusers, from an anti-child-abuse agency? What if the visual check gets accidentally signed off on, or even gets fraudulently marked as positive by a burned out/lazy/competent-but-distracted employee? There are just so many failure modes for this dragnet that all end with innocent people suffering a very difficult process, even if they don't eventually land in prison. I don't think it's unreasonable to not want to be volunteered to participate.
SXX 5y ago
Why would attacker need to create anything from scratch? If you want to build a dataset of images with people with specific skin and hair color, body types, etc in very specific poses it's can be quite hard and expensive to do so. Because even photo stocks have limited number of such photos. Unless... Unless you're looking to build a porn dataset and you're don't care about copyright. Porn is industry where exabytes of material are produced and published on internet almost every week. Who will agency come to? To some OnlyFans creators?