10 ms·
This is just getting wilder and wilder by the day, how spectacularly this move has backfired. As others have commented, at this point all you need is someone wi
by onepunchedman 5y ago
This is just getting wilder and wilder by the day, how spectacularly this move has backfired. As others have commented, at this point all you need is someone willing to sell you the CSAM hashes on the darknet, and this system is transparently broken.
Until that day, just send known CSAM to any person you'd like to get in trouble (make sure they have icloud sync enabled), be it your neighbour or a political figure, and start a PR campaign accusing the person of being investigated for it. The whole concept is so inherently flawed it's crazy they haven't been sued yet.
- deleted 5y ago[deleted]
- dannyw 5y agoThe "send known CSAM" attack has existed for a while but never made sense. However, this technology enables a new class of attacks: "send legal porn, collided to match CSAM perceptual hashes". With the previous status quo: 1. The attacker faces charges of possessing and distributing child pornography 2. The victim may be investigated and charged with child pornography if LEO is somehow alerted (which requires work, and can be traced to the attacker). Poor risk/reward payoff, specifically the risk outweighs the reward. So it doesn't happen (often). --- With the new status quo of lossy, on-device CSAM scanning and automated LEO alerting: 1. The attacker never sends CSAM, only material that collides with CSAM hashes. They will be looking at charges of CFAA, extortion, and blackmail. 2. The victim will be automatically investigated by law enforcement, due to Apple's "Safety Voucher" system. The victim will be investigated for possessing child pornography, particularly if the attacker collides legal pornography that may fool a reviewer inspecting a 'visual derivative'. Great risk/reward payoff. The reward dramatically outweighs the risk, as you can get someone in trouble for CSAM without ever touching CSAM yourself. If you think ransomware is bad, just imagine CSAM-collision ransomware. Your files will be replaced* with legal pornography that is designed specifically to collide with CSAM hashes and result in automated alerting to law enforcement. Pay X monero within the next 30 minutes, or quite literally, you may go to jail, and be charged with possessing child pornography, until you spend $XXX,XXX on lawyers and expert testimony that demonstrates your innocence. * Another delivery mechanism for this is simply sending collided photos over WhatsApp, as WhatsApp allows for up to 30 media images in one message, and has settings that will automatically add these images to your iCloud photo library.
- onepunchedman 5y agoSomehow this didn't solidify my trust in Apple! By this standard you can probably mount a half decent defence off "ignorance" if you are even caught sending the colliding material. Add this whole debacle on top of what's going on in the EU parliament and 2021 has been WILD for privacy.
- robertoandred 5y agoLEO is not alerted automatically, where’d you get that idea?
- hypothesis 5y agoThey will be if you collide a low-res image that resembles CSAM. Why would person doing manual review risk his job in case if he’s unsure? Naturally he will just play it safe and report images.
- ec109685 5y agoNot resembles. The adversarial image has to match a private perceptual hash function of the same CSAM image that the NeuralHash function matched before a human reviewer ever looks at it.
- onepunchedman 5y agoDo you have any material on this private function?
- ec109685 5y agoNot beyond the documents Apple has shared. Presumably it will be kept that way given it prevents an adversarial attack against it.
- silisili 5y agoThey'd more or less have to be. Well, not necessarily 'police', but NCMEC. I did work in automating abuse detection years back, and the US govt clearly tells you are not to open/confirm suspected, reported, or happened upon cp. There's a lot of other seemingly weird laws and rules around it.
- robertoandred 5y agoWhy would anyone save CSAM to their photo library?
- dannyw 5y agoA hash collision allows you to create material that matches CSAM signatures, without being CSAM. This opens up a new class of attacks. Specifically, many criminal actors don't touch CSAM because it's wrong. But some of these criminal actors will happily abuse legal systems, e.g. SWATTing.
- seph-reed 5y agoI would gladly have a mobile phone full of memes that have been modified to match, just for the lulz. I honestly think every meme should be put through just to have "illegal memes"
- zepto 5y ago> A hash collision allows you to create material that matches CSAM signatures, without being CSAM. This is not correct. Hash collisions won’t match the visual derivative.
- 5y ago
- shuckles 5y agoWhy wait? Just send them the pictures on Facebook Messenger or Gmail or Dropbox today.
- onepunchedman 5y agoNah that's so 2020, 2021 is all about low resolution legitimate porn being transformed to match CSAM. Get with the times!
- shuckles 5y agoThose will trip up 2020’s systems as well!
- norov 5y agoBut why low resolution porn?
- onepunchedman 5y agoSo that you are able to bypass the manual reviews. It still looks like CSAM, but it isn't.
- deleted 5y ago[deleted]
- SCLeo 5y agoI can't tell if you are being sarcastic. In case you are not, isn't the act of sending those pictures completely illegal?
- shuckles 5y agoPeople here are proposing intentionally creating image assets which collide with perceptual hashes of known CSAM (ignoring whether that is legal or ethical) and sharing those assets to effectively SWAT unaware targets.
- deleted 5y ago[deleted]
- cookiengineer 5y agoImagine being a parent that made pictures of their own children that bathed naked in their own backyard. I don't know about you, but my parents certainly have lots of embarassing pictures of me in their photo album. There will be so many false positives in that system, it's ridiculous. It doesn't necessarily have to be a false colliding hash, but legitimate use cases that - by definition - are impossible to train neural nets on unless the data is being used illegally by Apple.
- onepunchedman 5y agoWhat does Apple even do in this situation? That media won't match known CSAM, but if you modify childhood images so that its hash matches CSAM, what does Apple do. There are just SO MANY things that can and will go wrong as people try to exploit this system.
- zepto 5y agoYou can’t modify your childhood images so their hash matches csam because the visual derivative won’t match.
- ec109685 5y agoThat’s not how Apple’s system works. It’s not an image classifier. Only actual images that are derivatives of known CSAM images (a database of 250k images) will match. Random images of kids will not match those at any greater frequency than any other image.
- cookiengineer 5y agoCounter-question: At what point is child porn actually child porn, socially and statistically speaking? If I share that picture of my child with my friends and loved ones on Facebook - at what "scale" is it considered to be added to that database as child porn? 1k shares? 10k? Who's the one eligible to decide that? The judicatives? I think this scenario is a constitutional crisis because there's no good solution to it in terms of law and order.
- zepto 5y agoExcept that it isn’t. The hashes don’t enable an attack.