5 ms·
You can extract the hashes with a few hours spent on the darknet. Doing that is certainly illegal and not to mention VERY morally wrong, but criminals exist and
by dannyw 5y ago
You can extract the hashes with a few hours spent on the darknet. Doing that is certainly illegal and not to mention VERY morally wrong, but criminals exist and criminals won't hesitate to abuse this as a mechanism for framing, extortion, or ransom.
It's also possible for someone (Attacker A) to go on the darknet and get a list of 96-bit neural hashes, and then publish or sell this list somewhere to another party, Attacker B. The second party would never have to interact with CSAM.
Imagine Ransomware v2: We have inserted 29 photos of CSAM-matching material into your photo library. Pay X monero to this address in 30 minutes, or we will insert 2 additional photos, which will cross the threshold and may result in serious and life-changing consequences to you[1].
The difference here (versus the status quo) is that an easily-broken perceptual hashing enables the attacker to never send or possess any CSAM images[2]. From my experiences with being victims of various hackers, I know a lot of them won't touch CSAM because they know it's wrong, but they'll salivate at an opportunity to weaponise automated CSAM scanning.
[1]: If you think Apple's human review will mitigate this attack, you can permute legal pornography to match CSAM signatures. If Apple's reviewers see 30 CSAM matches and the visual derivatives look like porn, they will be legally required to report to to NCMEC (a statutory quasi-government agency staffed by the FBI), even if all the photos are actually consensual adults.
[2]: If you never possess nor touch CSAM, it might be harder for you to get charged with CP charges. You might be looking at CFAA, blackmail or extortion charges; while your victim faces child pornography charges. This is basically an "amplification attack" on the real world judicial system.
- d0100 5y ago> If Apple's reviewers see 30 CSAM matches and the visual derivatives look like porn Even worse, just get a "teen" porn screengrab, pass it through the collider and you have pretty much a smoking gun
- XorNot 5y agoThe "visual derivative" is not something any of us have been shown an example of either. Whatever it is, I suspect you only need to be vaguely in the same ballpark (I would wager humanoid shaped skin tones maybe). So I suspect it would be easier then that (particularly since this whole hashing scheme has been surrounded with a lot of clear garbage - "1 in a trillion" -> on demand collisions in a couple of weeks?
- ComputerGuru 5y agoI think visual derivative is just a beating-around-the-bush way of saying “thumbnail.”
- XorNot 5y agoThat's their words that they feel no need to elaborate on. Obviously they actually seem to just be doing the "technically the truth" thing - which shows that someone realized no one would like hearing what it actually is.
- ComputerGuru 5y agoYeah, it was very pointedly awkwardly worded. It’s intended for human reviewers to distinguish a false positive from a real positive. An eigen vector mapped image isn’t going to do that, a heavily Gaussian-blurred image isn’t going to do that - it needs to be something that a minimum wage person who’s only been trained a day or to can distinguish as “CSAM” or “not CSAM” and that means it’s a thumbnail of sorts.
- Scaevolus 5y agoOne of the grosser clean room designs. It's certainly possible, but I posit that the exploit chain necessary to get the capability to inject photos onto an arbitrary user's iPhone is valuable enough that it's more likely to be used for spying by repressive regimes than straight up blackmail-- and if you had such a capability, why bother with hash-colliding permutations of legal pornography? Why not plant CSAM directly onto the user's device? Nearly all cloud storage services implement a scanner like this, and permit the same level of blackmail with a simpler attack chain, such as phishing Dropbox credentials to inject illegal material. I think the more interesting attacks are governments colluding to add politically motivated non-CSAM material to the lists and then requiring Apple allow them to perform the human review to discover dissidents.
- dannyw 5y agoIf you plant CSAM, you must possess, distribute, and transmit CSAM. That creates moral and legal barriers. The median ransomware actor probably finds CSAM repulsive and wrong. If you plant material that matches CSAM hashes, you do none of that. The median ransomware actor might find this to thje fastest way to collect a thousand monero. Also, you can distribute 30 media items per message via WhatsApp. There is a configurable setting for WhatsApp to save all received photos to your iCloud photo library. No exploits needed, you could probably weaponise this via an WhatsApp bot.
- user-the-name 5y agoThis attack does not work, as Apple uses two hashing algorithms, one on the device which is now public, and one which is secret. You would have to collide both, which would be hard enough if you knew what the second one was, which you don't.