3 ms·

> The second hash algorithm is not currently public knowledge; fooling an unknown algorithm presents a different set of challenges At a minimum it is known to

by nullc 5y ago

> The second hash algorithm is not currently public knowledge; fooling an unknown algorithm presents a different set of challenges At a minimum it is known to the state actors generating these databases. Apple itself has directly acknowledged abuse by state actors as a threat vector, so it is entirely inappropriate to rely on obscurity here. (and would be a bad security practice regardless, since presumably other attackers could steal the other algorithims). > For the purposes of fooling Apple's NeuralHash to incorrectly flag a CSAM false positive, "easily" involves being in possession of the absolute worst-of-the-worst CSAM image This is false. It requires knowing the hash of the target image, which is completely lawful to possess. You don't have to be in possession of the illegal images. You cannot assume these hashes are impossible to obtain-- because if they were it would mean the whole scheme was pointless (since it would also mean the unlawful images were impossible to obtain). But even if they were impossible to obtain, an attacker could simply create new "apparent" child abuse material submit it to the databases while at the same time creating whatever false matches against it that they wanted. You appear to be assuming an attack model where the attackers won't do "bad" things. Under that set of assumptions all systems are secure. Reality disagrees with this model.

simondotau 5y ago
> You appear to be assuming an attack model where the attackers won't do "bad" things. Under that set of assumptions all systems are secure. Reality disagrees with this model. What I'm saying is that a whole bunch of stars need to align before any practical use for this exploit becomes even remotely plausible. This is slightly more likely if your adversary is a state actor with strong motivations and no issues accessing and handling CSAM. But consider the necessary steps and ask how likely it is: 1. Hope that your client uses an iPhone, pays for iCloud storage, and has enabled the cloud photo library feature, not on by default. 2. Acquire the NeuralHash and secondary hash for thirty A1-class images in the CSAM database. 3. Acquire the secondary hash algorithm and build a tool that generates hash collisions for that. 4. Find thirty images which have even the most vanishing small hope of being accidentally identifiable as CSAM by a human reviewer, and that your target would like so much that they'd be inclined to import all thirty into their iCloud Photo library upon receiving them. 5. Mangle these thirty images to each match both hashes. But without mangling them so much that the target's desire to import them into their iCloud photo gallery diminishes, whether for aesthetic reasons or because the target becomes suspicious. 6. Hope that your target does actually import all thirty images into their iCloud photo gallery. 7. Hope that your hash collsions all work correctly in practice. 8. Hope that Apple's human reviewers screw up. 9. Hope that none of the 30 images cause any of Apple's reviewers to become suspicious, flagging the matter for technical review, causing Apple to blacklist these hashes and rev their hash algorithms in the next version of iOS. 10. Hope that the child safety organisation which Apple reports this to also screws up. 11. Hope that the law enforcement body which this child safety organisation reports this to also screws up. 12. Hope that bumbling actions of law enforcement cause desirable negative consequences to your target. That's a dozen steps, many of which are exceedingly non-trivial. There's probably a dozen more steps I'm missing or unaware of. Upshot—a lot of work for very little outcome. Far more effective would be to break into the guy's house, stuff some printouts of CSAM under their mattress and then make a bunch of anonymous tip-offs to local law enforcement. Or just pay a guy to shoot him.
- nullc 5y ago
  > 1. Hope that your client uses an iPhone, pays for iCloud storage, and has enabled the cloud photo library feature, not on by default. True for millions of people, also assumes the functionality won't be expanded once initially deployed. > 2. Acquire the NeuralHash and secondary hash for thirty A1-class images in the CSAM database. Authorities claim these databases have millions of hits per year (NCMEC says 21.7 million hits in 2020), so clearly these are not that hard to obtain. > Acquire the secondary hash algorithm and build a tool that generates hash collisions for that. Because this is protected by security through obscurity we can't assess this step, but it's a standard practice in security assessment to assume obscure mechanisms add zero security. > Find thirty images which have even the most vanishing small hope of being accidentally identifiable as CSAM by a human reviewer, and that your target would like so much that they'd be inclined to import all thirty into their iCloud Photo library upon receiving them. So. Ordinary, perfectly lawful hardcore pornography-- just minimally selected so that you can't tell how old the talent is. This is indistinguishable from child abuse material especially in review context which has been described (viewing low resolution sobel transformed 'extracts'). But also: At this point when someone is reviewing any of their images the users privacy is already violated. Moreover, a hacker or state actor which has compromised Apple's systems could intercept the content at this point and monitor it regardless of what the review found. > Mangle these thirty images to each match both hashes. But without mangling them so much that the target's desire to import them into their iCloud photo gallery diminishes, whether for aesthetic reasons or because the target becomes suspicious. I can't speak to the claimed other hash, but I've demonstrated that for neuralhash you can make images match hashes without looking modified, and certainly without making them look unappealing. > 6. Hope that your target does actually import all thirty images into their iCloud photo gallery. well there you have me, because no one who cares about personal privacy or civil rights should ever use an apple device. :) > 12. Hope that bumbling actions of law enforcement cause desirable negative consequences to your target. Just being accused or questioned is a devastating negative consequence. It's also not too difficult to find examples of people being criminally charged over totally lawful pornography because the prosecutors couldn't bother to use the internet to look up the actresses DOB. > Upshot—a lot of work for very little outcome. Far more effective would be to break into the guy's house, stuff some printouts of CSAM under their mattress and then make a bunch of anonymous tip-offs to local law enforcement. Or just pay a guy to shoot him. That may be true, but it has a different risk reward tradeoff. By your argument swatting would never happen-- it requires a lot of complicated steps, more so than just pulling the trigger on a gun or swinging a baseball bat yourself. And for what benefit? As the article shows-- these system's are easily evaded. Should our privately owned computing devices be tattling on us based on opaque naughty lists which are cryptographically protected against accountability? -- just to catch some perverts trading widely circulated gross pictures which they're too stupid to fuzz against detection? and not even that-- as you noted in other posts: The pervs will just use other tools. So we degrade everyone's privacy, further establish the precedent that it's morally acceptable for your own private property to conspire against you, and probably don't actually protect any children. I think we should admit: Especially in light of the news on the same week this was announced about the doctor that was keeping lots of child porn in their iphotos and was caught by another service. The purpose of this scheme isn't to protect children, it isn't to preserve user privacy, it exists to increase and preserve Apple's profit margin by distancing them from creeps and it does so at the expense of the privacy of their users. Apple is converting their customer privacy into a private profit. ISTM the whole scheme may actually encourage abusing more children: The one reliable way to get sexual pictures of young people that this system cannot catch is to make new images. If this sort of thing were effective it could be expected to enlarge the commercial market for new abuse images by shifting demand from the back catalog of existing material to new material.

by nullc 5y ago

simondotau 5y ago

> You appear to be assuming an attack model where the attackers won't do "bad" things. Under that set of assumptions all systems are secure. Reality disagrees with this model. What I'm saying is that a whole bunch of stars need to align before any practical use for this exploit becomes even remotely plausible. This is slightly more likely if your adversary is a state actor with strong motivations and no issues accessing and handling CSAM. But consider the necessary steps and ask how likely it is: 1. Hope that your client uses an iPhone, pays for iCloud storage, and has enabled the cloud photo library feature, not on by default. 2. Acquire the NeuralHash and secondary hash for thirty A1-class images in the CSAM database. 3. Acquire the secondary hash algorithm and build a tool that generates hash collisions for that. 4. Find thirty images which have even the most vanishing small hope of being accidentally identifiable as CSAM by a human reviewer, and that your target would like so much that they'd be inclined to import all thirty into their iCloud Photo library upon receiving them. 5. Mangle these thirty images to each match both hashes. But without mangling them so much that the target's desire to import them into their iCloud photo gallery diminishes, whether for aesthetic reasons or because the target becomes suspicious. 6. Hope that your target does actually import all thirty images into their iCloud photo gallery. 7. Hope that your hash collsions all work correctly in practice. 8. Hope that Apple's human reviewers screw up. 9. Hope that none of the 30 images cause any of Apple's reviewers to become suspicious, flagging the matter for technical review, causing Apple to blacklist these hashes and rev their hash algorithms in the next version of iOS. 10. Hope that the child safety organisation which Apple reports this to also screws up. 11. Hope that the law enforcement body which this child safety organisation reports this to also screws up. 12. Hope that bumbling actions of law enforcement cause desirable negative consequences to your target. That's a dozen steps, many of which are exceedingly non-trivial. There's probably a dozen more steps I'm missing or unaware of. Upshot—a lot of work for very little outcome. Far more effective would be to break into the guy's house, stuff some printouts of CSAM under their mattress and then make a bunch of anonymous tip-offs to local law enforcement. Or just pay a guy to shoot him.

nullc 5y ago
> 1. Hope that your client uses an iPhone, pays for iCloud storage, and has enabled the cloud photo library feature, not on by default. True for millions of people, also assumes the functionality won't be expanded once initially deployed. > 2. Acquire the NeuralHash and secondary hash for thirty A1-class images in the CSAM database. Authorities claim these databases have millions of hits per year (NCMEC says 21.7 million hits in 2020), so clearly these are not that hard to obtain. > Acquire the secondary hash algorithm and build a tool that generates hash collisions for that. Because this is protected by security through obscurity we can't assess this step, but it's a standard practice in security assessment to assume obscure mechanisms add zero security. > Find thirty images which have even the most vanishing small hope of being accidentally identifiable as CSAM by a human reviewer, and that your target would like so much that they'd be inclined to import all thirty into their iCloud Photo library upon receiving them. So. Ordinary, perfectly lawful hardcore pornography-- just minimally selected so that you can't tell how old the talent is. This is indistinguishable from child abuse material especially in review context which has been described (viewing low resolution sobel transformed 'extracts'). But also: At this point when someone is reviewing any of their images the users privacy is already violated. Moreover, a hacker or state actor which has compromised Apple's systems could intercept the content at this point and monitor it regardless of what the review found. > Mangle these thirty images to each match both hashes. But without mangling them so much that the target's desire to import them into their iCloud photo gallery diminishes, whether for aesthetic reasons or because the target becomes suspicious. I can't speak to the claimed other hash, but I've demonstrated that for neuralhash you can make images match hashes without looking modified, and certainly without making them look unappealing. > 6. Hope that your target does actually import all thirty images into their iCloud photo gallery. well there you have me, because no one who cares about personal privacy or civil rights should ever use an apple device. :) > 12. Hope that bumbling actions of law enforcement cause desirable negative consequences to your target. Just being accused or questioned is a devastating negative consequence. It's also not too difficult to find examples of people being criminally charged over totally lawful pornography because the prosecutors couldn't bother to use the internet to look up the actresses DOB. > Upshot—a lot of work for very little outcome. Far more effective would be to break into the guy's house, stuff some printouts of CSAM under their mattress and then make a bunch of anonymous tip-offs to local law enforcement. Or just pay a guy to shoot him. That may be true, but it has a different risk reward tradeoff. By your argument swatting would never happen-- it requires a lot of complicated steps, more so than just pulling the trigger on a gun or swinging a baseball bat yourself. And for what benefit? As the article shows-- these system's are easily evaded. Should our privately owned computing devices be tattling on us based on opaque naughty lists which are cryptographically protected against accountability? -- just to catch some perverts trading widely circulated gross pictures which they're too stupid to fuzz against detection? and not even that-- as you noted in other posts: The pervs will just use other tools. So we degrade everyone's privacy, further establish the precedent that it's morally acceptable for your own private property to conspire against you, and probably don't actually protect any children. I think we should admit: Especially in light of the news on the same week this was announced about the doctor that was keeping lots of child porn in their iphotos and was caught by another service. The purpose of this scheme isn't to protect children, it isn't to preserve user privacy, it exists to increase and preserve Apple's profit margin by distancing them from creeps and it does so at the expense of the privacy of their users. Apple is converting their customer privacy into a private profit. ISTM the whole scheme may actually encourage abusing more children: The one reliable way to get sexual pictures of young people that this system cannot catch is to make new images. If this sort of thing were effective it could be expected to enlarge the commercial market for new abuse images by shifting demand from the back catalog of existing material to new material.