5 ms·

NeuralHash collisions are interesting, but the way Apple is implementing their scanner it's impossible to extract the banned hashes directly from the local data

NeuralHash collisions are interesting, but the way Apple is implementing their scanner it's impossible to extract the banned hashes directly from the local database. There are other ways to guess what the hashes are, but I can't think of legal ones. > Matching-Database Setup. The system begins by setting up the matching database using the known CSAM image hashes provided by NCMEC and other child-safety organizations. First, Apple receives the NeuralHashes corresponding to known CSAM from the above child-safety organizations. Next, these NeuralHashes go through a series of transformations that includes a final blinding step, powered by elliptic curve cryptography. The blinding is done using a server-side blinding secret, known only to Apple. The blinded CSAM hashes are placed in a hash table, where the position in the hash table is purely a function of the NeuralHash of the CSAM image. This blinded database is securely stored on users’ devices. The properties of elliptic curve cryptography ensure that no device can infer anything about the underlying CSAM image hashes from the blinded database. https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

dannyw 5y ago
You can extract the hashes with a few hours spent on the darknet. Doing that is certainly illegal and not to mention VERY morally wrong, but criminals exist and criminals won't hesitate to abuse this as a mechanism for framing, extortion, or ransom. It's also possible for someone (Attacker A) to go on the darknet and get a list of 96-bit neural hashes, and then publish or sell this list somewhere to another party, Attacker B. The second party would never have to interact with CSAM. Imagine Ransomware v2: We have inserted 29 photos of CSAM-matching material into your photo library. Pay X monero to this address in 30 minutes, or we will insert 2 additional photos, which will cross the threshold and may result in serious and life-changing consequences to you[1]. The difference here (versus the status quo) is that an easily-broken perceptual hashing enables the attacker to never send or possess any CSAM images[2]. From my experiences with being victims of various hackers, I know a lot of them won't touch CSAM because they know it's wrong, but they'll salivate at an opportunity to weaponise automated CSAM scanning. [1]: If you think Apple's human review will mitigate this attack, you can permute legal pornography to match CSAM signatures. If Apple's reviewers see 30 CSAM matches and the visual derivatives look like porn, they will be legally required to report to to NCMEC (a statutory quasi-government agency staffed by the FBI), even if all the photos are actually consensual adults. [2]: If you never possess nor touch CSAM, it might be harder for you to get charged with CP charges. You might be looking at CFAA, blackmail or extortion charges; while your victim faces child pornography charges. This is basically an "amplification attack" on the real world judicial system.
- d0100 5y ago
  > If Apple's reviewers see 30 CSAM matches and the visual derivatives look like porn Even worse, just get a "teen" porn screengrab, pass it through the collider and you have pretty much a smoking gun
  - XorNot 5y ago
    The "visual derivative" is not something any of us have been shown an example of either. Whatever it is, I suspect you only need to be vaguely in the same ballpark (I would wager humanoid shaped skin tones maybe). So I suspect it would be easier then that (particularly since this whole hashing scheme has been surrounded with a lot of clear garbage - "1 in a trillion" -> on demand collisions in a couple of weeks?
    - ComputerGuru 5y ago
      I think visual derivative is just a beating-around-the-bush way of saying “thumbnail.”
      - XorNot 5y ago
        That's their words that they feel no need to elaborate on. Obviously they actually seem to just be doing the "technically the truth" thing - which shows that someone realized no one would like hearing what it actually is.
        ComputerGuru 5y ago
        Yeah, it was very pointedly awkwardly worded. It’s intended for human reviewers to distinguish a false positive from a real positive. An eigen vector mapped image isn’t going to do that, a heavily Gaussian-blurred image isn’t going to do that - it needs to be something that a minimum wage person who’s only been trained a day or to can distinguish as “CSAM” or “not CSAM” and that means it’s a thumbnail of sorts.
- Scaevolus 5y ago
bArray 5y ago
To defeat this, all you need to be is a state actor with a database of child porn at your disposal (which is stored for exactly the purpose of training detection systems). Then you run the hashing algorithm against images you know are in the database (Apple suggested that they would accept suggestions by some kind of multi-Country vote). Then you can pull out the hashes and figure out how to trigger false positives on the important vectors. Next, embed your images in sites of interest, like: * A meme in some group * A document or 'leak' * An email to a journalist Wait for somebody to save it to their Apply device. Wait for it to be flagged and then use that as 'reasonable means to conduct a search'. When asking for a warrant, the agency would say something like "we detected possible CSAM on a device, the likelihood of a false match is extremely low" - a judge will hardly press further. You now essentially have a weapon where you can search any Apple device in the name of preventing the distribution of CSAM. Failing that, you could just have `document_leak.pdf` and download a file that is both a valid PDF and a child porn image, depending on which program you open it with.
- godelski 5y ago
  There's already a problem that Apple can't verify the hashes. Say a government wants to investigate a certain set of people. Those people probably share specific memes and photos. Add those hashes to the list and now you have reasonable cause to investigate these people. Honestly this even adds to the danger of hash collisions because now you can get someone on a terrorist watch list as well as the kiddy porn list.
  - ec109685 5y ago
    Apple is the one doing the first line of investigation.
- ec109685 5y ago
  This doesn’t work for two reasons: 1) There’s no way to know the perceptual hash value of Apple’s private NeuralHash function that is run on the derivative of the image server side to verify a hit really is CSAM. So while you could cause a collision with the on device neural hash if you possessed illegal content, you wouldn’t know if you successfully faked Apple’s private neuralhash implementation. 2) An Apple reviewer must verify the image is illegal before it’s passed along to law enforcement.
jazzyjackson 5y ago
is this an example of homomorphic encryption? checking for hashes in a 'blinded' table I mean.

dannyw 5y ago

You can extract the hashes with a few hours spent on the darknet. Doing that is certainly illegal and not to mention VERY morally wrong, but criminals exist and criminals won't hesitate to abuse this as a mechanism for framing, extortion, or ransom. It's also possible for someone (Attacker A) to go on the darknet and get a list of 96-bit neural hashes, and then publish or sell this list somewhere to another party, Attacker B. The second party would never have to interact with CSAM. Imagine Ransomware v2: We have inserted 29 photos of CSAM-matching material into your photo library. Pay X monero to this address in 30 minutes, or we will insert 2 additional photos, which will cross the threshold and may result in serious and life-changing consequences to you[1]. The difference here (versus the status quo) is that an easily-broken perceptual hashing enables the attacker to never send or possess any CSAM images[2]. From my experiences with being victims of various hackers, I know a lot of them won't touch CSAM because they know it's wrong, but they'll salivate at an opportunity to weaponise automated CSAM scanning. [1]: If you think Apple's human review will mitigate this attack, you can permute legal pornography to match CSAM signatures. If Apple's reviewers see 30 CSAM matches and the visual derivatives look like porn, they will be legally required to report to to NCMEC (a statutory quasi-government agency staffed by the FBI), even if all the photos are actually consensual adults. [2]: If you never possess nor touch CSAM, it might be harder for you to get charged with CP charges. You might be looking at CFAA, blackmail or extortion charges; while your victim faces child pornography charges. This is basically an "amplification attack" on the real world judicial system.

d0100 5y ago
> If Apple's reviewers see 30 CSAM matches and the visual derivatives look like porn Even worse, just get a "teen" porn screengrab, pass it through the collider and you have pretty much a smoking gun
- XorNot 5y ago
  The "visual derivative" is not something any of us have been shown an example of either. Whatever it is, I suspect you only need to be vaguely in the same ballpark (I would wager humanoid shaped skin tones maybe). So I suspect it would be easier then that (particularly since this whole hashing scheme has been surrounded with a lot of clear garbage - "1 in a trillion" -> on demand collisions in a couple of weeks?
  - ComputerGuru 5y ago
    I think visual derivative is just a beating-around-the-bush way of saying “thumbnail.”
    - XorNot 5y ago
      That's their words that they feel no need to elaborate on. Obviously they actually seem to just be doing the "technically the truth" thing - which shows that someone realized no one would like hearing what it actually is.
      - ComputerGuru 5y ago
        Yeah, it was very pointedly awkwardly worded. It’s intended for human reviewers to distinguish a false positive from a real positive. An eigen vector mapped image isn’t going to do that, a heavily Gaussian-blurred image isn’t going to do that - it needs to be something that a minimum wage person who’s only been trained a day or to can distinguish as “CSAM” or “not CSAM” and that means it’s a thumbnail of sorts.
Scaevolus 5y ago

bArray 5y ago

To defeat this, all you need to be is a state actor with a database of child porn at your disposal (which is stored for exactly the purpose of training detection systems). Then you run the hashing algorithm against images you know are in the database (Apple suggested that they would accept suggestions by some kind of multi-Country vote). Then you can pull out the hashes and figure out how to trigger false positives on the important vectors. Next, embed your images in sites of interest, like: * A meme in some group * A document or 'leak' * An email to a journalist Wait for somebody to save it to their Apply device. Wait for it to be flagged and then use that as 'reasonable means to conduct a search'. When asking for a warrant, the agency would say something like "we detected possible CSAM on a device, the likelihood of a false match is extremely low" - a judge will hardly press further. You now essentially have a weapon where you can search any Apple device in the name of preventing the distribution of CSAM. Failing that, you could just have `document_leak.pdf` and download a file that is both a valid PDF and a child porn image, depending on which program you open it with.

godelski 5y ago
There's already a problem that Apple can't verify the hashes. Say a government wants to investigate a certain set of people. Those people probably share specific memes and photos. Add those hashes to the list and now you have reasonable cause to investigate these people. Honestly this even adds to the danger of hash collisions because now you can get someone on a terrorist watch list as well as the kiddy porn list.
- ec109685 5y ago
  Apple is the one doing the first line of investigation.
ec109685 5y ago
This doesn’t work for two reasons: 1) There’s no way to know the perceptual hash value of Apple’s private NeuralHash function that is run on the derivative of the image server side to verify a hit really is CSAM. So while you could cause a collision with the on device neural hash if you possessed illegal content, you wouldn’t know if you successfully faked Apple’s private neuralhash implementation. 2) An Apple reviewer must verify the image is illegal before it’s passed along to law enforcement.

jazzyjackson 5y ago

is this an example of homomorphic encryption? checking for hashes in a 'blinded' table I mean.