5 ms·

Can someone ELI5? I understand that a person can now generate an image with the same hash as an illegal image (such as child porn), but I don't understand how t

by magpi3 5y ago

Can someone ELI5? I understand that a person can now generate an image with the same hash as an illegal image (such as child porn), but I don't understand how they can get it on someone's phone and I don't understand why someone would get in trouble for an image, when finally examined, that is clearly not child pornography.

varispeed 5y ago
> that is clearly not For one, you can't know if that's true as the image could have been manipulated to appear as such. For example you wouldn't know if a kind of steganography has been used to hide image in an image and that neuralhash picked on a hidden image. > but I don't understand how they can get it on someone's phone There is many vectors. For example you can leave phone unattended and someone can snap a picture of an image or since a collision may look innocent to you, you would overlook it in an email etc...
- mukesh610 5y ago
  > For example you wouldn't know if a kind of steganography has been used to hide image in an image and that neuralhash picked on a hidden image. How would NeuralHash pick a "hidden image"? It only uses the pixels of the image to get the hash. Any hidden image in the metadata would not even be picked up and no amount of steganography can fool NeuralHash. > There is many vectors. For example you can leave phone unattended and someone can snap a picture of an image or since a collision may look innocent to you, you would overlook it in an email etc... As iterated elsewhere in this thread, random gibberish pixels colliding with CSAM would definitely not be useful in incriminating anyone. The manual process would catch that. Also, if the manual process is overloaded, I'm pretty sure basic object recognition can filter out most of the colliding gibberish .
  - varispeed 5y ago
    The steganography is not to fool the NeuralHash, but to fool the viewer. The NeuralHash would "see" the planted image, but for the viewer it would appear innocent. I am trying to say that a person reviewing image manually, without special tools will not be able to tell if the image is a false positive and would have to report everything.
vnchr 5y ago
I would think a Message with the attached photo from a burner phone/account would be enough.
- manmal 5y ago
  Currently, the image would have to be imported into the photos library, and iCloud upload must be enabled.
  - zug_zug 5y ago
    This conversely means that all illegal content can be freely texted and this system won't even catch the distribution of CP unless those pictures are imported into the photos library and icloud updates enabled. There's a pretty good chance that it was inevitably going to get expanded to handle pictures arriving at the phone through other means.
    - cyanite 5y ago
      We can take that discussion if and when that happens.
  - Invictus0 5y ago
    Whatsapp has a feature where all images are automatically saved to device as they are received. If automatic iCloud upload is on, then all the conditions are there for a person to innocently click on a spammy Whatsapp message, see a bunch of nonsense grayscale images, and continue on with their day--not realizing they are now being monitored for CSAM.
    - manmal 5y ago
      I agree, that will become a problem.
  - dannyw 5y ago
    Pegasus says hi.
bitneuker 5y ago
A vulnerability by itself is not that dangerous, but in combination with a sophisticated attack, or another vulnerability can be disastrous. State actors have the resources to exploit a number of unknown bugs in combination with this collision to have Apple's systems flag persons of interest. This, combined with human error during the manual review process might result in someone getting reported. Seeing as twitter (and other social media sites) jump on the bandwagon whenever someone gets accused of being a pedophile, this might destroy someones life. The entire story might seem a bit to far fetched, but based on past events, you never know how bad something 'simple' as a hash collision can be.
- rootusrootus 5y ago
  > State actors have the resources You can end the conversation right there. If you are up against a state actor, you have already lost.
  - bitneuker 5y ago
    Very true, just wanted to paint a picture on how this could be abused.
    - dannyw 5y ago
      It's not true at all. You're assuming state actors are in the same jurisdiction. This isn't always the case - think an oppressive authoritian regime wanting to get an American journalist arrested for child pornography. It's always possible before, but client-side CSAM detection and alerting has weaponised this. Previously, you always had to somehow alert an unfriendly jurisdiction. Now, you just use malware like Pegasus to drop CSAM, whether real or disturbed from legal porn, and watch as Apple tips off the Feds on your enemies.
  - dannyw 5y ago
    Incorrect. A Chinese state actor can't just go around imprisoning journalists they don't like in America, but they can now do this through planting child porngoraphy via remote malware (Pegasus) and watch their enemies get arrested by the US Feds.

by magpi3 5y ago

varispeed 5y ago

> that is clearly not For one, you can't know if that's true as the image could have been manipulated to appear as such. For example you wouldn't know if a kind of steganography has been used to hide image in an image and that neuralhash picked on a hidden image. > but I don't understand how they can get it on someone's phone There is many vectors. For example you can leave phone unattended and someone can snap a picture of an image or since a collision may look innocent to you, you would overlook it in an email etc...

mukesh610 5y ago
> For example you wouldn't know if a kind of steganography has been used to hide image in an image and that neuralhash picked on a hidden image. How would NeuralHash pick a "hidden image"? It only uses the pixels of the image to get the hash. Any hidden image in the metadata would not even be picked up and no amount of steganography can fool NeuralHash. > There is many vectors. For example you can leave phone unattended and someone can snap a picture of an image or since a collision may look innocent to you, you would overlook it in an email etc... As iterated elsewhere in this thread, random gibberish pixels colliding with CSAM would definitely not be useful in incriminating anyone. The manual process would catch that. Also, if the manual process is overloaded, I'm pretty sure basic object recognition can filter out most of the colliding gibberish .
- varispeed 5y ago
  The steganography is not to fool the NeuralHash, but to fool the viewer. The NeuralHash would "see" the planted image, but for the viewer it would appear innocent. I am trying to say that a person reviewing image manually, without special tools will not be able to tell if the image is a false positive and would have to report everything.

vnchr 5y ago

I would think a Message with the attached photo from a burner phone/account would be enough.

manmal 5y ago
Currently, the image would have to be imported into the photos library, and iCloud upload must be enabled.
- zug_zug 5y ago
  This conversely means that all illegal content can be freely texted and this system won't even catch the distribution of CP unless those pictures are imported into the photos library and icloud updates enabled. There's a pretty good chance that it was inevitably going to get expanded to handle pictures arriving at the phone through other means.
  - cyanite 5y ago
    We can take that discussion if and when that happens.
- Invictus0 5y ago
  Whatsapp has a feature where all images are automatically saved to device as they are received. If automatic iCloud upload is on, then all the conditions are there for a person to innocently click on a spammy Whatsapp message, see a bunch of nonsense grayscale images, and continue on with their day--not realizing they are now being monitored for CSAM.
  - manmal 5y ago
    I agree, that will become a problem.
- dannyw 5y ago
  Pegasus says hi.

bitneuker 5y ago

A vulnerability by itself is not that dangerous, but in combination with a sophisticated attack, or another vulnerability can be disastrous. State actors have the resources to exploit a number of unknown bugs in combination with this collision to have Apple's systems flag persons of interest. This, combined with human error during the manual review process might result in someone getting reported. Seeing as twitter (and other social media sites) jump on the bandwagon whenever someone gets accused of being a pedophile, this might destroy someones life. The entire story might seem a bit to far fetched, but based on past events, you never know how bad something 'simple' as a hash collision can be.

rootusrootus 5y ago
> State actors have the resources You can end the conversation right there. If you are up against a state actor, you have already lost.
- bitneuker 5y ago
  Very true, just wanted to paint a picture on how this could be abused.
  - dannyw 5y ago
    It's not true at all. You're assuming state actors are in the same jurisdiction. This isn't always the case - think an oppressive authoritian regime wanting to get an American journalist arrested for child pornography. It's always possible before, but client-side CSAM detection and alerting has weaponised this. Previously, you always had to somehow alert an unfriendly jurisdiction. Now, you just use malware like Pegasus to drop CSAM, whether real or disturbed from legal porn, and watch as Apple tips off the Feds on your enemies.
- dannyw 5y ago
  Incorrect. A Chinese state actor can't just go around imprisoning journalists they don't like in America, but they can now do this through planting child porngoraphy via remote malware (Pegasus) and watch their enemies get arrested by the US Feds.