5 ms·

A vulnerability by itself is not that dangerous, but in combination with a sophisticated attack, or another vulnerability can be disastrous. State actors have t

A vulnerability by itself is not that dangerous, but in combination with a sophisticated attack, or another vulnerability can be disastrous. State actors have the resources to exploit a number of unknown bugs in combination with this collision to have Apple's systems flag persons of interest. This, combined with human error during the manual review process might result in someone getting reported. Seeing as twitter (and other social media sites) jump on the bandwagon whenever someone gets accused of being a pedophile, this might destroy someones life. The entire story might seem a bit to far fetched, but based on past events, you never know how bad something 'simple' as a hash collision can be.

rootusrootus 5y ago
> State actors have the resources You can end the conversation right there. If you are up against a state actor, you have already lost.
- bitneuker 5y ago
  Very true, just wanted to paint a picture on how this could be abused.
  - dannyw 5y ago
    It's not true at all. You're assuming state actors are in the same jurisdiction. This isn't always the case - think an oppressive authoritian regime wanting to get an American journalist arrested for child pornography. It's always possible before, but client-side CSAM detection and alerting has weaponised this. Previously, you always had to somehow alert an unfriendly jurisdiction. Now, you just use malware like Pegasus to drop CSAM, whether real or disturbed from legal porn, and watch as Apple tips off the Feds on your enemies.
- dannyw 5y ago
  Incorrect. A Chinese state actor can't just go around imprisoning journalists they don't like in America, but they can now do this through planting child porngoraphy via remote malware (Pegasus) and watch their enemies get arrested by the US Feds.
  - rootusrootus 5y ago
    Disagree. It isn't just jurisdiction. It is resource access. If the Chinese gov't were coming after little old me right now, I'd be properly worried, even though I'm safely within the boundaries of the US.
floatingatoll 5y ago
State actors will just Gitmo you, without all this wasteful effort on hashes. This system offers no benefit sufficient to make it worth their time if they want to cull you from the population somehow.
- dannyw 5y ago
  No, China can't just Gitmo an American journalist on American soil. But now China can send some legal pornography (eg closeup pussy pictures), disturbed to match a CSAM hit, to a journalist they don't like and get them in jail. Why can't China do this before? Because previously, they'd still need to tip off authorities, which has an attribution trail and credibility barrier. Now, they can just use Pegasus to plant these images and then watch as Apple turns them into the Feds. Zero links to the attacker.
  - floatingatoll 5y ago
    The scenario you describe has already been extant for the past ten years. Unreported zerodays could have been used at any time to inject a CSAM hit into someone's camera roll, way back in time where they wouldn't see it, in order to get them investigated. Their phone would have uploaded it to iCloud or Google Photos or Dropbox Whatever and the CSAM detections at each place would have fired off. No need for any of this fancy AI static nonsense. I know of zero instances of this attack being executed on anyone, so apparently even though it's been possible for years, it isn't a material threat to any Apple customers today. If you have information to the contrary, please present it. What new attacks are possible upon device owners when the CSAM scanning of iCloud uploads is shifted to the device, that were not already a viable attack at any time in the past decade?
dannyw 5y ago
No one is going to send gray blobs, they will be finding legal porn (like pussy close ups, tongue pics, whatever) and then disturbing it to trigger a CSAM hit. The low res derivative will match, perhaps even closely, because pussy closeups look similar to an apple employee when its grayscale 64 by 64 pixels (remember: it's illegal for Apple to transmit CSAM, so it must be so visually degraded to the point where it's arguably not visual). The victim will get raided, be considered a paedophile by their workplace, media, and family, and perhaps even go into jail. The attacker in this case can be users of Pegasus unhappy with a journalist.
- cyanite 5y ago
  There is a lot of speculation about things that haven’t happened in that comment.
- FabHK 5y ago
  Ok, so you posit an attacker could find/generate 30+ pictures that are 1. accepted by innocent user, 2. flagged as known CSAM by NeuralHash, 2b. also flagged by the second algorithm Apple will run over flagged images server side as known CSAM, 3. apparently CSAM in the "visual derivative". That strikes me as a rather remote scenario, but worth investigating. Having said that, if it's a 3-letter adversary using Pegasus unhappy with a journalist, couldn't they just put actual CSAM onto the journalist's phone? And couldn't they have done that for many years?

rootusrootus 5y ago

> State actors have the resources You can end the conversation right there. If you are up against a state actor, you have already lost.

bitneuker 5y ago
Very true, just wanted to paint a picture on how this could be abused.
- dannyw 5y ago
  It's not true at all. You're assuming state actors are in the same jurisdiction. This isn't always the case - think an oppressive authoritian regime wanting to get an American journalist arrested for child pornography. It's always possible before, but client-side CSAM detection and alerting has weaponised this. Previously, you always had to somehow alert an unfriendly jurisdiction. Now, you just use malware like Pegasus to drop CSAM, whether real or disturbed from legal porn, and watch as Apple tips off the Feds on your enemies.
dannyw 5y ago
Incorrect. A Chinese state actor can't just go around imprisoning journalists they don't like in America, but they can now do this through planting child porngoraphy via remote malware (Pegasus) and watch their enemies get arrested by the US Feds.
- rootusrootus 5y ago
  Disagree. It isn't just jurisdiction. It is resource access. If the Chinese gov't were coming after little old me right now, I'd be properly worried, even though I'm safely within the boundaries of the US.

floatingatoll 5y ago

State actors will just Gitmo you, without all this wasteful effort on hashes. This system offers no benefit sufficient to make it worth their time if they want to cull you from the population somehow.

dannyw 5y ago
No, China can't just Gitmo an American journalist on American soil. But now China can send some legal pornography (eg closeup pussy pictures), disturbed to match a CSAM hit, to a journalist they don't like and get them in jail. Why can't China do this before? Because previously, they'd still need to tip off authorities, which has an attribution trail and credibility barrier. Now, they can just use Pegasus to plant these images and then watch as Apple turns them into the Feds. Zero links to the attacker.
- floatingatoll 5y ago
  The scenario you describe has already been extant for the past ten years. Unreported zerodays could have been used at any time to inject a CSAM hit into someone's camera roll, way back in time where they wouldn't see it, in order to get them investigated. Their phone would have uploaded it to iCloud or Google Photos or Dropbox Whatever and the CSAM detections at each place would have fired off. No need for any of this fancy AI static nonsense. I know of zero instances of this attack being executed on anyone, so apparently even though it's been possible for years, it isn't a material threat to any Apple customers today. If you have information to the contrary, please present it. What new attacks are possible upon device owners when the CSAM scanning of iCloud uploads is shifted to the device, that were not already a viable attack at any time in the past decade?

dannyw 5y ago

No one is going to send gray blobs, they will be finding legal porn (like pussy close ups, tongue pics, whatever) and then disturbing it to trigger a CSAM hit. The low res derivative will match, perhaps even closely, because pussy closeups look similar to an apple employee when its grayscale 64 by 64 pixels (remember: it's illegal for Apple to transmit CSAM, so it must be so visually degraded to the point where it's arguably not visual). The victim will get raided, be considered a paedophile by their workplace, media, and family, and perhaps even go into jail. The attacker in this case can be users of Pegasus unhappy with a journalist.

cyanite 5y ago
There is a lot of speculation about things that haven’t happened in that comment.
FabHK 5y ago
Ok, so you posit an attacker could find/generate 30+ pictures that are 1. accepted by innocent user, 2. flagged as known CSAM by NeuralHash, 2b. also flagged by the second algorithm Apple will run over flagged images server side as known CSAM, 3. apparently CSAM in the "visual derivative". That strikes me as a rather remote scenario, but worth investigating. Having said that, if it's a 3-letter adversary using Pegasus unhappy with a journalist, couldn't they just put actual CSAM onto the journalist's phone? And couldn't they have done that for many years?