4 ms·

How does this theoretical attack get them to save the attached image to iCloud?

skoskie 5y ago
I think there's an unspoken assumption that photos are only the first part of this scanning exercise. IMO, Apple will likely end up scanning all content you choose to put on their servers. Tangentially, I think keeping their servers clear of illegal material is actually Apple's main motivation. This, in turn, supports claims made by nay-sayers that Apple could scan for other types of images/content in more repressive countries (but not necessarily report the people who did it). However, this assumption also contradicts arguments that Apple will start scanning for pictures of (e.g.) drugs, or weapons. Such images are not inherently illegal and therefore of no interest to Apple.
- misnome 5y ago
  > Tangentially, I think keeping their servers clear of illegal material is actually Apple's main motivation My guess about this whole debacle is that - with pressure from the government to scan their cloud storage - that this is the alternate scenario to avoid giving up (or being forced to) the "encryption" guarantees of their cloud. I'm not sure what technical process they have in place to "only decrypt with valid law enforcement requests" or allow account rescue, but it seems likely that not just any employee can view whatever they want, before or after this system. Saying that I can maybe see a way the pressures are on this doesn't mean that I'm saying this is a good solution though. Clearly technically implementing this is opening a can of worms that can't really be closed again and makes a lot of other scenarios "closer". Also, evidently, people are a lot more comfortable with the idea of them actively scanning stuff people store in the cloud than transmitting the information in a side channel so they don't even need to handle decrypted data without a hit.
  - skoskie 5y ago
    > My guess about this whole debacle is that - with pressure from the government to scan their cloud storage - that this is the alternate scenario to avoid giving up (or being forced to) the "encryption" guarantees of their cloud. This is exactly the case. https://www.eff.org/deeplinks/2019/12/senate-judiciary-committee-wants-everyone-know-its-concerned-about-encryption https://www.eff.org/deeplinks/2019/12/senate-judiciary-commi... > I'm not sure what technical process they have in place to "only decrypt with valid law enforcement requests" or allow account rescue, but it seems likely that not just any employee can view whatever they want, before or after this system. They have master keys that can be used to decrypt almost everything you upload. They can be compelled to decrypt and turn over information on anyone. Another (unsourced) comment in this thread indicated they do so 30,000 times per year. The new encryption scheme will effectively stop this for photos, and no doubt other files in the future. Apple's side will begin using shared key encryption, which will require ALL ~31 keys to decrypt the offending images. The decryption keys are only generated on-device, and only from a hash that results from a CSAM match. The other photos, simply won't have the decryption keys generated, so they don't even exist. As an interesting side note, this means that a person who surpasses the CSAM threshold will still only reveal the images that actually match the CSAM database. Every other image, including those that have CSAM unknown to authorities remain encrypted. This is hardly a big win for the big scary government. They now have far less ability to search for evidence of any other crimes. You could upload video of your bank robbery to iCloud, and as long as your personal device remains secure, nobody will know.
- dwaite 5y ago
  > IMO, Apple will likely end up scanning all content you choose to put on their servers. Even if that was a goal (and I would argue they have a hard stance against it), this system as built is not usable for that. While they can scan locally, every step of recording, thresholds, and subsequent automated/manual auditing is built to require content to be uploaded to iCloud Photos.
  - skoskie 5y ago
    I agree that the current photo scanning system won't work for other types of files. Further, I don't think they really even need to scan text files except to see if it's actually a filetype that should be scanned. And they can already easily scan music and such, but have shown no interest in it from a legal standpoint. Video seems like a prime candidate for the next generation of this tech. And I very much think they will be able to do that before the decade is over.
newbamboo 5y ago
iCloud is trivial to hack. Recall the massive leak of celebrity nudes a few years back. As long as iCloud is forced on users (you cannot apply security updates to an apple device without an Apple ID and an attached iCloud account) these attacks will be simple to do in mass, with very little risk to the perpetrators. Terrible security model, with a long history of spectacular and avoidable breaches which go totally unsolved.
- misnome 5y ago
  Let’s ignore the fact that you just shifted the goalposts from “emailing a 1 pixel image SWATs them” to an extremely wide scope for a minute - Even assuming that is true that iCloud is trivially “hackable” - and as I understand it, that was never clear how those leaks happened - how does uploading to iCloud help when it specifically needs to be uploaded from the users phone along with the scanning metadata. In fact, isn’t apples proposed implementation here the _only_ cloud service that protects against your proposed attack - while other clouds scan stored data and can be triggered by your attack, Apple’s requires you to upload specifically from a registered phone on their account; data stored on-cloud is never scanned.
  - newbamboo 5y ago
    You can choose to use an iPhone. Most people will never be targeted. Hell you could use a phone with no security at all and odds are you’ll be safe. But if you have enemies or are a high profile target, apple has made you easy to destroy. iPhones themselves can be hacked with no click attacks and we know this because macron’s phone was one of many that was listed in the recent fiasco. Icloud can be hacked because legions of celebrities had their nudes published. If the device is not secure, even for the president of France, and iCloud is not secure for legions if celebrities, then people can and will be hit with this attack and have their lives permanently destroyed. Worse than having you sex photos leaked, worse than having all you calls and communications intercepted, you’ll become known as a pedo. You can’t undo that. To be sure, that probably won’t happen to most people, but political figures, those with enemies working in infosec, etc…. I want a secure phone, and iPhone no longer meets that need for me and many others. You can do you.
AmericanChopper 5y ago
The described attack is completely unnecessary. Just send colliding images to people on WhatsApp. They go right into Photos, and then to iCloud if that’s enabled. There’s no reason most people would assume that this is what was happening if somebody sent them some slightly weird looking images. The response to this is “yeah but then a human will review it and nothing will happen to the victim of the attack, because it’s just some slightly blurry ordinary images”. But it ignores to entirely likely harms that could result from that. 1) Law enforcement use it as the basis of getting a search warrant, but conveniently leave the bit about the alerts being false alarms off the warrant application. 2) The list of people who have had CSAM alerts is inevitably leaked to the public, and the victim has to spend the rest of their lives explaining to people like employers why Apple flagged them as possessing child sexual abuse material. At the end of the day, all the gaslighting about “no potential for inadvertent harm” is bs, because it’s my device, so get lost. Go run your anti-privacy software somewhere else, imo.
- misnome 5y ago
  As I understand it; - Law enforcement doesn't get _anything_ unless it triggers a large number of images that match the perceptual hash - It also needs to match a -private- perceptual hash, that isn't distributed to devices, and so we don't have a reliable way of generating collisions for or even knowing that collisions are generated I mean this whole thing is bad enough on its own without having to artificially manufacture extremely specific scenarios and extrapolating from there to invent hysteric conclusions.
  - AmericanChopper 5y ago
    Well if you think a tech giant that’s partnered in a surveillance program with the NSA isn’t going to share anything they feel like with the government, then go ahead as use that as the basis of coming up with your own opinions. But you’re right though, the possibility of the list being leaked and ruining countless innocent lives is the much more likely of the two scenarios I described.
    - 5y ago

I think there's an unspoken assumption that photos are only the first part of this scanning exercise. IMO, Apple will likely end up scanning all content you choose to put on their servers. Tangentially, I think keeping their servers clear of illegal material is actually Apple's main motivation. This, in turn, supports claims made by nay-sayers that Apple could scan for other types of images/content in more repressive countries (but not necessarily report the people who did it). However, this assumption also contradicts arguments that Apple will start scanning for pictures of (e.g.) drugs, or weapons. Such images are not inherently illegal and therefore of no interest to Apple.

misnome 5y ago
> Tangentially, I think keeping their servers clear of illegal material is actually Apple's main motivation My guess about this whole debacle is that - with pressure from the government to scan their cloud storage - that this is the alternate scenario to avoid giving up (or being forced to) the "encryption" guarantees of their cloud. I'm not sure what technical process they have in place to "only decrypt with valid law enforcement requests" or allow account rescue, but it seems likely that not just any employee can view whatever they want, before or after this system. Saying that I can maybe see a way the pressures are on this doesn't mean that I'm saying this is a good solution though. Clearly technically implementing this is opening a can of worms that can't really be closed again and makes a lot of other scenarios "closer". Also, evidently, people are a lot more comfortable with the idea of them actively scanning stuff people store in the cloud than transmitting the information in a side channel so they don't even need to handle decrypted data without a hit.
- skoskie 5y ago
  > My guess about this whole debacle is that - with pressure from the government to scan their cloud storage - that this is the alternate scenario to avoid giving up (or being forced to) the "encryption" guarantees of their cloud. This is exactly the case. https://www.eff.org/deeplinks/2019/12/senate-judiciary-committee-wants-everyone-know-its-concerned-about-encryption https://www.eff.org/deeplinks/2019/12/senate-judiciary-commi... > I'm not sure what technical process they have in place to "only decrypt with valid law enforcement requests" or allow account rescue, but it seems likely that not just any employee can view whatever they want, before or after this system. They have master keys that can be used to decrypt almost everything you upload. They can be compelled to decrypt and turn over information on anyone. Another (unsourced) comment in this thread indicated they do so 30,000 times per year. The new encryption scheme will effectively stop this for photos, and no doubt other files in the future. Apple's side will begin using shared key encryption, which will require ALL ~31 keys to decrypt the offending images. The decryption keys are only generated on-device, and only from a hash that results from a CSAM match. The other photos, simply won't have the decryption keys generated, so they don't even exist. As an interesting side note, this means that a person who surpasses the CSAM threshold will still only reveal the images that actually match the CSAM database. Every other image, including those that have CSAM unknown to authorities remain encrypted. This is hardly a big win for the big scary government. They now have far less ability to search for evidence of any other crimes. You could upload video of your bank robbery to iCloud, and as long as your personal device remains secure, nobody will know.
dwaite 5y ago
> IMO, Apple will likely end up scanning all content you choose to put on their servers. Even if that was a goal (and I would argue they have a hard stance against it), this system as built is not usable for that. While they can scan locally, every step of recording, thresholds, and subsequent automated/manual auditing is built to require content to be uploaded to iCloud Photos.
- skoskie 5y ago
  I agree that the current photo scanning system won't work for other types of files. Further, I don't think they really even need to scan text files except to see if it's actually a filetype that should be scanned. And they can already easily scan music and such, but have shown no interest in it from a legal standpoint. Video seems like a prime candidate for the next generation of this tech. And I very much think they will be able to do that before the decade is over.

newbamboo 5y ago

iCloud is trivial to hack. Recall the massive leak of celebrity nudes a few years back. As long as iCloud is forced on users (you cannot apply security updates to an apple device without an Apple ID and an attached iCloud account) these attacks will be simple to do in mass, with very little risk to the perpetrators. Terrible security model, with a long history of spectacular and avoidable breaches which go totally unsolved.

misnome 5y ago
Let’s ignore the fact that you just shifted the goalposts from “emailing a 1 pixel image SWATs them” to an extremely wide scope for a minute - Even assuming that is true that iCloud is trivially “hackable” - and as I understand it, that was never clear how those leaks happened - how does uploading to iCloud help when it specifically needs to be uploaded from the users phone along with the scanning metadata. In fact, isn’t apples proposed implementation here the _only_ cloud service that protects against your proposed attack - while other clouds scan stored data and can be triggered by your attack, Apple’s requires you to upload specifically from a registered phone on their account; data stored on-cloud is never scanned.
- newbamboo 5y ago
  You can choose to use an iPhone. Most people will never be targeted. Hell you could use a phone with no security at all and odds are you’ll be safe. But if you have enemies or are a high profile target, apple has made you easy to destroy. iPhones themselves can be hacked with no click attacks and we know this because macron’s phone was one of many that was listed in the recent fiasco. Icloud can be hacked because legions of celebrities had their nudes published. If the device is not secure, even for the president of France, and iCloud is not secure for legions if celebrities, then people can and will be hit with this attack and have their lives permanently destroyed. Worse than having you sex photos leaked, worse than having all you calls and communications intercepted, you’ll become known as a pedo. You can’t undo that. To be sure, that probably won’t happen to most people, but political figures, those with enemies working in infosec, etc…. I want a secure phone, and iPhone no longer meets that need for me and many others. You can do you.

AmericanChopper 5y ago

The described attack is completely unnecessary. Just send colliding images to people on WhatsApp. They go right into Photos, and then to iCloud if that’s enabled. There’s no reason most people would assume that this is what was happening if somebody sent them some slightly weird looking images. The response to this is “yeah but then a human will review it and nothing will happen to the victim of the attack, because it’s just some slightly blurry ordinary images”. But it ignores to entirely likely harms that could result from that. 1) Law enforcement use it as the basis of getting a search warrant, but conveniently leave the bit about the alerts being false alarms off the warrant application. 2) The list of people who have had CSAM alerts is inevitably leaked to the public, and the victim has to spend the rest of their lives explaining to people like employers why Apple flagged them as possessing child sexual abuse material. At the end of the day, all the gaslighting about “no potential for inadvertent harm” is bs, because it’s my device, so get lost. Go run your anti-privacy software somewhere else, imo.

misnome 5y ago
As I understand it; - Law enforcement doesn't get _anything_ unless it triggers a large number of images that match the perceptual hash - It also needs to match a -private- perceptual hash, that isn't distributed to devices, and so we don't have a reliable way of generating collisions for or even knowing that collisions are generated I mean this whole thing is bad enough on its own without having to artificially manufacture extremely specific scenarios and extrapolating from there to invent hysteric conclusions.
- AmericanChopper 5y ago
  Well if you think a tech giant that’s partnered in a surveillance program with the NSA isn’t going to share anything they feel like with the government, then go ahead as use that as the basis of coming up with your own opinions. But you’re right though, the possibility of the list being leaked and ruining countless innocent lives is the much more likely of the two scenarios I described.
  - 5y ago