5 ms·

The reported response from Apple offers little reassurance: > The executives acknowledged that a user could be implicated by malicious actors who win control o

The reported response from Apple offers little reassurance: > The executives acknowledged that a user could be implicated by malicious actors who win control of a device and remotely install known child abuse material. But they said they expected any such attacks to be very rare and that in any case a review would then look for other signs of criminal hacking. What triggers them to look for signs of criminal hacking? Does every manual review process involve such checks? Are they searching device backups for indicators of compromise [IoC]? What if there's no device backup or device image to scan? What if the scan fails to notice IoC? What if the device was compromised after the last backup? What if the device was compromised via physical access? What if the device isn't compromised and the material was pushed maliciously or via drive-by download? It's dangerous to assume that all material on a network-connected device arrived with the consent of the user when it can accept incoming messages from strangers, trick people into downloading files, or be compromised without your knowledge. “That isn't mine” is going to be a tough defence if you can't even take measures to log where content came from. Client-side scanning seems to amplify this issue (which could still happen with cloud storage) because at least cloud storage doesn't generally ship with or integrate deeply with messaging apps, social media, a web browser, QR codes, App Clip Codes[1] etc. The impact might be fairly low right now with the current proposal (images would have to be uploaded to iCloud, so cached browser images don't get scanned as far as we know), but the existence of the non-consensual scan in the first place is worrying, because it means such attacks are only a policy change away. [1] : https://developer.apple.com/design/human-interface-guidelines/app-clips/overview/app-clip-codes/ https://developer.apple.com/design/human-interface-guideline...

GeekyBear 5y ago
> The executives acknowledged that a user could be implicated by malicious actors who win control of a device and remotely install known child abuse material. Since Google has been scanning your account for kiddie porn for the past decade, wouldn't this apply equally to Google accounts? >a man [was] arrested on child pornography charges, after Google tipped off authorities about illegal images found in the Houston suspect's Gmail account https://techcrunch.com/2014/08/06/why-the-gmail-scan-that-led-to-a-mans-arrest-for-child-porn-was-not-a-privacy-violation/ https://techcrunch.com/2014/08/06/why-the-gmail-scan-that-le... All people have to do is email you kiddie porn and Google will have you arrested?
- ipaddr 5y ago
  I thought google wasn't scanning your emails anymore.
  - GeekyBear 5y ago
    Wasn't that claim limited to children's accounts? Since Google is saving a history of what you purchase from third party merchants by scraping invoices and receipts sent to you through your Gmail account, it's safe to say that they are scanning your emails. https://news.ycombinator.com/item?id=26248486 https://news.ycombinator.com/item?id=26248486
  - jxdxbx 5y ago
    Google scans all photos in the cloud (Gmail, Drive, Google Photos) for CSAM and has for a long time. It just doesn't show contextual ads against email anymore, since those all sucked.
- modernerd 5y ago
  No, but the person who sent that message could get in trouble. In the case you linked to the person was reported for sending email to a friend with attached CSAM, not for receiving it.[1] Apple's system scans images client-side if they're due to be uploaded to iCloud. That process can happen without user consent or action. For example, WhatsApp and other messaging apps save images to photos, which are auto-synced to iCloud. (If you use WhatsApp and iCloud you'll find your Photos section full of memes from WhatsApp group chats when you log in at icloud.com, for example. This was a surprise to me at first.) So the risk of malice seems higher with Apple's system than with the long-running PhotoDNA implementations backing Gmail/Google Drive/OneDrive etc. Gaining access to someone's email and sending attached CSAM is likely to cause them more issues than receiving it. But that's harder because you need their login info and not just their email address/phone number, which is all that an attacker potentially requires to trigger action from Apple's automated scans. [1]: https://nakedsecurity.sophos.com/2014/07/31/google-tips-off-cops-after-spotting-child-abuse-images-in-email/ https://nakedsecurity.sophos.com/2014/07/31/google-tips-off-... > The investigation was apparently sparked by a tip-off sent by Google to the National Center for Missing and Exploited Children, after explicit images of a child were detected in an email he was sending.
artificialLimbs 5y ago
>> they expected any such attacks to be very rare Very rarely will your life be completely ruined based on inaccurate information.
fortran77 5y ago
> “That isn't mine” is going to be a tough defence if you can't even take measures to log where content came from. It's not a defense at all. This material is prosecuted under a "strict liability." It doesn't matter how you got it, you're liable.
- 35fbe7d3d5b9 5y ago
  > This material is prosecuted under a "strict liability." It doesn't matter how you got it, you're liable. You're overselling it. First, there is a statutory affirmative defense: if I obtain CSAM and "promptly and in good faith" delete it or report what happened to law enforcement, liability does not attach. Additionally, federal laws are clear that you have to knowingly receive CSAM. That's not just a legal flourish or a word – knowledge is an element that a jury or judge will rule on. If I ask you to send me an illegal video and you do, we've both knowingly violated federal law. If you send me to a webpage that purports to offer me a job, but actually has images hidden with CSS to poison my cache, I've not knowingly received anything.
  - vineyardmike 5y ago
    > knowledge is an element that a jury or judge will rule on. And yet, i never want to be in this court case at all.

GeekyBear 5y ago

> The executives acknowledged that a user could be implicated by malicious actors who win control of a device and remotely install known child abuse material. Since Google has been scanning your account for kiddie porn for the past decade, wouldn't this apply equally to Google accounts? >a man [was] arrested on child pornography charges, after Google tipped off authorities about illegal images found in the Houston suspect's Gmail account https://techcrunch.com/2014/08/06/why-the-gmail-scan-that-led-to-a-mans-arrest-for-child-porn-was-not-a-privacy-violation/ https://techcrunch.com/2014/08/06/why-the-gmail-scan-that-le... All people have to do is email you kiddie porn and Google will have you arrested?

ipaddr 5y ago
I thought google wasn't scanning your emails anymore.
- GeekyBear 5y ago
  Wasn't that claim limited to children's accounts? Since Google is saving a history of what you purchase from third party merchants by scraping invoices and receipts sent to you through your Gmail account, it's safe to say that they are scanning your emails. https://news.ycombinator.com/item?id=26248486 https://news.ycombinator.com/item?id=26248486
- jxdxbx 5y ago
  Google scans all photos in the cloud (Gmail, Drive, Google Photos) for CSAM and has for a long time. It just doesn't show contextual ads against email anymore, since those all sucked.
modernerd 5y ago
No, but the person who sent that message could get in trouble. In the case you linked to the person was reported for sending email to a friend with attached CSAM, not for receiving it.[1] Apple's system scans images client-side if they're due to be uploaded to iCloud. That process can happen without user consent or action. For example, WhatsApp and other messaging apps save images to photos, which are auto-synced to iCloud. (If you use WhatsApp and iCloud you'll find your Photos section full of memes from WhatsApp group chats when you log in at icloud.com, for example. This was a surprise to me at first.) So the risk of malice seems higher with Apple's system than with the long-running PhotoDNA implementations backing Gmail/Google Drive/OneDrive etc. Gaining access to someone's email and sending attached CSAM is likely to cause them more issues than receiving it. But that's harder because you need their login info and not just their email address/phone number, which is all that an attacker potentially requires to trigger action from Apple's automated scans. [1]: https://nakedsecurity.sophos.com/2014/07/31/google-tips-off-cops-after-spotting-child-abuse-images-in-email/ https://nakedsecurity.sophos.com/2014/07/31/google-tips-off-... > The investigation was apparently sparked by a tip-off sent by Google to the National Center for Missing and Exploited Children, after explicit images of a child were detected in an email he was sending.

artificialLimbs 5y ago

>> they expected any such attacks to be very rare Very rarely will your life be completely ruined based on inaccurate information.

fortran77 5y ago

> “That isn't mine” is going to be a tough defence if you can't even take measures to log where content came from. It's not a defense at all. This material is prosecuted under a "strict liability." It doesn't matter how you got it, you're liable.

35fbe7d3d5b9 5y ago
> This material is prosecuted under a "strict liability." It doesn't matter how you got it, you're liable. You're overselling it. First, there is a statutory affirmative defense: if I obtain CSAM and "promptly and in good faith" delete it or report what happened to law enforcement, liability does not attach. Additionally, federal laws are clear that you have to knowingly receive CSAM. That's not just a legal flourish or a word – knowledge is an element that a jury or judge will rule on. If I ask you to send me an illegal video and you do, we've both knowingly violated federal law. If you send me to a webpage that purports to offer me a job, but actually has images hidden with CSS to poison my cache, I've not knowingly received anything.
- vineyardmike 5y ago
  > knowledge is an element that a jury or judge will rule on. And yet, i never want to be in this court case at all.