4 ms·

Having read the coverage and the documents that Apple has been putting out to manage the … what, crisis? Backlash? … I’ve been thinking about the design of the

by bnj 5y ago

Having read the coverage and the documents that Apple has been putting out to manage the … what, crisis? Backlash? … I’ve been thinking about the design of the system and how I would want CSAM to be detected, if different than this. Honestly, I’m pretty impressed by what this approach accomplishes. Your comment says this obviously isn’t the way to do it— what’s better than this? In a legal context where detecting CSAM is a strong requirement, what’s preferable to this approach?

patmcc 5y ago
>>>In a legal context where detecting CSAM is a strong requirement, what’s preferable to this approach? It's not a strong requirement though? Only if the company sees it (unencrypted) do they need to report it, detecting it on-device is wholly different. It would honestly be better if they adjusted their TOS and started scanning the files on upload (on server side). They have the encryption keys already. Apple is planning to install on all (recent) iPhones software that will let them scan for any image similar (using perceptual hashing) to some set of images. Right now, that's CSAM, and only on upload to iCloud. But what do you want to bet China is salivating at the idea of including images of Tank Man and other "seditious" content? And maybe checking images anytime they're added to the phone, or sent to someone else, not just up to iCloud? It completely ruins the idea that Apple has your privacy/security in mind.
- nonbirithm 5y ago
  Even if scanning for the content is not explicitly required by law, what happens when a pedophile ring hoarding thousands of images of CSAM on iCloud is busted, and the news gets out? The article at [1] makes it sound like Apple choosing not to scan any of its users' videos in iCloud was seen as evidence of Apple lagging behind the status quo of companies like Facebook that were proactively reporting CSAM. According to that article, in the last year Apple only submitted 265 reports to the NCEMC while Facebook submitted 20 million. Would law enforcement believe they'd be missing out on catching abusers after seeing this disparity? If a company is found to allow criminals to store CSAM on their servers for extended periods of time, the law is going to want to know why they let it pass, irrespective of the extent the company chose to scan for it. Apple probably doesn't want to deal with that fallout, so maybe they figured that being proactive about scanning for CSAM in a way that could enable the use of E2EE wouldn't hurt, and that pushing the privacy narrative would satisfy enough people - which it didn't. [1] https://www.nytimes.com/2021/08/05/technology/apple-iphones-privacy.html https://www.nytimes.com/2021/08/05/technology/apple-iphones-...
  - patmcc 5y ago
    >>>If a company is found to allow criminals to store CSAM on their servers I'd bet dollars to doughnuts that AWS, GCP, Azure, et al. have terabytes of CSAM stored within their data centers - because users have uploaded encrypted files and the companies (rightly) don't have the keys. I'll also bet there are many WD hard drives full of CSAM, many Linux servers hosting it, many nginx or apache installs serving it up, etc. Should we mandate that all hard drives scan files as they're written to see if it's CSAM? Or maybe nginx should alert law enforcement anytime a CSAM image is served. Apple should have actually let their users have E2EE or given up on that and just scanned stuff server-side.

by bnj 5y ago

patmcc 5y ago

>>>In a legal context where detecting CSAM is a strong requirement, what’s preferable to this approach? It's not a strong requirement though? Only if the company sees it (unencrypted) do they need to report it, detecting it on-device is wholly different. It would honestly be better if they adjusted their TOS and started scanning the files on upload (on server side). They have the encryption keys already. Apple is planning to install on all (recent) iPhones software that will let them scan for any image similar (using perceptual hashing) to some set of images. Right now, that's CSAM, and only on upload to iCloud. But what do you want to bet China is salivating at the idea of including images of Tank Man and other "seditious" content? And maybe checking images anytime they're added to the phone, or sent to someone else, not just up to iCloud? It completely ruins the idea that Apple has your privacy/security in mind.

nonbirithm 5y ago
Even if scanning for the content is not explicitly required by law, what happens when a pedophile ring hoarding thousands of images of CSAM on iCloud is busted, and the news gets out? The article at [1] makes it sound like Apple choosing not to scan any of its users' videos in iCloud was seen as evidence of Apple lagging behind the status quo of companies like Facebook that were proactively reporting CSAM. According to that article, in the last year Apple only submitted 265 reports to the NCEMC while Facebook submitted 20 million. Would law enforcement believe they'd be missing out on catching abusers after seeing this disparity? If a company is found to allow criminals to store CSAM on their servers for extended periods of time, the law is going to want to know why they let it pass, irrespective of the extent the company chose to scan for it. Apple probably doesn't want to deal with that fallout, so maybe they figured that being proactive about scanning for CSAM in a way that could enable the use of E2EE wouldn't hurt, and that pushing the privacy narrative would satisfy enough people - which it didn't. [1] https://www.nytimes.com/2021/08/05/technology/apple-iphones-privacy.html https://www.nytimes.com/2021/08/05/technology/apple-iphones-...
- patmcc 5y ago
  >>>If a company is found to allow criminals to store CSAM on their servers I'd bet dollars to doughnuts that AWS, GCP, Azure, et al. have terabytes of CSAM stored within their data centers - because users have uploaded encrypted files and the companies (rightly) don't have the keys. I'll also bet there are many WD hard drives full of CSAM, many Linux servers hosting it, many nginx or apache installs serving it up, etc. Should we mandate that all hard drives scan files as they're written to see if it's CSAM? Or maybe nginx should alert law enforcement anytime a CSAM image is served. Apple should have actually let their users have E2EE or given up on that and just scanned stuff server-side.