3 ms·

> We now live in a world, where our personal computers come with "AI" that detects illegal use of those computers and automatically turns us in to the authoriti

by notJim 5y ago

> We now live in a world, where our personal computers come with "AI" that detects illegal use of those computers and automatically turns us in to the authorities. Wait a second, but only if you're uploading photos to iCloud? They were presumably already doing this once you upload them, so what is the big difference? I have totally missed this detail in a lot of the coverage, and it changes my view of it.

chipotle_coyote 5y ago
It's my understanding that they weren't actually checking photos on their servers against the CSAM database, which is why Apple's reporting of CSAM on their network has historically been orders of magnitude less than other companies -- Facebook last year reported ~20M, other companies reported tens or hundreds of thousands, and Apple reported, IIRC, 265. I'm aware the general sentiment here is "don't assume governments are putting pressure on Apple to step up their reporting, this is all on Apple bad bad bad bad bad," but sorry -- those numbers do indeed lead me to strongly suspect governments are putting pressure on Apple to step up their reporting. And I don't think Jason Snell (the author of the linked article, who probably knows more about Apple's world view and internal politics than almost anyone outside the executive team does) is wrong to theorize that Apple sees this CSAM-hash-detection system as a way to meet that pressure while protecting user privacy as much as possible, because they've convinced themselves that "protecting user privacy" means "do as much on device and as little in the cloud as possible". To be clear, I don't think this is a good idea -- letting the camel's nose in the tent through a different opening is still letting the camel's nose in the tent. The irony, though, is that I think Apple's inventively bad solution comes from the most canonically Silicon Valley mindset ever: all problems are engineering problems.
- notJim 5y ago
  Yeah it's kind of odd, because now law enforcement/the courts can say "well, you have the capability to scan content on device, you just choose not to in some cases", which would seem to open Apple up to being forced to do it in far more cases. I think we have to look at this through the lens of governments around the world attempting to ban E2E encryption. E2E encryption has both security and privacy benefits, and IMO most people are far more affected by the security benefits, given the frequency of data leaks/hacks. So maybe Apple is saying "let's preserve some of the E2E encryption benefits", but show it can be compatible with LEO demands. I think the unfortunate reality may be that that is the least bad option.

by notJim 5y ago

chipotle_coyote 5y ago

It's my understanding that they weren't actually checking photos on their servers against the CSAM database, which is why Apple's reporting of CSAM on their network has historically been orders of magnitude less than other companies -- Facebook last year reported ~20M, other companies reported tens or hundreds of thousands, and Apple reported, IIRC, 265. I'm aware the general sentiment here is "don't assume governments are putting pressure on Apple to step up their reporting, this is all on Apple bad bad bad bad bad," but sorry -- those numbers do indeed lead me to strongly suspect governments are putting pressure on Apple to step up their reporting. And I don't think Jason Snell (the author of the linked article, who probably knows more about Apple's world view and internal politics than almost anyone outside the executive team does) is wrong to theorize that Apple sees this CSAM-hash-detection system as a way to meet that pressure while protecting user privacy as much as possible, because they've convinced themselves that "protecting user privacy" means "do as much on device and as little in the cloud as possible". To be clear, I don't think this is a good idea -- letting the camel's nose in the tent through a different opening is still letting the camel's nose in the tent. The irony, though, is that I think Apple's inventively bad solution comes from the most canonically Silicon Valley mindset ever: all problems are engineering problems.

notJim 5y ago
Yeah it's kind of odd, because now law enforcement/the courts can say "well, you have the capability to scan content on device, you just choose not to in some cases", which would seem to open Apple up to being forced to do it in far more cases. I think we have to look at this through the lens of governments around the world attempting to ban E2E encryption. E2E encryption has both security and privacy benefits, and IMO most people are far more affected by the security benefits, given the frequency of data leaks/hacks. So maybe Apple is saying "let's preserve some of the E2E encryption benefits", but show it can be compatible with LEO demands. I think the unfortunate reality may be that that is the least bad option.