7 ms·

> there's a legitimate "slippery slope" argument The slippery slope argument is the only useful argument here. The fundamental issue with their PSI/CSAM syste

by new299 5y ago

> there's a legitimate "slippery slope" argument The slippery slope argument is the only useful argument here. The fundamental issue with their PSI/CSAM system is that they already were scanning iCloud content [1] and that they're seemingly not removing the ability to do that. If the PSI/CSAM system had been announced along side E2E encryption for iCloud backups, it would be clear that they were attempting to act in their users best interests. But it wasn't, so as it stands there's no obvious user benefit. It then becomes a question of trust. Apple are clearly willing to add functionality at the request of governments, on device. By doing this they lose user trust (in my opinion). > In the long run, we solve this, at least in liberal democracies, by voting people into office who understand technology, understand the value of personal encryption But this is clearly not the way it happens in practice. At least in part, we vote with our wallets and give money to companies willing to push back on governmental over-reach. Until now, Apple was one such company [2]. I realize that Apple likely don't "care" about privacy (it's a company, not a individual human). But in a purely cynical sense, positioning themselves as caring about privacy, and pushing back against governmental over-reach on users behalf was useful. And while it's "just marketing" it benefits users. By implementing this functionality, they've lost this "marketing benefit". Users can't buy devices believing they're supporting a company willing to defend their privacy. [1] https://nakedsecurity.sophos.com/2020/01/09/apples-scanning-icloud-photos-for-child-abuse-images/ https://nakedsecurity.sophos.com/2020/01/09/apples-scanning-... [2] https://epic.org/amicus/crypto/apple/ https://epic.org/amicus/crypto/apple/

shuckles 5y ago
> This is an area we’ve been looking at for some time, including current state of the art techniques which mostly involves scanning through entire contents of users’ libraries on cloud services that — as you point out — isn’t something that we’ve ever done; to look through users’ iCloud Photos. https://techcrunch.com/2021/08/10/interview-apples-head-of-privacy-details-child-abuse-detection-and-messages-safety-features/ https://techcrunch.com/2021/08/10/interview-apples-head-of-p... > This moment calls for public discussion, and we want our customers and people around the country to understand what is at stake. - Tim Cook, Apple At what point does the fact that half a decade has passed since those words were written yet the hacker community has made little contribution to that discourse about the importance of privacy start implicating us in the collective failure to act?
- hypothesis 5y ago
  And yet we can go e.g. on Twitter and observe comments from relevant security researchers that appear to describe a chilling atmosphere surrounding privacy research, including resistance to even consider white paper. That’s not my area of expertise and don’t know how to fix that, but that should be an important consideration.
- new299 5y ago
  > The voucher generation is actually exactly what enables us not to have to begin processing all users’ content on our servers, which we’ve never done for iCloud Photos. I really dislike this statement. It's likely designed to be "technically true". But it's been reported elsewhere that they do scan iCloud content: https://nakedsecurity.sophos.com/2020/01/09/apples-scanning-icloud-photos-for-child-abuse-images/ https://nakedsecurity.sophos.com/2020/01/09/apples-scanning-... Perhaps they scan as the data is being ingested. Perhaps it's scanned on a third party server. But it seems clear that it is being scanned.
  - shuckles 5y ago
    My understanding based on piecing together the various poorly cited news stories is that Apple used to scan iCloud Mail for this material, and that’s it.
    - new299 5y ago
      If you have references to also help me piece this together I'd find that really helpful.
      - selsta 5y ago
        > Last year, for instance, Apple reported 265 cases to the National Center for Missing & Exploited Children, while Facebook reported 20.3 million According to [1] it does seem like Apple didn't do any wide scale scanning of iCloud Data. [1] https://www.nytimes.com/2021/08/05/technology/apple-iphones-privacy.html https://www.nytimes.com/2021/08/05/technology/apple-iphones-...
chipotle_coyote 5y ago
> If the PSI/CSAM system had been announced along side E2E encryption for iCloud backups, it would be clear that they were attempting to act in their users best interests. Absolutely. I don't know whether there's a reason for this timing (that is, if they are planning E2E encryption, why they announced this first), but this is probably the biggest PR bungle Apple has had since "you're holding it wrong," if not ever. > Apple are clearly willing to add functionality at the request of governments, on device. Maybe? I'm not as willing to state that quite as definitively, given the pushback Apple gave in the San Bernardino shooter case. Some of what their Privacy Engineering head said in that TechCrunch article suggests that Apple has engineered this to be strategically awkward, e.g., generating the hashes by using ML trained on the CSAM data set (so the hashing system isn't as effective on other data sets) and making the on-device hashing component part of the operating system itself rather than a separately updatable data set. That in turn suggests to me Apple is still looking for an engineering way to say "no" if they're asked "hey, can you just add these other images to your data set." (Of course, my contention that this is not ultimately an engineering problem applies here, too: even if I'm right about Apple playing an engineering shell game here, I'm not convinced it's enough if a government is sufficiently insistent.) A minor interesting tidbit: your linked Sophos story is based on a Telegraph UK story that has this disclaimer at the bottom: > This story originally said Apple screens photos when they are uploaded to iCloud, Apple's cloud storage service. Ms Horvath and Apple's disclaimer did not mention iCloud, and the company has not specified how it screens material, saying this information could help criminals. It's hard to say what they were actually doing, but it's reasonable to suspect it's an earlier, perhaps entirely cloud-based rather than partially cloud-based, version of NeuralHash.
- new299 5y ago
  Right, and in the interview linked [1] above they state: > The voucher generation is actually exactly what enables us not to have to begin processing all users’ content on our servers, which we’ve never done for iCloud Photos. But they do appear to do "something" server-side. It's possible that all data in scanned as it is ingested for example. I dislike this statement, because it's probably technically correct but doesn't help clarify the situation in a helpful way. It makes me trust Apple less. [1] https://techcrunch.com/2021/08/10/interview-apples-head-of-privacy-details-child-abuse-detection-and-messages-safety-features/ https://techcrunch.com/2021/08/10/interview-apples-head-of-p...
dwaite 5y ago
> The fundamental issue with their PSI/CSAM system is that they already were scanning iCloud content This scanning was of email attachments being sent through an iCloud-hosted account, not of other iCloud hosted data (which is encrypted during operation.)
- new299 5y ago
  Do you have a public reference for this?
- cm2187 5y ago
  If it’s encrypted but apple has the key, it’s not encrypted to them.
- BostonEnginerd 5y ago
  I don’t think that photos are encrypted as you can view them from the iCloud website.
nicce 5y ago
> If the PSI/CSAM system had been announced along side E2E encryption for iCloud backups, it would be clear that they were attempting to act in their users best interests. Most likely because the only way this announcement makes sense, is that they tried to respond for misleading leaks. We'll see on September probably some E2EE announcements, since iOS 15 beta supports tokens for backup recovery. At least, let's hope so.
HWR_14 5y ago
> If the PSI/CSAM system had been announced along side E2E encryption for iCloud backups, it would be clear that they were attempting to act in their users best interests. As I understand it, this was more "leaked" than "announced". That is, it wasn't part of Apple's planned rollout strategy. My thought was that they'll announce that soon, but then again, I'm shocked soon wasn't last week. I have no idea. Maybe there is some other limitation (legal) on he iCloud E2E backups they needed to solve first?

by new299 5y ago

shuckles 5y ago

> This is an area we’ve been looking at for some time, including current state of the art techniques which mostly involves scanning through entire contents of users’ libraries on cloud services that — as you point out — isn’t something that we’ve ever done; to look through users’ iCloud Photos. https://techcrunch.com/2021/08/10/interview-apples-head-of-privacy-details-child-abuse-detection-and-messages-safety-features/ https://techcrunch.com/2021/08/10/interview-apples-head-of-p... > This moment calls for public discussion, and we want our customers and people around the country to understand what is at stake. - Tim Cook, Apple At what point does the fact that half a decade has passed since those words were written yet the hacker community has made little contribution to that discourse about the importance of privacy start implicating us in the collective failure to act?

hypothesis 5y ago
And yet we can go e.g. on Twitter and observe comments from relevant security researchers that appear to describe a chilling atmosphere surrounding privacy research, including resistance to even consider white paper. That’s not my area of expertise and don’t know how to fix that, but that should be an important consideration.
new299 5y ago
> The voucher generation is actually exactly what enables us not to have to begin processing all users’ content on our servers, which we’ve never done for iCloud Photos. I really dislike this statement. It's likely designed to be "technically true". But it's been reported elsewhere that they do scan iCloud content: https://nakedsecurity.sophos.com/2020/01/09/apples-scanning-icloud-photos-for-child-abuse-images/ https://nakedsecurity.sophos.com/2020/01/09/apples-scanning-... Perhaps they scan as the data is being ingested. Perhaps it's scanned on a third party server. But it seems clear that it is being scanned.
- shuckles 5y ago
  My understanding based on piecing together the various poorly cited news stories is that Apple used to scan iCloud Mail for this material, and that’s it.
  - new299 5y ago
    If you have references to also help me piece this together I'd find that really helpful.
    - selsta 5y ago
      > Last year, for instance, Apple reported 265 cases to the National Center for Missing & Exploited Children, while Facebook reported 20.3 million According to [1] it does seem like Apple didn't do any wide scale scanning of iCloud Data. [1] https://www.nytimes.com/2021/08/05/technology/apple-iphones-privacy.html https://www.nytimes.com/2021/08/05/technology/apple-iphones-...

chipotle_coyote 5y ago

> If the PSI/CSAM system had been announced along side E2E encryption for iCloud backups, it would be clear that they were attempting to act in their users best interests. Absolutely. I don't know whether there's a reason for this timing (that is, if they are planning E2E encryption, why they announced this first), but this is probably the biggest PR bungle Apple has had since "you're holding it wrong," if not ever. > Apple are clearly willing to add functionality at the request of governments, on device. Maybe? I'm not as willing to state that quite as definitively, given the pushback Apple gave in the San Bernardino shooter case. Some of what their Privacy Engineering head said in that TechCrunch article suggests that Apple has engineered this to be strategically awkward, e.g., generating the hashes by using ML trained on the CSAM data set (so the hashing system isn't as effective on other data sets) and making the on-device hashing component part of the operating system itself rather than a separately updatable data set. That in turn suggests to me Apple is still looking for an engineering way to say "no" if they're asked "hey, can you just add these other images to your data set." (Of course, my contention that this is not ultimately an engineering problem applies here, too: even if I'm right about Apple playing an engineering shell game here, I'm not convinced it's enough if a government is sufficiently insistent.) A minor interesting tidbit: your linked Sophos story is based on a Telegraph UK story that has this disclaimer at the bottom: > This story originally said Apple screens photos when they are uploaded to iCloud, Apple's cloud storage service. Ms Horvath and Apple's disclaimer did not mention iCloud, and the company has not specified how it screens material, saying this information could help criminals. It's hard to say what they were actually doing, but it's reasonable to suspect it's an earlier, perhaps entirely cloud-based rather than partially cloud-based, version of NeuralHash.

new299 5y ago
Right, and in the interview linked [1] above they state: > The voucher generation is actually exactly what enables us not to have to begin processing all users’ content on our servers, which we’ve never done for iCloud Photos. But they do appear to do "something" server-side. It's possible that all data in scanned as it is ingested for example. I dislike this statement, because it's probably technically correct but doesn't help clarify the situation in a helpful way. It makes me trust Apple less. [1] https://techcrunch.com/2021/08/10/interview-apples-head-of-privacy-details-child-abuse-detection-and-messages-safety-features/ https://techcrunch.com/2021/08/10/interview-apples-head-of-p...

dwaite 5y ago

> The fundamental issue with their PSI/CSAM system is that they already were scanning iCloud content This scanning was of email attachments being sent through an iCloud-hosted account, not of other iCloud hosted data (which is encrypted during operation.)

new299 5y ago
Do you have a public reference for this?
cm2187 5y ago
If it’s encrypted but apple has the key, it’s not encrypted to them.
BostonEnginerd 5y ago
I don’t think that photos are encrypted as you can view them from the iCloud website.

nicce 5y ago

> If the PSI/CSAM system had been announced along side E2E encryption for iCloud backups, it would be clear that they were attempting to act in their users best interests. Most likely because the only way this announcement makes sense, is that they tried to respond for misleading leaks. We'll see on September probably some E2EE announcements, since iOS 15 beta supports tokens for backup recovery. At least, let's hope so.

HWR_14 5y ago

> If the PSI/CSAM system had been announced along side E2E encryption for iCloud backups, it would be clear that they were attempting to act in their users best interests. As I understand it, this was more "leaked" than "announced". That is, it wasn't part of Apple's planned rollout strategy. My thought was that they'll announce that soon, but then again, I'm shocked soon wasn't last week. I have no idea. Maybe there is some other limitation (legal) on he iCloud E2E backups they needed to solve first?