13 ms·

> But with Apple's CSAM proposal is NOT possible to have an iPhone that Apple isn't continuously scanning. As currently implemented, iOS will only scan photos

by js2 5y ago

> But with Apple's CSAM proposal is NOT possible to have an iPhone that Apple isn't continuously scanning. As currently implemented, iOS will only scan photos to be uploaded to iCloud Photos. If iCloud Photos is not enabled, then Apple isn't scanning the phone.

hn_throwaway_99 5y ago
Woops, you're right, thanks for the correction. I think the issue is that, as Ben Thompson pointed out, the only thing preventing them from scanning other stuff now is just policy, not capability, and now that Pandora's box is open it's going to be much more difficult to resist when a government comes to them and says "we want you to scan messages for subversive content".
- js2 5y ago
  Yes, of course that is true. I use iCloud Photos and find this terribly creepy. If Apple must scan my photos, I'd rather they do it on their servers. I could maybe understand the new implementation if Apple had announced they'd also be enabling E2E encryption of everything in iCloud, and explained it as "this is the only way we can prevent CSAM from being stored on our servers."
  - AnthonyMouse 5y ago
    > "this is the only way we can prevent CSAM from being stored on our servers." Why is that supposed to be their responsibility? Given the scale of their business, there is effectively a certainty that people are transmitting CSAM via Comcast's network. That's no excuse for them to ban end to end encryption or start pushing out code to scan client devices. When you hail a taxi, they don't search you for drugs before letting you inside. Because they're not the police. It isn't their role to enforce the law.
    - emodendroket 5y ago
      If you become known as a hub for this stuff Congress probably won't see it that way.
    - alwillis 5y ago
      Why is that supposed to be their responsibility? They (and Google, Microsoft, Facebook, etc.) are essentially mandated reporters; if CSAM is on their servers, they're required to report it. It's like how a doctor is a mandated reporter regarding physical abuse and other issues. Because they're not the police. It isn't their role to enforce the law. They're not enforcing the law; if the CSAM reaches a certain threshold, they check it out and if it's the real deal, they report to National Center for Missing and Exploited Children (NCMEC); they get law enforcement involved if necessary.
      - kelnos 5y ago
        > They (and Google, Microsoft, Facebook, etc.) are essentially mandated reporters; if CSAM is on their servers, they're required to report it. I don't think that's correct. My understanding is that if they find CSAM, they're obligated to report it (just like anyone is). I don't believe they are legally obligated to proactively look for it. (It would be a PR nightmare for them to have an unchecked CSAM problem on their services, so they do look for it and report it.) Consider that Apple likely does have CSAM on their servers, because they apparently don't scan iCloud backups right now. I don't believe they're breaking any laws, at least until and unless they find any of it and (hypothetically) don't report it
        kmeisthax 5y ago
        Last year Congress mulled over a bill (EARN IT Act) that would explicitly require online services to proactively search for CSAM, by allowing services to be sued by governments for failing to do so. It would also allow services to be sued for failing to provide law-enforcement back doors to encrypted data. There's also SESTA/FOSTA, already in law, that rescinded CDA 230 protection for cases involving sex trafficking. Quite honestly, my opinion is that any service not scanning for CSAM is living on borrowed time.
  - alwillis 5y ago
    If Apple must scan my photos, I'd rather they do it on their servers. Scanning on their servers is much less privacy preserving for users but don't fret, Dropbox, Facebook, Microsoft, etc. already scan stored photos.
    - lifty 5y ago
      Not necessarily. The scanning implementation can be similar to what they plan on doing on your device. I don't want them to do the scanning on my phone. If shit hits the fan and I become a dissident, I would prefer to have the option to stop using iCloud, Dropbox or any other service that might enable my government or the secret police to suppress me.
    - zekrioca 5y ago
      One can encrypt a picture before uploading it to the aforementioned services, and as such nothing would really happen. That’s not a possibility on pictures taken on iPhones that get uploaded to iCloud.
- tshaddox 5y ago
  Given that they obviously already have the capability to send software updates that add new on-device capability, this seems like a meaningless distinction. It’s already “just policy” preventing them from sending any conceivable software update to their phones.
  - Dylan16807 5y ago
    Making a custom OS update is pretty different from adding an entry into a database, and could also be mitigated against.
  - hn_throwaway_99 5y ago
    I disagree, strongly. Let's say you're authoritarian government EvilGov. Before this announcement, if you went to Apple and said "we want you to push this spyware to your iPhones", Apple would and could have easily pushed back both in the court of public opinion and the court of law. Now though, Apple is already saying "We'll take this database of illegal image hashes provided by the government and use it to scan your phone." It's now quite trivial for a government to say "We don't have a special database of just CSAM, and a different database of just Winnie the Pooh memes. We just have one big DB of 'illegal images', and that's what we want you to use to scan the phones."
    - ribosometronome 5y ago
      Why do you think essentially no one is complaining about using ML to understand the content of photos, then (especially in comparison to this rather targeted CSAM feature)? My impression is that both Apple and Google have already been doing that since what? 2016? Earlier? There's been no need for a database of photos, either company could silently update those algorithms to ping on guns, drugs, Winnie the Pooh memes, etc.
      - salawat 5y ago
        raises hand Have been whining about big data for and ML recognition systems for years.
      - 5y ago
- Razengan 5y ago
  Woops, yet another example of uninformed outrage. People leading pitchfork parties without knowing the full story.
callmeal 5y ago
>As currently implemented, iOS will only scan photos to be uploaded to iCloud Photos. If iCloud Photos is not enabled, then Apple isn't scanning the phone. Except for the "oops, due to an unexpected bug in our code, every image, document, and message on your device was being continuously scanned" mea culpa we will see a few months after this goes live.
- resfirestar 5y ago
  The potential for this is overblown and I think a lot of people haven’t taken the time to understand how the system is set up. iOS is not scanning for hash matches and phoning home every time it sees one, it’s attaching a cryptographic voucher to every photo uploaded to iCloud that, if some number of photos represent hash matches (the fact of the match is not revealed until the threshold number is reached), then allow Apple to decrypt the photos that match. This is not something where “oops bug got non-iCloud photos too” or “court told us to flip the switch to scan the other photos” would make any sense, some significant modification would be required. Which I agree is a risk especially with governments like China/EU that like to set conditions for market access, just not a very immediate one.
  - spsful 5y ago
    I hear you, 100%, but as a longtime Apple and iCloud user I have been through the absolute ringer when it comes to iCloud and have little faith in it operating correctly. 1. About 3 years ago, there were empty files taking up storage on my iCloud account, and they weren't visible on the website so I couldn't delete them. All it showed was that an excessive amount of storage was taken up. Apple advisors had no idea what was going on and my entire account had to be sent to iCloud engineers to resolve the issue. They never followed up, but it took months for these random ghost files to stop taking up my iCloud storage. 2. Sometimes flipping them on and off a lot causes delays in the OS and then you have to kill the Settings app and re open it to see if they're actually enabled. Back when ATT was just being implemented, I noticed it was grayed out on my phone every time I was signed in to iCloud, but was completely operational when I was signed out. Many others had this issue and there was literally no setting to change within iCloud; it was a bug that engineering literally acknowledged to me in an email (they acknowledged that it happened for some users and fixed it in a software update). Screwups happen in an increasingly complex OS and I just feel that there will be a day when this type of bug surfaces in addition to everything that already happens to us end users.
whoisjuan 5y ago
That’s wrong. They are also enabling on-device detection for Messages.
- js2 5y ago
  Communication safety in Messages is a different feature than CSAM detection in iCloud Photos. The two features are not the same and do not use the same technology. In addition: Communication safety in Messages is only available for accounts set up as families in iCloud. Parent/guardian accounts must opt in to turn on the feature for their family group. Parental notifications can only be enabled by parents/guardians for child accounts age 12 or younger. https://www.apple.com/child-safety/pdf/Expanded_Protections_for_Children_Frequently_Asked_Questions.pdf https://www.apple.com/child-safety/pdf/Expanded_Protections_...
  - b0tzzzzzzman 5y ago
    Yeah, for now this is what is stated. It should be alarming how much they pivoted and changed with out any background statment. Something more is at play.
    - m4rtink 5y ago
      Thats the thing - with an unauditable black box like iOS you have to trust the vendor that it actually does what they say. If they suddenly start to break you trust - who knows what they will push to that black box next time ? You have no way to tell or prevent that, other than leaving the whole platform altogether.
browningstreet 5y ago
For now, in v1.
tremon 5y ago
> As currently implemented We don't actually know what is implemented. We only know what Apple's PR machine is saying about it, or do we have the source code somewhere?

by js2 5y ago

hn_throwaway_99 5y ago

Woops, you're right, thanks for the correction. I think the issue is that, as Ben Thompson pointed out, the only thing preventing them from scanning other stuff now is just policy, not capability, and now that Pandora's box is open it's going to be much more difficult to resist when a government comes to them and says "we want you to scan messages for subversive content".

js2 5y ago
Yes, of course that is true. I use iCloud Photos and find this terribly creepy. If Apple must scan my photos, I'd rather they do it on their servers. I could maybe understand the new implementation if Apple had announced they'd also be enabling E2E encryption of everything in iCloud, and explained it as "this is the only way we can prevent CSAM from being stored on our servers."
- AnthonyMouse 5y ago
  > "this is the only way we can prevent CSAM from being stored on our servers." Why is that supposed to be their responsibility? Given the scale of their business, there is effectively a certainty that people are transmitting CSAM via Comcast's network. That's no excuse for them to ban end to end encryption or start pushing out code to scan client devices. When you hail a taxi, they don't search you for drugs before letting you inside. Because they're not the police. It isn't their role to enforce the law.
  - emodendroket 5y ago
    If you become known as a hub for this stuff Congress probably won't see it that way.
  - alwillis 5y ago
    Why is that supposed to be their responsibility? They (and Google, Microsoft, Facebook, etc.) are essentially mandated reporters; if CSAM is on their servers, they're required to report it. It's like how a doctor is a mandated reporter regarding physical abuse and other issues. Because they're not the police. It isn't their role to enforce the law. They're not enforcing the law; if the CSAM reaches a certain threshold, they check it out and if it's the real deal, they report to National Center for Missing and Exploited Children (NCMEC); they get law enforcement involved if necessary.
    - kelnos 5y ago
      > They (and Google, Microsoft, Facebook, etc.) are essentially mandated reporters; if CSAM is on their servers, they're required to report it. I don't think that's correct. My understanding is that if they find CSAM, they're obligated to report it (just like anyone is). I don't believe they are legally obligated to proactively look for it. (It would be a PR nightmare for them to have an unchecked CSAM problem on their services, so they do look for it and report it.) Consider that Apple likely does have CSAM on their servers, because they apparently don't scan iCloud backups right now. I don't believe they're breaking any laws, at least until and unless they find any of it and (hypothetically) don't report it
      - kmeisthax 5y ago
        Last year Congress mulled over a bill (EARN IT Act) that would explicitly require online services to proactively search for CSAM, by allowing services to be sued by governments for failing to do so. It would also allow services to be sued for failing to provide law-enforcement back doors to encrypted data. There's also SESTA/FOSTA, already in law, that rescinded CDA 230 protection for cases involving sex trafficking. Quite honestly, my opinion is that any service not scanning for CSAM is living on borrowed time.
- alwillis 5y ago
  If Apple must scan my photos, I'd rather they do it on their servers. Scanning on their servers is much less privacy preserving for users but don't fret, Dropbox, Facebook, Microsoft, etc. already scan stored photos.
  - lifty 5y ago
    Not necessarily. The scanning implementation can be similar to what they plan on doing on your device. I don't want them to do the scanning on my phone. If shit hits the fan and I become a dissident, I would prefer to have the option to stop using iCloud, Dropbox or any other service that might enable my government or the secret police to suppress me.
  - zekrioca 5y ago
    One can encrypt a picture before uploading it to the aforementioned services, and as such nothing would really happen. That’s not a possibility on pictures taken on iPhones that get uploaded to iCloud.
tshaddox 5y ago
Given that they obviously already have the capability to send software updates that add new on-device capability, this seems like a meaningless distinction. It’s already “just policy” preventing them from sending any conceivable software update to their phones.
- Dylan16807 5y ago
  Making a custom OS update is pretty different from adding an entry into a database, and could also be mitigated against.
- hn_throwaway_99 5y ago
  I disagree, strongly. Let's say you're authoritarian government EvilGov. Before this announcement, if you went to Apple and said "we want you to push this spyware to your iPhones", Apple would and could have easily pushed back both in the court of public opinion and the court of law. Now though, Apple is already saying "We'll take this database of illegal image hashes provided by the government and use it to scan your phone." It's now quite trivial for a government to say "We don't have a special database of just CSAM, and a different database of just Winnie the Pooh memes. We just have one big DB of 'illegal images', and that's what we want you to use to scan the phones."
  - ribosometronome 5y ago
    Why do you think essentially no one is complaining about using ML to understand the content of photos, then (especially in comparison to this rather targeted CSAM feature)? My impression is that both Apple and Google have already been doing that since what? 2016? Earlier? There's been no need for a database of photos, either company could silently update those algorithms to ping on guns, drugs, Winnie the Pooh memes, etc.
    - salawat 5y ago
      raises hand Have been whining about big data for and ML recognition systems for years.
    - 5y ago
Razengan 5y ago
Woops, yet another example of uninformed outrage. People leading pitchfork parties without knowing the full story.

callmeal 5y ago

>As currently implemented, iOS will only scan photos to be uploaded to iCloud Photos. If iCloud Photos is not enabled, then Apple isn't scanning the phone. Except for the "oops, due to an unexpected bug in our code, every image, document, and message on your device was being continuously scanned" mea culpa we will see a few months after this goes live.

resfirestar 5y ago
The potential for this is overblown and I think a lot of people haven’t taken the time to understand how the system is set up. iOS is not scanning for hash matches and phoning home every time it sees one, it’s attaching a cryptographic voucher to every photo uploaded to iCloud that, if some number of photos represent hash matches (the fact of the match is not revealed until the threshold number is reached), then allow Apple to decrypt the photos that match. This is not something where “oops bug got non-iCloud photos too” or “court told us to flip the switch to scan the other photos” would make any sense, some significant modification would be required. Which I agree is a risk especially with governments like China/EU that like to set conditions for market access, just not a very immediate one.
- spsful 5y ago
  I hear you, 100%, but as a longtime Apple and iCloud user I have been through the absolute ringer when it comes to iCloud and have little faith in it operating correctly. 1. About 3 years ago, there were empty files taking up storage on my iCloud account, and they weren't visible on the website so I couldn't delete them. All it showed was that an excessive amount of storage was taken up. Apple advisors had no idea what was going on and my entire account had to be sent to iCloud engineers to resolve the issue. They never followed up, but it took months for these random ghost files to stop taking up my iCloud storage. 2. Sometimes flipping them on and off a lot causes delays in the OS and then you have to kill the Settings app and re open it to see if they're actually enabled. Back when ATT was just being implemented, I noticed it was grayed out on my phone every time I was signed in to iCloud, but was completely operational when I was signed out. Many others had this issue and there was literally no setting to change within iCloud; it was a bug that engineering literally acknowledged to me in an email (they acknowledged that it happened for some users and fixed it in a software update). Screwups happen in an increasingly complex OS and I just feel that there will be a day when this type of bug surfaces in addition to everything that already happens to us end users.

whoisjuan 5y ago

That’s wrong. They are also enabling on-device detection for Messages.

js2 5y ago
Communication safety in Messages is a different feature than CSAM detection in iCloud Photos. The two features are not the same and do not use the same technology. In addition: Communication safety in Messages is only available for accounts set up as families in iCloud. Parent/guardian accounts must opt in to turn on the feature for their family group. Parental notifications can only be enabled by parents/guardians for child accounts age 12 or younger. https://www.apple.com/child-safety/pdf/Expanded_Protections_for_Children_Frequently_Asked_Questions.pdf https://www.apple.com/child-safety/pdf/Expanded_Protections_...
- b0tzzzzzzman 5y ago
  Yeah, for now this is what is stated. It should be alarming how much they pivoted and changed with out any background statment. Something more is at play.
  - m4rtink 5y ago
    Thats the thing - with an unauditable black box like iOS you have to trust the vendor that it actually does what they say. If they suddenly start to break you trust - who knows what they will push to that black box next time ? You have no way to tell or prevent that, other than leaving the whole platform altogether.

browningstreet 5y ago

For now, in v1.

tremon 5y ago

> As currently implemented We don't actually know what is implemented. We only know what Apple's PR machine is saying about it, or do we have the source code somewhere?