6 ms·

This is limited to users of iCloud photos. If you want to store your photos on Apple servers, shouldn’t they have the right to exclude CSAM content? Apple owns

This is limited to users of iCloud photos. If you want to store your photos on Apple servers, shouldn’t they have the right to exclude CSAM content? Apple owns those servers and is legally liable. Why is this such a big issue?

hypothesis 5y ago
Note how they use your device to do the dirty work for them, instead of doing what everyone else is doing and scanning stuff on their servers.
- gowld 5y ago
  Because they don't want to see your private files.
- ithkuil 5y ago
  Because this way they can encrypt things on your phone and claim that they can't see your photos once on the server (because whatever they had to do with those photos, was already done on your own phone). (There are many ways this can be a slippery slope, but we don't have to pretend they could just so what ever body else is doing just as easily and they just want to do it on your phone because they are lazy or whatever. This is a solution to a legitimate problem and also it turns out that people are rightfully worried about what's next; those two facts can coexist)
btkramer9 5y ago
The issue is that the scanning happens on your device just before upload. So now your own device is scanning for illegal activity _on_ your phone not the servers. The second issue is that it will alert authorities. In regards to CSAM content those issues may not sound terrible. But the second it is expanded to texts, things you say, websites you visit or apps you use it's a lot scarier. And what if instead of CSAM content it is extended to alert authorities for _any_ activity deemed undesirable by your government
- gambiting 5y ago
  Logically the next step is to scan for any copyrighted content and notify authorities that you're watching a movie without paying for it. After all, it's all about catching criminals, how could you possibly object.
  - rootusrootus 5y ago
    That's a fairly large step, though. Apple cares first and foremost about their reputation. If this feature catches a real predator, it is 100% good PR. Every single false positive that makes it into the news is a huge loss, which strongly incentivizes them to avoid that. The last thing I expect them to do is expand the risk surface for something as trivial as copyright enforcement.
    - anon9001 5y ago
      > Every single false positive that makes it into the news is a huge loss, which strongly incentivizes them to avoid that. Just to be clear, "false positive" in this case means an innocent person is accused of trafficking in child sexual abuse material. It's likely they will be raided. Sure, that's bad if you're Apple, but it's a lot worse if you're the alleged predator.
    - FireBeyond 5y ago
      Except they tout needing multiple matches before they do anything about it. Which could also be spun as "Apple allows X freebies of known/highly suspected CSAM on your device before they'll tell anybody". The amount of PR failure that has gone into all this is huge and multi-level.
    - karaterobot 5y ago
      The thing (many) people are worried about is that, having instituted a backdoor on the user's device, Apple may lose the ability to control how it is used. In the past, their strongest defense when the FBI asked them to use a backdoor to decrypt a user's device password has been "we don't have a backdoor that allows us to decrypt a user's device password, so we couldn't do that for you even if we wanted to". From now on, when asked to check whether a user's encrypted phone contains arbitrary content the FBI wants to know about, Apple can no longer say "we don't have a way to do that." Sooner or later, you can bet they will start doing it, whether they want to or not.
chrismcb 5y ago
While apple owns the servers they shouldn't be legally lake. No more than a self storage facility is liable for the items individuals sure in their units.
- Miner49er 5y ago
  I'm not so sure that is true anymore. FOSTA-SESTA makes them liable, I think?
  - monocasa 5y ago
    FOSTA-SESTA is all about "knowingly" participating in "sex trafficking". a) I'm not sure that you can knowingly have an idea what's in encrypted blobs, so I don't think this would enable E2EE; they seem orthogonal. b) They're not policing a communication medium, but local files. Sex trafficking almost tautologically implies some kind of communication, so the target being local files instead of, say, local storage of apps like messenger and signal seems a bit off as an explanation.
psyc 5y ago
> If you want to store your photos on Apple servers, shouldn’t they have the right to exclude CSAM content? This seems worded to get a Yes answer. So, yes. It's a big deal because it's unprecedented (to my knowledge) outside of the domain of malware*. Other cloud providers run checks of their own property, on their own property. This runs a check of your property, on your property. That's why people care now. The fact that this occurs because of an intention to upload to their server doesn't really change the problem, not unless you're only looking at this like an architectural diagram. Which I fear many people are. A techie might look at this and see a simple architectural choice. Client-side code instead of server-side. Ok, neat. A more sophisticated techie might see a master plan to pave the way for E2EE. A net-win for privacy. Cool. But the problem doesn't go away. My phone, in my pocket, is now checking itself for evidence of a heinous crime. *I hope the comparison isn't too extra. I was thinking, the idea of code running on my device, that I don't want to run, that can gather criminal evidence against me, and report it over the internet... yeah I can't get around it, that really reminds me of malware. Not from society's perspective. From society's perspective maybe it's verygoodware. But from the traditional user's perspective, code that runs on your device, that hurts you, is at least vigilante malware, even if you are terrible.
- innagadadavida 5y ago
  > My phone, in my pocket, is now checking itself for evidence of a heinous crime. I see your point here - this is a slippery slope for Apple. However I don’t see how anyone could achieve both purposes - no fingerprint reporting and prevention of CSAM storage on Apple servers. Also, a practical thing to do is to just not store your photos on iCloud but use something else for sync and backup - there might be a startup opportunity here if enough people care.
- fogof 5y ago
  I don't understand your argument. You say people care about this because it "runs a check of your property, on your property". Would iCloud servers running this check after the photos are uploaded to iCloud be any better?
- jodrellblank 5y ago
  > "Other cloud providers run checks of their own property, on their own property. This runs a check of your property, on your property. That's why people care now." iTunes Match is an iCloud service which (if you buy it and opt-in) scans your local on-device music library for copyrighted songs, tells Apple you have them, and then they let you listen to high quality versions of those songs on all your devices. And it's not filename or id3 tag matching, it's doing a fuzzy match that can identify the same song in low quality rips and in different file formats. It would be concievable for them to scan for banned audio lectures, or scan your whole device outside the iTunes library, or change it to check for other copyrighted files e.g. movies. It could concievably be reporting you to the MPAA/RIAA if it finds certain songs along with your public IP address so they can check if that IP address has ever been logged as torrenting those songs. It could "in future" be changed to look for and report evidence of torrenting or movie copying. There's nothing technical or regulatory(?) stopping Apple from saying "people with CSAM on their computers can't use iTunes Match" and making it scan the computer as a condition of use, is there? There's nothing technical stopping a government from asking "can your iTunes Match scan engine report video files in the iTunes library which match popular Tiannamen Square video MD5 hashes?", is there? I do get that these are not the same seriousness, iTunes Match isn't (so far as we know) scanning to report crime but in so far as "Unprecedented on-device scanning for known content using an opaque database and a closed-source fuzzy-matching engine, it would only take a small change to make it look for other things, governments will definitely pressure them to do that and since they willingly built this system they will definitely agree, and all you can do is trust them", are they not samey enough to be relevant?
gambiting 5y ago
Because if the content is entirely encrypted(like apple says it is) they aren't legally liable and it's entirely voluntary that they do this. Also, no one(well, most people) has any issue with photos being scanned in the iCloud. Photos in Google Photos have been scanned for years and no one cares. The problem is that apple said that photos are encrypted on your device and in the cloud, but now your phone will scan the pictures and if they fail some magical test that you can't inspect, your pictures will be sent unencrypted for verification without telling you. So you think you're sending pictures to secure storage, but nope, actually their algorithm decided that the picture is dodgy in some way so in fact it's sent for viewing by some unknown person. But hey don't worry, you can trust apple, they will definitely only verify it and do nothing else. Because a big American corporation is totally trustworthy.
- strzibny 5y ago
  I mean even the verification is problematic. At no point I want an "certified" Apple employee or another parent looking at naked pictures of my kids, for example.
  - Bud 5y ago
    But there's nothing in this proposed implementation that could ever possibly result in that, because random pics of your kids would not be in a database of known CSAM content. So your pics wouldn't match the hash values.
    - short_sells_poo 5y ago
      So you are saying the hash values can never result in a collision? That in fact there is literally zero chance that two different images could result in the same hash? Because that doesn’t sound correct to me…
      - loeg 5y ago
        It is essentially true that a cryptographic hash can never collide. But what Apple claims to be using is a perceptual hash. Perceptual hashes can not just be scaled wider trivially, to reduce probability of collision, like cryptographic hashes can.
frosted-flakes 5y ago
If they were scanning images that were uploaded to icloud on Apple's servers, no one would care. iCloud is not encrypted and Apple provides governments access to iClod data, everyone knows that, and other cloud providers already scan content for CSAM material. The difference is that Apple is doing this scanning on your phone/computer. Right now, they say that only images that uploaded to iCloud will be scanned, but what's to stop them from scanning other files too? There's been a lot of pushback because this is essentially a back door into the device that governments can abuse.
- gowld 5y ago
  How can Apple scan encrypted photos?
  - anon9001 5y ago
    Apple has the keys to decrypt them.
  - frosted-flakes 5y ago
    I should have said, "iCloud is not end-to-end encrypted". Apple has full access to everything you upload to iCloud, because they control the encryption keys, not you.
- Joeri 5y ago
  Apple can do anything they want on every iphone, always have, and always will. Whether this feature exists or not changes that in no way, their technical ability to snoop through everyone’s stuff is the same. So far they’ve shown restraint with that ability. I think what people are getting riled up about is not the technical ability, it’s the lack of restraint, the willingness to search through everyone’s personal stuff on their phones. This is like the cops sending a drug-sniffing dog into everyone’s home once a day, with the excuse that it is privacy-preserving because no human enters the premises, and that only truly bad people will get caught. There is a difference between scanning in the cloud and scanning on device. One is looking through your stuff after you’ve stored it in a storage unit, and the other is looking through your stuff while it is still in your home. Apple’s excuse is that you were going to move it anyway, but somehow that doesn’t actually excuse things.
hackinthebochs 5y ago
Personally I don't see on device scanning as significantly different than cloud scanning. I think the widespread acceptance of scanning personal data stored on the cloud is a serious mistake. Cloud storage services are acting as agents of the user and so should not be doing any scanning or interpreting of data not explicitly for providing the service to the end user. Scanning/interpreting should only happen when data is shared or disseminated, as that is a non-personal action. If I own my data, someone processing this data on my behalf has no right or obligation to scan it for illegal content. The fact that this data sometimes sits on hard drives owned by another party just isn't a relevant factor. Presumably I still own my car when it sits in the garage at the shop. They have no right or obligation to rummage around looking for evidence of a crime. I don't see abstract data as any different.
- rootusrootus 5y ago
  What does the law say, though? Possession of CSAM by any organization or person other than NCMEC is flatly illegal. Even other branches of government, including law enforcement, may not have any in their possession. My question is -- does CSAM residing on Apple's servers, even when it is 'owned' by a customer, count as them possessing it? What about if it is encrypted?
  - fakedang 5y ago
    Ben's Stratechery article explains the distinction: > (f)Protection of Privacy.—Nothing in this section shall be construed to require a provider to— (1) monitor any user, subscriber, or customer of that provider; (2) monitor the content of any communication of any person described in paragraph (1); or (3) affirmatively search, screen, or scan for facts or circumstances described in sections (a) and (b). https://stratechery.com/2021/apples-mistake/ https://stratechery.com/2021/apples-mistake/
  - short_sells_poo 5y ago
    If it is encrypted, the data storage provider has no chance to know what it is. Which is exactly why these policies are so dim witted. Dragnet violation of everyone’s privacy while anyone even remotely sophisticated can easily evade it by just encrypting the data upfront.
short_sells_poo 5y ago
I’d expect a secure and privacy focused cloud data storage provider to not know what I’m storing _at all_. Let’s not beat about the bush, if someone wants to store information in a form that can’t be decrypted by Apple, they can. This is a stupid dragnet policy that won’t catch anyone sophisticated. Apple focused the last years pitching themselves as the tech giant who actually cares about privacy. They seemed to be consciously building this image. To now implement scanning of private information and then try and sell this obvious 180degree slippery slope turnaround in the most weasel worded “but think of the children” trope is an insult to the customers’ intelligence. I was a keen Apple consumer because I felt that even if their motivation was profit, this was a company who focused on privacy. It was a distinct selling point. I certainly won’t be buying more Apple products. For me, Apple lost the main reason to buy their stuff. If they are going to do the same thing everyone else is doing, I refuse to pay the premium they charge.

hypothesis 5y ago

Note how they use your device to do the dirty work for them, instead of doing what everyone else is doing and scanning stuff on their servers.

gowld 5y ago
Because they don't want to see your private files.
ithkuil 5y ago
Because this way they can encrypt things on your phone and claim that they can't see your photos once on the server (because whatever they had to do with those photos, was already done on your own phone). (There are many ways this can be a slippery slope, but we don't have to pretend they could just so what ever body else is doing just as easily and they just want to do it on your phone because they are lazy or whatever. This is a solution to a legitimate problem and also it turns out that people are rightfully worried about what's next; those two facts can coexist)

btkramer9 5y ago

The issue is that the scanning happens on your device just before upload. So now your own device is scanning for illegal activity _on_ your phone not the servers. The second issue is that it will alert authorities. In regards to CSAM content those issues may not sound terrible. But the second it is expanded to texts, things you say, websites you visit or apps you use it's a lot scarier. And what if instead of CSAM content it is extended to alert authorities for _any_ activity deemed undesirable by your government

gambiting 5y ago
Logically the next step is to scan for any copyrighted content and notify authorities that you're watching a movie without paying for it. After all, it's all about catching criminals, how could you possibly object.
- rootusrootus 5y ago
  That's a fairly large step, though. Apple cares first and foremost about their reputation. If this feature catches a real predator, it is 100% good PR. Every single false positive that makes it into the news is a huge loss, which strongly incentivizes them to avoid that. The last thing I expect them to do is expand the risk surface for something as trivial as copyright enforcement.
  - anon9001 5y ago
    > Every single false positive that makes it into the news is a huge loss, which strongly incentivizes them to avoid that. Just to be clear, "false positive" in this case means an innocent person is accused of trafficking in child sexual abuse material. It's likely they will be raided. Sure, that's bad if you're Apple, but it's a lot worse if you're the alleged predator.
  - FireBeyond 5y ago
    Except they tout needing multiple matches before they do anything about it. Which could also be spun as "Apple allows X freebies of known/highly suspected CSAM on your device before they'll tell anybody". The amount of PR failure that has gone into all this is huge and multi-level.
  - karaterobot 5y ago
    The thing (many) people are worried about is that, having instituted a backdoor on the user's device, Apple may lose the ability to control how it is used. In the past, their strongest defense when the FBI asked them to use a backdoor to decrypt a user's device password has been "we don't have a backdoor that allows us to decrypt a user's device password, so we couldn't do that for you even if we wanted to". From now on, when asked to check whether a user's encrypted phone contains arbitrary content the FBI wants to know about, Apple can no longer say "we don't have a way to do that." Sooner or later, you can bet they will start doing it, whether they want to or not.

chrismcb 5y ago

While apple owns the servers they shouldn't be legally lake. No more than a self storage facility is liable for the items individuals sure in their units.

Miner49er 5y ago
I'm not so sure that is true anymore. FOSTA-SESTA makes them liable, I think?
- monocasa 5y ago
  FOSTA-SESTA is all about "knowingly" participating in "sex trafficking". a) I'm not sure that you can knowingly have an idea what's in encrypted blobs, so I don't think this would enable E2EE; they seem orthogonal. b) They're not policing a communication medium, but local files. Sex trafficking almost tautologically implies some kind of communication, so the target being local files instead of, say, local storage of apps like messenger and signal seems a bit off as an explanation.

psyc 5y ago

> If you want to store your photos on Apple servers, shouldn’t they have the right to exclude CSAM content? This seems worded to get a Yes answer. So, yes. It's a big deal because it's unprecedented (to my knowledge) outside of the domain of malware*. Other cloud providers run checks of their own property, on their own property. This runs a check of your property, on your property. That's why people care now. The fact that this occurs because of an intention to upload to their server doesn't really change the problem, not unless you're only looking at this like an architectural diagram. Which I fear many people are. A techie might look at this and see a simple architectural choice. Client-side code instead of server-side. Ok, neat. A more sophisticated techie might see a master plan to pave the way for E2EE. A net-win for privacy. Cool. But the problem doesn't go away. My phone, in my pocket, is now checking itself for evidence of a heinous crime. *I hope the comparison isn't too extra. I was thinking, the idea of code running on my device, that I don't want to run, that can gather criminal evidence against me, and report it over the internet... yeah I can't get around it, that really reminds me of malware. Not from society's perspective. From society's perspective maybe it's verygoodware. But from the traditional user's perspective, code that runs on your device, that hurts you, is at least vigilante malware, even if you are terrible.

innagadadavida 5y ago
> My phone, in my pocket, is now checking itself for evidence of a heinous crime. I see your point here - this is a slippery slope for Apple. However I don’t see how anyone could achieve both purposes - no fingerprint reporting and prevention of CSAM storage on Apple servers. Also, a practical thing to do is to just not store your photos on iCloud but use something else for sync and backup - there might be a startup opportunity here if enough people care.
fogof 5y ago
I don't understand your argument. You say people care about this because it "runs a check of your property, on your property". Would iCloud servers running this check after the photos are uploaded to iCloud be any better?
jodrellblank 5y ago
> "Other cloud providers run checks of their own property, on their own property. This runs a check of your property, on your property. That's why people care now." iTunes Match is an iCloud service which (if you buy it and opt-in) scans your local on-device music library for copyrighted songs, tells Apple you have them, and then they let you listen to high quality versions of those songs on all your devices. And it's not filename or id3 tag matching, it's doing a fuzzy match that can identify the same song in low quality rips and in different file formats. It would be concievable for them to scan for banned audio lectures, or scan your whole device outside the iTunes library, or change it to check for other copyrighted files e.g. movies. It could concievably be reporting you to the MPAA/RIAA if it finds certain songs along with your public IP address so they can check if that IP address has ever been logged as torrenting those songs. It could "in future" be changed to look for and report evidence of torrenting or movie copying. There's nothing technical or regulatory(?) stopping Apple from saying "people with CSAM on their computers can't use iTunes Match" and making it scan the computer as a condition of use, is there? There's nothing technical stopping a government from asking "can your iTunes Match scan engine report video files in the iTunes library which match popular Tiannamen Square video MD5 hashes?", is there? I do get that these are not the same seriousness, iTunes Match isn't (so far as we know) scanning to report crime but in so far as "Unprecedented on-device scanning for known content using an opaque database and a closed-source fuzzy-matching engine, it would only take a small change to make it look for other things, governments will definitely pressure them to do that and since they willingly built this system they will definitely agree, and all you can do is trust them", are they not samey enough to be relevant?

gambiting 5y ago

Because if the content is entirely encrypted(like apple says it is) they aren't legally liable and it's entirely voluntary that they do this. Also, no one(well, most people) has any issue with photos being scanned in the iCloud. Photos in Google Photos have been scanned for years and no one cares. The problem is that apple said that photos are encrypted on your device and in the cloud, but now your phone will scan the pictures and if they fail some magical test that you can't inspect, your pictures will be sent unencrypted for verification without telling you. So you think you're sending pictures to secure storage, but nope, actually their algorithm decided that the picture is dodgy in some way so in fact it's sent for viewing by some unknown person. But hey don't worry, you can trust apple, they will definitely only verify it and do nothing else. Because a big American corporation is totally trustworthy.

strzibny 5y ago
I mean even the verification is problematic. At no point I want an "certified" Apple employee or another parent looking at naked pictures of my kids, for example.
- Bud 5y ago
  But there's nothing in this proposed implementation that could ever possibly result in that, because random pics of your kids would not be in a database of known CSAM content. So your pics wouldn't match the hash values.
  - short_sells_poo 5y ago
    So you are saying the hash values can never result in a collision? That in fact there is literally zero chance that two different images could result in the same hash? Because that doesn’t sound correct to me…
    - loeg 5y ago
      It is essentially true that a cryptographic hash can never collide. But what Apple claims to be using is a perceptual hash. Perceptual hashes can not just be scaled wider trivially, to reduce probability of collision, like cryptographic hashes can.

frosted-flakes 5y ago

If they were scanning images that were uploaded to icloud on Apple's servers, no one would care. iCloud is not encrypted and Apple provides governments access to iClod data, everyone knows that, and other cloud providers already scan content for CSAM material. The difference is that Apple is doing this scanning on your phone/computer. Right now, they say that only images that uploaded to iCloud will be scanned, but what's to stop them from scanning other files too? There's been a lot of pushback because this is essentially a back door into the device that governments can abuse.

gowld 5y ago
How can Apple scan encrypted photos?
- anon9001 5y ago
  Apple has the keys to decrypt them.
- frosted-flakes 5y ago
  I should have said, "iCloud is not end-to-end encrypted". Apple has full access to everything you upload to iCloud, because they control the encryption keys, not you.
Joeri 5y ago
Apple can do anything they want on every iphone, always have, and always will. Whether this feature exists or not changes that in no way, their technical ability to snoop through everyone’s stuff is the same. So far they’ve shown restraint with that ability. I think what people are getting riled up about is not the technical ability, it’s the lack of restraint, the willingness to search through everyone’s personal stuff on their phones. This is like the cops sending a drug-sniffing dog into everyone’s home once a day, with the excuse that it is privacy-preserving because no human enters the premises, and that only truly bad people will get caught. There is a difference between scanning in the cloud and scanning on device. One is looking through your stuff after you’ve stored it in a storage unit, and the other is looking through your stuff while it is still in your home. Apple’s excuse is that you were going to move it anyway, but somehow that doesn’t actually excuse things.

hackinthebochs 5y ago

Personally I don't see on device scanning as significantly different than cloud scanning. I think the widespread acceptance of scanning personal data stored on the cloud is a serious mistake. Cloud storage services are acting as agents of the user and so should not be doing any scanning or interpreting of data not explicitly for providing the service to the end user. Scanning/interpreting should only happen when data is shared or disseminated, as that is a non-personal action. If I own my data, someone processing this data on my behalf has no right or obligation to scan it for illegal content. The fact that this data sometimes sits on hard drives owned by another party just isn't a relevant factor. Presumably I still own my car when it sits in the garage at the shop. They have no right or obligation to rummage around looking for evidence of a crime. I don't see abstract data as any different.

rootusrootus 5y ago
What does the law say, though? Possession of CSAM by any organization or person other than NCMEC is flatly illegal. Even other branches of government, including law enforcement, may not have any in their possession. My question is -- does CSAM residing on Apple's servers, even when it is 'owned' by a customer, count as them possessing it? What about if it is encrypted?
- fakedang 5y ago
  Ben's Stratechery article explains the distinction: > (f)Protection of Privacy.—Nothing in this section shall be construed to require a provider to— (1) monitor any user, subscriber, or customer of that provider; (2) monitor the content of any communication of any person described in paragraph (1); or (3) affirmatively search, screen, or scan for facts or circumstances described in sections (a) and (b). https://stratechery.com/2021/apples-mistake/ https://stratechery.com/2021/apples-mistake/
- short_sells_poo 5y ago
  If it is encrypted, the data storage provider has no chance to know what it is. Which is exactly why these policies are so dim witted. Dragnet violation of everyone’s privacy while anyone even remotely sophisticated can easily evade it by just encrypting the data upfront.

short_sells_poo 5y ago

I’d expect a secure and privacy focused cloud data storage provider to not know what I’m storing _at all_. Let’s not beat about the bush, if someone wants to store information in a form that can’t be decrypted by Apple, they can. This is a stupid dragnet policy that won’t catch anyone sophisticated. Apple focused the last years pitching themselves as the tech giant who actually cares about privacy. They seemed to be consciously building this image. To now implement scanning of private information and then try and sell this obvious 180degree slippery slope turnaround in the most weasel worded “but think of the children” trope is an insult to the customers’ intelligence. I was a keen Apple consumer because I felt that even if their motivation was profit, this was a company who focused on privacy. It was a distinct selling point. I certainly won’t be buying more Apple products. For me, Apple lost the main reason to buy their stuff. If they are going to do the same thing everyone else is doing, I refuse to pay the premium they charge.