6 ms·

Personally I don't see on device scanning as significantly different than cloud scanning. I think the widespread acceptance of scanning personal data stored on

Personally I don't see on device scanning as significantly different than cloud scanning. I think the widespread acceptance of scanning personal data stored on the cloud is a serious mistake. Cloud storage services are acting as agents of the user and so should not be doing any scanning or interpreting of data not explicitly for providing the service to the end user. Scanning/interpreting should only happen when data is shared or disseminated, as that is a non-personal action. If I own my data, someone processing this data on my behalf has no right or obligation to scan it for illegal content. The fact that this data sometimes sits on hard drives owned by another party just isn't a relevant factor. Presumably I still own my car when it sits in the garage at the shop. They have no right or obligation to rummage around looking for evidence of a crime. I don't see abstract data as any different.

rootusrootus 5y ago
What does the law say, though? Possession of CSAM by any organization or person other than NCMEC is flatly illegal. Even other branches of government, including law enforcement, may not have any in their possession. My question is -- does CSAM residing on Apple's servers, even when it is 'owned' by a customer, count as them possessing it? What about if it is encrypted?
- fakedang 5y ago
  Ben's Stratechery article explains the distinction: > (f)Protection of Privacy.—Nothing in this section shall be construed to require a provider to— (1) monitor any user, subscriber, or customer of that provider; (2) monitor the content of any communication of any person described in paragraph (1); or (3) affirmatively search, screen, or scan for facts or circumstances described in sections (a) and (b). https://stratechery.com/2021/apples-mistake/ https://stratechery.com/2021/apples-mistake/
- short_sells_poo 5y ago
  If it is encrypted, the data storage provider has no chance to know what it is. Which is exactly why these policies are so dim witted. Dragnet violation of everyone’s privacy while anyone even remotely sophisticated can easily evade it by just encrypting the data upfront.
- frenchyatwork 5y ago
  (Edit: it seems like the algorithm does not work like I thought it did, you can basically disregard this comment. This has been mentioned on here before, but it's known CSAM possession that's illegal. Apple keeps your files encrypted until its algorithm thinks your encrypted file is too similar to CSAM, and then it decrypts it and sends it to Apple for review. There's a few things here. - The algorithm is a black box, so nobody knows how many false positives it hits. - Apple's willingness to decrypt files without the consent of the owner makes the encryption seem like a bit of a sham. - I imagine many are skeptical of Apple's ability to judge CSAM accurately. If I take a photo of my kids in a bathtub, is that CSAM? What about teenagers in a relationship sharing nudes. The law is a blunt and cruel instrument, and we've gotten away without hurting too many innocent people so far because the process is run by humans, but computers are not known for being gracious.
  - rootusrootus 5y ago
    > The algorithm is a black box So we know for sure they're not just using PhotoDNA? > If I take a photo of my kids in a bathtub, ..... Kinda the same question. If they're using PhotoDNA, then that's not really a risk, right? Isn't this technology well understood at this point?
    - frenchyatwork 5y ago
      Looks like you're right. I edited my comment. It looks like there's a couple fairly different changes that are happening: - There's a system to catch CSAM that is either PhotoDNA or something that works similarly. - There's a system to detect novel nudes, and notify parents if their children view them. I think I got these two mixed together.
      - rootusrootus 5y ago
        > I think I got these two mixed together. That's fair. Apple did a shit job of explaining themselves, and it has been compounded by a lot of misinformation (deliberate or not) in response. I'm trying really, really hard to moderate my reaction to this whole mess until I feel like I actually understand what Apple intends to do. I don't make platform jumps lightly.
- cwkoss 5y ago
  Should every minecraft server be checking if any arbitrary sequence of blocks on that server can encode to a binary representation of CSAM which when hashed matches something in the NCMEC database? You could argue that a minecraft server is technically in possession of CSAM if that's the case, but you could spend an infinite amount of money looking at various possible sequences and are bound to find many more false positives than true positives. Services should have a duty to report CSAM when they notice it, but the lengths they should go to search for CSAM should be limited by cost/benefit and privacy concerns.
- nonbirithm 5y ago
  My impression is that letting people upload CSAM to a cloud service has no positive benefit because of the supposed link between CSAM consumption and CSA, and it carries a very high risk of criminal liability, so there's no incentive for companies to completely ignore the files that users upload. Otherwise, people will eventually notice and the service will be denounced as "pedophile-friendly," and then the law will take notice and force them to give up the data. This type of scenario is what happened with the messaging service Kik, which was reportedly used to distribute CSAM in private chats. Law enforcement agencies said the company wasn't providing timely responses and that children were being actively abused as a result. This is about as damaging of an accusation you can leverage against a company. Laws against CSAM worldwide are not going away for good reasons, so there is always going to be a justifiable argument that storing certain classes of data is illegal. Hence, anyone wanting to run a cloud service that stores user data will have to obey by those laws, regardless of how proactive they are in scanning for the material. Absolute privacy in the cloud is impossible to achieve with those rules in place.
secondaryacct 5y ago
But at least you sent them away and can have reasonable expectation to be betrayed. But that your camera device also scans and report what it doesn't like.
throwaway81523 5y ago
According to Krawetz's article, Apple is not allowed to scan the files on their servers without a warrant. That's why they want to run the scan on your phone instead.

rootusrootus 5y ago

What does the law say, though? Possession of CSAM by any organization or person other than NCMEC is flatly illegal. Even other branches of government, including law enforcement, may not have any in their possession. My question is -- does CSAM residing on Apple's servers, even when it is 'owned' by a customer, count as them possessing it? What about if it is encrypted?

fakedang 5y ago
Ben's Stratechery article explains the distinction: > (f)Protection of Privacy.—Nothing in this section shall be construed to require a provider to— (1) monitor any user, subscriber, or customer of that provider; (2) monitor the content of any communication of any person described in paragraph (1); or (3) affirmatively search, screen, or scan for facts or circumstances described in sections (a) and (b). https://stratechery.com/2021/apples-mistake/ https://stratechery.com/2021/apples-mistake/
short_sells_poo 5y ago
If it is encrypted, the data storage provider has no chance to know what it is. Which is exactly why these policies are so dim witted. Dragnet violation of everyone’s privacy while anyone even remotely sophisticated can easily evade it by just encrypting the data upfront.
frenchyatwork 5y ago
(Edit: it seems like the algorithm does not work like I thought it did, you can basically disregard this comment. This has been mentioned on here before, but it's known CSAM possession that's illegal. Apple keeps your files encrypted until its algorithm thinks your encrypted file is too similar to CSAM, and then it decrypts it and sends it to Apple for review. There's a few things here. - The algorithm is a black box, so nobody knows how many false positives it hits. - Apple's willingness to decrypt files without the consent of the owner makes the encryption seem like a bit of a sham. - I imagine many are skeptical of Apple's ability to judge CSAM accurately. If I take a photo of my kids in a bathtub, is that CSAM? What about teenagers in a relationship sharing nudes. The law is a blunt and cruel instrument, and we've gotten away without hurting too many innocent people so far because the process is run by humans, but computers are not known for being gracious.
- rootusrootus 5y ago
  > The algorithm is a black box So we know for sure they're not just using PhotoDNA? > If I take a photo of my kids in a bathtub, ..... Kinda the same question. If they're using PhotoDNA, then that's not really a risk, right? Isn't this technology well understood at this point?
  - frenchyatwork 5y ago
    Looks like you're right. I edited my comment. It looks like there's a couple fairly different changes that are happening: - There's a system to catch CSAM that is either PhotoDNA or something that works similarly. - There's a system to detect novel nudes, and notify parents if their children view them. I think I got these two mixed together.
    - rootusrootus 5y ago
      > I think I got these two mixed together. That's fair. Apple did a shit job of explaining themselves, and it has been compounded by a lot of misinformation (deliberate or not) in response. I'm trying really, really hard to moderate my reaction to this whole mess until I feel like I actually understand what Apple intends to do. I don't make platform jumps lightly.
cwkoss 5y ago
Should every minecraft server be checking if any arbitrary sequence of blocks on that server can encode to a binary representation of CSAM which when hashed matches something in the NCMEC database? You could argue that a minecraft server is technically in possession of CSAM if that's the case, but you could spend an infinite amount of money looking at various possible sequences and are bound to find many more false positives than true positives. Services should have a duty to report CSAM when they notice it, but the lengths they should go to search for CSAM should be limited by cost/benefit and privacy concerns.
nonbirithm 5y ago
My impression is that letting people upload CSAM to a cloud service has no positive benefit because of the supposed link between CSAM consumption and CSA, and it carries a very high risk of criminal liability, so there's no incentive for companies to completely ignore the files that users upload. Otherwise, people will eventually notice and the service will be denounced as "pedophile-friendly," and then the law will take notice and force them to give up the data. This type of scenario is what happened with the messaging service Kik, which was reportedly used to distribute CSAM in private chats. Law enforcement agencies said the company wasn't providing timely responses and that children were being actively abused as a result. This is about as damaging of an accusation you can leverage against a company. Laws against CSAM worldwide are not going away for good reasons, so there is always going to be a justifiable argument that storing certain classes of data is illegal. Hence, anyone wanting to run a cloud service that stores user data will have to obey by those laws, regardless of how proactive they are in scanning for the material. Absolute privacy in the cloud is impossible to achieve with those rules in place.

secondaryacct 5y ago

But at least you sent them away and can have reasonable expectation to be betrayed. But that your camera device also scans and report what it doesn't like.

throwaway81523 5y ago

According to Krawetz's article, Apple is not allowed to scan the files on their servers without a warrant. That's why they want to run the scan on your phone instead.