3 ms·

A very relevant point on this entire discourse about Apple’s on-device CSAM scanning: According to the U.S. law, key snippets of which are quoted on the Strate

by AnonC 5y ago

A very relevant point on this entire discourse about Apple’s on-device CSAM scanning: According to the U.S. law, key snippets of which are quoted on the Stratechery blog (by Ben Thompson), Apple isn’t obligated to scan for CSAM. It’s only obligated to act on CSAM if it finds them. While it’s good for Apple to scan on its systems (iCloud) like Facebook, Google and other companies do on their servers, it’s inappropriate to do it on individual devices, which starts with the assumption that anyone who has iCloud photos enabled is a potential CSAM hoarder and needs to pay with their device’s battery life and time for the scanning to happen and report back. It’s a sort of micro-robbery that Apple is doing on the devices when there is no legal compulsion to do so. Everything else on trusting Apple’s NeuralHash or the sanctity of the NCMEC hashes come later, IMO. I sincerely hope Apple realizes that it’s got a dud solution on hand, eats humble pie (which it’s usually not capable of) and ditches this whole thing. I know a lot of egos at Apple are at stake here. But doing the right thing matters for a company that claims that “privacy is a fundamental human right” and has a CEO who’s a member of a marginalized/discriminated community and understands the risks of these efforts.

Lamad123 5y ago
They say only if they find 30 matching images, they'd act. So if they find 20 or 29 and don't report them, they are actually breaking the law!! I am wondering why they chose that magical number!
- Tagbert 5y ago
  Just getting a match on the neural hash is not sufficient to declare an image CSAM. It just flags it as a candidate for review. Apple is not required to report it unless it can be verified to be CSAM. there is a multi-step process involving multiple processes and organizations to do so.
  - Lamad123 5y ago
    They are opening the door for a lot of questions like this. They didn't say (as far as I know) if this 30-number cover multiply upload sessions or it gets reset with every upload. I feel like this number gets reset to 0 each for each upload session. Somebody can deliberately safely upload 29 actual CSAM photos each time. I know Apple is not stupid, but you know the government and lawmakers will ask it for a smaller magical number...
- nullc 5y ago
  They think they've outsmarted the law on that one. The system is setup to generate a bunch of false positives on its own, so a "match" which is below threshold may not actually be a match-- apple can plausibly deny it. It's unclear if their claimed threshold of 30 is before or after the false positives they intentionally introduce. I'm going to guess it's before.
fleddr 5y ago
"While it’s good for Apple to scan on its systems (iCloud) like Facebook, Google and other companies do on their servers, it’s inappropriate to do it on individual devices" I would even challenge the justification to do this on servers, unless the data is public. If it's behind a personal login, you might as well consider it personal property/data. I find the distinction of where data is stored not very meaningful. Allowing things to be searched for criminal content just because it's not in your immediate physical sphere makes no sense. It doesn't work like that in the physical world either. When I send a letter, and it leaves my house, no authority has the right to check its contents without a legitimate reason. Likewise, if I put stuff in a storage box in some warehouse, no authority can search it without a warrant. Note that I'm talking about personal storage (iCloud, Gmail), not public social networks like Facebook.
- hackinthebochs 5y ago
  Absolutely. If I own my data, someone processing this data on my behalf has no right or obligation to scan it for illegal content. The fact that this data sometimes sits on hard drives owned by another party just isn't a relevant factor. Presumably I still own my car when it sits in the garage at the shop. They have no right or obligation to rummage around looking for evidence of a crime. I don't see abstract data as any different. Our major privacy blunder was accepting scanning of private data in any context. The fight should be for the absolute privacy of personal data. Where the scanning happens is mostly irrelevant.
  - fleddr 5y ago
    Fully agree. Only this week did I learn that companies were already doing this on servers since at least 2018-2019. Can't say I ever read anything about it before. I find it quite shocking that a foundational element of criminal justice, innocent until proven guilty and needing a reason to search individual property, is tossed aside like it's nothing.

by AnonC 5y ago

Lamad123 5y ago

They say only if they find 30 matching images, they'd act. So if they find 20 or 29 and don't report them, they are actually breaking the law!! I am wondering why they chose that magical number!

Tagbert 5y ago
Just getting a match on the neural hash is not sufficient to declare an image CSAM. It just flags it as a candidate for review. Apple is not required to report it unless it can be verified to be CSAM. there is a multi-step process involving multiple processes and organizations to do so.
- Lamad123 5y ago
  They are opening the door for a lot of questions like this. They didn't say (as far as I know) if this 30-number cover multiply upload sessions or it gets reset with every upload. I feel like this number gets reset to 0 each for each upload session. Somebody can deliberately safely upload 29 actual CSAM photos each time. I know Apple is not stupid, but you know the government and lawmakers will ask it for a smaller magical number...
nullc 5y ago
They think they've outsmarted the law on that one. The system is setup to generate a bunch of false positives on its own, so a "match" which is below threshold may not actually be a match-- apple can plausibly deny it. It's unclear if their claimed threshold of 30 is before or after the false positives they intentionally introduce. I'm going to guess it's before.

fleddr 5y ago

"While it’s good for Apple to scan on its systems (iCloud) like Facebook, Google and other companies do on their servers, it’s inappropriate to do it on individual devices" I would even challenge the justification to do this on servers, unless the data is public. If it's behind a personal login, you might as well consider it personal property/data. I find the distinction of where data is stored not very meaningful. Allowing things to be searched for criminal content just because it's not in your immediate physical sphere makes no sense. It doesn't work like that in the physical world either. When I send a letter, and it leaves my house, no authority has the right to check its contents without a legitimate reason. Likewise, if I put stuff in a storage box in some warehouse, no authority can search it without a warrant. Note that I'm talking about personal storage (iCloud, Gmail), not public social networks like Facebook.

hackinthebochs 5y ago
Absolutely. If I own my data, someone processing this data on my behalf has no right or obligation to scan it for illegal content. The fact that this data sometimes sits on hard drives owned by another party just isn't a relevant factor. Presumably I still own my car when it sits in the garage at the shop. They have no right or obligation to rummage around looking for evidence of a crime. I don't see abstract data as any different. Our major privacy blunder was accepting scanning of private data in any context. The fight should be for the absolute privacy of personal data. Where the scanning happens is mostly irrelevant.
- fleddr 5y ago
  Fully agree. Only this week did I learn that companies were already doing this on servers since at least 2018-2019. Can't say I ever read anything about it before. I find it quite shocking that a foundational element of criminal justice, innocent until proven guilty and needing a reason to search individual property, is tossed aside like it's nothing.