3 ms·

I think the author, amongst others, speculates that Apple will announce E2E encryption of your iCloud data, such as backups, in the future. This is not an unrea

I think the author, amongst others, speculates that Apple will announce E2E encryption of your iCloud data, such as backups, in the future. This is not an unreasonable speculation, nor does it imply support of the whole CSAM scheme. BTW: I've been an Apple fanboy since the mid-80s, but I find the whole concept of my phone suspecting me of child pornography appalling. It's the thin end of the wedge, and IMHO Apple have made a huge mistake.

headmelted 5y ago
“It’s the thin end of the wedge” This is the biggest worry. Apple essentially waved their hand in their FAQ and said something to the effect of they’d never do something like that. Yes you would. You’ll get called to congress and told in no uncertain terms what you’re expected to do, you’ll say no, the law will be changed, then the gentleman in a suit that people don’t say no to will call to your nice headquarters for you to walk him through his shiny new surveillance tools.
- avalys 5y ago
  Yes, Apple's defense is literally the "If you're not doing anything wrong you have nothing to worry about" line that is universally recognized as laughable.
  - headmelted 5y ago
    Not only that, when they suggest that morally they would or would not do something they’re either being dishonest (which to be fair I don’t think is the case) or genuinely naive as to the limits of their control here, even over their own products. Once the tool exists, they’ll add whatever features governments in major economies tell them to or they’ll be forced into adding them. All this said, I don’t blame Apple alone here and I have mixed feelings about it still - as I really do believe in their fight against CSAM. I just don’t think this is the magic solution to solving CSAM while preserving an innocent user’s right to privacy that they’re selling it as. I’m also certain there are engineers who worked on this system who also share these concerns.
- kmeisthax 5y ago
  Congress has already gone a few steps into your hypothetical. The EARN IT Act[0] proposed to ban E2E encryption based entirely on making companies criminally liable for any CSAM they can't scan for. While EARN IT wasn't passed, I wouldn't be surprised if Apple sees the writing on the wall and figured they'd get out ahead of things. I also feel it's important to note that every major image host is doing the same thing Apple is doing, but server-side on unencrypted images. What Apple is planning to implement is, out of context, an improvement; in the same way that Google's FLoC is an improvement over ad tracking based on third-party cookies. Get a trusted client to cooperate in some smaller act of self-surveillance and we don't need the server to do full take. That being said, I don't think we even need to rely on governments threatening Apple into doing surveillance. The scanning scheme relies on paying human moderators to basically look at CSAM all day and check if it's a valid match. Big tech moderation is already understaffed and overworked as is - so it's almost certain that the people involved will just forward everything as their sanity degrades. Apple will almost certainly wind up accidentally or unknowingly forwarding along bogus CSAM reports, no coercion required. [0] https://en.wikipedia.org/wiki/EARN_IT_Act_of_2020 https://en.wikipedia.org/wiki/EARN_IT_Act_of_2020
snarf21 5y ago
I agree Apple is giving in just like they did in China. However, they were already doing this on the server. Wasn't that already the thin edge of the wedge?
Spivak 5y ago
At this point "my phone" is a pretty meaningless distinction. It's just Apple's phone as a service. For whatever reason people have drawn this, to me arbitrary, line where it's decided that it's not your MySpace page or your Squarespace site just because someone else distributing it on your behalf but once you hold it in your hand that rule can't apply because reasons. It's been a decade and a half of software not being the user's agent and instead some company's agent that in some cases you buy the hardware for. Unless how we talk about software changes this is just the world we live in now. How many apps do you have that are genuinely extensions of your intent as a user and not company kiosks? Of probably about 100 apps on my phone I think I have maybe 5.
AnimalMuppet 5y ago
> I've been an Apple fanboy since the mid-80s, but I find the whole concept of my phone suspecting me of child pornography appalling. In The Cuckoo's Egg, Cliff Stoll tells about having his computer page him when it detected a hacking attempt. Telling about one time when his pager went off, he said, "My computer wanted me." Now we have so many notifications and suggested actions from our phones and computers, it feels like we have become computer peripherals. Our computers want us to do things. They have agency, and we're responding to it. But at least we still had agency, too. The computer had the initiative, and we could choose to respond, or not. Now, though, the computers are limiting our agency. They are becoming the controllers, and we the controlled. I don't think it's just that Apple has made a huge mistake. I think humanity is making a huge mistake. We are allowing tech to turn us into children.
kevinpet 5y ago
E2E encryption of iCloud data is completely useless if it also comes with a feature to selectively upload some of that data unencrypted for snooping purposes. Even ignoring the issue that child porn will turn into child porn or terrorism or the cause du jour, the imperfect nature of the scanning includes a review process. This is more like Youtube content id than it is like an md5sum of known CSAM files. That's the violation. Another article recently points out that even the md5sum databases include bad matches (a rhesus monkey was mentioned).
refulgentis 5y ago
It is unreasonable speculation, insomuch as it begs the question _why_ - they're unrelated propositions. There's no legal framework where youre only allowed to encrypt data if you check if it's child porn first. My biggest fear is the only reason I've seen this come up this week is the Apple community, that built me into a programmer from a waiter, has become susceptible to the same kind of irrational tribalism we see everywhere else - "surely this bad thing is happening because it's part of a secret plan to fix the other bad thing!"

headmelted 5y ago

“It’s the thin end of the wedge” This is the biggest worry. Apple essentially waved their hand in their FAQ and said something to the effect of they’d never do something like that. Yes you would. You’ll get called to congress and told in no uncertain terms what you’re expected to do, you’ll say no, the law will be changed, then the gentleman in a suit that people don’t say no to will call to your nice headquarters for you to walk him through his shiny new surveillance tools.

avalys 5y ago
Yes, Apple's defense is literally the "If you're not doing anything wrong you have nothing to worry about" line that is universally recognized as laughable.
- headmelted 5y ago
  Not only that, when they suggest that morally they would or would not do something they’re either being dishonest (which to be fair I don’t think is the case) or genuinely naive as to the limits of their control here, even over their own products. Once the tool exists, they’ll add whatever features governments in major economies tell them to or they’ll be forced into adding them. All this said, I don’t blame Apple alone here and I have mixed feelings about it still - as I really do believe in their fight against CSAM. I just don’t think this is the magic solution to solving CSAM while preserving an innocent user’s right to privacy that they’re selling it as. I’m also certain there are engineers who worked on this system who also share these concerns.
kmeisthax 5y ago
Congress has already gone a few steps into your hypothetical. The EARN IT Act[0] proposed to ban E2E encryption based entirely on making companies criminally liable for any CSAM they can't scan for. While EARN IT wasn't passed, I wouldn't be surprised if Apple sees the writing on the wall and figured they'd get out ahead of things. I also feel it's important to note that every major image host is doing the same thing Apple is doing, but server-side on unencrypted images. What Apple is planning to implement is, out of context, an improvement; in the same way that Google's FLoC is an improvement over ad tracking based on third-party cookies. Get a trusted client to cooperate in some smaller act of self-surveillance and we don't need the server to do full take. That being said, I don't think we even need to rely on governments threatening Apple into doing surveillance. The scanning scheme relies on paying human moderators to basically look at CSAM all day and check if it's a valid match. Big tech moderation is already understaffed and overworked as is - so it's almost certain that the people involved will just forward everything as their sanity degrades. Apple will almost certainly wind up accidentally or unknowingly forwarding along bogus CSAM reports, no coercion required. [0] https://en.wikipedia.org/wiki/EARN_IT_Act_of_2020 https://en.wikipedia.org/wiki/EARN_IT_Act_of_2020

snarf21 5y ago

I agree Apple is giving in just like they did in China. However, they were already doing this on the server. Wasn't that already the thin edge of the wedge?

Spivak 5y ago

At this point "my phone" is a pretty meaningless distinction. It's just Apple's phone as a service. For whatever reason people have drawn this, to me arbitrary, line where it's decided that it's not your MySpace page or your Squarespace site just because someone else distributing it on your behalf but once you hold it in your hand that rule can't apply because reasons. It's been a decade and a half of software not being the user's agent and instead some company's agent that in some cases you buy the hardware for. Unless how we talk about software changes this is just the world we live in now. How many apps do you have that are genuinely extensions of your intent as a user and not company kiosks? Of probably about 100 apps on my phone I think I have maybe 5.

AnimalMuppet 5y ago

> I've been an Apple fanboy since the mid-80s, but I find the whole concept of my phone suspecting me of child pornography appalling. In The Cuckoo's Egg, Cliff Stoll tells about having his computer page him when it detected a hacking attempt. Telling about one time when his pager went off, he said, "My computer wanted me." Now we have so many notifications and suggested actions from our phones and computers, it feels like we have become computer peripherals. Our computers want us to do things. They have agency, and we're responding to it. But at least we still had agency, too. The computer had the initiative, and we could choose to respond, or not. Now, though, the computers are limiting our agency. They are becoming the controllers, and we the controlled. I don't think it's just that Apple has made a huge mistake. I think humanity is making a huge mistake. We are allowing tech to turn us into children.

kevinpet 5y ago

E2E encryption of iCloud data is completely useless if it also comes with a feature to selectively upload some of that data unencrypted for snooping purposes. Even ignoring the issue that child porn will turn into child porn or terrorism or the cause du jour, the imperfect nature of the scanning includes a review process. This is more like Youtube content id than it is like an md5sum of known CSAM files. That's the violation. Another article recently points out that even the md5sum databases include bad matches (a rhesus monkey was mentioned).

refulgentis 5y ago

It is unreasonable speculation, insomuch as it begs the question _why_ - they're unrelated propositions. There's no legal framework where youre only allowed to encrypt data if you check if it's child porn first. My biggest fear is the only reason I've seen this come up this week is the Apple community, that built me into a programmer from a waiter, has become susceptible to the same kind of irrational tribalism we see everywhere else - "surely this bad thing is happening because it's part of a secret plan to fix the other bad thing!"