4 ms·

“It’s the thin end of the wedge” This is the biggest worry. Apple essentially waved their hand in their FAQ and said something to the effect of they’d never do

“It’s the thin end of the wedge” This is the biggest worry. Apple essentially waved their hand in their FAQ and said something to the effect of they’d never do something like that. Yes you would. You’ll get called to congress and told in no uncertain terms what you’re expected to do, you’ll say no, the law will be changed, then the gentleman in a suit that people don’t say no to will call to your nice headquarters for you to walk him through his shiny new surveillance tools.

avalys 5y ago
Yes, Apple's defense is literally the "If you're not doing anything wrong you have nothing to worry about" line that is universally recognized as laughable.
- headmelted 5y ago
  Not only that, when they suggest that morally they would or would not do something they’re either being dishonest (which to be fair I don’t think is the case) or genuinely naive as to the limits of their control here, even over their own products. Once the tool exists, they’ll add whatever features governments in major economies tell them to or they’ll be forced into adding them. All this said, I don’t blame Apple alone here and I have mixed feelings about it still - as I really do believe in their fight against CSAM. I just don’t think this is the magic solution to solving CSAM while preserving an innocent user’s right to privacy that they’re selling it as. I’m also certain there are engineers who worked on this system who also share these concerns.
kmeisthax 5y ago
Congress has already gone a few steps into your hypothetical. The EARN IT Act[0] proposed to ban E2E encryption based entirely on making companies criminally liable for any CSAM they can't scan for. While EARN IT wasn't passed, I wouldn't be surprised if Apple sees the writing on the wall and figured they'd get out ahead of things. I also feel it's important to note that every major image host is doing the same thing Apple is doing, but server-side on unencrypted images. What Apple is planning to implement is, out of context, an improvement; in the same way that Google's FLoC is an improvement over ad tracking based on third-party cookies. Get a trusted client to cooperate in some smaller act of self-surveillance and we don't need the server to do full take. That being said, I don't think we even need to rely on governments threatening Apple into doing surveillance. The scanning scheme relies on paying human moderators to basically look at CSAM all day and check if it's a valid match. Big tech moderation is already understaffed and overworked as is - so it's almost certain that the people involved will just forward everything as their sanity degrades. Apple will almost certainly wind up accidentally or unknowingly forwarding along bogus CSAM reports, no coercion required. [0] https://en.wikipedia.org/wiki/EARN_IT_Act_of_2020 https://en.wikipedia.org/wiki/EARN_IT_Act_of_2020

avalys 5y ago

Yes, Apple's defense is literally the "If you're not doing anything wrong you have nothing to worry about" line that is universally recognized as laughable.

headmelted 5y ago
Not only that, when they suggest that morally they would or would not do something they’re either being dishonest (which to be fair I don’t think is the case) or genuinely naive as to the limits of their control here, even over their own products. Once the tool exists, they’ll add whatever features governments in major economies tell them to or they’ll be forced into adding them. All this said, I don’t blame Apple alone here and I have mixed feelings about it still - as I really do believe in their fight against CSAM. I just don’t think this is the magic solution to solving CSAM while preserving an innocent user’s right to privacy that they’re selling it as. I’m also certain there are engineers who worked on this system who also share these concerns.

kmeisthax 5y ago

Congress has already gone a few steps into your hypothetical. The EARN IT Act[0] proposed to ban E2E encryption based entirely on making companies criminally liable for any CSAM they can't scan for. While EARN IT wasn't passed, I wouldn't be surprised if Apple sees the writing on the wall and figured they'd get out ahead of things. I also feel it's important to note that every major image host is doing the same thing Apple is doing, but server-side on unencrypted images. What Apple is planning to implement is, out of context, an improvement; in the same way that Google's FLoC is an improvement over ad tracking based on third-party cookies. Get a trusted client to cooperate in some smaller act of self-surveillance and we don't need the server to do full take. That being said, I don't think we even need to rely on governments threatening Apple into doing surveillance. The scanning scheme relies on paying human moderators to basically look at CSAM all day and check if it's a valid match. Big tech moderation is already understaffed and overworked as is - so it's almost certain that the people involved will just forward everything as their sanity degrades. Apple will almost certainly wind up accidentally or unknowingly forwarding along bogus CSAM reports, no coercion required. [0] https://en.wikipedia.org/wiki/EARN_IT_Act_of_2020 https://en.wikipedia.org/wiki/EARN_IT_Act_of_2020