4 ms·

After seeing all these posts here and not hearing much about it outside of this space I’m starting to think that the cost benefit analysis that Apple did was th

by werber 5y ago

After seeing all these posts here and not hearing much about it outside of this space I’m starting to think that the cost benefit analysis that Apple did was that the profit from very likely convincing parents to spend a little more to get their children an iPhone as their first phone (and probably lock those children into their ecosystem) because of these features was greater than the amount of users who would switch to something else and that any blowback would be diminished because “what about the children”

sandworm101 5y ago
I'm not sure we have an understanding of Apple's intentions. Given the massive backlash and the apparent maturity of their plans this must have been in the works for a while. They may have been under pressure from government actors to develop a program. Or they might see writing on the wall. Either way, I doubt they will ever share the total picture behind this move.
- dionian 5y ago
  The leak of the email describing people with concerns as "screeching" didnt help.
  - theshrike79 5y ago
    The "screeching" comment was from NCMEC, not Apple.
- deleted 5y ago
  [deleted]
wizzwizz4 5y ago
And the NN totally-offline¹ optional configurable check is a good thing! Heck, I'd've enabled it on my own phone if I had an iPhone; it'd've made me feel that much better about carrying an internet-connected camera around with me wherever I went, if I knew it'd warn me if I accidentally photographed myself naked (or somehow decided to do so deliberately, in a moment of foolishness). They built two different systems, one that I think is good / that would be a selling point to parents, and one that could be a powerful weapon. ¹: The “notifying parents” thing is a little iffy; I'm not sure what to think about it.
- VortexDream 5y ago
  Honestly, this is deeply concerning. Apple shouldn't be in a position to be able to police what photos we take. If I'm a teen and I sext with a partner, does that mean that in the near future, our messages/photos will be flagged and reviewed for "child porn" and forwarded to police enforcement? That's horrifying and incredibly unfair to young people exploring their sexuality in a completely consensual and normal way. There's nothing wrong with taking pictures of yourself nude and/or sharing these with whoever you want. This goes towards territory that I find deeply uncomfortable.
  - UncleMeat 5y ago
    > If I'm a teen and I sext with a partner, does that mean that in the near future, our messages/photos will be flagged and reviewed for "child porn" and forwarded to police enforcement? No. The system looks for files that are similar to known illegal content. It is not a general purpose underage nude photograph detector.
    - wutbrodo 5y ago
      > does that mean that in the near future I think the parent comment is in reference not to the current system, but to potential future functionality, enabled by opening the Pandora's Box of client-side scanning. Though I agree that underage nudity detection in particular is a little fanciful
    - halostatue 5y ago
      That's the other feature. Feature 1: CSAM detection. Only on upload of images to iCloud Photos. Feature 2: nudity detection for minor children controlled via the parents' iCloud account. It is not an “underage” nude photograph detector, but a nude photograph detector. All done on-device. The only person(s) notified are THE PARENTS. This is all opt-in and age-restricted. I do not know whether it will affect teens sexting or only under-13s.
      - wizzwizz4 5y ago
wutbrodo 5y ago
> convincing parents to spend a little more to get their children an iPhone as their first phone What do you mean here: I don't think makes things any safer for child _users_ of iPhones, does it? It's scanning for images of child sexual abuse. (To anyone inclined to respond without reading the context, I'm not suggesting that there's no privacy concern if you're not sharing CP. I'm addressing OP's hypothetical that some parents will see owning an iPhone as safer for their child)
- halostatue 5y ago
  There are two main features that cause concern. The one that has been getting the most attention and noise is the CSAM scanning. That only happens on upload of images to iCloud Photos (currently only in the US). No automatic notification of any law enforcement agency is done, and there is a threshold requirement before Apple reviews the account and then submits a report. The second feature is iMessage specific. Parents have to enable this on their account on behalf of their children, and it only applies to minors (_probably_ just children under 13, possibly older; I am not clear on that). This looks for (mostly outbound?) messages going to the child's phone for “unsafe” images (e.g., nudity). If those messages are found, _the parent_ is notified. No one else. I have _more_ concerns about the second feature as it is likely to be abused by controlling parents and is more likely to negatively impact children who are questioning their gender or sexuality. But this is _explicitly_ a feature to protect kids from sharing sexual images of themselves.
- wutbrodo 5y ago
  Ah, thanks for the details. Though I still don't see how the parent comment's theory holds: I suspect that the PR backlash would be far smaller/nonexistent for this feature than for the CSAM one, as we already accept that the rights of minors are heavily curtailed.

by werber 5y ago

sandworm101 5y ago

I'm not sure we have an understanding of Apple's intentions. Given the massive backlash and the apparent maturity of their plans this must have been in the works for a while. They may have been under pressure from government actors to develop a program. Or they might see writing on the wall. Either way, I doubt they will ever share the total picture behind this move.

dionian 5y ago
The leak of the email describing people with concerns as "screeching" didnt help.
- theshrike79 5y ago
  The "screeching" comment was from NCMEC, not Apple.
deleted 5y ago
[deleted]

wizzwizz4 5y ago

And the NN totally-offline¹ optional configurable check is a good thing! Heck, I'd've enabled it on my own phone if I had an iPhone; it'd've made me feel that much better about carrying an internet-connected camera around with me wherever I went, if I knew it'd warn me if I accidentally photographed myself naked (or somehow decided to do so deliberately, in a moment of foolishness). They built two different systems, one that I think is good / that would be a selling point to parents, and one that could be a powerful weapon. ¹: The “notifying parents” thing is a little iffy; I'm not sure what to think about it.

VortexDream 5y ago
Honestly, this is deeply concerning. Apple shouldn't be in a position to be able to police what photos we take. If I'm a teen and I sext with a partner, does that mean that in the near future, our messages/photos will be flagged and reviewed for "child porn" and forwarded to police enforcement? That's horrifying and incredibly unfair to young people exploring their sexuality in a completely consensual and normal way. There's nothing wrong with taking pictures of yourself nude and/or sharing these with whoever you want. This goes towards territory that I find deeply uncomfortable.
- UncleMeat 5y ago
  > If I'm a teen and I sext with a partner, does that mean that in the near future, our messages/photos will be flagged and reviewed for "child porn" and forwarded to police enforcement? No. The system looks for files that are similar to known illegal content. It is not a general purpose underage nude photograph detector.
  - wutbrodo 5y ago
    > does that mean that in the near future I think the parent comment is in reference not to the current system, but to potential future functionality, enabled by opening the Pandora's Box of client-side scanning. Though I agree that underage nudity detection in particular is a little fanciful
  - halostatue 5y ago
    That's the other feature. Feature 1: CSAM detection. Only on upload of images to iCloud Photos. Feature 2: nudity detection for minor children controlled via the parents' iCloud account. It is not an “underage” nude photograph detector, but a nude photograph detector. All done on-device. The only person(s) notified are THE PARENTS. This is all opt-in and age-restricted. I do not know whether it will affect teens sexting or only under-13s.
    - wizzwizz4 5y ago

wutbrodo 5y ago

> convincing parents to spend a little more to get their children an iPhone as their first phone What do you mean here: I don't think makes things any safer for child _users_ of iPhones, does it? It's scanning for images of child sexual abuse. (To anyone inclined to respond without reading the context, I'm not suggesting that there's no privacy concern if you're not sharing CP. I'm addressing OP's hypothetical that some parents will see owning an iPhone as safer for their child)

halostatue 5y ago
There are two main features that cause concern. The one that has been getting the most attention and noise is the CSAM scanning. That only happens on upload of images to iCloud Photos (currently only in the US). No automatic notification of any law enforcement agency is done, and there is a threshold requirement before Apple reviews the account and then submits a report. The second feature is iMessage specific. Parents have to enable this on their account on behalf of their children, and it only applies to minors (_probably_ just children under 13, possibly older; I am not clear on that). This looks for (mostly outbound?) messages going to the child's phone for “unsafe” images (e.g., nudity). If those messages are found, _the parent_ is notified. No one else. I have _more_ concerns about the second feature as it is likely to be abused by controlling parents and is more likely to negatively impact children who are questioning their gender or sexuality. But this is _explicitly_ a feature to protect kids from sharing sexual images of themselves.
wutbrodo 5y ago
Ah, thanks for the details. Though I still don't see how the parent comment's theory holds: I suspect that the PR backlash would be far smaller/nonexistent for this feature than for the CSAM one, as we already accept that the rights of minors are heavily curtailed.