5 ms·

If you read their Security Threat Model Review [1] they're only using the "intersection of hashes provided by at least two child safety organizations operating

If you read their Security Threat Model Review [1] they're only using the "intersection of hashes provided by at least two child safety organizations operating in separate sovereign jurisdictions". So you'd have to pressure NEMEC and another org under a different government to both add the non-CSAM hash, plus Apple would need to be pressured to verify a non-CSAM derivative image, plus you'd need other hash matches on-device to exceed the threshold before they could even do the review in the first place (they can't even tell if there was a match unless the threshold is exceeded). I get why people are concerned, but between this thread and the other thread yesterday it's clear that pretty much everyone discussing this has no idea how it works. 1: https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model...

jjulius 5y ago
I think you're missing what concerns people; what guarantee do we have that Apple will, always and forever, only use the "intersection of hashes provided by at least two child safety organizations operating in separate sovereign jurisdictions"?
- karlshea 5y ago
  It’s almost like you didn’t read the paper or my whole comment.
  - deleted 5y ago
    [deleted]
  - jjulius 5y ago
    I've actually read your comment, along with the linked PDF, in full multiple times in an effort to make sure I wasn't missing a point you were trying to make. Before I continue, I want to make it clear that I fully understand that adding non-CSAM content into the existing NCMEC/CSAM/IWF environment would more than likely raise a lot of red flags. You and I have an understanding there. As it happens, I still think you're missing the point that myself and others are trying to make. Perhaps the following is a better way of phrasing my inquiry: What guarantees to we have that Apple will, always and forever, only use the NCMEC/CSAM/IWF system to scan images on phones? By that I mean, isn't it perfectly plausible that Apple can leave the NCMEC/CSAM/IWF system in place, as it is described in your post and the linked PDF, and at the same time partner with [random body X/Y/Z], who has their own database of [whatever] that they scan separately from the NCMEC/CSAM/IWF system? These two different scans wouldn't ever need to communicate with each other, and could run at the same time.
    - karlshea 5y ago
      Well yeah it's plausible they would just turn the contents of everyone's entire iCloud backups over to the FBI, which they can totally do right now at any time since they already have the keys! So what? What you're describing is not what this system does. You could argue against literally any possibility anyone could dream up, because they're the platform vendor and they can make any change they want. If that's a problem when you're evaluating your threat model then this new system is the least of your worries. And as far as I can see this system is the least intrusive version of what any of the other cloud vendors are doing. Don't want it? Turn off iCloud Photos and sync to something else like your own Nextcloud instance.
      - jjulius 5y ago
        >What you're describing is not what this system does. I said that the hypothetical system could be set up, and function, in the same way as the NCMEC/CSAM/IWF system, just with different entities behind it. Despite chastising me for what you thought was a failure to comprehend your post, this is the third time you seem to have been unable to grasp that concept. That's part of what people are concerned about - an identical system running in tandem to this, but with less savory characters behind it. Given how many comments on HN have addressed that this week, including all of mine right here, I am at a loss for how to make that any clearer to you.
        karlshea 5y ago
        Hypothetical system. Yes, a lot of comments have addressed something that isn't happening.
        jjulius 5y ago
        >Hypothetical system. Right! >Yes, a lot of comments have addressed something that isn't happening. No, and the confusion lies in the tense, "isn't" vs. "won't". A lot of comments have addressed a lack of trust that such a scenario won't happen.
- tylerchr 5y ago
  We have the same guarantee about that today as we did a month ago before this was announced. Apple writes the software that runs their devices, so they’ll always have the option to ship something like that. Apple could have quietly implemented CSAM scanning server-side, and left the door open to it being quietly exploited in who knows what way. But they didn’t: instead they put a whole bunch of infrastructure in place that all but guarantees that they’ll be immediately caught (and publicly excoriated) if they try to use this CSAM mechanism for anything other than CSAM. (See the PDF that GP linked for technical details on why.) Of course, they could still do it with some other mechanism. But in that case none of these CSAM changes are at all relevant to the concern, as that risk is unchanged from a month ago.
  - zimpenfish 5y ago
    > Apple could have quietly implemented CSAM scanning server-side Aren't iCloud Photos already scanned for CSAM though?
    - zepto 5y ago
      No
      - zimpenfish 5y ago
        Their privacy page[1] certainly suggests they have been since 2019 - "pre-screening or scanning uploaded content for potentially illegal content, including child sexual exploitation material". Do you have some source to indicate definitively that they have not been scanning iCloud Photos for CSAM? [1] https://web.archive.org/web/20190701000647/https://www.apple.com/legal/privacy/en-ww/ https://web.archive.org/web/20190701000647/https://www.apple...
        zepto 5y ago
        Craig Federighi said they don’t in a recent interview.
zimpenfish 5y ago
> So you'd have to pressure NEMEC and another org under a different government To be fair, if the other organisation is IWF[1] under the UK government, I don't think there'd be much pressure needed to get them to comply - just offer to bung them and their mates a few million in contracts and you'd be golden. It's a sensible plan, it just might not be as strong as it seems. [1] https://www.iwf.org.uk https://www.iwf.org.uk
- karlshea 5y ago
  Looks like you forgot to take into account the entire rest of that sentence you quoted.
  - zimpenfish 5y ago
    No, just pointing out that the first part isn't as strong as Apple seem to think it might be. The rest of it might well catch this, yes, (and that's good!) but it doesn't obviate that "only considering images from two or more providers" can be subverted, possibly easily, given the pressure USGOV can bring to bear on their partners.

jjulius 5y ago

I think you're missing what concerns people; what guarantee do we have that Apple will, always and forever, only use the "intersection of hashes provided by at least two child safety organizations operating in separate sovereign jurisdictions"?

karlshea 5y ago
It’s almost like you didn’t read the paper or my whole comment.
- deleted 5y ago
  [deleted]
- jjulius 5y ago
  I've actually read your comment, along with the linked PDF, in full multiple times in an effort to make sure I wasn't missing a point you were trying to make. Before I continue, I want to make it clear that I fully understand that adding non-CSAM content into the existing NCMEC/CSAM/IWF environment would more than likely raise a lot of red flags. You and I have an understanding there. As it happens, I still think you're missing the point that myself and others are trying to make. Perhaps the following is a better way of phrasing my inquiry: What guarantees to we have that Apple will, always and forever, only use the NCMEC/CSAM/IWF system to scan images on phones? By that I mean, isn't it perfectly plausible that Apple can leave the NCMEC/CSAM/IWF system in place, as it is described in your post and the linked PDF, and at the same time partner with [random body X/Y/Z], who has their own database of [whatever] that they scan separately from the NCMEC/CSAM/IWF system? These two different scans wouldn't ever need to communicate with each other, and could run at the same time.
  - karlshea 5y ago
    Well yeah it's plausible they would just turn the contents of everyone's entire iCloud backups over to the FBI, which they can totally do right now at any time since they already have the keys! So what? What you're describing is not what this system does. You could argue against literally any possibility anyone could dream up, because they're the platform vendor and they can make any change they want. If that's a problem when you're evaluating your threat model then this new system is the least of your worries. And as far as I can see this system is the least intrusive version of what any of the other cloud vendors are doing. Don't want it? Turn off iCloud Photos and sync to something else like your own Nextcloud instance.
    - jjulius 5y ago
      >What you're describing is not what this system does. I said that the hypothetical system could be set up, and function, in the same way as the NCMEC/CSAM/IWF system, just with different entities behind it. Despite chastising me for what you thought was a failure to comprehend your post, this is the third time you seem to have been unable to grasp that concept. That's part of what people are concerned about - an identical system running in tandem to this, but with less savory characters behind it. Given how many comments on HN have addressed that this week, including all of mine right here, I am at a loss for how to make that any clearer to you.
      - karlshea 5y ago
        Hypothetical system. Yes, a lot of comments have addressed something that isn't happening.
        jjulius 5y ago
        >Hypothetical system. Right! >Yes, a lot of comments have addressed something that isn't happening. No, and the confusion lies in the tense, "isn't" vs. "won't". A lot of comments have addressed a lack of trust that such a scenario won't happen.
tylerchr 5y ago
We have the same guarantee about that today as we did a month ago before this was announced. Apple writes the software that runs their devices, so they’ll always have the option to ship something like that. Apple could have quietly implemented CSAM scanning server-side, and left the door open to it being quietly exploited in who knows what way. But they didn’t: instead they put a whole bunch of infrastructure in place that all but guarantees that they’ll be immediately caught (and publicly excoriated) if they try to use this CSAM mechanism for anything other than CSAM. (See the PDF that GP linked for technical details on why.) Of course, they could still do it with some other mechanism. But in that case none of these CSAM changes are at all relevant to the concern, as that risk is unchanged from a month ago.
- zimpenfish 5y ago
  > Apple could have quietly implemented CSAM scanning server-side Aren't iCloud Photos already scanned for CSAM though?
  - zepto 5y ago
    No
    - zimpenfish 5y ago
      Their privacy page[1] certainly suggests they have been since 2019 - "pre-screening or scanning uploaded content for potentially illegal content, including child sexual exploitation material". Do you have some source to indicate definitively that they have not been scanning iCloud Photos for CSAM? [1] https://web.archive.org/web/20190701000647/https://www.apple.com/legal/privacy/en-ww/ https://web.archive.org/web/20190701000647/https://www.apple...
      - zepto 5y ago
        Craig Federighi said they don’t in a recent interview.

zimpenfish 5y ago

> So you'd have to pressure NEMEC and another org under a different government To be fair, if the other organisation is IWF[1] under the UK government, I don't think there'd be much pressure needed to get them to comply - just offer to bung them and their mates a few million in contracts and you'd be golden. It's a sensible plan, it just might not be as strong as it seems. [1] https://www.iwf.org.uk https://www.iwf.org.uk

karlshea 5y ago
Looks like you forgot to take into account the entire rest of that sentence you quoted.
- zimpenfish 5y ago
  No, just pointing out that the first part isn't as strong as Apple seem to think it might be. The rest of it might well catch this, yes, (and that's good!) but it doesn't obviate that "only considering images from two or more providers" can be subverted, possibly easily, given the pressure USGOV can bring to bear on their partners.