4 ms·

I think the author of the article got a few things wrong. As far as I understood, SafetyCore is not "scanning all your photos". It is an API that apps on your p

I think the author of the article got a few things wrong. As far as I understood, SafetyCore is not "scanning all your photos". It is an API that apps on your phone CAN use to LOCALLY detect nudity and other potentially inappropriate things in photos.

theshrike79 2y ago
Just like the Apple system COULD LOCALLY detect CSAM on your phone. And it was the Worst Thing Ever according to hackernews and the internet at large. Now when Google does it, it's just fine and dandy.
- gumby271 2y ago
  I'm pretty sure what Apple proposed was an offline system for scanning, but any matches would be submitted to them. This is just the offline parts, and apps can choose to use it to detect nudity or other NSFW images. There's no scanning going on at all actually.
  - JimDabell 2y ago
    This is not correct. This is how most people assumed the Apple proposal worked, but it actually worked in a very different way. The device never knew if an image matched. Matches could only be determined via the combination of a receipt calculated on the device, plus information on the server, plus meeting a threshold of many matches. It was not offline scanning and uploading matches.
    - fsflover 2y ago
      Does it matter? A tiny and invisible update would be able to change that in no time and with no possibility for a user to opt out.
      - JimDabell 2y ago
        This is true of literally any software with auto updates. You’re criticising them for something they did not do, did not intend to do, and designed a system that worked in an entirely different way… just because they could do it differently to what they actually proposed doing. If they wanted to do it that other way, they could have just done it that other way in the first place and saved themselves a lot of effort.
        fsflover 2y ago
        It's only true for proprietary software, which actively prevents users from studying it. Especially when the trust in company plunges due to the enshittification, https://pluralistic.net/2025/02/26/ursula-franklin/ https://pluralistic.net/2025/02/26/ursula-franklin/ and https://news.ycombinator.com/item?id=43243075 https://news.ycombinator.com/item?id=43243075
        JimDabell 2y ago
        > It's only true for proprietary software No, this is not true. There is nothing stopping open-source software from pushing malicious updates. But you are avoiding my main point. You are criticising Apple for something they haven’t done and had no plans to do.
        fsflover 2y ago
        > There is nothing stopping open-source software from pushing malicious updates. Two things stop it: (much simplified) oversight by the community and a possibility to fix the code. > You are criticising Apple for something they haven’t done and had no plans to do. The do participate in the enshittification (https://pluralistic.net/2025/02/26/ursula-franklin/ https://pluralistic.net/2025/02/26/ursula-franklin/ and https://news.ycombinator.com/item?id=43243075 https://news.ycombinator.com/item?id=43243075), so my only expectation is that they won't do what's best for the users. In addition, they removed many features requested by the users like the headphone jack. In general, the less you trust in a company, the better. Free software allows to decrease the trust in the vendor by watching the code and forking whenever you have to.
    - gumby271 2y ago
      Oh that's right, they would upload a hash basically then flag it using whatever criteria they wanted on the server side. I think some of the outrage too was the question of what happens when they detect something, it was the issue of it reporting content outside of your control at all, which again, this Google thing doesnt seem to do.
      - JimDabell 2y ago
        > Oh that's right, they would upload a hash basically then flag it using whatever criteria they wanted on the server side. No, that’s not right either. You should really read their white paper, it’s very interesting and not at all what people assumed it was like.
        gumby271 2y ago
        That's fair, but the point is still that "The Worst Thing Ever" was the online part, which this doesn't do, so the comparison makes little to no sense. I'll check out their write up though, sounds interesting.
        deleted 2y ago
        [deleted]
      - jodrellblank 2y ago
        The proposal was to scan photos people were uploading to iCloud not all photos[1]. The panic on HN was a) misunderstanding or deliberately misrepresenting the proposal as if it was scanning all offline photos, b) fantasising what if they don't do what they announced and instead scan all offline photos, c) realising that all Silicon Valley tech companies can change client software through updates, therefore Apple bad. [There were non-panicky comments about whether it's a legally significant move, whether it's a wedge change, whether it's abusable by governments in other ways, etc. the panicky ones were not those]. > "it was the issue of it reporting content outside of your control at all, which again, this Google thing doesnt seem to do." All the big cloud providers [Google, Microsoft, Facebook] report abusive imagery sent to their clouds to the authorities (search for annual NECMEC reports), except Apple. The others slurp up unencrypted data (Facebook photos, Google Drive, Microsoft OneDrive) and scan it and report on it, and nobody [on HN] says anything. Apple was trying to do a more privacy-preserving approach and it seemed like they might be pushing it to the client upload code so they could offer fully encrypted storage where they couldn't scan photos on their side, and a couple of years later they did, they announced optional Advanced Data Protection[2] which fully encrypts iCloud photos among other things. Dark patterns aside, it's in your control whether to upload data to companies, so 'reporting content outside your control' is deliberately misrepresenting it. [1] https://www.wired.com/story/apple-photo-scanning-csam-communication-safety-messages/ https://www.wired.com/story/apple-photo-scanning-csam-commun... [2] https://support.apple.com/en-gb/guide/security/sec973254c5f/web https://support.apple.com/en-gb/guide/security/sec973254c5f/...
    - gigel82 2y ago
      It was offline scanning and sending uploaded matches to law enforcement, this was very clearly laid out in their plans (that they eventually rolled back after massive uproar). Of course, you could argue some people that choose to not use iCloud would not get that last bit, but considering they're turning it on by default (and even turning it on whenever you switch devices even though you restore a backup with it off), I'd say that would be a tiny minority of their customers. Also, since we're on the subject of "unannounced scanning of all photos", Apple went and did it anyway, same as Google turning it on by default but claiming it's only to look for "landmarks" LOL :) https://news.ycombinator.com/item?id=42533685 https://news.ycombinator.com/item?id=42533685
      - JimDabell 2y ago
        > It was offline scanning It was not. The device was incapable of determining matches and the process only applied to photos being uploaded to iCloud. > Also, since we're on the subject of "unannounced scanning of all photos", Apple went and did it anyway, same as Google turning it on by default but claiming it's only to look for "landmarks" LOL :) https://news.ycombinator.com/item?id=42533685 https://news.ycombinator.com/item?id=42533685 They did not. The landmark process works in an entirely different way for an entirely different purpose.
- Gasp0de 2y ago
  As far as I remember, Apple was planning to detect CSAM, and upload the detected images to the cloud, showing the photos to apple employees. This is not at all comparable to a locally running API that returns something like "{nudity: true}".
  - theshrike79 2y ago
    Nope. You needed to have multiple (exact count undefined) confirmed matches before anything was sent anywhere. And even then "Apple employees" would only see a "reduced quality" (Can't remember the actual wording) version of the images. Basically just enough for them to determine if there's something actually illegal in there. You'd have to be the unluckiest person in the world to get 5 false matches against actual verified child abuse imagery. And even then you'd just slightly inconvenience a human checker. Now they're doing it in the cloud for every image you upload unless you turn on Advanced Data Protection. Just like every provider. I think I'm literally one of the very few people in the world who actually read and understood the white paper - and I have the downvotes to prove it :)
mkl 2y ago
It also seems to be 46.84MB on my phone, not 2GB.
ninalanyon 2y ago
> nudity and other potentially inappropriate things What counts as inappropriate? And who decides.
- hulitu 2y ago
  > What counts as inappropriate? And who decides. According to US supreme court: "I know it, when i see it"

theshrike79 2y ago

Just like the Apple system COULD LOCALLY detect CSAM on your phone. And it was the Worst Thing Ever according to hackernews and the internet at large. Now when Google does it, it's just fine and dandy.

gumby271 2y ago
I'm pretty sure what Apple proposed was an offline system for scanning, but any matches would be submitted to them. This is just the offline parts, and apps can choose to use it to detect nudity or other NSFW images. There's no scanning going on at all actually.
- JimDabell 2y ago
  This is not correct. This is how most people assumed the Apple proposal worked, but it actually worked in a very different way. The device never knew if an image matched. Matches could only be determined via the combination of a receipt calculated on the device, plus information on the server, plus meeting a threshold of many matches. It was not offline scanning and uploading matches.
  - fsflover 2y ago
    Does it matter? A tiny and invisible update would be able to change that in no time and with no possibility for a user to opt out.
    - JimDabell 2y ago
      This is true of literally any software with auto updates. You’re criticising them for something they did not do, did not intend to do, and designed a system that worked in an entirely different way… just because they could do it differently to what they actually proposed doing. If they wanted to do it that other way, they could have just done it that other way in the first place and saved themselves a lot of effort.
      - fsflover 2y ago
        It's only true for proprietary software, which actively prevents users from studying it. Especially when the trust in company plunges due to the enshittification, https://pluralistic.net/2025/02/26/ursula-franklin/ https://pluralistic.net/2025/02/26/ursula-franklin/ and https://news.ycombinator.com/item?id=43243075 https://news.ycombinator.com/item?id=43243075
        JimDabell 2y ago
        > It's only true for proprietary software No, this is not true. There is nothing stopping open-source software from pushing malicious updates. But you are avoiding my main point. You are criticising Apple for something they haven’t done and had no plans to do.
        fsflover 2y ago
        > There is nothing stopping open-source software from pushing malicious updates. Two things stop it: (much simplified) oversight by the community and a possibility to fix the code. > You are criticising Apple for something they haven’t done and had no plans to do. The do participate in the enshittification (https://pluralistic.net/2025/02/26/ursula-franklin/ https://pluralistic.net/2025/02/26/ursula-franklin/ and https://news.ycombinator.com/item?id=43243075 https://news.ycombinator.com/item?id=43243075), so my only expectation is that they won't do what's best for the users. In addition, they removed many features requested by the users like the headphone jack. In general, the less you trust in a company, the better. Free software allows to decrease the trust in the vendor by watching the code and forking whenever you have to.
  - gumby271 2y ago
    Oh that's right, they would upload a hash basically then flag it using whatever criteria they wanted on the server side. I think some of the outrage too was the question of what happens when they detect something, it was the issue of it reporting content outside of your control at all, which again, this Google thing doesnt seem to do.
    - JimDabell 2y ago
      > Oh that's right, they would upload a hash basically then flag it using whatever criteria they wanted on the server side. No, that’s not right either. You should really read their white paper, it’s very interesting and not at all what people assumed it was like.
      - gumby271 2y ago
        That's fair, but the point is still that "The Worst Thing Ever" was the online part, which this doesn't do, so the comparison makes little to no sense. I'll check out their write up though, sounds interesting.
        deleted 2y ago
        [deleted]
    - jodrellblank 2y ago
      The proposal was to scan photos people were uploading to iCloud not all photos[1]. The panic on HN was a) misunderstanding or deliberately misrepresenting the proposal as if it was scanning all offline photos, b) fantasising what if they don't do what they announced and instead scan all offline photos, c) realising that all Silicon Valley tech companies can change client software through updates, therefore Apple bad. [There were non-panicky comments about whether it's a legally significant move, whether it's a wedge change, whether it's abusable by governments in other ways, etc. the panicky ones were not those]. > "it was the issue of it reporting content outside of your control at all, which again, this Google thing doesnt seem to do." All the big cloud providers [Google, Microsoft, Facebook] report abusive imagery sent to their clouds to the authorities (search for annual NECMEC reports), except Apple. The others slurp up unencrypted data (Facebook photos, Google Drive, Microsoft OneDrive) and scan it and report on it, and nobody [on HN] says anything. Apple was trying to do a more privacy-preserving approach and it seemed like they might be pushing it to the client upload code so they could offer fully encrypted storage where they couldn't scan photos on their side, and a couple of years later they did, they announced optional Advanced Data Protection[2] which fully encrypts iCloud photos among other things. Dark patterns aside, it's in your control whether to upload data to companies, so 'reporting content outside your control' is deliberately misrepresenting it. [1] https://www.wired.com/story/apple-photo-scanning-csam-communication-safety-messages/ https://www.wired.com/story/apple-photo-scanning-csam-commun... [2] https://support.apple.com/en-gb/guide/security/sec973254c5f/web https://support.apple.com/en-gb/guide/security/sec973254c5f/...
  - gigel82 2y ago
    It was offline scanning and sending uploaded matches to law enforcement, this was very clearly laid out in their plans (that they eventually rolled back after massive uproar). Of course, you could argue some people that choose to not use iCloud would not get that last bit, but considering they're turning it on by default (and even turning it on whenever you switch devices even though you restore a backup with it off), I'd say that would be a tiny minority of their customers. Also, since we're on the subject of "unannounced scanning of all photos", Apple went and did it anyway, same as Google turning it on by default but claiming it's only to look for "landmarks" LOL :) https://news.ycombinator.com/item?id=42533685 https://news.ycombinator.com/item?id=42533685
    - JimDabell 2y ago
      > It was offline scanning It was not. The device was incapable of determining matches and the process only applied to photos being uploaded to iCloud. > Also, since we're on the subject of "unannounced scanning of all photos", Apple went and did it anyway, same as Google turning it on by default but claiming it's only to look for "landmarks" LOL :) https://news.ycombinator.com/item?id=42533685 https://news.ycombinator.com/item?id=42533685 They did not. The landmark process works in an entirely different way for an entirely different purpose.
Gasp0de 2y ago
As far as I remember, Apple was planning to detect CSAM, and upload the detected images to the cloud, showing the photos to apple employees. This is not at all comparable to a locally running API that returns something like "{nudity: true}".
- theshrike79 2y ago
  Nope. You needed to have multiple (exact count undefined) confirmed matches before anything was sent anywhere. And even then "Apple employees" would only see a "reduced quality" (Can't remember the actual wording) version of the images. Basically just enough for them to determine if there's something actually illegal in there. You'd have to be the unluckiest person in the world to get 5 false matches against actual verified child abuse imagery. And even then you'd just slightly inconvenience a human checker. Now they're doing it in the cloud for every image you upload unless you turn on Advanced Data Protection. Just like every provider. I think I'm literally one of the very few people in the world who actually read and understood the white paper - and I have the downvotes to prove it :)

mkl 2y ago

It also seems to be 46.84MB on my phone, not 2GB.

ninalanyon 2y ago

> nudity and other potentially inappropriate things What counts as inappropriate? And who decides.

hulitu 2y ago
> What counts as inappropriate? And who decides. According to US supreme court: "I know it, when i see it"