13 ms·

In their attempt to make this extra private by scanning 'on device', I think they've managed to make it feel worse. If they scan my iCloud photos in iCloud, we

In their attempt to make this extra private by scanning 'on device', I think they've managed to make it feel worse. If they scan my iCloud photos in iCloud, well lots of companies scan stuff when you upload it. It's on their servers, they're responsible for it. They don't want to be hosting CSAM. It feels much worse them turning your own, trusty iPhone against you. I know that isn't how you should look at it, but that's still how it feels.

nottorp 5y ago
That is how you should look at it. They have no business scanning the phones for anything, period. I'll refer you to the poem beginning with "First they came for the Jews...".
- petermcneeley 5y ago
  The poem begins with "First they came for the socialists.." https://en.wikipedia.org/wiki/First_they_came_ https://en.wikipedia.org/wiki/First_they_came_...
  - tpush 5y ago
    'Communists' instead of 'socialists' actually; the US version is inaccurate due to anti-communist propaganda.
    - stjohnswarts 5y ago
      Do you have a source for that?
      - FabHK 5y ago
        The original: Als die Nazis die Kommunisten holten, habe ich geschwiegen; ich war ja kein Kommunist. Als sie die Sozialdemokraten einsperrten, habe ich geschwiegen; ich war ja kein Sozialdemokrat. Als sie die Gewerkschafter holten, habe ich geschwiegen; ich war ja kein Gewerkschafter. Als sie die Juden holten, habe ich geschwiegen; ich war ja kein Jude. Als sie mich holten, gab es keinen mehr, der protestieren konnte. https://de.wikipedia.org/wiki/Martin_Niemöller#Habe_ich_geschwiegen https://de.wikipedia.org/wiki/Martin_Niemöller#Habe_ich_gesc...
- deleted 5y ago
  [deleted]
- userbinator 5y ago
  "First they came for the pedophiles..."
  - nottorp 5y ago
    Yeah, and then they'll come for the gays or christians, depending on which group you feel closer to.
hamburgerwah 5y ago
Also if you remotely think the scanning is going to stop at CSAM I have a bridge in brooklyn to sell you. Copyrighted material, national security, thought crime, are already inside this trojan horse waiting to jump out.
Wowfunhappy 5y ago
The practical difference is that with on-device scanning, the system is just a few bit flips away from scanning every photo on your device, instead of just the ones that are about to be uploaded. With server-side scanning, the separation is clear—what's on Apple's server can be scanned, and what's only on your phone cannot. This all plays so perfectly into my long-time fears about the locked down nature of the iPhone. Sure, it's more secure if Apple is a good steward, but no one outside of Apple can inspect or audit what it's doing!
- tmdg 5y ago
  I feel like this is a false sense of security. Even before this change, they can easily access and scan photos on your device. If they do any post-processing of the image on device, they already do.
  - shuckles 5y ago
    Apple already scans your photos for faces and syncs found faces through iCloud. I’d imagine updating that machine learning model is at least as straightforward as this one.
    - Wowfunhappy 5y ago
      They're searching for different things though. To my knowledge, before now iOS has never scanned for fingerprints of specific photographs. It would be so darn easy to replace the CSAM database with fingerprints of known tiananmen square photos...
      - shuckles 5y ago
        That is a distinction without a difference. I’m sure you could put together quite a good tank man classifier (proof: Google Reverse Image Search works quite well), and it’d catch variations which a perceptual hash wouldn’t. The only difference is intent. The technical risk has not changed at all.
guerrilla 5y ago
> I know that isn't how you should look at it, but that's still how it feels. Why do you think you know that? You didn't say. It seems like that is how you should see it since they are using your processing power on your device potentially against you.
jpxw 5y ago
In reality, the fact that they do the scanning on-device is actually better privacy-wise. But I can see how instinctually it “feels” wrong.
- rowanG077 5y ago
  No it's not. Not scanning any content on device at all is better privacy wise.
  - treesprite82 5y ago
    Often, despite all its disadvantages, on-device scanning would at least let know see what was being scanned for. But Apple have circumvented that with NeuralHash. There's no way for the user to verify that only CSAM is being detected - which could be partially accomplished by having iOS only accept hashes signed by multiple independent child protection organisations (with some outside of FVEY).
- eCa 5y ago
  Can you explain how it is better? If they only scanned icloud I can choose to not use that. If they scan on my iphone my choice would be to stop using that and get a gphone instead.
  - fossuser 5y ago
    It only runs if you have iCloud photos enabled. Then it only runs client side and alerts if there are matches to CSAM hashes above some threshold. The other way is it runs against unencrypted images in iCloud and they'd know of any match. The new method also paves the way for the iCloud photos to be encrypted (they're currently not).
    - AlexandrB 5y ago
      > It only runs if you have iCloud photos enabled. The detailed specifics of this would be interesting to know. If I accidentally flip iCloud photos on and then turn it off within a few minutes, would that give the system license to scan all my photos once? What if my phone is in airplane mode and I turn iCloud photos on? Will the system scan photos and create vouchers offline for later upload? Edit: On a related note, will turning on iCloud photos now come with a modal prompt and privacy warning so you're fully informed and can't do it "by accident"? Pardon my ignorance if this happens already. I've never turned mine on and am not about to start.
rowanG077 5y ago
Why is this not how you should look at it. This is exactly how I look at it. It's your device, no company should be able to scan shit on a device that they have sold to someone. That's like keeping the key of a house after you have sold it and then checking sneakily every week whether the new owners are doing illegal stuff.
vineyardmike 5y ago
> I know that isn't how you should look at it, but that's still how it feels. This is exactly how you should look at it. They turned your device against you.
- chrischattin 5y ago
  I downvoted this comment because it's hyperbole. There are plenty of over very real concerns with this. For example, say some activist intern at Apple decides they don't like your politics and hits the report button, next thing you know, the police show up to your door and are confiscating all your devices. Just one of many potential abuses.
pehtis 5y ago
Yes, thats exactly how i feel. I'd still hate it if my iCloud uploads are scanned but I'm already assuming that anyway. But the fact that my iOS device can potentially report me to any authorities, for whatever reason, is crossing a line that makes it impossible to ever own an iPhone again. I bought my first one in 2007 so I'm not saying this lightly.. Does anybody know if this policy will extend to macOS too?
- stephc_int13 5y ago
  It has already be announced, this is not only iOS, but all Apple devices.
  - blisterpeanuts 5y ago
    Great. I recently invested $3800 in an excellent, new iMac. Now I'm starting to wonder if I should have spent a couple thousand less for a barebones PC and installed my favorite Linux distro. It would have done 75% of what I needed, and the other 25% (music and video work)... well, that's the tradeoff. If anyone in my circle of family, friends, and social network asks my advice, the formerly easy answer "Get a Mac, get an iPhone; you won't regret it!" is probably going to be replaced with something more nuanced ("Buy Apple, but know what you're getting into; here's a few articles on privacy....").
    - selsta 5y ago
      The CSAM detection technical summary [1] only mentions iOS and iPadOS. If it does come to macOS it will be part of Photos.app, as that's the only way to interact with iCloud Photos. I would recommend you to avoid that app and cloud in general if you care about privacy. [1] https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
      - pehtis 5y ago
        If it stays like that then it's manageable i guess.
noobermin 5y ago
Only thing I disagree with, it is how you should look at it and that's why it feels wrong, because it is wrong.
skybrian 5y ago
They didn’t do themselves any favors by blurring the line between apps and the OS. If it’s just Apple’s Photos app doing this, you can install a different app and not use that one.
- stjohnswarts 5y ago
  That's true for now but once it gets normalized it will be extended laterally across the whole phone.
ksec 5y ago
I think what you describe and what is shown by Apple is "the" concrete piece of evidence, and what I have suspected for long that there is no longer any product person running the company. It used to be Steve Jobs. You have people arguing over the technical definition of privacy, or on devices, off devices terminology. It doesn't matter. The user doesn't understand any of it. And they just feel exactly what you wrote. An intrusion of their private space by a company that has been telling you how much they value your privacy for a better part of decade. These sort of technical explanation used to be a Tech Companies / Google / Microsoft things. Absolutely make sense to nerds, but nothing much to a layman. Now Apple are doing the same. If you watch their Keynotes, you should notice the increase usage of technical jargons or numbers for performance over the past 5 years. Something that Steve Jobs's keynote used to keep it to minimal.
farmerstan 5y ago
In a few years, all the pedophiles will stop using iPhone, and then only innocent people will be scanned in perpetuity. Remember, if someone makes new CSAM, it won't match any hashes so even "new" pedophiles won't get caught by this. So really, the steady state is just us regular folks getting scanned. As the years go on, what is defined as CSAM will morph into "objectionable material", and will then include "misinformation" like today. They say they won't right now, but where will things be in a few years?
- stjohnswarts 5y ago
  Apple just announce I would posit it will be a few weeks, not a few years. This news is everywhere.
- rootusrootus 5y ago
  > if someone makes new CSAM, it won't match any hashes so even "new" pedophiles won't get caught by this. That assumes the hashes aren't recorded permanently.
- benhurmarcel 5y ago
  > all the pedophiles will stop using iPhone Sounds like “mission accomplished” for Apple then
Kim_Bruning 5y ago
I wonder how much of this is shifting the overton window. I'd really not have people scanning my data at all really. "If you give me six lines written by the hand of the most honest of men, I will find something in them which will hang him." said Cardinal Richelieu in the 16th century. I'm pretty sure my phone and online accounts hold a lot more data than just six lines. How do I know there won't be accidents, or 'accidents'?
AceJohnny2 5y ago
Wild thought: could on-device scanning be considered a violation of the 3rd Amendment (quartering soldiers [enforcing agents of the government] in peoples homes [property]) ?
- COGlory 5y ago
  My understanding is that there isnt a whole lot of established anything re: the 3rd amendment. So...maybe, we can dream? I think warantless search is a much stronger argument, though.
ursugardaddy 5y ago
I wouldn't be surprised if there was some weird telemetry tied to this too
thefourthchime 5y ago
I agree. While the difference is slight, one feels almost expected, the other is creepy and invasive.
unityByFreedom 5y ago
> I know that isn't how you should look at it Why not? The argument seems perfectly reasonable to me.
feanaro 5y ago
> I know that isn't how you should look at it, but that's still how it feels It's definitely how you should look at it because it's right. Once a device is compromised in this way, there's no going back and the erosion of privacy and expansion of scope will become neverending. If a capability exists, it will just be too hard for spooks to keep their fingers out of the cookie jar.
pbasista 5y ago
> your own, trusty iPhone iPhone runs software which is in full control of Apple. They take active measures to stop general public from auditing its behavior (i.e. no source code, no API specifications, etc). In my opinion, such a device can never be considered as belonging to their users or "trusted" in any reasonable way. The reason is that its root functionality always remains in control of a different entity. I believe that such a relationship can more accurately be described as a lease. Instead of providing specifications and source code, which would help establish the fundamental trust in their products and their behavior, Apple seems to increase their reliance on the "trust me bro" promises, which their current user base has accepted. So far.
jamesholden 5y ago
Oh hey.. Roomba scanning for illicit drugs on your coffee table. It sounds ridiculous, but that thing is a literal camera roaming around your entire house, with machine learning.. so..
dangerface 5y ago
I think its just more visible, its easy to upload to the internet and think im uploading to the internet, you don't think I am sharing this with law enforcement and Apples third parties.
matt123456789 5y ago
Don’t second-guess yourself. The viewpoint you express is completely valid. Other child comments have pointed this out, but corporate messaging does not get to tell you how _you_ choose to look at things.

nottorp 5y ago

That is how you should look at it. They have no business scanning the phones for anything, period. I'll refer you to the poem beginning with "First they came for the Jews...".

petermcneeley 5y ago
The poem begins with "First they came for the socialists.." https://en.wikipedia.org/wiki/First_they_came_ https://en.wikipedia.org/wiki/First_they_came_...
- tpush 5y ago
  'Communists' instead of 'socialists' actually; the US version is inaccurate due to anti-communist propaganda.
  - stjohnswarts 5y ago
    Do you have a source for that?
    - FabHK 5y ago
      The original: Als die Nazis die Kommunisten holten, habe ich geschwiegen; ich war ja kein Kommunist. Als sie die Sozialdemokraten einsperrten, habe ich geschwiegen; ich war ja kein Sozialdemokrat. Als sie die Gewerkschafter holten, habe ich geschwiegen; ich war ja kein Gewerkschafter. Als sie die Juden holten, habe ich geschwiegen; ich war ja kein Jude. Als sie mich holten, gab es keinen mehr, der protestieren konnte. https://de.wikipedia.org/wiki/Martin_Niemöller#Habe_ich_geschwiegen https://de.wikipedia.org/wiki/Martin_Niemöller#Habe_ich_gesc...
deleted 5y ago
[deleted]
userbinator 5y ago
"First they came for the pedophiles..."
- nottorp 5y ago
  Yeah, and then they'll come for the gays or christians, depending on which group you feel closer to.

hamburgerwah 5y ago

Also if you remotely think the scanning is going to stop at CSAM I have a bridge in brooklyn to sell you. Copyrighted material, national security, thought crime, are already inside this trojan horse waiting to jump out.

Wowfunhappy 5y ago

The practical difference is that with on-device scanning, the system is just a few bit flips away from scanning every photo on your device, instead of just the ones that are about to be uploaded. With server-side scanning, the separation is clear—what's on Apple's server can be scanned, and what's only on your phone cannot. This all plays so perfectly into my long-time fears about the locked down nature of the iPhone. Sure, it's more secure if Apple is a good steward, but no one outside of Apple can inspect or audit what it's doing!

tmdg 5y ago
I feel like this is a false sense of security. Even before this change, they can easily access and scan photos on your device. If they do any post-processing of the image on device, they already do.
- shuckles 5y ago
  Apple already scans your photos for faces and syncs found faces through iCloud. I’d imagine updating that machine learning model is at least as straightforward as this one.
  - Wowfunhappy 5y ago
    They're searching for different things though. To my knowledge, before now iOS has never scanned for fingerprints of specific photographs. It would be so darn easy to replace the CSAM database with fingerprints of known tiananmen square photos...
    - shuckles 5y ago
      That is a distinction without a difference. I’m sure you could put together quite a good tank man classifier (proof: Google Reverse Image Search works quite well), and it’d catch variations which a perceptual hash wouldn’t. The only difference is intent. The technical risk has not changed at all.

guerrilla 5y ago

> I know that isn't how you should look at it, but that's still how it feels. Why do you think you know that? You didn't say. It seems like that is how you should see it since they are using your processing power on your device potentially against you.

jpxw 5y ago

In reality, the fact that they do the scanning on-device is actually better privacy-wise. But I can see how instinctually it “feels” wrong.

rowanG077 5y ago
No it's not. Not scanning any content on device at all is better privacy wise.
- treesprite82 5y ago
  Often, despite all its disadvantages, on-device scanning would at least let know see what was being scanned for. But Apple have circumvented that with NeuralHash. There's no way for the user to verify that only CSAM is being detected - which could be partially accomplished by having iOS only accept hashes signed by multiple independent child protection organisations (with some outside of FVEY).
eCa 5y ago
Can you explain how it is better? If they only scanned icloud I can choose to not use that. If they scan on my iphone my choice would be to stop using that and get a gphone instead.
- fossuser 5y ago
  It only runs if you have iCloud photos enabled. Then it only runs client side and alerts if there are matches to CSAM hashes above some threshold. The other way is it runs against unencrypted images in iCloud and they'd know of any match. The new method also paves the way for the iCloud photos to be encrypted (they're currently not).
  - AlexandrB 5y ago
    > It only runs if you have iCloud photos enabled. The detailed specifics of this would be interesting to know. If I accidentally flip iCloud photos on and then turn it off within a few minutes, would that give the system license to scan all my photos once? What if my phone is in airplane mode and I turn iCloud photos on? Will the system scan photos and create vouchers offline for later upload? Edit: On a related note, will turning on iCloud photos now come with a modal prompt and privacy warning so you're fully informed and can't do it "by accident"? Pardon my ignorance if this happens already. I've never turned mine on and am not about to start.

rowanG077 5y ago

Why is this not how you should look at it. This is exactly how I look at it. It's your device, no company should be able to scan shit on a device that they have sold to someone. That's like keeping the key of a house after you have sold it and then checking sneakily every week whether the new owners are doing illegal stuff.

vineyardmike 5y ago

> I know that isn't how you should look at it, but that's still how it feels. This is exactly how you should look at it. They turned your device against you.

chrischattin 5y ago
I downvoted this comment because it's hyperbole. There are plenty of over very real concerns with this. For example, say some activist intern at Apple decides they don't like your politics and hits the report button, next thing you know, the police show up to your door and are confiscating all your devices. Just one of many potential abuses.

pehtis 5y ago

Yes, thats exactly how i feel. I'd still hate it if my iCloud uploads are scanned but I'm already assuming that anyway. But the fact that my iOS device can potentially report me to any authorities, for whatever reason, is crossing a line that makes it impossible to ever own an iPhone again. I bought my first one in 2007 so I'm not saying this lightly.. Does anybody know if this policy will extend to macOS too?

stephc_int13 5y ago
It has already be announced, this is not only iOS, but all Apple devices.
- blisterpeanuts 5y ago
  Great. I recently invested $3800 in an excellent, new iMac. Now I'm starting to wonder if I should have spent a couple thousand less for a barebones PC and installed my favorite Linux distro. It would have done 75% of what I needed, and the other 25% (music and video work)... well, that's the tradeoff. If anyone in my circle of family, friends, and social network asks my advice, the formerly easy answer "Get a Mac, get an iPhone; you won't regret it!" is probably going to be replaced with something more nuanced ("Buy Apple, but know what you're getting into; here's a few articles on privacy....").
  - selsta 5y ago
    The CSAM detection technical summary [1] only mentions iOS and iPadOS. If it does come to macOS it will be part of Photos.app, as that's the only way to interact with iCloud Photos. I would recommend you to avoid that app and cloud in general if you care about privacy. [1] https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
    - pehtis 5y ago
      If it stays like that then it's manageable i guess.

noobermin 5y ago

Only thing I disagree with, it is how you should look at it and that's why it feels wrong, because it is wrong.

skybrian 5y ago

They didn’t do themselves any favors by blurring the line between apps and the OS. If it’s just Apple’s Photos app doing this, you can install a different app and not use that one.

stjohnswarts 5y ago
That's true for now but once it gets normalized it will be extended laterally across the whole phone.

ksec 5y ago

I think what you describe and what is shown by Apple is "the" concrete piece of evidence, and what I have suspected for long that there is no longer any product person running the company. It used to be Steve Jobs. You have people arguing over the technical definition of privacy, or on devices, off devices terminology. It doesn't matter. The user doesn't understand any of it. And they just feel exactly what you wrote. An intrusion of their private space by a company that has been telling you how much they value your privacy for a better part of decade. These sort of technical explanation used to be a Tech Companies / Google / Microsoft things. Absolutely make sense to nerds, but nothing much to a layman. Now Apple are doing the same. If you watch their Keynotes, you should notice the increase usage of technical jargons or numbers for performance over the past 5 years. Something that Steve Jobs's keynote used to keep it to minimal.

farmerstan 5y ago

In a few years, all the pedophiles will stop using iPhone, and then only innocent people will be scanned in perpetuity. Remember, if someone makes new CSAM, it won't match any hashes so even "new" pedophiles won't get caught by this. So really, the steady state is just us regular folks getting scanned. As the years go on, what is defined as CSAM will morph into "objectionable material", and will then include "misinformation" like today. They say they won't right now, but where will things be in a few years?

stjohnswarts 5y ago
Apple just announce I would posit it will be a few weeks, not a few years. This news is everywhere.
rootusrootus 5y ago
> if someone makes new CSAM, it won't match any hashes so even "new" pedophiles won't get caught by this. That assumes the hashes aren't recorded permanently.
benhurmarcel 5y ago
> all the pedophiles will stop using iPhone Sounds like “mission accomplished” for Apple then

Kim_Bruning 5y ago

I wonder how much of this is shifting the overton window. I'd really not have people scanning my data at all really. "If you give me six lines written by the hand of the most honest of men, I will find something in them which will hang him." said Cardinal Richelieu in the 16th century. I'm pretty sure my phone and online accounts hold a lot more data than just six lines. How do I know there won't be accidents, or 'accidents'?

AceJohnny2 5y ago

Wild thought: could on-device scanning be considered a violation of the 3rd Amendment (quartering soldiers [enforcing agents of the government] in peoples homes [property]) ?

COGlory 5y ago
My understanding is that there isnt a whole lot of established anything re: the 3rd amendment. So...maybe, we can dream? I think warantless search is a much stronger argument, though.

ursugardaddy 5y ago

I wouldn't be surprised if there was some weird telemetry tied to this too

thefourthchime 5y ago

I agree. While the difference is slight, one feels almost expected, the other is creepy and invasive.

unityByFreedom 5y ago

> I know that isn't how you should look at it Why not? The argument seems perfectly reasonable to me.

feanaro 5y ago

> I know that isn't how you should look at it, but that's still how it feels It's definitely how you should look at it because it's right. Once a device is compromised in this way, there's no going back and the erosion of privacy and expansion of scope will become neverending. If a capability exists, it will just be too hard for spooks to keep their fingers out of the cookie jar.

pbasista 5y ago

> your own, trusty iPhone iPhone runs software which is in full control of Apple. They take active measures to stop general public from auditing its behavior (i.e. no source code, no API specifications, etc). In my opinion, such a device can never be considered as belonging to their users or "trusted" in any reasonable way. The reason is that its root functionality always remains in control of a different entity. I believe that such a relationship can more accurately be described as a lease. Instead of providing specifications and source code, which would help establish the fundamental trust in their products and their behavior, Apple seems to increase their reliance on the "trust me bro" promises, which their current user base has accepted. So far.

jamesholden 5y ago

Oh hey.. Roomba scanning for illicit drugs on your coffee table. It sounds ridiculous, but that thing is a literal camera roaming around your entire house, with machine learning.. so..

dangerface 5y ago

I think its just more visible, its easy to upload to the internet and think im uploading to the internet, you don't think I am sharing this with law enforcement and Apples third parties.

matt123456789 5y ago

Don’t second-guess yourself. The viewpoint you express is completely valid. Other child comments have pointed this out, but corporate messaging does not get to tell you how _you_ choose to look at things.