5 ms·

So many people in this thread are convinced this whole thing is intentionally malicious, that Apple is doing this because they want to enable government spying,

by wilg 5y ago

So many people in this thread are convinced this whole thing is intentionally malicious, that Apple is doing this because they want to enable government spying, and they are intentionally using child sex abuse as a way of trying to make it palatable in a PR battle. I don't think that is the most likely situation at all. Apple has been, as part of their privacy initiatives, trying to do as much as possible on the device. That's how they have been defining privacy to themselves internally. Then someone said "can we do something about CSAM" and they came up with a pretty good technical solution that operates on device and therefore, to them, seemed like it would not be particularly controversial. They've been talking about doing ML and photo scanning object recognition on device for years, they're moving much of Siri to on-device in iOS 15, all as part of their privacy initiatives. It seems to have backfired in that people actually seem to prefer scanning in the cloud to on-device scanning for things like this, because it feels less like a violation of your ownership of the device. I think the security arguments about how this system can be misused are compelling and it's a fine position to be strongly against this, but I don't know that there's good justification that Apple has some ulterior motive and is faking caring about privacy. I think they were operating with a particular set of assumptions about how people view privacy that turned out to be wrong and they are genuinely surprised by the blowback.

willio58 5y ago
I agree with you. I’m all for privacy but have no issues whatsoever with companies scanning for CSAM. I do not feel this is an invasion of my privacy, because I know how hashing works and I know I do not have CSAM on my device.
- blintz 5y ago
  Do you know how NeuralHash works? NeuralHash is not a cryptographic hash. Unless you're an Apple employee, you can't - the model is private and not inspectable.
- gigel82 5y ago
  > I’m all for privacy but have no issues whatsoever with companies scanning That's an oxymoron.
- thephyber 5y ago
  I think some people here have similar feelings, but as other replies point out, this particular hash is not cryptographic and we have to trust Apple’s word for it that false positives are extremely rare. Also worth mentioning that the original corpus of data used to build the hashes isn’t inspectable, so we don’t know how much garbage exists on the input side. A recent blog post suggests that the corpus of CSAM isn’t even very comprehensive, so the value of this system is questionable.
  - willio58 5y ago
    Ahhh I did not know that they are using a non-cryptographic hash, I guess I assumed otherwise. Thanks for your reply.
firebaze 5y ago
That's one of the few good aspects of a legendary fuck-up like this: you learn about people defending it. People defending CSAM should go to hell, fast. But are we already done destroying all low-hanging fruits? Did we stop Johnny Savile? Did we put all clerical actors behind bars? Did we extinguish the child porn network behind Marc Dutroux (https://en.wikipedia.org/wiki/Marc_Dutroux https://en.wikipedia.org/wiki/Marc_Dutroux)? And even if we did, would that be enough of an excuse to implicitly accuse anyone? My spouses' family (well-off, so using iDevices) took photos of their young age kids playing, partially naked at the sea. They are now frightened if their photos could be stolen by someone and marketed as child porn. So unbelievable.
- notJim 5y ago
  > They are now frightened if their photos could be stolen by someone and marketed as child porn. That sounds bad, someone high up at Apple should do an interview clarifying that that's not what's happening!
  - cwkoss 5y ago
    If you don't think this is possible, you are not understanding the technical implementation they announced. Perceptual hashes on chunks of images will yield false positives.
    - notJim 5y ago
      I'm not sure what you mean by chunks of images. My understanding is that it's a perceptual hash meant to determine if it's the same image, with minor adjustments like cropping or adjusting the colors. There is no AI/ML classification running. Apple says false positives occur at a rate of one in a trillion or something like that. Someone elsewhere in the thread calculated that this would not be frequent enough for false positives to trigger a manual review.
      - cwkoss 5y ago
        The perceptual hashing are being applied to a chunk of the image - not the whole image but some part of it. These chunks contain less entropy than the whole image so necessarily have higher chance of triggering false positives than whole-image comparisons.
system2 5y ago
Doesn't matter how technically well done this is. I do not want my device to poke my files and send them to an AI software to make a decision. It makes me uncomfortable. This is malicious. I do not want them to touch my photos or anything personal. I paid for this device, now it is doing things against my will.
- jachee 5y ago
  It doesn’t happen against your will. You still have full control over whether scanning happens. Simply disable iCloud Photo Library, and nothing gets scanned.
  - asddubs 5y ago
    for now
  - zionic 5y ago
    And what about the hundreds of millions of people that enable iCloud by default thanks to Apple’s dark patterns, does their privacy not matter?
jachee 5y ago
Exactly! There has been so much FUD, conspiracy theory, and fear-mongering. None of the usual anti-regulation apologists have pointed out that Apple shouldn’t be forced to download and host potentially-illegal material in the interest of ensuring whether or not it’s actually illegal. This whole program is their intelligent solution to protecting as much user privacy as possible while still being compliant with the law. On-device hashing is actually pro privacy compared to in-the-cloud scanning (which all other cloud hosting providers are also required to do).
- m-ee 5y ago
  This is not about compliance, the relevant law specifically says that companies are not required to proactively scan for CSAM.
ipv6ipv4 5y ago
I agree it is likely not malicious at all. It’s the result of koolaid in an echo chamber. And inertia at this point. However, I also think this is the poster child of the proverb that the road to hell is paved with good intentions. Now Apple needs to cancel this misguided initiative and never speak of it again, if they want to salvage some of their reputation.
sagarm 5y ago
> because it feels less like a violation of your ownership of the device. Agreed that this was not intended to be malicious. Apple has always been pretty clear they they should decide what happens on their devices. This sort of on-device scanning that doesn't serve the user is just the latest example of it, and one that people who would never be affected by the code signing restrictions can relate to.
insulanus 5y ago
Actually, I'm with you in that I think Apple's motives are different than people think, but I think yours are incorrect as well. > Apple has been, as part of their privacy initiatives, trying to do as much as possible on the device. That's how they have been defining privacy to themselves internally. Then someone said "can we do something about CSAM" [...] As a separate issue, many people in the company certainly do care about privacy, and that may go all the way to to Tim Cook. Who knows. What is much more important to Apple the company, though, is making money. Governments have been hounding them for years about letting them spy on users. And they have painted themselves into a bit of a corner, by having the most secure phones. Now the government comes to them with an offer they can't refuse, cloaked in child porn motivations. I believe many (most?) of the people involved are sincere. It's clear they have tried to make the least invasive system that still does what the government wants. But that's not good enough in the crazy connected cyber-world we find ourselves in today. Apple doesn't have a motivation to do this themselves. But they will do what they calculate they need to do.
- cwkoss 5y ago
  Apple has a huge potential profit motive. Once they roll this out for US users, they can sell the exact same capabilities to authoritarian regimes for detecting subversive images, images of warcrimes, etc. China would happily pay billions of dollars per year for this capability.
  - thephyber 5y ago
    China already uses their police forces to mandate that certain citizens install specific apps on their phones. They also control the cloud service (a Chinese-only iCloud) that Apple uses in the China market. China doesn’t need this feature.
    - firebaze 5y ago
      Let's extrapolate a bit on this.
craftinator 5y ago
> because it feels less like a violation of your ownership of the device. It doesn't just feel like a violation, it is one. This opens doors for a number of future abuses of power, and abuses of rights. The average user has no way of knowing what's in the database of hashes being compared against, and we have no way of controlling what goes into those databases. If some party in the US government decided that they didn't like the Quran, and put a million pictures of it in the database, and served the Apple reviewers with a gag order or some other nonsense, we'd have a 80's dystopian blockbuster as everyday life in our country. My device, my property, my rules. Apple doesn't get to do a stop and frisk, even if it's "for the good of the children".
- Spivak 5y ago
  My device my property my rules is pretty rich talking about a platform where you can’t run your own apps without Apple’s approval, you have no control over how the OS behaves, and there is built-in DRM. Your phone has been Apple’s device since the moment you bought it and turned it on.
  - craftinator 5y ago
    I run Linux on my phone, and it's not an Apple device. Other than that, I completely agree with you.
gerash 5y ago
I doubt most think this is malicious but Apple did preach "what happens on an iphone stays on the iphone" which aged terribly given this feature. Apple has also benefited from the perception that Google (their smartphone rival) spies on people by collecting location information where as I am sure they also crowd source iphone location to improve their Maps and road traffic data
pcurve 5y ago
I don't think Apple has insidious reason, but their implementation can come across as betrayal. Unlike Android phone, iphone will know about your uploading potentially problematic image beforehand. Yet, it will not alert you or block you from doing so.
neop1x 5y ago
It may not be helping child abuse enough if it does not detect newly-taken child pornography. Couple of years fast forward and machine learning will be scanning all the photos you take and watch. A little bit forward and it will be forbidden to take photos of many things and there will be no-photo places. Apple and governments will dictate what is right and what is wrong to take and watch. They are doing it for the safety of everyone, right?

by wilg 5y ago

willio58 5y ago

I agree with you. I’m all for privacy but have no issues whatsoever with companies scanning for CSAM. I do not feel this is an invasion of my privacy, because I know how hashing works and I know I do not have CSAM on my device.

blintz 5y ago
Do you know how NeuralHash works? NeuralHash is not a cryptographic hash. Unless you're an Apple employee, you can't - the model is private and not inspectable.
gigel82 5y ago
> I’m all for privacy but have no issues whatsoever with companies scanning That's an oxymoron.
thephyber 5y ago
I think some people here have similar feelings, but as other replies point out, this particular hash is not cryptographic and we have to trust Apple’s word for it that false positives are extremely rare. Also worth mentioning that the original corpus of data used to build the hashes isn’t inspectable, so we don’t know how much garbage exists on the input side. A recent blog post suggests that the corpus of CSAM isn’t even very comprehensive, so the value of this system is questionable.
- willio58 5y ago
  Ahhh I did not know that they are using a non-cryptographic hash, I guess I assumed otherwise. Thanks for your reply.

firebaze 5y ago

That's one of the few good aspects of a legendary fuck-up like this: you learn about people defending it. People defending CSAM should go to hell, fast. But are we already done destroying all low-hanging fruits? Did we stop Johnny Savile? Did we put all clerical actors behind bars? Did we extinguish the child porn network behind Marc Dutroux (https://en.wikipedia.org/wiki/Marc_Dutroux https://en.wikipedia.org/wiki/Marc_Dutroux)? And even if we did, would that be enough of an excuse to implicitly accuse anyone? My spouses' family (well-off, so using iDevices) took photos of their young age kids playing, partially naked at the sea. They are now frightened if their photos could be stolen by someone and marketed as child porn. So unbelievable.

notJim 5y ago
> They are now frightened if their photos could be stolen by someone and marketed as child porn. That sounds bad, someone high up at Apple should do an interview clarifying that that's not what's happening!
- cwkoss 5y ago
  If you don't think this is possible, you are not understanding the technical implementation they announced. Perceptual hashes on chunks of images will yield false positives.
  - notJim 5y ago
    I'm not sure what you mean by chunks of images. My understanding is that it's a perceptual hash meant to determine if it's the same image, with minor adjustments like cropping or adjusting the colors. There is no AI/ML classification running. Apple says false positives occur at a rate of one in a trillion or something like that. Someone elsewhere in the thread calculated that this would not be frequent enough for false positives to trigger a manual review.
    - cwkoss 5y ago
      The perceptual hashing are being applied to a chunk of the image - not the whole image but some part of it. These chunks contain less entropy than the whole image so necessarily have higher chance of triggering false positives than whole-image comparisons.

system2 5y ago

Doesn't matter how technically well done this is. I do not want my device to poke my files and send them to an AI software to make a decision. It makes me uncomfortable. This is malicious. I do not want them to touch my photos or anything personal. I paid for this device, now it is doing things against my will.

jachee 5y ago
It doesn’t happen against your will. You still have full control over whether scanning happens. Simply disable iCloud Photo Library, and nothing gets scanned.
- asddubs 5y ago
  for now
- zionic 5y ago
  And what about the hundreds of millions of people that enable iCloud by default thanks to Apple’s dark patterns, does their privacy not matter?

jachee 5y ago

Exactly! There has been so much FUD, conspiracy theory, and fear-mongering. None of the usual anti-regulation apologists have pointed out that Apple shouldn’t be forced to download and host potentially-illegal material in the interest of ensuring whether or not it’s actually illegal. This whole program is their intelligent solution to protecting as much user privacy as possible while still being compliant with the law. On-device hashing is actually pro privacy compared to in-the-cloud scanning (which all other cloud hosting providers are also required to do).

m-ee 5y ago
This is not about compliance, the relevant law specifically says that companies are not required to proactively scan for CSAM.

ipv6ipv4 5y ago

I agree it is likely not malicious at all. It’s the result of koolaid in an echo chamber. And inertia at this point. However, I also think this is the poster child of the proverb that the road to hell is paved with good intentions. Now Apple needs to cancel this misguided initiative and never speak of it again, if they want to salvage some of their reputation.

sagarm 5y ago

> because it feels less like a violation of your ownership of the device. Agreed that this was not intended to be malicious. Apple has always been pretty clear they they should decide what happens on their devices. This sort of on-device scanning that doesn't serve the user is just the latest example of it, and one that people who would never be affected by the code signing restrictions can relate to.

insulanus 5y ago

Actually, I'm with you in that I think Apple's motives are different than people think, but I think yours are incorrect as well. > Apple has been, as part of their privacy initiatives, trying to do as much as possible on the device. That's how they have been defining privacy to themselves internally. Then someone said "can we do something about CSAM" [...] As a separate issue, many people in the company certainly do care about privacy, and that may go all the way to to Tim Cook. Who knows. What is much more important to Apple the company, though, is making money. Governments have been hounding them for years about letting them spy on users. And they have painted themselves into a bit of a corner, by having the most secure phones. Now the government comes to them with an offer they can't refuse, cloaked in child porn motivations. I believe many (most?) of the people involved are sincere. It's clear they have tried to make the least invasive system that still does what the government wants. But that's not good enough in the crazy connected cyber-world we find ourselves in today. Apple doesn't have a motivation to do this themselves. But they will do what they calculate they need to do.

cwkoss 5y ago
Apple has a huge potential profit motive. Once they roll this out for US users, they can sell the exact same capabilities to authoritarian regimes for detecting subversive images, images of warcrimes, etc. China would happily pay billions of dollars per year for this capability.
- thephyber 5y ago
  China already uses their police forces to mandate that certain citizens install specific apps on their phones. They also control the cloud service (a Chinese-only iCloud) that Apple uses in the China market. China doesn’t need this feature.
  - firebaze 5y ago
    Let's extrapolate a bit on this.

craftinator 5y ago

> because it feels less like a violation of your ownership of the device. It doesn't just feel like a violation, it is one. This opens doors for a number of future abuses of power, and abuses of rights. The average user has no way of knowing what's in the database of hashes being compared against, and we have no way of controlling what goes into those databases. If some party in the US government decided that they didn't like the Quran, and put a million pictures of it in the database, and served the Apple reviewers with a gag order or some other nonsense, we'd have a 80's dystopian blockbuster as everyday life in our country. My device, my property, my rules. Apple doesn't get to do a stop and frisk, even if it's "for the good of the children".

Spivak 5y ago
My device my property my rules is pretty rich talking about a platform where you can’t run your own apps without Apple’s approval, you have no control over how the OS behaves, and there is built-in DRM. Your phone has been Apple’s device since the moment you bought it and turned it on.
- craftinator 5y ago
  I run Linux on my phone, and it's not an Apple device. Other than that, I completely agree with you.

gerash 5y ago

I doubt most think this is malicious but Apple did preach "what happens on an iphone stays on the iphone" which aged terribly given this feature. Apple has also benefited from the perception that Google (their smartphone rival) spies on people by collecting location information where as I am sure they also crowd source iphone location to improve their Maps and road traffic data

pcurve 5y ago

I don't think Apple has insidious reason, but their implementation can come across as betrayal. Unlike Android phone, iphone will know about your uploading potentially problematic image beforehand. Yet, it will not alert you or block you from doing so.

neop1x 5y ago

It may not be helping child abuse enough if it does not detect newly-taken child pornography. Couple of years fast forward and machine learning will be scanning all the photos you take and watch. A little bit forward and it will be forbidden to take photos of many things and there will be no-photo places. Apple and governments will dictate what is right and what is wrong to take and watch. They are doing it for the safety of everyone, right?