14 ms·

> I just don't understand how they acted this way at all. There's a simple answer to this right? Despite everyone's reaction, Apple genuinely believe this is a

> I just don't understand how they acted this way at all. There's a simple answer to this right? Despite everyone's reaction, Apple genuinely believe this is a novel and unique method to catch CSAM without invading people's privacy. And if you look at it from Apple's point of view that's correct: other major cloud providers catch CSAM content on their platform by inspecting every file uploaded, i.e. total invasion of privacy. Apple found a way to preserve that privacy but still catch the bad people doing very bad things to children.

lixtra 5y ago
> other major cloud providers catch CSAM content on their platform by inspecting every file uploaded That is very unlikely. Most likely they compare some hash against a database - just like apple.
- jowsie 5y ago
  That's what they're saying, the key difference is Apples happens on your device, not theirs.
  - GiorgioG 5y ago
    That's not really better or different. In any meaningful way.
    - Spivak 5y ago
      Then why the outrage if they're the same?
      - JimBlackwood 5y ago
        Because this is Apple and people are very passionate about both hating and liking Apple.
    - nine_k 5y ago
      It is indeed meaningfully different: your sensitive data never leaves your device in order to be scanned. There can't be a crack in Apple servers that would expose files of millions of users uploaded for scanning. Eventually it could lead to Apple platform not having any CSAM content, or any known legally objectionable content, because any sane perpetrator would migrate to other platforms, and less sane, caught. This, of course, takes the user base to overwhelmingly agree that keeping legally objectionable content is a bad thing and should be exposed to police, and to trust the whatever body which defines the hashes of objectionable content. I'm afraid this order is not as tall as we could imagine.
      - heavyset_go 5y ago
        > your sensitive data never leaves your device in order to be scanned. There can't be a crack in Apple servers that would expose files of millions of users uploaded for scanning. And yet photos that get scanned are still uploaded to iCloud Photos, so they do end up on Apple's servers.
        GeekyBear 5y ago
        Photos that users choose to upload to iCloud Photos do indeed get uploaded to iCloud Photos? I'm failing to see the issue.
        heavyset_go 5y ago
        I think you might be tilting at windmills here. The issue is that the post I'm replying to claims that data never ends up on Apple's servers even though it does. Thanks for confirming that it does, though.
        GeekyBear 5y ago
        Am I supposed to be shocked that photos the user uploads to iCloud are on iCloud? The results of the scan looking for kiddie porn cannot be read by Apple until the device finds 30 examples of photos that match known kiddie porn, whereupon Apple gets the decryption key so they can see which images on their server need review, and a human review is triggered to make sure there haven't been 30 false positives.
    - GeekyBear 5y ago
      Apple keeps the scan results encrypted with a key they don't have until the device informs them that the threshold of 30 images that match known kiddie porn has been reached. After that, they get the decryption key and trigger a human review to make sure there haven't been 30 false positives at once. That is better, and more private, in a very meaningful way. False positive scan data sitting on the server is open to malicious misuse by anyone who can get a warrant. >Innocent man, 23, sues Arizona police for $1.5million after being arrested for murder and jailed for six days when Google's GPS tracker wrongly placed him at the scene of the 2018 crime https://www.dailymail.co.uk/news/article-7897319/Police-arrested-innocent-man-murder-using-Google-location-data.html https://www.dailymail.co.uk/news/article-7897319/Police-arre... Also, I sincerely doubt that Google has ended it's practice of refusing to hire a human being to check for false positives when a poorly performing algorithm is cheaper.
      - kelnos 5y ago
        I do agree that it's more private, but I'm not sure it's better. I'm fine with the idea that if I upload stuff to someone else's server, they may take a look at it and maybe even punish me for what I've uploaded. Certainly if I encrypt the data before I upload it, they can't do that. But if I don't, then it's fine with me if they do. But my device should not be snitching on me. Yes, this device-side scanning is supposedly gated on enabling uploads to iCloud, but that doesn't really make for much of a distinction to me. And since Apple certainly has the capability, they are likely one secret court order away from being required to scan even photos that aren't being uploaded, at least on some targeted subset of devices.
        tshaddox 5y ago
        > And since Apple certainly has the capability, they are likely one secret court order away from being required to scan even photos that aren't being uploaded, at least on some targeted subset of devices. Apple has the ability to upload literally any software to iPhones, so this argument applies equally to literally any conceivable bad thing that software could do on iPhones.
  - orangeoxidation 5y ago
    Making it obviously and unquestionably more invasive.
    - Spivak 5y ago
      I just don't get this. Say you're given two options when going through the TSA. 1. The TSA agent opens your luggage and searches everything for banned items. 2. The TSA agent hands you a scanner for you to wave over your luggage in private, it prints out a receipt of banned items it saw, and you present that receipt to the agent. Which one is more invasive?
      - rfd4sgmk8u 5y ago
        1. Cops on the street 2. Cops in your house Which one is more invasive? I don't think there is much to not understand. Apple is proposing putting a cop in your house. On device scanning is vastly vastly different than server side scanning. One is over apples property, the other is over YOUR property. Its a big deal, and a huge change in policy. And as myself and others have indicated from years of observation, it will grow and grow...
      - deleted 5y ago
        [deleted]
      - cma 5y ago
        The TSA installs this scanner in your house so it can detect before you even get to the airport. They give a pinky promise they will only run it when you book your ticket and intend on going to the airport where you would be expected to be scanned anyway. The scanner technology can also be used for detecting dissident journalism, but they say it won't be, it is just for preventing terrorism, but still they say, they really need to install it in your house even though it is only for scanning when you intend on going to the airport.
        bsql 5y ago
        > The scanner technology can also be used for detecting dissident journalism, but they say it won't be, it is just for preventing terrorism This has always been possible with server side scanning. Yeah we have to trust Apple to not enable it for users who don’t use iCloud photos but we’ve always had to trust Apple given their closed source system.
mox1 5y ago
But who was complaining about google and microsoft doing the cloud scanning? I don’t mind my one drive being scanned for “bad stuff”, I very much mind my personally owned data stores being scanned, with no opt out.
- GeekyBear 5y ago
  The only thing Apple scans are files you upload to iCloud Photos. If you turn off iCloud Photos, nothing is scanned. Microsoft scans everything. >The system that scans cloud drives for illegal images was created by Microsoft and Dartmouth College and donated to NCMEC. The organization creates signatures of the worst known images of child pornography, approximately 16,000 files at present. These file signatures are given to service providers who then try to match them to user files in order to prevent further distribution of the images themselves, a Microsoft spokesperson told NBC News. (Microsoft implemented image-matching technology in its own services, such as Bing and SkyDrive.) https://www.nbcnews.com/technolog/your-cloud-drive-really-private-not-according-fine-print-1c8881731 https://www.nbcnews.com/technolog/your-cloud-drive-really-pr...
  - Waterluvian 5y ago
    I have no idea what I expected but 16,000 photos feels… disgustingly high and ridiculously low at the same time.
    - ffhhj 5y ago
      It's well known that there are also videos. So these criminals won't be caught watching them.
  - kelnos 5y ago
    If Microsoft's scan is a server-side scan, how do they scan "everything" if "everything" includes things that you do not upload? By your definition of "everything", both Microsoft and Apple scan "everything". (Or Apple will, after this new system is rolled out.)
  - erklik 5y ago
mschuster91 5y ago
> Apple found a way to preserve that privacy but still catch the bad people doing very bad things to children. They're not going to find predators, all they are going to find is people incompetent enough to have years old (otherwise how would it end up at NCMEC?) CSAM on their iPhones and dumb enough to have iCloud backup turned on. To catch actual predators they would need to run AI scanning on phones for all photos and risk a lot of false positives by parents taking photos of their children on a beach. Plus a load of people who will inadvertently have stuff that "looks" like CSAM on their devices because some 4chan trolls will inevitably create colliding material and spread it in ads etc. so it ends up downloaded in browser caches. All of this "let's use technology to counter pedos" is utter, utter crap that is not going to save one single child and will only serve to tear down our freedoms, one law at a time. Want to fight against pedos? Make photos of your airbnb, hotel and motel rooms to help identify abuse recording locations and timeframes, and teach children from an early age about their bodies, sexuality and consent so that they actually have the words to tell you that they are being molested.
- zepto 5y ago
  > people incompetent enough to have years old (otherwise how would it end up at NCMEC?) CSAM on their iPhones Who do you think has 30 or more CSAM images on their phone?
  - atq2119 5y ago
    Let's be real, of course many of those people will be child abusers. But let's also be real about something else. Think of the Venn diagram of child abusers and people who share CSAM online. Those circles are not identical. Not everybody with CSAM is necessarily a child abuser. Worse, how many child abusers don't share CSAM online? Those can't ever be found by Apple-style invasions of privacy, so one has to wonder if we're being asked to give up significant privacy for a crime fighting strategy that may not even be all that effective. The cynic in me is also wondering what fraction of people working at places like NCMEC are pedophiles. It'd be a rather convenient job for them. And after all, there's a long history of ostensibly child-serving institutions harboring the worst kind of offenders.
carnitas 5y ago
Apple inspects every file on the local device Before its uploaded. It’s just pinky promise only matched with the on device database when an upload is intended.
- slg 5y ago
  Apple controls the hardware, software, and cloud service. It was always a pinky promise that they wouldn't look at your files. I don't know why we should doubt that pinky promise less today than we did a month ago.
  - fsflover 5y ago
    Because now Apple confirmed themselves that this promise is not kept.
    - slg 5y ago
      What is "this promise"? Because I would consider it "we will only scan files that you upload to iCloud". That was true a month ago and that would be true under this new system. The only part that is changing is that the scanning happens on your device before upload rather than on an Apple server after upload. I don't view that as a material difference when Apple already controls the hardware and software on both ends. If we can't trust Apple to follow their promise, their products should already have been considered compromised before this change was announced.
      - fsflover 5y ago
        > The only part that is changing is that the scanning happens on your device before upload This is the key point. 1. What if I change my mind and decide not to upload the picture? 2. This is a new mechanism for scanning private pictures on the device. What could go wrong? > If we can't trust Apple to follow their promise, their products should already have been considered compromised before this change was announced. Many people did trust Apple to keep their files private until now.
unyttigfjelltol 5y ago
Maybe Apple was focused on keeping CSAM off a cloud resource that really is Apple's in both a practical and technical sense. Merely scanning after upload maybe doesn't accomplish that because the material already has arrived in an Apple resource and someone might define that as a failure. Viewed from a perspective of iCloud compliance it sorta makes sense. No one ought to feel a lot of (legal) security deploying a system that could be used to host CSAM, from the Blockchain to whatever, it sounds like a don't-wizz-on-the-electric-fence kind of risk. Saddened by the privacy-adverse functionality on handsets but Apple seems to be hitting the nail on the head that poor communication principally is driving outrage.
- Syonyk 5y ago
  With the new system, it still arrives on the server, and they can't even know about it until some threshold has been passed - and then the servers still have to scan it. So it's not even accomplishing "keeping it off the servers."
  - simondotau 5y ago
    Actually it'll do an excellent job of keeping it off their servers. Apple's actions will be effective at making sure every CSAM aficionado will not trust their device. Or, if they're tech savvy, they will at least know not to co-mingle their deepest darkest secrets alongside photos of their mum and last night's dinner. If you think Apple's very big, very public blow-up is making waves in the hacker/security/libertarian crowd, just imagine how big it's blowing up in the CSAM community right now. I dare say it's probably all they've been talking about for the past two weeks. Unlike places like Hacker News where there's plenty of people desperate to drag Apple through the coals, the CSAM community will be highly motivated to have a precise understanding of what Apple is doing. I dare say that they'll be extremely well informed about exactly what Apple's plans entail and how to work around them.
noapologies 5y ago
It's really interesting to see the mental gymnastics people are willing to go through to defend their favorite trillion dollar corporations. > other major cloud providers catch CSAM content on their platform by inspecting every file uploaded, i.e. total invasion of privacy. > Apple found a way to preserve that privacy ... So scanning for CSAM in a third-party cloud is "total invasion of privacy", while scanning your own personal device is "preserving privacy"? The third-party clouds can only scan what one explicitly chooses to share with the third-party, while on-device scanning is a short slippery slope away from have its scope significantly expanded (to include non-shared and non-CSAM content).
- helen___keller 5y ago
  > defend their favorite trillion dollar corporations > is a short slippery slope away Obviously people who trust Apple aren't concerned about slippery slopes. What's the point of your post?
  - stjohnswarts 5y ago
    He's just commenting on how ludicrous it is for anyone to believe that apple is being honest about this and about spying for governments on its users.
- spullara 5y ago
  They are scanning files that are being uploaded. So, yes.
  - noapologies 5y ago
    As I mentioned, it is (by definition) impossible for the cloud to have access to something that does not exist in the cloud - only the content that is explicitly uploaded/shared by the user can be scanned. While local, on-device content access has no such guarantees - it's a question of policy rather than technical infeasibility. And policies change. > They are scanning files that are being uploaded. So, yes. Even considering the situation as it is today, the cloud providers (as well as Apple) are scanning just the files that are being uploaded. So I'm still not sure how one is "total invasion of privacy" and the other "preserving privacy".
speleding 5y ago
> catch the bad people doing very bad things to children You may catch a few perverts looking at the stuff, but I'm not convinced this will lead to catching the producers. How would that happen?
- shuckles 5y ago
  Known CSAM detection was one of three features they announced under the banner of child safety. Certainly they understand it is by itself an incomplete intervention for a much larger problem.
- kelnos 5y ago
  The theory is that if you make it too risky to possess CSAM, demand for it will drop, leading to less exploitation. I don't know if I buy that theory, but... there it is. Compare this to the war on drugs. There is huge demand for drugs. Many drugs are highly illegal to possess, and a lot of people get jailed for possessing them. And yet most of us consider the war on drugs to be an abject failure that hasn't done much more than create inequity. Why should something like CSAM possession be any different? I wish it was different, and these sorts of tactics would reduce this awful problem, but I'm just not convinced it does.
mirker 5y ago
> Apple genuinely believe this is a novel and unique method to catch CSAM without invading people's privacy What’s novel about this? The technique and issues with it are fairly obvious to anyone with experience in computer vision. It seems not too different from research from 1993 https://proceedings.neurips.cc/paper/1993/file/288cc0ff022877bd3df94bc9360b9c5d-Paper.pdf https://proceedings.neurips.cc/paper/1993/file/288cc0ff02287... The issues are also well known and encompass the whole subfield of adversarial examples.
- shuckles 5y ago
  Presumably the system is novel enough that a very similar proposal which lacked one important insight (using perceptual hashes to turn image similarity problems into set intersection problems) was accepted to USENIX this year: https://www.usenix.org/conference/usenixsecurity21/presentation/kulshrestha https://www.usenix.org/conference/usenixsecurity21/presentat...
  - mirker 5y ago
    The privacy aspect does seem novel, and that paper evaluates the various aspects of privacy and the quality of perceptual hashes. However, anything concerning privacy (of client or server data) is orthogonal to the vision methods underpinning the system, which are well understood and are the primary cause of the post. For that matter, I don’t see how the paper’s proofs of privacy and/or formalization of the problem help with hash collisions. And that particular paper doesn’t use deep learning methods which are probably easier to attack with off-the-shelf methods.
mapgrep 5y ago
> other major cloud providers catch CSAM content on their platform by inspecting every file uploaded So does Apple. EDIT: some people don’t like that answer, but “inspecting” in this context clearly means “digitally inspecting” (Google does not physically look at every file) and Apple does this with files that are uploaded. They do it in device, but it’s still inspected. That’s the whole point of this controversy, that there’s not much difference to people WHERE apple inspects and on device is actually arguably worse. Your sentence does not in any way distinguish what Apple does from what others do.
dabbledash 5y ago
There is no way to scan people’s content while “respecting their privacy.” The goal should be to create a system where you couldn’t do so even of you wanted to (or the state demanded it).
- chrismorgan 5y ago
  > The goal should be to create a system where you couldn’t do so even of you wanted to (or the state demanded it). There’s this bizarre notion that using end-to-end encryption can absolve you of responsibility, that the authorities will have to accept an answer of “we literally can’t access it”. That’s just not the case for centralised things: you’re deliberately facilitating some service, government will find you liable for some things in its operation, and if you don’t comply, they’ll fine or shut you down. E2EE doesn’t absolve you from law; law is all about saying you’re not allowed to do things that are physically possible. (Decentralised things, now they can be banned but not truly stopped because there’s no central party to shut down.)
  - dabbledash 5y ago
    There’s nothing stopping governments from banning E2EE, but in the absence of such bans, no one is under any obligation to build systems that empower them to spy on their users.
    - Retric 5y ago
      I wouldn’t be sure about that, the phone companies already have a legal obligation to allow wiretapping and the government is very happy to put gag orders on this stuff.
      - pengaru 5y ago
        Shouldn't the E2EE apple walled-garden app equivalent of wiretapping be pushing an app update to the suspect's phone with a sidechannel added for law enforcement to snoop, with warrant in hand?
      - 5y ago
varispeed 5y ago
> Apple found a way to preserve that privacy This is Orwellian doublespeak. You don't preserver someone's privacy by snooping on their devices.

lixtra 5y ago

> other major cloud providers catch CSAM content on their platform by inspecting every file uploaded That is very unlikely. Most likely they compare some hash against a database - just like apple.

jowsie 5y ago
That's what they're saying, the key difference is Apples happens on your device, not theirs.
- GiorgioG 5y ago
  That's not really better or different. In any meaningful way.
  - Spivak 5y ago
    Then why the outrage if they're the same?
    - JimBlackwood 5y ago
      Because this is Apple and people are very passionate about both hating and liking Apple.
  - nine_k 5y ago
    It is indeed meaningfully different: your sensitive data never leaves your device in order to be scanned. There can't be a crack in Apple servers that would expose files of millions of users uploaded for scanning. Eventually it could lead to Apple platform not having any CSAM content, or any known legally objectionable content, because any sane perpetrator would migrate to other platforms, and less sane, caught. This, of course, takes the user base to overwhelmingly agree that keeping legally objectionable content is a bad thing and should be exposed to police, and to trust the whatever body which defines the hashes of objectionable content. I'm afraid this order is not as tall as we could imagine.
    - heavyset_go 5y ago
      > your sensitive data never leaves your device in order to be scanned. There can't be a crack in Apple servers that would expose files of millions of users uploaded for scanning. And yet photos that get scanned are still uploaded to iCloud Photos, so they do end up on Apple's servers.
      - GeekyBear 5y ago
        Photos that users choose to upload to iCloud Photos do indeed get uploaded to iCloud Photos? I'm failing to see the issue.
        heavyset_go 5y ago
        I think you might be tilting at windmills here. The issue is that the post I'm replying to claims that data never ends up on Apple's servers even though it does. Thanks for confirming that it does, though.
        GeekyBear 5y ago
        Am I supposed to be shocked that photos the user uploads to iCloud are on iCloud? The results of the scan looking for kiddie porn cannot be read by Apple until the device finds 30 examples of photos that match known kiddie porn, whereupon Apple gets the decryption key so they can see which images on their server need review, and a human review is triggered to make sure there haven't been 30 false positives.
  - GeekyBear 5y ago
    Apple keeps the scan results encrypted with a key they don't have until the device informs them that the threshold of 30 images that match known kiddie porn has been reached. After that, they get the decryption key and trigger a human review to make sure there haven't been 30 false positives at once. That is better, and more private, in a very meaningful way. False positive scan data sitting on the server is open to malicious misuse by anyone who can get a warrant. >Innocent man, 23, sues Arizona police for $1.5million after being arrested for murder and jailed for six days when Google's GPS tracker wrongly placed him at the scene of the 2018 crime https://www.dailymail.co.uk/news/article-7897319/Police-arrested-innocent-man-murder-using-Google-location-data.html https://www.dailymail.co.uk/news/article-7897319/Police-arre... Also, I sincerely doubt that Google has ended it's practice of refusing to hire a human being to check for false positives when a poorly performing algorithm is cheaper.
    - kelnos 5y ago
      I do agree that it's more private, but I'm not sure it's better. I'm fine with the idea that if I upload stuff to someone else's server, they may take a look at it and maybe even punish me for what I've uploaded. Certainly if I encrypt the data before I upload it, they can't do that. But if I don't, then it's fine with me if they do. But my device should not be snitching on me. Yes, this device-side scanning is supposedly gated on enabling uploads to iCloud, but that doesn't really make for much of a distinction to me. And since Apple certainly has the capability, they are likely one secret court order away from being required to scan even photos that aren't being uploaded, at least on some targeted subset of devices.
      - tshaddox 5y ago
        > And since Apple certainly has the capability, they are likely one secret court order away from being required to scan even photos that aren't being uploaded, at least on some targeted subset of devices. Apple has the ability to upload literally any software to iPhones, so this argument applies equally to literally any conceivable bad thing that software could do on iPhones.
- orangeoxidation 5y ago
  Making it obviously and unquestionably more invasive.
  - Spivak 5y ago
    I just don't get this. Say you're given two options when going through the TSA. 1. The TSA agent opens your luggage and searches everything for banned items. 2. The TSA agent hands you a scanner for you to wave over your luggage in private, it prints out a receipt of banned items it saw, and you present that receipt to the agent. Which one is more invasive?
    - rfd4sgmk8u 5y ago
      1. Cops on the street 2. Cops in your house Which one is more invasive? I don't think there is much to not understand. Apple is proposing putting a cop in your house. On device scanning is vastly vastly different than server side scanning. One is over apples property, the other is over YOUR property. Its a big deal, and a huge change in policy. And as myself and others have indicated from years of observation, it will grow and grow...
    - deleted 5y ago
      [deleted]
    - cma 5y ago
      The TSA installs this scanner in your house so it can detect before you even get to the airport. They give a pinky promise they will only run it when you book your ticket and intend on going to the airport where you would be expected to be scanned anyway. The scanner technology can also be used for detecting dissident journalism, but they say it won't be, it is just for preventing terrorism, but still they say, they really need to install it in your house even though it is only for scanning when you intend on going to the airport.
      - bsql 5y ago
        > The scanner technology can also be used for detecting dissident journalism, but they say it won't be, it is just for preventing terrorism This has always been possible with server side scanning. Yeah we have to trust Apple to not enable it for users who don’t use iCloud photos but we’ve always had to trust Apple given their closed source system.

mox1 5y ago

But who was complaining about google and microsoft doing the cloud scanning? I don’t mind my one drive being scanned for “bad stuff”, I very much mind my personally owned data stores being scanned, with no opt out.

GeekyBear 5y ago
The only thing Apple scans are files you upload to iCloud Photos. If you turn off iCloud Photos, nothing is scanned. Microsoft scans everything. >The system that scans cloud drives for illegal images was created by Microsoft and Dartmouth College and donated to NCMEC. The organization creates signatures of the worst known images of child pornography, approximately 16,000 files at present. These file signatures are given to service providers who then try to match them to user files in order to prevent further distribution of the images themselves, a Microsoft spokesperson told NBC News. (Microsoft implemented image-matching technology in its own services, such as Bing and SkyDrive.) https://www.nbcnews.com/technolog/your-cloud-drive-really-private-not-according-fine-print-1c8881731 https://www.nbcnews.com/technolog/your-cloud-drive-really-pr...
- Waterluvian 5y ago
  I have no idea what I expected but 16,000 photos feels… disgustingly high and ridiculously low at the same time.
  - ffhhj 5y ago
    It's well known that there are also videos. So these criminals won't be caught watching them.
- kelnos 5y ago
  If Microsoft's scan is a server-side scan, how do they scan "everything" if "everything" includes things that you do not upload? By your definition of "everything", both Microsoft and Apple scan "everything". (Or Apple will, after this new system is rolled out.)
- erklik 5y ago

mschuster91 5y ago

> Apple found a way to preserve that privacy but still catch the bad people doing very bad things to children. They're not going to find predators, all they are going to find is people incompetent enough to have years old (otherwise how would it end up at NCMEC?) CSAM on their iPhones and dumb enough to have iCloud backup turned on. To catch actual predators they would need to run AI scanning on phones for all photos and risk a lot of false positives by parents taking photos of their children on a beach. Plus a load of people who will inadvertently have stuff that "looks" like CSAM on their devices because some 4chan trolls will inevitably create colliding material and spread it in ads etc. so it ends up downloaded in browser caches. All of this "let's use technology to counter pedos" is utter, utter crap that is not going to save one single child and will only serve to tear down our freedoms, one law at a time. Want to fight against pedos? Make photos of your airbnb, hotel and motel rooms to help identify abuse recording locations and timeframes, and teach children from an early age about their bodies, sexuality and consent so that they actually have the words to tell you that they are being molested.

zepto 5y ago
> people incompetent enough to have years old (otherwise how would it end up at NCMEC?) CSAM on their iPhones Who do you think has 30 or more CSAM images on their phone?
- atq2119 5y ago
  Let's be real, of course many of those people will be child abusers. But let's also be real about something else. Think of the Venn diagram of child abusers and people who share CSAM online. Those circles are not identical. Not everybody with CSAM is necessarily a child abuser. Worse, how many child abusers don't share CSAM online? Those can't ever be found by Apple-style invasions of privacy, so one has to wonder if we're being asked to give up significant privacy for a crime fighting strategy that may not even be all that effective. The cynic in me is also wondering what fraction of people working at places like NCMEC are pedophiles. It'd be a rather convenient job for them. And after all, there's a long history of ostensibly child-serving institutions harboring the worst kind of offenders.

carnitas 5y ago

Apple inspects every file on the local device Before its uploaded. It’s just pinky promise only matched with the on device database when an upload is intended.

slg 5y ago
Apple controls the hardware, software, and cloud service. It was always a pinky promise that they wouldn't look at your files. I don't know why we should doubt that pinky promise less today than we did a month ago.
- fsflover 5y ago
  Because now Apple confirmed themselves that this promise is not kept.
  - slg 5y ago
    What is "this promise"? Because I would consider it "we will only scan files that you upload to iCloud". That was true a month ago and that would be true under this new system. The only part that is changing is that the scanning happens on your device before upload rather than on an Apple server after upload. I don't view that as a material difference when Apple already controls the hardware and software on both ends. If we can't trust Apple to follow their promise, their products should already have been considered compromised before this change was announced.
    - fsflover 5y ago
      > The only part that is changing is that the scanning happens on your device before upload This is the key point. 1. What if I change my mind and decide not to upload the picture? 2. This is a new mechanism for scanning private pictures on the device. What could go wrong? > If we can't trust Apple to follow their promise, their products should already have been considered compromised before this change was announced. Many people did trust Apple to keep their files private until now.

unyttigfjelltol 5y ago

Maybe Apple was focused on keeping CSAM off a cloud resource that really is Apple's in both a practical and technical sense. Merely scanning after upload maybe doesn't accomplish that because the material already has arrived in an Apple resource and someone might define that as a failure. Viewed from a perspective of iCloud compliance it sorta makes sense. No one ought to feel a lot of (legal) security deploying a system that could be used to host CSAM, from the Blockchain to whatever, it sounds like a don't-wizz-on-the-electric-fence kind of risk. Saddened by the privacy-adverse functionality on handsets but Apple seems to be hitting the nail on the head that poor communication principally is driving outrage.

Syonyk 5y ago
With the new system, it still arrives on the server, and they can't even know about it until some threshold has been passed - and then the servers still have to scan it. So it's not even accomplishing "keeping it off the servers."
- simondotau 5y ago
  Actually it'll do an excellent job of keeping it off their servers. Apple's actions will be effective at making sure every CSAM aficionado will not trust their device. Or, if they're tech savvy, they will at least know not to co-mingle their deepest darkest secrets alongside photos of their mum and last night's dinner. If you think Apple's very big, very public blow-up is making waves in the hacker/security/libertarian crowd, just imagine how big it's blowing up in the CSAM community right now. I dare say it's probably all they've been talking about for the past two weeks. Unlike places like Hacker News where there's plenty of people desperate to drag Apple through the coals, the CSAM community will be highly motivated to have a precise understanding of what Apple is doing. I dare say that they'll be extremely well informed about exactly what Apple's plans entail and how to work around them.

noapologies 5y ago

It's really interesting to see the mental gymnastics people are willing to go through to defend their favorite trillion dollar corporations. > other major cloud providers catch CSAM content on their platform by inspecting every file uploaded, i.e. total invasion of privacy. > Apple found a way to preserve that privacy ... So scanning for CSAM in a third-party cloud is "total invasion of privacy", while scanning your own personal device is "preserving privacy"? The third-party clouds can only scan what one explicitly chooses to share with the third-party, while on-device scanning is a short slippery slope away from have its scope significantly expanded (to include non-shared and non-CSAM content).

helen___keller 5y ago
> defend their favorite trillion dollar corporations > is a short slippery slope away Obviously people who trust Apple aren't concerned about slippery slopes. What's the point of your post?
- stjohnswarts 5y ago
  He's just commenting on how ludicrous it is for anyone to believe that apple is being honest about this and about spying for governments on its users.
spullara 5y ago
They are scanning files that are being uploaded. So, yes.
- noapologies 5y ago
  As I mentioned, it is (by definition) impossible for the cloud to have access to something that does not exist in the cloud - only the content that is explicitly uploaded/shared by the user can be scanned. While local, on-device content access has no such guarantees - it's a question of policy rather than technical infeasibility. And policies change. > They are scanning files that are being uploaded. So, yes. Even considering the situation as it is today, the cloud providers (as well as Apple) are scanning just the files that are being uploaded. So I'm still not sure how one is "total invasion of privacy" and the other "preserving privacy".

speleding 5y ago

> catch the bad people doing very bad things to children You may catch a few perverts looking at the stuff, but I'm not convinced this will lead to catching the producers. How would that happen?

shuckles 5y ago
Known CSAM detection was one of three features they announced under the banner of child safety. Certainly they understand it is by itself an incomplete intervention for a much larger problem.
kelnos 5y ago
The theory is that if you make it too risky to possess CSAM, demand for it will drop, leading to less exploitation. I don't know if I buy that theory, but... there it is. Compare this to the war on drugs. There is huge demand for drugs. Many drugs are highly illegal to possess, and a lot of people get jailed for possessing them. And yet most of us consider the war on drugs to be an abject failure that hasn't done much more than create inequity. Why should something like CSAM possession be any different? I wish it was different, and these sorts of tactics would reduce this awful problem, but I'm just not convinced it does.

mirker 5y ago

> Apple genuinely believe this is a novel and unique method to catch CSAM without invading people's privacy What’s novel about this? The technique and issues with it are fairly obvious to anyone with experience in computer vision. It seems not too different from research from 1993 https://proceedings.neurips.cc/paper/1993/file/288cc0ff022877bd3df94bc9360b9c5d-Paper.pdf https://proceedings.neurips.cc/paper/1993/file/288cc0ff02287... The issues are also well known and encompass the whole subfield of adversarial examples.

shuckles 5y ago
Presumably the system is novel enough that a very similar proposal which lacked one important insight (using perceptual hashes to turn image similarity problems into set intersection problems) was accepted to USENIX this year: https://www.usenix.org/conference/usenixsecurity21/presentation/kulshrestha https://www.usenix.org/conference/usenixsecurity21/presentat...
- mirker 5y ago
  The privacy aspect does seem novel, and that paper evaluates the various aspects of privacy and the quality of perceptual hashes. However, anything concerning privacy (of client or server data) is orthogonal to the vision methods underpinning the system, which are well understood and are the primary cause of the post. For that matter, I don’t see how the paper’s proofs of privacy and/or formalization of the problem help with hash collisions. And that particular paper doesn’t use deep learning methods which are probably easier to attack with off-the-shelf methods.

mapgrep 5y ago

> other major cloud providers catch CSAM content on their platform by inspecting every file uploaded So does Apple. EDIT: some people don’t like that answer, but “inspecting” in this context clearly means “digitally inspecting” (Google does not physically look at every file) and Apple does this with files that are uploaded. They do it in device, but it’s still inspected. That’s the whole point of this controversy, that there’s not much difference to people WHERE apple inspects and on device is actually arguably worse. Your sentence does not in any way distinguish what Apple does from what others do.

dabbledash 5y ago

There is no way to scan people’s content while “respecting their privacy.” The goal should be to create a system where you couldn’t do so even of you wanted to (or the state demanded it).

chrismorgan 5y ago
> The goal should be to create a system where you couldn’t do so even of you wanted to (or the state demanded it). There’s this bizarre notion that using end-to-end encryption can absolve you of responsibility, that the authorities will have to accept an answer of “we literally can’t access it”. That’s just not the case for centralised things: you’re deliberately facilitating some service, government will find you liable for some things in its operation, and if you don’t comply, they’ll fine or shut you down. E2EE doesn’t absolve you from law; law is all about saying you’re not allowed to do things that are physically possible. (Decentralised things, now they can be banned but not truly stopped because there’s no central party to shut down.)
- dabbledash 5y ago
  There’s nothing stopping governments from banning E2EE, but in the absence of such bans, no one is under any obligation to build systems that empower them to spy on their users.
  - Retric 5y ago
    I wouldn’t be sure about that, the phone companies already have a legal obligation to allow wiretapping and the government is very happy to put gag orders on this stuff.
    - pengaru 5y ago
      Shouldn't the E2EE apple walled-garden app equivalent of wiretapping be pushing an app update to the suspect's phone with a sidechannel added for law enforcement to snoop, with warrant in hand?
    - 5y ago

varispeed 5y ago

> Apple found a way to preserve that privacy This is Orwellian doublespeak. You don't preserver someone's privacy by snooping on their devices.