10 ms·

Critically important? Really? A system that is trivial to circumvent and which only catches people sharing pics that have already been added to a database is

Critically important? Really? A system that is trivial to circumvent and which only catches people sharing pics that have already been added to a database is not going to move the needle on actual physical abuse of children.

madeofpalk 5y ago
Are you objecting to the general concept of CSAM scanning, something every online storage provider does? I think it's important to note that Apple is late to this, and they're just playing catch-up in implementing this scanning. The only difference between Apple and others is that they do it completely (or partially) on-device rather than all on their servers. And they they explicitly told people exactly what and how they're doing it all. It's kind of a funny situation where in Apple being more transparent than any other implementer of this same process, they've gotten themselves in more strife.
- fsflover 5y ago
  It doesn't matter what the purpose of this move is. It's not "catch-up" at all. The scanning will happen on the devices using the users' resources, before uploading. See also: https://news.ycombinator.com/item?id=28309202 https://news.ycombinator.com/item?id=28309202.
  - madeofpalk 5y ago
    I understand people's objection with Apple's on-device implementation of this scanning, but that didn't seem to be parent's objection. I was just trying to clarify.
  - judge2020 5y ago
    It’s catch-up in that Apple submits less than a thousand CSAM reports to NCMEC annually: > According to NCMEC, I submitted 608 reports to NCMEC in 2019, and 523 reports in 2020. In those same years, Apple submitted 205 and 265 reports (respectively). It isn't that Apple doesn't receive more picture than my service, or that they don't have more CP than I receive. Rather, it's that they don't seem to notice and therefore, don't report. > In 2020, FotoForensics received 931,466 pictures and submitted 523 reports to NCMEC; that's 0.056%. During the same year, Facebook submitted 20,307,216 reports to NCMEC. https://www.hackerfactor.com/blog/index.php?/archives/929-One-Bad-Apple.html https://www.hackerfactor.com/blog/index.php?/archives/929-On...
  - simondotau 5y ago
    If I were a US citizen, and I accepted that scanning cloud-stored photos for CSAM was neccesary, I would greatly prefer Apple's approach because any such search coerced by Government are protected by the Fifth Amendment. Whereas if the search occurs in the cloud, Government can force Apple or Google to search for whatever they want, using whichever algorithm they want.
- jpxw 5y ago
  Also, people are annoyed about the fact that the scanning is done client-side, which is ironic because Apple are doing that in order to increase the security/privacy of the system. I can see why people are uncomfortable about client-side scanning, but there’s still some amusing irony in it.
  - darksaints 5y ago
    Nothing about this "feature" increases security or privacy.
    - helen___keller 5y ago
      If you compare it against the industry standard of in-cloud scanning, the privacy comes from the fact that an on-device model results in 0 Apple servers ever examining your photos (unless you have 30 CSAM flags on your iCloud account), whereas an on-server model results in Apple servers examining every single photo you upload. You can argue it's better to have neither type of scanning, but if Apple considers it a critical business issue that their cloud is the preferred hosting site of CSAM[0], then they presumably have to pick one or the other. You can also argue that on-device seems creepier or more invasive, even if it doesn't result in Apple examining your photos, which is a reasonable reaction. It certainly breaks the illusion that it's "your" device. But it's a fact that the on-device model, as described, results in less prying eyes on your iCloud photos than the on-server model. [0] I'm not claiming this is the case, just saying for example
      - darksaints 5y ago
        None of this is an increase in security or privacy. What you described is merely a mitigation of a massive loss in security and privacy, when compared to other massive losses in security and privacy.
        helen___keller 5y ago
        > What you described is merely a mitigation of a massive loss in security and privacy If we're looking at the end result, a mitigation of a loss of privacy is an increase in privacy compared to the alternative, no? I mean clearly what you're saying here is "scanning always bad!". I understand that, I really do. I'm saying that scanning was never not on the table for a large corporation hosting photos on their server. Apple held out on the in-cloud scanning because they wanted a "better" scanning, and GP's point is that it's ironic that the one cloud provider willing to try to make a "less bad" scanning option is the one most demonized in the media. None of this is to argue that scanning is anything less than a loss in security and privacy. Yes, yes, E2EE running on free and open source software that I personally can recompile from source would be the best option.
- baggy_trough 5y ago
  That "only" is doing a huge amount of work in your comment. Turning my device into something that snitches on me is a huge difference from scanning my uploads.
  - jodrellblank 5y ago
    If you send any of those illegal photos to Facebook, Facebook will snitch on you. If you send them to iCloud (under this described system), Apple will snitch on you. It is no different to "scanning your uploads" because it /is/ "scanning your uploads". That it's the CPU in the phone doing most of the work and iCloud servers making the connection, vs Facebook's CPU and cloud doing all the work, makes zero practical difference to anything. Arguing about where in the combined system the processing happens is shifting the deck-chairs on the Titanic, it's not making one part of the system more or less responsible than any other part.
- washadjeffmad 5y ago
  They're pretty obviously not. CSAM scanning stops common proliferation of already identified materials and can help keep those from casually entering new markets. It does not protect children from being newly exploited by people using these services unless the services are also doing things they claim not to be doing. In that case, Apple's claim of critical importance, if we interpret that as any more than rhetoric, doesn't mean what they imply it to mean. Edit: I replied before you edited your comment. Leaving this one as is.
  - concinds 5y ago
    These systems rely on the premise that catching pdos and putting them out of circulation does* protect children, since many children are abused by relatives and such. I think it's a reasonable assumption!
    - only_as_i_fall 5y ago
      You'd have to actually have a reasonable chance of catching pedophiles though. If the system is as trivial to bypass as turning off photo sync it isn't a very good justification.
  - pvarangot 5y ago
    > It does not protect children from being newly exploited by people using these services unless the services are also doing things they claim not to be doing. While I only know about this through anecdotes people share, I understand this systems generate leads so that agents (usually federal) can infiltrate groups and catch people uploading new material, thus preventing them for further victimizing minors.
    - washadjeffmad 5y ago
      That smacks of those harebrained terrorist entrapment plots where the federal and undercover agents are the only members of groups they've "infiltrated". Scaling up detection also makes it more tempting for bad actors to seed more content to fabricate the appearance of an epidemic. We already have inaccurate gunshot detection systems, field drug tests, and expert bite mark analysis being used to convict people. Would a jury even be able to examine the evidence if it involved CSAM?
- newsbinator 5y ago
  Other implementers don’t scan on-device. It’s not only a matter of Apple’s openness about it. It’s the fact of what they’re announcing.
- djbebs 5y ago
  Yes, I am. I object to any and all forms of mass surveillance and spying. The fact that cloud providers aren't building their systems in such a way that they are unable of conducting such spying is cause for concern. The fact that they are conducting such spying is outrageous.
- Lamad123 5y ago
  You just said it!! Google is scanning its own servers.. This company is scanning something that is supposed to be yours!
  - vultour 5y ago
    Right before it's uploaded to their servers...
  - concinds 5y ago
    I fail to see any difference, when your data is as much "yours" whether cloud backup is turned on or off. Google and others scan data stored in Drive (and other cloud storage) not just for CP, but for copyright infringement, etc; even files not shared with anyone. Someone here just yesterday complained that Google in 2014 kept deleting personal files of ripped DVDs from their Drive. Given that many people use GDrive/OneDrive/iCloud as their main storage, where most of their PC files are stored in the cloud, I fail to see any logic in making a cloud vs personal device distinction.
    - stjohnswarts 5y ago
      When you hand it off to google it's now on their servers and as their user agreements point out your shit will be scanned and made available to law enforcement upon request and proper authority (whether that's a warrant or just Bill from computer forensics asking for it, I don't really know). When it's on your phone it is not being scanned. It's just like having to get a warrant to rifle through your house for evidence. The same should be required for personal devices. It being digital doesn't mean it should be any less safe than if it was on paper.
    - yunohn 5y ago
      Would you argue there’s no difference between the police searching a safety deposit box (which is a bad example, since it’s actually quite private) versus constantly going through and indexing the contents of your home, trying to find illegal objects?
  - Terretta 5y ago
- yyyk 5y ago
  >It's kind of a funny situation where in Apple being more transparent than any other implementer of this same process. It isn't the same process, it's an on-device scanning process and Apple is the first to implement this. Had Apple said they were scanning iCloud directly nobody bat an eye (I, for one, assumed they already did).
  - alibert 5y ago
    I would not say it is "scanning" as the device doesn't know the final result (match or not). If I had to summarize very broadly the thing to my understanding: it will be like doing a "loose" SHA on every photos in the Photos app and sending these to Apple (if iCloud is enabled). Server side, Apple checks if the hashes match or not. Isn't this like "scanning in iCloud" but without Apple needing to have a decrypted photos in their servers ?
    - yyyk 5y ago
      >I would not say it is "scanning" as the device doesn't know the final result (match or not) I don't think this is a relevant distinction. If the device had known the result and just sent Y/N to Apple, what would change in your argument? Nothing. Your last sentence would be just as arguable. >Isn't this like "scanning in iCloud" but without Apple needing to have a decrypted photos in their servers? Note that Apple right now already has the decrypted photos since they have the decryption key. There's no evidence they are even considering E2EE right now, and since there are other legal requirements for scanning (e.g. terrorist material) I'm not sure this allows them to implement E2EE. And I don't see how the client-side feature can remain as is. There are obvious cases that would be caught by the typical server-side scanning and won't be caught here, so once the door was opened, the government will pressure. For example: * What happens when the NCMEC database updates? It could be that the phone rescans all your images. Or that apple keeps the hash and rescans that. Note that the second case is identical to some server-side scanning implementations. * What happens when you upload something to iCloud, delete it from your phone and then the NCMEC database updates? If Apple keeps the hash, it's just server-side again. If Apple doesn't keep it but uses client side scanning, the phone has to keep the hashes so you can't ever truely delete an image you've taken. If there's no rescan, the bad guys get to keep CSAM on iCloud so long as they passed the original scanning with the older NCMEC database - surely the government wouldn't accept that. (I considered scanning on download but I don't know if Apple/the government would like this compromise, since with that approach CSAM can remain on iCloud undetected so long as it's not downloaded, anyway it's not in the original papers). Basically, either they scan on client-side more than they let on or we end up in a world with both server-side and client-side scanning. The latter is arguably worst of both worlds, the former has implications which need to be looked at.
- foxfluff 5y ago
  > Are you objecting to the general concept of CSAM scanning, something every online storage provider does? Not really. I'm just saying it's laughable to think that CSAM scanning in its current form is critically important. Maybe it's critically important for feelgood and PR but not for actually preventing child abuse. It's almost like saying scanning for catalogued images of violence stops violence. If it only were that easy, damn, the world would be a good place. Now as to what online providers should or shouldn't do, I can't say. But a part of me hopes that they continue the march towards egregious censorship and privacy violations so that people would eventually 𝐖𝐀𝐊𝐄 𝐓𝐇𝐄 𝐅𝐔𝐂𝐊 𝐔𝐏 and realize that there's no way to have freedom and privacy when you're handing your life to corporations and proprietary software. I really do wish for a world that is de-facto free software and fully controlled by the user. As for iCould, I have no skin in the game. I've never used an Apple product.
  - short_sells_poo 5y ago
    Unfortunately this clearly won't happen because of the staggering amount of people even on HN who see nothing wrong with having on-device scanners reporting back to HQ. I'm seriously baffled by the number of apologists and people who see nothing wrong with this or who flat out refuse to admit that this can lead to abuse. Once the on-device scanning Pandora's box is open, it's trivial for governments to request more stuff to be added to the databases and Apple can't claim the defense of "we have no such ability currently" anymore. If I was paranoid I'd wonder how much astroturfing is going on here.
    - xpe 5y ago
      > Unfortunately this clearly won't happen because of the staggering amount of people even on HN who see nothing wrong with having on-device scanners reporting back to HQ. This is a mischaracterization of many of the arguments. short_sells: My goal here is meta-goal: I am not trying to change your mind on this issue; rather I want you to acknowledge the strongest rational forms of disagreement. This is a complex issue. It is non obvious how to trade off goals of protecting kids, protecting privacy, minimizing surveillance, catching predators, and dealing with the effects of false positives. It is too simplistic (and self defeating) to think that just because people disagree with a particular conclusion that they are not allies with many of your goals.
- stjohnswarts 5y ago
  No other company is scanning for CSAM on your phone, if you have proof of that please bring it forward. I would love to see evidence of that on my Microsoft, Linux, and Android devices. We'll be waiting. Everyone and their uncle knows everything in the cloud is scanned unless it's encrypted before being uploaded.
  - jodrellblank 5y ago
    > "Everyone and their uncle knows everything in the cloud is scanned unless it's encrypted before being uploaded." And that's what Apple has announced. Comparing that to: > "No other company is scanning for CSAM on your phone" Suggests that you think the Apple system is scanning more than what you upload to the cloud, which it isn't. Or, less charitably, suggests you /want people to think it is/. Your other comment here https://news.ycombinator.com/item?id=28405476 https://news.ycombinator.com/item?id=28405476 suggests the same. And your comment here https://news.ycombinator.com/item?id=28405502 https://news.ycombinator.com/item?id=28405502 the same again.
  - kmlx 5y ago
    > No other company is scanning for CSAM on your phone no, they're just doing it in the cloud, after everything was uploaded automatically :)
- vorpalhex 5y ago
  So the only way to opt out of an intrusive, sloppy and flawed algorithm now is based on you hoping Apple sticks to their word? The same company that gives the CCP unfettered access to Chinese iCloud data?
burnished 5y ago
Regardless of the larger conversation, why do you think that adding friction wouldn't stop people from engaging in an activity? I mean, generally speaking A/B testing is used to find the path to most engagement and doesn't get this kind of pushback.. but as soon as you want to make something harder you get people saying "but that wont do anything". It demonstrably does! 'trivial to circumvent' in a world where people regularly have the experience of being called out to plug something in, when "have you checked the power button and power cord" has been a common trope for over two decades! I'm genuinely curious about your thoughts but to be clear my focus is on this very narrow, nigh nitpick tangent.
- foxfluff 5y ago
  > Regardless of the larger conversation, why do you think that adding friction wouldn't stop people from engaging in an activity? See war on drugs or piracy, or alcohol prohibition for instance. Now there's a million ways to share pictures online in a manner that bypasses the few big platforms' interference, and you really don't need to be a genius to use (say) password-protected archives. That's how a lot of casual online piracy happens. This thing does very little to prevent spreading CSAM pictures, and it does nothing to prevent actual child abuse.
  - burnished 5y ago
    Well, those things were actually pretty effective at stopping people. People clearly circumvented those restrictions, but from my perspective the number of people that smoke weed probably went up after it became easy to access in my state, and the quality and variety of products increased enormously. They weren't effective at achieving any broader aim, like stopping or slowing drug production but frankly that wasn't even the point. Regardless, I think we can both agree that these broad bans decreased the number of people participating in that behavior. I think the shift in our societies viewpoints on them came about as an awareness that they were ineffective at achieving any moral or justifiable aim. I think the correct criticism of them isn't that they failed to prevent or impede a given behavior, but that they weren't worth the negative externalities (if you'll forgive the understatement). Which brings me to your example of a password protected archive. I'm ignorant on specifics so I'm just going to sketch a broad argument and trust that you'll correct me if my premise is incorrect or my argument otherwise flawed. Essentially, if something is opt-in instead of opt-out, a non-trivial portion of the population won't do it. Especially if it is even slightly technical, there are just a lot of people who will just stop thinking as soon as they encounter a word they don't already know. So if the preventative measure is something that is not automatic and built in to whatever tool they are using to share images, then that preventative measure will not protect most of them. So to bring it full circle, I think it would do much to prevent the spreading of CSAM because other bans have been effective and I don't think most people have the technical literacy to even be aware of how to protect themselves form surveillance. As you say you don't have to be a genius, but I'd suggest you'd need to be above average, which gives you over half the population. Also thanks for responding, I hope I'm not coming across as unpleasantly argumentative, I mean all this in the most truth-seeking sense of a discussion (I was going to say 'sporting sense of a debate' when I realized I had never been part of a formal debate team and that might not mean what I thought it meant, heh).
tgv 5y ago
I'm betting a sizeable group has images from the database. They don't know which images are known, I think. But the fact that it's nearly trivial to circumvent does make me wonder.
tzs 5y ago
You (and quite a few others) seem to be making some implicit assumptions that are not at all obvious to me: 1. There is not much overlap between people sharing pictures from the database and people sharing new child porn that is not in it. 2. People sharing child porn are a lot more smarter or security conscious than the average person.
Joeri 5y ago
You would think that, but facebook detects 20 million shares of this material per year. Presumably at least part of that group that shares the known material are also abusing children and creating new material, so this does look like an effective detection method for finding some of the people who abuse children. https://www.missingkids.org/content/dam/missingkids/gethelp/2020-reports-by-esp.pdf https://www.missingkids.org/content/dam/missingkids/gethelp/...
- jrockway 5y ago
  I'm not convinced that's a good argument. Shouldn't every bag of coffee you buy include a camera that reports any illegal activity to the government? A lot of criminals drink coffee, after all!
  - c7DJTLrn 5y ago
    We must mandate speed monitoring devices for every car sold. Each time the vehicle speed exceeds the speed limit we will deduct 100 social credit points from the owner's balance. Hundreds of thousands of people die each year due to filthy criminals driving too fast. Introducing this measure will save lives. If you are uncomfortable with this measure, you drive too fast.
    - jrockway 5y ago
      Very good point. Actually, the next version of iOS should include a feature to make sure that you don't roll through a stop sign at 1MPH at 3 o'clock in the morning. STOP means stop, not kind of look both ways and carefully evaluate the situation! An Authorized Phone Receipticle built into your car will ensure that the phone's camera is facing forward so our Machine Learning Artificial Intelligence can detect stop signs, and the accelerometers will detect whether or not you stopped completely. If you didn't, your device will do some bullshit with bitcoins or something eventually leading to it sending a report to the Apple Safe Driving Headquarters ("ASDHQ") where totally trained technicians that definitely have health insurance and get paid more than $1/hour will check the data to see if it's a false positive. If it's not a false positive, and BTW, from exactly zero hours of real world testing we have determined that the false positive rate is less than 1 in 1 trillion, we'll report you to our government partners who will take care of your reeducation from there. Driving is a privilege, not a right!!! Over ONE HUNDRED people die per year from people not coming to a complete stop at stop signs, and at Apple, we care!!!!!! We'll never let the screeching voices of the minority stop us from invading your privacy for no good reason. We like the screeching, it makes us feel important and relevant. As the inventors of a small computer with a battery in it, we consider ourselves better than God.
sneak 5y ago
Not only that, but identifying those storing existing, known CSAM means that, unless they have previously unknown CSAM also, no additional harm has been done, as it's just file sharing. It's only production of new/novel CSAM that harms children. Sharing of existing, known CSAM (what this system detects) doesn't harm anyone.
Salgat 5y ago
I am so confounded on why they are burning through so much good will on something they don't even seem to really benefit from. Has anyone explained this?
- hannasanarion 5y ago
  It might only help a little bit, but it also won't hurt anyone so there's not really a reason not to. Stupid child abusers who put known-pornographic images in their iclouds are still child abusers. The fact that fruit is low hanging isn't a reason not to pick it.
- Drew_ 5y ago
  Pressure from governments of course. Apple would rather control how this works then cede control to governments/legislation. Problem is that this still doesn't stop governments from stepping in anyway. Apple has simply laid their own framework instead.

madeofpalk 5y ago

Are you objecting to the general concept of CSAM scanning, something every online storage provider does? I think it's important to note that Apple is late to this, and they're just playing catch-up in implementing this scanning. The only difference between Apple and others is that they do it completely (or partially) on-device rather than all on their servers. And they they explicitly told people exactly what and how they're doing it all. It's kind of a funny situation where in Apple being more transparent than any other implementer of this same process, they've gotten themselves in more strife.

fsflover 5y ago
It doesn't matter what the purpose of this move is. It's not "catch-up" at all. The scanning will happen on the devices using the users' resources, before uploading. See also: https://news.ycombinator.com/item?id=28309202 https://news.ycombinator.com/item?id=28309202.
- madeofpalk 5y ago
  I understand people's objection with Apple's on-device implementation of this scanning, but that didn't seem to be parent's objection. I was just trying to clarify.
- judge2020 5y ago
  It’s catch-up in that Apple submits less than a thousand CSAM reports to NCMEC annually: > According to NCMEC, I submitted 608 reports to NCMEC in 2019, and 523 reports in 2020. In those same years, Apple submitted 205 and 265 reports (respectively). It isn't that Apple doesn't receive more picture than my service, or that they don't have more CP than I receive. Rather, it's that they don't seem to notice and therefore, don't report. > In 2020, FotoForensics received 931,466 pictures and submitted 523 reports to NCMEC; that's 0.056%. During the same year, Facebook submitted 20,307,216 reports to NCMEC. https://www.hackerfactor.com/blog/index.php?/archives/929-One-Bad-Apple.html https://www.hackerfactor.com/blog/index.php?/archives/929-On...
- simondotau 5y ago
  If I were a US citizen, and I accepted that scanning cloud-stored photos for CSAM was neccesary, I would greatly prefer Apple's approach because any such search coerced by Government are protected by the Fifth Amendment. Whereas if the search occurs in the cloud, Government can force Apple or Google to search for whatever they want, using whichever algorithm they want.
jpxw 5y ago
Also, people are annoyed about the fact that the scanning is done client-side, which is ironic because Apple are doing that in order to increase the security/privacy of the system. I can see why people are uncomfortable about client-side scanning, but there’s still some amusing irony in it.
- darksaints 5y ago
  Nothing about this "feature" increases security or privacy.
  - helen___keller 5y ago
    If you compare it against the industry standard of in-cloud scanning, the privacy comes from the fact that an on-device model results in 0 Apple servers ever examining your photos (unless you have 30 CSAM flags on your iCloud account), whereas an on-server model results in Apple servers examining every single photo you upload. You can argue it's better to have neither type of scanning, but if Apple considers it a critical business issue that their cloud is the preferred hosting site of CSAM[0], then they presumably have to pick one or the other. You can also argue that on-device seems creepier or more invasive, even if it doesn't result in Apple examining your photos, which is a reasonable reaction. It certainly breaks the illusion that it's "your" device. But it's a fact that the on-device model, as described, results in less prying eyes on your iCloud photos than the on-server model. [0] I'm not claiming this is the case, just saying for example
    - darksaints 5y ago
      None of this is an increase in security or privacy. What you described is merely a mitigation of a massive loss in security and privacy, when compared to other massive losses in security and privacy.
      - helen___keller 5y ago
        > What you described is merely a mitigation of a massive loss in security and privacy If we're looking at the end result, a mitigation of a loss of privacy is an increase in privacy compared to the alternative, no? I mean clearly what you're saying here is "scanning always bad!". I understand that, I really do. I'm saying that scanning was never not on the table for a large corporation hosting photos on their server. Apple held out on the in-cloud scanning because they wanted a "better" scanning, and GP's point is that it's ironic that the one cloud provider willing to try to make a "less bad" scanning option is the one most demonized in the media. None of this is to argue that scanning is anything less than a loss in security and privacy. Yes, yes, E2EE running on free and open source software that I personally can recompile from source would be the best option.
baggy_trough 5y ago
That "only" is doing a huge amount of work in your comment. Turning my device into something that snitches on me is a huge difference from scanning my uploads.
- jodrellblank 5y ago
  If you send any of those illegal photos to Facebook, Facebook will snitch on you. If you send them to iCloud (under this described system), Apple will snitch on you. It is no different to "scanning your uploads" because it /is/ "scanning your uploads". That it's the CPU in the phone doing most of the work and iCloud servers making the connection, vs Facebook's CPU and cloud doing all the work, makes zero practical difference to anything. Arguing about where in the combined system the processing happens is shifting the deck-chairs on the Titanic, it's not making one part of the system more or less responsible than any other part.
washadjeffmad 5y ago
They're pretty obviously not. CSAM scanning stops common proliferation of already identified materials and can help keep those from casually entering new markets. It does not protect children from being newly exploited by people using these services unless the services are also doing things they claim not to be doing. In that case, Apple's claim of critical importance, if we interpret that as any more than rhetoric, doesn't mean what they imply it to mean. Edit: I replied before you edited your comment. Leaving this one as is.
- concinds 5y ago
  These systems rely on the premise that catching pdos and putting them out of circulation does* protect children, since many children are abused by relatives and such. I think it's a reasonable assumption!
  - only_as_i_fall 5y ago
    You'd have to actually have a reasonable chance of catching pedophiles though. If the system is as trivial to bypass as turning off photo sync it isn't a very good justification.
- pvarangot 5y ago
  > It does not protect children from being newly exploited by people using these services unless the services are also doing things they claim not to be doing. While I only know about this through anecdotes people share, I understand this systems generate leads so that agents (usually federal) can infiltrate groups and catch people uploading new material, thus preventing them for further victimizing minors.
  - washadjeffmad 5y ago
    That smacks of those harebrained terrorist entrapment plots where the federal and undercover agents are the only members of groups they've "infiltrated". Scaling up detection also makes it more tempting for bad actors to seed more content to fabricate the appearance of an epidemic. We already have inaccurate gunshot detection systems, field drug tests, and expert bite mark analysis being used to convict people. Would a jury even be able to examine the evidence if it involved CSAM?
newsbinator 5y ago
Other implementers don’t scan on-device. It’s not only a matter of Apple’s openness about it. It’s the fact of what they’re announcing.
djbebs 5y ago
Yes, I am. I object to any and all forms of mass surveillance and spying. The fact that cloud providers aren't building their systems in such a way that they are unable of conducting such spying is cause for concern. The fact that they are conducting such spying is outrageous.
Lamad123 5y ago
You just said it!! Google is scanning its own servers.. This company is scanning something that is supposed to be yours!
- vultour 5y ago
  Right before it's uploaded to their servers...
- concinds 5y ago
  I fail to see any difference, when your data is as much "yours" whether cloud backup is turned on or off. Google and others scan data stored in Drive (and other cloud storage) not just for CP, but for copyright infringement, etc; even files not shared with anyone. Someone here just yesterday complained that Google in 2014 kept deleting personal files of ripped DVDs from their Drive. Given that many people use GDrive/OneDrive/iCloud as their main storage, where most of their PC files are stored in the cloud, I fail to see any logic in making a cloud vs personal device distinction.
  - stjohnswarts 5y ago
    When you hand it off to google it's now on their servers and as their user agreements point out your shit will be scanned and made available to law enforcement upon request and proper authority (whether that's a warrant or just Bill from computer forensics asking for it, I don't really know). When it's on your phone it is not being scanned. It's just like having to get a warrant to rifle through your house for evidence. The same should be required for personal devices. It being digital doesn't mean it should be any less safe than if it was on paper.
  - yunohn 5y ago
    Would you argue there’s no difference between the police searching a safety deposit box (which is a bad example, since it’s actually quite private) versus constantly going through and indexing the contents of your home, trying to find illegal objects?
- Terretta 5y ago
yyyk 5y ago
>It's kind of a funny situation where in Apple being more transparent than any other implementer of this same process. It isn't the same process, it's an on-device scanning process and Apple is the first to implement this. Had Apple said they were scanning iCloud directly nobody bat an eye (I, for one, assumed they already did).
- alibert 5y ago
  I would not say it is "scanning" as the device doesn't know the final result (match or not). If I had to summarize very broadly the thing to my understanding: it will be like doing a "loose" SHA on every photos in the Photos app and sending these to Apple (if iCloud is enabled). Server side, Apple checks if the hashes match or not. Isn't this like "scanning in iCloud" but without Apple needing to have a decrypted photos in their servers ?
  - yyyk 5y ago
    >I would not say it is "scanning" as the device doesn't know the final result (match or not) I don't think this is a relevant distinction. If the device had known the result and just sent Y/N to Apple, what would change in your argument? Nothing. Your last sentence would be just as arguable. >Isn't this like "scanning in iCloud" but without Apple needing to have a decrypted photos in their servers? Note that Apple right now already has the decrypted photos since they have the decryption key. There's no evidence they are even considering E2EE right now, and since there are other legal requirements for scanning (e.g. terrorist material) I'm not sure this allows them to implement E2EE. And I don't see how the client-side feature can remain as is. There are obvious cases that would be caught by the typical server-side scanning and won't be caught here, so once the door was opened, the government will pressure. For example: * What happens when the NCMEC database updates? It could be that the phone rescans all your images. Or that apple keeps the hash and rescans that. Note that the second case is identical to some server-side scanning implementations. * What happens when you upload something to iCloud, delete it from your phone and then the NCMEC database updates? If Apple keeps the hash, it's just server-side again. If Apple doesn't keep it but uses client side scanning, the phone has to keep the hashes so you can't ever truely delete an image you've taken. If there's no rescan, the bad guys get to keep CSAM on iCloud so long as they passed the original scanning with the older NCMEC database - surely the government wouldn't accept that. (I considered scanning on download but I don't know if Apple/the government would like this compromise, since with that approach CSAM can remain on iCloud undetected so long as it's not downloaded, anyway it's not in the original papers). Basically, either they scan on client-side more than they let on or we end up in a world with both server-side and client-side scanning. The latter is arguably worst of both worlds, the former has implications which need to be looked at.
foxfluff 5y ago
> Are you objecting to the general concept of CSAM scanning, something every online storage provider does? Not really. I'm just saying it's laughable to think that CSAM scanning in its current form is critically important. Maybe it's critically important for feelgood and PR but not for actually preventing child abuse. It's almost like saying scanning for catalogued images of violence stops violence. If it only were that easy, damn, the world would be a good place. Now as to what online providers should or shouldn't do, I can't say. But a part of me hopes that they continue the march towards egregious censorship and privacy violations so that people would eventually 𝐖𝐀𝐊𝐄 𝐓𝐇𝐄 𝐅𝐔𝐂𝐊 𝐔𝐏 and realize that there's no way to have freedom and privacy when you're handing your life to corporations and proprietary software. I really do wish for a world that is de-facto free software and fully controlled by the user. As for iCould, I have no skin in the game. I've never used an Apple product.
- short_sells_poo 5y ago
  Unfortunately this clearly won't happen because of the staggering amount of people even on HN who see nothing wrong with having on-device scanners reporting back to HQ. I'm seriously baffled by the number of apologists and people who see nothing wrong with this or who flat out refuse to admit that this can lead to abuse. Once the on-device scanning Pandora's box is open, it's trivial for governments to request more stuff to be added to the databases and Apple can't claim the defense of "we have no such ability currently" anymore. If I was paranoid I'd wonder how much astroturfing is going on here.
  - xpe 5y ago
    > Unfortunately this clearly won't happen because of the staggering amount of people even on HN who see nothing wrong with having on-device scanners reporting back to HQ. This is a mischaracterization of many of the arguments. short_sells: My goal here is meta-goal: I am not trying to change your mind on this issue; rather I want you to acknowledge the strongest rational forms of disagreement. This is a complex issue. It is non obvious how to trade off goals of protecting kids, protecting privacy, minimizing surveillance, catching predators, and dealing with the effects of false positives. It is too simplistic (and self defeating) to think that just because people disagree with a particular conclusion that they are not allies with many of your goals.
stjohnswarts 5y ago
No other company is scanning for CSAM on your phone, if you have proof of that please bring it forward. I would love to see evidence of that on my Microsoft, Linux, and Android devices. We'll be waiting. Everyone and their uncle knows everything in the cloud is scanned unless it's encrypted before being uploaded.
- jodrellblank 5y ago
  > "Everyone and their uncle knows everything in the cloud is scanned unless it's encrypted before being uploaded." And that's what Apple has announced. Comparing that to: > "No other company is scanning for CSAM on your phone" Suggests that you think the Apple system is scanning more than what you upload to the cloud, which it isn't. Or, less charitably, suggests you /want people to think it is/. Your other comment here https://news.ycombinator.com/item?id=28405476 https://news.ycombinator.com/item?id=28405476 suggests the same. And your comment here https://news.ycombinator.com/item?id=28405502 https://news.ycombinator.com/item?id=28405502 the same again.
- kmlx 5y ago
  > No other company is scanning for CSAM on your phone no, they're just doing it in the cloud, after everything was uploaded automatically :)
vorpalhex 5y ago
So the only way to opt out of an intrusive, sloppy and flawed algorithm now is based on you hoping Apple sticks to their word? The same company that gives the CCP unfettered access to Chinese iCloud data?

burnished 5y ago

Regardless of the larger conversation, why do you think that adding friction wouldn't stop people from engaging in an activity? I mean, generally speaking A/B testing is used to find the path to most engagement and doesn't get this kind of pushback.. but as soon as you want to make something harder you get people saying "but that wont do anything". It demonstrably does! 'trivial to circumvent' in a world where people regularly have the experience of being called out to plug something in, when "have you checked the power button and power cord" has been a common trope for over two decades! I'm genuinely curious about your thoughts but to be clear my focus is on this very narrow, nigh nitpick tangent.

foxfluff 5y ago
> Regardless of the larger conversation, why do you think that adding friction wouldn't stop people from engaging in an activity? See war on drugs or piracy, or alcohol prohibition for instance. Now there's a million ways to share pictures online in a manner that bypasses the few big platforms' interference, and you really don't need to be a genius to use (say) password-protected archives. That's how a lot of casual online piracy happens. This thing does very little to prevent spreading CSAM pictures, and it does nothing to prevent actual child abuse.
- burnished 5y ago
  Well, those things were actually pretty effective at stopping people. People clearly circumvented those restrictions, but from my perspective the number of people that smoke weed probably went up after it became easy to access in my state, and the quality and variety of products increased enormously. They weren't effective at achieving any broader aim, like stopping or slowing drug production but frankly that wasn't even the point. Regardless, I think we can both agree that these broad bans decreased the number of people participating in that behavior. I think the shift in our societies viewpoints on them came about as an awareness that they were ineffective at achieving any moral or justifiable aim. I think the correct criticism of them isn't that they failed to prevent or impede a given behavior, but that they weren't worth the negative externalities (if you'll forgive the understatement). Which brings me to your example of a password protected archive. I'm ignorant on specifics so I'm just going to sketch a broad argument and trust that you'll correct me if my premise is incorrect or my argument otherwise flawed. Essentially, if something is opt-in instead of opt-out, a non-trivial portion of the population won't do it. Especially if it is even slightly technical, there are just a lot of people who will just stop thinking as soon as they encounter a word they don't already know. So if the preventative measure is something that is not automatic and built in to whatever tool they are using to share images, then that preventative measure will not protect most of them. So to bring it full circle, I think it would do much to prevent the spreading of CSAM because other bans have been effective and I don't think most people have the technical literacy to even be aware of how to protect themselves form surveillance. As you say you don't have to be a genius, but I'd suggest you'd need to be above average, which gives you over half the population. Also thanks for responding, I hope I'm not coming across as unpleasantly argumentative, I mean all this in the most truth-seeking sense of a discussion (I was going to say 'sporting sense of a debate' when I realized I had never been part of a formal debate team and that might not mean what I thought it meant, heh).

tgv 5y ago

I'm betting a sizeable group has images from the database. They don't know which images are known, I think. But the fact that it's nearly trivial to circumvent does make me wonder.

tzs 5y ago

You (and quite a few others) seem to be making some implicit assumptions that are not at all obvious to me: 1. There is not much overlap between people sharing pictures from the database and people sharing new child porn that is not in it. 2. People sharing child porn are a lot more smarter or security conscious than the average person.

Joeri 5y ago

You would think that, but facebook detects 20 million shares of this material per year. Presumably at least part of that group that shares the known material are also abusing children and creating new material, so this does look like an effective detection method for finding some of the people who abuse children. https://www.missingkids.org/content/dam/missingkids/gethelp/2020-reports-by-esp.pdf https://www.missingkids.org/content/dam/missingkids/gethelp/...

jrockway 5y ago
I'm not convinced that's a good argument. Shouldn't every bag of coffee you buy include a camera that reports any illegal activity to the government? A lot of criminals drink coffee, after all!
- c7DJTLrn 5y ago
  We must mandate speed monitoring devices for every car sold. Each time the vehicle speed exceeds the speed limit we will deduct 100 social credit points from the owner's balance. Hundreds of thousands of people die each year due to filthy criminals driving too fast. Introducing this measure will save lives. If you are uncomfortable with this measure, you drive too fast.
  - jrockway 5y ago
    Very good point. Actually, the next version of iOS should include a feature to make sure that you don't roll through a stop sign at 1MPH at 3 o'clock in the morning. STOP means stop, not kind of look both ways and carefully evaluate the situation! An Authorized Phone Receipticle built into your car will ensure that the phone's camera is facing forward so our Machine Learning Artificial Intelligence can detect stop signs, and the accelerometers will detect whether or not you stopped completely. If you didn't, your device will do some bullshit with bitcoins or something eventually leading to it sending a report to the Apple Safe Driving Headquarters ("ASDHQ") where totally trained technicians that definitely have health insurance and get paid more than $1/hour will check the data to see if it's a false positive. If it's not a false positive, and BTW, from exactly zero hours of real world testing we have determined that the false positive rate is less than 1 in 1 trillion, we'll report you to our government partners who will take care of your reeducation from there. Driving is a privilege, not a right!!! Over ONE HUNDRED people die per year from people not coming to a complete stop at stop signs, and at Apple, we care!!!!!! We'll never let the screeching voices of the minority stop us from invading your privacy for no good reason. We like the screeching, it makes us feel important and relevant. As the inventors of a small computer with a battery in it, we consider ourselves better than God.

sneak 5y ago

Not only that, but identifying those storing existing, known CSAM means that, unless they have previously unknown CSAM also, no additional harm has been done, as it's just file sharing. It's only production of new/novel CSAM that harms children. Sharing of existing, known CSAM (what this system detects) doesn't harm anyone.

Salgat 5y ago

I am so confounded on why they are burning through so much good will on something they don't even seem to really benefit from. Has anyone explained this?

hannasanarion 5y ago
It might only help a little bit, but it also won't hurt anyone so there's not really a reason not to. Stupid child abusers who put known-pornographic images in their iclouds are still child abusers. The fact that fruit is low hanging isn't a reason not to pick it.
Drew_ 5y ago
Pressure from governments of course. Apple would rather control how this works then cede control to governments/legislation. Problem is that this still doesn't stop governments from stepping in anyway. Apple has simply laid their own framework instead.