8 ms·

This is needless Apple apologetics. They deserve to be raked over the coals for this, there's no world where their current design is a "good" or "right" one.

by zionic 5y ago

This is needless Apple apologetics. They deserve to be raked over the coals for this, there's no world where their current design is a "good" or "right" one. Child abuse is a serious problem, but building a surveillance panopticon is not an acceptable solution to it. Better investment in education, health care, and reporting hotlines are the way forward to stop this issue at its source.

JasonFruit 5y ago
Right? Should cars have facial recognition cameras so they don't enable child trafficking?
- nicce 5y ago
  Not relevant example as Apple is only required to stop redistribution of existing material. They are not forced to look for new material.
  - cvwright 5y ago
    But the point is, once you accept something noble and difficult like "preventing CSAM" as your primary overriding goal, then there's nothing that's too far or too extreme if it will help you with your noble goal. Five years ago, the idea of Apple scanning photos on your phone would have been absurd. Five years from now, what will people think about hotels installing AI-powered cameras in every room? The vendor swears they only start recording when they detect an act of abuse. It sounds absurd now, but where do you draw the line?
    - nicce 5y ago
      Many (maybe 5) years ago Apple launched Neural Net to categorize your photos and this scans all of them, a lot whether they are in the cloud or not. Difference is, that we don’t know where this information is stored. Still nobody is worried about that. Feature, which allows more than this newly added CSAM functionality. If someone wants to misuse that in hidden, there is no difference of now or future. Because all we have is trust. Speculation suddenly raises, when common politic reasons are mentioned. It does not really matter if the scanning happens on device or iCloud in this situation, because you have to always trust their closed source system. Google has scanned your images since 2009 in the cloud unencrypted, but now when Apple makes situation better, it is suddenly bad. All tools have been out there already. There are no really other options to get more privacy than this, but people refuse to see that. Well, there is voting. Vote people who puts privacy over everything. That would make everything easy.
      - giantrobot 5y ago
        > Google has scanned your images since 2009 in the cloud unencrypted, but now when Apple makes situation better, it is suddenly bad. At the moment Apple's scanning policy is about the same as it was before. They claim they're only scanning photos if iCloud photos are enabled. The change they're advertising is doing the actual scanning process locally. The problem is two fold. The first is Apple went from scanning only explicitly uploaded content to local content. Since they've decided to intrude on local content once "for the children" it's not out of the realm of possibility (if not likely) they will make further intrusions in the future for prima facie noble reasons. Are third party apps going to be restricted on saving data unless they allow access to Apple's CSAM scanner? Will it start scanning texts or e-mails tomorrow letting and rando flood a person's phone with CSAM and get them arrested? Adding a local scanning system like this is a slippery slope. The second problem is the opaqueness of the system. This has multiple sub-problems. While the NCMEC has a laudable goal, involving them in the CSAM scanning process involves an outsize level of trust I don't think they have earned. They have law enforcement's unfortunate disdain for personal privacy coupled with a fanatical devotion to their cause. They believe their actions are always correct and just so long as they supposedly serve their goal of "protecting children". Due to the opaque nature of their content library it's not crazy to think repressive regimes will get self-serving content added to the source libraries for CSAM scanning. There's plenty of places where homosexuality is punishable by death and even mildly anti-government content will land you in jail. Obviously you and I can't go look at NCMEC/ICMEC CSAM libraries to check for falsely added content. So how are we supposed to trust a system run by fanatics to not have simple errors? Which leads to the other opaqueness sub-problem. Apple's design is interesting, if not laudable, but is closed source and full of black boxes. PhotoDNA, NeuralHash, and the like are not published algorithms anyone can verify. We don't even have a way of knowing if some image we have tripped a false positive and have to trust Apple's unknown "threshold" isn't 1. So not only does the public, the subject of these new intrusions, have no way of auditing the database but they have no way of auditing the code or process. A stupid bug in the scanning system could get a user reported to Apple which we then have to trust not to forward (and not to have additional bugs in their reporting system) them to law enforcement and ruin their life. So I am concerned with scope creep and bugs/false positives. I can live with a bug that causes video playback to stutter or a black box system in Maps that gives me the wrong hours for a restaurant. It's much harder to live with bugs that can get me arrested or even killed thanks to trigger happy police. Apple's system might be technically adept but their promises of future behavior aren't trustworthy since they've already changed their behavior with this new system.
  - Zak 5y ago
    Apple isn't required to stop redistribution of existing material; they're required to make a report if they have actual knowledge of users possessing or distributing apparent CSAM. This is different from what your comment implies in two ways. First, they do not have an obligation to actively look for CSAM; they only incur an obligation if they find it. Second, the obligation applies to apparent illegal content rather than known illegal content. What qualifies as apparent could end up in court. https://uscode.house.gov/view.xhtml?req=granuleid:USC-prelim-title18-section2258A&num=0&edition=prelim https://uscode.house.gov/view.xhtml?req=granuleid:USC-prelim...
    - nicce 5y ago
      > Apple isn't required to stop redistribution of existing material; they're required to make a report if they have actual knowledge of users possessing or distributing apparent CSAM. This isn't that simple. If NCMEC comes with the properties of CSAM (e.g. hashes) and asks provider especially those to be removed from their cloud, it is hard to remove them without looking for them. This is different than an obligation to actively look for CSAM in general.
      - Zak 5y ago
        Can you cite a statute that requires a provider to look for hashes when NCMEC asks them to? If NCMEC told a provider that a specific URL (or similarly unique identifier) contains CSAM, the provider would be obligated to destroy the associated file or be guilty of possession/distribution because at that point they know what they have. That's different from NCMEC providing hashes that could identify files the provider may or may not be storing.
camillomiller 5y ago
I think it’s actually a good way to look at the problem from a different, broader, perspective that isn’t the average HN user and privacy minded individual standpoint. Also, it interprets Apple’s decisions in the wider framework of their B2C business. Apple’s privacy engineers don’t have the luxury of being radical like their critics when it comes to taking a decision like this. Given this state of things, have they picked the lesser of two evils to solve the thorny problem of CSAM detection? I think it’s fair to say yes, they did, while still criticizing them for it (which is what they were of course expecting anyway).
- thesimon 5y ago
  > Given this state of things, have they picked the lesser of two evils to solve the thorny problem of CSAM detection? I think it’s fair to say yes, they did, Can you describe the two options they had?
  - nicce 5y ago
    First option is not to encrypt data at all (current state, server side does not count), second option is to use end-to-end encryption with hidden backdoor. They found a (third) way, to lock themselves out of most of the data, and for example FBI can’t ask them to show some arbitrary images. Sorry, this was three options.
- notheretoo 5y ago
  It’s not a privacy minded standpoint. It’s the god damn 4th amendment.
- nbzso 5y ago
  Nope. If scanning was implemented outside the device on the iCloud as everyone else, may be. But this is intrusion of privacy on a new "on device surveillance" level and Apple deserves hostile reaction. No form of apologetic or "technical" explanation can remove this from reality now. They are betting heavily on their "core" demographics to trust them automatically and without any form of critical thinking. If this implementation has no effect on Apples bottom line. Things are over. We will live in badly implemented version of the Minority Report.
wegs 5y ago
I think one of the problems here is a reality and perception don't align: - Apple has over a billion devices out there. - Child abuse is a rare problem, but with over a billion devices, there will be enough of it for a lot of newsworthy stories. - Child pornography takes just one abused child for an arbitrary number of viewers. Arguably, by the time you're limiting the number of viewers, most of the harm has been done. On the whole, I'm not quite sure how the Apple plan will protect actual children from rape (except to somewhat reduce the secondary harm of distribution). I can clearly see how it will protect Apple from bad press, though -- people won't use iPhones to record that. On the other hand, an investment in education, health care, reporting, and enforcement could significantly reduce the amount of child abuse, but with 7 billion people in the world, no expense would bring it to zero. So long as it's not zero, the potential for bad press is there. Indeed, usually if something happens a few times per year, it receives more bad press than if it happens a few times per day. Apple has every incentive to be (1) seen as doing something (2) do things which protect its brand value. Apple has no incentive to invest in education, health care, reporting, and enforcement. Those seem like good things to do, but if anything, if a scandal comes up, those sorts of things are used to say "See, Apple new, and was trying to buy an out." As a footnote, if we value all children equally, a lot of this is super-cheap. This is a good movie: https://en.wikipedia.org/wiki/Born_into_Brothels https://en.wikipedia.org/wiki/Born_into_Brothels And the problem it portrays could probably be solved with the same finances as the salaries of a few Apple engineers, and a focused, targeted effort to identify child prostitutes, help their families with the economics which force those kids to become child prostitutes, and get those kids into schools instead. I'm guessing the $100k raised from this film will do more to protect kids than this whole Apple initiative will do.
- throwavocado 5y ago
  > On the whole, I'm not quite sure how the Apple plan will protect actual children from rape (except to somewhat reduce the secondary harm of distribution). You bring up the distinction between "possession offenses" (i.e., a person who has CSAM content) and "hands-on offenses" (i.e., a person who abuses children and possibly, but not necessarily, produces CSAM). Detecting possession offenses (as Apple's sytem does) has the second-order effect of finding hands-on offenders because hands-on offenders tend to also collect CSAM and form large libraries of it. So finding a CSAM collection is the best way to find a hands-on offender and stop their abuse. Ideally, victims would always disclose their abuse so that the traditional investigatory process could handle it -- but child sexual abuse is special in that offenders are skilled in manipulating children and families in order to avoid detection. I think that the case of USA v. Rosenchein [0] is a good example because it shows the ins and outs of how the company->NCMEC->law enforcement system tends to work and how it leads to hands-on offenders. It's higher profile than most, perhaps because the defendant (a surgeon), seems to have plenty of resources for fighting the conviction on constitutional grounds (as opposed to actually claiming innocence). But the mechanism leading to the prosecution is by no means exceptional. Caveat: Not a lawyer. [0] https://www.anylaw.com/case/usa-v-rosenchein/d-new-mexico/11-11-2020/U23IwXUBvjaUG3Ruy4-x https://www.anylaw.com/case/usa-v-rosenchein/d-new-mexico/11...
SquishyPanda23 5y ago
> This is needless Apple apologetics. Just to be clear, I am neither defending nor attacking Apple. I don't even own any Apple devices. I'm just giving my interpretation of the dynamics behind what is going on.

by zionic 5y ago

JasonFruit 5y ago

Right? Should cars have facial recognition cameras so they don't enable child trafficking?

nicce 5y ago
Not relevant example as Apple is only required to stop redistribution of existing material. They are not forced to look for new material.
- cvwright 5y ago
  But the point is, once you accept something noble and difficult like "preventing CSAM" as your primary overriding goal, then there's nothing that's too far or too extreme if it will help you with your noble goal. Five years ago, the idea of Apple scanning photos on your phone would have been absurd. Five years from now, what will people think about hotels installing AI-powered cameras in every room? The vendor swears they only start recording when they detect an act of abuse. It sounds absurd now, but where do you draw the line?
  - nicce 5y ago
    Many (maybe 5) years ago Apple launched Neural Net to categorize your photos and this scans all of them, a lot whether they are in the cloud or not. Difference is, that we don’t know where this information is stored. Still nobody is worried about that. Feature, which allows more than this newly added CSAM functionality. If someone wants to misuse that in hidden, there is no difference of now or future. Because all we have is trust. Speculation suddenly raises, when common politic reasons are mentioned. It does not really matter if the scanning happens on device or iCloud in this situation, because you have to always trust their closed source system. Google has scanned your images since 2009 in the cloud unencrypted, but now when Apple makes situation better, it is suddenly bad. All tools have been out there already. There are no really other options to get more privacy than this, but people refuse to see that. Well, there is voting. Vote people who puts privacy over everything. That would make everything easy.
    - giantrobot 5y ago
      > Google has scanned your images since 2009 in the cloud unencrypted, but now when Apple makes situation better, it is suddenly bad. At the moment Apple's scanning policy is about the same as it was before. They claim they're only scanning photos if iCloud photos are enabled. The change they're advertising is doing the actual scanning process locally. The problem is two fold. The first is Apple went from scanning only explicitly uploaded content to local content. Since they've decided to intrude on local content once "for the children" it's not out of the realm of possibility (if not likely) they will make further intrusions in the future for prima facie noble reasons. Are third party apps going to be restricted on saving data unless they allow access to Apple's CSAM scanner? Will it start scanning texts or e-mails tomorrow letting and rando flood a person's phone with CSAM and get them arrested? Adding a local scanning system like this is a slippery slope. The second problem is the opaqueness of the system. This has multiple sub-problems. While the NCMEC has a laudable goal, involving them in the CSAM scanning process involves an outsize level of trust I don't think they have earned. They have law enforcement's unfortunate disdain for personal privacy coupled with a fanatical devotion to their cause. They believe their actions are always correct and just so long as they supposedly serve their goal of "protecting children". Due to the opaque nature of their content library it's not crazy to think repressive regimes will get self-serving content added to the source libraries for CSAM scanning. There's plenty of places where homosexuality is punishable by death and even mildly anti-government content will land you in jail. Obviously you and I can't go look at NCMEC/ICMEC CSAM libraries to check for falsely added content. So how are we supposed to trust a system run by fanatics to not have simple errors? Which leads to the other opaqueness sub-problem. Apple's design is interesting, if not laudable, but is closed source and full of black boxes. PhotoDNA, NeuralHash, and the like are not published algorithms anyone can verify. We don't even have a way of knowing if some image we have tripped a false positive and have to trust Apple's unknown "threshold" isn't 1. So not only does the public, the subject of these new intrusions, have no way of auditing the database but they have no way of auditing the code or process. A stupid bug in the scanning system could get a user reported to Apple which we then have to trust not to forward (and not to have additional bugs in their reporting system) them to law enforcement and ruin their life. So I am concerned with scope creep and bugs/false positives. I can live with a bug that causes video playback to stutter or a black box system in Maps that gives me the wrong hours for a restaurant. It's much harder to live with bugs that can get me arrested or even killed thanks to trigger happy police. Apple's system might be technically adept but their promises of future behavior aren't trustworthy since they've already changed their behavior with this new system.
- Zak 5y ago
  Apple isn't required to stop redistribution of existing material; they're required to make a report if they have actual knowledge of users possessing or distributing apparent CSAM. This is different from what your comment implies in two ways. First, they do not have an obligation to actively look for CSAM; they only incur an obligation if they find it. Second, the obligation applies to apparent illegal content rather than known illegal content. What qualifies as apparent could end up in court. https://uscode.house.gov/view.xhtml?req=granuleid:USC-prelim-title18-section2258A&num=0&edition=prelim https://uscode.house.gov/view.xhtml?req=granuleid:USC-prelim...
  - nicce 5y ago
    > Apple isn't required to stop redistribution of existing material; they're required to make a report if they have actual knowledge of users possessing or distributing apparent CSAM. This isn't that simple. If NCMEC comes with the properties of CSAM (e.g. hashes) and asks provider especially those to be removed from their cloud, it is hard to remove them without looking for them. This is different than an obligation to actively look for CSAM in general.
    - Zak 5y ago
      Can you cite a statute that requires a provider to look for hashes when NCMEC asks them to? If NCMEC told a provider that a specific URL (or similarly unique identifier) contains CSAM, the provider would be obligated to destroy the associated file or be guilty of possession/distribution because at that point they know what they have. That's different from NCMEC providing hashes that could identify files the provider may or may not be storing.

camillomiller 5y ago

I think it’s actually a good way to look at the problem from a different, broader, perspective that isn’t the average HN user and privacy minded individual standpoint. Also, it interprets Apple’s decisions in the wider framework of their B2C business. Apple’s privacy engineers don’t have the luxury of being radical like their critics when it comes to taking a decision like this. Given this state of things, have they picked the lesser of two evils to solve the thorny problem of CSAM detection? I think it’s fair to say yes, they did, while still criticizing them for it (which is what they were of course expecting anyway).

thesimon 5y ago
> Given this state of things, have they picked the lesser of two evils to solve the thorny problem of CSAM detection? I think it’s fair to say yes, they did, Can you describe the two options they had?
- nicce 5y ago
  First option is not to encrypt data at all (current state, server side does not count), second option is to use end-to-end encryption with hidden backdoor. They found a (third) way, to lock themselves out of most of the data, and for example FBI can’t ask them to show some arbitrary images. Sorry, this was three options.
notheretoo 5y ago
It’s not a privacy minded standpoint. It’s the god damn 4th amendment.
nbzso 5y ago
Nope. If scanning was implemented outside the device on the iCloud as everyone else, may be. But this is intrusion of privacy on a new "on device surveillance" level and Apple deserves hostile reaction. No form of apologetic or "technical" explanation can remove this from reality now. They are betting heavily on their "core" demographics to trust them automatically and without any form of critical thinking. If this implementation has no effect on Apples bottom line. Things are over. We will live in badly implemented version of the Minority Report.

wegs 5y ago

I think one of the problems here is a reality and perception don't align: - Apple has over a billion devices out there. - Child abuse is a rare problem, but with over a billion devices, there will be enough of it for a lot of newsworthy stories. - Child pornography takes just one abused child for an arbitrary number of viewers. Arguably, by the time you're limiting the number of viewers, most of the harm has been done. On the whole, I'm not quite sure how the Apple plan will protect actual children from rape (except to somewhat reduce the secondary harm of distribution). I can clearly see how it will protect Apple from bad press, though -- people won't use iPhones to record that. On the other hand, an investment in education, health care, reporting, and enforcement could significantly reduce the amount of child abuse, but with 7 billion people in the world, no expense would bring it to zero. So long as it's not zero, the potential for bad press is there. Indeed, usually if something happens a few times per year, it receives more bad press than if it happens a few times per day. Apple has every incentive to be (1) seen as doing something (2) do things which protect its brand value. Apple has no incentive to invest in education, health care, reporting, and enforcement. Those seem like good things to do, but if anything, if a scandal comes up, those sorts of things are used to say "See, Apple new, and was trying to buy an out." As a footnote, if we value all children equally, a lot of this is super-cheap. This is a good movie: https://en.wikipedia.org/wiki/Born_into_Brothels https://en.wikipedia.org/wiki/Born_into_Brothels And the problem it portrays could probably be solved with the same finances as the salaries of a few Apple engineers, and a focused, targeted effort to identify child prostitutes, help their families with the economics which force those kids to become child prostitutes, and get those kids into schools instead. I'm guessing the $100k raised from this film will do more to protect kids than this whole Apple initiative will do.

throwavocado 5y ago
> On the whole, I'm not quite sure how the Apple plan will protect actual children from rape (except to somewhat reduce the secondary harm of distribution). You bring up the distinction between "possession offenses" (i.e., a person who has CSAM content) and "hands-on offenses" (i.e., a person who abuses children and possibly, but not necessarily, produces CSAM). Detecting possession offenses (as Apple's sytem does) has the second-order effect of finding hands-on offenders because hands-on offenders tend to also collect CSAM and form large libraries of it. So finding a CSAM collection is the best way to find a hands-on offender and stop their abuse. Ideally, victims would always disclose their abuse so that the traditional investigatory process could handle it -- but child sexual abuse is special in that offenders are skilled in manipulating children and families in order to avoid detection. I think that the case of USA v. Rosenchein [0] is a good example because it shows the ins and outs of how the company->NCMEC->law enforcement system tends to work and how it leads to hands-on offenders. It's higher profile than most, perhaps because the defendant (a surgeon), seems to have plenty of resources for fighting the conviction on constitutional grounds (as opposed to actually claiming innocence). But the mechanism leading to the prosecution is by no means exceptional. Caveat: Not a lawyer. [0] https://www.anylaw.com/case/usa-v-rosenchein/d-new-mexico/11-11-2020/U23IwXUBvjaUG3Ruy4-x https://www.anylaw.com/case/usa-v-rosenchein/d-new-mexico/11...

SquishyPanda23 5y ago

> This is needless Apple apologetics. Just to be clear, I am neither defending nor attacking Apple. I don't even own any Apple devices. I'm just giving my interpretation of the dynamics behind what is going on.