5 ms·

Alex Stamos on Apple's new child safety protections

76 points by sbuccini 5y ago

kragen 5y ago
https://twitr.gq/alexstamos/status/1424054542879006728 https://twitr.gq/alexstamos/status/1424054542879006728 Links to https://storage.courtlistener.com/recap/gov.uscourts.insd.77308/gov.uscourts.insd.77308.172.0.pdf https://storage.courtlistener.com/recap/gov.uscourts.insd.77... (90 pages of harrowing public-domain court documents obtained through RECAP, the system Aaron Swartz first got investigated by the FBI for setting up; they document the misdeeds of a guy named Buster Hernandez who serially raped children over Facebook)
wyager 5y ago
> For the last couple of years, our team at @stanfordio has been hosting a series of conferences on how to balance the safety and privacy aspects of E2EE products. I am not interested in “balancing” anything. This is not a negotiation and it is not open to compromise. The opposite of privacy is not “safety”; it is surveillance. I refuse to use products that actively surveil me behind my back. Consumers deserve products that work for consumer interests, not products that ultimately serve someone else.
- deleted 5y ago
  [deleted]
- caeril 5y ago
  Indeed. However, there is no point getting mad at Alex. He used to be a real hacker, until he sold his soul for mammon at Facebook. Once someone like this makes the transition from hacker to wiping one's ass with the hacker manifesto, there's no turning back, and you just need to accept that their words are hollow. Nothing against Alex. He's brilliant, and you never know if you're offered $XMM to compromise your principles, you won't make the same decision until you're actually in that position. I'm not smart enough to ever be in his position, and I'd like to believe that if I were, I would hold true to principle, but who knows? We're all flawed.
- bluepizza 5y ago
  That is a naive and immature point of view. Technology enables crime in terrible ways, and those crimes need to be curtailed. Don't like it? Buy a different phone.
dvt 5y ago
The thread has a point, and there's definitely nuance here, but I think people throughout history have litigated this many times over, and we came to the same conclusion over and over and over again. To argue the contrary is revisionist or simply ignorant. This conclusion is one that has pushed our species to be more charitable, more law-abiding, more free, less tyrannical, more fair, etc. Not that we've reached some kind of utopia, but let's face it: the world is much better off than it was 100, 200, or 1000 years ago. The conclusion is that freedom (that is: negative, not positive rights) is the main force of progress (societal, technological, scientific). Of course, we have law enforcement, and judges, and lawyers, and an entire legal machinery: but it functions within well-defined boundaries. In the US, we have warrants. We have habeas corpus. We have chain of custody, and so on. Apple is fundamentally circumventing these norms. And this is wrong, no matter what is on the other side of the equation. I'm not sure what Stamos' counter-argument is here. I don't always agree with the EFF, but I think in this case, they're right on the money.
- bluepizza 5y ago
  Is it? How are these norms being circumvented?
  - dvt 5y ago
    By reporting potential criminals to the authorities by scanning what is essentially private data (images, messages), I'd likely argue they're violating unlawful search and seizure and I doubt a conviction would stick[1]. It's not like my landlord can barge in my apartment and look under the couch for illegal drugs because he heard a rumor I bought some. I mean, probable cause is already sketchy enough, what Apple doing is beyond the pale. [1] https://reason.com/volokh/2021/08/05/apple-plans-to-scan-us-iphones-for-child-abuse-imagery/ https://reason.com/volokh/2021/08/05/apple-plans-to-scan-us-...
    - ceejayoz 5y ago
      > By reporting potential criminals to the authorities by scanning what is essentially private data (images, messages), I'd likely argue they're violating unlawful search and seizure and I doubt a conviction would stick[1]. Why? Police use informants to obtain probable cause regularly.
joe_the_user 5y ago
Interestingly, "there are no easy answers" seems to be an ... easy answer to a lot of pretty obvious and compelling objections. This kind of thing is often said to "balance" between "your plan is despicable and ridiculous" and "yeah but the pressure to do something is huge"
dmm 5y ago
> I also don't understand why Apple is pushing the CSAM scanning for iCloud into the device Because it's a pretense for further surveillance.
- caeril 5y ago
  He understands, he's just playing the game.
- alisonkisk 5y ago
  Because they don't want to decrypt your non-child-porn files!
  - wyager 5y ago
    iCloud images are not encrypted. This is a concession Apple had already made to the FBI.
- Niksko 5y ago
  One theory put out by John Gruber was that the scanning was being done in device so that they could E2E encrypt iCloud in the future. So they satisfy the 'think of the children people', while giving you strong encryption elsewhere. Time will tell.
- simondotau 5y ago
  > Because it's a pretense for further surveillance. Alternatively, it could be a way of deflating future demands for increased surveillance. CSAM is one of the few emotional strings Governments have to insist upon surveillance. If we can mark CSAM as "solved" using basic fingerprinting then it's harder to push for real surveillance. We're currently seeing the widespread adoption of end-to-end encryption. If we don't have an answer to CSAM, then it will be used as a bludgeon to demand real back doors which can be exploited without manufacturer knowledge.
  - fmajid 5y ago
    Appeasing the authoritarians never works. On the contrary, it sets a precedent that is used to make more demands. The ability to scan for transformation-resistant image fingerprints will be used to identify whistleblowers who disclosed documents that embarrass the powers-that be (think Chelsea Manning, Reality Winner or the Panama Papers). It’s a matter of when, not if.
draw_down 5y ago
I think it’s alright to have an un-nuanced position about it. The whole problem of course is false positives, how they will be handled, what recourse innocent people will have to exonerate themselves, etc. None of that is going to be done in good faith or out in the open. Think Guantanamo, or PATRIOT act- this is the mode of operation in our political economy. How many lives did that save, how many attacks did it prevent? I’m not asking that rhetorically to imply the answer is zero, but to point out the whole problem is we can’t and never will know. It’s simply justified as “well we have to stand up to the baddies” and that is that. There never will be an accounting for the intrusion upon freedom versus the benefit to society, you’re meant to just shut up and accept it. Because, jeez, there’s no easy answers!
josh_today 5y ago
Even if Apple backtracks on this their trust has been broken
- ryantgtg 5y ago
  They lost my trust years ago when their software detected “me” in a photo. Why is their software analyzing and detecting things like that? It’s ludicrous.
  - macintux 5y ago
    > They lost my trust years ago when their software detected “me” in a photo. I don't understand. Their software, running on your phone, without interacting with any servers outside your phone, identified you for your convenience. What's harmful about that?
    - weikju 5y ago
      Yeah I'm confused about that statement too. From that point of view, a cron job running on a Linux system, daily, which updates the "locate" db is also a breach of trust and privacy..
      - ryantgtg 5y ago
        I was joking, of course. But what I was getting at is that I’m seeing an analog with this latest scan, in that the scanning software is now on the server where you are electing to store your photos. So, hard drive scan on your local is fine, yet hard drive scan on your remote is outrageous?
        weikju 5y ago
        This is the case where the scanning is being moved from iCloud to local devices -- still only for photos that are set to be uploaded to iCloud of course.. for now. I think overall what bothers people is more anxiety about what comes NEXT. Sure, it's innocuous enough that photos will be scanned locally instead of remotely, in either case they are going to be shared to iCloud therefore scanned. What about once they get pressure to expand this to all photos, uploaded to iCloud or not? Other kinds of materials, not just CSAM? Add other capabilities to scan other content, not just photos? etc etc etc...
gordon_freeman 5y ago
One thing I don't understand which Alex also points out in this thread is why Apple does not build a tool in iMessage to allow users to report/flag such inappropriate and abusive comms to Apple where Apple can then report to the authorities? Why Apple goes to this CSAM scanning route instead?
- ceejayoz 5y ago
  I'd presume that this sort of content is been sent around between people who are both into it. Not much reason to report in that scenario.
blintz 5y ago
There's a great live conversation between Alex, Riana Pfefferkorn, and Matthew Green here: https://www.youtube.com/watch?v=dbYZVNSOVy4 https://www.youtube.com/watch?v=dbYZVNSOVy4. Has more details on the reasoning and a little debate on the issues from some really good people. Even pretty non-hacker people intuit quickly that it would weird (and maybe dystopian) for your laptop to constantly scan all the files on your computer against a government database. Why this should be different for phones seems unclear to me, and none of the pro-scanning takes I’ve seen have made convincing arguments about it. Alex and folks in the broader abuse-prevention community don’t really make a principled argument about why phones should try to do this, but not laptops. I think it sounds a bit more obviously like something an authoritarian state would do, so the laptop analogy gets avoided. This isn’t really a theoretical concern; a natural next step for Apple would be to do this on the Mac too.
- dannyw 5y ago
  Apple said they will be doing this on macOS Big Sur as well alongside iOS 15.
rdli 5y ago
Pretty thoughtful take IMO. It is complicated and there isn’t a clear answer. I liked Alex’s suggestion though: when it’s a highly complex issue, the best solutions happen when you can bring together the right experts in their field together, and collaborate, and I think it’s a very fair critique of Apple. It’s hard for me to imagine a situation where showing up at the conference, sharing a preview of what they were thinking, and then hearing the feedback would not have been beneficial. (Speaking as a reasonably happy Apple user, I wish openness for some of these things was a little bit more of their DNA. When you’re as big as Apple, people need to just recognize that the standards are different. It may not be fair. But it’s just the way the world works.)
- bitexploder 5y ago
  I personally think there are pretty clear answers in terms of the technology. Apple is violating privacy. To what degree is debatable, but, I generally disagree with Alex here. He’s not making the stronger argument compared to privacy advocates here. Yes there is nuance. Yes we must operate with a high degree of empathy for victims of abuse. No one doubts or downplays that IMO. But comments right here in this thread hit the core of it. Would something like this ever be acceptable on a laptop? I get a choice because it’s my computer. I don’t want this. It’s violating my privacy and Apple has eliminated my ability to decide. They basically positioned this as the obvious and safe choice and anyone against them is a “screeching minority”, even though they knew this would be controversial. Alex is operating in good faith, but I think he’s in to deep on trying to bring nuance to an issue where his argument is simply the less principled one. It’s becoming a situation where if you care about privacy you simply can’t use a smart phone (I know there are options, but none of them are close to an iPhone).
  - reaperhulk 5y ago
    Without addressing any other points you’re making, NCMEC wrote the email to Apple that contained the “screeching minority” phrase. It was not something Apple wrote.
    - bitexploder 5y ago
lostgame 5y ago
I’ll be downvoted you hell for this, but I’m gonna put this out here. I’m a little concerned by the lack of concern towards the immense benefit this can have towards the terrifying problem of child pornography online. If Apple wants to use this for the CCP, they will; and there’s already so many companies who are bowing to that regime. False positives will be pretty obvious in court. I...I get the privacy concerns but honestly, I’m concerned with this getting almost universally negative reaction. There’s nothing you can really say about privacy here I haven’t already read - I’m not interested in that. I know I’m going to get shit on by 99% of this community for this - but someone has to offer the opposing opinion; and I guess I’m that bitch atm. I’m here to offer an opinion I’ve surprisedly never read in the maybe dozen comment threads on HN about this I’ve seen in the past few days. As a victim of child sexual assault, I highly encourage anything that prevents that from getting worse. I’m alright with this, I understand the community’s concerns; and tbh I don’t honestly think I need to read the - what I’m aware will be - scathing - responses to this comment. Happy to just see me at -50 karma tomorrow - heh. Yes, let’s help stop child rape and pedophiles. Yes; sometimes to be serious about that we will have to take some extreme measures. Thank God those extreme measures exist. May a hell of a lot more pedophiles go to jail from this. Looking hella forward to hearing how many arrests this was responsible for in a year’s time. <3
- macintux 5y ago
  I don’t believe they’ve effectively expanded the CSAM portion at all. Previously they were scanning photos after being shipped to iCloud; now they’re scanning them on the device as they’re prepared to be sent. So your wish that more criminals would be caught this way is unlikely to be fulfilled.
- wheybags 5y ago
  > I highly encourage anything that prevents that from getting worse. This is exactly the problem. There is no end to the effective new measures we can introduce. Until everyone is living in a prison cell, there will always be more we can do, that would be demonstrably effective at reducing child abuse. There has to be a line where we say the ends don't justify the means, and for a lot of people, this exceeds that line.
- 5y ago
disillusioned 5y ago
A virus, but it's just a bot that sends out CSAM-known-bad-images to get your device flagged as an offender...
- simondotau 5y ago
  Given that Google has been doing the same thing with their cloud photos service for many years, why hasn't this already happened?
g42gregory 5y ago
On one hand, yes, I totally agree with Alex that the "The scale of abuse that happens to kids online and the impact on those families is unfathomable." Furthermore, sex trafficking is another horrible, enormous and related problem that seemingly getting little attention. Per Alex tweet, I am not "verbally rolling my eyes at the invocation of child safety as a reason for these changes." Instead, I simply do not believe that this is the reason for which Apple is introducing these capabilities. There is something to be said about reputation of a business: it's hard to gain and easy to lose. Large, near-monopoly platforms, such as Apple, have lost their reputation in my eyes and now I am always questioning their motives. I simply do not believe that the Apple will refuse, when under serious anti-trust pressures, to extend these capabilities to other areas, including political views as an example. Paraphrasing Benjamin Franklin: "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety."
- yladiz 5y ago
  > Instead, I simply do not believe that this is the reason for which Apple is introducing these capabilities. Okay, what do you believe the reason is then? I can understand being skeptical of the claim that it's for child abuse and safety, but if you don't believe that then you must have something that you believe is the more likely reason. I also want to point out that, yes, the system could be abused for a multitude of reasons but that doesn't mean the actual reason for building the system is different.
  - btheshoe 5y ago
    There is real regulatory pressure on Apple, by a multitude of governments, to introduce backdoors into their systems for security reasons.
    - slowmovintarget 5y ago
      Not for "security" but rather for surveillance. The problem is the phones are too secure for governments to get the surveillance they want out of them.
andrew_ 5y ago
This is mass surveillance, there's just no way around it. The good intentions of this program will not be remembered when this is inevitably abused. https://twitter.com/Snowden/status/1423469854347169798 https://twitter.com/Snowden/status/1423469854347169798
version_five 5y ago
Why is this a nuanced conversation with no easy answers? There is literally no debate. Authoritarian elements want to look through your stuff because some people do bad things. I'm at a loss, to me nobody should even have to justify why the answer is no, it's a principle of natural justice. Its pretty screwed up when the onus is on people to explain why this is a bad thing. The correct action is really just to walk away.
- drooby 5y ago
  Huh? Natural justice? No such thing. Yes, you should explain why something is bad, or else we are just doing things based off of how we feel… does that sound better to you? The correct answer should be that the most well reasoned argument should be supported. And we should not expect anyone to intuit the best answer - repeating it as much as needed.
- augstein 5y ago
  I find it disturbing that they seemingly are allowed to do things on the devices of their US customers, that in most countries are reserved to law enforcement agencies exclusively. Also I think they should have to face serious repercussions in countries with strong data protection laws, should they decide to unilaterally change terms of service for their users in these jurisdictions, to reserve the right to search their devices for illegal images. For many people heavily invested in their ecosystem, it’s also not that easy to just walk away. For other users, the alternative (Google) is not necessarily more trustworthy.
  - moftz 5y ago
    From what I've read, the scanning only takes place on icloud images. You can choose not to backup to icloud and the images won't be scanned. Other cloud storage providers have done something similar in the past where if the hash of the content matches copyrighted movies or music, the content is deleted.
jacknews 5y ago
I spent the weekend (well, saturday) trying to get rid of an ant problem in the kitchen. They were coming up to the work-surfaces through cracks in the grouting. So I re-grouted the backsplash, and the joins in the worksurface etc. Of course, they started coming up through gaps in between the floor tiles, so I patched those. Then through the window sills. And I noticed they pushed out some of the new grout to make a couple of new runs. You will never completely stop this problem. And monitoring every. single. thing. is not the way to tackle it. Professionals need to target the source of the problem. Who is being abused? Why? Who are the abusers? Why? How and where are the two meeting? This is the work to traditional policing, improved social services and society in general, not mass-surveillance.
zx10 5y ago
I have some genuine questions regarding Apple's new CSAM detection. From what I have read 1) Most cloud providers including Apple already scans uploaded photos for child abuse. Photos are scanned server-side. 2) Apple wants to do this on-device instead of server-side so that, 1) photos are not looked up and scanned on the server. 2) the device can generate a cryptographic safety voucher to prevent leaking information about images that do not match the CSAM database. 3) threshold secret sharing can be used to prevent match results and encrypted data about the images from being accessed unless the account exceeds a threshold of matches. 3) Photos that are not uploaded to iCloud Photos are not affected. 4) Matching is done against *already known* database of CSAM images. 5) Data is manually reviewed before reporting to NCMEC. Isn't this better than what Apple and other companies(Google, Microsoft, Facebook etc.) have been doing on the server-side? Server-side and on-device CSAM detection both perform scan > match > manual review > report process. As far as I can see, the difference is that on-device detection adds more restrictions to Apple and third-parties. There are many claims focused on Apple's decision to move away from server-side to on-device CSAM detection that, 1) it is worse for privacy. (How is it worse than server-side detection? It is worse than no detection, but then there is no point of focusing on Apple's decision to move away from server-side detection.) 2) it is bad that other companies will follow this path. (Why is it bad if other companies replace server-side detection to on-device detection?) 3) it will let false positives to be triggered. (If it lets, were false positives previously(server-side) impossible to be triggered?) I'm all in for ceasing personal data scanning, both server-side and on-device, altogether. But based on what I have read, I think on-device detection is at least better than server-side detection. On the other hand, the amount of backlash laser-focused on Apple's decision to move away from server-side to on-device CSAM detection has led me to believe that I might be missing something. Am I?
literallyaduck 5y ago
Just stay home for two weeks to slow the spread, just don't wear masks they aren't effective, wear masks to stay safe, if you take the vaccine you don't need the mask, if you want to stay safe wear the mask and get the booster, your phone is secure, we only will use the backdoor to find people who have images of children, we will only use the back door to find criminals, your memes indicate you are a danger to society please report to a re-education camp, you are far too dangerous to be reintroduced here is your labor assignment, you no longer are of value to society your organs will go to those who contribute the most. That nuanced enough? Never allow a back door, because anyone can and will use it for nefarious purposes.
deleted 5y ago
[deleted]
ianmiers 5y ago
Its a nuanced conversation, but there two fundamental points that Apple totally missed.: 1) Don't build a system for reporting what images people make, store, or send privately. That is just begging for someone to force you to abuse it for things that aren't CSAM. You shouldn't expect Apple to be able to resist pressure to abuse such a system. 1)they already caved to China by basically handing iCloud + iMessage key infra to a local company. 2) The entire reason you have encrypted systems(and make no mistake, this entire scheme only makes sense to scan encrypted content) is precisely for the day when the social and repetitional guarantees of "trust us" fail. 2) We need alternative ways to stop grooming and effectively get the major players who produce CSAM. And that's not going to come from scanning for CSAM NCMEC already knows about. Yes, resharing images harms the victim, but if you have a finite amount of resources, you go after the original image production and the people literally raping kids. And indeed, thats apparently what the FBI does and why they mostly ignore reports flagged by these systems.
xylophoner 5y ago
How long until this surveillance attitude leads them to add permanent monitoring into all Apple products, like Notes, iMovie, Messages, etc? That’s one thing this move points to.
tgsovlerkhgsel 5y ago
I think the most offensive element of Apple's approach is that your device, that you paid for and own [1], is acting in a way that can only be against your own interest. There is really no good argument to allow that. Most of the other user-hostile features are at least somewhat explained away as also benefiting the user (e.g. claims that targeted ads are preferable to irrelevant ones, telemetry helps cover the users' use cases, buying the subscription service is actually a win-win, etc.) Nobody would choose a feature like this voluntarily. This is simply preinstalled malware. I really wonder whether Apple realized how badly they just burned down their entire privacy-centric advertising campaign, where the core selling point is that unlike others, they act in your interest. [1] Spare me the Stallmanism, I'm talking about the device itself.
btheshoe 5y ago
I don't see how there's any middle ground here. Anything apple does to try to prevent sexual abuse will by definition infringe on the privacy of users. And I don't think there's anything apple can do to prevent cp that doesn't create a slippery slope to preventing other things that governments (potentially authoritarian) deem unsavory, Frankly, this entire thread seems like a bad attempt to take a centrist position without providing any real justification for the position to have value.
nbzso 5y ago
"The greatest tyrannies are always perpetuated in the name of the noblest causes" Thomas Paine
fmajid 5y ago
Hmmm. Would that be the same Alex Stamos who was Chief Information Security Officer at Yahoo when they had one of the worst hacks ever, then at Facebook when they were used to subvert American democracy? He’s good at failing upwards, I’ll give you that. I don’t know why I waited so long to officially flip the bozo bit on him by adding him to my feed reader’s filter list.
sigsergv 5y ago
It's time to introduce new management title — Child Protection Officer.
cannabis_sam 5y ago
I realize I’m godwin-ing myself with this, but I’m always wondering how these fanatical people view the discrepancy in holocaust victims from countries with active registration of people’s religion, versus countries without religious registration.
cannabis_sam 5y ago
Actually, Alex Stamos, despite his apparent crusade against sexual abuse of children, has not installed video surveillance in his home. It’s unfathomable that he would not know that the majority of sexual abuse against children happens IN THE HOME!?! Why would he focus on this marginal problem of distribution, instead of stopping the actual assaults on children? It’s actually trivial to install video cameras in your children’s bedroom! If you won’t, what cruel child abuse are you trying to hide? Alex Stamos is clearly not telling the whole truth here? How do we know that he doesn’t abuse children without surveillance in his home? We obviously don’t, so naturally he should be reported to the police for refusing to cooperate in the investigation of child abuse. Why is he refusing? What is Alex Stamos hiding about his personal experience with child sexual abuse?

76 points by sbuccini 5y ago

https://twitr.gq/alexstamos/status/1424054542879006728 https://twitr.gq/alexstamos/status/1424054542879006728 Links to https://storage.courtlistener.com/recap/gov.uscourts.insd.77308/gov.uscourts.insd.77308.172.0.pdf https://storage.courtlistener.com/recap/gov.uscourts.insd.77... (90 pages of harrowing public-domain court documents obtained through RECAP, the system Aaron Swartz first got investigated by the FBI for setting up; they document the misdeeds of a guy named Buster Hernandez who serially raped children over Facebook)

wyager 5y ago

> For the last couple of years, our team at @stanfordio has been hosting a series of conferences on how to balance the safety and privacy aspects of E2EE products. I am not interested in “balancing” anything. This is not a negotiation and it is not open to compromise. The opposite of privacy is not “safety”; it is surveillance. I refuse to use products that actively surveil me behind my back. Consumers deserve products that work for consumer interests, not products that ultimately serve someone else.

deleted 5y ago
[deleted]
caeril 5y ago
Indeed. However, there is no point getting mad at Alex. He used to be a real hacker, until he sold his soul for mammon at Facebook. Once someone like this makes the transition from hacker to wiping one's ass with the hacker manifesto, there's no turning back, and you just need to accept that their words are hollow. Nothing against Alex. He's brilliant, and you never know if you're offered $XMM to compromise your principles, you won't make the same decision until you're actually in that position. I'm not smart enough to ever be in his position, and I'd like to believe that if I were, I would hold true to principle, but who knows? We're all flawed.
bluepizza 5y ago
That is a naive and immature point of view. Technology enables crime in terrible ways, and those crimes need to be curtailed. Don't like it? Buy a different phone.

dvt 5y ago

The thread has a point, and there's definitely nuance here, but I think people throughout history have litigated this many times over, and we came to the same conclusion over and over and over again. To argue the contrary is revisionist or simply ignorant. This conclusion is one that has pushed our species to be more charitable, more law-abiding, more free, less tyrannical, more fair, etc. Not that we've reached some kind of utopia, but let's face it: the world is much better off than it was 100, 200, or 1000 years ago. The conclusion is that freedom (that is: negative, not positive rights) is the main force of progress (societal, technological, scientific). Of course, we have law enforcement, and judges, and lawyers, and an entire legal machinery: but it functions within well-defined boundaries. In the US, we have warrants. We have habeas corpus. We have chain of custody, and so on. Apple is fundamentally circumventing these norms. And this is wrong, no matter what is on the other side of the equation. I'm not sure what Stamos' counter-argument is here. I don't always agree with the EFF, but I think in this case, they're right on the money.

bluepizza 5y ago
Is it? How are these norms being circumvented?
- dvt 5y ago
  By reporting potential criminals to the authorities by scanning what is essentially private data (images, messages), I'd likely argue they're violating unlawful search and seizure and I doubt a conviction would stick[1]. It's not like my landlord can barge in my apartment and look under the couch for illegal drugs because he heard a rumor I bought some. I mean, probable cause is already sketchy enough, what Apple doing is beyond the pale. [1] https://reason.com/volokh/2021/08/05/apple-plans-to-scan-us-iphones-for-child-abuse-imagery/ https://reason.com/volokh/2021/08/05/apple-plans-to-scan-us-...
  - ceejayoz 5y ago
    > By reporting potential criminals to the authorities by scanning what is essentially private data (images, messages), I'd likely argue they're violating unlawful search and seizure and I doubt a conviction would stick[1]. Why? Police use informants to obtain probable cause regularly.

joe_the_user 5y ago

Interestingly, "there are no easy answers" seems to be an ... easy answer to a lot of pretty obvious and compelling objections. This kind of thing is often said to "balance" between "your plan is despicable and ridiculous" and "yeah but the pressure to do something is huge"

dmm 5y ago

> I also don't understand why Apple is pushing the CSAM scanning for iCloud into the device Because it's a pretense for further surveillance.

caeril 5y ago
He understands, he's just playing the game.
alisonkisk 5y ago
Because they don't want to decrypt your non-child-porn files!
- wyager 5y ago
  iCloud images are not encrypted. This is a concession Apple had already made to the FBI.
Niksko 5y ago
One theory put out by John Gruber was that the scanning was being done in device so that they could E2E encrypt iCloud in the future. So they satisfy the 'think of the children people', while giving you strong encryption elsewhere. Time will tell.
simondotau 5y ago
> Because it's a pretense for further surveillance. Alternatively, it could be a way of deflating future demands for increased surveillance. CSAM is one of the few emotional strings Governments have to insist upon surveillance. If we can mark CSAM as "solved" using basic fingerprinting then it's harder to push for real surveillance. We're currently seeing the widespread adoption of end-to-end encryption. If we don't have an answer to CSAM, then it will be used as a bludgeon to demand real back doors which can be exploited without manufacturer knowledge.
- fmajid 5y ago
  Appeasing the authoritarians never works. On the contrary, it sets a precedent that is used to make more demands. The ability to scan for transformation-resistant image fingerprints will be used to identify whistleblowers who disclosed documents that embarrass the powers-that be (think Chelsea Manning, Reality Winner or the Panama Papers). It’s a matter of when, not if.

draw_down 5y ago

I think it’s alright to have an un-nuanced position about it. The whole problem of course is false positives, how they will be handled, what recourse innocent people will have to exonerate themselves, etc. None of that is going to be done in good faith or out in the open. Think Guantanamo, or PATRIOT act- this is the mode of operation in our political economy. How many lives did that save, how many attacks did it prevent? I’m not asking that rhetorically to imply the answer is zero, but to point out the whole problem is we can’t and never will know. It’s simply justified as “well we have to stand up to the baddies” and that is that. There never will be an accounting for the intrusion upon freedom versus the benefit to society, you’re meant to just shut up and accept it. Because, jeez, there’s no easy answers!

josh_today 5y ago

Even if Apple backtracks on this their trust has been broken

ryantgtg 5y ago
They lost my trust years ago when their software detected “me” in a photo. Why is their software analyzing and detecting things like that? It’s ludicrous.
- macintux 5y ago
  > They lost my trust years ago when their software detected “me” in a photo. I don't understand. Their software, running on your phone, without interacting with any servers outside your phone, identified you for your convenience. What's harmful about that?
  - weikju 5y ago
    Yeah I'm confused about that statement too. From that point of view, a cron job running on a Linux system, daily, which updates the "locate" db is also a breach of trust and privacy..
    - ryantgtg 5y ago
      I was joking, of course. But what I was getting at is that I’m seeing an analog with this latest scan, in that the scanning software is now on the server where you are electing to store your photos. So, hard drive scan on your local is fine, yet hard drive scan on your remote is outrageous?
      - weikju 5y ago
        This is the case where the scanning is being moved from iCloud to local devices -- still only for photos that are set to be uploaded to iCloud of course.. for now. I think overall what bothers people is more anxiety about what comes NEXT. Sure, it's innocuous enough that photos will be scanned locally instead of remotely, in either case they are going to be shared to iCloud therefore scanned. What about once they get pressure to expand this to all photos, uploaded to iCloud or not? Other kinds of materials, not just CSAM? Add other capabilities to scan other content, not just photos? etc etc etc...

gordon_freeman 5y ago

One thing I don't understand which Alex also points out in this thread is why Apple does not build a tool in iMessage to allow users to report/flag such inappropriate and abusive comms to Apple where Apple can then report to the authorities? Why Apple goes to this CSAM scanning route instead?

ceejayoz 5y ago
I'd presume that this sort of content is been sent around between people who are both into it. Not much reason to report in that scenario.

blintz 5y ago

There's a great live conversation between Alex, Riana Pfefferkorn, and Matthew Green here: https://www.youtube.com/watch?v=dbYZVNSOVy4 https://www.youtube.com/watch?v=dbYZVNSOVy4. Has more details on the reasoning and a little debate on the issues from some really good people. Even pretty non-hacker people intuit quickly that it would weird (and maybe dystopian) for your laptop to constantly scan all the files on your computer against a government database. Why this should be different for phones seems unclear to me, and none of the pro-scanning takes I’ve seen have made convincing arguments about it. Alex and folks in the broader abuse-prevention community don’t really make a principled argument about why phones should try to do this, but not laptops. I think it sounds a bit more obviously like something an authoritarian state would do, so the laptop analogy gets avoided. This isn’t really a theoretical concern; a natural next step for Apple would be to do this on the Mac too.

dannyw 5y ago
Apple said they will be doing this on macOS Big Sur as well alongside iOS 15.

rdli 5y ago

Pretty thoughtful take IMO. It is complicated and there isn’t a clear answer. I liked Alex’s suggestion though: when it’s a highly complex issue, the best solutions happen when you can bring together the right experts in their field together, and collaborate, and I think it’s a very fair critique of Apple. It’s hard for me to imagine a situation where showing up at the conference, sharing a preview of what they were thinking, and then hearing the feedback would not have been beneficial. (Speaking as a reasonably happy Apple user, I wish openness for some of these things was a little bit more of their DNA. When you’re as big as Apple, people need to just recognize that the standards are different. It may not be fair. But it’s just the way the world works.)

bitexploder 5y ago
I personally think there are pretty clear answers in terms of the technology. Apple is violating privacy. To what degree is debatable, but, I generally disagree with Alex here. He’s not making the stronger argument compared to privacy advocates here. Yes there is nuance. Yes we must operate with a high degree of empathy for victims of abuse. No one doubts or downplays that IMO. But comments right here in this thread hit the core of it. Would something like this ever be acceptable on a laptop? I get a choice because it’s my computer. I don’t want this. It’s violating my privacy and Apple has eliminated my ability to decide. They basically positioned this as the obvious and safe choice and anyone against them is a “screeching minority”, even though they knew this would be controversial. Alex is operating in good faith, but I think he’s in to deep on trying to bring nuance to an issue where his argument is simply the less principled one. It’s becoming a situation where if you care about privacy you simply can’t use a smart phone (I know there are options, but none of them are close to an iPhone).
- reaperhulk 5y ago
  Without addressing any other points you’re making, NCMEC wrote the email to Apple that contained the “screeching minority” phrase. It was not something Apple wrote.
  - bitexploder 5y ago

lostgame 5y ago

I’ll be downvoted you hell for this, but I’m gonna put this out here. I’m a little concerned by the lack of concern towards the immense benefit this can have towards the terrifying problem of child pornography online. If Apple wants to use this for the CCP, they will; and there’s already so many companies who are bowing to that regime. False positives will be pretty obvious in court. I...I get the privacy concerns but honestly, I’m concerned with this getting almost universally negative reaction. There’s nothing you can really say about privacy here I haven’t already read - I’m not interested in that. I know I’m going to get shit on by 99% of this community for this - but someone has to offer the opposing opinion; and I guess I’m that bitch atm. I’m here to offer an opinion I’ve surprisedly never read in the maybe dozen comment threads on HN about this I’ve seen in the past few days. As a victim of child sexual assault, I highly encourage anything that prevents that from getting worse. I’m alright with this, I understand the community’s concerns; and tbh I don’t honestly think I need to read the - what I’m aware will be - scathing - responses to this comment. Happy to just see me at -50 karma tomorrow - heh. Yes, let’s help stop child rape and pedophiles. Yes; sometimes to be serious about that we will have to take some extreme measures. Thank God those extreme measures exist. May a hell of a lot more pedophiles go to jail from this. Looking hella forward to hearing how many arrests this was responsible for in a year’s time. <3

macintux 5y ago
I don’t believe they’ve effectively expanded the CSAM portion at all. Previously they were scanning photos after being shipped to iCloud; now they’re scanning them on the device as they’re prepared to be sent. So your wish that more criminals would be caught this way is unlikely to be fulfilled.
wheybags 5y ago
> I highly encourage anything that prevents that from getting worse. This is exactly the problem. There is no end to the effective new measures we can introduce. Until everyone is living in a prison cell, there will always be more we can do, that would be demonstrably effective at reducing child abuse. There has to be a line where we say the ends don't justify the means, and for a lot of people, this exceeds that line.
5y ago

disillusioned 5y ago

A virus, but it's just a bot that sends out CSAM-known-bad-images to get your device flagged as an offender...

simondotau 5y ago
Given that Google has been doing the same thing with their cloud photos service for many years, why hasn't this already happened?

g42gregory 5y ago

On one hand, yes, I totally agree with Alex that the "The scale of abuse that happens to kids online and the impact on those families is unfathomable." Furthermore, sex trafficking is another horrible, enormous and related problem that seemingly getting little attention. Per Alex tweet, I am not "verbally rolling my eyes at the invocation of child safety as a reason for these changes." Instead, I simply do not believe that this is the reason for which Apple is introducing these capabilities. There is something to be said about reputation of a business: it's hard to gain and easy to lose. Large, near-monopoly platforms, such as Apple, have lost their reputation in my eyes and now I am always questioning their motives. I simply do not believe that the Apple will refuse, when under serious anti-trust pressures, to extend these capabilities to other areas, including political views as an example. Paraphrasing Benjamin Franklin: "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety."

yladiz 5y ago
> Instead, I simply do not believe that this is the reason for which Apple is introducing these capabilities. Okay, what do you believe the reason is then? I can understand being skeptical of the claim that it's for child abuse and safety, but if you don't believe that then you must have something that you believe is the more likely reason. I also want to point out that, yes, the system could be abused for a multitude of reasons but that doesn't mean the actual reason for building the system is different.
- btheshoe 5y ago
  There is real regulatory pressure on Apple, by a multitude of governments, to introduce backdoors into their systems for security reasons.
  - slowmovintarget 5y ago
    Not for "security" but rather for surveillance. The problem is the phones are too secure for governments to get the surveillance they want out of them.

andrew_ 5y ago

This is mass surveillance, there's just no way around it. The good intentions of this program will not be remembered when this is inevitably abused. https://twitter.com/Snowden/status/1423469854347169798 https://twitter.com/Snowden/status/1423469854347169798

version_five 5y ago

Why is this a nuanced conversation with no easy answers? There is literally no debate. Authoritarian elements want to look through your stuff because some people do bad things. I'm at a loss, to me nobody should even have to justify why the answer is no, it's a principle of natural justice. Its pretty screwed up when the onus is on people to explain why this is a bad thing. The correct action is really just to walk away.

drooby 5y ago
Huh? Natural justice? No such thing. Yes, you should explain why something is bad, or else we are just doing things based off of how we feel… does that sound better to you? The correct answer should be that the most well reasoned argument should be supported. And we should not expect anyone to intuit the best answer - repeating it as much as needed.
augstein 5y ago
I find it disturbing that they seemingly are allowed to do things on the devices of their US customers, that in most countries are reserved to law enforcement agencies exclusively. Also I think they should have to face serious repercussions in countries with strong data protection laws, should they decide to unilaterally change terms of service for their users in these jurisdictions, to reserve the right to search their devices for illegal images. For many people heavily invested in their ecosystem, it’s also not that easy to just walk away. For other users, the alternative (Google) is not necessarily more trustworthy.
- moftz 5y ago
  From what I've read, the scanning only takes place on icloud images. You can choose not to backup to icloud and the images won't be scanned. Other cloud storage providers have done something similar in the past where if the hash of the content matches copyrighted movies or music, the content is deleted.

jacknews 5y ago

I spent the weekend (well, saturday) trying to get rid of an ant problem in the kitchen. They were coming up to the work-surfaces through cracks in the grouting. So I re-grouted the backsplash, and the joins in the worksurface etc. Of course, they started coming up through gaps in between the floor tiles, so I patched those. Then through the window sills. And I noticed they pushed out some of the new grout to make a couple of new runs. You will never completely stop this problem. And monitoring every. single. thing. is not the way to tackle it. Professionals need to target the source of the problem. Who is being abused? Why? Who are the abusers? Why? How and where are the two meeting? This is the work to traditional policing, improved social services and society in general, not mass-surveillance.

zx10 5y ago

I have some genuine questions regarding Apple's new CSAM detection. From what I have read 1) Most cloud providers including Apple already scans uploaded photos for child abuse. Photos are scanned server-side. 2) Apple wants to do this on-device instead of server-side so that, 1) photos are not looked up and scanned on the server. 2) the device can generate a cryptographic safety voucher to prevent leaking information about images that do not match the CSAM database. 3) threshold secret sharing can be used to prevent match results and encrypted data about the images from being accessed unless the account exceeds a threshold of matches. 3) Photos that are not uploaded to iCloud Photos are not affected. 4) Matching is done against *already known* database of CSAM images. 5) Data is manually reviewed before reporting to NCMEC. Isn't this better than what Apple and other companies(Google, Microsoft, Facebook etc.) have been doing on the server-side? Server-side and on-device CSAM detection both perform scan > match > manual review > report process. As far as I can see, the difference is that on-device detection adds more restrictions to Apple and third-parties. There are many claims focused on Apple's decision to move away from server-side to on-device CSAM detection that, 1) it is worse for privacy. (How is it worse than server-side detection? It is worse than no detection, but then there is no point of focusing on Apple's decision to move away from server-side detection.) 2) it is bad that other companies will follow this path. (Why is it bad if other companies replace server-side detection to on-device detection?) 3) it will let false positives to be triggered. (If it lets, were false positives previously(server-side) impossible to be triggered?) I'm all in for ceasing personal data scanning, both server-side and on-device, altogether. But based on what I have read, I think on-device detection is at least better than server-side detection. On the other hand, the amount of backlash laser-focused on Apple's decision to move away from server-side to on-device CSAM detection has led me to believe that I might be missing something. Am I?

literallyaduck 5y ago

Just stay home for two weeks to slow the spread, just don't wear masks they aren't effective, wear masks to stay safe, if you take the vaccine you don't need the mask, if you want to stay safe wear the mask and get the booster, your phone is secure, we only will use the backdoor to find people who have images of children, we will only use the back door to find criminals, your memes indicate you are a danger to society please report to a re-education camp, you are far too dangerous to be reintroduced here is your labor assignment, you no longer are of value to society your organs will go to those who contribute the most. That nuanced enough? Never allow a back door, because anyone can and will use it for nefarious purposes.

deleted 5y ago

[deleted]

ianmiers 5y ago

Its a nuanced conversation, but there two fundamental points that Apple totally missed.: 1) Don't build a system for reporting what images people make, store, or send privately. That is just begging for someone to force you to abuse it for things that aren't CSAM. You shouldn't expect Apple to be able to resist pressure to abuse such a system. 1)they already caved to China by basically handing iCloud + iMessage key infra to a local company. 2) The entire reason you have encrypted systems(and make no mistake, this entire scheme only makes sense to scan encrypted content) is precisely for the day when the social and repetitional guarantees of "trust us" fail. 2) We need alternative ways to stop grooming and effectively get the major players who produce CSAM. And that's not going to come from scanning for CSAM NCMEC already knows about. Yes, resharing images harms the victim, but if you have a finite amount of resources, you go after the original image production and the people literally raping kids. And indeed, thats apparently what the FBI does and why they mostly ignore reports flagged by these systems.

xylophoner 5y ago

How long until this surveillance attitude leads them to add permanent monitoring into all Apple products, like Notes, iMovie, Messages, etc? That’s one thing this move points to.

tgsovlerkhgsel 5y ago

I think the most offensive element of Apple's approach is that your device, that you paid for and own [1], is acting in a way that can only be against your own interest. There is really no good argument to allow that. Most of the other user-hostile features are at least somewhat explained away as also benefiting the user (e.g. claims that targeted ads are preferable to irrelevant ones, telemetry helps cover the users' use cases, buying the subscription service is actually a win-win, etc.) Nobody would choose a feature like this voluntarily. This is simply preinstalled malware. I really wonder whether Apple realized how badly they just burned down their entire privacy-centric advertising campaign, where the core selling point is that unlike others, they act in your interest. [1] Spare me the Stallmanism, I'm talking about the device itself.

btheshoe 5y ago

I don't see how there's any middle ground here. Anything apple does to try to prevent sexual abuse will by definition infringe on the privacy of users. And I don't think there's anything apple can do to prevent cp that doesn't create a slippery slope to preventing other things that governments (potentially authoritarian) deem unsavory, Frankly, this entire thread seems like a bad attempt to take a centrist position without providing any real justification for the position to have value.

nbzso 5y ago

"The greatest tyrannies are always perpetuated in the name of the noblest causes" Thomas Paine

fmajid 5y ago

Hmmm. Would that be the same Alex Stamos who was Chief Information Security Officer at Yahoo when they had one of the worst hacks ever, then at Facebook when they were used to subvert American democracy? He’s good at failing upwards, I’ll give you that. I don’t know why I waited so long to officially flip the bozo bit on him by adding him to my feed reader’s filter list.

sigsergv 5y ago

It's time to introduce new management title — Child Protection Officer.

cannabis_sam 5y ago

I realize I’m godwin-ing myself with this, but I’m always wondering how these fanatical people view the discrepancy in holocaust victims from countries with active registration of people’s religion, versus countries without religious registration.

cannabis_sam 5y ago

Actually, Alex Stamos, despite his apparent crusade against sexual abuse of children, has not installed video surveillance in his home. It’s unfathomable that he would not know that the majority of sexual abuse against children happens IN THE HOME!?! Why would he focus on this marginal problem of distribution, instead of stopping the actual assaults on children? It’s actually trivial to install video cameras in your children’s bedroom! If you won’t, what cruel child abuse are you trying to hide? Alex Stamos is clearly not telling the whole truth here? How do we know that he doesn’t abuse children without surveillance in his home? We obviously don’t, so naturally he should be reported to the police for refusing to cooperate in the investigation of child abuse. Why is he refusing? What is Alex Stamos hiding about his personal experience with child sexual abuse?