15 ms·

After criticism, Apple to only seek abuse images flagged in multiple nations

263 points by ldayley 5y ago

metiscus 5y ago
This article misses the point completely. It was never about what they were looking for, it was that they were looking at all. The quiet part is, once they get acceptance of the snooping using the bogeyman of the day, it will eventually encompass other behavior up to and including political dissent etc.
- okamiueru 5y ago
  There is also the issue that closed door arbitration of heinous crimes are the perfect tool to put away dissidents. A very convenient excuse to avoid proper checks and balances. "No, you cannot verify, because you aren't allowed to look at the illegal material that proves their guilt"
  - hahajk 5y ago
    Is this actually true? If I was arrested for an image on my phone, would no one in the judge/jury team be allowed to see the image that was flagged? I understand it’s allegedly contraband, but if I were a judge I don’t think I’d take an algorithm’s word for it.
    - heavyset_go 5y ago
      > it’s allegedly contraband, but if I were a judge I don’t think I’d take an algorithm’s word for it. The prosecution can hire more expert witnesses with excessive credentials than you can, and they will explain how there's a one in one trillion possibility that the system is wrong, and that the defendant is assuredly a monster. Juries eat that up when it comes to bogus DNA, bite mark, fingerprint, or other forensic evidence claims. Most people think computers can't be wrong or biased, and people's perceptions of what can be deemed reasonable doubts or not seem to shift when computers are involved, or when smart, credentialed people tell them their reasonable doubts aren't reasonable at all because of that one in a trillion chance of the computers being wrong.
- jsjohnst 5y ago
  I agree. Further, is there any doubt that these “CSAM”² hash databases aren’t already shared between countries? Because if there was doubt, there shouldn’t be, because agencies do share across borders. ² - reason for the scare quotes is because I have first hand knowledge of non-CSAM content being in NCMEC’s database (most likely via data entry errors, but I can’t be for sure).
  - belorn 5y ago
    Images of bonsai trees?
    - jsjohnst 5y ago
      > Images of bonsai trees? Ha! Come to think of it, I think that was one example. The main examples that came to memory (it’s been almost 8 years since I was involved) when discussing this last week were essentially extremely common photos (like the stock Windows XP background, among others).
- deugo 5y ago
  I think this is (quietly) already happening: https://www.reuters.com/technology/exclusive-facebook-tech-giants-target-manifestos-militias-database-2021-07-26/ https://www.reuters.com/technology/exclusive-facebook-tech-g... Instead of hashes looking for sexual abuse images, hashes look at signs of white supremacist behavior. Instead of reporting such racism or behavior to the government, it is shared with all the major tech companies, including AirBnB and Paypal. So, if Facebook deems you to be a political outcast, based on your conversations on their platform, you may find yourself without access to hotels or online payment platforms. The bogeyman of the day is the white supremacist, and coordinated snooping by big tech is the quiet part. > Over the next few months, the group will add attacker manifestos - often shared by sympathizers after white supremacist violence - and other publications and links flagged by U.N. initiative Tech Against Terrorism. It will use lists from intelligence-sharing group Five Eyes, adding URLs and PDFs from more groups, including the Proud Boys, the Three Percenters and neo-Nazis. > The firms, which include Twitter (TWTR.N) and Alphabet Inc's (GOOGL.O) YouTube, share "hashes," unique numerical representations of original pieces of content that have been removed from their services. Other platforms use these to identify the same content on their own sites in order to review or remove it. As a company to become part of the Global Internet Forum to Counter Terrorism, you have to pledge your support to expanding from terrorism towards all online extremism. If you can't accept that, they will still send a speaker to educate your company.
- spoonjim 5y ago
  And, just introducing this idea can make the concept real, even if Apple cancels this plan tomorrow. Country X could introduce a law saying “all companies that sell phones here must scan and report for these hashes” and based on recent trends, the companies will just roll over and say Uncle.
codezero 5y ago
I expect any powerful nation could “share” with a desperate ally or otherwise leverage them into adopting the same database, creating a false overlap.
- bellyfullofbac 5y ago
  Yeah. "Dear debtor countries, the attached file contain hashes of new files deemed illegal for the harmonious well-being of peoples for this month, please enter them into your databases."
- deleted 5y ago
  [deleted]
adrr 5y ago
How does that stop a FISA subpoena with a gag order? The system exists that any secret government order can exploit.
- ls612 5y ago
  A FISA subpoena or an NSL can only compel Apple to give certain types of information that Apple already has. They can’t compel Apple to gather information they don’t already have.
  - kook_throwaway 5y ago
    You have far more faith in a system of secret courts and gag orders than I do.
    - busymom0 5y ago
      Yep. FBI Lawyer, Kevin Clinesmith gets to lie to the FISA court, fabricate emails to get 2-hop warrants and then gets zero days in prison and as of this month is allowed to practice law again. It's an absolute joke.
  - captainredbeard 5y ago
    Hahaha an NSL can do more than that.
  - adrr 5y ago
    That can’t be true because the old pen/trap orders required installation of a device to record the calls.
    - ls612 5y ago
      A wiretap order is a different kind of order and goes through the district court, not FISA. The legal standard for it is also much higher than for other subpoenas by statute.
      - 5y ago
istingray 5y ago
What happened to it being USA only? This sounds like one of those errors where they double down on it and call it a "solution". Sounds like they're misunderstanding people's concerns pretty badly. Sorry if we miscommunicated. Kill this "feature" yesterday. Thanks Tim!
- dannyw 5y ago
  This is just business. Governments around the world are thinking of breaking up the app store, and Apple is saying they'll help governments surveil on their citizens inside their walled garden. Would be a shame if users can escape from the surveillance by choosing alternates outside of the walled garden, wouldn't it? Expect an App Store rule soon - all photos apps must scan for CSAM. In a year, expect Signal to be banned from all app stores, just like how it's already banned from some countries.
- treesprite82 5y ago
  USA-only referred to which accounts would be scanned. Whereas it's theoretically a good thing to require the CSAM hashes to be verified by organisations in multiple countries (rather than a single US government-tied organisation that dismissed privacy concerns as "the screeching voices of the minority"). I'm hoping for something along the lines of "iOS will reject hashes that haven't been signed by independent organisations in US, Russia, China, and India" to make it very difficult to push through anything except actual CSAM. Won't be much of a guarantee if it's just Apple saying "we promise we're only using hashes that have been checked by Australia and the US".
- TillE 5y ago
  My impression was that it's currently US only because that's the only country so far where they've arranged a process to report their users to the cops. No reason they wouldn't expand it as they make arrangements with other countries. They've really bought into a ton of complexity at all levels by doing this instead of just scanning stuff on their own servers, which would at least have been a clear dividing line. Privacy is really all about control, and on-device scanning dangerously blurs that line.
Invictus0 5y ago
It should be obvious by now that Apple has been coerced into building this scanning system, and the CSAM scanning is just the front for a much more broad "national security" backdoor. Apple's PR team has surely been shitting bricks for the entire week--this is the worst PR debacle they've had since the iPhone 4 Antennagate in 2010.
- istingray 5y ago
  Completely agree. A friend of mine asked what was happening, and I just said "it's a big deal, biggest thing in a long time". Antennagate is a good comparison -- though this seems bigger. Impacts of Antennagate long term seemed minimal.
  - klipklop 5y ago
    Sadly this will likely be little to no impact on Apple long term either. I too agree this is coming from the government. They have found their loophole of using "private industry" to do all the dirty stuff they cannot directly do.
    - tgsovlerkhgsel 5y ago
      Given that privacy has been what Apple was using as they key selling point in their ad campaign, and now they've completely subverted that and mainstream media is picking it up and people are noticing, I suspect this will impact them far more than they expected. While Apple deserves all the fallout they're getting and more, I'm disappointed that the NCMEC that pushed for this isn't also receiving more scrutiny and criticism. Multiple people have now pointed out that their database contains false positives, which is absolutely terrifying. Completely legal, harmless, no-nudity-no-humans pictures can get your life ruined. The truth coming out later doesn't matter when your home gets raided and it slips out that you were caught sharing multiple images that matched hashes from the NCMEC child porn database.
  - musha68k 5y ago
    This is way bigger than any RF engineering mishap. I’m going to actively evangelize alternatives to Apple devices and that’s coming from someone who has been doing Apple evangelizing since OSX Panther days. Tens of thousands of dollars of bought devices and services not including all the people I convinced to make the switch over the years. I’m sure I’m not the only one in this regard.
whytaka 5y ago
Conveniently, the other participating nations are all members of Five Eyes. Probably.
2OEH8eoCRo0 5y ago
Wrong answer, Apple.
akomtu 5y ago
Translating from corporatese: "Apple hasn't given up the idea and is looking for a better excuse to push their surveillance solution."
schappim 5y ago
With this much bad PR (and Apple hates bad PR), why is Apple pretty much holding its position?
- chrischen 5y ago
  Potentially something like a national security letter and they can’t reveal the real reason behind it. I agree this feature is off-brand for Apple and came out of the blue.
  - vmception 5y ago
    Maybe this is the warrant canary Lets push that idea so allies in the government can go looking and void it
  - underseacables 5y ago
    What would happen if Apple reveal the national security letter? What’s the government going to do, arrest Tim Cook?
  - silasdavis 5y ago
    Is that consistent with the amount of time this feature took to implement?
- Overton-Window 5y ago
  Their handlers have them on a tight leash.
  - asteroidbelt 5y ago
    This sounds like a conspiracy theory unless you provide more explanations with some evidence. For example, who are these handlers? What are their motives? The simplest explanation is usually the correct one. The simplest explanation would be some director in Apple wants to get promoted to VP so he or she started this project, pitched it, and very few people opposed it because who wouldn't to prevent harm of children?
- 5y ago
grishka 5y ago
It's a policy decision, while the technical one is still there, unchanged.
underseacables 5y ago
Still not good enough. That’s like an invading army saying they have a “kinder, gentler machine gun hand.”
dkersten 5y ago
So they're still retaining the ability to look on users devices? I have nothing against them looking at the abuse material, I have a problem with them having power over my device, which could be piggy-backed on by state actors (or others) for their own purposes later. Nobody is against them trying to prevent child sexual abuse, pretty sure we all agree that fighting that is important, but doing so by creating what is essentially a back door of sorts into my devices isn't the right approach to doing so. It sounds to me that this is still allowing them that access, so this changes absolutely nothing.
- dwighttk 5y ago
  Don’t use iCloud photos
  - thoughtstheseus 5y ago
    Don’t use Apple products. The scanning is on device.
    - uninformedhn 5y ago
      ...if you use iCloud Photos. How is it day 47 of HN arguing about this and you all still can’t get basic facts right, and oh boy, you showed me downvoting me twice in 60 seconds for pointing out a lack of fundamental comprehension of what you’re actually upset about This community is obnoxious
  - wilkystyle 5y ago
    This is as unhelpful as saying "don't have CSAM on your phone." As has been reiterated many times, the problem is not the capability it is claimed to start off with, it is how the capability will be evolved, abused, and exploited as time goes on.
    - dwighttk 5y ago
      I mean, don’t do that either, but the venn diagram circles of those two statements are not identical.
amq 5y ago
A backdoor is a backdoor.
cletus 5y ago
And... Apple misses the point of the criticism completely. This problem is the capability, not what it's used for. Any such capability will be abused by new use cases be it terrorism, drug trafficking, human trafficking or whatever. Plus there will inevitably be unauthorized access. The only way to prevent all this is for the system not to exist. I don't buy into theories that Apple is being pressured or coerced on any of this. I believe it's far more likely this is just tone-deaf but well-intentioned incompetence. It's classic "why won't anyone think of the children?" and we've seen it time and time again with encryption backdoors and similar. The big question for me is how and why Tim Cook and Apple's board signed off on a plan to do huge damage to Apple's reputation and user trust. If they didn't know, it's a problem. If they knew and didn't realize the inevitable backlash, well that's a different problem.
- lucasyvas 5y ago
  You are completely correct and I find this turn of events hilarious. They are onto the bargaining stage of grief.
- heavyset_go 5y ago
  > This problem is the capability, not what it's used for. Any such capability will be abused by new use cases be it terrorism, drug trafficking, human trafficking or whatever. Plus there will inevitably be unauthorized access. Apple even says it themselves[1]: > This program is ambitious, and protecting children is an important responsibility. These efforts will evolve and expand over time. [1] https://www.apple.com/child-safety/ https://www.apple.com/child-safety/
  - goodells 5y ago
    I can't wait until /child-safety 404s or redirects to /safety and there's a wall of marketing blurb (possibly only in Chinese at first) that explains how 'national security' concerns are reported to the CCP. This has totally pushed me over the edge, though I'll admit I was oblivious to begin with. My plan is to replace the MacBooks with a Thinkpad P15 gen 2 running Ubuntu and replace the iPhone with something running Ubuntu Touch (Volla Phone, Fairphone, OnePlus One). Screw not having control.
paulie4542 5y ago
Trust lost last week. This doesn’t matter.
- tgsovlerkhgsel 5y ago
  I wouldn't say "this doesn't matter", I'd say "this makes it worse" because they're reaffirming that they want to push forward with this despite the criticism.
- newsbinator 5y ago
  Of course it matters- if Apple does a 180º reversal I'll applaud and buy more Apple products. I want companies to be able to acknowledge mistakes unequivocally and fix them. If Apple says "we heard the backlash, we're sorry, we'll never do it again", I'll be more of a supporter than I had been before. Until they do, trust lost.
- paulie4542 5y ago
  I’m saying it doesn’t matter to me because I’ve moved out of their ecosystem. I’m done.
cblconfederate 5y ago
lol, so first it's "at least 30 images", then it's "multiple nations" (how many? which pairs are allowed?) Soon your pictures will be subject to a 3-week review period to decide whether they 're fit to be scanned
heavyset_go 5y ago
This does not address the issue of perceptual hash collisions and false positives at all, nor does it address the issue of on device scanning nor their claim that, according to Apple[1]: > This program is ambitious, and protecting children is an important responsibility. These efforts will evolve and expand over time. People don't want their property spying on them and reporting them to the police. They don't want people looking at their photos or thumbnails. It's patronizing, invasive and embarrassing to have your privacy violated like that. [1] https://www.apple.com/child-safety/ https://www.apple.com/child-safety/
- concinds 5y ago
  You may be interested in this: Security Threat Model Review of the Apple Child Safety Features [pdf] (apple.com) https://news.ycombinator.com/item?id=28173134 https://news.ycombinator.com/item?id=28173134
swiley 5y ago
They're willing to kill the platform over this? What's going over there on the west coast?
- ud_0 5y ago
  They know they can essentially wait out the public outcry, plus they can even buffer things a little bit by calling their critics "confused" and introduce measures that don't address anything but sound reasonable to lay persons in an effort to distort the conversation. The platform is not in danger. But if it were, then yes, I think they'd still risk it. It's either that or getting shut out of markets gradually by law enforcement and governments.
  - swiley 5y ago
    > It's either that or getting shut out of markets gradually by law enforcement and governments. How come that's not happening to desktop Linux then?
    - ud_0 5y ago
      Desktop Linux in no way compares to iOS market share, and even if it did, it'd require a different type of attack. Apple is vulnerable precisely because it is the one single vendor in complete control of anything related to the iPhone. Personally I do think it's likely we'll see a successful anti-Linux smear campaign and/or laws in the next few years, but I have to admit that's a pretty pessimistic take.
concinds 5y ago
These stories have been top of HN for more than a week, indicating fascination. Yet non-technical mainstream articles make the bulk of it, and Apple's just released detailed (and convincing) threat report PDF is getting buried. An unfortunate recipe that'll solidify misunderstandings and keep the conversation much more emotional than it should be. Security Threat Model Review of the Apple Child Safety Features [pdf] (apple.com) https://news.ycombinator.com/item?id=28173134 https://news.ycombinator.com/item?id=28173134
- gigel82 5y ago
  They can publish the dev design doc, the test plan, even the fricking source code, that doesn't change the basic fact which is local device scanning. There is no misunderstanding of that fact by anyone with sufficient technical know-how. There is also no misunderstanding of what comes next once this Pandora box is opened.
- tgsovlerkhgsel 5y ago
  > solidify misunderstandings The key issue here is the very concept of "Apple turns your iPhone into a snitch", and I haven't seen any misunderstanding around that.
kook_throwaway 5y ago
This perfectly illustrates the problem: the content being scanned for can change arbitrarily and on a whim. Gun, meet foot.
thepasswordis 5y ago
We could probably increase the vaccination rate if we snooped on peoples' phones to find out if they are participating in vaccine-hesitant behavior, and then rate-limited traffic to websites and apps that peddle this dangerous misinformation. How many people have to lose their lives in the service of some pedantic idea of "privacy"? It's a computer looking at it, it's not even a human person. I think tech companies need a hippocratic oath similar to "first do no harm". Apple should not be engaging in misinformation trafficking, and should at the very least be working to minimize harm by preventing people form falling victim to dangerous, unsubstantiated, and un-fact-checked information. This is especially important when our elected officials use the considerable power that has been gifted to them by the people to put peoples' lives in danger by spreading dangerous misinformation. What role did apple's inaction on this have in the pandemic? In the January 6th insurrectionist's attempt to overturn a validated, secure, and duly certified democratic election? What role did Apple's inaction play in the attempted kidnapping of the governor of Michigan? By refusing to help, they are partially responsible for these things. It's time for us to demand that they do their fair share of helping. Inaction is itself an action. -- This is sarcasm, of course. For how long?
PierceJoy 5y ago
It was not that long ago that Apple wrote this letter https://www.apple.com/customer-letter/ https://www.apple.com/customer-letter/. > Building a version of iOS that bypasses security in this way would undeniably create a backdoor. And while the government may argue that its use would be limited to this case, there is no way to guarantee such control.
- nrabulinski 5y ago
  > For many years, we have used encryption to protect our customers’ personal data because we believe it’s the only way to keep their information safe. We have even put that data out of our own reach, because we believe the contents of your iPhone are none of our business. How ironic of you, Apple
post_break 5y ago
Apple these days is the type of company to make a mistake, think they are never wrong, blame anyone but themselves and then change a little bit, only for it to go over like a dead balloon. It reminds me of the Apple Silicon dev kit. No refunds, ok refunds but you have to spend them in a certain time, ok fine you can use them for a longer period and for anything. They are probably completely flabbergasted that people are upset about this so they make this change after a week, completely missing the point.
- adventured 5y ago
  They've been like that since the early days of the company, it's core to their culture, which was built by Steve Jobs. Apple has always had an elitist snobbery at their center, such that if you don't get a thing then you're wrong. That's what Think Different is all about, that's a hyper arrogant proclamation that could only be spewed out of an elitist never-wrong orifice, it's borderline assholic in nature. Part of the cult of Apple has always been that attitude, the Apple fans eat it up, they love the idea that buying Apple products made them feel different (or it used to, before Apple became mainstream), made them feel separate from the masses. Steve Jobs points that out in the D5 interview in 2007 with Gates, how Apple needs the inferior taste of the masses to prime their superiority off of (which is another way of saying: they're wrong, we're right). Go back to their response over the iPhone 4 reception problems ("you're holding it wrong"), same asshole culture then as now, and that culture has been in place for decades: https://www.engadget.com/2010-06-24-apple-responds-over-iphone-4-reception-issues-youre-holding-th.html https://www.engadget.com/2010-06-24-apple-responds-over-ipho...
  - whatever_dude 5y ago
    Eh, I would go even deeper: the original Macintosh keyboard didn't have arrow keys because you shouldn't need it according to z Jobs. People complained and Apple held their feet for a long time until giving up and adding the goddamn arrow keys. That's always been very emblematic of Apple for me. (Hard to find a good citation link on a phone, bit the story's out there.)
  - wincy 5y ago
sergiomattei 5y ago
We don't want it pushed back, we want it dead in the water.
SnowProblem 5y ago
I have a 4" iPhone SE. Love it, but Apple has continued to push me away these last four years. I no longer use any other Apple products except for this 4-year-old phone. What alternatives are there for small phone fans? Sony used to make an Xperia Compact that was nice, but I think they discontinued it.
- swiley 5y ago
  I think there's an out of tree kernel for the SE. You could give PostmarketOS a shot if you have a lot of patience.
deleted 5y ago
[deleted]
fennecfoxen 5y ago
In before China starts diplomatically leaning on "multiple nations" to flag dissident imagery for review.
tehnub 5y ago
They put out a new paper [0] describing the security thread model they were working with, and this paragraph on page 9 stood out to me: The perceptual CSAM hash database is included, in an encrypted form, as part of the signed operating system. It is never downloaded or updated separately over the Internet or through any other mechanism. This claim is subject to code inspection by security researchers like all other iOS device-side security claims. Could someone tell me how that inspection works? Are there researchers who are given the source code? [0]: https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model...
- dang 5y ago
  That paper is being discussed at https://news.ycombinator.com/item?id=28173134 https://news.ycombinator.com/item?id=28173134.
pembrook 5y ago
This is by far the biggest misstep of Tim Cook’s tenure at Apple thus far. If they don’t kill this program soon, it’s going to overshadow the entire upcoming iPhone event, and will follow Apple around like a dark cloud for years. I can see the headlines now: “New iPhone launches amid massive new privacy concerns.” Anytime someone praises Apple for privacy, anywhere on the internet, there will be a tidal wave of people bringing up this program in rebuttal. From people who would have previously defended Apple to the grave! I cannot fathom how on earth anybody thought this was a good idea. It’s like taking decades and billions of dollars worth of hard won reputation for privacy and throwing it in the garbage at the worst possible moment.
- n8cpdx 5y ago
  The folks who choose iPhone in part because of a perception of relative privacy and respect for the user (like myself) will think twice. My two concerns are that Android as an ecosystem is almost certainly still worse, and that the vast majority of users will not care. I’m tempted to jailbreak my devices going forward, although I guess the folks at Apple would say that makes me a pedophile. Edit: seeking recommendations for Android phones with strong performance and reasonable privacy protections. Ideally one that can be used without a Google account.
  - ucm_edge 5y ago
    I'm very nostalgic for Blackberry again. I feel like on a platform where the whole point was to prevent anyone who wasn't the corporate IT department from being able to access it, this local code execution would have been a massive non starter.
  - rkagerer 5y ago
    I put LineageOS on an S5 a while back (with gApps, but there are non-Google alternatives) and it went pretty smoothly. Works with several more modern phones.
  - lifty 5y ago
    You can use something like CalyxOS with a pixel 5
babesh 5y ago
I wonder what the other countries will be and whether they will be disclosed? My bet is on Canada, UK, Australia, and New Zealand. https://en.wikipedia.org/wiki/Five_Eyes https://en.wikipedia.org/wiki/Five_Eyes
firebaze 5y ago
I'm not sure if this addresses the main point. Apple promised privacy, as long as you're not violating laws. The most recent move changed this: now you're a suspect by default, your photos will be scanned even on your local device, and you won't know what the criteria are to flag you, since they're hidden from your knowledge and you can't defend yourself against that hidden criteria. So what's gotten better by this? In the kindest interpretation, you'll be safe from Fascististan's hashes. But what if Fascististan has been bribed by China or the US? And so on. A line which should never have been crossed was crossed.
deleted 5y ago
[deleted]
- dang 5y ago
  That paper is being discussed at https://news.ycombinator.com/item?id=28173134 https://news.ycombinator.com/item?id=28173134.
busymom0 5y ago
Multiple nations being Five Eyes nations? Is that supposed to make us trust them more??
josh_today 5y ago
Start with x and settled on x/2
blisterpeanuts 5y ago
This entire project has been a public relations catastrophe not only for Apple, but also for the NCMEC. Almost no one held the National Center for Missing & Exploited Children in low regard prior to last week. Why would we? It's one of the best, most noble causes that have ever existed. Then, Marita Rodriguez, an executive director at NCMEC, reassured employees in an internal memo that was unfortunately leaked:[1] "I know it’s been a long day and that many of you probably haven’t slept in 24 hours. We know that the days to come will be filled with the screeching voices of the minority. Our voices will be louder." The "screeching voices of the minority" phrase got picked up and (perhaps unfairly) has become a kind of rallying cry, exposing the arrogance of the NCMEC. I say "unfairly" because I do believe that their hearts were in the right place with this initiative, although they are desperately naive when it comes to the principles of privacy. "Let's save the children - how can we stop this toxicity of child porn - let's set something up with the cloud providers!" But the damage has been done. Now the only tenable outcome is for Apple to simply cancel the project. I, for one, was looking forward to upgrading from an Android to an iPhone 12 Pro Max later this year. That plan is now on hold! I doubt it will affect their sales too badly, at least at first. Maybe it will after the Chinese government starts using the back door to round up dissidents, or some Middle Eastern governments use it to crack down on homosexuality, or... the mind boggles. [1] https://www.howtogeek.com/746588/apple-discusses-screeching-voices-of-the-minority-in-internal-memos/ https://www.howtogeek.com/746588/apple-discusses-screeching-...
- ipv6ipv4 5y ago
  This affair has me wondering about NCMEC and if it has publicly accountable oversight.
peanut_worm 5y ago
Am I missing something or wont that just let more pedos get away without giving ANY more privacy?
type0 5y ago
Let's talk about a nanny state called Finland for a moment > the Finnish National Bureau of Investigation (NBI), had compiled a secret blacklist of websites that it deemed to contain child pornography and sent it to Finnish Internet service providers. > Analyzing the address list, Nikki also noted that the first three Google search results for "gay porn" are censored. Electronic Frontier Finland (EFFI) have noticed that the blacklist includes non-pornographic websites also, including a Windows advice forum, a computer repair service and the Internet Initiative Japan server nn.iij4u.or.jp that, among others, hosts websites for a violin factory, a doll store and a hearing aid manufacturer. from https://en.wikipedia.org/wiki/Lapsiporno.info https://en.wikipedia.org/wiki/Lapsiporno.info
thewarrior 5y ago
NSA agent 1: So we need Apple to scan this image on all phones but they won’t do it because only we need it. GHCQ agent 2: No worries we got you covered we’ll add it to our list as well.
sandworm101 5y ago
Multiple nations? USA+Canada? China+HongKong? Russia+Belarus? In practice, this statement means nothing good. It does mean that Apple is able and willing to manage country-specific blacklists.
jiggawatts 5y ago
My problem with all of this is not some abstract slippery slope of "what happens if China starts insisting that Tank Man be added to the hash list" or whatever. It's the practical scenarios where people in the United States have previously been indicted and their lives ruined over "child pornography" or "statutory rape" that stretches the definition to the breaking point. Young men just barely 18 years old have gone to jail because they have taken pictures of their technically under-aged girlfriends. Often their real crime is that they've simply upset someone rich and powerful, or that they were guilty of being black and dating a blonde white girl. I wish I was being facetious, but this happens all the time. More importantly, in this digital age, many stupid teenagers upload nude pictures of their under-aged girlfriends to the Internet. Sometimes as revenge for the girl breaking up with them, sometimes because they "hacked" a girl's account and downloaded their selfies, or they legitimately have the photos and just wanted those sweet internet karma points. Imageboards like 4chan are full of what is technically CP, even though nobody was directly harmed in its creation. Other stupid teenagers download these photos, as do older paedophiles that trawl these websites looking for CP. Some of them will share it with other paedophiles, internationally even. So what happens if one of these paedophiles gets arrested for a serious child sex crime? Their PCs and their phones will be searched, and the pictures will end up on the CP hash database, of course. Even with this "multiple nations" requirement, that just requires two arrests for the hash to be added. Nothing really changed, there's just a delay. Now what will happen to the stupid teenagers that downloaded the same photo? Teenagers that aren't child molesters! Teenagers that aren't paedophiles! Apple will probably say: No worries, we filter out the phones belonging to under-aged children from the results. ... until they turn 18. Get it? This is a legal landmine in the pocket of every stupid kid. YOUR stupid kid, that looks at porn on the internet, just like every other stupid kid. This is just one scenario where this could lead to a false positive that totally and irrevocably destroys someone's life. I can think of several more, and dozens once you start adding the political pressure from various other nations. There is just no way to make this kind of dragnet surveillance safe. Even a tiny false positive rate is unacceptable when there are hundreds of millions of people playing this distopian lottery.
- zepto 5y ago
  This is a highly informative comment. Thanks!
squarefoot 5y ago
That's just smoke in the eyes while hiding the real problem which is allowing monitoring, not where, when, by whom, and for what reason. And I would also like to get some real numbers about children abuse cases compared to drugs/domestic/gangs/organized/etc. crimes. In other words, how much they impact the society. Wanna bet that in the world there are for example a lot more normal citizens abused by the police every day than old men wanking to a kid picture? When will come the scanning of cops iPhones to catch the "bad apples"? I'm very serious. Child abuse is a cancer and must be stopped, but if they're employing all that technology for this and not for that, well, dear Apple, I want to see some numbers that would justify the choice. And don't reply "even saving only one child is worth ... yadda yadda yadda" You know what I mean.
squarefoot 5y ago
Just out of curiosity, would it be possible to fight this back by creating a huge number of false positives without involving actual CSAM content?
deleted 5y ago
[deleted]
aussieguy1234 5y ago
Since hearing about this whole affair, one potential scenario popped up in my mind... Currently, bad actors are randomly sharing illegal content in an unsolicited way to people who did not ask for it (1). Let's imagine this happens to you. Suddenly, you have illegal content on your device you didn't ask for. Maybe you are busy when the message with the content is received and you don't even know about it, or you think it's random spam from someone you don't know and just ignore it. Then, the content scanner picks it up and notifys authorities. What will happen in this scenario? (1) https://www.news4jax.com/news/2018/02/05/beware-child-porn-message-spreading-through-facebooks-messenger-app/ https://www.news4jax.com/news/2018/02/05/beware-child-porn-m...
- nabakin 5y ago
  Well, to be factually accurate, in this case with Apple, nothing will happen. Apple only reports when it matches 30 instances of this content. They have said they plan to reduce it from 30 as they improve their algorithm.
  - yunohn 5y ago
    > Well, to be factually accurate, in this case with Apple, nothing will happen. To be factually accurate, the commenter didn’t mention a number. Sending 30+ images of anything to someone’s number is trivial.
    - nabakin 5y ago
      True
deleted 5y ago
[deleted]
pcurve 5y ago
If they cared about privacy so much, why wouldn't they just alert you and block you from uploading in the first place? By this becoming such a hot topic, I think this will push the activity to more underground. I bet most people didn't even know that other cloud companies did search for CP.
ArcVRArthur 5y ago
I think the most reasonable solution to this problem is to not purchase Apple products for a few years given how they have responded to criticism and promoted the characterization of privacy advocates as "screeching voices". I was thinking of upgrading my iPhone given it's getting pretty old now but I can't really see myself doing that now.
ashneo76 5y ago
To everyone looking for alternatives, there are quite a few but strong alternatives. They are on a spectrum of complexity/privacy/convenience. Android probably is not much different. But and this a big one, android devices are not tied to the host google OS and are waaaay way more open than Apple. This is because of Apple's ir tight grip on their hardware ecosystem and lobbying against right to repair. Also, the abundance of android phone makers means you get features at a variety of price points. The OS options are LineageOS (my daily driver), CalyxOS worth more google support, microG lineage. More different OS are Ubuntu touch (very active), postmarket OS with multiple frontends just like on Linux desktop, Manjaro Mobile, etc. And hardware wise, there is Pine phone and Purism. But the more people paying for these, the more there is money for the competition. Apple id already sitting on a staggering $200billion dollars. $200 billion.
hvis 5y ago
Suppose we trust what they're saying, what proportion of "children abuse" photos is supposed to be flagged in "multiple nations"? How many children is this going to protect, to weigh against the loss of privacy across the whole userbase [in US] [for now]?
737min 5y ago
China will easily get multiple countries to flag content as problematic. Values are not reflected in UN votes, either.
johndonne 5y ago
I wouldn’t be surprised if Apple’s motivation is the liability they’re facing for having so much CSEM on they’re servers generated by children themselves during a year in near isolation. Section 203 may offer some protection but publicity would risk their revenue as parents reacted by ditching all of the Apple products their kids own.
reacharavindh 5y ago
What is a reasonably “Good enough” alternative for an iPhone that does not tie into Google services?
andrewmcwatters 5y ago
> The implications of the government’s demands are chilling. If the government can use the All Writs Act to make it easier to unlock your iPhone, it would have the power to reach into anyone’s device to capture their data. The government could extend this breach of privacy and demand that Apple build surveillance software to intercept your messages, access your health records or financial data, track your location, or even access your phone’s microphone or camera without your knowledge.[1] [1]: https://www.apple.com/customer-letter/ https://www.apple.com/customer-letter/ Chilling indeed, Tim, chilling indeed.
beezischillin 5y ago
I wonder if anyone would've really cared (or at least cared this much) if they just nixed the possible plans to do end to end on the entire iCloud and just did the scanning server-side. I probably wouldn't have started looking at other options, personally. It's somewhat funny to me that switching to a Mac and eventually to multiple Macs is what had me thinking that I could probably be fine just using Linux for my day-to-day computing that doesn't involve gaming (I've a Windows desktop for that) and now this has me thinking "yeah, I can pretty much make the switch and realistically not feel like I'm missing out on too much, anyway.".
deleted 5y ago
[deleted]
brokenmachine 5y ago
I hate Apple, they're ruining computers. 1. Devaluing general-purpose computing. 2. Disposable devices with soldered-in batteries and no upgradeability. 3. Removing headphone sockets so they can nickel-and-dime with irritating dongles and ridiculously expensive wireless headphones. 4. Disallowing personal backups using your own key to incentivize the use of expensive icloud storage. 5. App-store corruption. 6. Dodging tax in my country. 7. A million examples of shitty hardware design and then blaming the customer somehow, eg "you're holding it wrong", dust-attractant garbage keyboards, overheating gpus, etc etc What a shitty, penny pinching piece of shit company. The most expensive devices and they still want to squeeze their customers more. Literally everything they do trades privacy/usability for money, and the cherry on the shit sundae is always insulting their customers intelligence by somehow claiming it's for their own good. And now we get probably the very worst thing they've done: 7. Legitimizing on-device scanning and adversarial devices. I dearly wish they would go out of business because they're ruining technology, something I've loved my whole life. I never even bought one of their bullshit locked-down garbage disposable devices and they're still ruining things for me with their influence on the market. Fuck you, Apple. Fuck you in every way. You're the worst hardware manufacturer and the worst software manufacturer. Go to hell.

263 points by ldayley 5y ago

This article misses the point completely. It was never about what they were looking for, it was that they were looking at all. The quiet part is, once they get acceptance of the snooping using the bogeyman of the day, it will eventually encompass other behavior up to and including political dissent etc.

okamiueru 5y ago
There is also the issue that closed door arbitration of heinous crimes are the perfect tool to put away dissidents. A very convenient excuse to avoid proper checks and balances. "No, you cannot verify, because you aren't allowed to look at the illegal material that proves their guilt"
- hahajk 5y ago
  Is this actually true? If I was arrested for an image on my phone, would no one in the judge/jury team be allowed to see the image that was flagged? I understand it’s allegedly contraband, but if I were a judge I don’t think I’d take an algorithm’s word for it.
  - heavyset_go 5y ago
    > it’s allegedly contraband, but if I were a judge I don’t think I’d take an algorithm’s word for it. The prosecution can hire more expert witnesses with excessive credentials than you can, and they will explain how there's a one in one trillion possibility that the system is wrong, and that the defendant is assuredly a monster. Juries eat that up when it comes to bogus DNA, bite mark, fingerprint, or other forensic evidence claims. Most people think computers can't be wrong or biased, and people's perceptions of what can be deemed reasonable doubts or not seem to shift when computers are involved, or when smart, credentialed people tell them their reasonable doubts aren't reasonable at all because of that one in a trillion chance of the computers being wrong.
jsjohnst 5y ago
I agree. Further, is there any doubt that these “CSAM”² hash databases aren’t already shared between countries? Because if there was doubt, there shouldn’t be, because agencies do share across borders. ² - reason for the scare quotes is because I have first hand knowledge of non-CSAM content being in NCMEC’s database (most likely via data entry errors, but I can’t be for sure).
- belorn 5y ago
  Images of bonsai trees?
  - jsjohnst 5y ago
    > Images of bonsai trees? Ha! Come to think of it, I think that was one example. The main examples that came to memory (it’s been almost 8 years since I was involved) when discussing this last week were essentially extremely common photos (like the stock Windows XP background, among others).
deugo 5y ago
I think this is (quietly) already happening: https://www.reuters.com/technology/exclusive-facebook-tech-giants-target-manifestos-militias-database-2021-07-26/ https://www.reuters.com/technology/exclusive-facebook-tech-g... Instead of hashes looking for sexual abuse images, hashes look at signs of white supremacist behavior. Instead of reporting such racism or behavior to the government, it is shared with all the major tech companies, including AirBnB and Paypal. So, if Facebook deems you to be a political outcast, based on your conversations on their platform, you may find yourself without access to hotels or online payment platforms. The bogeyman of the day is the white supremacist, and coordinated snooping by big tech is the quiet part. > Over the next few months, the group will add attacker manifestos - often shared by sympathizers after white supremacist violence - and other publications and links flagged by U.N. initiative Tech Against Terrorism. It will use lists from intelligence-sharing group Five Eyes, adding URLs and PDFs from more groups, including the Proud Boys, the Three Percenters and neo-Nazis. > The firms, which include Twitter (TWTR.N) and Alphabet Inc's (GOOGL.O) YouTube, share "hashes," unique numerical representations of original pieces of content that have been removed from their services. Other platforms use these to identify the same content on their own sites in order to review or remove it. As a company to become part of the Global Internet Forum to Counter Terrorism, you have to pledge your support to expanding from terrorism towards all online extremism. If you can't accept that, they will still send a speaker to educate your company.
spoonjim 5y ago
And, just introducing this idea can make the concept real, even if Apple cancels this plan tomorrow. Country X could introduce a law saying “all companies that sell phones here must scan and report for these hashes” and based on recent trends, the companies will just roll over and say Uncle.

codezero 5y ago

I expect any powerful nation could “share” with a desperate ally or otherwise leverage them into adopting the same database, creating a false overlap.

bellyfullofbac 5y ago
Yeah. "Dear debtor countries, the attached file contain hashes of new files deemed illegal for the harmonious well-being of peoples for this month, please enter them into your databases."
deleted 5y ago
[deleted]

adrr 5y ago

How does that stop a FISA subpoena with a gag order? The system exists that any secret government order can exploit.

ls612 5y ago
A FISA subpoena or an NSL can only compel Apple to give certain types of information that Apple already has. They can’t compel Apple to gather information they don’t already have.
- kook_throwaway 5y ago
  You have far more faith in a system of secret courts and gag orders than I do.
  - busymom0 5y ago
    Yep. FBI Lawyer, Kevin Clinesmith gets to lie to the FISA court, fabricate emails to get 2-hop warrants and then gets zero days in prison and as of this month is allowed to practice law again. It's an absolute joke.
- captainredbeard 5y ago
  Hahaha an NSL can do more than that.
- adrr 5y ago
  That can’t be true because the old pen/trap orders required installation of a device to record the calls.
  - ls612 5y ago
    A wiretap order is a different kind of order and goes through the district court, not FISA. The legal standard for it is also much higher than for other subpoenas by statute.
    - 5y ago

istingray 5y ago

What happened to it being USA only? This sounds like one of those errors where they double down on it and call it a "solution". Sounds like they're misunderstanding people's concerns pretty badly. Sorry if we miscommunicated. Kill this "feature" yesterday. Thanks Tim!

dannyw 5y ago
This is just business. Governments around the world are thinking of breaking up the app store, and Apple is saying they'll help governments surveil on their citizens inside their walled garden. Would be a shame if users can escape from the surveillance by choosing alternates outside of the walled garden, wouldn't it? Expect an App Store rule soon - all photos apps must scan for CSAM. In a year, expect Signal to be banned from all app stores, just like how it's already banned from some countries.
treesprite82 5y ago
USA-only referred to which accounts would be scanned. Whereas it's theoretically a good thing to require the CSAM hashes to be verified by organisations in multiple countries (rather than a single US government-tied organisation that dismissed privacy concerns as "the screeching voices of the minority"). I'm hoping for something along the lines of "iOS will reject hashes that haven't been signed by independent organisations in US, Russia, China, and India" to make it very difficult to push through anything except actual CSAM. Won't be much of a guarantee if it's just Apple saying "we promise we're only using hashes that have been checked by Australia and the US".
TillE 5y ago
My impression was that it's currently US only because that's the only country so far where they've arranged a process to report their users to the cops. No reason they wouldn't expand it as they make arrangements with other countries. They've really bought into a ton of complexity at all levels by doing this instead of just scanning stuff on their own servers, which would at least have been a clear dividing line. Privacy is really all about control, and on-device scanning dangerously blurs that line.

Invictus0 5y ago

It should be obvious by now that Apple has been coerced into building this scanning system, and the CSAM scanning is just the front for a much more broad "national security" backdoor. Apple's PR team has surely been shitting bricks for the entire week--this is the worst PR debacle they've had since the iPhone 4 Antennagate in 2010.

istingray 5y ago
Completely agree. A friend of mine asked what was happening, and I just said "it's a big deal, biggest thing in a long time". Antennagate is a good comparison -- though this seems bigger. Impacts of Antennagate long term seemed minimal.
- klipklop 5y ago
  Sadly this will likely be little to no impact on Apple long term either. I too agree this is coming from the government. They have found their loophole of using "private industry" to do all the dirty stuff they cannot directly do.
  - tgsovlerkhgsel 5y ago
    Given that privacy has been what Apple was using as they key selling point in their ad campaign, and now they've completely subverted that and mainstream media is picking it up and people are noticing, I suspect this will impact them far more than they expected. While Apple deserves all the fallout they're getting and more, I'm disappointed that the NCMEC that pushed for this isn't also receiving more scrutiny and criticism. Multiple people have now pointed out that their database contains false positives, which is absolutely terrifying. Completely legal, harmless, no-nudity-no-humans pictures can get your life ruined. The truth coming out later doesn't matter when your home gets raided and it slips out that you were caught sharing multiple images that matched hashes from the NCMEC child porn database.
- musha68k 5y ago
  This is way bigger than any RF engineering mishap. I’m going to actively evangelize alternatives to Apple devices and that’s coming from someone who has been doing Apple evangelizing since OSX Panther days. Tens of thousands of dollars of bought devices and services not including all the people I convinced to make the switch over the years. I’m sure I’m not the only one in this regard.

whytaka 5y ago

Conveniently, the other participating nations are all members of Five Eyes. Probably.

2OEH8eoCRo0 5y ago

Wrong answer, Apple.

akomtu 5y ago

Translating from corporatese: "Apple hasn't given up the idea and is looking for a better excuse to push their surveillance solution."

schappim 5y ago

With this much bad PR (and Apple hates bad PR), why is Apple pretty much holding its position?

chrischen 5y ago
Potentially something like a national security letter and they can’t reveal the real reason behind it. I agree this feature is off-brand for Apple and came out of the blue.
- vmception 5y ago
  Maybe this is the warrant canary Lets push that idea so allies in the government can go looking and void it
- underseacables 5y ago
  What would happen if Apple reveal the national security letter? What’s the government going to do, arrest Tim Cook?
- silasdavis 5y ago
  Is that consistent with the amount of time this feature took to implement?
Overton-Window 5y ago
Their handlers have them on a tight leash.
- asteroidbelt 5y ago
  This sounds like a conspiracy theory unless you provide more explanations with some evidence. For example, who are these handlers? What are their motives? The simplest explanation is usually the correct one. The simplest explanation would be some director in Apple wants to get promoted to VP so he or she started this project, pitched it, and very few people opposed it because who wouldn't to prevent harm of children?
5y ago

grishka 5y ago

It's a policy decision, while the technical one is still there, unchanged.

underseacables 5y ago

Still not good enough. That’s like an invading army saying they have a “kinder, gentler machine gun hand.”

dkersten 5y ago

So they're still retaining the ability to look on users devices? I have nothing against them looking at the abuse material, I have a problem with them having power over my device, which could be piggy-backed on by state actors (or others) for their own purposes later. Nobody is against them trying to prevent child sexual abuse, pretty sure we all agree that fighting that is important, but doing so by creating what is essentially a back door of sorts into my devices isn't the right approach to doing so. It sounds to me that this is still allowing them that access, so this changes absolutely nothing.

dwighttk 5y ago
Don’t use iCloud photos
- thoughtstheseus 5y ago
  Don’t use Apple products. The scanning is on device.
  - uninformedhn 5y ago
    ...if you use iCloud Photos. How is it day 47 of HN arguing about this and you all still can’t get basic facts right, and oh boy, you showed me downvoting me twice in 60 seconds for pointing out a lack of fundamental comprehension of what you’re actually upset about This community is obnoxious
- wilkystyle 5y ago
  This is as unhelpful as saying "don't have CSAM on your phone." As has been reiterated many times, the problem is not the capability it is claimed to start off with, it is how the capability will be evolved, abused, and exploited as time goes on.
  - dwighttk 5y ago
    I mean, don’t do that either, but the venn diagram circles of those two statements are not identical.

amq 5y ago

A backdoor is a backdoor.

cletus 5y ago

And... Apple misses the point of the criticism completely. This problem is the capability, not what it's used for. Any such capability will be abused by new use cases be it terrorism, drug trafficking, human trafficking or whatever. Plus there will inevitably be unauthorized access. The only way to prevent all this is for the system not to exist. I don't buy into theories that Apple is being pressured or coerced on any of this. I believe it's far more likely this is just tone-deaf but well-intentioned incompetence. It's classic "why won't anyone think of the children?" and we've seen it time and time again with encryption backdoors and similar. The big question for me is how and why Tim Cook and Apple's board signed off on a plan to do huge damage to Apple's reputation and user trust. If they didn't know, it's a problem. If they knew and didn't realize the inevitable backlash, well that's a different problem.

lucasyvas 5y ago
You are completely correct and I find this turn of events hilarious. They are onto the bargaining stage of grief.
heavyset_go 5y ago
> This problem is the capability, not what it's used for. Any such capability will be abused by new use cases be it terrorism, drug trafficking, human trafficking or whatever. Plus there will inevitably be unauthorized access. Apple even says it themselves[1]: > This program is ambitious, and protecting children is an important responsibility. These efforts will evolve and expand over time. [1] https://www.apple.com/child-safety/ https://www.apple.com/child-safety/
- goodells 5y ago
  I can't wait until /child-safety 404s or redirects to /safety and there's a wall of marketing blurb (possibly only in Chinese at first) that explains how 'national security' concerns are reported to the CCP. This has totally pushed me over the edge, though I'll admit I was oblivious to begin with. My plan is to replace the MacBooks with a Thinkpad P15 gen 2 running Ubuntu and replace the iPhone with something running Ubuntu Touch (Volla Phone, Fairphone, OnePlus One). Screw not having control.

paulie4542 5y ago

Trust lost last week. This doesn’t matter.

tgsovlerkhgsel 5y ago
I wouldn't say "this doesn't matter", I'd say "this makes it worse" because they're reaffirming that they want to push forward with this despite the criticism.
newsbinator 5y ago
Of course it matters- if Apple does a 180º reversal I'll applaud and buy more Apple products. I want companies to be able to acknowledge mistakes unequivocally and fix them. If Apple says "we heard the backlash, we're sorry, we'll never do it again", I'll be more of a supporter than I had been before. Until they do, trust lost.
paulie4542 5y ago
I’m saying it doesn’t matter to me because I’ve moved out of their ecosystem. I’m done.

cblconfederate 5y ago

lol, so first it's "at least 30 images", then it's "multiple nations" (how many? which pairs are allowed?) Soon your pictures will be subject to a 3-week review period to decide whether they 're fit to be scanned

heavyset_go 5y ago

This does not address the issue of perceptual hash collisions and false positives at all, nor does it address the issue of on device scanning nor their claim that, according to Apple[1]: > This program is ambitious, and protecting children is an important responsibility. These efforts will evolve and expand over time. People don't want their property spying on them and reporting them to the police. They don't want people looking at their photos or thumbnails. It's patronizing, invasive and embarrassing to have your privacy violated like that. [1] https://www.apple.com/child-safety/ https://www.apple.com/child-safety/

concinds 5y ago
You may be interested in this: Security Threat Model Review of the Apple Child Safety Features [pdf] (apple.com) https://news.ycombinator.com/item?id=28173134 https://news.ycombinator.com/item?id=28173134

swiley 5y ago

They're willing to kill the platform over this? What's going over there on the west coast?

ud_0 5y ago
They know they can essentially wait out the public outcry, plus they can even buffer things a little bit by calling their critics "confused" and introduce measures that don't address anything but sound reasonable to lay persons in an effort to distort the conversation. The platform is not in danger. But if it were, then yes, I think they'd still risk it. It's either that or getting shut out of markets gradually by law enforcement and governments.
- swiley 5y ago
  > It's either that or getting shut out of markets gradually by law enforcement and governments. How come that's not happening to desktop Linux then?
  - ud_0 5y ago
    Desktop Linux in no way compares to iOS market share, and even if it did, it'd require a different type of attack. Apple is vulnerable precisely because it is the one single vendor in complete control of anything related to the iPhone. Personally I do think it's likely we'll see a successful anti-Linux smear campaign and/or laws in the next few years, but I have to admit that's a pretty pessimistic take.

concinds 5y ago

These stories have been top of HN for more than a week, indicating fascination. Yet non-technical mainstream articles make the bulk of it, and Apple's just released detailed (and convincing) threat report PDF is getting buried. An unfortunate recipe that'll solidify misunderstandings and keep the conversation much more emotional than it should be. Security Threat Model Review of the Apple Child Safety Features [pdf] (apple.com) https://news.ycombinator.com/item?id=28173134 https://news.ycombinator.com/item?id=28173134

gigel82 5y ago
They can publish the dev design doc, the test plan, even the fricking source code, that doesn't change the basic fact which is local device scanning. There is no misunderstanding of that fact by anyone with sufficient technical know-how. There is also no misunderstanding of what comes next once this Pandora box is opened.
tgsovlerkhgsel 5y ago
> solidify misunderstandings The key issue here is the very concept of "Apple turns your iPhone into a snitch", and I haven't seen any misunderstanding around that.

kook_throwaway 5y ago

This perfectly illustrates the problem: the content being scanned for can change arbitrarily and on a whim. Gun, meet foot.

thepasswordis 5y ago

We could probably increase the vaccination rate if we snooped on peoples' phones to find out if they are participating in vaccine-hesitant behavior, and then rate-limited traffic to websites and apps that peddle this dangerous misinformation. How many people have to lose their lives in the service of some pedantic idea of "privacy"? It's a computer looking at it, it's not even a human person. I think tech companies need a hippocratic oath similar to "first do no harm". Apple should not be engaging in misinformation trafficking, and should at the very least be working to minimize harm by preventing people form falling victim to dangerous, unsubstantiated, and un-fact-checked information. This is especially important when our elected officials use the considerable power that has been gifted to them by the people to put peoples' lives in danger by spreading dangerous misinformation. What role did apple's inaction on this have in the pandemic? In the January 6th insurrectionist's attempt to overturn a validated, secure, and duly certified democratic election? What role did Apple's inaction play in the attempted kidnapping of the governor of Michigan? By refusing to help, they are partially responsible for these things. It's time for us to demand that they do their fair share of helping. Inaction is itself an action. -- This is sarcasm, of course. For how long?

PierceJoy 5y ago

It was not that long ago that Apple wrote this letter https://www.apple.com/customer-letter/ https://www.apple.com/customer-letter/. > Building a version of iOS that bypasses security in this way would undeniably create a backdoor. And while the government may argue that its use would be limited to this case, there is no way to guarantee such control.

nrabulinski 5y ago
> For many years, we have used encryption to protect our customers’ personal data because we believe it’s the only way to keep their information safe. We have even put that data out of our own reach, because we believe the contents of your iPhone are none of our business. How ironic of you, Apple

post_break 5y ago

Apple these days is the type of company to make a mistake, think they are never wrong, blame anyone but themselves and then change a little bit, only for it to go over like a dead balloon. It reminds me of the Apple Silicon dev kit. No refunds, ok refunds but you have to spend them in a certain time, ok fine you can use them for a longer period and for anything. They are probably completely flabbergasted that people are upset about this so they make this change after a week, completely missing the point.

adventured 5y ago
They've been like that since the early days of the company, it's core to their culture, which was built by Steve Jobs. Apple has always had an elitist snobbery at their center, such that if you don't get a thing then you're wrong. That's what Think Different is all about, that's a hyper arrogant proclamation that could only be spewed out of an elitist never-wrong orifice, it's borderline assholic in nature. Part of the cult of Apple has always been that attitude, the Apple fans eat it up, they love the idea that buying Apple products made them feel different (or it used to, before Apple became mainstream), made them feel separate from the masses. Steve Jobs points that out in the D5 interview in 2007 with Gates, how Apple needs the inferior taste of the masses to prime their superiority off of (which is another way of saying: they're wrong, we're right). Go back to their response over the iPhone 4 reception problems ("you're holding it wrong"), same asshole culture then as now, and that culture has been in place for decades: https://www.engadget.com/2010-06-24-apple-responds-over-iphone-4-reception-issues-youre-holding-th.html https://www.engadget.com/2010-06-24-apple-responds-over-ipho...
- whatever_dude 5y ago
  Eh, I would go even deeper: the original Macintosh keyboard didn't have arrow keys because you shouldn't need it according to z Jobs. People complained and Apple held their feet for a long time until giving up and adding the goddamn arrow keys. That's always been very emblematic of Apple for me. (Hard to find a good citation link on a phone, bit the story's out there.)
- wincy 5y ago

sergiomattei 5y ago

We don't want it pushed back, we want it dead in the water.

SnowProblem 5y ago

I have a 4" iPhone SE. Love it, but Apple has continued to push me away these last four years. I no longer use any other Apple products except for this 4-year-old phone. What alternatives are there for small phone fans? Sony used to make an Xperia Compact that was nice, but I think they discontinued it.

swiley 5y ago
I think there's an out of tree kernel for the SE. You could give PostmarketOS a shot if you have a lot of patience.

deleted 5y ago

[deleted]

fennecfoxen 5y ago

In before China starts diplomatically leaning on "multiple nations" to flag dissident imagery for review.

tehnub 5y ago

They put out a new paper [0] describing the security thread model they were working with, and this paragraph on page 9 stood out to me: The perceptual CSAM hash database is included, in an encrypted form, as part of the signed operating system. It is never downloaded or updated separately over the Internet or through any other mechanism. This claim is subject to code inspection by security researchers like all other iOS device-side security claims. Could someone tell me how that inspection works? Are there researchers who are given the source code? [0]: https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model...

dang 5y ago
That paper is being discussed at https://news.ycombinator.com/item?id=28173134 https://news.ycombinator.com/item?id=28173134.

pembrook 5y ago

This is by far the biggest misstep of Tim Cook’s tenure at Apple thus far. If they don’t kill this program soon, it’s going to overshadow the entire upcoming iPhone event, and will follow Apple around like a dark cloud for years. I can see the headlines now: “New iPhone launches amid massive new privacy concerns.” Anytime someone praises Apple for privacy, anywhere on the internet, there will be a tidal wave of people bringing up this program in rebuttal. From people who would have previously defended Apple to the grave! I cannot fathom how on earth anybody thought this was a good idea. It’s like taking decades and billions of dollars worth of hard won reputation for privacy and throwing it in the garbage at the worst possible moment.

n8cpdx 5y ago
The folks who choose iPhone in part because of a perception of relative privacy and respect for the user (like myself) will think twice. My two concerns are that Android as an ecosystem is almost certainly still worse, and that the vast majority of users will not care. I’m tempted to jailbreak my devices going forward, although I guess the folks at Apple would say that makes me a pedophile. Edit: seeking recommendations for Android phones with strong performance and reasonable privacy protections. Ideally one that can be used without a Google account.
- ucm_edge 5y ago
  I'm very nostalgic for Blackberry again. I feel like on a platform where the whole point was to prevent anyone who wasn't the corporate IT department from being able to access it, this local code execution would have been a massive non starter.
- rkagerer 5y ago
  I put LineageOS on an S5 a while back (with gApps, but there are non-Google alternatives) and it went pretty smoothly. Works with several more modern phones.
- lifty 5y ago
  You can use something like CalyxOS with a pixel 5

babesh 5y ago

I wonder what the other countries will be and whether they will be disclosed? My bet is on Canada, UK, Australia, and New Zealand. https://en.wikipedia.org/wiki/Five_Eyes https://en.wikipedia.org/wiki/Five_Eyes

firebaze 5y ago

I'm not sure if this addresses the main point. Apple promised privacy, as long as you're not violating laws. The most recent move changed this: now you're a suspect by default, your photos will be scanned even on your local device, and you won't know what the criteria are to flag you, since they're hidden from your knowledge and you can't defend yourself against that hidden criteria. So what's gotten better by this? In the kindest interpretation, you'll be safe from Fascististan's hashes. But what if Fascististan has been bribed by China or the US? And so on. A line which should never have been crossed was crossed.

deleted 5y ago

[deleted]

dang 5y ago
That paper is being discussed at https://news.ycombinator.com/item?id=28173134 https://news.ycombinator.com/item?id=28173134.

busymom0 5y ago

Multiple nations being Five Eyes nations? Is that supposed to make us trust them more??

josh_today 5y ago

Start with x and settled on x/2

blisterpeanuts 5y ago

This entire project has been a public relations catastrophe not only for Apple, but also for the NCMEC. Almost no one held the National Center for Missing & Exploited Children in low regard prior to last week. Why would we? It's one of the best, most noble causes that have ever existed. Then, Marita Rodriguez, an executive director at NCMEC, reassured employees in an internal memo that was unfortunately leaked:[1] "I know it’s been a long day and that many of you probably haven’t slept in 24 hours. We know that the days to come will be filled with the screeching voices of the minority. Our voices will be louder." The "screeching voices of the minority" phrase got picked up and (perhaps unfairly) has become a kind of rallying cry, exposing the arrogance of the NCMEC. I say "unfairly" because I do believe that their hearts were in the right place with this initiative, although they are desperately naive when it comes to the principles of privacy. "Let's save the children - how can we stop this toxicity of child porn - let's set something up with the cloud providers!" But the damage has been done. Now the only tenable outcome is for Apple to simply cancel the project. I, for one, was looking forward to upgrading from an Android to an iPhone 12 Pro Max later this year. That plan is now on hold! I doubt it will affect their sales too badly, at least at first. Maybe it will after the Chinese government starts using the back door to round up dissidents, or some Middle Eastern governments use it to crack down on homosexuality, or... the mind boggles. [1] https://www.howtogeek.com/746588/apple-discusses-screeching-voices-of-the-minority-in-internal-memos/ https://www.howtogeek.com/746588/apple-discusses-screeching-...

ipv6ipv4 5y ago
This affair has me wondering about NCMEC and if it has publicly accountable oversight.

peanut_worm 5y ago

Am I missing something or wont that just let more pedos get away without giving ANY more privacy?

type0 5y ago

Let's talk about a nanny state called Finland for a moment > the Finnish National Bureau of Investigation (NBI), had compiled a secret blacklist of websites that it deemed to contain child pornography and sent it to Finnish Internet service providers. > Analyzing the address list, Nikki also noted that the first three Google search results for "gay porn" are censored. Electronic Frontier Finland (EFFI) have noticed that the blacklist includes non-pornographic websites also, including a Windows advice forum, a computer repair service and the Internet Initiative Japan server nn.iij4u.or.jp that, among others, hosts websites for a violin factory, a doll store and a hearing aid manufacturer. from https://en.wikipedia.org/wiki/Lapsiporno.info https://en.wikipedia.org/wiki/Lapsiporno.info

thewarrior 5y ago

NSA agent 1: So we need Apple to scan this image on all phones but they won’t do it because only we need it. GHCQ agent 2: No worries we got you covered we’ll add it to our list as well.

sandworm101 5y ago

Multiple nations? USA+Canada? China+HongKong? Russia+Belarus? In practice, this statement means nothing good. It does mean that Apple is able and willing to manage country-specific blacklists.

jiggawatts 5y ago

My problem with all of this is not some abstract slippery slope of "what happens if China starts insisting that Tank Man be added to the hash list" or whatever. It's the practical scenarios where people in the United States have previously been indicted and their lives ruined over "child pornography" or "statutory rape" that stretches the definition to the breaking point. Young men just barely 18 years old have gone to jail because they have taken pictures of their technically under-aged girlfriends. Often their real crime is that they've simply upset someone rich and powerful, or that they were guilty of being black and dating a blonde white girl. I wish I was being facetious, but this happens all the time. More importantly, in this digital age, many stupid teenagers upload nude pictures of their under-aged girlfriends to the Internet. Sometimes as revenge for the girl breaking up with them, sometimes because they "hacked" a girl's account and downloaded their selfies, or they legitimately have the photos and just wanted those sweet internet karma points. Imageboards like 4chan are full of what is technically CP, even though nobody was directly harmed in its creation. Other stupid teenagers download these photos, as do older paedophiles that trawl these websites looking for CP. Some of them will share it with other paedophiles, internationally even. So what happens if one of these paedophiles gets arrested for a serious child sex crime? Their PCs and their phones will be searched, and the pictures will end up on the CP hash database, of course. Even with this "multiple nations" requirement, that just requires two arrests for the hash to be added. Nothing really changed, there's just a delay. Now what will happen to the stupid teenagers that downloaded the same photo? Teenagers that aren't child molesters! Teenagers that aren't paedophiles! Apple will probably say: No worries, we filter out the phones belonging to under-aged children from the results. ... until they turn 18. Get it? This is a legal landmine in the pocket of every stupid kid. YOUR stupid kid, that looks at porn on the internet, just like every other stupid kid. This is just one scenario where this could lead to a false positive that totally and irrevocably destroys someone's life. I can think of several more, and dozens once you start adding the political pressure from various other nations. There is just no way to make this kind of dragnet surveillance safe. Even a tiny false positive rate is unacceptable when there are hundreds of millions of people playing this distopian lottery.

zepto 5y ago
This is a highly informative comment. Thanks!

squarefoot 5y ago

That's just smoke in the eyes while hiding the real problem which is allowing monitoring, not where, when, by whom, and for what reason. And I would also like to get some real numbers about children abuse cases compared to drugs/domestic/gangs/organized/etc. crimes. In other words, how much they impact the society. Wanna bet that in the world there are for example a lot more normal citizens abused by the police every day than old men wanking to a kid picture? When will come the scanning of cops iPhones to catch the "bad apples"? I'm very serious. Child abuse is a cancer and must be stopped, but if they're employing all that technology for this and not for that, well, dear Apple, I want to see some numbers that would justify the choice. And don't reply "even saving only one child is worth ... yadda yadda yadda" You know what I mean.

squarefoot 5y ago

Just out of curiosity, would it be possible to fight this back by creating a huge number of false positives without involving actual CSAM content?

deleted 5y ago

[deleted]

aussieguy1234 5y ago

Since hearing about this whole affair, one potential scenario popped up in my mind... Currently, bad actors are randomly sharing illegal content in an unsolicited way to people who did not ask for it (1). Let's imagine this happens to you. Suddenly, you have illegal content on your device you didn't ask for. Maybe you are busy when the message with the content is received and you don't even know about it, or you think it's random spam from someone you don't know and just ignore it. Then, the content scanner picks it up and notifys authorities. What will happen in this scenario? (1) https://www.news4jax.com/news/2018/02/05/beware-child-porn-message-spreading-through-facebooks-messenger-app/ https://www.news4jax.com/news/2018/02/05/beware-child-porn-m...

nabakin 5y ago
Well, to be factually accurate, in this case with Apple, nothing will happen. Apple only reports when it matches 30 instances of this content. They have said they plan to reduce it from 30 as they improve their algorithm.
- yunohn 5y ago
  > Well, to be factually accurate, in this case with Apple, nothing will happen. To be factually accurate, the commenter didn’t mention a number. Sending 30+ images of anything to someone’s number is trivial.
  - nabakin 5y ago
    True

deleted 5y ago

[deleted]

pcurve 5y ago

If they cared about privacy so much, why wouldn't they just alert you and block you from uploading in the first place? By this becoming such a hot topic, I think this will push the activity to more underground. I bet most people didn't even know that other cloud companies did search for CP.

ArcVRArthur 5y ago

I think the most reasonable solution to this problem is to not purchase Apple products for a few years given how they have responded to criticism and promoted the characterization of privacy advocates as "screeching voices". I was thinking of upgrading my iPhone given it's getting pretty old now but I can't really see myself doing that now.

ashneo76 5y ago

To everyone looking for alternatives, there are quite a few but strong alternatives. They are on a spectrum of complexity/privacy/convenience. Android probably is not much different. But and this a big one, android devices are not tied to the host google OS and are waaaay way more open than Apple. This is because of Apple's ir tight grip on their hardware ecosystem and lobbying against right to repair. Also, the abundance of android phone makers means you get features at a variety of price points. The OS options are LineageOS (my daily driver), CalyxOS worth more google support, microG lineage. More different OS are Ubuntu touch (very active), postmarket OS with multiple frontends just like on Linux desktop, Manjaro Mobile, etc. And hardware wise, there is Pine phone and Purism. But the more people paying for these, the more there is money for the competition. Apple id already sitting on a staggering $200billion dollars. $200 billion.

hvis 5y ago

Suppose we trust what they're saying, what proportion of "children abuse" photos is supposed to be flagged in "multiple nations"? How many children is this going to protect, to weigh against the loss of privacy across the whole userbase [in US] [for now]?

737min 5y ago

China will easily get multiple countries to flag content as problematic. Values are not reflected in UN votes, either.

johndonne 5y ago

I wouldn’t be surprised if Apple’s motivation is the liability they’re facing for having so much CSEM on they’re servers generated by children themselves during a year in near isolation. Section 203 may offer some protection but publicity would risk their revenue as parents reacted by ditching all of the Apple products their kids own.

reacharavindh 5y ago

What is a reasonably “Good enough” alternative for an iPhone that does not tie into Google services?

andrewmcwatters 5y ago

> The implications of the government’s demands are chilling. If the government can use the All Writs Act to make it easier to unlock your iPhone, it would have the power to reach into anyone’s device to capture their data. The government could extend this breach of privacy and demand that Apple build surveillance software to intercept your messages, access your health records or financial data, track your location, or even access your phone’s microphone or camera without your knowledge.[1] [1]: https://www.apple.com/customer-letter/ https://www.apple.com/customer-letter/ Chilling indeed, Tim, chilling indeed.

beezischillin 5y ago

I wonder if anyone would've really cared (or at least cared this much) if they just nixed the possible plans to do end to end on the entire iCloud and just did the scanning server-side. I probably wouldn't have started looking at other options, personally. It's somewhat funny to me that switching to a Mac and eventually to multiple Macs is what had me thinking that I could probably be fine just using Linux for my day-to-day computing that doesn't involve gaming (I've a Windows desktop for that) and now this has me thinking "yeah, I can pretty much make the switch and realistically not feel like I'm missing out on too much, anyway.".

deleted 5y ago

[deleted]

brokenmachine 5y ago

I hate Apple, they're ruining computers. 1. Devaluing general-purpose computing. 2. Disposable devices with soldered-in batteries and no upgradeability. 3. Removing headphone sockets so they can nickel-and-dime with irritating dongles and ridiculously expensive wireless headphones. 4. Disallowing personal backups using your own key to incentivize the use of expensive icloud storage. 5. App-store corruption. 6. Dodging tax in my country. 7. A million examples of shitty hardware design and then blaming the customer somehow, eg "you're holding it wrong", dust-attractant garbage keyboards, overheating gpus, etc etc What a shitty, penny pinching piece of shit company. The most expensive devices and they still want to squeeze their customers more. Literally everything they do trades privacy/usability for money, and the cherry on the shit sundae is always insulting their customers intelligence by somehow claiming it's for their own good. And now we get probably the very worst thing they've done: 7. Legitimizing on-device scanning and adversarial devices. I dearly wish they would go out of business because they're ruining technology, something I've loved my whole life. I never even bought one of their bullshit locked-down garbage disposable devices and they're still ruining things for me with their influence on the market. Fuck you, Apple. Fuck you in every way. You're the worst hardware manufacturer and the worst software manufacturer. Go to hell.