14 ms·
What's shocking to me is how little Apple management understood of what their actions looked like. Really stunning. For a company that marketed itself as one o
by epistasis 5y ago
What's shocking to me is how little Apple management understood of what their actions looked like. Really stunning.
For a company that marketed itself as one of the few digital service providers that consumers could trust, I just don't understand how they acted this way at all.
Either there will be heads rolling at management, or Apple takes a permanent hit to consumer trust.
- FabHK 5y ago> What's shocking to me is how little Apple management understood of what their actions looked like. Really stunning. Maybe because they underestimated people's ignorance. I think they saw (and still do see) it as a better, more privacy preserving technique than what everyone else is doing.
- fraa-orolo 5y agoThe thing is that this is a better and more privacy preserving technique. Their hubris is in not seeing or thinking that they will be able to stand up to all kinds of abuses of this system that its mere existence will invite; their hubris is also in thinking that they will be able to perfectly and without mistakes manage and overview a system making accusations so heinous that even the mere act of accusing destroy people's lives and livelihoods.
- shuckles 5y agoWhat abuses apply to CSAM scanning in this hybrid pipeline which don’t apply to iCloud Backup? If they scanned on the server, why couldn’t governments “slippery slope” abuse the system by requiring that all files on iOS end up in iCloud Backup, where they can be scanned by the system? Since the announcement, I can think of a dozen ways Apple could be easily forced into scanning all the contents of your device by assembling features they’ve already shipped. Yet they haven’t. At some point, people need to produce evidence that Apple cannot hold the line they’ve said they will.
- candiodari 5y agoExactly. Also people are strongly objecting that their own device is being used to report them to law enforcement. Surely someone at Apple noticed this beforehand ...
- int_19h 5y agoThe governments always could do that, but then they'd be the target of the pushback, and there's already significant mistrust of governments wrt surveillance because of how it can be abused. What we have now is Apple, with its "strong privacy" record, normalizing this. If it succeeds, it would be that much easier for the governments to tackle other stuff onto it. Or, say, lower the threshold needed to submit images for review. I can easily picture some senator ranting about how unacceptable it is that somebody with only 20 CSAM photos won't be flagged, and won't somebody please think of the children? And yes, if it comes to that, Apple definitely cannot hold the line. After all, they already didn't hold it on encrypted cloud storage - and that wasn't even legally forced on them, merely "not recommended".
- justapassenger 5y ago> For a company that marketed itself as one of the few digital service providers that consumers could trust, I just don't understand how they acted this way at all. Because privacy stance is mostly PR to differentiate from Google. And while there're invalid reasons to get users data, there're also valid ones (at least from legal requirement point of view - let's not get into weeds about personal freedom here and if the laws and its implementations need to be changed). Their PR was just writing the checks they cannot cash without going on a war with governments.
- istingray 5y agoIn retrospect this makes sense to me. I don't like it, but I get it now. When Apple said "privacy" what they meant was "we don't like tracking cookies or hackers but everything else is fine".
- websites2023 5y agoPrivacy means “you pay for the device so we don’t sell your attention to advertisers.” That’s it. There’s no protection against state level actors (Pegasus, CSAM scanning, etc.). If your threat model includes being the target of someone who will plant child pornography on your phone, you are already fucked. And no, Apple isn’t suddenly going to scan Chinese iPhones for Winnie the Pooh memes. They don’t have to. China already has the 50 cent party to do that for them, on WeChat. Basically everything everyone seems to think is just around the corner has already been possible for years.
- shuckles 5y agoIs there a consumer computing device that’s more secure than iPhone against state actors? A Chromebook, maybe?
- websites2023 5y agoThere is no consumer grade device that is secure against state actors.
- daxuak 5y agoThey are very likely aware of backlash but since this is a easily defendable hill with a very slippery slope down the way, it is of their interest to push for it IMHO.
- floatingatoll 5y agoApple has declared in interviews that the slope shall not be slipped, but you're indicating that they chose this specifically because they can slip that slope. How did you determine that their intentions contradict their words? Please share the framework for your belief, so that we're able to understand how you arrived at that belief and to evaluate your evidence with an open mind. (Or, if your claim is unsupported conjecture, please don't misrepresent your opinions and beliefs as facts here at HN.)
- monadgonad 5y ago“IMHO” means “in my humble opinion”
- floatingatoll 5y agoI'm glad to see that there now, thanks!
- daxuak 5y agoWords in interviews mean little to me. If it's impossible to hold a cooperation accountable when the verbal promise is broken, that promise, not word-by-word but in its spirit, is void from the beginning. This is just my biased view but there _could_ be countless ways to slip the slope without violating the current framing. For example: Q: What happens when other governments ask Apple to use this for other purposes. A: We will inform them that we did not build the thing they’re thinking of.[1] Note that the question could be interpreted as "use this exact implementation" (rather than use the algorithm in general), and the answer does not rule out the possibility of "but we can build the thing for them then" (rather than "we don't have it and we will never have it"). The reader is free to interprete the conversation as they see it. [1] https://daringfireball.net/2021/08/apple_child_safety_initiatives_slippery_slope https://daringfireball.net/2021/08/apple_child_safety_initia... citing [2] [2] https://www.nytimes.com/2021/08/05/technology/apple-iphones-privacy.html https://www.nytimes.com/2021/08/05/technology/apple-iphones-...
- camillomiller 5y agoApple's management can also be prone to hubris. This is also very much a case where the engineers were left unbridled without any proper check from marketing and comms, I suspect because of the extreme complexity of the problem and the sheer impossibility of putting it into layman terms effectively.
- bhawks 5y agoWas this a pure engineering driven exercise? I hadn't heard that before and it doesn't match up with what I've heard about Apple's internal culture. The level of defensive pushback really makes me feel that they are very bought into the system. Definitely would appreciate a link to anything substantial indicating that this was a bunch of eng in over their heads.
- emptysongglass 5y agoAlso have to agree: I don't see how this could originate from engineers. Every engineer I've spoken with at the company I work for has been mortified by this insanity.
- sorrytim 5y agoIt’s really unpopular inside the fruit company. I’ve not spoken with a single engineer in Cupertino who thinks this is a good path. There has been radio silence from management on how to address this. It almost feels like they want this to fail. In the past management has given us resources and talking points we can use with our friends and family. Not this time.
- orangeoxidation 5y ago> In the past management has given us resources and talking points we can use with our friends and family. Wow. That feels like an overreach. 'We don't just buy your labor, but also your private opinion and tell you how to talk to your family'.
- nojito 5y agoWhy are hash collisions relevent? There are atleast 2-3 further checks to account for this.
- foobiekr 5y agoyour trust in the system is charming
- fraa-orolo 5y agoBecause it's not a cryptographic hash where a one bit difference results in a completely different hash. It's a perceptual hash that operates on a smaller bitmap derived from the image so it's plausible that some innocuous images might result in similar derivations; and there might be intentionally crafted innocently-looking images that result in an offensive derivative. Salvador Dali could do something similar by hand in 1973 in Gala Contemplating the Mediterranean Sea [1] [1] https://en.wikipedia.org/wiki/Lincoln_in_Dalivision https://en.wikipedia.org/wiki/Lincoln_in_Dalivision
- __blockcipher__ 5y agoThis is a great answer but that’s not actually the GP’s contention. Their argument is essentially “so what if there’s a collision, the human review will catch it”. And to that I’d say that the same is supposed to occur for the no-fly list and we all know how that works in practice. The mere accusal itself of possessing CSAM can be life ruining if it gets to that stage. More importantly, a collision will effectively allow warrantless searches, at least of the collided images.
- fraa-orolo 5y agoIndeed, I touched on that in another comment: https://news.ycombinator.com/item?id=28227141 https://news.ycombinator.com/item?id=28227141
- nojito 5y agoThat’s one check. There are other system checks to make the client side hash collision meaningless.
- GeekyBear 5y agoThe thing that's shocking to me is that Google, Microsoft and all the big names in tech have been scanning everything in your account (email, cloud drive, photos, etc) for the past decade, without any noticeable uproar. Apple announces that it is going to start scanning iCloud Photos only, and that their system is set to ignore anything below a threshold of ~30 positives before triggering a human review, and people lose their minds.
- rblatz 5y agoBecause it isn’t about the CSAM scanning. Apple keeps reframing the argument back to that. most people expect that google and Microsoft to scan files on their servers for CSAM, people object to Apple turning your phone against its owner, and once building out the capability to do this only policy prevents it from being abused.
- websites2023 5y agoBigCos, take note: you’re better off doing nefarious shit without telling anyone. Because, if you come clean, you’ll only invite an endless parade of bloggers who will misconstrue your technology to make you look bad.
- justin_oaks 5y agoIt's important to keep nefarious stuff on the server side because eventually someone will reverse engineer what's on the client side. Imagine if Apple had done this on the client side without telling anyone, and later it was discovered. I think things would be a whole worse for Apple in that case.
- websites2023 5y agoThat being the case, why do it client side at all, when presumably every claim they make is verifiable?
- squarefoot 5y agoDevices with proprietary OSes spend more and more time phoning home, then exchanging data officially for "updates". They probably tell the truth, but should one of them decide to hide users data exfiltration or other monitoring practices behind those updates, it would be quite hard to catch them. In other words we have no way to tell that they're not already doing this.
- helen___keller 5y ago> I just don't understand how they acted this way at all. There's a simple answer to this right? Despite everyone's reaction, Apple genuinely believe this is a novel and unique method to catch CSAM without invading people's privacy. And if you look at it from Apple's point of view that's correct: other major cloud providers catch CSAM content on their platform by inspecting every file uploaded, i.e. total invasion of privacy. Apple found a way to preserve that privacy but still catch the bad people doing very bad things to children.
- lixtra 5y ago> other major cloud providers catch CSAM content on their platform by inspecting every file uploaded That is very unlikely. Most likely they compare some hash against a database - just like apple.
- jowsie 5y agoThat's what they're saying, the key difference is Apples happens on your device, not theirs.
- GiorgioG 5y agoThat's not really better or different. In any meaningful way.
- Spivak 5y agoThen why the outrage if they're the same?
- JimBlackwood 5y agoBecause this is Apple and people are very passionate about both hating and liking Apple.
- nine_k 5y agoIt is indeed meaningfully different: your sensitive data never leaves your device in order to be scanned. There can't be a crack in Apple servers that would expose files of millions of users uploaded for scanning. Eventually it could lead to Apple platform not having any CSAM content, or any known legally objectionable content, because any sane perpetrator would migrate to other platforms, and less sane, caught. This, of course, takes the user base to overwhelmingly agree that keeping legally objectionable content is a bad thing and should be exposed to police, and to trust the whatever body which defines the hashes of objectionable content. I'm afraid this order is not as tall as we could imagine.
- duxup 5y agoI'm kinda amazed. I mentioned I was thinking of moving from an Android phone to Apple soon, somewhat privacy related. My friends lectured me on "they're scanning your photos" ... meanwhile they share their google photos albums with me and marvel about how easy they are to search ... Maybe we (humans) only get outraged based on more specific narratives and not so much the general topics / issues? I don't know but they didn't seem to notice the conflict.
- kzrdude 5y agoIsn't there a small difference between these? A) They scan everything I have released to google photos B) They scan everything that exists on my device Psychologically, you'll feel a difference in what you accept between the two, I think
- shapefrog 5y ago> B) They scan everything that exists on my device No ... They scan everything that I have released to apple photos that exists on my device. Same scan - different place.
- __blockcipher__ 5y agoThe issue is that as soon as you set that precedent, it’s only a matter of time before it extends beyond iCloud. That’s the problem with doing any device-level scanning. This is dystopian and scary. And yes I understand the technology in its current iteration. The current form has problems (weaponizing collisions etc) but the real issue comes with future developments.
- robertoandred 5y agoWas scanning server side not a precedent?
- Multicomp 5y ago
- whoknowswhat11 5y agoWe will see. I've heard lots of these predictions and I don't buy them AT ALL. What I have seen is a selling point for apple products. I'd encourage folks to get out of the HN bubble on this - talk to a wife, a family especially those with kids.
- __blockcipher__ 5y agoYeah, talk to people that know nothing on the issue besides that they want to “protect the kids”. Why stop there? Get out of the HN bubble on the patriot act, instead ask your neighbor’s wife her thoughts on it. Get out of the HN bubble on immigration, go ask a stereotypical boomer conservative about it. I think my sarcasm already made it overtly obvious but, this is horrible advice you are giving and the fact that you don’t seem to be aware that pedophilia and terrorism are the two most classic “this gives us an excuse to exert totalitarian control” topics betrays your own ignorance (or, worse, you are aware and just don’t care).
- floatingatoll 5y agoRight now, it seems like there are two specific groups of people that are upset with Apple: Freedom evangelists (e.g. EFF) and tech futurists (e.g. HN). They're saying, essentially: "Apple does not have my permission to use my device to scan my iCloud uploads for CSAM" and "This is a slippery slope that could result in Apple enforcing thoughtcrimes" Neither of these viewpoints are particularly agreeable to the general public in the US, as far as I can determine from my non-tech farming city. Once the fuss in tech dies down, I expect Apple will see a net increase in iCloud adoption — all the fuss we're generating is free advertising for their efforts to stop child porn, and the objections raised are too domain-specific to matter. It's impossible to say for certain which of your outcomes will occur, but there's definitely two missing from your list. Corrected, it reads: "Either there will be heads rolling at management, or Apple takes a permanent hit to consumer trust, or Apple sees no effect whatsoever on consumer trust, or Apple sees a permanent boost in consumer trust." I expect it'll be "no effect", but if I had to pick a second guess, it would be "permanent boost", well offsetting any losses among the tech/free/lib crowd.
- drvdevd 5y agoYou forgot the most important point of the last 24 hours: "Apple has created a system for detecting CSAM on local devices which has already proven vulnerable to cheap perceptual hash collision attacks. It's now highly inconceivable Apple will be able to deploy this technology as-is without having their users exploited." In other words it's not just about privacy or thoughtcrimes anymore but should be viewed as actually dangerous to use their devices. I feel a bit dramatic even typing that out but I.. think it's true?
- floatingatoll 5y agoI would also warn you against owning any device with a radio. Carriers control the radio towers and can be compelled by government agencies and selfish corporate interests to exploit remote execution vulnerabilities in radio chips in order to plant CSAM content onto devices. How dramatic is too dramatic? When does something that hasn’t happened to you or anyone you know become a risk you’re willing to sacrifice personal convenience to mitigate? Will you be divesting yourself of all wireless radio hardware? If not, then why would you be worried about users being exploited through a more clumsy and less effective process such as CSAM signature hacking? The piece of information you’re taking for granted, that few in free/tech/lib are confronting, is the assumption that this process can be exploited at scale to harm millions of people. So far as I can tell, there will probably be zero or one false positive CSAM matches that pass the known algo, the unknown algo, the human blurred comparison, and the human unblurred comparison — all steps that must occur before law enforcement is invoked to collect digital evidence - in the first year. How many false positives (to the nearest 10^X) do you think the system will generate in the first year that result in law enforcement actions? Your words suggest that everyone is vulnerable, and there are 10^9 users, so do you believe there will be 10^9 false positives in the first year? Do you think only a thousand people will be affected, so 10^3? How do you judge which is more likely correct? It is unlikely that this system will generate 10^9 false positives, or else it never would have passed QA. I encourage you to consider how you would personally quantify this risk, and then also look up the quantified risks for killing someone while driving a car or getting struck by lightning while indoors. I don’t know what the actual reality will be, but I don’t think it's a very large X.
- zepto 5y agoConsumers are probably in favor this, or don’t care. The only people who are bothered are people claiming this is going to be misused by authoritarian governments.
- ravenstine 5y agoNah, their stock is up which means they'll continue on this trajectory.
- teclordphrack2 5y agoI have a feeling they already did a test of this with some small sample of people who did not know they were test subjects. I think that in the future it will come out that what they found was disturbing enough that they thought doing it network wide was a worth while endeavor. This is not me agreeing or disagreeing.
- jiocrag 5y agoRepeating myself from another thread but… China. Conjecture, admittedly: 1. Apple cannot lose the Chinese market. Huge and more important fastest growing geo for the company. 2. China is self deprecating elements of its own tech sector (aggressive crackdowns on both established companies like tencent and Alibaba as well as individual web sites). They are clearly cleaning house from a surveillance and control perspective. Apple is not immune, but it’s an American behemoth, so open door crackdowns are impossible. I don’t think Apple’s CSAM push and China’s crackdown are purely coincidental. Who can argue with stemming child abuse? It’s the type of hot button issue that affords broad acceptance for intrusive tech. The leap from scanning for abuse to scanning for anti regime content is more like a tiny step. It seems obvious from afar that the company adamant about refusal to unlock a potential terrorist’s iPhone on privacy principles (with the attendant marketing benefits) would so suddenly force push (and therefore ensure collection massive training data with or without opt-in for Chinas v2.0) such a boldly invasive feature addition. Turns out vertical integration is both gift and curse (dependent on the whims of the integrator) for on-device privacy and autonomy.
- strogonoff 5y agoIf you use Apple devices, you are agreeing to their ToS which since 2019 give them the right to pre-screen your uploads for any potentially illegal (sic) content, not specifically CSAM imagery. If Apple happens to use similar ToS in China (no idea if true), you bet CCP would be all over this clause. A worst-case wild guess from the pessimist in me: they had to add this clause in 2019 to appease CCP, and then they got thinking if they could make use of it for the greater good (tm), too. Hopefully it’s very incorrect.
- stjohnswarts 5y agoOf course the CCP loves stuff like this. The big thing is will apple acquiesce to keeping mum about scanning everyone's stuff at the directive of the government. Also will that just be a "china" build or will it be in the USA code base but "flipped off"
- antpls 5y agoHN's Point of View doesn't represent the PoV of the 1 billion active iPhone owners. Most people won't have any idea about the meaning of "hashes" and "databases". Not everyone is trying to actively fight the system and shit on everything, most people just want to live happily with their friends and family, they won't care that Apple scans their devices. > "Either there will be heads rolling at management, or Apple takes a permanent hit to consumer trust." Oh god ! How wasn't all of this obvious to the top Apple management, but so obvious to epistasis! Damn, thanks man for correcting and leading Apple to the right track !