16 ms·
I actually think something else happened, and to be honest I think many at Apple behind this decision are likely pretty surprised by the blowback. That is, it
by hn_throwaway_99 5y ago
I actually think something else happened, and to be honest I think many at Apple behind this decision are likely pretty surprised by the blowback.
That is, it seems like Apple really wanted to preserve "end-to-end" encryption, but they needed to do something to address the CSAM issue lest governments come down on them hard. Thus, my guess is, at least at the beginning, they saw this as a strong win for privacy. As the project progressed, however, it's like they missed "the forest for the trees" and didn't quite realize that their plan to protect end-to-end encryption doesn't mean that much if one "end" is now forced to run tech that can be abused by governments.
This kind of thing isn't that uncommon with large companies. To me it seems very similar to the Netflix Qwikster debacle of a decade ago. Netflix could very clearly see the writing on the wall, that the DVD-by-mail business needed to morph into a streaming business. But they fumbled the implementation, and what largely came across to the public was "you're raising prices by 40%."
I give Netflix huge props for actually reversing the decision pretty quickly - I find that exceptionally rare among large corporations. Remains to be seen if Apple will do the same.
- roody15 5y agoThat is, it seems like Apple really wanted to preserve "end-to-end" encryption,“ … except they still have not mentioned anything about E2E encryption… and they currently don’t encrypt icloud backups. You would think apple would get ahead of this story and mention … or maybe they don’t have any E2E plans at all.
- hn_throwaway_99 5y agoThis blog post got a lot of commentary on HN a couple days ago: https://news.ycombinator.com/item?id=28118350 https://news.ycombinator.com/item?id=28118350. iMessages are already E2E encrypted, but you are correct, iCloud backups are decryptable with a warrant (and that was reportedly added at the FBI's request). But I agree with Ben Thompson's point in that blog post, that it's OK to not have strong, unbreakable encryption be the default, and that it's still possible to use an iPhone without iCloud and get full E2E. But with Apple's CSAM proposal is NOT possible to have an iPhone that Apple isn't continuously scanning.
- js2 5y ago> But with Apple's CSAM proposal is NOT possible to have an iPhone that Apple isn't continuously scanning. As currently implemented, iOS will only scan photos to be uploaded to iCloud Photos. If iCloud Photos is not enabled, then Apple isn't scanning the phone.
- hn_throwaway_99 5y agoWoops, you're right, thanks for the correction. I think the issue is that, as Ben Thompson pointed out, the only thing preventing them from scanning other stuff now is just policy, not capability, and now that Pandora's box is open it's going to be much more difficult to resist when a government comes to them and says "we want you to scan messages for subversive content".
- js2 5y agoYes, of course that is true. I use iCloud Photos and find this terribly creepy. If Apple must scan my photos, I'd rather they do it on their servers. I could maybe understand the new implementation if Apple had announced they'd also be enabling E2E encryption of everything in iCloud, and explained it as "this is the only way we can prevent CSAM from being stored on our servers."
- AnthonyMouse 5y ago> "this is the only way we can prevent CSAM from being stored on our servers." Why is that supposed to be their responsibility? Given the scale of their business, there is effectively a certainty that people are transmitting CSAM via Comcast's network. That's no excuse for them to ban end to end encryption or start pushing out code to scan client devices. When you hail a taxi, they don't search you for drugs before letting you inside. Because they're not the police. It isn't their role to enforce the law.
- emodendroket 5y agoIf you become known as a hub for this stuff Congress probably won't see it that way.
- chrisfinazzo 5y agoCSAM scanning is probably the easiest thing they could do to satisfy this demand, and if you don't use iCloud Photos, you're not affected by it at all. As far as encrypted backups go, it's an open question whether they want to deal with the legal and support headaches that such a change would bring. If they continued to do nothing, Congress might force their hand by legislatively outlawing stronger encryption - they had to shit or get off the pot. For users, if you enable this feature, but then lose your password, you are entirely screwed and Apple can't help you. Encrypting "In-transit" as a middle ground is likely good enough for most people, until researchers manage to come up with a better solution.
- mapgrep 5y ago> For users, if you enable this feature, but then lose your password, you are entirely screwed and Apple can't help you. Exactly the same with the phone if you forget your PIN/passcode. So they already do this.
- raxxorrax 5y agoEspecially on Apple devices. You cannot just reset the device if it was ever associated with an account and you don't know the password anymore. You must contact Apple support. Some people still want to employ Apple devices in a business environment. The amount of money that is spend on their devices is just insane.
- chrisfinazzo 5y agoFirst, while the "Erase after X unsuccessful attempts" feature exists, I've never used it and really don't recommend anyone else do so unless you are exceptionally careful. Second, my comment was meant in the context of iCloud - if you don't enable the feature as described, Apple may be able to decrypt and recover something, but if you do go all the way to the logical end, yes, you're out of options.
- mapgrep 5y agoMy point is, Apple has already crossed the rubicon of allowing users to lock themselves out of their data. On an iPhone, regardless of auto-erase settings, if you lock yourself out, Apple cannot help you.
- nicce 5y agoWell, iCloud Photos are now ”kinda” E2E encrypted. iOS 15 beta has new recovery option by secret (which is useful only on E2EE?) And Child Safety was released most likely because the leaks. Motive of the leaks is unknown. They might be waiting for September now.
- deleted 5y ago[deleted]
- gidam 5y agowhat best way to "to preserve end-to-end encryption" than shipping a technology that can bypass it anytime?
- deleted 5y ago[deleted]
- etempleton 5y agoSomething probably did happen. Apple no doubt became increasingly aware of just how legally culpable they are for photos uploaded to iCloud. For content that is pirated Apple would receive a slap on the wrist, but for content that shows the exploitation of children? And that Apple is hosting on their owned servers? There is no doubt every government will throw their full legal weight behind that case. Now they are stuck between a rock and a hard place. Do they or do they not look at iCloud Photos? Well, not in the cloud i guess, but right before you upload them. Is that any better? I don't know? Probably not. They really have no choice though. They either scan in the cloud of scan on the device. Scanning in the cloud would at least let people know where their content truly stands, but then they can't advertise end-to-end encryption and on paper it may have felt like a bigger reversal. I think the long and short answer is. Apple cannot really legally protect your privacy when it comes to data stored on their servers. The end.
- calvinmorrison 5y agoI doubt a judge can find apple liable for hosting encrypted data if its illegal. That would mean every e2e storage solution, or even just encryption and storing files on a s3 bucket host would be liable. Apple should use it's gigantic legal arm to set good precedents for privacy in court.
- closeparen 5y agoFirst, there is nothing stopping a full legislative assault on end to end encryption, various such proposals have had traction recently in both the US and EU. The lawyerly phrase is “knew or should have known.” It could be argued that now that this technology is feasible, failure to apply it is complicity. Think about GDPR. Eventually homomorphic encryption is going to be at a point where “we need your data in the clear so we can do computations with it on the server” will cross from “failure to be on the bleeding edge of privacy research” to “careless and backwards.”
- calvinmorrison 5y agoThe way to win that situation is to not play. Pull all apple products from the shelves for a month and launch a billion dollar ad campaign about what the hell our government is up too. At least try to stick to one governments. Tech companies are entering tricky waters by trying to follow laws of many nations (which conflict) simply because their customers are there
- Unklejoe 5y agoThat would be the best case scenario, but then why wouldn’t they come out and say that? They could have announced this all in one shot and saved themselves a lot of blowback. I believe they have to understand that, which makes me think the theory of them planning to add E2E may not be true.
- spuz 5y agoThey would not talk about this in their public messaging because it would require them to admit that their privacy policies are influenced not just by their own company morals but also by the government. They would have to say something like "We know some users want to fully encrypt their cloud data. We also want to encrypt their data. But the government won't allow us to do it until we protect their interests in stopping child abuse and terrorism". That would contradict their marketing image of having their interests being aligned with their users'.
- krumpet 5y agoI sold all the Netflix stock I had purchased when they announced Qwikster. Woe is me!
- jensensbutton 5y ago> didn't quite realize that their plan to protect end-to-end encryption doesn't mean that much if one "end" is now forced to run tech that can be abused by governments Not just one end, but specifically the end that their users thought they had control over.
- joe_the_user 5y agoThus, my guess is, at least at the beginning, they saw this as a strong win for privacy. As the project progressed, however, it's like they missed "the forest for the trees" and didn't quite realize that their plan to protect end-to-end encryption doesn't mean that much if one "end" is now forced to run tech that can be abused by governments. Sincere or not, the "the only way we could do true X is by making X utterly useless and meaningless" thing certainly seems like bureaucracy speaking. It's a testament to "the reality distortion field" or "drinking the Koolaid" but it shows that the "fall-off" of such fields can be steep.
- onedognight 5y agoWhat has changed again? The practical state is that photos are uploaded in the clear to iCloud today, and Apple can scan them if they want or more importantly if the US Government asks and with any valid subpoena must hand them over. Consider the this new hypothetical future state: 1) photos are encrypted on iCloud 2) photos are hash scanned before being encrypted and uploaded. In this new state no valid subpoena can get access to the encrypted photos on iCloud without also getting the key on your device. What is your metric where this is not a better state than the current state where every photo is available in the clear to any government that asks and must be preserved when given a valid order to do so? If you don’t upload photos to iCloud there is no change. In all scenarios you have to trust Apple to write software that does what they say and that they won’t write software in response to a subpoena to target you specifically. The attack with the hash scanning is someone gets a hash for something in your photos specifically (that doesn’t match a ton of people) and through some means gets it into the database (plausible), but can’t get anything without going though this route. Specifically subpoenas can’t just get all your photos without going through this route. My conclusion is that you are more protected in the hash scan and encrypt state than in current state.
- shakezula 5y agoThere are methods for dealing with CP that would maintain privacy, not subject humans to child pornography, and would jive with their end to end encryption goals. But we’ll never know what they’re doing under the hood and I think that’s the concerning part.
- new_realist 5y agoPlease elaborate on these magical methods.
- shakezula 5y agoThey're not magical. They're just smart use of cryptography. Ashton Kutcher's foundation Thorn https://en.wikipedia.org/wiki/Thorn_(organization) https://en.wikipedia.org/wiki/Thorn_(organization) does a lot of work in this space. They have hashed content databases, they have suspected and watchlisted IP addresses, etc... I don't know why I'm being downvoted. This is an active area of cryptography research, and I've applied these types of content checks in real life. You can maintain good privacy and still help combat child pornography. Sometimes HN takes digital cynicism a step too far.
- anchpop 5y agoChecking the images you upload to icloud against a database of CSAM image hashes is what Apple is updating people’s phones to do. I actually don’t really see the issue and I don’t think the average consumer will care about this one bit. apple has always been able to see your icloud backups in plan text anyway
- heavyset_go 5y ago> That is, it seems like Apple really wanted to preserve "end-to-end" encryption, but they needed to do something to address the CSAM issue lest governments come down on them hard. This seems like an incredible reach to me.
- hn_throwaway_99 5y agoWhy? It's already been well-reported that Apple makes a teeny fraction of the NCMEC reports that other large cloud providers make, and that the reason they hold the keys to iCloud backups was from some requests/pressure from the FBI.
- rossjudson 5y agoWhat other "large cloud providers"? Ones that are similar to Apple? Or general cloud companies? And is "pressure from the FBI" really the only reason why Apple might hold backup keys? Please sketch out your "lost device, need to restore from backup" user journey. For a real person, who has only an iPhone.
- hn_throwaway_99 5y ago> Please sketch out your "lost device, need to restore from backup" user journey. For a real person, who has only an iPhone. How about "enter your master passphrase to access your photos". Like literally all syncing password managers work, or things like Authy work to back up your one-time password seeds. It's not complicated.
- heavyset_go 5y agoBecause the government doesn't just care about CSAM, they care about terrorism, drug and human trafficking, gangs etc. Scanning for CSAM won't change the government's opposition to E2EE, and Apple knows this because, according to their transparency reports, they respond with customers' data to NSL and FISA data requests about 60,000 times a year. Occam's razor also says they aren't playing 11th dimensional chess, and that they just fucked up.
- emodendroket 5y agoI agree with your analysis -- I also think it was probably meant to mollify law enforcement without sacrificing encryption. The same thought occured to me independently right when it was announced.
- adam_ellsworth 5y agoWhat follows is conjecture: >... to address the CSAM issue lest governments come down on them hard. And I think that is the springboard of why this is happening. It's no secret that Apple has lobbyists which by turns have a "pulse" on the trajectory of Bills in the House and Senate. What immediately came to mind to me was H.R.485 of 2021's House Session: https://www.congress.gov/bill/117th-congress/house-bill/485/text https://www.congress.gov/bill/117th-congress/house-bill/485/... My (layperson) estimation is that Apple knows the rewording of it is liable pass both House/Senate in some way either in the 2022 or 2023 sessions and are attempting to get ahead of the issue. (most likely as a "rider" bill at this point) That's my thought process on what may have precipitated Apple's move toward "CSAM"; and I welcome it in some ways (unequivocally, children should not be subjected to sexual abuse) and am wary of it in others (ML is in its infancy in too many respects, and if a false-positive arises who's to say what's liable to happen on the Law Enforcement level). There are infinitely many concerns to be discussed on both sides of this argument, and I want both to succeed in their way. CSAM, however, feels like an attempt to throw a blanket over Apple's domains-of-future-interests while playing an Altruism Card. One thing is certain, though: they've opened the floodgates to these discussions on an international level...
- cronix 5y ago> It's no secret that Apple has lobbyists which by turns have a "pulse" on the trajectory of Bills in the House and Senate. Great, let the bills pass and then they get to publicly blame congress for being "forced to legally comply" instead of jumping the gun and doing an about face on 80% of your reputation built over the last 20+ years. There is 0 reason to do this before being legally forced to. It's not a gamble with a possible upside somewhere. It's directly shooting yourself in the foot.
- unityByFreedom 5y agoYes, 100% agree, make congress go through with passing such a law. Make it a public discussion. Make them consider if they would like their own devices to be built insecurely. They're among the biggest targets of people who would like to subvert US policy.
- AnonC 5y ago> …to be honest I think many at Apple behind this decision are likely pretty surprised by the blowback. If Apple is surprised by the blowback, it’s entirely Apple’s fault because it didn’t engage with many outside experts on the privacy front. I recall reading either in Alex Stamos’s tweets or Matthew Green’s tweets that Apple was invited before for discussions related to handling CSAM, but it didn’t participate. Secretiveness is helpful to keep an advantage, but I can’t believe that Apple is incapable of engaging with knowledgeable people outside with legal agreements to keep things under wraps. This blowback is well deserved for the way Apple operates. And I’m writing this as a disappointed Apple ecosystem user.
- shuckles 5y agoJust because Apple didn’t attend Alex’s mini conference for his contacts does not mean they didn’t engage any privacy experts. Furthermore, Alex is a security expert and not a privacy expert. Finally, if that conference was really pushing the envelope on privacy, where are the innovations from those who did attend? The status quo of CSAM scanning is at least as dangerous as an announcement of an alternative.
- volta83 5y ago> but they needed to do something to address the CSAM issue lest governments come down on them hard. If Apple does not have decryption keys for iCloud content, they can't decrypt it, only the user can. For Apple, it's not technically feasible, and the law supports that. Apple has now demonstrated that it is technically feasible to overcome this challenge by doing things "pre-encryption" on the user's device. From a user's point-of-view, Apple's privacy is worth nothing. It makes no sense for them to encrypt anything, if they leak all data before encryption. Children today, terrorism tomorrow. All your messages, photos, and audio, can be leaked. There are thousands of people at Apple involved in the release of any single feature, from devs and testers, to managers all the way up to VPs. The fact that this feature made it to production proofs that Apple is an organization that I can't trust with my privacy. Their privacy team must be pulling their hairs out. They better start looking for a new job.
- unityByFreedom 5y ago> Apple has now demonstrated that it is technically feasible to overcome this challenge by doing things "pre-encryption" on the user's device. > From a legal stand point, they are now screwed, because now they _must_ do it. That's not the right takeaway. They're only required to turn over data to which they have access. If they continue writing software that makes it so they don't have access to the data, then they do not need to turn it over because they can't. The fact that you can write software to store or transmit unencrypted data is irrelevant.
- mrweasel 5y agoBecause Apple write the software, there will never be a time where they do not at some point have access to the data.
- noduerme 5y agoRight, but there is a big difference between them having the software in place already to steal files off a phone for a government, versus a government telling them "you must deploy this new software." In the past Apple has said no to writing any new spyware for the government. They would not be able to say no very easily if the software is already on the devices.
- bobiny 5y agoI'm not a techie, but I think iOS already has similar technology: Camera detects faces, Photos can recognize people and objects (or cats) — all with help of AI (or ML). I think they are just expanding this and combining with "share".
- xiphias2 5y ago,,Netflix could very clearly see the writing on the wall, that the DVD-by-mail business needed to morph into a streaming business.'' I don't argue with your point, but actually Reed Hastings originally created Netflix the DVD rental company so that he could turn it into a movie streaming company. He had to wait for internet to be fast enough. His interview with Reid Hoffman is pretty cool: https://youtu.be/jYhP08uuffs https://youtu.be/jYhP08uuffs
- shp0ngle 5y agoIt all seems so weird. It's actually easier to the the scanning on the backend compared to scanning of the user's phones. My opinion, maybe entirely wrong, is that it's related to how Apple operates in China. In China, Apple does not actually operate their backends, but their BE is operated by government owned business, which is "leased" by Apple. Maybe they don't want to enable scanning on BE for that reason, and over-engineered local device scanning, that really doesn't make much sense if you think about it twice?
- GuB-42 5y agoAdding to the confusion, they announced both the AI powered parental control feature for iMessage and hash-based scanning of iCloud pictures. The iMessage thing is actual spyware, by design, it is a way for parents to spy on their children, which is fine as long as it is a choice and the parents are in control. And that's the case here. The iCloud thing is much more restrictive, comparing hashes of pictures before you send them to Apple servers. For some reason, they do it client side and not server side. AFAIK, iCloud photos are not E2E encrypted so I don't really know why they are doing it that way when everyone else does it server-side. But combine the news and it sounds like Apple is using AI to scan all your pictures. Which is not true. Unless you are a child and still under your parents authority, it will only compare hashes of picture you send in the cloud, no fancy AI here, and it is kind of understandable for Apple not to want to host child porn.
- sneak 5y agoThey're doing it clientside so that they can do it for images and files that aren't ever sent off the device. I am increasingly convinced this is a backroom deal, just like the iCloud Backup e2e feature (designed and, as I understand it, partially implemented) being nixed on demand (but not legal compulsion) from the FBI. If you get big enough, the 1A doesn't apply to you anymore.
- hereme888 5y agoG
- TriStateFarmer 5y agoYou have misunderstood the entire tech story. Apple are scanning the images only when they are uploaded to iCloud, because they don't want the CSAM pictures on their servers. And you're not alone. Reddit, Facebook, Twitter etc are full of shouting angry people who haven't understood the story at all. It's almost became a Rorschach test between those who understand technology and those who like to shout at things.
- mdoms 5y ago> there's no easy way to explain it and still defend Apple's Privacy narrative, wihout doing extreme mental gymnastics. > > Apple really wanted to preserve "end-to-end" encryption, but they needed to do something to address the CSAM issue There we are.
- randallsquared 5y ago> I think many at Apple behind this decision are likely pretty surprised by the blowback. Maybe they would have been surprised where it ended up, but someone in charge there is definitely on board with a reversal on the previous privacy stance, given the doubling-down with an email signal-boosting the NCMEC calling privacy concerns the "screeching of the minority".
- fidesomnes 5y agoThe Apple employee throwaway justifying how much they suck.
- neop1x 5y agoThey announced CSAM scanning but would you trust there is no scanning if they said they reversed their decision and eventually gave you E2E? Considering how big the firmware size is and that all images are processed by their APIs and they control both SW and HW they could hide such scanning in the firmware or even hardware if they wanted or were forced to. They could add a new separate microcontroller similar to Secure Enclave which would (in addition to "AI-enhancing" the picture) compute and compare CSAM hash. On a positive match, the main CPU would decrypt and run a hidden reporting code which would report the image over TLS-secured channel with Secure Enclave-backed certificate pinning to prevent any MITM. No one would notice because it would be decrypted only after a positive match and no one could see the image being sent out on a positive match because of TLS. It is not just Apple, though. We currently don't have enough control over our devices, especially smart phones. There are some vendor binary blobs on Android too. Now or in the future, these proprietary, very capable devices are ideal for many types of spying. Stallman warned about this many years ago.
- deleted 5y ago[deleted]
- dukeofdoom 5y agoApple know what they are doing. Don't fool yourself by giving plausible excuses. The further away you are from your phone the more privacy you will have. Almost everyone was traced back to the Jan 6th insurrection, because unlike antifa they took their phones with them, so easy trace for government. Now they are in solitary confinement in prison while the world loughs at their expectations of privacy from their phones. If they get a covid passport implemented, you won't be able to go out in public without your phone to prove you've been vaccinated. So leaving a phone behind will not be much of an option if this is to pass. Metadata and contact tracing will keep track of everyone you're near. This dystopian future is possibly just a few weeks away.
- mulmen 5y agoSo Apple has rightfully lost the trust of its users then. We submit to their walled garden because we believe they are benevolent. But when they make this kind of a mistake how can we trust them? I don't really see how they can recover from this. Will they go out of business? no. Will they even lose market share? No. But there will always be that example of the time Apple got it wrong. If I was an Apple employee I would certainly be concerned about how something like this could happen.
- bluepoint 5y agoI also think that some kind of negotiation between Apple and the government happened in the background. Probably something like “don’t break up the company and we give you a backdoor”. Maybe we throw stones at the wrong window? I wish we could know more about this, but it is probably that the inner workings of state surveillance are not meant to be public. What do we do?