12 ms·

"I don't understand most of the technical details of Apple's blog post" I do: - Client side vectorization: the photo is processed locally, preparing a non-rev

"I don't understand most of the technical details of Apple's blog post" I do: - Client side vectorization: the photo is processed locally, preparing a non-reversible vector representation before sending (think semantic hash). - Differential privacy: a decent amount of noise is added the the vector before sending it. Enough to make it impossible to reverse lookup the vector. The noise level here is ε = 0.8, which is quite good privacy. - OHTTP relay: it's sent through a 3rd party so Apple never knows your IP address. The contents are encrypted so the 3rd party never doesn't learn anything either (some risk of exposing "IP X is an apple photos user", but nothing about the content of the library). - Homomorphic encryption: The lookup work is performed on server with encrypted data. Apple can't decrypt the vector contents, or response contents. Only the client can decrypt the result of the lookup. This is what a good privacy story looks like. Multiple levels of privacy security, when any one of the latter 3 should be enough alone to protect privacy. "It ought to be up to the individual user to decide their own tolerance for the risk of privacy violations." -> The author themselves looks to be an Apple security researcher, and are saying they can't make an informed choice here. I'm not sure what the right call is here. But the conclusion "Thus, the only way to guarantee computing privacy is to not send data off the device." isn't true. There are other tools to provide privacy (DP, homomorphic encryption), while also using services. They are immensely complicated, and user's can't realistically evaluate risk. But if you want features that require larger-than-disk datasets, or frequently changing content, you need tools like this.

AJRF 2y ago
> This is what a good privacy story looks like. Not at all. A good privacy story is not sending this data anywhere.
chikere232 2y ago
The nearest neighbour search is sharded, which apple's blog admits is a privacy issue, which is why they're running the DP and OHTTP parts. If apple were to add additional clusters that match "sensitive" content and endeavour to put them in their own shards distinct from landmarks, they defeat the homomorphic encryption part while still technically doing it. The DP part can be defeated with just statistics over time; someone with any volume of sensitive content will hit these sensitive clusters with a higher likelihood than someone generateing noise injected fake searches. The OHTTP part can be defeated in several ways, the simplest of which is just having a clause in a non-public contract allowing apple to request logs for some purpose. They're paying them and they can make up the rules as they go.
geoffmac 2y ago
This answer should be much higher. Thank you
Gabriel54 2y ago
I appreciate the explanation. However, I think you do not address the main problem, which is that my data is being sent off my device by default and without any (reasonable) notice. Many users may agree to such a feature (as you say, it may be very secure), but to assume that everyone ought to be opted in by default is the issue.
- sgammon 2y ago
  "Your data" is not actually being sent off your device, actually, it is being scrambled into completely unusable form for anyone except you. This is a much greater level of security than what you would expect from a bank, for example, who needs to fully decrypt the data you send it. When using your banking apps over HTTPS (TLS), you are trusting the CA infrastructure, you are trusting all sorts of things. You have fewer points of failure when a key for homomorphic encryption resides only on your device. "Opting-in by default" is therefore not unsafe.
- jeffybefffy519 2y ago
  The big mistake here is ownership of your apple devices is an illusion...
- JustExAWS 2y ago
  Do you use iCloud to store your photos?
  - latexr 2y ago
    I’m not the person you asked, but I agree with them. To answer your question: No, I do not use iCloud to store my photos. Even if I did, consent to store data is not the same as consent to scan or run checks on it. For a company whose messaging is all about user consent and privacy, that matters. This would be easily solvable: On first run show a window with: > Hey, we have this new cool feature that does X and is totally private because of Y [link to Learn More] > Do you want to turn it on? You can change your mind later in Settings > [Yes] [No]
    - 2y ago
talldayo 2y ago
The right call is to provide the feature and let users opt-in. Apple knows this is bad, they've directly witnessed the backlash to OCSP, lawful intercept and client-side-scanning. There is no world in which they did not realize the problem and decided to enable it by default anyways knowing full-well that users aren't comfortable with this. People won't trust homomorphic encryption, entropy seeding or relaying when none of it is transparent and all of it is enabled in an OTA update. > This is what a good privacy story looks like. This is what a coverup looks like. Good privacy stories never force third-party services on a user, period. When you see that many puppets on stage in one security theater, it's only natural for things to feel off.
- oneplane 2y ago
  It's not that binary. Nobody is forcing anything, you can not buy a phone, you can not use the internet. Heck, you can even not install any updates! What is happening, is that people make tradeoffs, and decide to what degree they trust who and what they interact with. Plenty of people might just 'go with the flow', but putting what Apple did here in the same bucket as what for example Microsoft or Google does is a gross misrepresentation. Present it all as equals just kills the discussion, and doesn't inform anyone to a better degree. When you want to take part in an interconnected network, you cannot do that on your own, and you will have to trust other parties to some degree. This includes things that might 'feel' like you can judge them (like your browser used to access HN right here), but you actually can't unless you understand the entire codebase of your OS and Browser, all the firmware on the I/O paths, and the silicon it all runs on. So you make a choice, which as you are reading this, is apparently that you trust this entire chain enough to take part in it. It would be reasonable to make this optional (as in, opt-in), but the problem is that you end up asking a user for a ton of "do you want this" questions, almost every upgrade and install cycle, which is not what they want (we have had this since Mavericks and Vista, people were not happy). So if you can engineer a feature to be as privacy-centric yet automated as possible, it's a win for everyone.
  - talldayo 2y ago
    > What is happening, is that people make tradeoffs, and decide to what degree they trust who and what they interact with. People aren't making tradeoffs - that's the problem. Apple is making the tradeoffs for them, and then retroactively asking their users "is this okay?" Users shouldn't need to buy a new phone to circumevent arbitrary restrictions on the hardware that is their legal property. If America had functional consumer protections, Apple would have been reprimanded harder than their smackdowns in the EU.
cma 2y ago
Quantum makes the homomorphic stuff ineffective in the mid-term. All they have to do is hold on to the data and they can get the results of the lookup table computation, in maybe 10-25 years. Shouldn't be on by default.
- oneplane 2y ago
  What makes you think that this is the biggest problem if things like AES and RSA are suddenly breakable? If someone wanted to get a hold of your cloud hosted data at that point, they would use their capacity to simply extract enough key material to impersonate a Secure Enclave. That that point, you "are" the device and as such you "are" the user. No need to make it more complicated than that. In theory, Apple and other manufacturers would already use PQC to prevent such scenarios. Then again, QC has been "coming soon" for so long, it's doubtful that any information that is currently protected by encryption will still be valuable by the time it can be cracked. Most real-world process implementations don't rely on some "infinite insurance", but assume it will be breached at some point and just try to make it difficult or costly enough to run out the clock on confidentiality, which is all that really matters. Nothing that exists really needs to be confidential forever. Things either get lost/destroyed or become irrelevant.
  - cma 2y ago
    This is ostensibly for non-cloud data, derivatives of it auto uploaded after an update.
latexr 2y ago
> The author themselves looks to be an Apple security researcher They’re not. Jeff Johnson develops apps (specifically Safari extensions) for Apple platforms and frequently blogs about their annoyances with Apple, but they’re not a security researcher.
gigel82 2y ago
This may be a "good" privacy story but a way better one is to just not send any of your data anywhere, especially without prior consent.
jazzyjackson 2y ago
This must be the first consumer or commercial product implementing homomorphic encryption is it not? I would be surprised if doing noisy vector comparisons is actually the most effective way to tell if someone is in front of the Eiffel tower. A small large language model could caption it just as well on device, my spider sense tells me someone saw an opportunity to apply bleeding edge, very cool tech so that they can gain experience and do it bigger and better in the future, but they're fumbling their reputation by doing this kind of user data scanning.
- Klonoar 2y ago
  Apple themselves have already used it in the past (Caller ID)
- do_not_redeem 2y ago
  > This must be the first consumer or commercial product implementing homomorphic encryption is it not? Not really, it's been around for a bit now. From 2021: > The other major reason we’re talking about HE and FL now is who is using them. According to a recent repository of PETs, there are 19 publicly announced pilots, products, and proofs of concept for homomorphic encryption and federated analytics (another term for federated learning) combined. That doesn’t seem like a lot … but the companies offering them include Apple,7 Google, Microsoft, Nvidia, IBM, and the National Health Service in the United Kingdom, and users and investors include DARPA, Intel, Oracle, Mastercard, and Scotiabank. Also, the industries involved in these early projects are among the largest. Use cases are led by health and social care and finance, with their use in digital and crime and justice also nontrivial (figure 1). https://www2.deloitte.com/us/en/insights/industry/technology/technology-media-and-telecom-predictions/2022/homomorphic-encryption-federated-learning.html https://www2.deloitte.com/us/en/insights/industry/technology... I do wonder why we don't hear about it more often though. "Homomorphic encryption" as a buzzword has a lot of headline potential, so I'm surprised companies don't brag about it more.
  - fragmede 2y ago
    But what are the products from them that implement HE and that consumers are using? Microsoft, IBM, Intel, and Google have all released libraries for HE, and there's Duality SecurePlu, but as far as actual consumer products, Apple's caller ID phone number lookup and other features in iOS 18 is very possibly the first. As far as why it's not more of a buzzword, it's far too in the weeds and ultimately consumers either trust you or they don't. And even if they don't trust you, many of them are still going to use Apple/Google/Facebook system anyway.
lysace 2y ago
> - OHTTP relay: it's sent through a 3rd party so Apple never knows your IP address. The contents are encrypted so the 3rd party never doesn't learn anything either (some risk of exposing "IP X is an apple photos user", but nothing about the content of the library). Which 3rd party is that?
- gigel82 2y ago
  The NSA, the CCP, etc. depending on jurisdiction. (joking, but not really)
- oneplane 2y ago
  I don't have a list on hand, but at least Cloudflare and Akamai are part of the network hops. Technically you only need 2 hops to make sure no origin or data extraction can be done.
  - jazzyjackson 2y ago
    O good, cloudflare gets one more data point on me, a ping every time I add a photo to my library.
    - threeseed 2y ago
      a) Cloudflare doesn't know about you. It sees an IP address. b) If we follow your tortured logic then every hop along the path from your phone to Apple will have one more data point on you. That's thousands of companies a day.
      - jazzyjackson 2y ago
        I'm just griping that cloudflare has many eyeballs and sees most of the traffic on the internet at this point. How many websites have I used with Cloudflare DDoS protection that checks if I'm a bot by fingerprinting my device? They know plenty about me. I'm also griping that "the data is encrypted !" is not a good enough excuse seeing as how we've known for years that the metadata is a bigger pot of gold for intelligence agencies. That my mac address is taking a photo and hitting a particular cell tower is pretty detailed information, even without knowing the content of the photo.
homofermic 2y ago
Regarding HE: since the lookup is generated by the requestor, it can be used as an adversarial vector, which can result in exfiltration by nearest neighbor (closest point to vector) methods. In other words, you can change what you are searching for, and much like differential power analysis attacks on crypto, extract information.
- brookst 2y ago
  Does the noise addition not help? Is that a mitigation for a different attack?
ustad 2y ago
You're presenting a false dichotomy between "perfect user understanding" and "no user choice." The issue isn't whether users can fully comprehend homomorphic encryption or differential privacy – it's about basic consent and transparency. Consider these points: 1. Users don't need a PhD to understand "This feature will send data about your photos to Apple's servers to enable better search." 2. The complexity of the privacy protections doesn't justify removing user choice. By that logic, we should never ask users about any technical feature. 3. Many privacy-conscious users follow a simple principle: they want control over what leaves their device, regardless of how it's protected. The "it's too complex to explain" argument could justify any privacy-invasive default. Would you apply the same logic to, say, enabling location services by default because explaining GPS technology is too complex? The real solution is simple: explain the feature in plain language, highlight the benefits, outline the privacy protections, and let users make their own choice. Apple already does this for many other features. "Default off with opt-in" is a core principle of privacy-respecting design, regardless of how robust the underlying protections are.
- scosman 2y ago
  I don't believe I said or implied that anywhere: 'You're presenting a false dichotomy between "perfect user understanding" and "no user choice."'? Happy to be corrected if wrong. Closest I come to presenting an opinion on the right way UX was "I'm not sure what the right call is here.". The thing I disagreed with was a technical statement "the only way to guarantee computing privacy is to not send data off the device.". Privacy respecting design and tech is a passion of mine. I'm pointing out "user choice" gets hard as the techniques used for privacy exceed the understanding of users. Users can intuitively understand "send my location to Google [once/always]" without understanding GPS satellites. User's can't understand the difference between "send my photo" and "send homomorphicly encrypted locally differentially private vector of e=0.8" and "send differentially private vector of e=50". Your prompt "send data about your photos..." would allow for much less private designs than this. If we want to move beyond "ask the user then do it", we need to get into the nitty gritty details here. I'd love to see more tech like this in consumer products, where it's private when used, even when opted-in.
  - 2y ago
deleted 2y ago
[deleted]
yapyap 2y ago
> "It ought to be up to the individual user to decide their own tolerance for the risk of privacy violations." -> The author themselves looks to be an Apple security researcher, and are saying they can't make an informed choice here I don’t think that that’s what the author is saying at all, I think he’s saying that Apple should let the user decide for themself if they want to send all this shit to Apple, freedom for the individual. They’re not saying “I dunno”
omolobo 2y ago
So what? Why should the application talk over the Internet to begin with? And why isn't that functionality off by default under a settings option that clearly warns the user of the consequences? I think you're missing the forest for the trees here. And the claims that this is good privacy/security are not at all obvious either. And who are those third-parties anyway? Did you verify each one of them?
jncfhnb 2y ago
> if you want features that require larger-than-disk datasets, or frequently changing content, you need tools like this. Well I want them to fuck off. Hidden in your commentary here is the fact that the vector representation of the image is _the contents of the image_. It very well may be that they cannot reverse the exact image. But it’s still a representation of the image that has to be good for something. Without being too familiar I would be willing to hazard a guess that this could include textual labels and classifications of what is in the image. I don’t give a shit how good your internal controls are. I don’t have anything particularly interesting to hide. I still do not want you taking my pictures.
- scratchyone 2y ago
  If you read the research you'd know that they don't have access to the vector either. They never decrypt the data. All operations on their server are done directly on the encrypted data. They get 0 information about your photos. They cannot even see which landmark your vector was closest to.
  - jncfhnb 2y ago
    I don’t care! I do not want them helping themselves to representations of my data. I don’t care if it’s encrypted or run through a one way hash. I don’t care if they only interact with it via homomorphic methods. They can, again, fuck the hell off. A private corporation has no business reading my files for its own benefit.
    - scratchyone 2y ago
      > A private corporation has no business reading my files for its own benefit. that's what i'm saying!! because of all these special privacy measures it's genuinely impossible for them to get any benefit from this. it is only used for your benefit to improve photo search on your local device. they don't see any of the data, they simply perform computations blindly over the encrypted data that are then returned to your device to then decrypt the results locally and use them to improve local photo indexing for you.
griomnib 2y ago
I’m deeply familiar with all of these techniques, the core issue here is informed consent which they have not obtained. Furthermore, Apples privacy stance is generally a sham as their definition of “human rights” doesn’t extend to China. Which either means Apple doesn’t respect human rights, or they don’t view Chinese people as human.
- slashdave 2y ago
  Apple follows the law. First you need to get the Chinese government to respect those rights. The only other choice is to stop doing business entirely in the country.
  - griomnib 2y ago
    A choice many companies have made. Apple is in China to make money, which is what a corporation is set up to do. My point is them claiming the moral high ground of a human rights defender is utterly laughable bullshit.
- iammattmurphy 2y ago
  That's not really fair; Apple's in a sticky wicket when it comes to the Chinese government, and they're not the only ones. The Chinese government are debatably inhuman. They've literally censored the word "censorship." (Then they censored what people used euphemistically for censorship--"harmonious.") It's funny from the outside but also a miserable state of affairs in 2024.
  - griomnib 2y ago
    It’s very fair, Apple has historically been very happy to be the sponsor of horrible human rights violations in their supply chain, only marginally paying attention to suicides in their factories when the PR got too bad. Apples version of “human right” includes suicide nets as an alternative to treating people humanely. That’s why their stance is pure marketing - they have blood on their hands. And guess what? You can’t use Google in China, and while Google isn’t by any means perfect, they aren’t Apple.
thefounder 2y ago
The best you can hope is integrity and security until your information reaches the destination but to assume that Apple or the U.S government cannot decipher the information you sent it or use it against you(i.e. set a person of interest as "landmark" and find out who's iPhone matches that "landmark) you must be foolish. It's no longer a conspiracy. I think we are all over past that time(i.e with Snowden and Wikileaks). We live in a surveillance world and "They're guarding all the doors and holding all the keys".
StanislavPetrov 2y ago
>There are other tools to provide privacy (DP, homomorphic encryption), while also using services. They are immensely complicated, and user's can't realistically evaluate risk. It is simple for any user to evaluate risk the risk of their data being breached on 3rd party servers when their data isn't being sent off the device - there is none. It is only when corporations insist that they are going to send the data off your device whether you like it or not that evaluating risk becomes necessary.
throwaway81523 2y ago
> This is what a good privacy story looks like. A good privacy story actually looks like not sending any info to anyone else anywhere at any time.
- bobsomers 2y ago
  Sure, but if we follow that line of thinking to its logical conclusion, we must move to a cabin in the woods, 100 miles from the nearest civilization, growing our own food and never connecting our computing devices to anything resembling a network.
  - ketralnis 2y ago
    No? You can have a photos app that doesn't phone home while not having to move to a cabin in the woods. See: every photos app that doesn't phone home, and I currently don't live in a cabin in the woods.
  - eptcyka 2y ago
    It is a spectrum, and I'd love to be able to draw my own line. I have my own photo storage solution. I do not need Apple's.
    - dialup_sounds 2y ago
      This entire post is about a setting in Apple's Photos app.
  - codys 2y ago
    It's not charitable to construct a contrived situation no one is talking about and place that into the mouth of a commenter.
  - nimih 2y ago
    I've read the post you're responding to like 3 times, and after pondering it deeply, I'm pretty sure the conclusion of their line of thinking pretty definitively stops at "Apple should not be sending data off the device without the user requesting it." If you think otherwise, you should maybe provide more of an argument.
hahahacorn 2y ago
Thank you for this comment. I found the author's ignorance to be fairly discrediting, and was surprised to find so many follow up comments equally railing on Apple. Between the quote you pointed out and: "One thing I do know, however, is that Apple computers are constantly full of privacy and security vulnerabilities, as proved by Apple's own security release notes" which just reeks of survivorship bias. I think the final call of what is right here _shouldn't_ be informed by the linked article. IMO, enabled by default without opt-in is absolutely the right call when judging between 1: Feature value 2: Security risk 3: Consent Fatigue. If you're data-conscious enough to disagree with my prior statement, you should consider having lockdown mode enabled. If you disagree with my prior statement because of how Apple locks you into Photos, :shake_hands:. If Enhanced Visual Search is still enabled by default in lockdown mode, then I think that's worth a conversation.
- pama 2y ago
  Enhanced Visual Search was enabled despite my default lockdown mode. I worry about enhanced visual search capabilities much less than several of the other risky features that lockdown mode disables, but was a bit surprised by the default opt-in in my lockdown mode phone.
  - hahahacorn 2y ago
    Yeah that’s genuinely surprising and I disagree with that decision.
- lapcat 2y ago
  > I found the author's ignorance to be fairly discrediting Why in the world am I supposed to be an expert on homomorphic encryption? How many people in the world are experts on homomorphic encryption? > which just reeks of survivorship bias. What does that even mean in this context? > 1: Feature value What is the value of the feature? As the article notes, this new feature is flying so low under the radar that Apple hasn't bothered to advertise it, and the Apple media haven't bothered to mention it either. You have to wonder how many people even wanted it. > If you're data-conscious enough to disagree with my prior statement, you should consider having lockdown mode enabled. That's ridiculous. Apple itself has said, "Lockdown Mode is an optional, extreme protection that’s designed for the very few individuals who, because of who they are or what they do, might be personally targeted by some of the most sophisticated digital threats. Most people are never targeted by attacks of this nature." https://support.apple.com/105120 https://support.apple.com/105120 Lockdown mode is basically for famous people and nobody else.
amiga386 2y ago
> This is what a good privacy story looks like. A good privacy story starts with "Do you consent" and not transmitting a byte if you answer "no"
blindriver 2y ago
This sounds exactly like that CSAM "feature" they wanted to add but created a huge outrage because of how incredibly invasive it was. It sounds like it only needs a few extra lines of code to get exactly what they wanted before, they just packaged it differently and we all fell for it like frogs getting boiled in water.
- nntwozz 2y ago
  A frog that is gradually heated will actually jump out. https://en.m.wikipedia.org/wiki/Boiling_frog https://en.m.wikipedia.org/wiki/Boiling_frog
  - userbinator 2y ago
    Let's hope that part of the analogy also applies.
- theshrike79 2y ago
  Oh ffs, not this one again. The CSAM filtering was a best of class implementation. I'm pretty sure I'm one of maybe a dozen people who actually read the spec before throwing a hissy-fit about "muh privacy!" The only actual "flaw" was that maybe a state-level actor could make it scan for bad stuff on your device. BUT they can do it to your cloud data _today_. And if you disabled cloud uploads, the local scanning was disabled too.
  - 4ad 2y ago
    > BUT they can do it to your cloud data _today_. No, they can't if you enable ADP[1]. [1] https://support.apple.com/en-us/108756 https://support.apple.com/en-us/108756
schappim 2y ago
For context, @scosman is self-described as “Formerly Apple Photos” in his Twitter bio.
thousand_nights 2y ago
this is what gaslighting looks like how about they don't send anything about my photos to their servers and i get to keep my shit on my own device i suppose we're past that to the point where techbros like you will defend personal data exfiltration because.. uhh idk? trillion dollar corporation knows best?
matheusmoreira 2y ago
> I'm not sure what the right call is here. I am sure. The right call is to never send any data from the device to anyone unless the user explicitly tells the device to do it. The only thing the device should do is whatever its user tells it to do. The user didn't tell it to do this. Apple did. > But the conclusion "Thus, the only way to guarantee computing privacy is to not send data off the device." isn't true Irrelevant. It was never about privacy to begin with. It was always about power, who owns the keys to the machine, who commands it. Vectorization, differential privacy, relays, homomorphic encryption, none of it matters. What matters is the device is going behind the user's back, doing somebody else's bidding, protecting somebody else's interests. That they were careful about it offers little comfort to users who are now aware of the fact "their" devices are doing things they weren't supposed to be doing.
- parasubvert 2y ago
  Complete nonsense. *All networked devices do things behind their users back* at this point, and have for years, and do not ask for consent for most of it. And users would REJECT granular opt-in as a terrible UX. Let's look at the primary alternative, Android. It generally does not provide you this level of granular control on network access either without rooted hacks. Apps and the phone vendor can do whatever they want with far less user control unless you're a deep Android nerd and know how to install root-level restriction software.
  - matheusmoreira 2y ago
    Yes, apps going behind people's back and exfiltrating personal information has become normal. That's not an argument, it's merely a statement of fact. This shouldn't be happening at all. The fact it got to this point doesn't imply it shouldn't be stopped. No one's advocating for granular opt in either. There are much better ways. We have to make it so that data is toxic to corporations. Turn data into expensive legal liabilities they don't want to deal with. These corporations should not even be thinking about it. They should be scrambling to forget all about us the second we are done with them, not covertly collecting all the data they possibly can for "legitimate" purposes. People should be able to use their computers without ever worrying that corporations are exploiting them in any way whatsoever. The Android situation is just as bad, by the way. Rooting is completely irrelevant. You may think you can hack it but if you actually do it the phone fails remote attestation and the corporations discriminate against you based on that, usually by straight up denying you service. On a very small number of devices, Google's phones ironically, you can access those keys and even set your own. And it doesn't matter, because the corporations don't trust your keys, they only trust the keys of other corporations. They don't care to know your device is secure, they want to know it's fully owned by Google so that you can't do things the corporations don't like. It's not something that can be solved with technology. Computer freedom needs to become law.
user3939382 2y ago
> This is what a good privacy story looks like. What a good privacy story looks like is that my photos aren’t sent anywhere in any way shape or form without explicit opt in permission.
- notyourwork 2y ago
  Do you not sync to iCloud?
  - ternnoburn 2y ago
    That sounds like "with opt in permission".
    - EA-3167 2y ago
      You aren't wrong, but... it's odd coming here to HN and seeing people talk about privacy like we aren't in the nth generation of people trading theirs away for a pittance. I think the market for the sort of privacy envisioned by some here is incredibly small, incredibly niche, and honestly one of the least likely to buy an iPhone in the first place. Most people broadcast their lives on social media, happily opt in to all sorts of schemes that track them just for minor conveniences. For people like that, the idea that the privacy protection outlined by the OP isn't enough rings really hollow. Or to put it bluntly, at some point this really stops feeling like a practical debate, and more of an ideological one.
  - chatmasta 2y ago
    You can enable Advanced Data Protection and all your iCloud data will be stored as encrypted blobs.
    - sneak 2y ago
      ADP sends hashes of the plaintext and filenames to Apple effectively in the clear (non e2ee). If only you and three other people have a unique file, Apple knows you are a group.
  - 2y ago
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]
pton_xd 2y ago
> This is what a good privacy story looks like. I have an idea: send an encrypted, relayed, non-reversible, noised vector representation of your daily phone habits and interactions. That way you can be bucketed, completely anonymously of course, with other user cohorts for tracking, advertising, and other yet-to-be discovered purposes. It's a great privacy story! Why would you have a problem with that?
- 1123581321 2y ago
  What would be the value to the user in your scenario? In the photos app real scenario, it’s to enable a search feature that requires pairing photos with data not on the phone. (I understand you’re being sarcastic.)
  - pton_xd 2y ago
    Maybe we can do some analysis and optimize phone battery life based on your cohorts usage patterns. I don't know, I'm sure we'll figure something out once we have your data!
    - giancarlostoro 2y ago
      The entire point is that you don't actually have the data, only the client can decrypt any of it.
      - deleted 2y ago
        [deleted]
    - 1123581321 2y ago
      That doesn’t make sense, and the other user is right that you can’t give up personal data with this scheme. Perhaps focus on the real privacy leaks from cell phones like tower connections and sign-ins to Instagram.
- jrk 2y ago
  They don't "have your data," even at an aggregated and noised level, due to the homomorphic encryption part. Restating the layers above, in reverse: - They don't see either your data or the results of the query (it's fully encrypted even from them where they compute the query -- this is what homomorphic encryption means) - Even if they broke the encryption and had your query data / the query result, they don't know who "you" are (the relay part) - Even if they had your query hash and your identity, they couldn't reverse the hash to identify which specific photos you have in your library (the client-side vectorization + differential privacy part), though by the this point they could know what records in the places database were hits. So they could know that you took a photo of a landmark, but only if the encryption and relay were both broken.
matthewdgreen 2y ago
The devil is in the differential privacy budget. I am in Japan and I’ve taken hundreds of photos this week. What does that budget cover?
waynecochran 2y ago
Can this be verified?
SoftTalker 2y ago
It’s a nice story but Apple can change the implementation any time they want or are forced to.
themaninthedark 2y ago
I am bit bit confused: Data is being sent to Apple, in such a way that it can not be traced back to the user. Apple does some processing on it. Then somehow magically, the pictures on your phone are updated with tags based on Apple's processing....but Apple doesn't know who you are.....
- kalleboo 2y ago
  There is a way to perform processing on encrypted data so the result is also encrypted and the person doing the processing never knows anything about the data that was processed on or the result (which can only be decrypted by the user with the original encryption keys) https://en.wikipedia.org/wiki/Homomorphic_encryption https://en.wikipedia.org/wiki/Homomorphic_encryption > Homomorphic encryption is a form of encryption that allows computations to be performed on encrypted data without first having to decrypt it. The resulting computations are left in an encrypted form which, when decrypted, result in an output that is identical to that produced had the operations been performed on the unencrypted data. Homomorphic encryption can be used for privacy-preserving outsourced storage and computation. This allows data to be encrypted and outsourced to commercial cloud environments for processing, all while encrypted And the way the data comes back to you is via the third-party relay which knows your IP but nothing else
  - themaninthedark 2y ago
    Ok, that's the step that was missing. I couldn't figure out how there was a benefit to the users without data being fed back and data can't be fed back without knowing some ID. So, while Apple doesn't know the ID of the person sending the data, they have a 'room number' that links back to an ID. If Apple were to decide to scan photos for pictures of 'lines of white powder' they couldn't tell the police your name but they could say that the 3rd party knows who you are.
- deleted 2y ago
  [deleted]
- raincole 2y ago
  You joked, but you accidentally described what homomorphic encryption does. (if implemented correctly) > Then somehow magically, the pictures on your phone are updated with tags based on Apple's processing....but Apple doesn't know who you are..... Yes, this is the whole point.
dannyw 2y ago
Isn’t this 95% the same as Apple’s CSAM scanning paper?

AJRF 2y ago

> This is what a good privacy story looks like. Not at all. A good privacy story is not sending this data anywhere.

chikere232 2y ago

The nearest neighbour search is sharded, which apple's blog admits is a privacy issue, which is why they're running the DP and OHTTP parts. If apple were to add additional clusters that match "sensitive" content and endeavour to put them in their own shards distinct from landmarks, they defeat the homomorphic encryption part while still technically doing it. The DP part can be defeated with just statistics over time; someone with any volume of sensitive content will hit these sensitive clusters with a higher likelihood than someone generateing noise injected fake searches. The OHTTP part can be defeated in several ways, the simplest of which is just having a clause in a non-public contract allowing apple to request logs for some purpose. They're paying them and they can make up the rules as they go.

geoffmac 2y ago

This answer should be much higher. Thank you

Gabriel54 2y ago

I appreciate the explanation. However, I think you do not address the main problem, which is that my data is being sent off my device by default and without any (reasonable) notice. Many users may agree to such a feature (as you say, it may be very secure), but to assume that everyone ought to be opted in by default is the issue.

sgammon 2y ago
"Your data" is not actually being sent off your device, actually, it is being scrambled into completely unusable form for anyone except you. This is a much greater level of security than what you would expect from a bank, for example, who needs to fully decrypt the data you send it. When using your banking apps over HTTPS (TLS), you are trusting the CA infrastructure, you are trusting all sorts of things. You have fewer points of failure when a key for homomorphic encryption resides only on your device. "Opting-in by default" is therefore not unsafe.
jeffybefffy519 2y ago
The big mistake here is ownership of your apple devices is an illusion...
JustExAWS 2y ago
Do you use iCloud to store your photos?
- latexr 2y ago
  I’m not the person you asked, but I agree with them. To answer your question: No, I do not use iCloud to store my photos. Even if I did, consent to store data is not the same as consent to scan or run checks on it. For a company whose messaging is all about user consent and privacy, that matters. This would be easily solvable: On first run show a window with: > Hey, we have this new cool feature that does X and is totally private because of Y [link to Learn More] > Do you want to turn it on? You can change your mind later in Settings > [Yes] [No]
  - 2y ago

talldayo 2y ago

The right call is to provide the feature and let users opt-in. Apple knows this is bad, they've directly witnessed the backlash to OCSP, lawful intercept and client-side-scanning. There is no world in which they did not realize the problem and decided to enable it by default anyways knowing full-well that users aren't comfortable with this. People won't trust homomorphic encryption, entropy seeding or relaying when none of it is transparent and all of it is enabled in an OTA update. > This is what a good privacy story looks like. This is what a coverup looks like. Good privacy stories never force third-party services on a user, period. When you see that many puppets on stage in one security theater, it's only natural for things to feel off.

oneplane 2y ago
It's not that binary. Nobody is forcing anything, you can not buy a phone, you can not use the internet. Heck, you can even not install any updates! What is happening, is that people make tradeoffs, and decide to what degree they trust who and what they interact with. Plenty of people might just 'go with the flow', but putting what Apple did here in the same bucket as what for example Microsoft or Google does is a gross misrepresentation. Present it all as equals just kills the discussion, and doesn't inform anyone to a better degree. When you want to take part in an interconnected network, you cannot do that on your own, and you will have to trust other parties to some degree. This includes things that might 'feel' like you can judge them (like your browser used to access HN right here), but you actually can't unless you understand the entire codebase of your OS and Browser, all the firmware on the I/O paths, and the silicon it all runs on. So you make a choice, which as you are reading this, is apparently that you trust this entire chain enough to take part in it. It would be reasonable to make this optional (as in, opt-in), but the problem is that you end up asking a user for a ton of "do you want this" questions, almost every upgrade and install cycle, which is not what they want (we have had this since Mavericks and Vista, people were not happy). So if you can engineer a feature to be as privacy-centric yet automated as possible, it's a win for everyone.
- talldayo 2y ago
  > What is happening, is that people make tradeoffs, and decide to what degree they trust who and what they interact with. People aren't making tradeoffs - that's the problem. Apple is making the tradeoffs for them, and then retroactively asking their users "is this okay?" Users shouldn't need to buy a new phone to circumevent arbitrary restrictions on the hardware that is their legal property. If America had functional consumer protections, Apple would have been reprimanded harder than their smackdowns in the EU.

cma 2y ago

Quantum makes the homomorphic stuff ineffective in the mid-term. All they have to do is hold on to the data and they can get the results of the lookup table computation, in maybe 10-25 years. Shouldn't be on by default.

oneplane 2y ago
What makes you think that this is the biggest problem if things like AES and RSA are suddenly breakable? If someone wanted to get a hold of your cloud hosted data at that point, they would use their capacity to simply extract enough key material to impersonate a Secure Enclave. That that point, you "are" the device and as such you "are" the user. No need to make it more complicated than that. In theory, Apple and other manufacturers would already use PQC to prevent such scenarios. Then again, QC has been "coming soon" for so long, it's doubtful that any information that is currently protected by encryption will still be valuable by the time it can be cracked. Most real-world process implementations don't rely on some "infinite insurance", but assume it will be breached at some point and just try to make it difficult or costly enough to run out the clock on confidentiality, which is all that really matters. Nothing that exists really needs to be confidential forever. Things either get lost/destroyed or become irrelevant.
- cma 2y ago
  This is ostensibly for non-cloud data, derivatives of it auto uploaded after an update.

latexr 2y ago

> The author themselves looks to be an Apple security researcher They’re not. Jeff Johnson develops apps (specifically Safari extensions) for Apple platforms and frequently blogs about their annoyances with Apple, but they’re not a security researcher.

gigel82 2y ago

This may be a "good" privacy story but a way better one is to just not send any of your data anywhere, especially without prior consent.

jazzyjackson 2y ago

This must be the first consumer or commercial product implementing homomorphic encryption is it not? I would be surprised if doing noisy vector comparisons is actually the most effective way to tell if someone is in front of the Eiffel tower. A small large language model could caption it just as well on device, my spider sense tells me someone saw an opportunity to apply bleeding edge, very cool tech so that they can gain experience and do it bigger and better in the future, but they're fumbling their reputation by doing this kind of user data scanning.

Klonoar 2y ago
Apple themselves have already used it in the past (Caller ID)
do_not_redeem 2y ago
> This must be the first consumer or commercial product implementing homomorphic encryption is it not? Not really, it's been around for a bit now. From 2021: > The other major reason we’re talking about HE and FL now is who is using them. According to a recent repository of PETs, there are 19 publicly announced pilots, products, and proofs of concept for homomorphic encryption and federated analytics (another term for federated learning) combined. That doesn’t seem like a lot … but the companies offering them include Apple,7 Google, Microsoft, Nvidia, IBM, and the National Health Service in the United Kingdom, and users and investors include DARPA, Intel, Oracle, Mastercard, and Scotiabank. Also, the industries involved in these early projects are among the largest. Use cases are led by health and social care and finance, with their use in digital and crime and justice also nontrivial (figure 1). https://www2.deloitte.com/us/en/insights/industry/technology/technology-media-and-telecom-predictions/2022/homomorphic-encryption-federated-learning.html https://www2.deloitte.com/us/en/insights/industry/technology... I do wonder why we don't hear about it more often though. "Homomorphic encryption" as a buzzword has a lot of headline potential, so I'm surprised companies don't brag about it more.
- fragmede 2y ago
  But what are the products from them that implement HE and that consumers are using? Microsoft, IBM, Intel, and Google have all released libraries for HE, and there's Duality SecurePlu, but as far as actual consumer products, Apple's caller ID phone number lookup and other features in iOS 18 is very possibly the first. As far as why it's not more of a buzzword, it's far too in the weeds and ultimately consumers either trust you or they don't. And even if they don't trust you, many of them are still going to use Apple/Google/Facebook system anyway.

lysace 2y ago

> - OHTTP relay: it's sent through a 3rd party so Apple never knows your IP address. The contents are encrypted so the 3rd party never doesn't learn anything either (some risk of exposing "IP X is an apple photos user", but nothing about the content of the library). Which 3rd party is that?

gigel82 2y ago
The NSA, the CCP, etc. depending on jurisdiction. (joking, but not really)
oneplane 2y ago
I don't have a list on hand, but at least Cloudflare and Akamai are part of the network hops. Technically you only need 2 hops to make sure no origin or data extraction can be done.
- jazzyjackson 2y ago
  O good, cloudflare gets one more data point on me, a ping every time I add a photo to my library.
  - threeseed 2y ago
    a) Cloudflare doesn't know about you. It sees an IP address. b) If we follow your tortured logic then every hop along the path from your phone to Apple will have one more data point on you. That's thousands of companies a day.
    - jazzyjackson 2y ago
      I'm just griping that cloudflare has many eyeballs and sees most of the traffic on the internet at this point. How many websites have I used with Cloudflare DDoS protection that checks if I'm a bot by fingerprinting my device? They know plenty about me. I'm also griping that "the data is encrypted !" is not a good enough excuse seeing as how we've known for years that the metadata is a bigger pot of gold for intelligence agencies. That my mac address is taking a photo and hitting a particular cell tower is pretty detailed information, even without knowing the content of the photo.

homofermic 2y ago

Regarding HE: since the lookup is generated by the requestor, it can be used as an adversarial vector, which can result in exfiltration by nearest neighbor (closest point to vector) methods. In other words, you can change what you are searching for, and much like differential power analysis attacks on crypto, extract information.

brookst 2y ago
Does the noise addition not help? Is that a mitigation for a different attack?

ustad 2y ago

You're presenting a false dichotomy between "perfect user understanding" and "no user choice." The issue isn't whether users can fully comprehend homomorphic encryption or differential privacy – it's about basic consent and transparency. Consider these points: 1. Users don't need a PhD to understand "This feature will send data about your photos to Apple's servers to enable better search." 2. The complexity of the privacy protections doesn't justify removing user choice. By that logic, we should never ask users about any technical feature. 3. Many privacy-conscious users follow a simple principle: they want control over what leaves their device, regardless of how it's protected. The "it's too complex to explain" argument could justify any privacy-invasive default. Would you apply the same logic to, say, enabling location services by default because explaining GPS technology is too complex? The real solution is simple: explain the feature in plain language, highlight the benefits, outline the privacy protections, and let users make their own choice. Apple already does this for many other features. "Default off with opt-in" is a core principle of privacy-respecting design, regardless of how robust the underlying protections are.

scosman 2y ago
I don't believe I said or implied that anywhere: 'You're presenting a false dichotomy between "perfect user understanding" and "no user choice."'? Happy to be corrected if wrong. Closest I come to presenting an opinion on the right way UX was "I'm not sure what the right call is here.". The thing I disagreed with was a technical statement "the only way to guarantee computing privacy is to not send data off the device.". Privacy respecting design and tech is a passion of mine. I'm pointing out "user choice" gets hard as the techniques used for privacy exceed the understanding of users. Users can intuitively understand "send my location to Google [once/always]" without understanding GPS satellites. User's can't understand the difference between "send my photo" and "send homomorphicly encrypted locally differentially private vector of e=0.8" and "send differentially private vector of e=50". Your prompt "send data about your photos..." would allow for much less private designs than this. If we want to move beyond "ask the user then do it", we need to get into the nitty gritty details here. I'd love to see more tech like this in consumer products, where it's private when used, even when opted-in.
- 2y ago

deleted 2y ago

[deleted]

yapyap 2y ago

> "It ought to be up to the individual user to decide their own tolerance for the risk of privacy violations." -> The author themselves looks to be an Apple security researcher, and are saying they can't make an informed choice here I don’t think that that’s what the author is saying at all, I think he’s saying that Apple should let the user decide for themself if they want to send all this shit to Apple, freedom for the individual. They’re not saying “I dunno”

omolobo 2y ago

So what? Why should the application talk over the Internet to begin with? And why isn't that functionality off by default under a settings option that clearly warns the user of the consequences? I think you're missing the forest for the trees here. And the claims that this is good privacy/security are not at all obvious either. And who are those third-parties anyway? Did you verify each one of them?

jncfhnb 2y ago

> if you want features that require larger-than-disk datasets, or frequently changing content, you need tools like this. Well I want them to fuck off. Hidden in your commentary here is the fact that the vector representation of the image is _the contents of the image_. It very well may be that they cannot reverse the exact image. But it’s still a representation of the image that has to be good for something. Without being too familiar I would be willing to hazard a guess that this could include textual labels and classifications of what is in the image. I don’t give a shit how good your internal controls are. I don’t have anything particularly interesting to hide. I still do not want you taking my pictures.

scratchyone 2y ago
If you read the research you'd know that they don't have access to the vector either. They never decrypt the data. All operations on their server are done directly on the encrypted data. They get 0 information about your photos. They cannot even see which landmark your vector was closest to.
- jncfhnb 2y ago
  I don’t care! I do not want them helping themselves to representations of my data. I don’t care if it’s encrypted or run through a one way hash. I don’t care if they only interact with it via homomorphic methods. They can, again, fuck the hell off. A private corporation has no business reading my files for its own benefit.
  - scratchyone 2y ago
    > A private corporation has no business reading my files for its own benefit. that's what i'm saying!! because of all these special privacy measures it's genuinely impossible for them to get any benefit from this. it is only used for your benefit to improve photo search on your local device. they don't see any of the data, they simply perform computations blindly over the encrypted data that are then returned to your device to then decrypt the results locally and use them to improve local photo indexing for you.

griomnib 2y ago

I’m deeply familiar with all of these techniques, the core issue here is informed consent which they have not obtained. Furthermore, Apples privacy stance is generally a sham as their definition of “human rights” doesn’t extend to China. Which either means Apple doesn’t respect human rights, or they don’t view Chinese people as human.

slashdave 2y ago
Apple follows the law. First you need to get the Chinese government to respect those rights. The only other choice is to stop doing business entirely in the country.
- griomnib 2y ago
  A choice many companies have made. Apple is in China to make money, which is what a corporation is set up to do. My point is them claiming the moral high ground of a human rights defender is utterly laughable bullshit.
iammattmurphy 2y ago
That's not really fair; Apple's in a sticky wicket when it comes to the Chinese government, and they're not the only ones. The Chinese government are debatably inhuman. They've literally censored the word "censorship." (Then they censored what people used euphemistically for censorship--"harmonious.") It's funny from the outside but also a miserable state of affairs in 2024.
- griomnib 2y ago
  It’s very fair, Apple has historically been very happy to be the sponsor of horrible human rights violations in their supply chain, only marginally paying attention to suicides in their factories when the PR got too bad. Apples version of “human right” includes suicide nets as an alternative to treating people humanely. That’s why their stance is pure marketing - they have blood on their hands. And guess what? You can’t use Google in China, and while Google isn’t by any means perfect, they aren’t Apple.

thefounder 2y ago

The best you can hope is integrity and security until your information reaches the destination but to assume that Apple or the U.S government cannot decipher the information you sent it or use it against you(i.e. set a person of interest as "landmark" and find out who's iPhone matches that "landmark) you must be foolish. It's no longer a conspiracy. I think we are all over past that time(i.e with Snowden and Wikileaks). We live in a surveillance world and "They're guarding all the doors and holding all the keys".

StanislavPetrov 2y ago

>There are other tools to provide privacy (DP, homomorphic encryption), while also using services. They are immensely complicated, and user's can't realistically evaluate risk. It is simple for any user to evaluate risk the risk of their data being breached on 3rd party servers when their data isn't being sent off the device - there is none. It is only when corporations insist that they are going to send the data off your device whether you like it or not that evaluating risk becomes necessary.

throwaway81523 2y ago

> This is what a good privacy story looks like. A good privacy story actually looks like not sending any info to anyone else anywhere at any time.

bobsomers 2y ago
Sure, but if we follow that line of thinking to its logical conclusion, we must move to a cabin in the woods, 100 miles from the nearest civilization, growing our own food and never connecting our computing devices to anything resembling a network.
- ketralnis 2y ago
  No? You can have a photos app that doesn't phone home while not having to move to a cabin in the woods. See: every photos app that doesn't phone home, and I currently don't live in a cabin in the woods.
- eptcyka 2y ago
  It is a spectrum, and I'd love to be able to draw my own line. I have my own photo storage solution. I do not need Apple's.
  - dialup_sounds 2y ago
    This entire post is about a setting in Apple's Photos app.
- codys 2y ago
  It's not charitable to construct a contrived situation no one is talking about and place that into the mouth of a commenter.
- nimih 2y ago
  I've read the post you're responding to like 3 times, and after pondering it deeply, I'm pretty sure the conclusion of their line of thinking pretty definitively stops at "Apple should not be sending data off the device without the user requesting it." If you think otherwise, you should maybe provide more of an argument.

hahahacorn 2y ago

Thank you for this comment. I found the author's ignorance to be fairly discrediting, and was surprised to find so many follow up comments equally railing on Apple. Between the quote you pointed out and: "One thing I do know, however, is that Apple computers are constantly full of privacy and security vulnerabilities, as proved by Apple's own security release notes" which just reeks of survivorship bias. I think the final call of what is right here _shouldn't_ be informed by the linked article. IMO, enabled by default without opt-in is absolutely the right call when judging between 1: Feature value 2: Security risk 3: Consent Fatigue. If you're data-conscious enough to disagree with my prior statement, you should consider having lockdown mode enabled. If you disagree with my prior statement because of how Apple locks you into Photos, :shake_hands:. If Enhanced Visual Search is still enabled by default in lockdown mode, then I think that's worth a conversation.

pama 2y ago
Enhanced Visual Search was enabled despite my default lockdown mode. I worry about enhanced visual search capabilities much less than several of the other risky features that lockdown mode disables, but was a bit surprised by the default opt-in in my lockdown mode phone.
- hahahacorn 2y ago
  Yeah that’s genuinely surprising and I disagree with that decision.
lapcat 2y ago
> I found the author's ignorance to be fairly discrediting Why in the world am I supposed to be an expert on homomorphic encryption? How many people in the world are experts on homomorphic encryption? > which just reeks of survivorship bias. What does that even mean in this context? > 1: Feature value What is the value of the feature? As the article notes, this new feature is flying so low under the radar that Apple hasn't bothered to advertise it, and the Apple media haven't bothered to mention it either. You have to wonder how many people even wanted it. > If you're data-conscious enough to disagree with my prior statement, you should consider having lockdown mode enabled. That's ridiculous. Apple itself has said, "Lockdown Mode is an optional, extreme protection that’s designed for the very few individuals who, because of who they are or what they do, might be personally targeted by some of the most sophisticated digital threats. Most people are never targeted by attacks of this nature." https://support.apple.com/105120 https://support.apple.com/105120 Lockdown mode is basically for famous people and nobody else.

amiga386 2y ago

> This is what a good privacy story looks like. A good privacy story starts with "Do you consent" and not transmitting a byte if you answer "no"

blindriver 2y ago

This sounds exactly like that CSAM "feature" they wanted to add but created a huge outrage because of how incredibly invasive it was. It sounds like it only needs a few extra lines of code to get exactly what they wanted before, they just packaged it differently and we all fell for it like frogs getting boiled in water.

nntwozz 2y ago
A frog that is gradually heated will actually jump out. https://en.m.wikipedia.org/wiki/Boiling_frog https://en.m.wikipedia.org/wiki/Boiling_frog
- userbinator 2y ago
  Let's hope that part of the analogy also applies.
theshrike79 2y ago
Oh ffs, not this one again. The CSAM filtering was a best of class implementation. I'm pretty sure I'm one of maybe a dozen people who actually read the spec before throwing a hissy-fit about "muh privacy!" The only actual "flaw" was that maybe a state-level actor could make it scan for bad stuff on your device. BUT they can do it to your cloud data _today_. And if you disabled cloud uploads, the local scanning was disabled too.
- 4ad 2y ago
  > BUT they can do it to your cloud data _today_. No, they can't if you enable ADP[1]. [1] https://support.apple.com/en-us/108756 https://support.apple.com/en-us/108756

schappim 2y ago

For context, @scosman is self-described as “Formerly Apple Photos” in his Twitter bio.

thousand_nights 2y ago

this is what gaslighting looks like how about they don't send anything about my photos to their servers and i get to keep my shit on my own device i suppose we're past that to the point where techbros like you will defend personal data exfiltration because.. uhh idk? trillion dollar corporation knows best?

matheusmoreira 2y ago

> I'm not sure what the right call is here. I am sure. The right call is to never send any data from the device to anyone unless the user explicitly tells the device to do it. The only thing the device should do is whatever its user tells it to do. The user didn't tell it to do this. Apple did. > But the conclusion "Thus, the only way to guarantee computing privacy is to not send data off the device." isn't true Irrelevant. It was never about privacy to begin with. It was always about power, who owns the keys to the machine, who commands it. Vectorization, differential privacy, relays, homomorphic encryption, none of it matters. What matters is the device is going behind the user's back, doing somebody else's bidding, protecting somebody else's interests. That they were careful about it offers little comfort to users who are now aware of the fact "their" devices are doing things they weren't supposed to be doing.

parasubvert 2y ago
Complete nonsense. *All networked devices do things behind their users back* at this point, and have for years, and do not ask for consent for most of it. And users would REJECT granular opt-in as a terrible UX. Let's look at the primary alternative, Android. It generally does not provide you this level of granular control on network access either without rooted hacks. Apps and the phone vendor can do whatever they want with far less user control unless you're a deep Android nerd and know how to install root-level restriction software.
- matheusmoreira 2y ago
  Yes, apps going behind people's back and exfiltrating personal information has become normal. That's not an argument, it's merely a statement of fact. This shouldn't be happening at all. The fact it got to this point doesn't imply it shouldn't be stopped. No one's advocating for granular opt in either. There are much better ways. We have to make it so that data is toxic to corporations. Turn data into expensive legal liabilities they don't want to deal with. These corporations should not even be thinking about it. They should be scrambling to forget all about us the second we are done with them, not covertly collecting all the data they possibly can for "legitimate" purposes. People should be able to use their computers without ever worrying that corporations are exploiting them in any way whatsoever. The Android situation is just as bad, by the way. Rooting is completely irrelevant. You may think you can hack it but if you actually do it the phone fails remote attestation and the corporations discriminate against you based on that, usually by straight up denying you service. On a very small number of devices, Google's phones ironically, you can access those keys and even set your own. And it doesn't matter, because the corporations don't trust your keys, they only trust the keys of other corporations. They don't care to know your device is secure, they want to know it's fully owned by Google so that you can't do things the corporations don't like. It's not something that can be solved with technology. Computer freedom needs to become law.

user3939382 2y ago

> This is what a good privacy story looks like. What a good privacy story looks like is that my photos aren’t sent anywhere in any way shape or form without explicit opt in permission.

notyourwork 2y ago
Do you not sync to iCloud?
- ternnoburn 2y ago
  That sounds like "with opt in permission".
  - EA-3167 2y ago
    You aren't wrong, but... it's odd coming here to HN and seeing people talk about privacy like we aren't in the nth generation of people trading theirs away for a pittance. I think the market for the sort of privacy envisioned by some here is incredibly small, incredibly niche, and honestly one of the least likely to buy an iPhone in the first place. Most people broadcast their lives on social media, happily opt in to all sorts of schemes that track them just for minor conveniences. For people like that, the idea that the privacy protection outlined by the OP isn't enough rings really hollow. Or to put it bluntly, at some point this really stops feeling like a practical debate, and more of an ideological one.
- chatmasta 2y ago
  You can enable Advanced Data Protection and all your iCloud data will be stored as encrypted blobs.
  - sneak 2y ago
    ADP sends hashes of the plaintext and filenames to Apple effectively in the clear (non e2ee). If only you and three other people have a unique file, Apple knows you are a group.
- 2y ago

deleted 2y ago

[deleted]

deleted 2y ago

[deleted]

pton_xd 2y ago

> This is what a good privacy story looks like. I have an idea: send an encrypted, relayed, non-reversible, noised vector representation of your daily phone habits and interactions. That way you can be bucketed, completely anonymously of course, with other user cohorts for tracking, advertising, and other yet-to-be discovered purposes. It's a great privacy story! Why would you have a problem with that?

1123581321 2y ago
What would be the value to the user in your scenario? In the photos app real scenario, it’s to enable a search feature that requires pairing photos with data not on the phone. (I understand you’re being sarcastic.)
- pton_xd 2y ago
  Maybe we can do some analysis and optimize phone battery life based on your cohorts usage patterns. I don't know, I'm sure we'll figure something out once we have your data!
  - giancarlostoro 2y ago
    The entire point is that you don't actually have the data, only the client can decrypt any of it.
    - deleted 2y ago
      [deleted]
  - 1123581321 2y ago
    That doesn’t make sense, and the other user is right that you can’t give up personal data with this scheme. Perhaps focus on the real privacy leaks from cell phones like tower connections and sign-ins to Instagram.
jrk 2y ago
They don't "have your data," even at an aggregated and noised level, due to the homomorphic encryption part. Restating the layers above, in reverse: - They don't see either your data or the results of the query (it's fully encrypted even from them where they compute the query -- this is what homomorphic encryption means) - Even if they broke the encryption and had your query data / the query result, they don't know who "you" are (the relay part) - Even if they had your query hash and your identity, they couldn't reverse the hash to identify which specific photos you have in your library (the client-side vectorization + differential privacy part), though by the this point they could know what records in the places database were hits. So they could know that you took a photo of a landmark, but only if the encryption and relay were both broken.

matthewdgreen 2y ago

The devil is in the differential privacy budget. I am in Japan and I’ve taken hundreds of photos this week. What does that budget cover?

waynecochran 2y ago

Can this be verified?

SoftTalker 2y ago

It’s a nice story but Apple can change the implementation any time they want or are forced to.

themaninthedark 2y ago

I am bit bit confused: Data is being sent to Apple, in such a way that it can not be traced back to the user. Apple does some processing on it. Then somehow magically, the pictures on your phone are updated with tags based on Apple's processing....but Apple doesn't know who you are.....

kalleboo 2y ago
There is a way to perform processing on encrypted data so the result is also encrypted and the person doing the processing never knows anything about the data that was processed on or the result (which can only be decrypted by the user with the original encryption keys) https://en.wikipedia.org/wiki/Homomorphic_encryption https://en.wikipedia.org/wiki/Homomorphic_encryption > Homomorphic encryption is a form of encryption that allows computations to be performed on encrypted data without first having to decrypt it. The resulting computations are left in an encrypted form which, when decrypted, result in an output that is identical to that produced had the operations been performed on the unencrypted data. Homomorphic encryption can be used for privacy-preserving outsourced storage and computation. This allows data to be encrypted and outsourced to commercial cloud environments for processing, all while encrypted And the way the data comes back to you is via the third-party relay which knows your IP but nothing else
- themaninthedark 2y ago
  Ok, that's the step that was missing. I couldn't figure out how there was a benefit to the users without data being fed back and data can't be fed back without knowing some ID. So, while Apple doesn't know the ID of the person sending the data, they have a 'room number' that links back to an ID. If Apple were to decide to scan photos for pictures of 'lines of white powder' they couldn't tell the police your name but they could say that the 3rd party knows who you are.
deleted 2y ago
[deleted]
raincole 2y ago
You joked, but you accidentally described what homomorphic encryption does. (if implemented correctly) > Then somehow magically, the pictures on your phone are updated with tags based on Apple's processing....but Apple doesn't know who you are..... Yes, this is the whole point.

dannyw 2y ago

Isn’t this 95% the same as Apple’s CSAM scanning paper?