9 ms·
What is Differential Privacy?
- hfsbtnye 10y agoI have to admit, I'm really starting to like the direction that Apple is heading despite being previously disenchanted. I only wish that they would go ahead and put everything under a free software license, since they're in the business of selling hardware that's coincidentally bundled with their software.
- applester 10y agoTo paint another viewpoint, Apple initially went all gung ho about privacy and wanted to make not collecting data a big play (and fairly so, full respect to them). The recent WWDC obviously shows a big shift towards AI and ML applications within the company. Some things are possible on the device, but many neural nets just cannot be served from an iPhone reasonably. Hence, the move towards more data collection. I really wish they give out more information here. Until then, I'm not sure how much they are actually collecting after their realization that they do need the data to do AI well.
- tjl 10y agoThey talked a bit more about differential privacy in the State of the Union. Basically, they hash the data and add noise. By collecting data from a bunch of people that noise gets averaged out. They also limit the amount of samples (over a relatively short period of time) they can get from a single person so they won't be able to identify them.
- xufi 10y agoIntersting. That's a smart way to collect data while not having too much noise flood in the dataset. I need to watch this State of the Union.
- jon-wood 10y agoState of the Union is what the WWDC keynote used to be before it started being watched by press and the public. Much more technical detail, and information in the underlying frameworks rather than user visible features.
- tjl 10y agohttps://developer.apple.com/videos/play/wwdc2016/102/ https://developer.apple.com/videos/play/wwdc2016/102/ You can find all the videos from WWDC 2016 some time after the session is done. I usually check the next day. They have the videos for several previous WWDCs up as well.
- quantumtremor 10y agoAt some point, regardless of adding noise, you're definitely losing your privacy. I'd be happy with an "opt-out" feature that I know worked (as far as I can see, only if it was open-source). I didn't watch WWDC, perhaps they mentioned this.
- johncolanduoni 10y agoI agree opt-out is definitely something that should be deployed alongside differential privacy, but what makes you so sure that it doesn't work "at some point"? If the noise means an specific query to one user's information has a significant chance of being wrong, how does this not equate to privacy? You can add a lot more noise than you might imagine if you know the kind of analysis you'll be doing with the data; for example, a lot of statistical techniques are constructed to be mostly immune to Gaussian noise since it's very common with some kinds of data.
- fauigerzigerk 10y agoThe whole point of collecting the data is to predict the actions or information needs of individual users. That in itself is a privacy issue. If a recommender system for iTunes can predict the likelihood of me appreciating movies that contain violence against women, that information could be subpoenaed when I am falsely accused of having strangled my girlfriend. I appreciate that Apple is trying to protect our privacy where they can. But if we want them to make predictions about or behavior, we have to be aware of the fact that we are necessarily giving up some privacy.
- johncolanduoni 10y agoYou're misunderstanding where this is to be used. It is specifically not for things like iTunes suggestions, where it would be useless. It's for situations where they want to get aggregated metrics without collecting identifiable information. The obfuscation can be performed by the client so that they never have a database on the server with accurate (at the specific user level) data.
- denzil_correa 10y ago> Apple initially went all gung ho about privacy and wanted to make not collecting data a big play My impression always has been that Apple does not collect data that can lead you to be personally identified. I never got any impression that "Apple does not collect data".
- dudusmaximus 10y agoDifferential Privacy is now $AAPL's licence to collect more "anonymised" personal data & benefit. Such personal data collection is not an evil anymore. I'll have some of the "PR" they're having!
- yrhcz 10y agoWho knows, maybe one day that will be the reality. They are taking a lot of steps in the right direction towards facilitating development on their platform – In my opinion it doesn't seem too far fetched to believe that one day they might open up their software too.
- forgotpwtomain 10y ago> I only wish that they would go ahead and put everything under a free software license, since they're in the business of selling hardware that's coincidentally bundled with their software. That's never going to happen. Apple sells a 'User Experience' not just hardware - having a complete and mostly closed product is an inevitable consequence of the former - and the number of Linux users that would buy a Macbook isn't a large enough part of the market for them to worry about. With that said I've had a Macbook Pro and it was pretty much a better piece of hardware (at least as far as build quality) than any other notebook I've used.
- hfsbtnye 10y agoIn what way does changing the software license impede user experience? No other company would have the proverbial balls to straight up copy their software either.
- nixy 10y agoHave you heard about a company called Xiaomi? They have the balls to straight up copy the industrial design of Apple, so pretty sure they would have no problem copying the software.
- deleted 10y ago[deleted]
- 7Z7 10y ago>No other company would have the proverbial balls to straight up copy their software either. How can you say this looking at the hardware landscape?
- lalos 10y agoThey know it's one of the ways to differentiate from the other big companies, by offering privacy and marketing in such a way that it's impossible for the other big players to do the classic "me too" approach.
- kordless 10y agoFuck Apple. The only thing they are programmed to care about is the bottom line and getting more users. That programming is complicated and unlikely to change to suit the consumer's needs, or desires.
- liquidise 10y agoYou say that despite this privacy push being a direct response to consumer sentiments. Apple knows well that as tech becomes wearables and the Internet of things, privacy concerns skyrocket. My grandparents won't buy things online. Soon my generation will be those old and anxious curmudgeons unless our concerns are eased. Do they care about their bottom line? Of course. It's for that very reason they are investing the time now to secure the trust of generations of consumers.
- gear54rus 10y agoThis 'push' is just another unprovable unverifiable marketing bs that is just enough for the type of audience they attract until it's all opened to FS licenses and hardware is made to accept any software that user wants it to (like it should if you bought it).
- kordless 10y agoAgreed. Apple is only doing as much security as they reasonably needed to satisfy customers. Most customers don't understand the various types of suffering (and suffering risk I might add) caused by software, but large companies do. Large companies end up "making decisions" for users en masse. This is fine for button placement but falls apart logically when presented in a discussion based on trusted infrastruture. You will note that liquidse's response is conflicted. Those rationalizations are the indicators of dissonance in the privacy debate.
- ekianjo 10y ago> On the other hand, when the budget was reduced to a level that achieved meaningful privacy, the "noise-ridden" model had a tendency to kill its "patients". Uh, the graph is just showing you get an increased 25% estimated risk of mortality from Warfarin, nothing close to "killing patients". Complete exageration, since the mortality baseline is probably very low in the first place.
- aub3bhat 10y agoGood luck selling that to doctors. People who are trained to cut open a patient aren't receptive to hypothetical harms and imaginary "budgets". Consider the fact that cause of death records are public. The usual perception of privacy by theory and security researchers and norms practiced in healthcare differ enormously.
- cyphar 10y agoThe point of differential privacy is to allow for aggregate analysis, without destroying the privacy of outliers. Researchers deal with noise all the time, so is it so odd that a field of researchers believe that adding enough noise to data released with studies will allow for conclusive analysis without ruining privacy for individuals?
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]
- Create 10y agoOutliers are the reason databases exist. Any "average" is simply readily apparent, therefore irrelevant for serious in depth analysis. Adding noise and fuzzing has a long history in statistics since the '70s [1], and while it does work on large numbers, it almost always messes up the details ie. the error bars. C.D. DP is essentially a cheap ripoff of the ideas implemented in ARGUS[2]. [1] 1977 Dalenius, see Do Not Fold, Spindle or Mutilate movement and earlier: http://tpscongress.indiana.edu/impact-of-congress/gallery/first-census.html http://tpscongress.indiana.edu/impact-of-congress/gallery/fi... [2] http://neon.vb.cbs.nl/casc/ http://neon.vb.cbs.nl/casc/
- bo1024 10y agoI've read a couple articles and haven't seen any details about how they're going to apply DP (differential privacy). It's important to clearly distinguish what DP can and cannot do. DP is just a technique for taking a database and outputting some statistic or fact about it. The output has some noise added to it. The guarantee of DP is (roughly) that anyone looking at the output alone won't learn much about anyone in the database. This also holds for anything you do with that statistic. Think about this carefully when thinking about what DP does and doesn't promise. Also think about the difference between "privacy" and "security". Example of what DP does protect against: If Apple is recommending products to people based on others' download habits, and this recommendation is based on differentially private statistics, then no other user or group of users can infer anything about my downloads. In fact, even engineers at Apple, if they can only see the statistics and not the original database, cannot infer anything about my downloads. Example of what DP does not protect against: government accessing the data. The database still has to exist on Apple's servers. The government can get to it just as easily as before via warrants or so on. DP is not cryptography. My assessment: On one hand it is awesome that Apple is taking a lead in using differential privacy and thinking about mathematical approaches to privacy. On the other, there are many facets of privacy and right now I think people are more concerned about security of their data and privacy from the government, or else privacy from companies like Apple itself. DP doesn't address these; it only addresses the case where Apple has a bunch of data and wants the algorithms it runs not to leak much info about that data to the world at large.
- frankmcsherry 10y ago> The database still has to exist on Apple's servers. It doesn't, which is part of the reason Apple wants to do this. You can still do differential privacy without collecting all the data, you just get less accurate results. See page 232 in [1], re "The Local Model". [1]: http://www.cis.upenn.edu/~aaroth/Papers/privacybook.pdf http://www.cis.upenn.edu/~aaroth/Papers/privacybook.pdf Edit: the article even says this down when it talks about RAPPOR.
- conradev 10y agoEven though the technique doesn't require collecting all of the data, Apple still does collect a huge well of data. They store all iCloud sync material (backups, photos, contacts, calendars, mail, documents, etc.) without end to end encryption, and have all of the iMessage metadata.
- kordless 10y agoIt's time for an Open Communications initiative. Time for companies to stop owning the platform. Time for all of us to stand up for our right to communicate with who we want, when we want, without being monitored, inspected, blamed, or advertised to. Enough is enough. It's time for a change.
- deleted 10y ago[deleted]
- chmike 10y agoDoes it mean that Apple will randomly insert turds in my messages so that it looks like the average user ?
- guelo 10y agoApple backed themselves into a corner by marketing themselves as the super-privacy company in contrast to Google. The problem is that all the data collection lets you do some really useful stuff that benefits the user. So now they're spreading FUD while trying to pretend that they're not collecting the same type of data that Google does. Google has been using differential privacy for a while in different projects.
- Twisell 10y agoBut I can't forget that Google is the company that somehow managed to suggest ads on my personal phone based on browsing on my professional PC. Theses devices are never on the same network, the only shared parameters is an exchange account. As a rule I always log-out of the only Google service I rarely use, so this must be some cookie/tracker dark magic. Sadly I have no proof, but I use gosthery to block trackers since then. (Side note: gosthery also claim to use DP btw)
- techie64 10y agoIt's fairly straight forward - cookies + Google's ad network + Analytics on your phone allow them to track you across devices.
- bla2 10y agoHow would the cookie from the work get to the phone?
- Twisell 10y agoI must have been connected to the same Gmail or Facebook account at some point on both devices. However as I said I purposely never stay logged in to google services. Yeah, I can flush all cookies (thus annoyingly resetting all legit cookies as well). But this is not how privacy should work, cause there is a lot of people out there that don't read HN and only recently found out that there is a lot pastry inside their computers.
- cromwellian 10y agoSome dissenting views on the utility of differential privacy: https://medium.com/@Practical/differential-privacy-considered-harmful-f420ce717846 https://medium.com/@Practical/differential-privacy-considere... Also, Apple is woefully low on details, theoretical privacy should be accompanied by openly published research papers that are peer reviewed. I understand they won't release the source, but would you trust Apple if they said they invented a new encryption algorithm, but refuse to publish an academic paper on it? I'd be interested precisely in what they're doing. Are they claiming they're doing federated learning, by gathering anonymous image data from photos, uploading it to their cloud, training DNNs on it, and then shipping the results back down to clients for local recognition? Surely they're not training on device, as this is very RAM and CPU intensive.
- EGreg 10y agoIf you collect values of random variable Y from phones, where Y = X + N (N being normally distributed with mean 0 and var Y = var X, say) then many statistics can be calculated with that. The law of large numbers says that after gathering statistics from many values of Y, they will converge (for continuously differentiable functions of X) to the values for X. Yes? Meanwhile each individual user will not send so many samples as to identify the true values of X with any useful accuracy.
- nxzero 10y agoSeems to be a huge amount of speculative commentary, which is acknowledged, but to me, not a way that shows the potential variation in implementing DP. For example, Apple could easily download all the data, do a DP on the impact of adding the data to the existing aggregate data, clean out indentiers, and add it to the database. Key here is that Apple has all the data, then purges the indentiers from it, which is completely different than removing the indentifiers before sending to Apple. _______ (Apple:) "Hi, I'm Apple, Trust Me! Don't mind the black bag, I just likely being mysterious, it's cool, right?" (Me:) "Umm, no, no thanks!" _________ Apple needs to let go of the whole security through secrecy ploy, since it looks more and more shady. Imagine if security modules for devices where public and non-secure section of the devices had to be encapsulated for EmSec and tamper proof. If this was the case, security literally wouldn't be an issue; either everyone is impacted, or no is impacted.
- ianmiers 10y agoThis is something Apple really needs to release all the details of. Even if they got the crypto exactly right, they could have picked a privacy budget/ security parameters that just leaks everything. And there is every reason to be skeptical about Apple's ability to design even mildly complex crypto given iMessage's flaws. Although the break in iMessage wasn't practically exploitable, that was luck and the fact that the only way to detect if a mulled ciphertext decrypted required attachment messages. The cryptographic mistakes were bad. Given any way to detect decryption of mulled ciphertexts for standard messages (e.g. sequence numbers, timing, actively synching messages between devices, delivery receipts from iMessage instead of APSD), Apple's crypto design bugs would have eliminated nearly all of the E2E security of iMessage. Remember, this isn't a boon for user privacy. Apple is now collecting far more invasive data about users under the claim that they have protections in place. At best it preserves the status quo and does so only if Apple both picked the parameters correctly and implemented it correctly. At this point Apple's position should be best summed up as: we have drastically reduced your privacy except not because magic that we (i.e. Apple) do not fully understand.
- mnem 10y agoApple have designed/implemented several quite successful crypto and security systems too.
- ianmiers 10y agoWhat have they actually designed from scratch? Most of what comes to mind (Facetime, filevault) it is off the shelf stuff/ at least there were well known designs to ape which had been subject to analysis. This, well if they just copied what google did, then maybe. But at some level, my comment is pretty harsh on Apple when they have a better track record than most for privacy and encryption. But remember, this is Apple making you less secure by grabbing data and then saying they took care of the issue. With iMessage, Facetime, Filefault, if it failed, you were were no worse off than if you used something else. In this case, you actually are. That means there is a higher burden to getting it right than just name dropping magic crypto pixie dust.