9 ms·
Debunking the Myth of "Anonymous" Data
- jruohonen 3y agoA good popular take, but they, either intentionally or out of ignorance, omit newer, proven techniques like differential privacy.
- buttercraft 3y agoOr they don't want you collecting data in the first place, in which case the techniques are irrelevant.
- JohnFen 3y agoEntirely this. That data is so widely collected is the root problem.
- pixl97 3y agoIn particular areas things like differential privacy may work, for example medical sets where there are lots of regulations and potential fines for the companies involved. But do you think your average ad tech company gives a fuck? They are going to keep the original data because that's where the money is. Yea, maybe they'll have privacy datasets they sell/release to other groups, but all the real data will remain in a database, and with most companies in this industry, be given to government agencies on demand.
- willseth 3y agoIt's not because of the money. Most could make money just fine with properly anonymized data. They just don't care and/or don't want to take the effort or spend money to do it because the consequences for leaking private info are so minimal.
- Forbo 3y agoStill sounds potentially problematic. Per wikipedia: "Differential privacy provides a quantified measure of privacy loss and an upper bound and allows curators to choose the explicit trade-off between privacy and accuracy. It is robust to still unknown privacy attacks. However, it encourages greater data sharing, which if done poorly, increases privacy risk. Differential privacy implies that privacy is protected, but this depends very much on the privacy loss parameter chosen and may instead lead to a false sense of security. Finally, though it is robust against unforeseen future privacy attacks, a countermeasure may be devised that we cannot predict." If I am dependent on the curator to determine the level of privacy then I've already lost.
- JohnFen 3y agoI agree. DP is better than no privacy at all, but it certainly doesn't make me feel comfortable.
- 95014_refugee 3y agoYou are already dependent on a set of curators to not simply outright lie and export the data as captured. This take lacks subtlety; unless you are going to abandon the set of functionality ("where is the best fried chicken near me?") that this sort of metadata facilitates, you need to make decisions about which curators you trust, and then participate in driving them to honesty and accountability. From my perspective this article leans too heavily into the FUD, and really doesn't succeed at keeping the call to action ticking over. On a good day, the EFF can be really good. Today, not so much so.
- c0pium 3y agoThe problem with the EFF is that it’s full of people who made up their minds about what’s ok and what isn’t over a decade ago, and they are largely just playing the hits now. No new material, and no consideration that things might change.
- c0pium 3y ago> …it encourages greater data sharing, which if done poorly, increases privacy risk. This is a really useful argument, because it’s the equivalent of the FDA’s “generally believed to be safe”. If you look into something and this is the risk you find, then it’s safe.
- godelski 3y agoThere's also Zero Knowledge Proofs which instead has a privacy guarantee Edit: Not trying to start a DP vs ZKP (or Homomorphic Encryption) flame war. We're all on the same side of privacy, right? Different tools for different situations.
- jandrewrogers 3y agoTechniques like differential privacy do not work for some types of data models, including many of the more interesting/risky ones. I've never seen a technique that can deliver an analytical model at scale that is both analytically effective and anonymous while also robust against sophisticated de-anonymization attacks. There is no good theoretical foundation to suggest that such things are possible. Most modern techniques for ensuring anonymization make assumptions that won't constrain sophisticated blackhats. They are good policy in a legal ass-covering sense and increase the cost required to de-anonymize but that is about it.
- willseth 3y agoCan you give an example? Global DP systems generally offer excellent privacy protection for the analytical distortion cost. Global DP is often not implemented simply due to workflow inconveniences but otherwise it’s great for analytical use cases.
- dfcowell 3y agoIf your scale is large enough and you don’t care about identifying individuals, synthetic data does this fairly effectively.
- orbital-decay 3y agoDifferential privacy does not anonymize your data. It's a (mathematically solid) instrument to make it coarse enough to only give up the part you want. The EFF is completely correct in their assessment - there's no way to make your data anonymous and usable by the third party at the same time. The root issue lies elsewhere. There's FHE, which does allow your data to be processed without accessing it, but it has prohibitive compute requirements and isn't practical for most purposes. I'm not sure if it allows the data to be usable enough without revealing it, either.
- Godel_unicode 3y ago> there's no way to make your data anonymous and usable by the third party at the same time. The This is absurd hyperbole, here’s a trivial proof. The US population is 51.1% female. If you live in the US your gender information is included in this data point, but it cannot be de-anonymized. It is however useful for determining TAM for many products. Anonymity is a spectrum, and there are many points along that spectrum which are solutions to usable data which also respects privacy.
- JohnFen 3y agoPrivacy is a spectrum, anonymity is not.
- jdietrich 3y agoGDPR specifically mentions pseudonymous data in Recital 26: "The principles of data protection should apply to any information concerning an identified or identifiable natural person. Personal data which have undergone pseudonymisation, which could be attributed to a natural person by the use of additional information should be considered to be information on an identifiable natural person. To determine whether a natural person is identifiable, account should be taken of all the means reasonably likely to be used, such as singling out, either by the controller or by another person to identify the natural person directly or indirectly." Under EU law, there is no special class of "personally identifying information". Any data that relates to a person or could be related to a person is protected. It isn't enough to just strip the name and SSN field out of the database and call it anonymous, you need to demonstrate that the data couldn't be attributed to anyone through any reasonably practical process. https://gdpr-info.eu/recitals/no-26/ https://gdpr-info.eu/recitals/no-26/
- c0pium 3y agoThat is almost verbatim the definition of PII from Wikipedia and from OMB Memorandum M-07-1616 (the US government definition of PII).
- oezi 3y agoThe crucial piece is your link is: > 4To ascertain whether means are reasonably likely to be used to identify the natural person, account should be taken of all objective factors, such as the costs of and the amount of time required for identification, taking into consideration the available technology at the time of the processing and technological developments. This has been interpreted that any objectively possible way to de-anonymize is reasonable. In particular, considering that all the data of the organization doing the anonymization can be used to try to de-anonymize.
- kwillets 3y agoI think they're trying to keep cryptographically secure data out of scope.
- bertil 3y ago
- euniceee3 3y agoGreat article putting all the relevant content in one place. Does anyone know of any de-anonymization services? The startup I am working at is privacy focused and we are looking for a way to demonstrate why you need an additional layer to protect and compartmentalize. Short of us buying up data in bulk and then doing the de-anonymization in-house I am not seeing an easy way to do this. Or even an advertised partner, seems like all the articles are really careful to not do free marketing for companies in this space.
- willseth 3y agoThere are a small handful of us out there, targeting slightly different but overlapping use cases, but this is me https://www.privacydynamics.io https://www.privacydynamics.io. Happy to answer any questions about it.
- bertil 3y agoThis is the same problem as white-hat security: you need people who know an awful lot about how bad guys work but are good guys. Security has developed a legitimate market over the years, but that took time. You could try to ask independent consultant(s) who you trust to work on that problem and let them grow that practice. The problem is that they would need to have access to marketing platforms.
- willseth 3y agoI just realized you wrote “de-anonymization”. We don’t do that, but we do a risk analysis that simulates an attack, making some pessimistic assumptions about what data an attacker may have for linkage, to estimate how risky data is to share, essentially as a benchmark. TLDR; more unique combinations of quasi-identifiers make data more susceptible to a linkage attack. We also pitch our system as a risk analysis only tool, though no one is using it for that currently. If you’re interested in methods, I’d recommend El Emam’s book, “Guide to the De-Identification of Personal Health Information”, which covers a lot in detail. Our attack simulation is based on his work.
- deleted 3y ago[deleted]
- spiffytech 3y agoEvery time I hear "anonymous data", I think of that time AOL published anonymized search logs (for academic research). The anonymization was negligent, and an NYT reporter de-anonymized and tracked down one of the users with the local & personal info present in the search queries. https://en.wikipedia.org/wiki/AOL_search_log_release https://en.wikipedia.org/wiki/AOL_search_log_release https://web.archive.org/web/20130404175032/http://www.nytimes.com/2006/08/09/technology/09aol.html?_r=1 https://web.archive.org/web/20130404175032/http://www.nytime...
- willseth 3y agoAnother fun one was the Netflix Prize, where Netflix published an anonymized dataset, but some researchers at UT Austin were able to de-identify many/most of the users in the dataset by linking them to IMDB profiles based on preferences.
- jeffbee 3y agoThere's no way most of the users in the Netflix data had IMDb accounts.
- e12e 3y agoFrom a skim, it looks like their main point is that with 8 movie ratings, it's trivial to link two datasets (eg anonymized Netflix rating to public IMDb profile). https://scholar.google.com/citations?view_op=view_citation&hl=en&user=rejZUEkAAAAJ&citation_for_view=rejZUEkAAAAJ:u-x6o8ySG0sC https://scholar.google.com/citations?view_op=view_citation&h... https://systems.cs.columbia.edu/private-systems-class/papers/Narayanan2008Robust.pdf https://systems.cs.columbia.edu/private-systems-class/papers...
- frankyg 3y agoI have thoughts like that all the time but then I remember how many people have accounts at xyz at all and how often I created an account on abc, made a list or two and never returned (until I did). I have over 600 logins saved in my password manager and "holy fuck shit" some of the data on these sites is still "part of me", I just don't actively spend time on it. It's all quite usable.
- willseth 3y agoI work for a data privacy startup, and this article unfortunately groups all forms of anonymization together. It is specifically criticizing forms of anonymization that only treat direct identifiers like names, addresses, and phone numbers. That is usually referred to as "pseudonymization", and they are correct to point out that an only moderately sophisticated attacker can still link people in the dataset using combinations of indirect identifiers like age, birthday, and zipcode. Pseudonymization is a weak form of privacy. More sophisticated methods of privacy also anonymize indirect identifiers, and in some cases personal attributes. They do this by adding noise to the data in such a way that the noise has relatively* minimal impact on the results of computations made over the dataset, but a significant impact on the ability to re-identify someone using indirect identifiers or attributes. *There is always a tradeoff between privacy and utility. The only way to achieve 100% private data is 100% noise, but the privacy-utility tradeoff curve isn't linear, and you can still achieve very good utility and very good privacy in many cases, especially with the best tools. Methods are also improving over time, reducing the impact of the tradeoff.
- genewitch 3y agoIn the interest of charitable-ness, can you define "noise" and explain how rudimentary denoising algorithms that have existed since the 1960s can't penetrate it? Time and time again people think they've anonymized data and they're always proven incorrect. Just come to grips with the fact you're participating in the sale of my (or whoever's) private data. Hope you sleep well!
- yowlingcat 3y ago> Just come to grips with the fact you're participating in the sale of my (or whoever's) private data. Hope you sleep well! I think it's really sad that the open source community has historically cultivated a culture which doesn't recognize this kind of paranoia as a lack of mental wellness. I think it's because the zealotry comports well with the extreme amount of energy needed to bootstrap the ecosystem into existence. You can care about privacy while being unrealistic about why you don't truly have it and never will, because of how much of a risk it is to the powers that be. You can live in a world ruled by proxy wars between gigantic political superpowers and empires who control the upstream levers of the macroeconomy to ensure that the world will always go in a direction opposite to what you'd prefer it to be. It's been like this for thousands if not tens or hundreds of thousands of years. But moreover, you can and likely do live in a society, full of people who will always value convenience over the freedom you seem to cherish. But even you likely are willing to give up some freedom and privacy for some convenience, unless you employ private security personnel to guard your private compound. And my guess is that if you were inclined in that direction, you wouldn't really be posting on HN. Short of that, to live in a modern society is to some degree accept this compromise and to accept one's own hypocrisy -- and my heart goes out to you as much as to myself because the existential anguish of this never goes away. Nonetheless, it is generally historically accepted that as we grow older, we are to develop the ability to constructively manage around this and gain the wisdom to do so. Perhaps the reason you sarcastically asked GP that you "hope you sleep well" is because you are projecting and you know that you yourself do not. My advice to you and those of your ilk: stop obsessing over theoeretical purity and get on with enjoying your life in the scarce time you have left on this planet. You'll have far less regrets at the end. And you will finally sleep well.
- hasoleju 3y agoAnonymization of personal data is a very tricky thing. It is so tricky that the GDPR even doesn't really mention it. I even don't know of any official SOP for doing it. Of course k-anonymity and differential privacy can help you with a closed data set, but once you add records to a dataset over time everything breaks down. I once tried to find a way to anonymize data on different clients and then connect the records of the same entity on the server. This method had the main problem that one could artificially generate a new non-identifying data point for an individual and then see in which record it will end up in the central data base.
- yoaviram 3y agoMore about why location information IS personal information: https://consciousdigital.org/location-data-is-personally-identifiable-information/ https://consciousdigital.org/location-data-is-personally-ide...
- ndr 3y ago"De-identified data isn't. Either it's not de-identified, or it's not data anymore." -- Cynthia Dwork, one of the invertors of Differential Privacy, Knuth Prize and Gödel Prize winner. https://youtu.be/RWpG0ag6j9c?feature=shared&t=274 https://youtu.be/RWpG0ag6j9c?feature=shared&t=274
- hinkley 3y agoThere's anonymizing data so that creepy guy two floors down can't stalk the barista across the street, and there's anonymizing data so that you can publish it to the world. Barely even comparable.
- jmatthews 3y agoI like to make an appearance in threads in this domain and just say, yeah, it's not just possible but "fun" to put the puzzle pieces together. It is a very tough problem to solve. Especially when you consider the richness of the datasets you use to put the pieces together. In my experience the only effective means is to poison the data, in addition to the common sense steps mentioned here. Poisoning a dataset means seeding a wide variety of the datasets you use to discover PII with fictional look alikes that resist debunking. Additionally you can poison the core set if you are very clever about it.
- lsh123 3y agoAs others pointed out, the article mixes a lot of things together. EU (GDPR) has very specific and very hard to meet anonymization bar (tldr; it requires anonymization to be at the level where it’s mathematically improbable to de-anonymize the user). None of the “anonymization” examples in the article would pass this EU bar.
- deafpolygon 3y agoAnd yet this does not prevent anyone from doing it in the EU.
- lsh123 3y agoI can’t speak for everyone, but the company I work for takes this very seriously and does its best to comply with the law. Said that, all of it is very complicated. Most (if not all) privacy problems I observed so far are caused not by malice but by incompetence. And those get fixed as soon as found.
- daamien 3y agoActually the GDPR "just" requires to protect the data against "reasonably likely" attacks. « To determine whether a natural person is identifiable, account should be taken of all the means reasonably likely to be used, such as singling out, either by the controller or by another person to identify the natural person directly or indirectly. To ascertain whether means are reasonably likely to be used to identify the natural person, account should be taken of all objective factors, such as the costs of and the amount of time required for identification, taking into consideration the available technology at the time of the processing and technological developments. » https://www.privacy-regulation.eu/en/r26.htm https://www.privacy-regulation.eu/en/r26.htm
- goodboyjojo 3y agodata is never anonymous. if you get enough of it you can correlate who is who and turn it into information on the user.
- PeterStuer 3y agoReminds me about the telco offer to sell gps location traces on anyone you wanted, but to ensure 'anonymity' you had to order a batch of 30 people minimum and you did not get information on which trace belongen to who in that set. Figuring out which of those traces belonged to which employee was a real puzzle /s
- frankyg 3y agoI don't think humanity is going to get around disclosing how exactly all that data flows into marketing, UX design and policies. Not in the form of pop-science books or blockbuster documentaries but detailed statistics and open reports by the companies themselves.