4 ms·

When organizations claim to have "anonymized" a data set, what exactly does that mean? I.e., do they mean that nobody they talked to could think of a way to re

When organizations claim to have "anonymized" a data set, what exactly does that mean? I.e., do they mean that nobody they talked to could think of a way to recover the identity of even one individual in the set with 100% certainty? Or is there some information-theoretical or legal standard of anonymization they're claiming to have met?

throwawayjava 7y ago
There are some mathematical definitions [1], but the fundamental problem is that with enough cross-referencing between databases it's hard to say anything for sure [2]. You never know what data other people might publish in the future. I'm not aware of any legal definitions, but given the thorniness of reidentification I would assume they're insufficient. [1] https://en.wikipedia.org/wiki/K-anonymity https://en.wikipedia.org/wiki/K-anonymity [2] https://www.wired.com/2007/12/why-anonymous-data-sometimes-isnt/ https://www.wired.com/2007/12/why-anonymous-data-sometimes-i...
emmelaich 7y ago
In this particular example, they produced a random number which use the real id[0] as a seed, then mixed the result with the original id. It was not enough, and, as Teague et al note: > Indeed, encryption was not necessary – a randomly chosen unique number for each person would have worked. Scroll down from here: https://www.oaic.gov.au/privacy/privacy-decisions/investigation-reports/mbspbs-data-publication/#information-about-medicare-service-providers https://www.oaic.gov.au/privacy/privacy-decisions/investigat... [0] The data had ids for providers (e.g. doctors) as well as patients.
MaulingMonkey 7y ago
> When organizations claim to have "anonymized" a data set, what exactly to that mean? For "organizations" in general? It means approximately nothing, or if you're feeling particularly generous, it means "we probably remembered to drop the column containing your social security number before publishing this data... this time". You're asking exact specifics of a vague and broad category. There are some legal standards, information theory, and non-legal organization standards that might being met in some cases - involving adding noise or removing data / making it sparse. https://en.wikipedia.org/wiki/Data_re-identification https://en.wikipedia.org/wiki/Data_re-identification goes into all the ways that it can go wrong despite the best of intentions. My basic take on this all is: data "always" gets more identifying, not less. Two datasets that were successfully anonymized individually can still be correlated to de-anonymize some or all of the data when combined. Even organizations applying information theory with the best of intentions and proper diligence will eventually make a mistake.
DEADBEEFC0FFEE 7y ago
In hwalthcare there usually an ethics panel, that will look at the data, and look for way to reduce re-identification. The common example is the one-legged child with cancer from a remote town. You can remove a the PII columns and it's pretty easy to find that person.
- rzzzt 7y ago
  One way around that is to drop all cases below a certain occurrence threshold, ie. if there aren't at least 1000 people in the same town with the same condition, they aren't getting into the dataset. (The downside is that rare diseases might fall through the cracks.)

throwawayjava 7y ago

There are some mathematical definitions [1], but the fundamental problem is that with enough cross-referencing between databases it's hard to say anything for sure [2]. You never know what data other people might publish in the future. I'm not aware of any legal definitions, but given the thorniness of reidentification I would assume they're insufficient. [1] https://en.wikipedia.org/wiki/K-anonymity https://en.wikipedia.org/wiki/K-anonymity [2] https://www.wired.com/2007/12/why-anonymous-data-sometimes-isnt/ https://www.wired.com/2007/12/why-anonymous-data-sometimes-i...

emmelaich 7y ago

In this particular example, they produced a random number which use the real id[0] as a seed, then mixed the result with the original id. It was not enough, and, as Teague et al note: > Indeed, encryption was not necessary – a randomly chosen unique number for each person would have worked. Scroll down from here: https://www.oaic.gov.au/privacy/privacy-decisions/investigation-reports/mbspbs-data-publication/#information-about-medicare-service-providers https://www.oaic.gov.au/privacy/privacy-decisions/investigat... [0] The data had ids for providers (e.g. doctors) as well as patients.

MaulingMonkey 7y ago

> When organizations claim to have "anonymized" a data set, what exactly to that mean? For "organizations" in general? It means approximately nothing, or if you're feeling particularly generous, it means "we probably remembered to drop the column containing your social security number before publishing this data... this time". You're asking exact specifics of a vague and broad category. There are some legal standards, information theory, and non-legal organization standards that might being met in some cases - involving adding noise or removing data / making it sparse. https://en.wikipedia.org/wiki/Data_re-identification https://en.wikipedia.org/wiki/Data_re-identification goes into all the ways that it can go wrong despite the best of intentions. My basic take on this all is: data "always" gets more identifying, not less. Two datasets that were successfully anonymized individually can still be correlated to de-anonymize some or all of the data when combined. Even organizations applying information theory with the best of intentions and proper diligence will eventually make a mistake.

DEADBEEFC0FFEE 7y ago

In hwalthcare there usually an ethics panel, that will look at the data, and look for way to reduce re-identification. The common example is the one-legged child with cancer from a remote town. You can remove a the PII columns and it's pretty easy to find that person.

rzzzt 7y ago
One way around that is to drop all cases below a certain occurrence threshold, ie. if there aren't at least 1000 people in the same town with the same condition, they aren't getting into the dataset. (The downside is that rare diseases might fall through the cracks.)