4 ms·
It's a mathematical problem, not a technical problem. You can remove features from a dataset that, on their own, serve to uniquely identify someone, but that do
by nonameiguess 5y ago
It's a mathematical problem, not a technical problem. You can remove features from a dataset that, on their own, serve to uniquely identify someone, but that doesn't change that the remaining features still each partition the population into smaller and smaller subsets until eventually the subset is one or only a few people. Once you get down to "only a few people," metadata may be able to bring that to one, i.e. there are a few hundred people it could potentially be, but only one was actually at the hospital the data was collected from at the time it was collected. You don't necessarily even need the hospital to leak that metadata. Maybe this person was the only one who even lived near that hospital near that time.
The only way to anonymize the data in a foolproof way is to remove so many features that it becomes useless for statistical research.
Modeling-wise, regression, classification, and clustering alike all rely on being able to construct a model that minimizes entropy. But anonymity relies upon maximizing entropy. This fundamental conflict can't really be resolved. You pick some point in the middle of the spectrum that ends up serving neither purpose well.
- unishark 5y agoMath problems aren't technical problems? Do you mean it's a mathematically-impossible technical problem? Noise and obfuscation can be added to the data such that the unique person it identifies doesn't exist. For example changing the dates by some limited random amount. Replacing values with roughly-similar choices (replace engineer with scientist, replace Mexico with el Salvador, etc.) The technical problem is doing that in a way that doesn't harm its research value, never allows the final set of possible similar identities to be below some lower limit, and can't be reversed of course.
- poplarstand 5y agoWould you be willing to provide a few references for this? My professional work has skirted around the discussion at times, and I would be fantastically interested in learning more.