4 ms·

> Re-identify based on what ? The fact that you poo-poo this question is telling. Preventing re-identification attacks is incredibly subtle and sometimes impos

> Re-identify based on what ? The fact that you poo-poo this question is telling. Preventing re-identification attacks is incredibly subtle and sometimes impossible. Removing "city and date of birth" is nowhere near sufficient. > this raw data is already available, but under a loose NDA for commercial partners and researchers... Yes. It's available to people who have their real identities tied to their access and can be sued into oblivion or possibly even prosecuted for wanton misuse of the data. Surely you see how this is different from throwing it on a public s3 bucket, right?

rvnx 5y ago
It looks like from what you are saying that it's an incommensurable effort to anonymize reasonably the dataset (the majority of the risks can be hedge by bucketing, and then there can be subtle deanonymization edge cases but they don't scale). It reminds me the FDA saying they can't release the COVID-19 documents because reviewing 44'000 documents would take 50 years. It didn't take 50 years to produce them... Yes there is some effort needed to anonymize reasonably the data but it's not an impossible task. It's a question of motivation. Here, clearly, the labs and administration don't really want to put efforts into that.
- throwawaygh 5y ago
  Presumable FDAs analysis assumed current staffing levels. Going to go out on a limb and assume they’ve got a LOT more people involved in creating docs than in deidentifying docs. And, yes, I think public data on this sort of stuff is important and should be properly resourced. I think: Step 1. ASK PERMISSION before changing the way data is handled. Step 2. Share full datasets more generously, but still gated by use and handling agreements. This probably means a private citizens without ethics research training and a supporting it dept can’t get a copy and peruse it on their personal laptop, but also that a truly enormous number of researchers would be able to access the data. The gold standard of a CSV in a public s3 bucket shouldn’t be the enemy of good enough.

rvnx 5y ago

It looks like from what you are saying that it's an incommensurable effort to anonymize reasonably the dataset (the majority of the risks can be hedge by bucketing, and then there can be subtle deanonymization edge cases but they don't scale). It reminds me the FDA saying they can't release the COVID-19 documents because reviewing 44'000 documents would take 50 years. It didn't take 50 years to produce them... Yes there is some effort needed to anonymize reasonably the data but it's not an impossible task. It's a question of motivation. Here, clearly, the labs and administration don't really want to put efforts into that.

throwawaygh 5y ago
Presumable FDAs analysis assumed current staffing levels. Going to go out on a limb and assume they’ve got a LOT more people involved in creating docs than in deidentifying docs. And, yes, I think public data on this sort of stuff is important and should be properly resourced. I think: Step 1. ASK PERMISSION before changing the way data is handled. Step 2. Share full datasets more generously, but still gated by use and handling agreements. This probably means a private citizens without ethics research training and a supporting it dept can’t get a copy and peruse it on their personal laptop, but also that a truly enormous number of researchers would be able to access the data. The gold standard of a CSV in a public s3 bucket shouldn’t be the enemy of good enough.