5 ms·
"sunlight all of their data and code in a repository" Good idea where possible, but "all their data" will be problematic in many social science studies, and al
by Someone 10y ago
"sunlight all of their data and code in a repository"
Good idea where possible, but "all their data" will be problematic in many social science studies, and also in medicine, because of privacy laws.
- finid 10y agoKeep in mind that data can always be anonymized. So even most medical or social science research data can be put in a repo.
- rando832 10y ago> Keep in mind that data can always be anonymized. No it can't. Some data can be anonymized, some can't.
- RyanZAG 10y agoThere's always a clever solution to anonymize any data. Post a description of the data and the problems with anonymizing it, and someone out there will always find a clever way to handle it. You don't have to solve everything yourself. That's the whole point of a peer driven scientific community.
- dahart 10y agoInferences in large data sets are more powerful than you think. Individual people have been identified from multiple "anonymized" scrubbed datasets, and its only getting easier as the datasets get bigger and the analytics tech gets better.
- wyager 10y agoAnonymize the data then, if plausible. We ought not to trust studies that are inherently irreproducible.
- Someone 10y agoIt's practically impossible to anonymize data without throwing away the information contained in it, doubly so if you don't know at the time what questions critics will want to see answered on that data.
- wgyn 10y agoThat's not quite true. E.g. Google's email spam models are trained without access to the raw text. In general the field of differential privacy is developing rather quickly.
- vitno 10y agoI'm somewhat confused about how they would be doing that. Do you have any references to blogs/papers on this technique?
- bjterry 10y agoOne typical strategy for spam detection is to convert text to a "bag of words" representation[0]. If you take this bag of words representation and hash all the values, then rather than training words like ED, you are training on word number 19213123. The number of these hashed values is smaller than the number of words, just like a hash table, and this generally doesn't harm the accuracy of the machine learning. When you receive feedback on the classification (from people reporting spam or people marking things as not spam), you just turn the reported email into a bag of hashed words and feed that change into your model. Because the order of the entries in the bag of words is arbitrary, and the words have been hashed, it is impossible to go back from a bag of words representation to the original email. I don't know if this is what google does, but it is pretty normal to do so. 0: https://en.wikipedia.org/wiki/Bag-of-words_model https://en.wikipedia.org/wiki/Bag-of-words_model
- cvwright 10y agoThis is actually a really tricky topic. Things that sound like they should give very good security, often don't in practice. The "hashed bag of words" technique that you describe here sounds an awful lot like some recent attempts at letting legacy systems search on encrypted data [1,2]. We took at look at this recently [3], and it turns out that mapping the word numbers back to the original words is actually a lot more doable than you'd think. [1] ShadowCrypt: Encrypted Web Applications for Everyone http://dl.acm.org/citation.cfm?doid=2660267.2660326 http://dl.acm.org/citation.cfm?doid=2660267.2660326 [2] Mimesis Aegis: A Mimicry Privacy Shield–A System’s Approach to Data Privacy on Public Cloud https://www.usenix.org/conference/usenixsecurity14/technical-sessions/presentation/lau https://www.usenix.org/conference/usenixsecurity14/technical... [3] The Shadow Nemesis: Inference Attacks on Efficiently Deployable, Efficiently Searchable Encryption https://www.sigsac.org/ccs/CCS2016/agenda/ https://www.sigsac.org/ccs/CCS2016/agenda/
- semi-extrinsic 10y agoAlso, in many cases in the physical sciences, the data sets (both from experiments and simulations) may simply be too big for long term storage, let alone sharing. CERN is pushing the envelope at around 1 TB new shared data per day, but projects run by groups as small as five people can easily generate tens of terabytes in a day. E.g. with turbulence simulations or tomographic PIV or even just multiple high-speed cameras.
- s_q_b 10y agoConservatively, it's about $7.00 per terabyte to store and retrieve data on Amazon Glacier per month. Working with petabyte size streams and storage is a pain, but it can be done. https://aws.amazon.com/glacier/pricing/ https://aws.amazon.com/glacier/pricing/
- nradov 10y agoIn the USA the federal government has established clear rules for anonymizing (de-identifying) healthcare data. So it's clearly possible to do so. http://www.hhs.gov/hipaa/for-professionals/privacy/special-topics/de-identification/ http://www.hhs.gov/hipaa/for-professionals/privacy/special-t...
- jandrese 10y agoIt should be pointed out that thanks to the explosion of data anonymizing is a lot harder than you might expect.
- Someone 10y agoYou can only conclude it is clearly possible from that if you assume the USA federal government is infallible. I don't think that's the case. The way I read it, the "Safe Harbor" part of that standard allows keeping around part of the zip code if that designates over 20,000 people. For such a group of just over 20,000, add in birth year and sex (both allowed), and you're down to smallest groups of around 200 people. For the topic at hand (publish the data set so that critics can draw their own conclusions), often general health, education level and race, maybe even line of work (blue collar/white collar/agriculture) must be added, so that critics can check that your sample is representative. I bet that gets you down to a single person in quite a few of those groups. Yes, they end with "The covered entity does not have actual knowledge that the information could be used alone or in combination with other information to identify an individual who is a subject of the information.", but the earlier list doesn't make me confident that the USA federal government can make that judgment.