3 ms·

https://www.federalregister.gov/documents/2018/04/30/2018-09078/strengthening-transparency-in-regulatory-science https://www.federalregister.gov/documents/2018/

https://www.federalregister.gov/documents/2018/04/30/2018-09078/strengthening-transparency-in-regulatory-science https://www.federalregister.gov/documents/2018/04/30/2018-09... The proposed regulation provides that, for the science pivotal to its significant regulatory actions, EPA will ensure that the data and models underlying the science is publicly available The public availability requirement sounds nice on paper, but in practice, it means ignoring a lot of science during rule-making because the original data can't be published due to genuine privacy concerns. There are some common-sense workarounds. For example, requiring an EPA or third-party audit of the original dataset and analysis. Or simply carving out exceptions for research whenever it can be demonstrated that an IRB (and therefore federal law) would not have allowed the research without restricting public access to data/analysis.

microcolonel 8y ago
> The public availability requirement sounds nice on paper, but in practice, it means ignoring a lot of science during rule-making because the original data can't be published due to genuine privacy concerns. It is more limited if you read the whole sentence, which goes on to qualify which data they are interested in (specifically "publicly available in a manner sufficient for validation and analysis.").
- throwawayjava 8y ago
  which de facto excludes 90% of medical research because medical patients have privacy rights and researchers have legal obligations to respect those rights. These rules are a hot political topic that are years in the making. The people pushing them are not scientists and have never expressed genuine interest in reproducibility outside of this one issue. And scientists who have spent years fighting for reproducibility have publicly stated they oppose this rule. And all the major proponents have strong anti-regulatory preferences. These rules are not about reproducibility. This rule is an intentionally crafted Catch-22 designed to exclude medical research from consideration so it's easier to justify repealing or not implementing environmental regulations. Otherwise, the rules would find a middle ground; e.g., by allowing the EPA access to relevant scientific data and analysis without risking https://en.wikipedia.org/wiki/De-anonymization https://en.wikipedia.org/wiki/De-anonymization by posting sensitive medical information about thousands of patients on the public internet.
  - smsm42 8y ago
    It's very hard to believe it is not possible to publish medical data in an anonymized fashion that does not infringe privacy rights. If that's true, how people validate any medical research at all? Do they just ignore the validation, and if some research has an error - well, tough luck, we'd base policy on it anyway and if it turns out to be a disaster, we'd just shrug? Giving up on reproducibility and validation means giving up on one of the most important features of modern science, I have very hard time believing that's the only way to do modern medical research known to our civilization. > Otherwise, the rules would find a middle ground; They actually did, if you read the actual proposal, which talks about disclosure in ways that enough for validation but does not infringe privacy. But you seem to have arrived to pre-determined conclusion already, and there in fact can be no middle ground that would be acceptable to you, as it seems.
    - throwawayjava 8y ago
      > It's very hard to believe it is not possible to publish medical data in an anonymized fashion that does not infringe privacy rights Then you're one of today's lucky 10,000: "Robust De-anonymization of Large Sparse Datasets" http://www.cs.utexas.edu/~shmat/shmat_oak08netflix.pdf http://www.cs.utexas.edu/~shmat/shmat_oak08netflix.pdf A lot of environmental medical research results in dense and small data instead of sparse and large data, which obviously means it's even easier to de-anonymize. And the really important take away from the above paper is that you need to know about all the other public data in the universe in order to really ensure anonymity. Even when anonymity seems like it's "obviously" not going to be a problem. You never know what dataset will be leaked/published tomorrow correlating columns D-F in the spreadsheet you gave EPA to someone's first and last name. So privacy is like computer security: you can never be totally sure. So when the stakes are high you should be extra cautious. And the stakes when sharing medical research are very often high. > If that's true, how people validate any medical research at all?.. I have very hard time believing that's the only way to do modern medical research known to our civilization. This isn't exactly a new problem, so scientists have had a little while to figure out how to make peer review work. You share the data. You share with trusted, relevant third parties who promise to preserve the anonymity of study participants. Just becuase you do not publish possibly de-anonymizable and definitely sensitive medical information on a public website does not mean you don't share the data with interested and relevant third parties. > But you seem to have arrived to pre-determined conclusion already, and there in fact can be no middle ground that would be acceptable to you, as it seems. On the contrary, I do literally exactly that in my very first post in this thread: There are some common-sense workarounds. For example, requiring an EPA or third-party audit of the original dataset and analysis. Or simply carving out exceptions for research whenever it can be demonstrated that an IRB (and therefore federal law) would not have allowed the research without restricting public access to data/analysis. The situation where IRB says "no" to public disclosure should explicitly trigger exemption, with the duty falling back to the agency to prove the IRB was wrong in the first place. The fact that this exemption is not automatic is what makes this a catch-22 that pins researchers in-between their IRB and the EPA administrator in charge of evaluating their exception.

microcolonel 8y ago

> The public availability requirement sounds nice on paper, but in practice, it means ignoring a lot of science during rule-making because the original data can't be published due to genuine privacy concerns. It is more limited if you read the whole sentence, which goes on to qualify which data they are interested in (specifically "publicly available in a manner sufficient for validation and analysis.").

throwawayjava 8y ago
which de facto excludes 90% of medical research because medical patients have privacy rights and researchers have legal obligations to respect those rights. These rules are a hot political topic that are years in the making. The people pushing them are not scientists and have never expressed genuine interest in reproducibility outside of this one issue. And scientists who have spent years fighting for reproducibility have publicly stated they oppose this rule. And all the major proponents have strong anti-regulatory preferences. These rules are not about reproducibility. This rule is an intentionally crafted Catch-22 designed to exclude medical research from consideration so it's easier to justify repealing or not implementing environmental regulations. Otherwise, the rules would find a middle ground; e.g., by allowing the EPA access to relevant scientific data and analysis without risking https://en.wikipedia.org/wiki/De-anonymization https://en.wikipedia.org/wiki/De-anonymization by posting sensitive medical information about thousands of patients on the public internet.
- smsm42 8y ago
  It's very hard to believe it is not possible to publish medical data in an anonymized fashion that does not infringe privacy rights. If that's true, how people validate any medical research at all? Do they just ignore the validation, and if some research has an error - well, tough luck, we'd base policy on it anyway and if it turns out to be a disaster, we'd just shrug? Giving up on reproducibility and validation means giving up on one of the most important features of modern science, I have very hard time believing that's the only way to do modern medical research known to our civilization. > Otherwise, the rules would find a middle ground; They actually did, if you read the actual proposal, which talks about disclosure in ways that enough for validation but does not infringe privacy. But you seem to have arrived to pre-determined conclusion already, and there in fact can be no middle ground that would be acceptable to you, as it seems.
  - throwawayjava 8y ago
    > It's very hard to believe it is not possible to publish medical data in an anonymized fashion that does not infringe privacy rights Then you're one of today's lucky 10,000: "Robust De-anonymization of Large Sparse Datasets" http://www.cs.utexas.edu/~shmat/shmat_oak08netflix.pdf http://www.cs.utexas.edu/~shmat/shmat_oak08netflix.pdf A lot of environmental medical research results in dense and small data instead of sparse and large data, which obviously means it's even easier to de-anonymize. And the really important take away from the above paper is that you need to know about all the other public data in the universe in order to really ensure anonymity. Even when anonymity seems like it's "obviously" not going to be a problem. You never know what dataset will be leaked/published tomorrow correlating columns D-F in the spreadsheet you gave EPA to someone's first and last name. So privacy is like computer security: you can never be totally sure. So when the stakes are high you should be extra cautious. And the stakes when sharing medical research are very often high. > If that's true, how people validate any medical research at all?.. I have very hard time believing that's the only way to do modern medical research known to our civilization. This isn't exactly a new problem, so scientists have had a little while to figure out how to make peer review work. You share the data. You share with trusted, relevant third parties who promise to preserve the anonymity of study participants. Just becuase you do not publish possibly de-anonymizable and definitely sensitive medical information on a public website does not mean you don't share the data with interested and relevant third parties. > But you seem to have arrived to pre-determined conclusion already, and there in fact can be no middle ground that would be acceptable to you, as it seems. On the contrary, I do literally exactly that in my very first post in this thread: There are some common-sense workarounds. For example, requiring an EPA or third-party audit of the original dataset and analysis. Or simply carving out exceptions for research whenever it can be demonstrated that an IRB (and therefore federal law) would not have allowed the research without restricting public access to data/analysis. The situation where IRB says "no" to public disclosure should explicitly trigger exemption, with the duty falling back to the agency to prove the IRB was wrong in the first place. The fact that this exemption is not automatic is what makes this a catch-22 that pins researchers in-between their IRB and the EPA administrator in charge of evaluating their exception.