4 ms·

I am a research scientist published via R, Stata, and Excel analyses. My code documents wouldn't be helpful since the data is all locked up due to HIPAA concern

I am a research scientist published via R, Stata, and Excel analyses. My code documents wouldn't be helpful since the data is all locked up due to HIPAA concerns. We're talking names, health conditions, scrambled SSN, this isn't reproducible because the data is locked to those without security clearance. The code itself is a ton of munging and then some basic stat functions. This information can be gleaned from the methods section of the article anyway. So, really, my field of public health doesn't use GitHub or sharing much, there's simply too little benefit to the researcher to share their code. There's an unwarranted fear of getting your work poached. In modern science, publications are everything, they determine your career. Enabling your direct competitors, those who want the same grants and students and glories, is not common in science.

asdf_snar 6y ago
I don't disagree with you on any points. I have some academic friends who mostly do "a ton of munging and then some basic stat functions", as you say (but with less sensitive data). The problem is that their workflow is prone to human error. Even though the stat functions are simple, the proper labeling of inputs and outputs is less reliable. I have some research published for which I wrote MATLAB code years ago. I trust the fundamental results but not the values displayed in the tables. I would have personally benefited from rudimentary version control and unit testing.
stult 6y ago
In well designed software, data ingestion should be easily separable from the core logic of the application. Which is the point the parent comment is making. Some basic best practices would allow you to share your core code without implicating HIPAA. Even if it’s just basic stats, sharing the code makes it easier to reproduce your results and to check your logic. Although I agree with your analysis that enabling competitors in science is not common, it really, really should be. That’s kinda the point of publication, at least in theory. Sharing knowledge and methods.
- jimmyvalmer 6y ago
  > enabling competitors in science ... really, really should be. Said someone whose livelihood doesn't depend on said competition.
vharuck 6y ago
As a public health statistician, I am very grateful to all researchers who publish code. More so for those who publish packages that make their techniques easy to use. I am not an expert in the field of statistics, just a grunt applying what you guys devise. It takes a while for me to do enough research and testing to be sure I'm correctly implementing new techniques. Even a basic pseudo-code walkthrough would immensely help. >My code documents wouldn't be helpful since the data is all locked up due to HIPAA concerns. We're talking names, health conditions, scrambled SSN, this isn't reproducible because the data is locked to those without security clearance. Is there a standard format for this kind of data? If so, consider using it. That way, others can easily create artificial datasets to test it. Even if you have no control over your data source, you can convert the raw data to the standard as a "pre-munging" step. >So, really, my field of public health doesn't use GitHub or sharing much, there's simply too little benefit to the researcher to share their code. Sad but true.

asdf_snar 6y ago

I don't disagree with you on any points. I have some academic friends who mostly do "a ton of munging and then some basic stat functions", as you say (but with less sensitive data). The problem is that their workflow is prone to human error. Even though the stat functions are simple, the proper labeling of inputs and outputs is less reliable. I have some research published for which I wrote MATLAB code years ago. I trust the fundamental results but not the values displayed in the tables. I would have personally benefited from rudimentary version control and unit testing.

stult 6y ago

In well designed software, data ingestion should be easily separable from the core logic of the application. Which is the point the parent comment is making. Some basic best practices would allow you to share your core code without implicating HIPAA. Even if it’s just basic stats, sharing the code makes it easier to reproduce your results and to check your logic. Although I agree with your analysis that enabling competitors in science is not common, it really, really should be. That’s kinda the point of publication, at least in theory. Sharing knowledge and methods.

jimmyvalmer 6y ago
> enabling competitors in science ... really, really should be. Said someone whose livelihood doesn't depend on said competition.

vharuck 6y ago

As a public health statistician, I am very grateful to all researchers who publish code. More so for those who publish packages that make their techniques easy to use. I am not an expert in the field of statistics, just a grunt applying what you guys devise. It takes a while for me to do enough research and testing to be sure I'm correctly implementing new techniques. Even a basic pseudo-code walkthrough would immensely help. >My code documents wouldn't be helpful since the data is all locked up due to HIPAA concerns. We're talking names, health conditions, scrambled SSN, this isn't reproducible because the data is locked to those without security clearance. Is there a standard format for this kind of data? If so, consider using it. That way, others can easily create artificial datasets to test it. Even if you have no control over your data source, you can convert the raw data to the standard as a "pre-munging" step. >So, really, my field of public health doesn't use GitHub or sharing much, there's simply too little benefit to the researcher to share their code. Sad but true.