30 ms·

I've been debating about whether I want to open-source the scientific code I've been writing. A lot of it could be useful to other people in the molecular dyna

I've been debating about whether I want to open-source the scientific code I've been writing. A lot of it could be useful to other people in the molecular dynamics field. I recently introduced my advisor to Github, and he thought it was a good idea; however, there were a few hesitations. The first, most importantly, is the likeliness of a bug. If you put your code on a very public website like Github, there is a chance it's going to be scrutinized by everyone in your field. Now, unless you are one of the best programmers who has ever lived, there are bound to be bugs in your software, and when someone discovers them, it could have a deleterious effect on any journal articles you've written that used that code. The issue is that even though most bugs do not lead to significant changes in results, you would still need to redo all of your data to make sure that is the case. The software industry has long recognized buggy software as a reality, but I don't think the scientific community is as tolerant of it (hence the reason a lot of people hide their code). For my MD simulations, I use the well-known LAMMPS package. Bugs in it are discovered all the time! (http://lammps.sandia.gov/bug.html http://lammps.sandia.gov/bug.html). So I think there needs to be a collective realization among the scientific community that these are bound to occur and authors of journal articles can't be persecuted all the time for it. A lot of computational work is the art of approximation so I would just lump "human incompetency" under one of those approximation factors. Despite this risk, I think I'm still going to release my code at some point as I would personally welcome critique and improvement suggestions. I'd like to think I'm a better coder than most scientists since I've been coding since I was twelve in multiple language paradigms and have won a major hackathon, but eh, who knows. I'm quite sure my environment isn't up to industry standards because I've always coded solo rather than in a team.

plg 14y ago
holy moly. It sounds like you don't want to release your code because someone else might find a bug that you missed (and that might have a "deleterious" effect on your journal articles). You would rather leave potentially incorrect work standing than have a bug corrected? What the f*ck has happened to science.
- dantheman 14y ago
  Yeah, this is what make me fed up with academia -- it's no longer really about the work. I like Feynman's quote "It's a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty--a kind of leaning over backwards. For example, if you're doing an experiment, you should report everything that you think might make it invalid--not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you've eliminated by some other experiment, and how they worked--to make sure the other fellow can tell they have been eliminated." [1] [1] http://neurotheory.columbia.edu/~ken/cargo_cult.html http://neurotheory.columbia.edu/~ken/cargo_cult.html
- nagrom 14y ago
  What the fuck has happened to science. This is fairly standard in the science industry (I use the term deliberately). I get rated (in nuclear and particle physics) on how many papers I've published, what indictions of recognition I get from my peers and at which conferences I've been invited to speak. Short of something like a Millennium Prize or a Nobel (one-in-a-million, and both still very political), there are almost no direct rewards to accuracy or importance. Academics tend not to be the people with the most understanding or the best direct insight - they're the people with the most friends on committees and who do the most fashionable things (usually badly). And so they appoint new academics who are a little bit worse than them (no point in opening yourself to competition - instead bring in people who are going to be grateful to you!) and the cycle continues.
  - michaelbarton 14y ago
    I strongly agree with your statement. I am the author of the original post. I removed a sentence stating that job search committees care about the number of publications and citations more than they do about if your software is well documented and has a good command line interface. I feel as a post doc I am in a vulnerable career position and I always have to think about how 'will this lead to a publication' before anything else.
    - 14y ago
michaelbarton 14y ago
I have a feeling that there is great deal of buggy software in biology. Taking non-scientific software for example such as Rails, if there is a bug this becomes obvious a page is loaded incorrectly or a model is pulled incorrectly from the database. In contrast imagine an academic scenario where I'm testing a hypothesis using someone else's software. How do I notice a bug? I'll notice the bugs where the output is in the wrong format for example. More subtle bugs I won't notice because I don't have any expectation on the results. This will affect the conclusion I draw though.
pjscott 14y ago
Suppose that your code does become widely visible. How is that a bad thing for you? Your reputation will be enhanced, you will have contributed something that other people find useful, and if there are serious bugs affecting your results, it'll be nice to find those. You might want to automate the process of re-running your analysis, though. It's a good idea in general, and especially if you anticipate needing to make minor tweaks to the software involved.
- michaelbarton 14y ago
  I think that is a big 'if.' If I create one of the pieces of bioinformatics software that becomes widely used then this is great for my career. However I believe a large majority of research software goes unused and therefore uncited. The extra effort I made to create a website and documentation is therefore time I could have spent more usefully on creating another publishable unit.

plg 14y ago

holy moly. It sounds like you don't want to release your code because someone else might find a bug that you missed (and that might have a "deleterious" effect on your journal articles). You would rather leave potentially incorrect work standing than have a bug corrected? What the f*ck has happened to science.

dantheman 14y ago
Yeah, this is what make me fed up with academia -- it's no longer really about the work. I like Feynman's quote "It's a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty--a kind of leaning over backwards. For example, if you're doing an experiment, you should report everything that you think might make it invalid--not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you've eliminated by some other experiment, and how they worked--to make sure the other fellow can tell they have been eliminated." [1] [1] http://neurotheory.columbia.edu/~ken/cargo_cult.html http://neurotheory.columbia.edu/~ken/cargo_cult.html
nagrom 14y ago
What the fuck has happened to science. This is fairly standard in the science industry (I use the term deliberately). I get rated (in nuclear and particle physics) on how many papers I've published, what indictions of recognition I get from my peers and at which conferences I've been invited to speak. Short of something like a Millennium Prize or a Nobel (one-in-a-million, and both still very political), there are almost no direct rewards to accuracy or importance. Academics tend not to be the people with the most understanding or the best direct insight - they're the people with the most friends on committees and who do the most fashionable things (usually badly). And so they appoint new academics who are a little bit worse than them (no point in opening yourself to competition - instead bring in people who are going to be grateful to you!) and the cycle continues.
- michaelbarton 14y ago
  I strongly agree with your statement. I am the author of the original post. I removed a sentence stating that job search committees care about the number of publications and citations more than they do about if your software is well documented and has a good command line interface. I feel as a post doc I am in a vulnerable career position and I always have to think about how 'will this lead to a publication' before anything else.
  - 14y ago

michaelbarton 14y ago

I have a feeling that there is great deal of buggy software in biology. Taking non-scientific software for example such as Rails, if there is a bug this becomes obvious a page is loaded incorrectly or a model is pulled incorrectly from the database. In contrast imagine an academic scenario where I'm testing a hypothesis using someone else's software. How do I notice a bug? I'll notice the bugs where the output is in the wrong format for example. More subtle bugs I won't notice because I don't have any expectation on the results. This will affect the conclusion I draw though.

pjscott 14y ago

Suppose that your code does become widely visible. How is that a bad thing for you? Your reputation will be enhanced, you will have contributed something that other people find useful, and if there are serious bugs affecting your results, it'll be nice to find those. You might want to automate the process of re-running your analysis, though. It's a good idea in general, and especially if you anticipate needing to make minor tweaks to the software involved.

michaelbarton 14y ago
I think that is a big 'if.' If I create one of the pieces of bioinformatics software that becomes widely used then this is great for my career. However I believe a large majority of research software goes unused and therefore uncited. The extra effort I made to create a website and documentation is therefore time I could have spent more usefully on creating another publishable unit.