4 ms·
Sure I am. There's general disrespect in academia for work that is incompletely reported in a way that hinders review and reproduction. It's a historical acci
by dkarl 16y ago
Sure I am. There's general disrespect in academia for work that is incompletely reported in a way that hinders review and reproduction. It's a historical accident that refusing to publish source code has come to be treated as a special exception to the principles of good scientific practice. The more the basis for that exception is undermined, the more justified researchers will feel in demanding source code when they're skeptical of a result.
And here's the key incentive: lack of source code can be used as an excuse for ignoring and devaluing competitors' contributions. Scientists may be reluctant to raise a standard that would result in more work for everybody, but they can't resist a means of denigrating their rivals' work. Once the cat is out of the bag, scientists with weak reputations and disruptive results will be forced to publish source code to prevent their work being round-filed. Established scientists will find it untenable to hold themselves to an explicitly lower standard than everybody else, so they will eventually be forced to start publishing their code, too.
- scott_s 16y agoIt's not a refusal, it's a neglect. What you're saying is, "If the culture was different, the incentives would be there." But the culture is not different. You can't just wave a magic wand. Changing culture is hard.
- dkarl 16y agoI explained how the culture will change. It will change because academics will use failure to release source code to throw doubt on the results of their rivals, and people (especially those who aren't already established in their fields) will answer that criticism by releasing code, which will raise the standards for everybody else. There's disincentive to be first, because everybody will pretend to be shocked at the buggy, undocumented code, but not everyone writes bad code and not everyone will feel able to refuse. You want me to actually change the culture right now in a comment thread on Hacker News? I think predicting how it will happen is the best I can do, sorry.
- scott_s 16y agoYou explained the manner in which it would change, not why it would change. That's the hard problem. If that's not clear, you did not explain why academics would "use failure to release source code to throw doubt on the results of their rivals" when they do not do that now. I don't disagree with you that this would be preferable to the status-quo, nor am I disagreeing that it would be sustainable once the change occurred. I just don't see any reason why the change will occur in the first place.
- chc 16y agoI think you're talking past each other. As I read it, dkarl is saying that we should encourage academics to do this. That is the implicit reason why they would do it — because we're pushing them to.