5 ms·
> “I had and was very willing to share the code,” he says. What he didn’t have, he says, was the time or personnel to prepare the code in a form that could be u
by bigiain 8y ago
> “I had and was very willing to share the code,” he says. What he didn’t have, he says, was the time or personnel to prepare the code in a form that could be useful to an outsider.
That's what this license is for: http://matt.might.net/articles/crapl/ http://matt.might.net/articles/crapl/
- itronitron 8y agoConsidering that a paper is run through a subjective and highly critical peer-review I can see why authors would be reluctant to share their code.
- bigiain 8y agoIt's an interesting observation then that the people doing the "highly critical peer review" might promote papers with no source code showing how the calculations were done over papers with messy-but-available source code. I know some fields intentionally/explicitly hide their working ("Law and sausage are two things you do not want to see being made."), but I wouldn't have thought academia would subscribe to that... There's a lot to be said for the IETF's policy of "rough consensus and running code".
- itronitron 8y agoThe paper should show how the calculations can be made, along with the results of that calculation, preferably on a publicly available dataset. A common strategy is to describe the calculation in highly convoluted or specialized language (mathematical formulas) in order to reduce the number of people that may try to reproduce the work. Having reviewed papers on multiple occasions, I wouldn't spend much if any time looking at the actual source code if it is available. Every paper submission is rushed because it needs to be ahead of what anyone else is doing (at least in CS and related fields).
- bigiain 8y agoDo you see this as an inevitable "that's the way it is" part of how the peer review process works? Or as a flaw in the process? Or something in between. (A deeply cynical and suspicious part of me suspects this is because the prime motivator for "peer reviewed journals" is to make Elsiver richer, and academic knowledge sharing and reputation building is just a side effect of that...)
- itronitron 8y agoElsevier is like the cocktail lounge of the alcohol industry. They are just running a business by controlling access to the good stuff, that people with money have a taste for, and they will keep prices high for as long as they can. To be fair, some domains really need the good stuff and the cost for access to those journals is probably a drop in the bucket relative to their other operating expenses (or marketing expenses if we are talking about the pharmaceutical industry.) Every peer review process is different from the next, it varies by journal, by conference, and by month of the year. I probably can't emphasize enough how highly subjective every peer review is. Anyone that has a paper rejected should consider the validity of reviewer comments (which may be total BS on occasion), then edit and resubmit the paper to another venue.
- nikanj 8y agoOnce you release code, people googling you will find that code - and use it as a measuring stick of your coding skills. In todays rockstar-obsessed recruiting hell, I would not want any quick-and-dirty hacks published under my name.
- ur-whale 8y agoJust say one of our grad students implemented your ideas. Done.
- adrianN 8y agoCondensed matter theorists usually don't look for a job that measures them on the quality of their research code.
- nikanj 8y agoTenure track is a cruel funnel, and if you don't make the cut, it's time to find a job outside of academia.