4 ms·
> "How is this different from any other academic research?" Well it's not, although CS should be particularly amenable to reproducibility. > "He is asking tha
by polemic 10y ago
> "How is this different from any other academic research?"
Well it's not, although CS should be particularly amenable to reproducibility.
> "He is asking that researchers produce code that he can put into production"
No, he asked for code [full stop]. "Because CS researchers don't publish code or data. They publish LaTeX-templated Word docs as paywalled PDFs."
> "The way to use academic research results is to (1) read and understand the paper, (2) reproduce the code -- ideally, from scratch, so that his results are (hopefully) unaffected by authors' bugs, (3) verify on a test problem, and (4) apply to his data."
He wants to re-run the authors analysis with new data. He's not looking to recreate the research from scratch or publish a new paper. Saying that this is the only valid usage of the results is awfully short sighted. It misses the point that the research has use beyond usage by other researchers.
Imagine if rather than open source software, we published the results of our new modules and told potential collaborators to build it from scratch to verify the implementation first. You'd learn a lot about building that piece of software, but you've missed an enormous opportunity along the way.
- mcguire 10y ago"He wants to re-run the authors analysis with new data. He's not looking to recreate the research from scratch or publish a new paper. Saying that this is the only valid usage of the results is awfully short sighted. It misses the point that the research has use beyond usage by other researchers." That sort of thing has been done for years in the scientific computing world. The end result is that you are making decisions based on code that may have worked once (Definition of 'academic code': it works on the three problems in the author's dissertation. There's production code, beta code, alpha code, proofs-of-concepts, and academic code.) but that you have no reason to trust on other inputs and no reason to believe correct. Case in point: I had lunch a while back with someone whose job was to run and report the results of a launch vehicle simulation. Ya know, rocket science. It required at least one dedicated human since getting the parameters close to right was an exercise in intuition. Apparently someone at Georgia Tech wanted to compare results so they got a copy. The results, however, turned out to be completely different because they had inadvertently given the researchers a newer version of the code.