2 ms·
If I claim, "An implementation of this algorithm (described in pseudocode) achieves O(N) peformance on that task" and no one can reproduce it based on the pseud
by tjradcliffe 11y ago
If I claim, "An implementation of this algorithm (described in pseudocode) achieves O(N) peformance on that task" and no one can reproduce it based on the pseudocode, I've published a lie.
This is a phenomenon I've seen many times in my work as a consultant translating academic discoveries to commercial code. This work invovles digging through both published papers and actual code to find that some grad student has implemented a clever heuristic that is not described in the paper because it is "inessential" that is what the claimed performance actually relies on.
The article says, "It's unrealistic to think contract labs or university core facilities can get the same results as a highly specialized team of academic researchers, they say."
I can understand this sentiment: science is hard, and at the cutting edge it's really hard. I've certainly got results that I think it would be hard for others to achieve and I've succeed with experiments where others have failed simply because I have good hands.
I once took over an experiment that had been worked on by a heavy smoker with shaky hands. He had developed a complicated apparatus for streching ultra-thin plastic foils over a frame. I just used my fingers. If I had published the way I did it and someone like him had tried to reproduce it, they would have failed.
However... the kind of reproducibilty being talked about here is not (primarily) technique, it is results. If a result can't be reproduced because the cell line was so very special or whatever, then it likely isn't robust enough to be very interesting in practical biology. It certainly isn't of much interest in drug discovery. So while the science may be OK, results that are incredibly hard to reproduce are of very low value for biotech, and that knowledge is worth something in and of itself.