3 ms·
Back when I was active in academia, our publishers were reluctant to print source code or even repository links (that was largely before GitHub) but they could
by okaleniuk 3y ago
Back when I was active in academia, our publishers were reluctant to print source code or even repository links (that was largely before GitHub) but they could still share a paper source on demand. If you reference someone else's paper and want to quote some formula, it is easier and less error prone to copy rather than retype.
At that point I thought about making a TeX interpreter so one could easily "run a paper" on their own data to see if the papers claims hold. As it turned out, people often write the same formula in multiple ways and to make a TeX interpreter you'd have to specify a "runnable" subset and convince anyone to use that subset instead of what they got used to. So the idea stalled.
In a few years, publishing a GitHub link along the paper became the norm, and the problem disappeared. At least in applied geometry, people do replicate each other results all the time.
- kjkjadksj 3y ago“Running a paper” is honestly challenging these days because of the resource requirements of a lot of scientific code, or the size of certain datasets. One group might have access to a beefy cluster and there’s no pressure for very performant code when they can parallelize the work across a few dozen xeons or have access to tbs of memory. Another group might be running their code on a laptop. Maybe if your data is much larger than the authors data, their code doesn’t even work since it was designed for much smaller datasets. Tools like nextflow or snakemake help with respect to having a one liner to generate all data in a paper potentially, handle dependencies, list resource expectations, use your own profile to handle your environment specific job scheduling commands and parameters. However, this still doesn’t do anything for whether you have access to the resources needed.