4 ms·
For subjects with a lot of data analysis you could largely automate a part of the validation as scientists use programming languages. Papers would have to prov
by aaren 13y ago
For subjects with a lot of data analysis you could largely automate a part of the validation as scientists use programming languages.
Papers would have to provide access to their raw data and the code used to process it. You could then just run code(data) to generate the figures / results in the paper.
Two problems here:
1) assumes that the raw data is ok (which is a big assumption).
2) existing scientific code is largely terrible (completely imperative, no abstraction, no documentation, poor typing conventions), and it is very unlikely you can do
figures <- code(data)
to regenerate a paper.
This is what forced validation would aim to change though.
- a-priori 13y agoThat only proves that their analysis is right. It doesn't rule out the possibility that their results are caused by a quirk of their implementation. To really validate a paper you have to be able to recreate their code from a high-level description, then check that.
- aaren 13y agoI agree, but this would give you a starting point in that you could see exactly what steps were used to create the figures in a paper. It might be that their high level description (usually in papers anyway) is correct but their implementation is flawed in some subtle way that peer review doesn't pick up. Assuming a correct implementation this would be useful for anyone wanting to use the methods of the paper. For example, I've spent the last 3 weeks coding a numerical solver for some equations in a fluid mechanics paper. Having the code for this available would have saved me figuring out all the quirks of the solution.