3 ms·

So I have some grad/phd student friends and a small computational lab - enough cores, ram, storage and jobbing engines that in 2000s it would have been a respec

So I have some grad/phd student friends and a small computational lab - enough cores, ram, storage and jobbing engines that in 2000s it would have been a respectable not quite super computer but none the less very powerful system capable of crunching lots of numbers quickly. My friends sometimes try to validate conclusions of the papers or just re-run the data crunching presented in those papers. They never succeed. Either the data sets don't actually match the data sets in papers, or something in the formulas is off, or the code does not even compile. I did computation physics in college. We had to write code as the paper of homework/tests. Our TA would never run our binaries. He would take our source code, compile it, feed it two of his data sets (we had one) and check the result. If the student's code did not compile, the student would get an F for that homework. If the student's code bombed on the data set that was shared with students, the student would automatically get an F. And only if the second data set got different results than he expected would he go chasing the errors and potentially still give a good grade. By the end of two semesters everyone knew one never tweaked the code to just run the known data set and get the known result. Science would benefit tremendously if this was the approach.

nonbel 8y ago
>"Science would benefit tremendously if this was the approach." This is just the original idea behind science as developed in the renaissance. The resistance to checking each others work is a recent phenomenon. Edit to add a source: > "the foundation of scientific societies compelled scientists to change the way in which experiments were presented so that not only members of the scientific community but also the inexperienced reader would be able to repeat them by following the same steps. [...] Robert Boyle also played an important role in the evolution of scientific writing (Gotti,1996, 2001; Hunter, 2007). Fulton (1932: 78) has highlighted the contribution of the scientist, describing Boyle’s undertaking “to establish science as an integral part of the intellectual life of ordinary men”. He encouraged scientists to write in a language that their contemporaries could understand, and he undoubtedly did more than anyone of his time to make scientific methodology part of the intellectual equipment of educated men. [...] According to Gotti, one of the features of experimental essays is brevity, but this brevity of form does not imply brevity in the exposition of the subject. On the contrary, experiments are usually reported in full, furnishing the reader with all the minutest details. The aim of such a report is, after all, to make it possible for the experiment to be repeated, making the precision of the description an important factor." https://doi.org/10.14198/raei.2007.20.06 https://doi.org/10.14198/raei.2007.20.06

nonbel 8y ago

>"Science would benefit tremendously if this was the approach." This is just the original idea behind science as developed in the renaissance. The resistance to checking each others work is a recent phenomenon. Edit to add a source: > "the foundation of scientific societies compelled scientists to change the way in which experiments were presented so that not only members of the scientific community but also the inexperienced reader would be able to repeat them by following the same steps. [...] Robert Boyle also played an important role in the evolution of scientific writing (Gotti,1996, 2001; Hunter, 2007). Fulton (1932: 78) has highlighted the contribution of the scientist, describing Boyle’s undertaking “to establish science as an integral part of the intellectual life of ordinary men”. He encouraged scientists to write in a language that their contemporaries could understand, and he undoubtedly did more than anyone of his time to make scientific methodology part of the intellectual equipment of educated men. [...] According to Gotti, one of the features of experimental essays is brevity, but this brevity of form does not imply brevity in the exposition of the subject. On the contrary, experiments are usually reported in full, furnishing the reader with all the minutest details. The aim of such a report is, after all, to make it possible for the experiment to be repeated, making the precision of the description an important factor." https://doi.org/10.14198/raei.2007.20.06 https://doi.org/10.14198/raei.2007.20.06