5 ms·
The problem is more complex. Some errors will damage the scientific conclusions, others will show up as fractions of a percentage point inside a much larger con
by alextp 15y ago
The problem is more complex. Some errors will damage the scientific conclusions, others will show up as fractions of a percentage point inside a much larger confidence interval, and will ultimately not matter much.
In machine learning research, for example, most evaluations consist of running a new program on soem data, getting results back, and from these results computing some aggregate measure of performance. A bug on the code that computes this measure of performance is _really bad_ and can invalidate all your conclusions. If that code is right, however, a but on the code that trains your model is completely meaningless, because as long as your results are good you can argue that you actually meant to write a paper about the model actually implemented rather than the model you were supposed to implement.
I'm sure other scientific areas have similar distinctions, and a naive code reader might fail to notice that a bug is harmless (and there's also the fact that scientific code carries within it a lot of assumptions about the data which, if broken, can be buggy, but are not broken by real data).