3 ms·

I think the bigger reason it's false in science while somewhat more true in CS is that it's basically free to check the source code for something and setup a sm

by rtkwe 7y ago

I think the bigger reason it's false in science while somewhat more true in CS is that it's basically free to check the source code for something and setup a small localhost test where replication in science is tough and expensive. Even if someone does have the funds to check experimental descriptions don't actually cover all the details usually and there's some combination of small but important details left out and well meaning but imperfect directions. Also there's just the fact that in CS we're working on a defined system. From top to bottom it's possible to definitively say when I execute X the following Y things happen. The details may be buried deep within the hardware for the truly eldritch bugs but fundamentally it's all human defined and the bits that do exist outside human design are well understood enough they can largely be ignored (no one is reasonably going to find a bug that traces down to some unknown quirk of semiconductor physics). In the sciences we're poking at the edges to see what the system actually is.

btrettel 7y ago
While I agree that checking source code and running software is relatively easier than running experiments, there's more to science than experiments. Many wrong things in science are even easier to identify than the "easy" case you identify, but you still don't see them checked. In fact, the example I gave was for a math error that should be much easier to check than looking at someone's software or source code. All you needed to do was go through the claimed derivation to see that something was off. No fancy math was involved either, just algebra. I think problems like this are much more common than is appreciated.
- rtkwe 7y ago
  Yeah the issue is probably partially weird results can be lazily explained as a number of sources; genuinely new science, experimental design problems, or equipment errors to list a few, so if you're quickly reviewing a paper it's easy to just gloss over checking all the math because it's easy to trust on that. With computers we know exactly what we expect a piece of code to do and if it doesn't we're not discovering some new truth about the universe it's just that something was done incorrectly or our assumptions were wrong if it's someone else's code/program/library.

by rtkwe 7y ago

btrettel 7y ago

While I agree that checking source code and running software is relatively easier than running experiments, there's more to science than experiments. Many wrong things in science are even easier to identify than the "easy" case you identify, but you still don't see them checked. In fact, the example I gave was for a math error that should be much easier to check than looking at someone's software or source code. All you needed to do was go through the claimed derivation to see that something was off. No fancy math was involved either, just algebra. I think problems like this are much more common than is appreciated.

rtkwe 7y ago
Yeah the issue is probably partially weird results can be lazily explained as a number of sources; genuinely new science, experimental design problems, or equipment errors to list a few, so if you're quickly reviewing a paper it's easy to just gloss over checking all the math because it's easy to trust on that. With computers we know exactly what we expect a piece of code to do and if it doesn't we're not discovering some new truth about the universe it's just that something was done incorrectly or our assumptions were wrong if it's someone else's code/program/library.