5 ms·
I'm under no illusion that it's inherently easy (I consider such to be my job and that's partly why I get money), but that's why one should be 'professional' an
by ACow_Adonis 7y ago
I'm under no illusion that it's inherently easy (I consider such to be my job and that's partly why I get money), but that's why one should be 'professional' and have to work for it.
If you make a calculation, you should have processes to verify if that calculation or result is correct, that you can repeat that calculation, think about why it might be wrong even if you think it's right etc.
If your technique relies on sorting things in a particular order, then you both explicitly sort them in that order, and then you figure out an independent method to verify they are sorted in that order.
Indeed in one of my jobs that was a bug: old bank code where sql statements were inherently assumed to be in a consistent/stable-sorted order for downstream processing because they used an 'order by' clause. When the vendor upgraded the sql engine to be distributed/multithreaded, the relative order inside equal keys changed (and indeed was now non-deterministic when it was technically deterministic at the time the code was written even though that was an assumption not a requirement).
But I don't think I'm unreasonable: yes, there is no black and white rule for how far down the rabbit-hole you go: is it too far to test each individual instance of integer arithmetic, or floating point calcs, or individual os's or processors... in practice probably yes. But you do need higher-level tests to determine if calcs are going wrong (that might then later start you down that route when your results suggest they are going wrong). And yes, there are bugs that are inherently difficult to track down, especially as you go up and up in complexity and abstraction.
And there are parts of science where the feasibility of the calculation itself or its outcome is the great unknown. But these are edge cases and not 99% of science papers.
What's alarming is the reuse of code they don't inspect, the use of libraries and techniques they don't understand, and practically no critical thinking or inspection towards their data, results or methods.
They throw stats or a library or a technique at a wall, get a positive result and: bang! into (at least one) paper.
In my eyes that's not how science should work.
I'll also go one step further and say that if you have unexplained bugs, warnings, errors, and if your calculations are not reproducible, then for 99% of science you should not be publishing just because most of the time when you run it you get a positive result. Would it suck if that one in a million obscure machine specific problem happens when producing your paper? Sure. Would you be negligent in publishing anyway without a way to prove or assert the correctness your calculation? Yes.
Is that what everyone's doing? Hell, I think it's orders of magnitude worse. Some languages (like R) are almost built around the philosophy of 'the show must go on' rather than 'be correct or no result for you'. I think that's part of the reason why they're popular.
Again, I don't think that's software engineering specific, I think it's a mindset that is required to make a good scientist.
- BlueTemplar 7y agoCould you please tell us more about this R 'the show must go on' philosophy?
- ACow_Adonis 7y agoIn short, R contains silent coercions, vector recycling, lazy evaluation, partial matching on strings in various calls, exceptions to the rules, surprises, unexpected values, inconsistent and weird operations on edge cases and often chooses to continue on with these incorrect and bad values rather than throw errors and stop computation. I'm not a fan of its writing style (there's a point where being funny hinders readability), but I believe chapter 8 of the R Inferno is replete with examples. The combination of these factors results in a language where actually trying to reason about the veracity of a computation and coding defensively is exceptionally difficult, but people can load packages and make the wrong computations that look right exceptionally quickly, so they think of it as convenient.
- BlueTemplar 7y agoThanks, bookmarking this answer...