5 ms·

That's exactly the philosophy we follow e.g. in particle physics and its a common excuse to dismiss all guidelines made in the article. However, this kind of v

That's exactly the philosophy we follow e.g. in particle physics and its a common excuse to dismiss all guidelines made in the article. However, this kind of validation/falsification is often done between different research groups (maybe using different but formally equivalent approaches) while people within the same group have to deal with the 10 years old code base. I myself had very bad experience with extending the undocumented Fortran 77 code (lots of gotos and common blocks) of my supervisor. Finally, I decided to rewrite the whole thing including my new results instead of just somehow embedding my results into the old code for two reasons: (1) I'm presumably faster in rewriting the whole thing including my new research rather than struggling with the old code and (2) I simply would not trust in the numerical results/phenomenology produced by the code. After all, I'm wasting 2 months of my PhD for the marriage of my own results with known results which -in principle- could have been done within one day if the code base would allow for it. So yes, If it's a one-man-show I would not give too much on code quality (though unit tests and git can safe quite a lot of time during development) but if there is a chance that someone else is going to touch the code in near future it will save time to your colleagues and improve the overall (scientific) productivity. PS: quite excited about my first post here

MaxBarraclough 6y ago
> If it's a one-man-show I would not give too much on code quality This makes me a little uneasy, as I'm not too worried about code quality can easily translate into Yes I know my code is full of undefined behaviour, and I don't care. > PS: quite excited about my first post here Welcome to HN! reddit has more cats, Slashdot has more jokes about sharks and laserbeams, but somehow we get by.
- ByteJockey 6y ago
  Are we talking actual undefined behavior or just behavior that's undefined by the language standard? The latter isn't great practice, but if your environment handles behavior deterministically, and you publish the version of the compiler you're using, it doesn't seem to be a problem for this type of code.
  - MaxBarraclough 6y ago
    > Are we talking actual undefined behavior or just behavior that's undefined by the language standard? 'Undefined behaviour' is a term-of-art in C/C++ programming, there's no ambiguity. > if your environment handles behavior deterministically, and you publish the version of the compiler you're using, it doesn't seem to be a problem for this type of code. Code should be correct by construction, not correct by coincidence. Results from such code shouldn't be considered publishable. Mathematicians don't get credit for invalid proofs that happen to reach a conclusion which is correct. Again, this isn't some theoretical quibble. There are plenty of sneaky ways undefined behaviour can manifest and cause trouble. [0][1][2] In the domain of safety-critical software development in C, extreme measures are taken to ensure the absence of undefined behaviour. If scientists adopt a sloppier attitude toward code quality, they should expect to end up publishing invalid results. Frankly, this isn't news, and I'm surprised the standards seem to be so low. Also, of all the languages out there, C and C++ are among the most unforgiving of minor bugs, and are a bad choice of language for writing poor-quality code. Ada and Java, for instance, won't give you undefined behaviour for writing int i; int j = i;. [0] https://devblogs.microsoft.com/oldnewthing/20140627-00/?p=633 https://devblogs.microsoft.com/oldnewthing/20140627-00/?p=63... [1] https://blog.regehr.org/archives/213 https://blog.regehr.org/archives/213 [2] https://cryptoservices.github.io/fde/2018/11/30/undefined-behavior.html https://cryptoservices.github.io/fde/2018/11/30/undefined-be... See also my longer ramble on this topic at https://news.ycombinator.com/item?id=24264376 https://news.ycombinator.com/item?id=24264376
    - kazagistar 6y ago
      I think its poor practice, but undefined behavior shouldn't instantly invalidate results. In fact, this mindset is what keeps people from publishing the code in the first place. Let the scientists publish UB code, and even the artifacts produced, the executables. Then, if such problems are found in the code by professionals, they can investigate it fully and find if it leads to a tangible flaw that invalidates the research or not. You would drive yourself mad pointing out places in math proofs where some steps, even seemingly important ones, were skipped. But the papers are not retracted unless such a gap actually holds a flaw that invalidates the rest of thr proof. Let thdm publish their gross, awful, and even buggy code. Sometimes the bugs don't effect the outcomes.
jnxx 6y ago
> After all, I'm wasting 2 months of my PhD for the marriage of my own results with known results which -in principle- could have been done within one day if the code base would allow for it. Sounds like it is quite good science to do that, because it puts the computation on a pair of independent feet. Otherwise, it could just be that the code you are using as a bug and nobody notes until it is too late.
- martingab 6y ago
  I see your and MaxBarraclough concerns. In my case, there exist 5-6 codes which do -at their core- the same thing as ours does and they all have been cross-checked against each other within either theoretical or numerical precision (where possible). That's the spirit that sjburt was referring to, I guess, and which triggered me because it is only true to a certain extend. The cross-checking is anyways good scientific practise, not only because of bugs in the code (that's actually a sub-leading problem imho), but because of the degree of difficulty of the problems and the complexity of their solutions (and their reproducibility). In that sense, cross-checking should discover both, scientific "bugs" and programming-bugs. The "debugging" is partly also done at the community level - at least in our field of research. However, it is also a matter of efficiency. I -and many others too- need to re-implement not because of bug-hunting/cross-checking but simply because we do not understand the "ugly" code of our colleagues and instead of taking the risk to break existing code we simply write new one which is extremely inefficient (others may take the risk and then waste months on debugging and reverse-engineering which is also inefficient). So my point on writing "good code" is not so much about avoiding bugs but about being kind to you colleagues, saving them nerves and time (which they can then spend on actual science) and thus also saving taxpayers money...

MaxBarraclough 6y ago

> If it's a one-man-show I would not give too much on code quality This makes me a little uneasy, as I'm not too worried about code quality can easily translate into Yes I know my code is full of undefined behaviour, and I don't care. > PS: quite excited about my first post here Welcome to HN! reddit has more cats, Slashdot has more jokes about sharks and laserbeams, but somehow we get by.

ByteJockey 6y ago
Are we talking actual undefined behavior or just behavior that's undefined by the language standard? The latter isn't great practice, but if your environment handles behavior deterministically, and you publish the version of the compiler you're using, it doesn't seem to be a problem for this type of code.
- MaxBarraclough 6y ago
  > Are we talking actual undefined behavior or just behavior that's undefined by the language standard? 'Undefined behaviour' is a term-of-art in C/C++ programming, there's no ambiguity. > if your environment handles behavior deterministically, and you publish the version of the compiler you're using, it doesn't seem to be a problem for this type of code. Code should be correct by construction, not correct by coincidence. Results from such code shouldn't be considered publishable. Mathematicians don't get credit for invalid proofs that happen to reach a conclusion which is correct. Again, this isn't some theoretical quibble. There are plenty of sneaky ways undefined behaviour can manifest and cause trouble. [0][1][2] In the domain of safety-critical software development in C, extreme measures are taken to ensure the absence of undefined behaviour. If scientists adopt a sloppier attitude toward code quality, they should expect to end up publishing invalid results. Frankly, this isn't news, and I'm surprised the standards seem to be so low. Also, of all the languages out there, C and C++ are among the most unforgiving of minor bugs, and are a bad choice of language for writing poor-quality code. Ada and Java, for instance, won't give you undefined behaviour for writing int i; int j = i;. [0] https://devblogs.microsoft.com/oldnewthing/20140627-00/?p=633 https://devblogs.microsoft.com/oldnewthing/20140627-00/?p=63... [1] https://blog.regehr.org/archives/213 https://blog.regehr.org/archives/213 [2] https://cryptoservices.github.io/fde/2018/11/30/undefined-behavior.html https://cryptoservices.github.io/fde/2018/11/30/undefined-be... See also my longer ramble on this topic at https://news.ycombinator.com/item?id=24264376 https://news.ycombinator.com/item?id=24264376
  - kazagistar 6y ago
    I think its poor practice, but undefined behavior shouldn't instantly invalidate results. In fact, this mindset is what keeps people from publishing the code in the first place. Let the scientists publish UB code, and even the artifacts produced, the executables. Then, if such problems are found in the code by professionals, they can investigate it fully and find if it leads to a tangible flaw that invalidates the research or not. You would drive yourself mad pointing out places in math proofs where some steps, even seemingly important ones, were skipped. But the papers are not retracted unless such a gap actually holds a flaw that invalidates the rest of thr proof. Let thdm publish their gross, awful, and even buggy code. Sometimes the bugs don't effect the outcomes.

jnxx 6y ago

> After all, I'm wasting 2 months of my PhD for the marriage of my own results with known results which -in principle- could have been done within one day if the code base would allow for it. Sounds like it is quite good science to do that, because it puts the computation on a pair of independent feet. Otherwise, it could just be that the code you are using as a bug and nobody notes until it is too late.

martingab 6y ago
I see your and MaxBarraclough concerns. In my case, there exist 5-6 codes which do -at their core- the same thing as ours does and they all have been cross-checked against each other within either theoretical or numerical precision (where possible). That's the spirit that sjburt was referring to, I guess, and which triggered me because it is only true to a certain extend. The cross-checking is anyways good scientific practise, not only because of bugs in the code (that's actually a sub-leading problem imho), but because of the degree of difficulty of the problems and the complexity of their solutions (and their reproducibility). In that sense, cross-checking should discover both, scientific "bugs" and programming-bugs. The "debugging" is partly also done at the community level - at least in our field of research. However, it is also a matter of efficiency. I -and many others too- need to re-implement not because of bug-hunting/cross-checking but simply because we do not understand the "ugly" code of our colleagues and instead of taking the risk to break existing code we simply write new one which is extremely inefficient (others may take the risk and then waste months on debugging and reverse-engineering which is also inefficient). So my point on writing "good code" is not so much about avoiding bugs but about being kind to you colleagues, saving them nerves and time (which they can then spend on actual science) and thus also saving taxpayers money...