4 ms·

> If it's a one-man-show I would not give too much on code quality This makes me a little uneasy, as I'm not too worried about code quality can easily translat

> If it's a one-man-show I would not give too much on code quality This makes me a little uneasy, as I'm not too worried about code quality can easily translate into Yes I know my code is full of undefined behaviour, and I don't care. > PS: quite excited about my first post here Welcome to HN! reddit has more cats, Slashdot has more jokes about sharks and laserbeams, but somehow we get by.

ByteJockey 6y ago
Are we talking actual undefined behavior or just behavior that's undefined by the language standard? The latter isn't great practice, but if your environment handles behavior deterministically, and you publish the version of the compiler you're using, it doesn't seem to be a problem for this type of code.
- MaxBarraclough 6y ago
  > Are we talking actual undefined behavior or just behavior that's undefined by the language standard? 'Undefined behaviour' is a term-of-art in C/C++ programming, there's no ambiguity. > if your environment handles behavior deterministically, and you publish the version of the compiler you're using, it doesn't seem to be a problem for this type of code. Code should be correct by construction, not correct by coincidence. Results from such code shouldn't be considered publishable. Mathematicians don't get credit for invalid proofs that happen to reach a conclusion which is correct. Again, this isn't some theoretical quibble. There are plenty of sneaky ways undefined behaviour can manifest and cause trouble. [0][1][2] In the domain of safety-critical software development in C, extreme measures are taken to ensure the absence of undefined behaviour. If scientists adopt a sloppier attitude toward code quality, they should expect to end up publishing invalid results. Frankly, this isn't news, and I'm surprised the standards seem to be so low. Also, of all the languages out there, C and C++ are among the most unforgiving of minor bugs, and are a bad choice of language for writing poor-quality code. Ada and Java, for instance, won't give you undefined behaviour for writing int i; int j = i;. [0] https://devblogs.microsoft.com/oldnewthing/20140627-00/?p=633 https://devblogs.microsoft.com/oldnewthing/20140627-00/?p=63... [1] https://blog.regehr.org/archives/213 https://blog.regehr.org/archives/213 [2] https://cryptoservices.github.io/fde/2018/11/30/undefined-behavior.html https://cryptoservices.github.io/fde/2018/11/30/undefined-be... See also my longer ramble on this topic at https://news.ycombinator.com/item?id=24264376 https://news.ycombinator.com/item?id=24264376
  - kazagistar 6y ago
    I think its poor practice, but undefined behavior shouldn't instantly invalidate results. In fact, this mindset is what keeps people from publishing the code in the first place. Let the scientists publish UB code, and even the artifacts produced, the executables. Then, if such problems are found in the code by professionals, they can investigate it fully and find if it leads to a tangible flaw that invalidates the research or not. You would drive yourself mad pointing out places in math proofs where some steps, even seemingly important ones, were skipped. But the papers are not retracted unless such a gap actually holds a flaw that invalidates the rest of thr proof. Let thdm publish their gross, awful, and even buggy code. Sometimes the bugs don't effect the outcomes.
    - MaxBarraclough 6y ago
      > undefined behavior shouldn't instantly invalidate results Granted, it's not a guarantee that the results are wrong, but it's a serious issue with the experiment. I agree it wouldn't generally make sense to retract a publication unless it can be determined that the results are invalid. It should be possible to independently investigate this, if the source-code and input data are published, as they should be. (It isn't universally true that reproduction of the experiment should be practical given that the source and data are published, as it may be difficult to reproduce supercomputer-powered experiments. iirc, training AlphaGo cost several million dollars of compute time, for instance.) > this mindset is what keeps people from publishing the code in the first place As I explained in [0], this attitude makes no sense at all. It has no place in modern science, and it's unfortunate the publication norms haven't caught up. Scientific publication is meant to enable critical independent review of work, not to shield scientists from criticism from their peers, which is the exact opposite. > Let the scientists publish UB code, and even the artifacts produced, the executables. Then, if such problems are found in the code by professionals, they can investigate it fully and find if it leads to a tangible flaw that invalidates the research or not. I'm not sure what to make of 'professionals', but otherwise I agree, go ahead and publish the binaries too, as much as applicable. Could be a valuable addition. (In some cases it might not be possible/practical to publish machine-code binaries, such as when working with GPUs, or Java. These platforms tend to be JIT based, and hostile to dumping and restoring exact binaries.) I agree with your final two paragraphs. [0] https://news.ycombinator.com/item?id=24264376 https://news.ycombinator.com/item?id=24264376

ByteJockey 6y ago

Are we talking actual undefined behavior or just behavior that's undefined by the language standard? The latter isn't great practice, but if your environment handles behavior deterministically, and you publish the version of the compiler you're using, it doesn't seem to be a problem for this type of code.

MaxBarraclough 6y ago
> Are we talking actual undefined behavior or just behavior that's undefined by the language standard? 'Undefined behaviour' is a term-of-art in C/C++ programming, there's no ambiguity. > if your environment handles behavior deterministically, and you publish the version of the compiler you're using, it doesn't seem to be a problem for this type of code. Code should be correct by construction, not correct by coincidence. Results from such code shouldn't be considered publishable. Mathematicians don't get credit for invalid proofs that happen to reach a conclusion which is correct. Again, this isn't some theoretical quibble. There are plenty of sneaky ways undefined behaviour can manifest and cause trouble. [0][1][2] In the domain of safety-critical software development in C, extreme measures are taken to ensure the absence of undefined behaviour. If scientists adopt a sloppier attitude toward code quality, they should expect to end up publishing invalid results. Frankly, this isn't news, and I'm surprised the standards seem to be so low. Also, of all the languages out there, C and C++ are among the most unforgiving of minor bugs, and are a bad choice of language for writing poor-quality code. Ada and Java, for instance, won't give you undefined behaviour for writing int i; int j = i;. [0] https://devblogs.microsoft.com/oldnewthing/20140627-00/?p=633 https://devblogs.microsoft.com/oldnewthing/20140627-00/?p=63... [1] https://blog.regehr.org/archives/213 https://blog.regehr.org/archives/213 [2] https://cryptoservices.github.io/fde/2018/11/30/undefined-behavior.html https://cryptoservices.github.io/fde/2018/11/30/undefined-be... See also my longer ramble on this topic at https://news.ycombinator.com/item?id=24264376 https://news.ycombinator.com/item?id=24264376
- kazagistar 6y ago
  I think its poor practice, but undefined behavior shouldn't instantly invalidate results. In fact, this mindset is what keeps people from publishing the code in the first place. Let the scientists publish UB code, and even the artifacts produced, the executables. Then, if such problems are found in the code by professionals, they can investigate it fully and find if it leads to a tangible flaw that invalidates the research or not. You would drive yourself mad pointing out places in math proofs where some steps, even seemingly important ones, were skipped. But the papers are not retracted unless such a gap actually holds a flaw that invalidates the rest of thr proof. Let thdm publish their gross, awful, and even buggy code. Sometimes the bugs don't effect the outcomes.
  - MaxBarraclough 6y ago
    > undefined behavior shouldn't instantly invalidate results Granted, it's not a guarantee that the results are wrong, but it's a serious issue with the experiment. I agree it wouldn't generally make sense to retract a publication unless it can be determined that the results are invalid. It should be possible to independently investigate this, if the source-code and input data are published, as they should be. (It isn't universally true that reproduction of the experiment should be practical given that the source and data are published, as it may be difficult to reproduce supercomputer-powered experiments. iirc, training AlphaGo cost several million dollars of compute time, for instance.) > this mindset is what keeps people from publishing the code in the first place As I explained in [0], this attitude makes no sense at all. It has no place in modern science, and it's unfortunate the publication norms haven't caught up. Scientific publication is meant to enable critical independent review of work, not to shield scientists from criticism from their peers, which is the exact opposite. > Let the scientists publish UB code, and even the artifacts produced, the executables. Then, if such problems are found in the code by professionals, they can investigate it fully and find if it leads to a tangible flaw that invalidates the research or not. I'm not sure what to make of 'professionals', but otherwise I agree, go ahead and publish the binaries too, as much as applicable. Could be a valuable addition. (In some cases it might not be possible/practical to publish machine-code binaries, such as when working with GPUs, or Java. These platforms tend to be JIT based, and hostile to dumping and restoring exact binaries.) I agree with your final two paragraphs. [0] https://news.ycombinator.com/item?id=24264376 https://news.ycombinator.com/item?id=24264376