4 ms·

Not sure if there is so much to support - they most likely run some archaic Fortran code that only 5 people in the world understand, compiled for some specific

Not sure if there is so much to support - they most likely run some archaic Fortran code that only 5 people in the world understand, compiled for some specific cluster, where it runs multiple days for one dataset, and is only numerically stable in that particular environment.

wolfgke 10y ago
This is why on https://news.ycombinator.com/item?id=12099720 https://news.ycombinator.com/item?id=12099720 I also suggested to provide a virtual machine image if possible (unluckily providing one is in my opinion not always possible). On the other hand if another team ran the code independently and we really saw that it is numerically unstable in another environment, this would be, in my opinion, a strong case for the importance of providing the source code.
- dalke 10y ago
  I don't think you understand what elcapitan means by numerically stable. Numeric stability is a function of the environment. If I take an algorithm designed for 128 bit IEEE binary floats and implement it on a system with 64 bit floats, then it may be unstable. That doesn't say anything about the correctness of the original method for the original environment. It only means the method wasn't designed for the new environment. You are used to a world where computers are a commodity, likely based around the Intel architecture, and almost certainly using IEEE floats. What do you do with a computer program written for Anton, a specialized computer for doing molecular dynamics built using specialized ASICs? https://en.wikipedia.org/wiki/Anton_(computer) https://en.wikipedia.org/wiki/Anton_(computer)
  - wolfgke 10y ago
    First: IEEE 754 defines exactly (bit per bit) what the result of any floating point operation in each of the five rounding modes as long as no NaN is produced (but in this case only not all bits are defined exactly, but it will still be a NaN). Unluckily C allows some optimizations to be done to code containing floating point code that violates this principle of bit-for-bit reproducability of floating point code. If this causes problems the code can be considered as numerically unstable. In other words: Writing code that uses the IEE 754 defined "gold standard" in its code should be the goal scientific results. The correct way for independent checking, if we really need 128 bit floating point numbers, is to wrap these operations by a software floating point library. > It only means the method wasn't designed for the new environment. If this property (what kind of environment the algorithm depends on) of the algorithm is not documented properly (and an explanation is given why we need this property (say: size of floating points, in particular unusual floating point sizes as 128 bit) for the algorithm to work correctly) any independent reviewer should better not assume these properties to hold. If we indeed find out that this causes problems, this is a strong sign to me that there might be subtile errors in the code. In other words: We should be really careful of the results that the algorithm gave. > What do you do with a computer program written for Anton, a specialized computer for doing molecular dynamics built using specialized ASICs? I gave the answer in the post above: "[U]nluckily providing [a virtual machine] is in my opinion not always possible".
    - dalke 10y ago
      I don't follow the point of your first four paragraphs. I specifically said the code was numerically stable on 128 bits - why are you introducing those irrelevant facts, like how C compiler details can make some algorithms unstable? Really it comes down to what you mean by "another environment" and what your case is for access to the source code. Let me give a simpler example. I wrote code which expects ILP32. It's stable under ILP32. However, it produces different and non-deterministic answers under LP64. That is "another environment." Does the numerical instability under a different environment cast doubt on its validity in the original environment? Why is it a strong sign that there may be subtle errors?

wolfgke 10y ago

This is why on https://news.ycombinator.com/item?id=12099720 https://news.ycombinator.com/item?id=12099720 I also suggested to provide a virtual machine image if possible (unluckily providing one is in my opinion not always possible). On the other hand if another team ran the code independently and we really saw that it is numerically unstable in another environment, this would be, in my opinion, a strong case for the importance of providing the source code.

dalke 10y ago
I don't think you understand what elcapitan means by numerically stable. Numeric stability is a function of the environment. If I take an algorithm designed for 128 bit IEEE binary floats and implement it on a system with 64 bit floats, then it may be unstable. That doesn't say anything about the correctness of the original method for the original environment. It only means the method wasn't designed for the new environment. You are used to a world where computers are a commodity, likely based around the Intel architecture, and almost certainly using IEEE floats. What do you do with a computer program written for Anton, a specialized computer for doing molecular dynamics built using specialized ASICs? https://en.wikipedia.org/wiki/Anton_(computer) https://en.wikipedia.org/wiki/Anton_(computer)
- wolfgke 10y ago
  First: IEEE 754 defines exactly (bit per bit) what the result of any floating point operation in each of the five rounding modes as long as no NaN is produced (but in this case only not all bits are defined exactly, but it will still be a NaN). Unluckily C allows some optimizations to be done to code containing floating point code that violates this principle of bit-for-bit reproducability of floating point code. If this causes problems the code can be considered as numerically unstable. In other words: Writing code that uses the IEE 754 defined "gold standard" in its code should be the goal scientific results. The correct way for independent checking, if we really need 128 bit floating point numbers, is to wrap these operations by a software floating point library. > It only means the method wasn't designed for the new environment. If this property (what kind of environment the algorithm depends on) of the algorithm is not documented properly (and an explanation is given why we need this property (say: size of floating points, in particular unusual floating point sizes as 128 bit) for the algorithm to work correctly) any independent reviewer should better not assume these properties to hold. If we indeed find out that this causes problems, this is a strong sign to me that there might be subtile errors in the code. In other words: We should be really careful of the results that the algorithm gave. > What do you do with a computer program written for Anton, a specialized computer for doing molecular dynamics built using specialized ASICs? I gave the answer in the post above: "[U]nluckily providing [a virtual machine] is in my opinion not always possible".
  - dalke 10y ago
    I don't follow the point of your first four paragraphs. I specifically said the code was numerically stable on 128 bits - why are you introducing those irrelevant facts, like how C compiler details can make some algorithms unstable? Really it comes down to what you mean by "another environment" and what your case is for access to the source code. Let me give a simpler example. I wrote code which expects ILP32. It's stable under ILP32. However, it produces different and non-deterministic answers under LP64. That is "another environment." Does the numerical instability under a different environment cast doubt on its validity in the original environment? Why is it a strong sign that there may be subtle errors?