6 ms·

> `a+b` will have the same value as `b+a` No, as IEEE754 doesn't even guarantee that `a + b` == `a + b`. You always need to compare the absolute difference of

> `a+b` will have the same value as `b+a` No, as IEEE754 doesn't even guarantee that `a + b` == `a + b`. You always need to compare the absolute difference of two values against an ε. So a == b <=> |a - b| < ε But for sensible comparison of floats, the addition is commutative.

zokier 5y ago
> No, as IEEE754 doesn't even guarantee that `a + b` == `a + b` What? I'm pretty sure that is plain incorrect. Can you back that claim up with references?
- deleted 5y ago
  [deleted]
- futharkshill 5y ago
  Of course it is incorrect, the system is of course not stochastic, not even floating points.
  - c-cube 5y ago
    What if a is nan? You don't have that nan == nan...
- anonymoushn 5y ago
  Floating point addition is not commutative if you're adding negative zero to positive zero, but you won't be able to check that with `a + b == a + b`
- ReleaseCandidat 5y ago
  The rounding can depend 'where' (e.g. the width of the register) the float is stored in the CPU ('destination' in IEEE 754 speak). So, if you compare the result stored in two different 'destinations' (because one has been computed in another register or at a GPU or ....), they can differ even in the same program with the same optimizations. See for example: int main() { double q; q = 3.0/7.0; if (q == 3.0/7.0) printf("Equal\n"); else printf("Not Equal\n"); return 0; } On an extended-based system, even though the expression 3.0/7.0 has type double, the quotient will be computed in a register in extended double format, and thus in the default mode, it will be rounded to extended double precision. When the resulting value is assigned to the variable q, however, it may then be stored in memory, and since q is declared double, the value will be rounded to double precision. In the next line, the expression 3.0/7.0 may again be evaluated in extended precision yielding a result that differs from the double precision value stored in q, causing the program to print "Not Equal". Of course, other outcomes are possible, too: the compiler could decide to store and thus round the value of the expression 3.0/7.0 in the second line before comparing it with q, or it could keep q in a register in extended precision without storing it. An optimizing compiler might evaluate the expression 3.0/7.0 at compile time, perhaps in double precision or perhaps in extended double precision. (With one x86 compiler, the program prints "Equal" when compiled with optimization and "Not Equal" when compiled for debugging.) Finally, some compilers for extended-based systems automatically change the rounding precision mode to cause operations producing results in registers to round those results to single or double precision, albeit possibly with a wider range. Thus, on these systems, we can't predict the behavior of the program simply by reading its source code and applying a basic understanding of IEEE 754 arithmetic. The relevant conclusion: Neither can we accuse the hardware or the compiler of failing to provide an IEEE 754 compliant environment; the hardware has delivered a correctly rounded result to each destination, as it is required to do, and the compiler has assigned some intermediate results to destinations that are beyond the user's control, as it is allowed to do. https://grouper.ieee.org/groups/msc/ANSI_IEEE-Std-754-2019/background/addendum.html https://grouper.ieee.org/groups/msc/ANSI_IEEE-Std-754-2019/b...
  - zokier 5y ago
    I find the argument unconvincing. While IEEE 758 does not specify what `/` (or `+`) operator in particular does (that falls under C standard), it does specify that: > Implementations shall provide the following formatOf general-computational operations, for destinations of > all supported arithmetic formats, and, for each destination format, for operands of all supported arithmetic > formats with the same radix as the destination format Which to my interpretation sounds like providing a `division(double, double) -> double` operation is required. I suppose the argument could be that in C the way of invoking that specific operation would be to add an explicit cast, i.e. (double)(a op b) But I do think that this is more a quirk on how operators are done specifically in C and not a general matter; the actual IEEE 758 operations are consistent and don't have such surprises. So I guess you were technically correct that IEEE 758 does not guarantee `a + b == a + b`, because IEEE 758 does not specify `+` (or `==` for that matter) operators at all. What IEEE 758 does guarantee is that you get the same result for the same operation. In terms of C language, it is maybe interesting question if `FP_CONTRACT` pragma influences the result here at all: > A floating expression may be contracted, that is, evaluated as though it were a single opera- > tion, thereby omitting rounding errors implied by the source code and the expression evalua- > tion method. The FP_CONTRACT pragma in <math.h> provides a way to disallow contracted > expressions. Otherwise, whether and how expressions are contracted is implementation-defined. Are the comparison and arithmetic operations considered to be contracted in these sort of situations?
    - ReleaseCandidat 5y ago
      > But I do think that this is more a quirk on how operators are done specifically in C and not a general matte That has nothing to do with C, but with the CPU and its registers. x86 has 80bit floating point registers (too), so if the compiler/CPU (doesn't matter which language) saves a value in a 80 bit floating point register and moves it from there to memory where it is stored in 64 bits, the number gets rounded (not truncated, it's actually converted). See also the GCC page: For instance, in the following code segment, depending on the compilation flags and numbers and calculations used to find tmp, the following code may print out that the values are different: double tmp, X[2]; tmp = .... tmp += .... ... X[0] = tmp; if (X[0] == tmp) printf("Values are the same, as expected!\n"); else printf("Values are different!\n"); This is because tmp will typically be moved to a register during register assignment, which means tmp may hold a full 80 bits of accuracy, some of which are lost in the store to X[0], and thus the numbers are no longer equal. You may workaround this problem by always explicitly storing to memory to force the round-down. https://gcc.gnu.org/wiki/x87note https://gcc.gnu.org/wiki/x87note > Are the comparison and arithmetic operations considered to be contracted in these sort of situations? No. Or well, maybe. 'Contracted' means using e.g. FMA (fused multiply and add), so addition and multiplication like `ab + c` is done in a single step instead of two. Which means that the result is only rounded once (`ab + c` is rounded), whereas without this fusing/contracting `ab` would be rounded and `ab + c` would be rounded again (actually depending on the optimization/compiler flags). So the results (may) differ.

zokier 5y ago

> No, as IEEE754 doesn't even guarantee that `a + b` == `a + b` What? I'm pretty sure that is plain incorrect. Can you back that claim up with references?

deleted 5y ago
[deleted]
futharkshill 5y ago
Of course it is incorrect, the system is of course not stochastic, not even floating points.
- c-cube 5y ago
  What if a is nan? You don't have that nan == nan...
anonymoushn 5y ago
Floating point addition is not commutative if you're adding negative zero to positive zero, but you won't be able to check that with `a + b == a + b`
ReleaseCandidat 5y ago
The rounding can depend 'where' (e.g. the width of the register) the float is stored in the CPU ('destination' in IEEE 754 speak). So, if you compare the result stored in two different 'destinations' (because one has been computed in another register or at a GPU or ....), they can differ even in the same program with the same optimizations. See for example: int main() { double q; q = 3.0/7.0; if (q == 3.0/7.0) printf("Equal\n"); else printf("Not Equal\n"); return 0; } On an extended-based system, even though the expression 3.0/7.0 has type double, the quotient will be computed in a register in extended double format, and thus in the default mode, it will be rounded to extended double precision. When the resulting value is assigned to the variable q, however, it may then be stored in memory, and since q is declared double, the value will be rounded to double precision. In the next line, the expression 3.0/7.0 may again be evaluated in extended precision yielding a result that differs from the double precision value stored in q, causing the program to print "Not Equal". Of course, other outcomes are possible, too: the compiler could decide to store and thus round the value of the expression 3.0/7.0 in the second line before comparing it with q, or it could keep q in a register in extended precision without storing it. An optimizing compiler might evaluate the expression 3.0/7.0 at compile time, perhaps in double precision or perhaps in extended double precision. (With one x86 compiler, the program prints "Equal" when compiled with optimization and "Not Equal" when compiled for debugging.) Finally, some compilers for extended-based systems automatically change the rounding precision mode to cause operations producing results in registers to round those results to single or double precision, albeit possibly with a wider range. Thus, on these systems, we can't predict the behavior of the program simply by reading its source code and applying a basic understanding of IEEE 754 arithmetic. The relevant conclusion: Neither can we accuse the hardware or the compiler of failing to provide an IEEE 754 compliant environment; the hardware has delivered a correctly rounded result to each destination, as it is required to do, and the compiler has assigned some intermediate results to destinations that are beyond the user's control, as it is allowed to do. https://grouper.ieee.org/groups/msc/ANSI_IEEE-Std-754-2019/background/addendum.html https://grouper.ieee.org/groups/msc/ANSI_IEEE-Std-754-2019/b...
- zokier 5y ago
  I find the argument unconvincing. While IEEE 758 does not specify what `/` (or `+`) operator in particular does (that falls under C standard), it does specify that: > Implementations shall provide the following formatOf general-computational operations, for destinations of > all supported arithmetic formats, and, for each destination format, for operands of all supported arithmetic > formats with the same radix as the destination format Which to my interpretation sounds like providing a `division(double, double) -> double` operation is required. I suppose the argument could be that in C the way of invoking that specific operation would be to add an explicit cast, i.e. (double)(a op b) But I do think that this is more a quirk on how operators are done specifically in C and not a general matter; the actual IEEE 758 operations are consistent and don't have such surprises. So I guess you were technically correct that IEEE 758 does not guarantee `a + b == a + b`, because IEEE 758 does not specify `+` (or `==` for that matter) operators at all. What IEEE 758 does guarantee is that you get the same result for the same operation. In terms of C language, it is maybe interesting question if `FP_CONTRACT` pragma influences the result here at all: > A floating expression may be contracted, that is, evaluated as though it were a single opera- > tion, thereby omitting rounding errors implied by the source code and the expression evalua- > tion method. The FP_CONTRACT pragma in <math.h> provides a way to disallow contracted > expressions. Otherwise, whether and how expressions are contracted is implementation-defined. Are the comparison and arithmetic operations considered to be contracted in these sort of situations?
  - ReleaseCandidat 5y ago
    > But I do think that this is more a quirk on how operators are done specifically in C and not a general matte That has nothing to do with C, but with the CPU and its registers. x86 has 80bit floating point registers (too), so if the compiler/CPU (doesn't matter which language) saves a value in a 80 bit floating point register and moves it from there to memory where it is stored in 64 bits, the number gets rounded (not truncated, it's actually converted). See also the GCC page: For instance, in the following code segment, depending on the compilation flags and numbers and calculations used to find tmp, the following code may print out that the values are different: double tmp, X[2]; tmp = .... tmp += .... ... X[0] = tmp; if (X[0] == tmp) printf("Values are the same, as expected!\n"); else printf("Values are different!\n"); This is because tmp will typically be moved to a register during register assignment, which means tmp may hold a full 80 bits of accuracy, some of which are lost in the store to X[0], and thus the numbers are no longer equal. You may workaround this problem by always explicitly storing to memory to force the round-down. https://gcc.gnu.org/wiki/x87note https://gcc.gnu.org/wiki/x87note > Are the comparison and arithmetic operations considered to be contracted in these sort of situations? No. Or well, maybe. 'Contracted' means using e.g. FMA (fused multiply and add), so addition and multiplication like `ab + c` is done in a single step instead of two. Which means that the result is only rounded once (`ab + c` is rounded), whereas without this fusing/contracting `ab` would be rounded and `ab + c` would be rounded again (actually depending on the optimization/compiler flags). So the results (may) differ.