4 ms·

A clarification, since this is a thing that frequently confuses people: the importance of correct rounding is that it is the easiest mechanism to unlock reprodu

A clarification, since this is a thing that frequently confuses people: the importance of correct rounding is that it is the easiest mechanism to unlock reproducibility. If I use library A, and you use library B, but both are correctly rounded, then we know that we get the same result. When reproducibility doesn't matter, it's basically always better to use a not-necessarily-correctly-rounded implementation at somewhat higher precision, as this gets you smaller overall errors for less effort. But there are a lot of domains where reproducibility matters, and that's where correct rounding comes in.

lisper 4y ago
Wait, now I'm confused. If what you want is reproducibility, why does it matter that the rounding be correct? It seems to me that what matters in this case is that the rounding conform to some standard or specification. Why should it matter what that standard actually is?
- MauranKilom 4y ago
  Sure, you could invent a different standard, but what are you going to write into it? "If you input 0.1 into atan, the result should round up when outputting 5 digits, down with 8 digits and up with 12 digits"? Also, "round up" is not any easier than "round correctly" if you require it to be consistent. Say you compute a result like 9.004 and want to round to two digits - if the exact result would have been 8.991, then rounding your computation as 9.01 is different from the standard-specified result of 9.00. So your standard would literally have to specify the desired output for every combination of operation, input, and output precision - obviously impossible.
- jabl 4y ago
  One usecase, for instance, is if you're writing a compiler and you want to constant propagate floating point operations, and the compile host doesn't necessarily support the same FP formats the target hardware does. But you still want the results to be the same as if you compute it on the target HW at runtime (barring usage of a "damn-IEEE754-just-go-faster" compile option). This, FWIW, is what GCC uses MPFR for.
  - adgjlsfhk1 4y ago
    Note that GCC actually has a bunch of bugs related to this where it will give you different results depending on whether a value gets computed at compile time or runtime.
    - jabl 4y ago
      Been a while since I was actively involved in GCC development, but back then most issues like this AFAIR were due to math functions and not basic arithmetic. GCC doesn't control the target libm, and correctly rounded libm's are rare and much slower than ones which permit a bit more slop in the results.
ChrisLomont 4y ago
>When reproducibility doesn't matter, If it doesn't matter, then high precision rarely matters either. Once you have results with unknown noise in them, and you do a few operations on those, the noise compounds, at approximately sqrt(N) bits for N steps of computation (rough estimate, it certainly depends on algorithm selection, compiler nuances, stability of problem, etc.). So high precision degrades quickly when doing any work without using a reproducible and understood rounding mode. >correct rounding is that it is the easiest mechanism to unlock reproducibility Any rounding method works works for reproducibility - truncate, round to zero, Bankers rounding, etc., all (roughly) equivalently hard/easy to implement. However, pretty much none of these are possible without the ability to do correcting rounding first as a result of the Table Makers' Dilemma. For example, Banker's rounding is generally better for doing computation than the usual correctly rounded. Knuth has some papers on this somewhere which are fascinating. Also note most of the non-reproducible issues in numerical computation are compiler added - compile code with GCC version X, get one result, compile with later version, and results may change. Or across compilers. Negative 0, denormals, rounding modes, all need carefully dealt with. It can be done, but is not trivial. Keeping floating point solid is very, very tricky over compilers and time.
- adgjlsfhk1 4y ago
  Note that bankers rounding is a subset of correct rounding since bankers rounding only affects ties.
- stephencanon 4y ago
  > If it doesn't matter, then high precision rarely matters either. Once you have results with unknown noise in them, and you do a few operations on those, the noise compounds, at approximately sqrt(N) bits for N steps of computation (rough estimate, it certainly depends on algorithm selection, compiler nuances, stability of problem, etc.). So high precision degrades quickly when doing any work without using a reproducible and understood rounding mode. This isn't really true for any of the numerous numerical algorithms that satisfy backwards error bounds; there you would always prefer to have a well-rounded arithmetic at higher precision, if reproducibility is not required. > Any rounding method works works for reproducibility Only if you round _correctly_ in that direction. > Banker's rounding is generally better for doing computation than the usual correctly rounded. Banker's rounding _is_ the default IEEE 754 rounding mode.

lisper 4y ago

Wait, now I'm confused. If what you want is reproducibility, why does it matter that the rounding be correct? It seems to me that what matters in this case is that the rounding conform to some standard or specification. Why should it matter what that standard actually is?

MauranKilom 4y ago
Sure, you could invent a different standard, but what are you going to write into it? "If you input 0.1 into atan, the result should round up when outputting 5 digits, down with 8 digits and up with 12 digits"? Also, "round up" is not any easier than "round correctly" if you require it to be consistent. Say you compute a result like 9.004 and want to round to two digits - if the exact result would have been 8.991, then rounding your computation as 9.01 is different from the standard-specified result of 9.00. So your standard would literally have to specify the desired output for every combination of operation, input, and output precision - obviously impossible.
jabl 4y ago
One usecase, for instance, is if you're writing a compiler and you want to constant propagate floating point operations, and the compile host doesn't necessarily support the same FP formats the target hardware does. But you still want the results to be the same as if you compute it on the target HW at runtime (barring usage of a "damn-IEEE754-just-go-faster" compile option). This, FWIW, is what GCC uses MPFR for.
- adgjlsfhk1 4y ago
  Note that GCC actually has a bunch of bugs related to this where it will give you different results depending on whether a value gets computed at compile time or runtime.
  - jabl 4y ago
    Been a while since I was actively involved in GCC development, but back then most issues like this AFAIR were due to math functions and not basic arithmetic. GCC doesn't control the target libm, and correctly rounded libm's are rare and much slower than ones which permit a bit more slop in the results.

ChrisLomont 4y ago

>When reproducibility doesn't matter, If it doesn't matter, then high precision rarely matters either. Once you have results with unknown noise in them, and you do a few operations on those, the noise compounds, at approximately sqrt(N) bits for N steps of computation (rough estimate, it certainly depends on algorithm selection, compiler nuances, stability of problem, etc.). So high precision degrades quickly when doing any work without using a reproducible and understood rounding mode. >correct rounding is that it is the easiest mechanism to unlock reproducibility Any rounding method works works for reproducibility - truncate, round to zero, Bankers rounding, etc., all (roughly) equivalently hard/easy to implement. However, pretty much none of these are possible without the ability to do correcting rounding first as a result of the Table Makers' Dilemma. For example, Banker's rounding is generally better for doing computation than the usual correctly rounded. Knuth has some papers on this somewhere which are fascinating. Also note most of the non-reproducible issues in numerical computation are compiler added - compile code with GCC version X, get one result, compile with later version, and results may change. Or across compilers. Negative 0, denormals, rounding modes, all need carefully dealt with. It can be done, but is not trivial. Keeping floating point solid is very, very tricky over compilers and time.

adgjlsfhk1 4y ago
Note that bankers rounding is a subset of correct rounding since bankers rounding only affects ties.
stephencanon 4y ago
> If it doesn't matter, then high precision rarely matters either. Once you have results with unknown noise in them, and you do a few operations on those, the noise compounds, at approximately sqrt(N) bits for N steps of computation (rough estimate, it certainly depends on algorithm selection, compiler nuances, stability of problem, etc.). So high precision degrades quickly when doing any work without using a reproducible and understood rounding mode. This isn't really true for any of the numerous numerical algorithms that satisfy backwards error bounds; there you would always prefer to have a well-rounded arithmetic at higher precision, if reproducibility is not required. > Any rounding method works works for reproducibility Only if you round _correctly_ in that direction. > Banker's rounding is generally better for doing computation than the usual correctly rounded. Banker's rounding _is_ the default IEEE 754 rounding mode.