5 ms·

This seems excessively glib. In my view floating point numbers are one of the most valuable and significant innovations in the history of science, and the wides

This seems excessively glib. In my view floating point numbers are one of the most valuable and significant innovations in the history of science, and the widespread availability of IEEE-754-compliant float hardware is one of the best things about computers. We all owe an enormous debt of gratitude to the folks who spent a ton of (technical and political) work to make it happen. Instead I would say: numerical analysis is hard irrespective of the number representation, and we should try to spread at least basic knowledge about it so people can accomplish their goals while avoiding some of the pitfalls.

jjmarr 2y ago
"Floating points are the worst method of representing rational numbers, except for all the others." —Winston Churchill probably
- lmm 2y ago
  Nah, arbitrary precision rationals are much better if you actually only need to represent rationals. And frankly even for reals, decimal types are better for most use cases.
  - jacobolus 2y ago
    For "most use cases" (by far) of arbitrary fractional numbers, floats are significantly better than integer based representations, which is why they are pervasive. For certain niche uses (such as some kinds of money accounting calculations) alternative integer-based representations have some advantages, and those who need them use them where appropriate.
    - lmm 2y ago
      Disagree. IEEE floats see a disproportionate amount of use because most programming languages make them much easier to use than more appropriate representations, and people (understandably) use the representation the language guides them to rather than seriously analysing and considering the options.
  - jjmarr 2y ago
    Arbitrary precision sounds good but my registers have 64 bits and it takes several instructions to load arbitrary bit slices from memory. It's a lot more performant to use the equivalent of scientific notation.
    - lmm 2y ago
      Sure, but the vast majority of the time that performance difference doesn't matter, or at least isn't worth the large sacrifices in usability and consistency that it costs.
      - jacobolus 2y ago
        The vast majority of the time floats are a significantly more convenient representation, with more than enough precision for the job, and alternatives are so slow that they are somewhere between painful and literally impossible to use. Arbitrary precision computation grows exponentially more expensive as you perform arithmetic operations on it, and people routinely apply transcendental functions whose results cannot be represented using a finite number of binary digits. If you are finding otherwise in your personal work, you are likely in some niche field sheltered far from the bulk of numerical computing applications.
bell-cot 2y ago
Motor vehicles are one of the most valuable and significant innovations in history, etc. etc. But that doesn't make them safe.
adgjlsfhk1 2y ago
IMO IEE-754 is pretty good, but could be a bunch better. I really wish they had gotten rid of -0, Infs and all but one of the NaNs (which you would encode as -0 currently is encoded and be equal to itself). Currently the behavior of which NaN gets produced is unspecified and inconsistent across hardware which really sucks. Also, the exponent bias should be changed so that every nonzero finite value has a representable finite inverse. Currently 1/subnormal is Inf which is really annoying since ti means transforming x/y to x*(1/y) can overflow.
- jacobolus 2y ago
  Without -0 or Infs, IEEE-754 would be significantly worse for doing scientific computations. I personally find these to be among the most useful and important features of floating point arithmetic.
  - adgjlsfhk1 2y ago
    Do you have an example? I have a hard time believing this since any scientific data will have uncertainty that should make it completely random as to what sign your zero is supposed to be. Similarly with Inf, if any intermediate result gives an Inf, you've lost all precision, and the final result is probably going to be NaN anyway.
    - jacobolus 2y ago
      > random as to what sign your zero is supposed to be The purpose of negative zero is to preserve the sign of underflow, so you can e.g. get the branch cuts right when you are implementing some complex function. Cf. https://grouper.ieee.org/groups/msc/ANSI_IEEE-Std-754-2019/background/kahan86branch.pdf https://grouper.ieee.org/groups/msc/ANSI_IEEE-Std-754-2019/b... Functions with singularities are ubiquitous in physics and many other branches of science, https://dlmf.nist.gov https://dlmf.nist.gov
      - adgjlsfhk1 2y ago
        I understand that that is the purpose, but the problem is as soon as you have any error in your computations, you lose the sign of the zero. From the paper you posted: (ii) Rounding errors can obscure the singularities. That is why, for example, sqrt(z^2-1) = sqrt((z+1)(z-1)) fails so badly when either z^2=1 very nearly or when z^2<0 very nearly. To avoid this problem, the programmer may have to decompose complex arithmetic expressions into separate computations of real and imaginary parts. thereby forgoing Some of the advantages of a compact notation. (iii) Careless handling can turn infinity or the sign of zero into misinformation that subsequently disappears leaving behind only a plausible but incorrect result. That is why compilers must not transform z-1 into z-(1+0i) as we have seen above, nor -(-x-x^2) into x+x^2 as we shall see below, lest a subsequent logarithm or square root produce a nonzero imaginary part whose sign is opposite to what was intended. Branch cuts are a fundamentally dumb idea in finite precision math because if your input number has any uncertainty (which it always does), you don't know which side of the branch cut you are on. With a massive amount of manual work, you can sometimes hack the system well enough to work for extremely contrived cases, but in reality, it will never be useful because within the broader scope of the problem, if you've hit a singularity, you don't know what side you hit it from. Functions with singularities are ubiquitous in physics and other branches of science, but symbolic math is needed to correctly track them.

jjmarr 2y ago

"Floating points are the worst method of representing rational numbers, except for all the others." —Winston Churchill probably

lmm 2y ago
Nah, arbitrary precision rationals are much better if you actually only need to represent rationals. And frankly even for reals, decimal types are better for most use cases.
- jacobolus 2y ago
  For "most use cases" (by far) of arbitrary fractional numbers, floats are significantly better than integer based representations, which is why they are pervasive. For certain niche uses (such as some kinds of money accounting calculations) alternative integer-based representations have some advantages, and those who need them use them where appropriate.
  - lmm 2y ago
    Disagree. IEEE floats see a disproportionate amount of use because most programming languages make them much easier to use than more appropriate representations, and people (understandably) use the representation the language guides them to rather than seriously analysing and considering the options.
- jjmarr 2y ago
  Arbitrary precision sounds good but my registers have 64 bits and it takes several instructions to load arbitrary bit slices from memory. It's a lot more performant to use the equivalent of scientific notation.
  - lmm 2y ago
    Sure, but the vast majority of the time that performance difference doesn't matter, or at least isn't worth the large sacrifices in usability and consistency that it costs.
    - jacobolus 2y ago
      The vast majority of the time floats are a significantly more convenient representation, with more than enough precision for the job, and alternatives are so slow that they are somewhere between painful and literally impossible to use. Arbitrary precision computation grows exponentially more expensive as you perform arithmetic operations on it, and people routinely apply transcendental functions whose results cannot be represented using a finite number of binary digits. If you are finding otherwise in your personal work, you are likely in some niche field sheltered far from the bulk of numerical computing applications.

bell-cot 2y ago

Motor vehicles are one of the most valuable and significant innovations in history, etc. etc. But that doesn't make them safe.

adgjlsfhk1 2y ago

IMO IEE-754 is pretty good, but could be a bunch better. I really wish they had gotten rid of -0, Infs and all but one of the NaNs (which you would encode as -0 currently is encoded and be equal to itself). Currently the behavior of which NaN gets produced is unspecified and inconsistent across hardware which really sucks. Also, the exponent bias should be changed so that every nonzero finite value has a representable finite inverse. Currently 1/subnormal is Inf which is really annoying since ti means transforming x/y to x*(1/y) can overflow.

jacobolus 2y ago
Without -0 or Infs, IEEE-754 would be significantly worse for doing scientific computations. I personally find these to be among the most useful and important features of floating point arithmetic.
- adgjlsfhk1 2y ago
  Do you have an example? I have a hard time believing this since any scientific data will have uncertainty that should make it completely random as to what sign your zero is supposed to be. Similarly with Inf, if any intermediate result gives an Inf, you've lost all precision, and the final result is probably going to be NaN anyway.
  - jacobolus 2y ago
    > random as to what sign your zero is supposed to be The purpose of negative zero is to preserve the sign of underflow, so you can e.g. get the branch cuts right when you are implementing some complex function. Cf. https://grouper.ieee.org/groups/msc/ANSI_IEEE-Std-754-2019/background/kahan86branch.pdf https://grouper.ieee.org/groups/msc/ANSI_IEEE-Std-754-2019/b... Functions with singularities are ubiquitous in physics and many other branches of science, https://dlmf.nist.gov https://dlmf.nist.gov
    - adgjlsfhk1 2y ago
      I understand that that is the purpose, but the problem is as soon as you have any error in your computations, you lose the sign of the zero. From the paper you posted: (ii) Rounding errors can obscure the singularities. That is why, for example, sqrt(z^2-1) = sqrt((z+1)(z-1)) fails so badly when either z^2=1 very nearly or when z^2<0 very nearly. To avoid this problem, the programmer may have to decompose complex arithmetic expressions into separate computations of real and imaginary parts. thereby forgoing Some of the advantages of a compact notation. (iii) Careless handling can turn infinity or the sign of zero into misinformation that subsequently disappears leaving behind only a plausible but incorrect result. That is why compilers must not transform z-1 into z-(1+0i) as we have seen above, nor -(-x-x^2) into x+x^2 as we shall see below, lest a subsequent logarithm or square root produce a nonzero imaginary part whose sign is opposite to what was intended. Branch cuts are a fundamentally dumb idea in finite precision math because if your input number has any uncertainty (which it always does), you don't know which side of the branch cut you are on. With a massive amount of manual work, you can sometimes hack the system well enough to work for extremely contrived cases, but in reality, it will never be useful because within the broader scope of the problem, if you've hit a singularity, you don't know what side you hit it from. Functions with singularities are ubiquitous in physics and other branches of science, but symbolic math is needed to correctly track them.