5 ms·

Floating point considered harmful Edit: this is not a blanket statement. It was meant in the context.

by arnon 6y ago

phoe-krk 6y ago
Misunderstanding floating point is more harmful than floating point itself.
- arnon 6y ago
  Hence why it's harmful.
  - MaxBarraclough 6y ago
    Floating point arithmetic powers everything from computer graphics to weather simulations. Seems rather silly to dismiss it as an anti-pattern.
  - OskarS 6y ago
    Floating point is fine. Non-integers are inherently tricky to represent, especially when you have to pack it into 32 bits. You could maybe quibble with some of the decisions around NaN and denormals and things like that, but mostly IEEE-754 got it right. There's a reason it's been the standard for three and a half decades now, and it's served the computer industry very well. Incidentally: the fact that 0.1+0.2 does not equal 0.3 is not something you could quibble with, that is absolutely reasonable for a floating point standard. It would be insanity to base it off of base 10 instead of base 2.
    - aidenn0 6y ago
      Of course it shouldn't be base 10. It should be base 60. That has more prime factors and wastes fewer bits than BCD does.
      - lmilcin 6y ago
        Or base 256 so that it fits a byte efficiently. Oh... wait...
    - UncleMeat 6y ago
      I think one of the problems today is that floating point remains the default even for scripting languages like python. I'd wager that the huge majority of users of floating point arithmetic in their programs actually want correct math rather than efficient operations. This feels a bit like Random vs SecureRandom. So many people use the default and then accidentally shoot themselves in the foot. IMO, for scripting languages we should have infinite precision math by default and then be able to use floating point if you really care about speed.
      - edflsafoiewq 6y ago
        What precise number representation do you want that's free of such "surprises"?
        lmilcin 6y ago
        There exists no representation without "surprises" and it is easy to show why that must be. The closest you can get is some systems like Matlab/Octave that specialize in working with numbers.
      - lifthrasiir 6y ago
        This is not a trivial decision because rational numbers can have an unbounded denominator and it can cause a serious performance problem. Python explicitly ruled rational numbers out due to this issue [1]. There are multiple possible answers: - Keep rational numbers; you also have an unbounded integer so that's to be expected. (Well, unbounded integers are visible, while unbounded denominators are mostly invisble. For example it is very rare to explicitly test denominators.) - Use decimal types with precision controlled in run time, like Python `decimal` module. (That's still inexact, also we need some heavy language and/or runtime support for varying precision.) - Use interval arithmetic with ordinary floating point numbers. (This would be okay only if every user knows pitfalls of IA: for example, comparison no longer returns true or false but also "unsure". There may be some clever language constructs that can make this doable though.) - You don't have real numbers, just integers. (I think this is actually okay for surprisingly large use cases, but not all.) [1] https://python-history.blogspot.com/2009/02/early-language-design-and-development.html https://python-history.blogspot.com/2009/02/early-language-d... (search for "anecdote")
lmilcin 6y ago
"People repeating stuff without understanding it considered harmful." Floating point is extremely useful. Too bad so many people have no idea how and when to use it. Including some people that design programming languages. Please, tell me, mister, how would you perform complex numerical calculations efficiently? I guess we should just forget about drones and bunch other stuff because 90% of developers have no clue how to use FP?
- lifthrasiir 6y ago
  > Please, tell me, mister, how would you perform complex numerical calculations efficiently? If your calculation turned out to be incorrect it doesn't matter if it's efficient. Correct FP calculation requires error analysis, which is a concrete definition of "how to use it". If you mostly use packaged routines like LAPACK, then you don't exactly need FP; you need routines that internally use FP.
  - lmilcin 6y ago
    > Correct FP calculation requires error analysis No, it does not. Please, don't make it seem harder than it needs to. 99% applications, if you don't do anything stupid you are completely fine. If you care for precision so much the last digit make difference for you you are probably one of very few cases. I remember somebody giving an example circumference of solar system showing uncertainty of the value of Pi available as FP to cause couple centimeters of error at the orbit of Pluto, or something like that. (Edit: found it: https://kottke.org/16/03/how-many-digits-of-pi-does-nasa-use https://kottke.org/16/03/how-many-digits-of-pi-does-nasa-use) Most of the time floating point input is already coming with its own error, you are just adding calculation error to the input uncertainty. But the calculation error is so much smaller than in most cases it is safe to ignore it. For example, if you program a drone, you have readings from IMU which have not nearly the precision of the double or even float you will be working on. There is also various techniques of ordering the operations to minimize the resulting error. If you are aware which kinds of operations in which situations can cause huge resulting error it is usually very easy to avoid it. Only very special case is if you try to subtract two values calculated separately and matching almost exactly. This should be avoided.
dragonwriter 6y ago
Not so much floating point as “using float point type for exact decimal literals”.
- arnon 6y ago
  Fair...
bidirectional 6y ago
Not at all. For all it's faults, floating point is incredibly fast. It's not some convenience hack that we lazy programmers have come up with, it's an incredibly quick way to do numerical computation. It will always have it's place (sometimes even in finance to represent money, to many people's shock).
- lifthrasiir 6y ago
  To add to that, in modern processors FP calculation is faster than integer calculation, both in terms of latency and throughput (as long as you don't hit subnormal numbers). This is very unintuitive and mostly due to disproportional demands.

Floating point considered harmful Edit: this is not a blanket statement. It was meant in the context.

by arnon 6y ago

Floating point considered harmful Edit: this is not a blanket statement. It was meant in the context.

phoe-krk 6y ago

Misunderstanding floating point is more harmful than floating point itself.

arnon 6y ago
Hence why it's harmful.
- MaxBarraclough 6y ago
  Floating point arithmetic powers everything from computer graphics to weather simulations. Seems rather silly to dismiss it as an anti-pattern.
- OskarS 6y ago
  Floating point is fine. Non-integers are inherently tricky to represent, especially when you have to pack it into 32 bits. You could maybe quibble with some of the decisions around NaN and denormals and things like that, but mostly IEEE-754 got it right. There's a reason it's been the standard for three and a half decades now, and it's served the computer industry very well. Incidentally: the fact that 0.1+0.2 does not equal 0.3 is not something you could quibble with, that is absolutely reasonable for a floating point standard. It would be insanity to base it off of base 10 instead of base 2.
  - aidenn0 6y ago
    Of course it shouldn't be base 10. It should be base 60. That has more prime factors and wastes fewer bits than BCD does.
    - lmilcin 6y ago
      Or base 256 so that it fits a byte efficiently. Oh... wait...
  - UncleMeat 6y ago
    I think one of the problems today is that floating point remains the default even for scripting languages like python. I'd wager that the huge majority of users of floating point arithmetic in their programs actually want correct math rather than efficient operations. This feels a bit like Random vs SecureRandom. So many people use the default and then accidentally shoot themselves in the foot. IMO, for scripting languages we should have infinite precision math by default and then be able to use floating point if you really care about speed.
    - edflsafoiewq 6y ago
      What precise number representation do you want that's free of such "surprises"?
      - lmilcin 6y ago
        There exists no representation without "surprises" and it is easy to show why that must be. The closest you can get is some systems like Matlab/Octave that specialize in working with numbers.
    - lifthrasiir 6y ago
      This is not a trivial decision because rational numbers can have an unbounded denominator and it can cause a serious performance problem. Python explicitly ruled rational numbers out due to this issue [1]. There are multiple possible answers: - Keep rational numbers; you also have an unbounded integer so that's to be expected. (Well, unbounded integers are visible, while unbounded denominators are mostly invisble. For example it is very rare to explicitly test denominators.) - Use decimal types with precision controlled in run time, like Python `decimal` module. (That's still inexact, also we need some heavy language and/or runtime support for varying precision.) - Use interval arithmetic with ordinary floating point numbers. (This would be okay only if every user knows pitfalls of IA: for example, comparison no longer returns true or false but also "unsure". There may be some clever language constructs that can make this doable though.) - You don't have real numbers, just integers. (I think this is actually okay for surprisingly large use cases, but not all.) [1] https://python-history.blogspot.com/2009/02/early-language-design-and-development.html https://python-history.blogspot.com/2009/02/early-language-d... (search for "anecdote")

lmilcin 6y ago

"People repeating stuff without understanding it considered harmful." Floating point is extremely useful. Too bad so many people have no idea how and when to use it. Including some people that design programming languages. Please, tell me, mister, how would you perform complex numerical calculations efficiently? I guess we should just forget about drones and bunch other stuff because 90% of developers have no clue how to use FP?

lifthrasiir 6y ago
> Please, tell me, mister, how would you perform complex numerical calculations efficiently? If your calculation turned out to be incorrect it doesn't matter if it's efficient. Correct FP calculation requires error analysis, which is a concrete definition of "how to use it". If you mostly use packaged routines like LAPACK, then you don't exactly need FP; you need routines that internally use FP.
- lmilcin 6y ago
  > Correct FP calculation requires error analysis No, it does not. Please, don't make it seem harder than it needs to. 99% applications, if you don't do anything stupid you are completely fine. If you care for precision so much the last digit make difference for you you are probably one of very few cases. I remember somebody giving an example circumference of solar system showing uncertainty of the value of Pi available as FP to cause couple centimeters of error at the orbit of Pluto, or something like that. (Edit: found it: https://kottke.org/16/03/how-many-digits-of-pi-does-nasa-use https://kottke.org/16/03/how-many-digits-of-pi-does-nasa-use) Most of the time floating point input is already coming with its own error, you are just adding calculation error to the input uncertainty. But the calculation error is so much smaller than in most cases it is safe to ignore it. For example, if you program a drone, you have readings from IMU which have not nearly the precision of the double or even float you will be working on. There is also various techniques of ordering the operations to minimize the resulting error. If you are aware which kinds of operations in which situations can cause huge resulting error it is usually very easy to avoid it. Only very special case is if you try to subtract two values calculated separately and matching almost exactly. This should be avoided.

dragonwriter 6y ago

Not so much floating point as “using float point type for exact decimal literals”.

arnon 6y ago
Fair...

bidirectional 6y ago

Not at all. For all it's faults, floating point is incredibly fast. It's not some convenience hack that we lazy programmers have come up with, it's an incredibly quick way to do numerical computation. It will always have it's place (sometimes even in finance to represent money, to many people's shock).

lifthrasiir 6y ago
To add to that, in modern processors FP calculation is faster than integer calculation, both in terms of latency and throughput (as long as you don't hit subnormal numbers). This is very unintuitive and mostly due to disproportional demands.