4 ms·

Floats are such an embarrassment to have to explain to new developers. Then you have to explain that there is no intention to fix it. And then there are all the

by 6510 3y ago

Floats are such an embarrassment to have to explain to new developers. Then you have to explain that there is no intention to fix it. And then there are all these developers who make the field look like a cult by endlessly chanting how it works the way it should work. Who seem not to know the difference between correct and incorrect, the difference between right and wrong.

FabHK 3y ago
There are infinitely many real numbers (uncountably even), but your computer can only handle finitely many things. Something has to give. While many people complain about it, IEEE 754 is a very thoughtful and thorough standard, and the reason it is still around is that nothing unambiguously better has been proposed yet.
- Groxx 3y ago
  For the vast majority of uses: big-number libraries are far better. They act like people expect numbers to act. Save floats for performance optimizations. Start out correct, optimize when you can show both need for speed and non-need for correctness.
  - galangalalgol 3y ago
    If you are doing signal processing, ML, simulations, gaming or the like, you can know going in that you will need to use primitive types from a performance standpoint which sometimes means fixed point but often means floating point.
    - aidenn0 3y ago
      Many simulations and gaming are both better off using fixed point. The range available with a 64-bit integer is staggering and precision varying with distance from an arbitrary origin is a bug, not a feature for those cases.
      - gyrovagueGeist 3y ago
        This is absolutely not the case for most physical simulations where the significant digits are more important than fixed decimal accuracy.
        aidenn0 3y ago
        Consider a simple newtonian physics simulation. What you say is probably true for both force and acceleration. It may be true for velocity, but is almost certainly not true for position. Consider an object with a very small velocity traveling away from the origin with no forces acting upon it. With floating-point arithmetic, it will eventually stop at some distance away from the origin. In general objects with small velocity will only move if they are close to the origin.
        gyrovagueGeist 3y ago
        Sure, but most PIC methods are centered at their element or local domain. (You can likely even do the same with ewald/nbody). I definitely agree with you though if you need high accuracy on position when everything is on the same huge grid. A simulation that computes bulk properties by simulating particles directly is very rare. At least for me. Observable like temperature, electric/magnetic field strength, stress, velocity, etc are much more common outputs. These often all vary over many order of magnitude across a domain. Working with continuum mechanics, performing large inverted solves on matrices in fixed precision, or heck, even an FFT sounds terrifying to guard against overflow.
      - thebooktocome 3y ago
        When exp(45) overflows that 64-bit integer doesn’t seem so impressive.
      - Dylan16807 3y ago
        The range available with 54 bit fixed point is also staggering. And almost any code that would work with 54 bit fixed point works even better with 64 bit floating point, and the floating point version is much easier to code. So sure, floating point isn't optional when you start off assuming the same number of bits. But if you treat it as a small overhead to make code more robust to large numbers, easier to code, and often faster to run, then it looks a lot better.
    - lmm 3y ago
      Occasionally yes. But the niche where you need something other than decimal bignums is narrow, and the niche where you need that but don't need to get into the weeds of custom float formats etc. is narrower still. IEEE 754 shouldn't be front-and-center in general-purpose programming languages.
      - galangalalgol 3y ago
        I've never even heard of people using custom float formats outside of an fpga. And bignum seems useful for things that are not cpu-bound. If you need simd, or vectorization, including gpu acceleration, you use the format the hardware and compiler/optimizer support to meet your throughput needs. And on x86_64 aarch64 or spirv, thats a ieee floating point for better or worse.
        lmm 3y ago
        > I've never even heard of people using custom float formats outside of an fpga. It's normal and expected for ML these days (not completely custom, but smaller than the IEEE floats). And I think for gaming graphics too. > If you need simd, or vectorization, including gpu acceleration, you use the format the hardware and compiler/optimizer support to meet your throughput needs. And on x86_64 aarch64 or spirv, thats a ieee floating point for better or worse. If you need to do it on the CPU, sure. (Even then, you probably don't want to use full IEEE754, you will likely disable subnormals for performance). If you're using the GPU then you'll likely be using something else. The niche that IEEE754 fits is pretty narrow - it works for non-HPC physics simulations because it was designed for that, but that's about it.
  - jcranmer 3y ago
    Big-number libraries break down if you want to do anything other than be a 4-function calculator (and honestly, even division can be questionable, since some big-number libraries use fixed base-10 and thus even ⅓ isn't accurately representable). Want to throw in a call to sin or exp? Your only realistic option is some sort of floating-point, and that quickly boils down to "do we use IEEE 754 single or double precision? Or are we in the rare case when we need quad [1]?" [1] I believe the biggest use of quad precision is evaluating the accuracy of double precision arithmetic.
  - constantcrying 3y ago
    >For the vast majority of uses: big-number libraries are far better. They act like people expect numbers to act. They are usually useless. Almost all programs need fast floats.
    - 6510 3y ago
      Languages should cater to all programmers and all programs not most. I never needed fast anything, things are plenty fast for my needs, insanely fast.
      - constantcrying 3y ago
        People like you are the reason why, even though computers have become incredibly fast, almost all software is complete shit and geting worse faster than hardware can keep up with. If your software isn't manipulating Gigabytes of data or solving a hard math problem everything should be instantaneous. Just because you might not understand basic CS knowledge is no reason to deprive the world of the best approximation to real numbers we have. By the way. Big num libraries absolutely do not solve the problems of float. They, just as floats, can suffer from unbounded large errors. And 0.3 isn't representable regardless how long you make mantissa and exponent.
        6510 3y ago
        > People like you are the reason why, even though computers have become incredibly fast, almost all software is complete shit and geting worse faster than hardware can keep up with. I appreciate the sentiment but in this case I do not want to hurl around a bignum lib to add up 2 tiny numbers. > If your software isn't manipulating Gigabytes of data or solving a hard math problem everything should be instantaneous. I agree, NO amount of bandwidth should be spend on this, build in functionality should do the small number arithmetic. > Just because you might not understand basic CS knowledge is no reason to deprive the world of the best approximation to real numbers we have. Meanwhile, back on the construction site I'm hammering screws. No, I don't want to take your nails away from you.
deleted 3y ago
[deleted]
dikei 3y ago
I'm sorry, but what's hard to say about "all floating point arithmetic operations can contain errors". It's such a simple statement, that every developers learn in CS 101, that you don't even need to understand the reason to remember.
dreamcompiler 3y ago
Floats cannot be "fixed" because they are not broken. Floats are an extremely limited subset of the general rationals, and the general rationals are an extremely limited subset of the general reals. It's theoretically impossible to compute with the general reals. Full stop. It's theoretically possible to compute with the general rationals (Common Lisp for example lets you do so) but it's often impractical because you can easily end up running out of memory and/or time for the computation to complete. It's practical to compute with floats because you can guarantee hard limits on memory usage and time. You just have to live with the fact that you can't represent the vast majority of real numbers and you lose associative arithmetic. There are various other, weirder, computational number systems out there but the tradeoffs don't go away; they just move around.
- jcranmer 3y ago
  > It's theoretically impossible to compute with the general reals. Full stop. You could limit yourself to the computable numbers. Although you do have an issue determining equality on general computable numbers... But yeah, I would agree that floating-point is generally the best compromise out there for a general-purpose computational approximation to real numbers.
  - deleted 3y ago
    [deleted]
- constantcrying 3y ago
  >It's theoretically impossible to compute with the general reals. Full stop. While that is true. It is possible to make mathematical true statememts about results of real number arithmetic on a computer. >you lose associative arithmetic. People like to complain about this, but it is a mathematical consequence of the reuirements. You can not fix that problem.
drivebycomment 3y ago
So, what do you think is the fix / "correct" / "right" ?
constantcrying 3y ago
All of the flaws of floating point numbers are the mathematical results of its requirements. You have a fixed size datatype which almost always behaves like the real numbers. Floats can be fixed just as much free energy can be invented.

by 6510 3y ago

FabHK 3y ago

There are infinitely many real numbers (uncountably even), but your computer can only handle finitely many things. Something has to give. While many people complain about it, IEEE 754 is a very thoughtful and thorough standard, and the reason it is still around is that nothing unambiguously better has been proposed yet.

Groxx 3y ago
For the vast majority of uses: big-number libraries are far better. They act like people expect numbers to act. Save floats for performance optimizations. Start out correct, optimize when you can show both need for speed and non-need for correctness.
- galangalalgol 3y ago
  If you are doing signal processing, ML, simulations, gaming or the like, you can know going in that you will need to use primitive types from a performance standpoint which sometimes means fixed point but often means floating point.
  - aidenn0 3y ago
    Many simulations and gaming are both better off using fixed point. The range available with a 64-bit integer is staggering and precision varying with distance from an arbitrary origin is a bug, not a feature for those cases.
    - gyrovagueGeist 3y ago
      This is absolutely not the case for most physical simulations where the significant digits are more important than fixed decimal accuracy.
      - aidenn0 3y ago
        Consider a simple newtonian physics simulation. What you say is probably true for both force and acceleration. It may be true for velocity, but is almost certainly not true for position. Consider an object with a very small velocity traveling away from the origin with no forces acting upon it. With floating-point arithmetic, it will eventually stop at some distance away from the origin. In general objects with small velocity will only move if they are close to the origin.
        gyrovagueGeist 3y ago
        Sure, but most PIC methods are centered at their element or local domain. (You can likely even do the same with ewald/nbody). I definitely agree with you though if you need high accuracy on position when everything is on the same huge grid. A simulation that computes bulk properties by simulating particles directly is very rare. At least for me. Observable like temperature, electric/magnetic field strength, stress, velocity, etc are much more common outputs. These often all vary over many order of magnitude across a domain. Working with continuum mechanics, performing large inverted solves on matrices in fixed precision, or heck, even an FFT sounds terrifying to guard against overflow.
    - thebooktocome 3y ago
      When exp(45) overflows that 64-bit integer doesn’t seem so impressive.
    - Dylan16807 3y ago
      The range available with 54 bit fixed point is also staggering. And almost any code that would work with 54 bit fixed point works even better with 64 bit floating point, and the floating point version is much easier to code. So sure, floating point isn't optional when you start off assuming the same number of bits. But if you treat it as a small overhead to make code more robust to large numbers, easier to code, and often faster to run, then it looks a lot better.
  - lmm 3y ago
    Occasionally yes. But the niche where you need something other than decimal bignums is narrow, and the niche where you need that but don't need to get into the weeds of custom float formats etc. is narrower still. IEEE 754 shouldn't be front-and-center in general-purpose programming languages.
    - galangalalgol 3y ago
      I've never even heard of people using custom float formats outside of an fpga. And bignum seems useful for things that are not cpu-bound. If you need simd, or vectorization, including gpu acceleration, you use the format the hardware and compiler/optimizer support to meet your throughput needs. And on x86_64 aarch64 or spirv, thats a ieee floating point for better or worse.
      - lmm 3y ago
        > I've never even heard of people using custom float formats outside of an fpga. It's normal and expected for ML these days (not completely custom, but smaller than the IEEE floats). And I think for gaming graphics too. > If you need simd, or vectorization, including gpu acceleration, you use the format the hardware and compiler/optimizer support to meet your throughput needs. And on x86_64 aarch64 or spirv, thats a ieee floating point for better or worse. If you need to do it on the CPU, sure. (Even then, you probably don't want to use full IEEE754, you will likely disable subnormals for performance). If you're using the GPU then you'll likely be using something else. The niche that IEEE754 fits is pretty narrow - it works for non-HPC physics simulations because it was designed for that, but that's about it.
- jcranmer 3y ago
  Big-number libraries break down if you want to do anything other than be a 4-function calculator (and honestly, even division can be questionable, since some big-number libraries use fixed base-10 and thus even ⅓ isn't accurately representable). Want to throw in a call to sin or exp? Your only realistic option is some sort of floating-point, and that quickly boils down to "do we use IEEE 754 single or double precision? Or are we in the rare case when we need quad [1]?" [1] I believe the biggest use of quad precision is evaluating the accuracy of double precision arithmetic.
- constantcrying 3y ago
  >For the vast majority of uses: big-number libraries are far better. They act like people expect numbers to act. They are usually useless. Almost all programs need fast floats.
  - 6510 3y ago
    Languages should cater to all programmers and all programs not most. I never needed fast anything, things are plenty fast for my needs, insanely fast.
    - constantcrying 3y ago
      People like you are the reason why, even though computers have become incredibly fast, almost all software is complete shit and geting worse faster than hardware can keep up with. If your software isn't manipulating Gigabytes of data or solving a hard math problem everything should be instantaneous. Just because you might not understand basic CS knowledge is no reason to deprive the world of the best approximation to real numbers we have. By the way. Big num libraries absolutely do not solve the problems of float. They, just as floats, can suffer from unbounded large errors. And 0.3 isn't representable regardless how long you make mantissa and exponent.
      - 6510 3y ago
        > People like you are the reason why, even though computers have become incredibly fast, almost all software is complete shit and geting worse faster than hardware can keep up with. I appreciate the sentiment but in this case I do not want to hurl around a bignum lib to add up 2 tiny numbers. > If your software isn't manipulating Gigabytes of data or solving a hard math problem everything should be instantaneous. I agree, NO amount of bandwidth should be spend on this, build in functionality should do the small number arithmetic. > Just because you might not understand basic CS knowledge is no reason to deprive the world of the best approximation to real numbers we have. Meanwhile, back on the construction site I'm hammering screws. No, I don't want to take your nails away from you.

deleted 3y ago

[deleted]

dikei 3y ago

I'm sorry, but what's hard to say about "all floating point arithmetic operations can contain errors". It's such a simple statement, that every developers learn in CS 101, that you don't even need to understand the reason to remember.

dreamcompiler 3y ago

Floats cannot be "fixed" because they are not broken. Floats are an extremely limited subset of the general rationals, and the general rationals are an extremely limited subset of the general reals. It's theoretically impossible to compute with the general reals. Full stop. It's theoretically possible to compute with the general rationals (Common Lisp for example lets you do so) but it's often impractical because you can easily end up running out of memory and/or time for the computation to complete. It's practical to compute with floats because you can guarantee hard limits on memory usage and time. You just have to live with the fact that you can't represent the vast majority of real numbers and you lose associative arithmetic. There are various other, weirder, computational number systems out there but the tradeoffs don't go away; they just move around.

jcranmer 3y ago
> It's theoretically impossible to compute with the general reals. Full stop. You could limit yourself to the computable numbers. Although you do have an issue determining equality on general computable numbers... But yeah, I would agree that floating-point is generally the best compromise out there for a general-purpose computational approximation to real numbers.
- deleted 3y ago
  [deleted]
constantcrying 3y ago
>It's theoretically impossible to compute with the general reals. Full stop. While that is true. It is possible to make mathematical true statememts about results of real number arithmetic on a computer. >you lose associative arithmetic. People like to complain about this, but it is a mathematical consequence of the reuirements. You can not fix that problem.

drivebycomment 3y ago

So, what do you think is the fix / "correct" / "right" ?

constantcrying 3y ago

All of the flaws of floating point numbers are the mathematical results of its requirements. You have a fixed size datatype which almost always behaves like the real numbers. Floats can be fixed just as much free energy can be invented.