3 ms·

IEEE floats are exact. That there exist real numbers that aren't IEEE floats is why we say IEEE floats are a subset of real numbers, and why operations on IEEE

by Veedrac 11y ago

IEEE floats are exact. That there exist real numbers that aren't IEEE floats is why we say IEEE floats are a subset of real numbers, and why operations on IEEE floats have to round to a representable value.

jacobolus 11y ago
Floats are also a subset of the rational numbers. To be specific, they’re a particular subset of the dyadic rationals with a bounded number of significant figures and a bounded exponent. I think it’s misleading / counterproductive to talk about IEEE floats as subsets of “real numbers”, because e.g. Math.PI is a clearly different thing than the “real number” π (whatever that might be). Calling floats “exact” is also IMO problematic, since in practice every computation using floats involves rounding. (I also think it’s problematic for Gustafson to call the numbers in his system “real numbers”, even if you could in principle build new hardware with the idea to perform arithmetic on any particular (decided before hardware design) small set of real numbers you want.)
- Veedrac 11y ago
  I say subset of the real numbers because other number systems don't restrict themselves to rationals. Logarithmic number systems, for example, almost never express rational numbers. It's true that most floating point computations involve rounding, but there's no contradiction in that. If you think floats are inexact you can get into a lot of confusion about things; you might miss, for example, that `(a + b) / 2` is exact (except on overflow to infinity, or languages that don't enforce floating point correctness) or that Javascript's integers are actually just floats. Considering floats as inexact might lead you to believe they represent ranges of numbers, eg. x ± ½eps, which is confusing. I've seen more damage from this belief than most other myths about floats, at least among experienced programmers.
  - jacobolus 11y ago
    My point though is that every non-trivial computation involving floats includes some rounding. Typically the inputs are also coming from some inexact source. The whole point of floating point arithmetic is to be approximate, maintaining only a particular precision because otherwise the number of bits in the answer doubles every time we multiply. If someone cares about exact arithmetic, floats are the wrong tool for the job. As such, to do pretty much anything with floats you have to keep some idea in mind of the relative accuracy of various values, and the condition number of every operation/algorithm. None of this is explicitly included in the number, you just have to track it manually. Instead of thinking of a float coming out of some algorithm as an exact value, you should generally think of it as a representative of some implicit range of possible values, a range typically somewhat bigger than ±½eps.
    - Veedrac 11y ago
      I gave examples of floating computation that doesn't involve rounding. A lot of programs depend on it; Javascript ones in particular. In fact, in a case like `random() -> [0, 1)`, one often relies on the exactness of the bounds. Rather than thinking of a float as some range of values, you should think of it as a randomized sample inside some (probably unknown) range.

by Veedrac 11y ago

jacobolus 11y ago

Floats are also a subset of the rational numbers. To be specific, they’re a particular subset of the dyadic rationals with a bounded number of significant figures and a bounded exponent. I think it’s misleading / counterproductive to talk about IEEE floats as subsets of “real numbers”, because e.g. Math.PI is a clearly different thing than the “real number” π (whatever that might be). Calling floats “exact” is also IMO problematic, since in practice every computation using floats involves rounding. (I also think it’s problematic for Gustafson to call the numbers in his system “real numbers”, even if you could in principle build new hardware with the idea to perform arithmetic on any particular (decided before hardware design) small set of real numbers you want.)

Veedrac 11y ago
I say subset of the real numbers because other number systems don't restrict themselves to rationals. Logarithmic number systems, for example, almost never express rational numbers. It's true that most floating point computations involve rounding, but there's no contradiction in that. If you think floats are inexact you can get into a lot of confusion about things; you might miss, for example, that `(a + b) / 2` is exact (except on overflow to infinity, or languages that don't enforce floating point correctness) or that Javascript's integers are actually just floats. Considering floats as inexact might lead you to believe they represent ranges of numbers, eg. x ± ½eps, which is confusing. I've seen more damage from this belief than most other myths about floats, at least among experienced programmers.
- jacobolus 11y ago
  My point though is that every non-trivial computation involving floats includes some rounding. Typically the inputs are also coming from some inexact source. The whole point of floating point arithmetic is to be approximate, maintaining only a particular precision because otherwise the number of bits in the answer doubles every time we multiply. If someone cares about exact arithmetic, floats are the wrong tool for the job. As such, to do pretty much anything with floats you have to keep some idea in mind of the relative accuracy of various values, and the condition number of every operation/algorithm. None of this is explicitly included in the number, you just have to track it manually. Instead of thinking of a float coming out of some algorithm as an exact value, you should generally think of it as a representative of some implicit range of possible values, a range typically somewhat bigger than ±½eps.
  - Veedrac 11y ago
    I gave examples of floating computation that doesn't involve rounding. A lot of programs depend on it; Javascript ones in particular. In fact, in a case like `random() -> [0, 1)`, one often relies on the exactness of the bounds. Rather than thinking of a float as some range of values, you should think of it as a randomized sample inside some (probably unknown) range.