6 ms·

Exposing Floating Point (2019)

71 points by eindiran 3y ago

CoastalCoder 3y ago
Anyone know how he generates those illustrations? If he's not keeping it a secret, I'd love to have that in my toolbox.
- rtheunissen 3y ago
  You can view the source on the page, it's all there.
- gavinhoward 3y ago
  IIRC, custom WebGL stuff that he writes himself. So he's a 10x technical writer and a 10x developer.
- bigdict 3y ago
  It's literally in the page JS source.
  - turtledragonfly 3y ago
    I don't entirely follow... Most of the diagrams in this article are SVG files. Yes, you can look at the source (a big pile of numbers), but that doesn't say how they were created. I realize that a lot of this author's other articles have animated WebGL stuff, and the source code is illuminating, but not so much here, unless I'm missing something.
    - bigdict 3y ago
      You are right, my comment applies to other articles by this author but not this one.
- eindiran 3y ago
  I'm not certain of what tool he is using to generate the SVG files but a great option is Inkscape, which is FOSS and free-as-in-beer. If you want an illustration tool, Inkscape is definitely the best non-Adobe option I've used (and as someone with a monthly subscription to all of the Adobe tools, probably the best option overall).
turtledragonfly 3y ago
Pretty solid article. I'm glad there was mention of the non-uniform-distribution of floats, though my biased view wishes there were a little more emphasis that floats really really want to be "near 1", to keep precision. The practice of normalizing data to a [0,1] space, doing your math there, then transforming back to "big" space is useful not just for conceptual simplicity, but also for maintaining precision in intermediate results. Also, there's a pretty good range of integers that can be represented exactly by floats; one of the reasons the Lua developers were resistant to adding an integer type for many years. I believe you can represent about 24 bits of exact integers in a 32 bit float. I've been wrangling with fixed-point representations a lot lately, so the differences are stark in my mind (:
- deleted 3y ago
  [deleted]
- Veserv 3y ago
  Floating point is a piecewise linear approximation of a exponential function where each piece has fixed precision. For FP32, which has 23 mantissa bits, you divide each piece into 2^23 evenly spaced segments e.g. between 1 and 2 there are 2^23 segments and between 2 and 4 there are 2^23 segments, etc. The strength of floating point is that it bounds % error rather than absolute error. Similar to how scientific notation has a % error represented in the form of number of significant figures.
  - mananaysiempre 3y ago
    > The strength of floating point is that it bounds % error rather than absolute error. Similar to how scientific notation has a % error represented in the form of number of significant figures. Not just similar, precisely the same: scientific notation with a fixed number of significant figures is pencil-and-paper decimal floats. Having mentioned decimal floats, though, I have to note that they have worse precision bounds because the pieces in the piecewise approximation are larger. For example, with four binary digits, one unit of last place expressed as a relative error lies from binary 0.001 / 1.111 to binary 0.001 / 1.001, that is to say from about 2^-4 to about 2^-3, a factor of two, whereas with four decimal digits it will be from 0.001 / 9.999 to 0.001 / 1.001, or about 10^-4 to about 10^-3, a factor of ten. The usual phrasing is that the wobble of the number system is smaller with a smaller radix. Trading more wobble for less digits is reasonable when calculating by hand, but makes no sense on a computer. Even on paper some books will tell you to refrain from rounding down to ...1 when a value ends in ...11, ...12, or ...13 in order to avoid the worst of the wobble.
jbandela1 3y ago
> It seems that there is no way to configure the printing of floating point values to automatically maintain exact number of decimal digits needed to accurately represent the value. C++17 has to_ chars/from_chars and c++20 has std::format which will output the shortest output the shortest output required to round trip the floating point value exactly.

71 points by eindiran 3y ago

CoastalCoder 3y ago

Anyone know how he generates those illustrations? If he's not keeping it a secret, I'd love to have that in my toolbox.

rtheunissen 3y ago
You can view the source on the page, it's all there.
gavinhoward 3y ago
IIRC, custom WebGL stuff that he writes himself. So he's a 10x technical writer and a 10x developer.
bigdict 3y ago
It's literally in the page JS source.
- turtledragonfly 3y ago
  I don't entirely follow... Most of the diagrams in this article are SVG files. Yes, you can look at the source (a big pile of numbers), but that doesn't say how they were created. I realize that a lot of this author's other articles have animated WebGL stuff, and the source code is illuminating, but not so much here, unless I'm missing something.
  - bigdict 3y ago
    You are right, my comment applies to other articles by this author but not this one.
eindiran 3y ago
I'm not certain of what tool he is using to generate the SVG files but a great option is Inkscape, which is FOSS and free-as-in-beer. If you want an illustration tool, Inkscape is definitely the best non-Adobe option I've used (and as someone with a monthly subscription to all of the Adobe tools, probably the best option overall).

turtledragonfly 3y ago

Pretty solid article. I'm glad there was mention of the non-uniform-distribution of floats, though my biased view wishes there were a little more emphasis that floats really really want to be "near 1", to keep precision. The practice of normalizing data to a [0,1] space, doing your math there, then transforming back to "big" space is useful not just for conceptual simplicity, but also for maintaining precision in intermediate results. Also, there's a pretty good range of integers that can be represented exactly by floats; one of the reasons the Lua developers were resistant to adding an integer type for many years. I believe you can represent about 24 bits of exact integers in a 32 bit float. I've been wrangling with fixed-point representations a lot lately, so the differences are stark in my mind (:

deleted 3y ago
[deleted]
Veserv 3y ago
Floating point is a piecewise linear approximation of a exponential function where each piece has fixed precision. For FP32, which has 23 mantissa bits, you divide each piece into 2^23 evenly spaced segments e.g. between 1 and 2 there are 2^23 segments and between 2 and 4 there are 2^23 segments, etc. The strength of floating point is that it bounds % error rather than absolute error. Similar to how scientific notation has a % error represented in the form of number of significant figures.
- mananaysiempre 3y ago
  > The strength of floating point is that it bounds % error rather than absolute error. Similar to how scientific notation has a % error represented in the form of number of significant figures. Not just similar, precisely the same: scientific notation with a fixed number of significant figures is pencil-and-paper decimal floats. Having mentioned decimal floats, though, I have to note that they have worse precision bounds because the pieces in the piecewise approximation are larger. For example, with four binary digits, one unit of last place expressed as a relative error lies from binary 0.001 / 1.111 to binary 0.001 / 1.001, that is to say from about 2^-4 to about 2^-3, a factor of two, whereas with four decimal digits it will be from 0.001 / 9.999 to 0.001 / 1.001, or about 10^-4 to about 10^-3, a factor of ten. The usual phrasing is that the wobble of the number system is smaller with a smaller radix. Trading more wobble for less digits is reasonable when calculating by hand, but makes no sense on a computer. Even on paper some books will tell you to refrain from rounding down to ...1 when a value ends in ...11, ...12, or ...13 in order to avoid the worst of the wobble.

jbandela1 3y ago

> It seems that there is no way to configure the printing of floating point values to automatically maintain exact number of decimal digits needed to accurately represent the value. C++17 has to_ chars/from_chars and c++20 has std::format which will output the shortest output the shortest output required to round trip the floating point value exactly.