9 ms·

Float Toy

649 points by lelf 7y ago

FriedPickles 7y ago
This rocks! I bet I grokked it way faster with this toy compared to a written explanation. An ounce of demonstration is worth a pound of documentation.
- FabHK 7y ago
  Did you grok infs and NaNs (the two different kinds, and their payload) and zero (both zeros?) and subnormal numbers? Otherwise a pound of written explanation might still make sense.
earthboundkid 7y ago
Neat, but it should special case -0.
Aardwolf 7y ago
So great! I believe it could also show the difference between quiet and signaling NaN
TYPE_FASTER 7y ago
Awesome. It would be cool to incorporate it into the Wikipedia article (or at least, link to it).
vardump 7y ago
Nice! Would be nice to get various 16-bit float formats as well. They're getting more and more common in graphics and NN domains. Or perhaps generic n-width exponent and m-width mantissa.
- thomasballinger 7y ago
  rreusser made a more generic version inspired by Float Toy: https://observablehq.com/@rreusser/binary-input https://observablehq.com/@rreusser/binary-input
- dunham 7y ago
  There is also an 8-bit float used in Lucene. I was surprised that someone was using an 8-bit float format when I first came across that.
evanjonr 7y ago
A similar probably more well presented site: https://float.exposed/0x40490fdb https://float.exposed/0x40490fdb
- deleted 7y ago
  [deleted]
- zamadatix 7y ago
  It's certainly got more information but that combined with the "exact" values makes it a bit overwhelming/hard to understand, especially if you're trying to get a feel for it the first time and the half and the double have the same number of digits on screen. It's almost made more for investigation of something you ran across or need to hardcode than gaining intuition. But most importantly I think best feature of the one in the submission is something it doesn't even mention in the description - it has a "click and drag to write bits" feature that I think is much easier to use/follow than the 2 dragging section on that one.
  - jacobolus 7y ago
    The 2-dragging section is much more efficient if you want to roughly pick arbitrary floats, and gives a better spatial map of the range of possible floats. The individual bit flips also can be useful. It really depends what you want to use it for.
  - TazeTSchnitzel 7y ago
    You can click to flip individual bits on float.exposed too, and they don't have “the same number of digits”…?
    - mkl 7y ago
      They do have the same number of decimal digits displayed. You can see it by clicking between "float" and "double": the decimal representation stays the same.
      - 7y ago
AceJohnny2 7y ago
With the 32-bit one, if I set the highest bit of the exponent from 1 to 0, the exponent goes to -126 (ok), but if I then toggle the lowest bit of the exponent (rightmost green bit), it modifies the mantissa, not the exponent. Is that a bug or do I not understand how IEEE754 works (very likely :p)?
- sillysaurusx 7y ago
  Hmm, could you show a screenshot? When I toggle the rightmost green bit, it doubles the value as expected. Note that float toy doesn't actually show the mantissa value, only the final value. So if the value goes from 6.709679e-39 to 1.8464622e-38, then it was doubled. EDIT: Oh! I see what you mean. I was wrong; obviously it does show the mantissa. Yes, that seems like a bug. Shouldn't the mantissa always be in the range [0.0 .. 1.0)? On the other hand, maybe it behaves differently when the exponent is negative. If you don't understand how IEEE754 works, then apparently I don't either, so thanks for pointing that out. EDIT2: Ok, comparing with https://float.exposed/0x40490fdb https://float.exposed/0x40490fdb makes me more certain that both sites are wrong. On float.exposed, the exponent can’t ever go negative, which is clearly bogus. Exponent “128” should read “1”. It’s showing the raw binary unsigned value, which is misleading, since you want to know the int8 signed value. But toggling the exponent bits should never affect the mantissa. That’s also bogus. I think float toy is the superior visualization, and they should fix the bug. I wonder if there’s anywhere to report it....
  - sillysaurusx 7y ago
    Turns out, there's no bug in float toy. The only case where the mantissa becomes 0.xx instead of 1.xx is when exponent == 0, which is a subnormal number and is working as expected.
- asdvjonbn 7y ago
  That's called a 'subnormal number'. https://stackoverflow.com/questions/8341395/what-is-a-subnormal-floating-point-number https://stackoverflow.com/questions/8341395/what-is-a-subnor... If the exponent is 0, then: the leading bit becomes 0 the exponent is fixed to -126 (not -127 as if we didn't have this exception) Such numbers are called subnormal numbers (or denormal numbers which is synonym). All normal floating-point numbers have an implicit leading significant bit of 1. If the exponent field goes from 1 to 0, then this corresponds to changing the implicit leading bit to a 0, not to decreasing the exponent.
divbzero 7y ago
Does anyone have a similar “toy” for Unicode? (UTF-8 vs. UTF-16 vs. UTF-32)
seanalltogether 7y ago
I feel like I groked floating point numbers easier when looking at 16bit floating point numbers. The values are much smaller and easier to juggle in your head. The fact that the fraction part literally translated to x / 1024 suddenly became clear to me when you're staring at just 10 bits https://en.wikipedia.org/wiki/Half-precision_floating-point_format#Half_precision_examples https://en.wikipedia.org/wiki/Half-precision_floating-point_...
- jacobolus 7y ago
  It’s even easier if you use 8-bit floats. Then there are only 256 of them and you can plot them all.
TheMagicHorsey 7y ago
I just realized I did not actually understand how floats worked. I just kind of assumed I knew ... and I did not actually know the details correctly. Thanks for this!
- baq 7y ago
  well, the details are an IEEE standard, after all. most people should be fine knowing why 9.999999999 is equal to 10 without having to go to infinity.
mrspeaker 7y ago
I was playing with this yesterday for about an hour... getting so frustrated trying to figure out how to count from 0 to 20. But when I got there, I suddenly had a "omg, I know floating point!" moment. Then this morning on the train I started wondering: "wait, but how does floating point addition even make sense?!"... I'm still stumped on that one - I need "Float Addition Toy" ;) (Pro-tip: it took me way too long to figure out that you can click and drag to set/clear the bits quickly)
- sillysaurusx 7y ago
  https://www.doc.ic.ac.uk/~eedwards/compsys/float/ https://www.doc.ic.ac.uk/~eedwards/compsys/float/ Basically, you rewrite one of the numbers so that the exponents match, then sum the mantissas. There are a few extra steps, though: https://imgur.com/fgOH4sS https://imgur.com/fgOH4sS A "floating point addition toy" would be way cool!
- Analemma_ 7y ago
  https://ciechanow.ski/exposing-floating-point/ https://ciechanow.ski/exposing-floating-point/ is the clearest and most coherent explanation of floating-point numbers I've found; it was really what gave me the lightbulb moment I had been missing. This site is a great companion to it though.
  - animal531 7y ago
    Thanks, that's a great write-up.
amelius 7y ago
Suggestions for improvement: Add a (+) and (-) button to move up or down to the nearest floating point number. Show the local precision (not sure if this is the correct term), i.e. the maximum difference between the current float and the two nearest ones.
- mceachen 7y ago
  Nice. Also, if you make the css transition for what bits flip on ++ and -- slowish, you can see what's going on.
fctorial 7y ago
So it's scientific notation in base 2
- netsharc 7y ago
  He could've explained the red bits better, it's 2^-1, 2^-2, 2^-3, etc So if the third red bit is 1, it's 1 x 2^-3, i.e 1 x 1/(2^3), i.e. 1 x 1/8, i.e. 0.125.
  - fctorial 7y ago
    And +/-0 are special cased.
pcwalton 7y ago
The thing that made floating point easy to understand for me was the concept of a dyadic rational [1]. Those are just the rational numbers whose denominator is a power of two (e.g. 1/2, 3/16, 61/32). Fundamentally, binary floating point represents dyadic rationals (not counting special stuff like NaN, infinity, etc.) To represent such a number in floating point form, just write it as a/2^b. Then the mantissa is a, and the exponent is -b. This intuitively makes it obvious why, for example, 1/3 can't be represented. I find the rational form like a/b easier to understand than the "scientific notation" form like a×2^b. [1]: https://en.wikipedia.org/wiki/Dyadic_rational https://en.wikipedia.org/wiki/Dyadic_rational
- jacobolus 7y ago
  > a/2^b. Then the mantissa is a, and the exponent is -b Why not just write a * 2^b. Your a here, if supposed to represent the mantissa, is not an integer. This form doesn’t really clarify the relationship to the concept of a dyadic rational. To write the floating point number as a dyadic rational treating a as an integer you would have something like (2^(53) + a) / 2^(53 - b). But arbitrary dydadic rationals cannot be represented by a single floating point number. If you want arbitrary dyadic rationals you need the sum of a list of floats. * * * I think what you might be getting at is starting by teaching people about fixed-point binary numbers? Or just binary fractions in general, analogous to the usual decimal fractions? That would be clearer to say than anything about “dyadic rationals”. After going through a typical primary school, people have the concept of “terminating decimals”, rather than a concept of “ten-adic rationals”.
- twic 7y ago
  I think of a float as an integer with a shift. Take the mantissa, stick on the implicit leading 1, put it in the middle of an infinitely big string of zeroes with a decimal (well, binary) point at the right-hand end, then shift it by the exponent (which can be positive or negative). If you're comfortable thinking about shifting integers, then perhaps this is a simple way to think about floats. For me, it's easier than thinking about them as fractions.
  - deleted 7y ago
    [deleted]
qwerty456127 7y ago
Very valuable. Thanks!
gautamcgoel 7y ago
This is awesome, I honestly learned more about floating point in five minutes of playing with this tool than I have in all my years as a coder. One of the main takeaways should be that 64 bits is usually way more precision than we need, 32 is usually more than enough. It would be cool if the tool also showed the new Bfloat16 type, so we could get some intuition for that as well. IIRC, Intel created Bfloat16 to better suit the needs of deep learning practitioners, who don't need much precision (especially for large values).
- moultano 7y ago
  32 is plenty to represent the numbers we care about, but you'll find you probably want 64 to do math with those numbers. 64 has always been enough for me to never have to care about the precision, but whenever I've used 32 to save some ram, I've ended up having to think carefully about how some operation would reduce the precision in a material way.
  - platz 7y ago
    At 32-bit you already have a whole 16 number gap per-bit in the 200 millions: 210,828,704 210,828,720
- nwallin 7y ago
  Notably, 32 bit floats are insufficiently precise for representing latitudes and longitudes. I seem to recall doing the math on it, and in California the next line of longitude representable by 32 bit floats was like half a mile away. Don't quote me on the number though. 32 bit integers are precise enough. If you define 180 degrees West as INT_MIN and 180 degrees East as INT_MAX your precision is about a foot. Although 32 bit floats are precise enough in some areas of Africa because it's closer to the 0,0 point. But it'll fall down if you're anywhere else in the world.
quelsolaar 7y ago
I think I found a bug: If you set all bits to 0 except the sign bit you get 0 when you should get -0. In IEEE floating point representation 0 and -0 is not the same. This is why: f *= 0; Is not the same operation as: f = 0; Since the former will retain the sign bit.
- tboerstad 7y ago
  Thanks for that bit of information, pun intended. I hope I'll never have to rely on it in production.
snek 7y ago
I really like this one: https://bartaz.github.io/ieee754-visualization/ https://bartaz.github.io/ieee754-visualization/
sj4nz 7y ago
Definitely looking forward to the Unum Type III alternative to floats. This is a nice visualization and it gives you an idea of how many NaN's waste the bits being used.
- pmarreck 7y ago
  yeah I was going to ask, what's the deal with all those wasted bits? I'm not a floating-point fan (I do all computations in integer wherever possible/feasible), but I've been following the universal number story since it was announced because it seems less painful/gross than IEEE floating-point on many fronts, here's a github with a good about page about it: https://github.com/stillwater-sc/universal https://github.com/stillwater-sc/universal
userbinator 7y ago
I wrote something similar ~25 years ago, when Win32 first appeared, and still use it today on the (relatively rare) occasion that I need to debug something that uses floating-point. One interesting thing you'll notice about IEEE754 is that the values sort naturally if you interpret them as sign-magnitude integers: 00000000 0 00000001 1.4e-45 (smallest representable positive value) 00000002 2.8e-45 (next-smallest representable positive value) 007fffff 1.1754942e-38 00800000 1.17549435e-38 3f000000 0.5 3f800000 1. 3fffffff 1.9999999 40000000 2. 40400000 3. 60000000 3.6893488e19 7f000000 1.7014118e38 7f7fffff 3.4028235e38 (largest finite value) 7f800000 Infinity 7f800001 first NaN 7fffffff last NaN In other words, if you count up in floats, it starts at 0 and goes up by increasingly larger amounts, then the number right "after" the "largest" is Infinity, followed by all the NaNs to the end of the range. The negative side is a "mirror image", the only difference being the highest (sign) bit is set.
- filmor 7y ago
  Just a little nit-pick: The NaNs always compare false, so this property only holds for values up to +inf (0x7f800000).
  - deleted 7y ago
    [deleted]
- vanderZwan 7y ago
  > One interesting thing you'll notice about IEEE754 is that the values sort naturally if you interpret them as sign-magnitude integers With a few bit-twiddling tricks you can even make them radix-sortable, as explained by Malte Skarupke: > Michael Herf has also discovered a good way to generalize this to floating point numbers: Reinterpret cast the float to a uint32, then flip every bit if the float was negative, but flip only the sign bit if the float was positive. The same trick works for doubles and uint64s. Michael Herf explains why this works in the linked piece, but the short version of it is this: Positive floating point numbers already sort correctly if we just reinterpret cast them to a uint32. The exponent comes before the mantissa, so we would sort by the exponent first, then by the mantissa. Everything works out. Negative floating point numbers however would sort the wrong way. Flipping all the bits on them fixes that. The final remaining problem is that positive floating point numbers need to sort as bigger than negative numbers, and the easiest way to do that is to flip the sign bit since it’s the most significant bit. Up until recently in most JS engines it was faster to sort typedarrays with a custom radix sort than with the built-in sort (seems to no longer be the case, thank goodness) [0] https://probablydance.com/2016/12/27/i-wrote-a-faster-sorting-algorithm/ https://probablydance.com/2016/12/27/i-wrote-a-faster-sortin... [1] http://www.codercorner.com/RadixSortRevisited.htm http://www.codercorner.com/RadixSortRevisited.htm [2] http://stereopsis.com/radix.html http://stereopsis.com/radix.html
kstenerud 7y ago
I deep dove into ieee754 binary and decimal float (DPD and BID) while researching how to compress floating point values [1]. For 25 years it had been a magic black box that I couldn't trust for comparisons or printing, but now I get it. I get why it happened this way, what the benefits were, how to mitigate against its weirdness, and why ultimately decimal float needs to replace binary float in future. It's definitely worth learning. https://github.com/kstenerud/compact-float/blob/master/compact-float-specification.md https://github.com/kstenerud/compact-float/blob/master/compa...
constexpr 7y ago
Wow, I didn't expect a four year old project of mine to hit HN front page! If you're interested in this, you might enjoy some of the stuff we're working on at Figma. Here's a link to learn more: https://www.figma.com/careers/ https://www.figma.com/careers/
- bostonvaulter2 7y ago
  I use figma at work and it's really neat. Keep up the good work! Now if you could only convince the designer to give me write access ;)
vortico 7y ago
Nice work on the "mouse down to toggle" and "drag to copy value" interaction, a much underused but useful tool to set a collection of boolean states.
kecskesadam 7y ago
https://www.h-schmidt.net/FloatConverter/IEEE754.html https://www.h-schmidt.net/FloatConverter/IEEE754.html
riceslush 7y ago
I built something similar https://www.isfloat.com/ https://www.isfloat.com/ It also tells you whether the decimal value given can be exactly represented as a float. For example https://www.isfloat.com/?v=0.3 https://www.isfloat.com/?v=0.3
kazinator 7y ago
Click, hold and swipe to quickly replicate a bit value across a range of places.

649 points by lelf 7y ago

This rocks! I bet I grokked it way faster with this toy compared to a written explanation. An ounce of demonstration is worth a pound of documentation.

FabHK 7y ago
Did you grok infs and NaNs (the two different kinds, and their payload) and zero (both zeros?) and subnormal numbers? Otherwise a pound of written explanation might still make sense.

earthboundkid 7y ago

Neat, but it should special case -0.

Aardwolf 7y ago

So great! I believe it could also show the difference between quiet and signaling NaN

TYPE_FASTER 7y ago

Awesome. It would be cool to incorporate it into the Wikipedia article (or at least, link to it).

vardump 7y ago

Nice! Would be nice to get various 16-bit float formats as well. They're getting more and more common in graphics and NN domains. Or perhaps generic n-width exponent and m-width mantissa.

thomasballinger 7y ago
rreusser made a more generic version inspired by Float Toy: https://observablehq.com/@rreusser/binary-input https://observablehq.com/@rreusser/binary-input
dunham 7y ago
There is also an 8-bit float used in Lucene. I was surprised that someone was using an 8-bit float format when I first came across that.

evanjonr 7y ago

A similar probably more well presented site: https://float.exposed/0x40490fdb https://float.exposed/0x40490fdb

deleted 7y ago
[deleted]
zamadatix 7y ago
It's certainly got more information but that combined with the "exact" values makes it a bit overwhelming/hard to understand, especially if you're trying to get a feel for it the first time and the half and the double have the same number of digits on screen. It's almost made more for investigation of something you ran across or need to hardcode than gaining intuition. But most importantly I think best feature of the one in the submission is something it doesn't even mention in the description - it has a "click and drag to write bits" feature that I think is much easier to use/follow than the 2 dragging section on that one.
- jacobolus 7y ago
  The 2-dragging section is much more efficient if you want to roughly pick arbitrary floats, and gives a better spatial map of the range of possible floats. The individual bit flips also can be useful. It really depends what you want to use it for.
- TazeTSchnitzel 7y ago
  You can click to flip individual bits on float.exposed too, and they don't have “the same number of digits”…?
  - mkl 7y ago
    They do have the same number of decimal digits displayed. You can see it by clicking between "float" and "double": the decimal representation stays the same.
    - 7y ago

AceJohnny2 7y ago

With the 32-bit one, if I set the highest bit of the exponent from 1 to 0, the exponent goes to -126 (ok), but if I then toggle the lowest bit of the exponent (rightmost green bit), it modifies the mantissa, not the exponent. Is that a bug or do I not understand how IEEE754 works (very likely :p)?

sillysaurusx 7y ago
Hmm, could you show a screenshot? When I toggle the rightmost green bit, it doubles the value as expected. Note that float toy doesn't actually show the mantissa value, only the final value. So if the value goes from 6.709679e-39 to 1.8464622e-38, then it was doubled. EDIT: Oh! I see what you mean. I was wrong; obviously it does show the mantissa. Yes, that seems like a bug. Shouldn't the mantissa always be in the range [0.0 .. 1.0)? On the other hand, maybe it behaves differently when the exponent is negative. If you don't understand how IEEE754 works, then apparently I don't either, so thanks for pointing that out. EDIT2: Ok, comparing with https://float.exposed/0x40490fdb https://float.exposed/0x40490fdb makes me more certain that both sites are wrong. On float.exposed, the exponent can’t ever go negative, which is clearly bogus. Exponent “128” should read “1”. It’s showing the raw binary unsigned value, which is misleading, since you want to know the int8 signed value. But toggling the exponent bits should never affect the mantissa. That’s also bogus. I think float toy is the superior visualization, and they should fix the bug. I wonder if there’s anywhere to report it....
- sillysaurusx 7y ago
  Turns out, there's no bug in float toy. The only case where the mantissa becomes 0.xx instead of 1.xx is when exponent == 0, which is a subnormal number and is working as expected.
asdvjonbn 7y ago
That's called a 'subnormal number'. https://stackoverflow.com/questions/8341395/what-is-a-subnormal-floating-point-number https://stackoverflow.com/questions/8341395/what-is-a-subnor... If the exponent is 0, then: the leading bit becomes 0 the exponent is fixed to -126 (not -127 as if we didn't have this exception) Such numbers are called subnormal numbers (or denormal numbers which is synonym). All normal floating-point numbers have an implicit leading significant bit of 1. If the exponent field goes from 1 to 0, then this corresponds to changing the implicit leading bit to a 0, not to decreasing the exponent.

divbzero 7y ago

Does anyone have a similar “toy” for Unicode? (UTF-8 vs. UTF-16 vs. UTF-32)

seanalltogether 7y ago

I feel like I groked floating point numbers easier when looking at 16bit floating point numbers. The values are much smaller and easier to juggle in your head. The fact that the fraction part literally translated to x / 1024 suddenly became clear to me when you're staring at just 10 bits https://en.wikipedia.org/wiki/Half-precision_floating-point_format#Half_precision_examples https://en.wikipedia.org/wiki/Half-precision_floating-point_...

jacobolus 7y ago
It’s even easier if you use 8-bit floats. Then there are only 256 of them and you can plot them all.

TheMagicHorsey 7y ago

I just realized I did not actually understand how floats worked. I just kind of assumed I knew ... and I did not actually know the details correctly. Thanks for this!

baq 7y ago
well, the details are an IEEE standard, after all. most people should be fine knowing why 9.999999999 is equal to 10 without having to go to infinity.

mrspeaker 7y ago

I was playing with this yesterday for about an hour... getting so frustrated trying to figure out how to count from 0 to 20. But when I got there, I suddenly had a "omg, I know floating point!" moment. Then this morning on the train I started wondering: "wait, but how does floating point addition even make sense?!"... I'm still stumped on that one - I need "Float Addition Toy" ;) (Pro-tip: it took me way too long to figure out that you can click and drag to set/clear the bits quickly)

sillysaurusx 7y ago
https://www.doc.ic.ac.uk/~eedwards/compsys/float/ https://www.doc.ic.ac.uk/~eedwards/compsys/float/ Basically, you rewrite one of the numbers so that the exponents match, then sum the mantissas. There are a few extra steps, though: https://imgur.com/fgOH4sS https://imgur.com/fgOH4sS A "floating point addition toy" would be way cool!
Analemma_ 7y ago
https://ciechanow.ski/exposing-floating-point/ https://ciechanow.ski/exposing-floating-point/ is the clearest and most coherent explanation of floating-point numbers I've found; it was really what gave me the lightbulb moment I had been missing. This site is a great companion to it though.
- animal531 7y ago
  Thanks, that's a great write-up.

amelius 7y ago

Suggestions for improvement: Add a (+) and (-) button to move up or down to the nearest floating point number. Show the local precision (not sure if this is the correct term), i.e. the maximum difference between the current float and the two nearest ones.

mceachen 7y ago
Nice. Also, if you make the css transition for what bits flip on ++ and -- slowish, you can see what's going on.

fctorial 7y ago

So it's scientific notation in base 2

netsharc 7y ago
He could've explained the red bits better, it's 2^-1, 2^-2, 2^-3, etc So if the third red bit is 1, it's 1 x 2^-3, i.e 1 x 1/(2^3), i.e. 1 x 1/8, i.e. 0.125.
- fctorial 7y ago
  And +/-0 are special cased.

pcwalton 7y ago

The thing that made floating point easy to understand for me was the concept of a dyadic rational [1]. Those are just the rational numbers whose denominator is a power of two (e.g. 1/2, 3/16, 61/32). Fundamentally, binary floating point represents dyadic rationals (not counting special stuff like NaN, infinity, etc.) To represent such a number in floating point form, just write it as a/2^b. Then the mantissa is a, and the exponent is -b. This intuitively makes it obvious why, for example, 1/3 can't be represented. I find the rational form like a/b easier to understand than the "scientific notation" form like a×2^b. [1]: https://en.wikipedia.org/wiki/Dyadic_rational https://en.wikipedia.org/wiki/Dyadic_rational

jacobolus 7y ago
> a/2^b. Then the mantissa is a, and the exponent is -b Why not just write a * 2^b. Your a here, if supposed to represent the mantissa, is not an integer. This form doesn’t really clarify the relationship to the concept of a dyadic rational. To write the floating point number as a dyadic rational treating a as an integer you would have something like (2^(53) + a) / 2^(53 - b). But arbitrary dydadic rationals cannot be represented by a single floating point number. If you want arbitrary dyadic rationals you need the sum of a list of floats. * * * I think what you might be getting at is starting by teaching people about fixed-point binary numbers? Or just binary fractions in general, analogous to the usual decimal fractions? That would be clearer to say than anything about “dyadic rationals”. After going through a typical primary school, people have the concept of “terminating decimals”, rather than a concept of “ten-adic rationals”.
twic 7y ago
I think of a float as an integer with a shift. Take the mantissa, stick on the implicit leading 1, put it in the middle of an infinitely big string of zeroes with a decimal (well, binary) point at the right-hand end, then shift it by the exponent (which can be positive or negative). If you're comfortable thinking about shifting integers, then perhaps this is a simple way to think about floats. For me, it's easier than thinking about them as fractions.
- deleted 7y ago
  [deleted]

qwerty456127 7y ago

Very valuable. Thanks!

gautamcgoel 7y ago

This is awesome, I honestly learned more about floating point in five minutes of playing with this tool than I have in all my years as a coder. One of the main takeaways should be that 64 bits is usually way more precision than we need, 32 is usually more than enough. It would be cool if the tool also showed the new Bfloat16 type, so we could get some intuition for that as well. IIRC, Intel created Bfloat16 to better suit the needs of deep learning practitioners, who don't need much precision (especially for large values).

moultano 7y ago
32 is plenty to represent the numbers we care about, but you'll find you probably want 64 to do math with those numbers. 64 has always been enough for me to never have to care about the precision, but whenever I've used 32 to save some ram, I've ended up having to think carefully about how some operation would reduce the precision in a material way.
- platz 7y ago
  At 32-bit you already have a whole 16 number gap per-bit in the 200 millions: 210,828,704 210,828,720
nwallin 7y ago
Notably, 32 bit floats are insufficiently precise for representing latitudes and longitudes. I seem to recall doing the math on it, and in California the next line of longitude representable by 32 bit floats was like half a mile away. Don't quote me on the number though. 32 bit integers are precise enough. If you define 180 degrees West as INT_MIN and 180 degrees East as INT_MAX your precision is about a foot. Although 32 bit floats are precise enough in some areas of Africa because it's closer to the 0,0 point. But it'll fall down if you're anywhere else in the world.

quelsolaar 7y ago

I think I found a bug: If you set all bits to 0 except the sign bit you get 0 when you should get -0. In IEEE floating point representation 0 and -0 is not the same. This is why: f *= 0; Is not the same operation as: f = 0; Since the former will retain the sign bit.

tboerstad 7y ago
Thanks for that bit of information, pun intended. I hope I'll never have to rely on it in production.

snek 7y ago

I really like this one: https://bartaz.github.io/ieee754-visualization/ https://bartaz.github.io/ieee754-visualization/

sj4nz 7y ago

Definitely looking forward to the Unum Type III alternative to floats. This is a nice visualization and it gives you an idea of how many NaN's waste the bits being used.

pmarreck 7y ago
yeah I was going to ask, what's the deal with all those wasted bits? I'm not a floating-point fan (I do all computations in integer wherever possible/feasible), but I've been following the universal number story since it was announced because it seems less painful/gross than IEEE floating-point on many fronts, here's a github with a good about page about it: https://github.com/stillwater-sc/universal https://github.com/stillwater-sc/universal

userbinator 7y ago

I wrote something similar ~25 years ago, when Win32 first appeared, and still use it today on the (relatively rare) occasion that I need to debug something that uses floating-point. One interesting thing you'll notice about IEEE754 is that the values sort naturally if you interpret them as sign-magnitude integers: 00000000 0 00000001 1.4e-45 (smallest representable positive value) 00000002 2.8e-45 (next-smallest representable positive value) 007fffff 1.1754942e-38 00800000 1.17549435e-38 3f000000 0.5 3f800000 1. 3fffffff 1.9999999 40000000 2. 40400000 3. 60000000 3.6893488e19 7f000000 1.7014118e38 7f7fffff 3.4028235e38 (largest finite value) 7f800000 Infinity 7f800001 first NaN 7fffffff last NaN In other words, if you count up in floats, it starts at 0 and goes up by increasingly larger amounts, then the number right "after" the "largest" is Infinity, followed by all the NaNs to the end of the range. The negative side is a "mirror image", the only difference being the highest (sign) bit is set.

filmor 7y ago
Just a little nit-pick: The NaNs always compare false, so this property only holds for values up to +inf (0x7f800000).
- deleted 7y ago
  [deleted]
vanderZwan 7y ago
> One interesting thing you'll notice about IEEE754 is that the values sort naturally if you interpret them as sign-magnitude integers With a few bit-twiddling tricks you can even make them radix-sortable, as explained by Malte Skarupke: > Michael Herf has also discovered a good way to generalize this to floating point numbers: Reinterpret cast the float to a uint32, then flip every bit if the float was negative, but flip only the sign bit if the float was positive. The same trick works for doubles and uint64s. Michael Herf explains why this works in the linked piece, but the short version of it is this: Positive floating point numbers already sort correctly if we just reinterpret cast them to a uint32. The exponent comes before the mantissa, so we would sort by the exponent first, then by the mantissa. Everything works out. Negative floating point numbers however would sort the wrong way. Flipping all the bits on them fixes that. The final remaining problem is that positive floating point numbers need to sort as bigger than negative numbers, and the easiest way to do that is to flip the sign bit since it’s the most significant bit. Up until recently in most JS engines it was faster to sort typedarrays with a custom radix sort than with the built-in sort (seems to no longer be the case, thank goodness) [0] https://probablydance.com/2016/12/27/i-wrote-a-faster-sorting-algorithm/ https://probablydance.com/2016/12/27/i-wrote-a-faster-sortin... [1] http://www.codercorner.com/RadixSortRevisited.htm http://www.codercorner.com/RadixSortRevisited.htm [2] http://stereopsis.com/radix.html http://stereopsis.com/radix.html

kstenerud 7y ago

I deep dove into ieee754 binary and decimal float (DPD and BID) while researching how to compress floating point values [1]. For 25 years it had been a magic black box that I couldn't trust for comparisons or printing, but now I get it. I get why it happened this way, what the benefits were, how to mitigate against its weirdness, and why ultimately decimal float needs to replace binary float in future. It's definitely worth learning. https://github.com/kstenerud/compact-float/blob/master/compact-float-specification.md https://github.com/kstenerud/compact-float/blob/master/compa...

constexpr 7y ago

Wow, I didn't expect a four year old project of mine to hit HN front page! If you're interested in this, you might enjoy some of the stuff we're working on at Figma. Here's a link to learn more: https://www.figma.com/careers/ https://www.figma.com/careers/

bostonvaulter2 7y ago
I use figma at work and it's really neat. Keep up the good work! Now if you could only convince the designer to give me write access ;)

vortico 7y ago

Nice work on the "mouse down to toggle" and "drag to copy value" interaction, a much underused but useful tool to set a collection of boolean states.

kecskesadam 7y ago

https://www.h-schmidt.net/FloatConverter/IEEE754.html https://www.h-schmidt.net/FloatConverter/IEEE754.html

riceslush 7y ago

I built something similar https://www.isfloat.com/ https://www.isfloat.com/ It also tells you whether the decimal value given can be exactly represented as a float. For example https://www.isfloat.com/?v=0.3 https://www.isfloat.com/?v=0.3

kazinator 7y ago

Click, hold and swipe to quickly replicate a bit value across a range of places.