5 ms·

Previously, in JF's "Can we acknowledge that every real computer works this way?" series: "Signed Integers are Two’s Complement" <https://www.open-std.org/jtc1/

Previously, in JF's "Can we acknowledge that every real computer works this way?" series: "Signed Integers are Two’s Complement" <https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0907r0.html https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p09...>

jsheard 2y ago
Maybe specifying that floats are always IEEE floats should be next? Though that would obsolete this Linux kernel classic so maybe not. https://github.com/torvalds/linux/blob/master/include/math-emu/double.h#L29 https://github.com/torvalds/linux/blob/master/include/math-e...
- NL807 2y ago
  Love it
- AnimalMuppet 2y ago
  That line is actually from a famous Dilbert cartoon. I found this snapshot of it, though it's not on the real Dilbert site: https://www.reddit.com/r/linux/comments/73in9/computer_holy_wars_png/ https://www.reddit.com/r/linux/comments/73in9/computer_holy_...
  - Jerrrrrrry 2y ago
    [flagged]
- FooBarBizBazz 2y ago
  Whether double floats can silently have 80 bit accumulators is a controversial thing. Numerical analysis people like it. Computer science types seem not to because it's unpredictable. I lean towards, "we should have it, but it should be explicit", but this is not the most considered opinion. I think there's a legitimate reason why Intel included it in x87, and why DSPs include it.
  - deleted 2y ago
    [deleted]
  - stephencanon 2y ago
    Numerical analysis people do not like it. Having _explicitly controlled_ wider accumulation available is great. Having compilers deciding to do it for you or not in unpredictable ways is anathema.
    - bee_rider 2y ago
      It isn’t harmful, right? Just like getting a little accuracy from a fused multiply add. It just isn’t useful if you can’t depend on it.
      - eternityforest 2y ago
        I suppose it could be harmful if you write code that depends on it without realizing it, and then something changes so it stops doing that.
      - Negitivefrags 2y ago
        It can be harmful. In GCC while compiling a 32 bit executable, making an std::map< float, T > can cause infinite loops or crashes in your program. This is because when you insert a value into the map, it has 80 bit precision, and that number of bits is used when comparing the value you are inserting during the traversal of the tree. After the float is stored in the tree, it's clamped to 32 bits. This can cause the element to be inserted into into the wrong order in the tree, and this breaks the assumptions of the algorithm leaidng to the crash or infinite loop. Compiling for 64 bits or explicitly disabling x87 float math makes this problem go away. I have actually had this bug in production and it was very hard to track down.
        blt 2y ago
        dang that's a good war story.
        2y ago
- jfbastien 2y ago
  Hi! I'm JF. I half-jokingly threatened to do IEEE float in 2018 https://youtu.be/JhUxIVf1qok?si=QxZN_fIU2Th8vhxv&t=3250 https://youtu.be/JhUxIVf1qok?si=QxZN_fIU2Th8vhxv&t=3250 I wouldn't want to lose the Linux humor tho!
- conradev 2y ago
  I was curious about float16, and TIL that the 2008 revision of the standard includes it as an interchange format: https://en.wikipedia.org/wiki/IEEE_754-2008_revision https://en.wikipedia.org/wiki/IEEE_754-2008_revision
  - tialaramex 2y ago
    Note that this type (which Rust will/ does in nightly call "f16" and a C-like language would probably name "half") is not the only popular 16-bit floating point type, as some people want to have https://en.wikipedia.org/wiki/Bfloat16_floating-point_format https://en.wikipedia.org/wiki/Bfloat16_floating-point_format
    - adrian_b 2y ago
      The IEEE FP16 format is what is useful in graphics applications, e.g. for storing color values. The Google BF16 format is useful strictly only for machine learning/AI applications, because its low precision is insufficient for anything else. BF16 has very low precision, but an exponent range equal to FP32, which makes overflows and underflows less likely.
- jcranmer 2y ago
  I'm literally giving a talk next week who's first slide is essentially "Why IEEE 754 is not a sufficient description of floating-point semantics" and I'm sitting here trying to figure out what needs to be thrown out of the talk to make it fit the time slot. One of the most surprising things about floating-point is that very little is actually IEEE 754; most things are merely IEEE 754-ish, and there's a long tail of fiddly things that are different that make it only -ish.
  - Terr_ 2y ago
    > there's a long tail of fiddly things that are different that make it only -ish. Perhaps a way to fill some time would be gradually revealing parts of a convoluted Venn diagram or mind-map of the fiddling things. (That is, assuming there's any sane categorization.)
  - speedgoose 2y ago
    I'm interested by your future talk, do you plan to publish a video or a transcript?
    - selimthegrim 2y ago
      I too would like to see it.
  - chungy 2y ago
    The IEEE 754 standard has been updated several times, often by relaxing previous mandates in order to make various hardware implementations become compliant retroactively (eg, adding Intel's 80-bit floats as a standard floating point size). It'll be interesting if the "-ish" bits are still "-ish" with the current standard.
    - stephencanon 2y ago
      The first 754 standard (1985) was essentially formalization of the x87 arithmetic; it defines a "double extended" format. It is not mandatory: > Implementations should support the extended format corresponding to the widest basic format supported. _if_ it exists, it is required to have at least as many bits as the x87 long double type.¹ The language around extended formats changed in the 2008 standard, but the meaning didn't: > Language standards or implementations should support an extended precision format that extends the widest basic format that is supported in that radix. That language is still present in the 2019 standard. So nothing has ever really changed here. Double-extended is recommended, but not required. If it exists, the significand and exponent must be at least as large as those of the Intel 80-bit format, but they may also be larger. --- ¹ At the beginning of the standardization process, Kahan and Intel engineers still hoped that the x87 format would gradually expand in subsequent CPU generations until it became what is now the standard 128b quad format; they didn't understand the inertia of binary compatibility yet. So the text only set out minimum precision and exponent range. By the time the standard was published in 1985, it was understood internally that they would never change the type, but by then other companies had introduced different extended-precision types (e.g. the 96-bit type in Apple's SANE), so it was never pinned down.
- Silphendio 2y ago
  At the very least, division by zero should not be undefined for floats.
- seoulbigchris 2y ago
  Which one? Remember the decimal IEEE 754 floating point formats exist too. Do folks in banking use IEEE decimal formats? I remember we used to have different math libs to link against depending, but this was like 40 years ago.
  - quietbritishjim 2y ago
    Nothing prevents banks (or anyone else) from using a compiler where "float" means binary floating point while some other native or user-defined type supports decimal floating point. In fact, that's probably for the best, since they'll probably have exacting requirements for that type so it makes sense for the application developer to write that type themselves.
    - seoulbigchris 2y ago
      I was referring to banks using decimal libraries because they work in base 10 numbers, and I recall a big announcement many years ago when the stock market officially switched from fractional stock pricing to cents "for the benefit of computers and rounding", or some such excuse. It always struck me as strange, since binary fixed and floating point represent those particular quantities exactly, without rounding error. Now with normal dollars and cents calculations, I can see why a decimal library might be preferred.
  - mu53 2y ago
    Java is big for banks, and `BigInteger` is common for monetary types.
  - stephencanon 2y ago
    Binding float to the IEEE 754 binary32 format would not preclude use of decimal formats; they have their own bindings (e.g. _Decimal64 in C23). (I think they're still a TR for C++, but I haven't been keeping track).
- heinrich5991 2y ago
  Permalink (press 'y' anywhere on GitHub): https://github.com/torvalds/linux/blob/4d939780b70592e0f4bc6c397e52e518f8fb7916/include/math-emu/double.h#L29 https://github.com/torvalds/linux/blob/4d939780b70592e0f4bc6....
  - crote 2y ago
    That file hasn't been touched in over 19 years. I don't think we have to worry about the non-permalink url breaking any time soon.

jsheard 2y ago

Maybe specifying that floats are always IEEE floats should be next? Though that would obsolete this Linux kernel classic so maybe not. https://github.com/torvalds/linux/blob/master/include/math-emu/double.h#L29 https://github.com/torvalds/linux/blob/master/include/math-e...

NL807 2y ago
Love it
AnimalMuppet 2y ago
That line is actually from a famous Dilbert cartoon. I found this snapshot of it, though it's not on the real Dilbert site: https://www.reddit.com/r/linux/comments/73in9/computer_holy_wars_png/ https://www.reddit.com/r/linux/comments/73in9/computer_holy_...
- Jerrrrrrry 2y ago
  [flagged]
FooBarBizBazz 2y ago
Whether double floats can silently have 80 bit accumulators is a controversial thing. Numerical analysis people like it. Computer science types seem not to because it's unpredictable. I lean towards, "we should have it, but it should be explicit", but this is not the most considered opinion. I think there's a legitimate reason why Intel included it in x87, and why DSPs include it.
- deleted 2y ago
  [deleted]
- stephencanon 2y ago
  Numerical analysis people do not like it. Having _explicitly controlled_ wider accumulation available is great. Having compilers deciding to do it for you or not in unpredictable ways is anathema.
  - bee_rider 2y ago
    It isn’t harmful, right? Just like getting a little accuracy from a fused multiply add. It just isn’t useful if you can’t depend on it.
    - eternityforest 2y ago
      I suppose it could be harmful if you write code that depends on it without realizing it, and then something changes so it stops doing that.
    - Negitivefrags 2y ago
      It can be harmful. In GCC while compiling a 32 bit executable, making an std::map< float, T > can cause infinite loops or crashes in your program. This is because when you insert a value into the map, it has 80 bit precision, and that number of bits is used when comparing the value you are inserting during the traversal of the tree. After the float is stored in the tree, it's clamped to 32 bits. This can cause the element to be inserted into into the wrong order in the tree, and this breaks the assumptions of the algorithm leaidng to the crash or infinite loop. Compiling for 64 bits or explicitly disabling x87 float math makes this problem go away. I have actually had this bug in production and it was very hard to track down.
      - blt 2y ago
        dang that's a good war story.
      - 2y ago
jfbastien 2y ago
Hi! I'm JF. I half-jokingly threatened to do IEEE float in 2018 https://youtu.be/JhUxIVf1qok?si=QxZN_fIU2Th8vhxv&t=3250 https://youtu.be/JhUxIVf1qok?si=QxZN_fIU2Th8vhxv&t=3250 I wouldn't want to lose the Linux humor tho!
conradev 2y ago
I was curious about float16, and TIL that the 2008 revision of the standard includes it as an interchange format: https://en.wikipedia.org/wiki/IEEE_754-2008_revision https://en.wikipedia.org/wiki/IEEE_754-2008_revision
- tialaramex 2y ago
  Note that this type (which Rust will/ does in nightly call "f16" and a C-like language would probably name "half") is not the only popular 16-bit floating point type, as some people want to have https://en.wikipedia.org/wiki/Bfloat16_floating-point_format https://en.wikipedia.org/wiki/Bfloat16_floating-point_format
  - adrian_b 2y ago
    The IEEE FP16 format is what is useful in graphics applications, e.g. for storing color values. The Google BF16 format is useful strictly only for machine learning/AI applications, because its low precision is insufficient for anything else. BF16 has very low precision, but an exponent range equal to FP32, which makes overflows and underflows less likely.
jcranmer 2y ago
I'm literally giving a talk next week who's first slide is essentially "Why IEEE 754 is not a sufficient description of floating-point semantics" and I'm sitting here trying to figure out what needs to be thrown out of the talk to make it fit the time slot. One of the most surprising things about floating-point is that very little is actually IEEE 754; most things are merely IEEE 754-ish, and there's a long tail of fiddly things that are different that make it only -ish.
- Terr_ 2y ago
  > there's a long tail of fiddly things that are different that make it only -ish. Perhaps a way to fill some time would be gradually revealing parts of a convoluted Venn diagram or mind-map of the fiddling things. (That is, assuming there's any sane categorization.)
- speedgoose 2y ago
  I'm interested by your future talk, do you plan to publish a video or a transcript?
  - selimthegrim 2y ago
    I too would like to see it.
- chungy 2y ago
  The IEEE 754 standard has been updated several times, often by relaxing previous mandates in order to make various hardware implementations become compliant retroactively (eg, adding Intel's 80-bit floats as a standard floating point size). It'll be interesting if the "-ish" bits are still "-ish" with the current standard.
  - stephencanon 2y ago
    The first 754 standard (1985) was essentially formalization of the x87 arithmetic; it defines a "double extended" format. It is not mandatory: > Implementations should support the extended format corresponding to the widest basic format supported. _if_ it exists, it is required to have at least as many bits as the x87 long double type.¹ The language around extended formats changed in the 2008 standard, but the meaning didn't: > Language standards or implementations should support an extended precision format that extends the widest basic format that is supported in that radix. That language is still present in the 2019 standard. So nothing has ever really changed here. Double-extended is recommended, but not required. If it exists, the significand and exponent must be at least as large as those of the Intel 80-bit format, but they may also be larger. --- ¹ At the beginning of the standardization process, Kahan and Intel engineers still hoped that the x87 format would gradually expand in subsequent CPU generations until it became what is now the standard 128b quad format; they didn't understand the inertia of binary compatibility yet. So the text only set out minimum precision and exponent range. By the time the standard was published in 1985, it was understood internally that they would never change the type, but by then other companies had introduced different extended-precision types (e.g. the 96-bit type in Apple's SANE), so it was never pinned down.
Silphendio 2y ago
At the very least, division by zero should not be undefined for floats.
seoulbigchris 2y ago
Which one? Remember the decimal IEEE 754 floating point formats exist too. Do folks in banking use IEEE decimal formats? I remember we used to have different math libs to link against depending, but this was like 40 years ago.
- quietbritishjim 2y ago
  Nothing prevents banks (or anyone else) from using a compiler where "float" means binary floating point while some other native or user-defined type supports decimal floating point. In fact, that's probably for the best, since they'll probably have exacting requirements for that type so it makes sense for the application developer to write that type themselves.
  - seoulbigchris 2y ago
    I was referring to banks using decimal libraries because they work in base 10 numbers, and I recall a big announcement many years ago when the stock market officially switched from fractional stock pricing to cents "for the benefit of computers and rounding", or some such excuse. It always struck me as strange, since binary fixed and floating point represent those particular quantities exactly, without rounding error. Now with normal dollars and cents calculations, I can see why a decimal library might be preferred.
- mu53 2y ago
  Java is big for banks, and `BigInteger` is common for monetary types.
- stephencanon 2y ago
  Binding float to the IEEE 754 binary32 format would not preclude use of decimal formats; they have their own bindings (e.g. _Decimal64 in C23). (I think they're still a TR for C++, but I haven't been keeping track).
heinrich5991 2y ago
Permalink (press 'y' anywhere on GitHub): https://github.com/torvalds/linux/blob/4d939780b70592e0f4bc6c397e52e518f8fb7916/include/math-emu/double.h#L29 https://github.com/torvalds/linux/blob/4d939780b70592e0f4bc6....
- crote 2y ago
  That file hasn't been touched in over 19 years. I don't think we have to worry about the non-permalink url breaking any time soon.