4 ms·

I thought this was about different numbers like rational and complex, but it is about number representation (of floats), see also https://en.wikipedia.org/wiki/

I thought this was about different numbers like rational and complex, but it is about number representation (of floats), see also https://en.wikipedia.org/wiki/Unum_(number_format)#Unum_III https://en.wikipedia.org/wiki/Unum_(number_format)#Unum_III

parasti 4y ago
Unfortunately the Wikipedia article is extremely light on details of the actual format.
- zeotroph 4y ago
  The(?) posit website has a pdf[1] which goes into more details under "3.3 Posit format encoding". It also mentions a "quire" (and the footnote for the quire value is self referential?) , can anyone explain that concept? 1: https://posithub.org/docs/posit_standard-2.pdf https://posithub.org/docs/posit_standard-2.pdf
  - SanjayMehta 4y ago
    As I understood it, it's a large internal register to prevent fractional errors from accumulating.
    - josefx 4y ago
      Wasn't that a gigantic issue with x87 floating point implementation? It would internally use 80 bit registers and the result of any computation was completely at the mercy of how the compiler used those 80 bit registers as every move to system memory would drop precision back to the 32 bit or 64 bit specified by the program.
      - SanjayMehta 4y ago
        I don't really know anything about that, but it appears here that all this is done in hardware without any compilers involved.
        josefx 4y ago
        The footnote for point 4.3 in the linked document above explicitly calls out optimization modes and possible non compliance. Yeah, its the x87 80 bit registers all over again.
        gliptic 4y ago
        Except the compilers will not be dumb enough to use it implicitly when you don't ask for it, just like they don't automatically use FMA now (except maybe with -ffast-math).
        adgjlsfhk1 4y ago
        Nope. C/C++ think it's OK to use fma when you didn't ask for it with default compiler settings. Also, with default settings, GCC is willing to replace single precision math with double precision math if it feels like it.
        jcranmer 4y ago
        > Nope. C/C++ think it's OK to use fma when you didn't ask for it with default compiler settings. No, it doesn't. You have to request #pragma STDC FP_CONTRACT ON explicitly, or use -ffp-contract flag-equivalent (implied by -ffast-math or -Ofast) on most compilers. icc is the only major compiler that actually defaults to fast-math flags. > Also, with default settings, GCC is willing to replace single precision math with double precision math if it feels like it. I'm less familiar with gcc than I am with LLVM, but I strongly doubt that this is the case. There is a provision in C/C++ for FLT_EVAL_METHOD, which indicates what the internal precision of arithmetic expressions (which excludes assignments and casts) is, and this is set to 2 on 32-bit x86, because x87 internally operates on all numbers as long double precision, only explicitly rounding to float/double when you tell it to in an extension. But on 64-bit x86, FLT_EVAL_METHOD is 0 (everybody executes according to their own type), because SSE can operate on single- or double-precision numbers directly.
        gliptic 4y ago
- btdmaster 4y ago
  I have found [1] and [2] but it is not clear how that helps with small numbers. [1] http://raden.fke.utm.my/blog/positnumbersystem http://raden.fke.utm.my/blog/positnumbersystem [2] https://www.johndcook.com/blog/2018/04/11/anatomy-of-a-posit-number/ https://www.johndcook.com/blog/2018/04/11/anatomy-of-a-posit...
FabHK 4y ago
Interesting. So, Posits are basically Unums, v3. I’d love to know what Kahan (the father of IEEE 754 floats) has to say about them. He thought that the „merits of schemes proposed in“ earlier versions were „greatly exaggerate[d]”. http://people.eecs.berkeley.edu/~wkahan/UnumSORN.pdf http://people.eecs.berkeley.edu/~wkahan/UnumSORN.pdf
- zasdffaa 4y ago
  I did some reading on unum a while back, inc. Kahan's criticisms. There's another side. There's a vid on youtube with Kahan and Gustafson talking over it. Kahan goes in like a wolf onto a lamb, much to gustafson's shock and hurt. Gustafson points out some of Kahan's criticisms are plain wrong, even citing the page in the book, then says something like "we should be working together on this, why aren't we working together?" Kahan doesn't respond. Kahan may be right or wrong but his attitude is weirdly hostile, and Gustafson is no n00b about floats. Edit: I think this is it (but it's been a while) https://www.youtube.com/watch?v=LZAeZBVAzVw https://www.youtube.com/watch?v=LZAeZBVAzVw
  - NohatCoder 4y ago
    Gustafson is a noob. Both his Unum I and II proposals are completely impractical, making absolutely no consideration for how hardware would implement his ideas. But by writing only about the imagined merits he managed to convince a lot of people that his ideas were great. Unum III/Posits can work, but mainly it is just a number compression format. Working with say 64 bit Posits pretty much requires implementing arithmetic equivalent to that of 80 bit IEEE floats, and then throwing away a larger or smaller portion of the mantissa depending on the exponent. And the extra accuracy around the sweet spot of 1 still comes at the cost of lost accuracy for large and small numbers, so some workloads will suffer.
  - jcranmer 4y ago
    Having watched that video, Gustafson seems to me much more vengeful and hostile than Kahan. There was one point--where an audience member was asking after the impact of variably-sized arithmetic types on implementing algorithms--where Gustafson basically says "you guys don't know how to code for modern computers anymore", which is when the moderator has to step in to keep things from escalating. The answer to your question--why isn't Kahan working with Gustafson--is that, in Kahan's view, interval arithmetic (this is, AIUI, the main thrust of unums) isn't actually an effective solution to the "problem" of needing numerical analysis. While Kahan isn't the best at explaining this in detail, he does point out two valid issues: interval arithmetic can give excessively pessimistic ranges (because it doesn't account for correlated error), and it can give just plain incorrect answers when you have singularities in ranges. I would note that, as Gustafson is no longer (as far as I know) pushing for the interval arithmetic approach, this is basically a concession that Kahan was right and Gustafson was wrong.

parasti 4y ago

Unfortunately the Wikipedia article is extremely light on details of the actual format.

zeotroph 4y ago
The(?) posit website has a pdf[1] which goes into more details under "3.3 Posit format encoding". It also mentions a "quire" (and the footnote for the quire value is self referential?) , can anyone explain that concept? 1: https://posithub.org/docs/posit_standard-2.pdf https://posithub.org/docs/posit_standard-2.pdf
- SanjayMehta 4y ago
  As I understood it, it's a large internal register to prevent fractional errors from accumulating.
  - josefx 4y ago
    Wasn't that a gigantic issue with x87 floating point implementation? It would internally use 80 bit registers and the result of any computation was completely at the mercy of how the compiler used those 80 bit registers as every move to system memory would drop precision back to the 32 bit or 64 bit specified by the program.
    - SanjayMehta 4y ago
      I don't really know anything about that, but it appears here that all this is done in hardware without any compilers involved.
      - josefx 4y ago
        The footnote for point 4.3 in the linked document above explicitly calls out optimization modes and possible non compliance. Yeah, its the x87 80 bit registers all over again.
        gliptic 4y ago
        Except the compilers will not be dumb enough to use it implicitly when you don't ask for it, just like they don't automatically use FMA now (except maybe with -ffast-math).
        adgjlsfhk1 4y ago
        Nope. C/C++ think it's OK to use fma when you didn't ask for it with default compiler settings. Also, with default settings, GCC is willing to replace single precision math with double precision math if it feels like it.
        jcranmer 4y ago
        > Nope. C/C++ think it's OK to use fma when you didn't ask for it with default compiler settings. No, it doesn't. You have to request #pragma STDC FP_CONTRACT ON explicitly, or use -ffp-contract flag-equivalent (implied by -ffast-math or -Ofast) on most compilers. icc is the only major compiler that actually defaults to fast-math flags. > Also, with default settings, GCC is willing to replace single precision math with double precision math if it feels like it. I'm less familiar with gcc than I am with LLVM, but I strongly doubt that this is the case. There is a provision in C/C++ for FLT_EVAL_METHOD, which indicates what the internal precision of arithmetic expressions (which excludes assignments and casts) is, and this is set to 2 on 32-bit x86, because x87 internally operates on all numbers as long double precision, only explicitly rounding to float/double when you tell it to in an extension. But on 64-bit x86, FLT_EVAL_METHOD is 0 (everybody executes according to their own type), because SSE can operate on single- or double-precision numbers directly.
        gliptic 4y ago
btdmaster 4y ago
I have found [1] and [2] but it is not clear how that helps with small numbers. [1] http://raden.fke.utm.my/blog/positnumbersystem http://raden.fke.utm.my/blog/positnumbersystem [2] https://www.johndcook.com/blog/2018/04/11/anatomy-of-a-posit-number/ https://www.johndcook.com/blog/2018/04/11/anatomy-of-a-posit...

FabHK 4y ago

Interesting. So, Posits are basically Unums, v3. I’d love to know what Kahan (the father of IEEE 754 floats) has to say about them. He thought that the „merits of schemes proposed in“ earlier versions were „greatly exaggerate[d]”. http://people.eecs.berkeley.edu/~wkahan/UnumSORN.pdf http://people.eecs.berkeley.edu/~wkahan/UnumSORN.pdf

zasdffaa 4y ago
I did some reading on unum a while back, inc. Kahan's criticisms. There's another side. There's a vid on youtube with Kahan and Gustafson talking over it. Kahan goes in like a wolf onto a lamb, much to gustafson's shock and hurt. Gustafson points out some of Kahan's criticisms are plain wrong, even citing the page in the book, then says something like "we should be working together on this, why aren't we working together?" Kahan doesn't respond. Kahan may be right or wrong but his attitude is weirdly hostile, and Gustafson is no n00b about floats. Edit: I think this is it (but it's been a while) https://www.youtube.com/watch?v=LZAeZBVAzVw https://www.youtube.com/watch?v=LZAeZBVAzVw
- NohatCoder 4y ago
  Gustafson is a noob. Both his Unum I and II proposals are completely impractical, making absolutely no consideration for how hardware would implement his ideas. But by writing only about the imagined merits he managed to convince a lot of people that his ideas were great. Unum III/Posits can work, but mainly it is just a number compression format. Working with say 64 bit Posits pretty much requires implementing arithmetic equivalent to that of 80 bit IEEE floats, and then throwing away a larger or smaller portion of the mantissa depending on the exponent. And the extra accuracy around the sweet spot of 1 still comes at the cost of lost accuracy for large and small numbers, so some workloads will suffer.
- jcranmer 4y ago
  Having watched that video, Gustafson seems to me much more vengeful and hostile than Kahan. There was one point--where an audience member was asking after the impact of variably-sized arithmetic types on implementing algorithms--where Gustafson basically says "you guys don't know how to code for modern computers anymore", which is when the moderator has to step in to keep things from escalating. The answer to your question--why isn't Kahan working with Gustafson--is that, in Kahan's view, interval arithmetic (this is, AIUI, the main thrust of unums) isn't actually an effective solution to the "problem" of needing numerical analysis. While Kahan isn't the best at explaining this in detail, he does point out two valid issues: interval arithmetic can give excessively pessimistic ranges (because it doesn't account for correlated error), and it can give just plain incorrect answers when you have singularities in ranges. I would note that, as Gustafson is no longer (as far as I know) pushing for the interval arithmetic approach, this is basically a concession that Kahan was right and Gustafson was wrong.