3 ms·
No, it's 64 decimal digits ~ 213 bits of precision! Octuple precision should have 8 * 32 = 256 bit word size. IEEE 754 single, double, and quadruple precision F
by throwaway_yy2Di 12y ago
No, it's 64 decimal digits ~ 213 bits of precision! Octuple precision should have 8 * 32 = 256 bit word size. IEEE 754 single, double, and quadruple precision FP have 32, 64, and 128-bit word sizes, with significand precisions of 24, 53, and 113 bits.
https://en.wikipedia.org/wiki/Floating_point#Internal_representation https://en.wikipedia.org/wiki/Floating_point#Internal_repres...
- HCIdivision17 12y agoEven more astounding. In retrospect, the word 'digits' is pretty unambiguous, and I shouldn't have gotten confused. But man, that is some hard-core numerical work. I'll need to bring the paper to the top of my dinner reading stack. The project just seems fascinating.