6 ms·
It's absolutely not. Everything I've seen has been 64-bit double. If you need to, you round either intermediate or final calcs to a set precision. I once worke
by timrobinson 11y ago
It's absolutely not. Everything I've seen has been 64-bit double. If you need to, you round either intermediate or final calcs to a set precision.
I once worked on replicating a particular index, and the engine used fixed point internally. In order to get the values to match with the ones published officially, we had to emulate Excel floating point: our fixed point values wouldn't line up with theirs.