3 ms·
So the complaint is that this loses half a bit of precision on average? Is that a lot?
by upofadown 6y ago
So the complaint is that this loses half a bit of precision on average? Is that a lot?
- kevingadd 6y agoIf you lose half a bit on every operation and do multiple operations, that could be pretty bad. If it's half a bit at the end of a long chain of operations, that's tolerable and not worse than the precision specified by the docs for some execution environments (GPU shaders, for example) It also matters a lot whether the error is predictable and possible to compensate for.