4 ms·
The problem with float32: you only get 16M values
- al2o3cr 4y agoThe author uses a definition of "precision" that doesn't make a ton of sense for most applications; the differences between 0.5 and 1.0 and 16000001 and 16000001.5 are "the same" numerically, but most applications are more concerned with _relative_ error which is 16 million times smaller for the second pair. Ultimately it's a data-dependent situation: VERY BAD if you're tracking money with floats (srsly, don't do that ffs) but harmless if you're tracking something like "detected particles" which already has an O(sqrt(N)) error bound.