3 ms·
Is your example of a float32 number correct, holding 24 ascii char representation? I had thought single-precision gonna be 7 digits and the exponent, sign and e
by banku_brougham 2y ago
Is your example of a float32 number correct, holding 24 ascii char representation? I had thought single-precision gonna be 7 digits and the exponent, sign and exp sign. Something like 7+2+1+1 or 10 char ascii representation? Rather than the 24 you mentioned?
- minimaxir 2y agoIt depends on the default print format. The example string I mentioned is pulled from what np.savetxt() does (fmt='%.18e') and there isn't any precision loss in that number. But I admit I'm not a sprintf() guru. In practice numbers with that much precision is overkill and verbose so tools don't print float32s to that level of precision.
- PaulHoule 2y agoOne of the things I remember from my PhD work is that you can do a stupendous number of FLOPs on floating point numbers in the time it takes to serialize/deserialize them to ASCII.