3 ms·
I got that but I am curious how to derive that number. Is it representable as a non-trivial ratio of integers?
by jdthedisciple 2y ago
I got that but I am curious how to derive that number.
Is it representable as a non-trivial ratio of integers?
- hmry 2y agoGood question! It's 1/(2**23), because 32 bit floats have 23 bits after the decimal point