4 ms·
Or you can think of it as precision in the number of significant digits for a number. Each floating point number can only store a certain amount of significant
by semaphoreP 10y ago
Or you can think of it as precision in the number of significant digits for a number. Each floating point number can only store a certain amount of significant digits. The fractional precision on each float is the same, but the absolute magnitude of the error in the float value increases as the value increases.
- mirimir 10y agoThat's how I learned it.