4 ms·

> For very small numbers (less than about FLT_EPSILON * 0.25) adding 0.5 gives 0.5 exactly, and this then rounds to zero. Since about 40% of the positive floati

by zaroth 6y ago

> For very small numbers (less than about FLT_EPSILON * 0.25) adding 0.5 gives 0.5 exactly, and this then rounds to zero. Since about 40% of the positive floating-point numbers are smaller than FLT_EPSILON * 0.25 this results in a lot of errors – over 850 million of them! I thought Float Epsilon was the smallest possible number which could be represented by a float? This statement doesn’t seem to make sense...

Sebb767 6y ago
I thought that too, but it's actually the difference between 1.0f and the next larger number; so it only represents the smallest difference for the non-dotzero range. See here: http://www.cplusplus.com/reference/cfloat/ http://www.cplusplus.com/reference/cfloat/
MatmaRex 6y ago
There are different "epsilon" constants in different languages that mean different things. In C and C++, FLT_EPSILON is "Difference between 1 and the least value greater than 1 that is representable." (http://www.cplusplus.com/reference/cfloat/ http://www.cplusplus.com/reference/cfloat/). You might be thinking of the C# constant.
neeeeees 6y ago
FLT_ESPILON is the bound on rounding error for certain operations. Plenty of numbers smaller than it can be represented in most floating point implementations
jcranmer 6y ago
When I took numerical analysis, the concept was described as "machine epsilon", which is the smallest number that can be added to 1.0 that is not 1.0. The utility of this metric is that it tells you the best possible relative error, as you can never guarantee a result will be more correct to the true result (in the domain of mathematical real numbers as opposed to machine floating-point numbers) than half of machine epsilon. This concept is often extended into discussions of "units in the last place" (ULPs), which is the magnitude of the value of last bit of the mantissa. So ulp(1.0) = ulp(1.5) = machine epsilon, ulp(2.0) = ulp(3.9) = 2 * machine epsilon, etc. A good floating-point library will document its accuracy in terms of ulps, so that we might say that the maximum error of the exp function is 1 ULP and that of pow might be 6 ULPs.
brucedawson 6y ago
FLT_EPSILON is the difference between 1.0f and the next float. With that we can work out the math: - If you add 1.0f to FLT_EPSILON then you get an exact result - If you add 1.0f to FLT_EPSILON * 0.5 then you get a number that cannot be represented and it will either round to 1.0 or 1.0+FLT_EPSILON - Similarly, if you add 0.5f to FLT_EPSILON * 0.25 then you get a number that cannot be represented and it will either round to 0.5 or to 0.5 + FLT_EPSILON * 0.5 With the default rounding mode that last calculation rounds to 0.5. There are a _lot_ of floats below FLT_EPSILON.

by zaroth 6y ago

Sebb767 6y ago

I thought that too, but it's actually the difference between 1.0f and the next larger number; so it only represents the smallest difference for the non-dotzero range. See here: http://www.cplusplus.com/reference/cfloat/ http://www.cplusplus.com/reference/cfloat/

MatmaRex 6y ago

There are different "epsilon" constants in different languages that mean different things. In C and C++, FLT_EPSILON is "Difference between 1 and the least value greater than 1 that is representable." (http://www.cplusplus.com/reference/cfloat/ http://www.cplusplus.com/reference/cfloat/). You might be thinking of the C# constant.

neeeeees 6y ago

FLT_ESPILON is the bound on rounding error for certain operations. Plenty of numbers smaller than it can be represented in most floating point implementations

jcranmer 6y ago

When I took numerical analysis, the concept was described as "machine epsilon", which is the smallest number that can be added to 1.0 that is not 1.0. The utility of this metric is that it tells you the best possible relative error, as you can never guarantee a result will be more correct to the true result (in the domain of mathematical real numbers as opposed to machine floating-point numbers) than half of machine epsilon. This concept is often extended into discussions of "units in the last place" (ULPs), which is the magnitude of the value of last bit of the mantissa. So ulp(1.0) = ulp(1.5) = machine epsilon, ulp(2.0) = ulp(3.9) = 2 * machine epsilon, etc. A good floating-point library will document its accuracy in terms of ulps, so that we might say that the maximum error of the exp function is 1 ULP and that of pow might be 6 ULPs.

brucedawson 6y ago

FLT_EPSILON is the difference between 1.0f and the next float. With that we can work out the math: - If you add 1.0f to FLT_EPSILON then you get an exact result - If you add 1.0f to FLT_EPSILON * 0.5 then you get a number that cannot be represented and it will either round to 1.0 or 1.0+FLT_EPSILON - Similarly, if you add 0.5f to FLT_EPSILON * 0.25 then you get a number that cannot be represented and it will either round to 0.5 or to 0.5 + FLT_EPSILON * 0.5 With the default rounding mode that last calculation rounds to 0.5. There are a _lot_ of floats below FLT_EPSILON.