4 ms·
I'm far from a floating point expert, but the fact that the article does everything in terms of epsilons, without mentioning ULPs a single time confuses me. In
by tux3 6y ago
I'm far from a floating point expert, but the fact that the article does everything in terms of epsilons, without mentioning ULPs a single time confuses me.
In fact, that fundamental axiom of FP arithmetic is NOT in terms of some absolute value epsilonMachine. As I understand it, it says that there exists a relative epsilon value (which depends on your input numbers), and that the error of any FP operation on those numbers will be bounded by that epsilon.
What the article doesn't explain, is that floating point numbers and operations are guaranteed to be accurate to the nearest half-ULP, not to the nearest absolute epsilon. And what exactly is the absolute value of an ULP? It depends!
The difference is so fundamental, I'm not sure you can have a useful mental model of floats if you don't start by addressing this.
- kd5bjo 6y agoWould you mind expanding the `ULP` acronym you’re using: if it’s so fundamental to understanding this, I’d like to know what it is.
- tux3 6y agoMy apology, ULP stands for "Unit in the Last Place". The simple explanation is that an ULP is defined as the difference between two consecutive floating point numbers, which means it isn't a fixed value. Another "intuitive" explanation, is that if you increment the binary representation of a float, the difference between the two numbers is 1 ULP. (I'd explain more, but I have to run to a meeting! More info on Wikipedia: https://en.wikipedia.org/wiki/Unit_in_the_last_place https://en.wikipedia.org/wiki/Unit_in_the_last_place)
- deleted 6y ago[deleted]
- carlmr 6y agoBut multiplying by (1+e) is relative. I guess it's much more convenient mathematically than dealing with the absolute error of the calculation.
- akdor1154 6y agoThe epsilon model is perfectly adequate, and does not assume epsilon is constant on a given machine (the article glosses over this in the first paragraph where epsilon is introduced, but it is spelled out later). In fact the epsilon model is key to being able to do a mathematically formulated error analysis at all.
- DougBTX 6y agoInstead of calling it "ULPs" they refer to a "next floating point number" which I'm pretty sure is the same idea: > Unlike the number line, for every representable floating point number x, there is a _next_ floating point number, and it is approximately x + ϵmachine x. So for an arbitrary real number x0, it falls between two floating point values x and x+ϵx (leaving off the subscript for conciseness). When we represent x0 in floating point, we will get one of these two values.
- fractionalhare 6y agoMachine epsilon is fine for doing error analysis. For example, I used machine epsilon instead of ULP for an article I recently wrote which walks through implementation and (numerical) analysis of various algorithms for e^x: https://www.pseudorandom.com/implementing-exp https://www.pseudorandom.com/implementing-exp So you don't need ULP (and if you did, you could convert to ULP from machine epsilon). But you are right that machine epsilon varies according to where you are in the interval of floating point numbers. Some authors don't mention that, but yes, since floating point values are not evenly distributed the machine epsilon for any particular x will vary. This is good to know, because even if error analysis demonstrates that your approximation function is bounded by machine epsilon, it's still going to exhibit less accuracy as you move away from the origin.