3 ms·

It doesn't seem to be doing too well on log(1+x). It does correctly identify that the Taylor expansion is more accurate for small x (although in my opinion it s

It doesn't seem to be doing too well on log(1+x). It does correctly identify that the Taylor expansion is more accurate for small x (although in my opinion it switches back to log(1+x) a bit too soon for positive x), however it doesn't seem to take into account the essential singularity at x=-1 and keeps using the Taylor series for negative values all the way up to x=-1 leading to horrendous accuracy.

pavpanchekha 6y ago
That's right. Herbie's error measure is average error over the whole range of inputs; one way to think about that is that it's really measuring the area of inputs that have reasonable error. There aren't many floating-point values near -1, so Herbie doesn't consider the error there that important. To be precise, Herbie tries to avoid overfitting by "charging" itself 1 bit of accuracy for every branch; adding a branch for, say, -1 <= x <= -0.8 isn't worth it. If you set a precondition, say -1 <= x <= -0.8, Herbie will instead focus on that range. Also note that the Herbie web demo has some options set that make it fast (to handle load) at the cost of lower accuracy. For example, if you download and install it yourself, you can turn on support for the special numeric functions (like log1p) or increase the number of search iterations done.
- contravariant 6y ago
  I see. I guess it ends up doing something a bit unexpected because the distribution it's sampling from is quite different from the numbers people often deal with (also unlike the example on the main page it can apparently fail to find a function that is more accurate for all inputs). Also this is a bit of a special case because it's easy to show that x - x^2 / 2 is within 10^-15 of the true value provided abs(x) is within 10^-6 or so, so it's easy to figure out how good Herbie is doing.
  - pavpanchekha 6y ago
    That's right, the implicit distribution is uniform over floating-point values, so for example about a quarter of samples are between -1e-150 and 1e-150.

pavpanchekha 6y ago

That's right. Herbie's error measure is average error over the whole range of inputs; one way to think about that is that it's really measuring the area of inputs that have reasonable error. There aren't many floating-point values near -1, so Herbie doesn't consider the error there that important. To be precise, Herbie tries to avoid overfitting by "charging" itself 1 bit of accuracy for every branch; adding a branch for, say, -1 <= x <= -0.8 isn't worth it. If you set a precondition, say -1 <= x <= -0.8, Herbie will instead focus on that range. Also note that the Herbie web demo has some options set that make it fast (to handle load) at the cost of lower accuracy. For example, if you download and install it yourself, you can turn on support for the special numeric functions (like log1p) or increase the number of search iterations done.

contravariant 6y ago
I see. I guess it ends up doing something a bit unexpected because the distribution it's sampling from is quite different from the numbers people often deal with (also unlike the example on the main page it can apparently fail to find a function that is more accurate for all inputs). Also this is a bit of a special case because it's easy to show that x - x^2 / 2 is within 10^-15 of the true value provided abs(x) is within 10^-6 or so, so it's easy to figure out how good Herbie is doing.
- pavpanchekha 6y ago
  That's right, the implicit distribution is uniform over floating-point values, so for example about a quarter of samples are between -1e-150 and 1e-150.