6 ms·
The implementation of the __cos kernel in Musl is actually quite elegant. After reducing the input to the range [-pi/4, pi/4], it just applies the best degree-1
by dancsi 5y ago
The implementation of the __cos kernel in Musl is actually quite elegant. After reducing the input to the range [-pi/4, pi/4], it just applies the best degree-14 polynomial for approximating the cosine on this interval. It turns out that this suffices for having an error that is less than the machine precision. The coefficients of this polynomial can be computed with the Remez algorithm, but even truncating the Chebyshev expansion is going to yield much better results than any of the methods proposed by the author.
- l- 5y agohttps://github.com/ifduyue/musl/blob/master/src/math/__cos.c https://github.com/ifduyue/musl/blob/master/src/math/__cos.c
- llimllib 5y ago> Input y is the tail of x. What does "tail" mean in this context?
- lifthrasiir 5y agoIt is a double-double representation [1], where a logical fp number is represented with a sum of two machine fp numbers x (head, larger) and y (tail, smaller). This effectively doubles the mantissa. In the context of musl this representation is produced from the range reduction process [2]. [1] https://en.wikipedia.org/wiki/Quadruple-precision_floating-point_format#Double-double_arithmetic https://en.wikipedia.org/wiki/Quadruple-precision_floating-p... [2] https://github.com/ifduyue/musl/blob/6d8a515796270eb6cec8a278cb353a078a10f09a/src/math/cos.c#L68-L76 https://github.com/ifduyue/musl/blob/6d8a515796270eb6cec8a27...
- jhgb 5y agoDoes it make sense to use a double-double input when you only have double output? Sine is Lispchitz-limited by 1 so I don't see how this makes a meaningful difference.
- lifthrasiir 5y agoThe input might be double but the constant pi is not. Let f64(x) be a function from any real number to double, so that an ordinary expression `a + b` actually computes f64(a + b) and so on. Then in general f64(sin(x)) may differ from f64(sin(f64(x mod 2pi))); since you can't directly compute f64(sin(x mod 2pi)), you necessarily need more precision during argument reduction so that f64(sin(x)) = f64(sin(f64timeswhatever(x mod 2pi))).
- jhgb 5y agoBut am I correct in thinking that is at worst a 0.5 ulp error in this case? The lesser term in double-double can't be more than 0.5 ulp of the greater term and sensitivity of both sine and cosine to an error in the input will not be more than 1. Also, in case of confusion, I was specifically commenting on the function over the [-pi/4, pi/4] domain in https://github.com/ifduyue/musl/blob/master/src/math/__cos.c https://github.com/ifduyue/musl/blob/master/src/math/__cos.c , which the comment in https://news.ycombinator.com/item?id=30846546 https://news.ycombinator.com/item?id=30846546 was presumably about.
- adgjlsfhk1 5y agoDouble rounding can still bite you. You are forced to incur up to half an ulp of error from your polynomial, so taking another half ulp in your reduction can lead to a total error of about 1 ulp.
- lifthrasiir 5y agoYeah, sine and cosine are not as sensitive (but note that many libms target 1 or 1.5 ulp error for them, so a 0.5 ulp error might still be significant). For tangent however you definitely need more accurate range reduction.
- deleted 5y ago[deleted]
- ufo 5y agoI think the polynomial calculation in the end looks interesting. It doesn't use Horner's rule.
- lifthrasiir 5y agoIt does use Horner's rule, but splits the expression into two halves in order to exploit instruction-level parallelism.
- jacobolus 5y agoConsidering the form of both halves is the same, are compilers smart enough to vectorize this code?
- adgjlsfhk1 5y agoI might be wrong but I would think for something like this vectorizing wouldn't save time (since you would have to move data around before and afterwards. The real benefit of this is it lets you run two fma operations in parallel.
- mnarayan01 5y agoFor anyone else confused by this, the control logic described in the comment happens in https://github.com/ifduyue/musl/blob/master/src/math/cos.c https://github.com/ifduyue/musl/blob/master/src/math/cos.c
- brandmeyer 5y agoThis has been the standard algorithm used by every libm for decades. Its not special to Musl.
- enriquto 5y agoBut isn't this code rarely called in practice? I guess on intel architectures the compiler just calls the fsin instruction of the cpu.
- adgjlsfhk1 5y agoNo. The fsin instruction is inaccurate enough to be useless. It gives 0 correct digits when the output is close to 0.
- enriquto 5y ago> 0 correct digits when the output is close to 0 this is an amusing way to describe the precision of sub-normal floating point numbers
- lifthrasiir 5y agoIt is much more amusing if you describe it in ulps; for some inputs the error can reach > 2^90 ulps, more than the mantissa size itself.
- adgjlsfhk1 5y agoIt's not just sub-normal numbers. As https://randomascii.wordpress.com/2014/10/09/intel-underestimates-error-bounds-by-1-3-quintillion/ https://randomascii.wordpress.com/2014/10/09/intel-underesti... shows, fsin only uses 66 bits of pi, which means you have roughly no precision whenever abs(sin(x))<10^-16 which is way bigger than the biggest subnormal (10^-307 or so)
- ramshorns 5y agoIn that range, just returning x would be way better. Maybe even perfect actually - if x is less than 10^-16, then the error of x^3/6 is less than the machine precision for x.
- toolslive 5y agoYup. Chebyshev is the way to go (after 30s of consideration)
- ajross 5y agoThat's got nothing to do with Musl per se, that's just the SunPro code that basically every C library uses. I'm sure the polynomials themselves (there's another one for the "crest" of the sin curve, and still more for log and exp and the rest of the family) date from somewhere in computing pre-history. Optimizing polynomial fits to analytic functions was something you could do on extremely early computers.
- bertr4nd 5y agoAre there libraries/tools that people use to do Remez/Chebyshev/etc. function expansions? I can do a basic Taylor series expansion by hand but I’m out of my depth with more sophisticated techniques.
- adgjlsfhk1 4y agosollya is the king here
- bertr4nd 4y agoThanks!
- jeremysalwen 5y agoThanks for explaining this. I actually wrote a SIMD implementation of trig functions years ago, using the techniques you describe. You can check it out: https://github.com/jeremysalwen/vectrig https://github.com/jeremysalwen/vectrig I compared several different methods of generating polynomials of different sizes for speed and precision (spoilers: taylor series were the worst and minimax polynomials (Remez algorithm) were the best). Another (surprising) thing which I learned during the project was that the range reduction was just as (if not more) important to the accuracy of the implementation than the polynomial. If you think about it, you will realize that it's actually pretty difficult to quickly and accurately compute the sin of large numbers like 2^50. I also tried to directly optimize the coefficients for the accuracy of the polynomial on the required range, but that experiment was unsuccessful. It's all there in the repository, the implementations, notes about the different polynomials used, and the accuracy/speed statistics for the different methods.
- jhgb 5y ago> I compared several different methods of generating polynomials of different sizes for speed and precision (spoilers: taylor series were the worst and minimax polynomials (Remez algorithm) were the best). I would have expected at least an LSQ approximation with a basis of Legendre polynomials thrown into the mix. I got that as a basic homework in my numerics class once, after we've shown to ourselves in the class that [1, x, x², x³...] is not a really good basis to project things onto.