Great post! The author touches on Chebychev polynomials, which are the basis for quite a few numerical analysis tricks, including conjugate gradient [1], accel

Great post! The author touches on Chebychev polynomials, which are the basis for quite a few numerical analysis tricks, including conjugate gradient [1], accelerated gradient descent [2], and Chebychev semi-iterative methods (which find the best combination of past iterates to use during an optimization procedure; sadly I can't find a good reference). There are a number of facts/folklore about Chebychev polynomials that seem to go stated without proof in a lot of papers, so a few years ago I wrote up some notes [3,4] with the most common claims. Maybe someone will find them useful! [1] (p35) http://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf http://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradi... [2] http://blog.mrtz.org/2013/09/07/the-zen-of-gradient-descent.html http://blog.mrtz.org/2013/09/07/the-zen-of-gradient-descent.... [3] https://benrbray.com/static/notes/chebychev-polynomials_dec16.pdf https://benrbray.com/static/notes/chebychev-polynomials_dec1... [4] https://benrbray.com/static/notes/minimax-approx_nov16.pdf https://benrbray.com/static/notes/minimax-approx_nov16.pdf