3 ms·

All interests I had in Taylor/Macluarin series were promptly beat out of me when having to manually calculate hundreds of them in Calculus courses :). The Reme

All interests I had in Taylor/Macluarin series were promptly beat out of me when having to manually calculate hundreds of them in Calculus courses :). The Remez algorithm linked at the end still inspires curiosity though. Any other "numerically more useful" approximation algorithms folks want to highlight? The Padé approximant looks like another interesting candidate to read about.

raphlinus 2y ago
Yes! The workhorse numerical technique is Chebyshev approximation. Remez exchange usually starts with it, for fine-tuning with respect to a "maximum error" norm, but it also works quite well by itself, and can be computed efficiently (even with a high degree) even from a function you can only evaluate numerically. A really good place to read up on it is the documentation for Chebfun. https://www.chebfun.org/docs/guide/guide04.html https://www.chebfun.org/docs/guide/guide04.html Also: be on the lookout for a blog post on using Chebyshev polynomials to efficiently compute error metrics for curves.
- bee_rider 2y ago
  Chebyshev’s polynomials seem to have eclipsed his semi-iterative method for solving linear systems, which is too bad IMO.
  - sfpotter 2y ago
    Why is that bad? Are there any cases where Chebyshev iteration is the best option? It seems like acquiring the necessary information about the spectrum would be prohibitive in practice, although I have never tried it out.
    - bee_rider 2y ago
      I’ve only really played with it on a single-node machine, which isn’t where it ought to shine. IMO it sits at a really interesting spot as a sort of “more robust” (hand-waves) iterative solver that doesn’t require inner products. You need to know something about the spectrum sure, but sneakily figuring out things about the spectrum is somewhere where people can show off their expertise I think.
      - sfpotter 2y ago
        That seems plausible. I actually spent a little time digging into this, and I'm not sure if this method is actually due to Chebyshev! This link has the most extensive references I found: https://encyclopediaofmath.org/wiki/Chebyshev_iteration_method and from what I can tell it's actually due to Richardson.
bee_rider 2y ago
I had the opposite reaction. Math department: “oh, look, we are very clever, see these pretty plots the differential equations generate and all the beautiful closed form solution we can make” Me: “Hmm, very nice, I am not smart enough for this.” Engineering department: “Complicated equation bad, smash with Taylor series, little part go bye-bye, big part is smooth.” Me: “Yes this equation will match my brain nicely.”
abdullahkhalids 2y ago
Weird. When I was doing boxing, having to run sprints every training session, doing dozens of pushups, situps and what not did not beat the interest I had in boxing :).
- borroka 2y ago
  Not at all weird, because you were also boxing. But if your boxing sessions were only to be sprints, pushups, situps etc, and no boxing, you would surely lose interest in doing boxing at that place. Likewise, if your jiu jitsu training were only drills and no sparring, there would be no people doing jiu jitsu after a while.
  - abdullahkhalids 2y ago
    My boxing training sessions were about 50-50 exercises and the padding/sparring session (in which there were big breaks). In my undergrad, I was made to take 6 math courses, and about 18 physics courses which used that math [1]. Plus additional Engineering/CS electives. So over a 25-75 ratio of exercise to "real" stuff. Much better ratio than boxing. [1] If you decide that the math courses have nothing real/interesting in them. Which is not true.
- zamadatix 2y ago
  The big difference is probably that I wasn't taking the math classes out of interest in being great at computations. Just like manually doing long division of the factorial denominator wouldn't have made me any better able to apply Taylor series, finding the 3rd derivative of a random trig function manually (well, for some reason a trig reference table was allowed but not a CAS) dozens of times didn't make me any better able to use them either but did wear out my energy for the topic anyways. Of course this "interested in applying concepts to solve the problem, not calculating the values in said solution" lean is probably why I was in a computer program instead of a math program in the first place :).
  - abdullahkhalids 2y ago
    Let me state my point explicitly: You are probably referring to real math as one of (1) building abstraction, (2) doing proofs of theorems, (3) solving applied math problems. To build competence at these problems, you need to exercise the basic skills, of which computational skills are a big part. If you don't exercise your computational skills, you can't become good at mathematics in the sense of (1), (2), or (3). The same way you can't get to be good at boxing without doing a lot of pushups. Now, not all teachers are good. Many certainly try to force students to do too much computational practice, compared to their level. But both as a student and later as a prof, I discovered that there is value is doing really dumb and complicated calculations by hand sometimes during your education. The reason is that many times when people your computational algorithm fails, you have to do the dirty thing by hand to figure out what went wrong. And depending on what you do, computing the 3rd derivative of a random trig function manually correctly with nuances is exactly what you need to do [1]. Well, not everyone. Many/most people do jobs that only require low levels of maths, and they might never need to do any of that. But a math course taught to everyone has to take a cookie cutter approach and that usually requires setting a high enough bar, so people who have to do the hard math at their job, have those skills. This is an unfortunate fact of resource optimization. [1] I am an industry researcher now, and have to do all kinds of these crappy calculations.

raphlinus 2y ago

Yes! The workhorse numerical technique is Chebyshev approximation. Remez exchange usually starts with it, for fine-tuning with respect to a "maximum error" norm, but it also works quite well by itself, and can be computed efficiently (even with a high degree) even from a function you can only evaluate numerically. A really good place to read up on it is the documentation for Chebfun. https://www.chebfun.org/docs/guide/guide04.html https://www.chebfun.org/docs/guide/guide04.html Also: be on the lookout for a blog post on using Chebyshev polynomials to efficiently compute error metrics for curves.

bee_rider 2y ago
Chebyshev’s polynomials seem to have eclipsed his semi-iterative method for solving linear systems, which is too bad IMO.
- sfpotter 2y ago
  Why is that bad? Are there any cases where Chebyshev iteration is the best option? It seems like acquiring the necessary information about the spectrum would be prohibitive in practice, although I have never tried it out.
  - bee_rider 2y ago
    I’ve only really played with it on a single-node machine, which isn’t where it ought to shine. IMO it sits at a really interesting spot as a sort of “more robust” (hand-waves) iterative solver that doesn’t require inner products. You need to know something about the spectrum sure, but sneakily figuring out things about the spectrum is somewhere where people can show off their expertise I think.
    - sfpotter 2y ago
      That seems plausible. I actually spent a little time digging into this, and I'm not sure if this method is actually due to Chebyshev! This link has the most extensive references I found: https://encyclopediaofmath.org/wiki/Chebyshev_iteration_method and from what I can tell it's actually due to Richardson.

bee_rider 2y ago

I had the opposite reaction. Math department: “oh, look, we are very clever, see these pretty plots the differential equations generate and all the beautiful closed form solution we can make” Me: “Hmm, very nice, I am not smart enough for this.” Engineering department: “Complicated equation bad, smash with Taylor series, little part go bye-bye, big part is smooth.” Me: “Yes this equation will match my brain nicely.”

abdullahkhalids 2y ago

Weird. When I was doing boxing, having to run sprints every training session, doing dozens of pushups, situps and what not did not beat the interest I had in boxing :).

borroka 2y ago
Not at all weird, because you were also boxing. But if your boxing sessions were only to be sprints, pushups, situps etc, and no boxing, you would surely lose interest in doing boxing at that place. Likewise, if your jiu jitsu training were only drills and no sparring, there would be no people doing jiu jitsu after a while.
- abdullahkhalids 2y ago
  My boxing training sessions were about 50-50 exercises and the padding/sparring session (in which there were big breaks). In my undergrad, I was made to take 6 math courses, and about 18 physics courses which used that math [1]. Plus additional Engineering/CS electives. So over a 25-75 ratio of exercise to "real" stuff. Much better ratio than boxing. [1] If you decide that the math courses have nothing real/interesting in them. Which is not true.
zamadatix 2y ago
The big difference is probably that I wasn't taking the math classes out of interest in being great at computations. Just like manually doing long division of the factorial denominator wouldn't have made me any better able to apply Taylor series, finding the 3rd derivative of a random trig function manually (well, for some reason a trig reference table was allowed but not a CAS) dozens of times didn't make me any better able to use them either but did wear out my energy for the topic anyways. Of course this "interested in applying concepts to solve the problem, not calculating the values in said solution" lean is probably why I was in a computer program instead of a math program in the first place :).
- abdullahkhalids 2y ago
  Let me state my point explicitly: You are probably referring to real math as one of (1) building abstraction, (2) doing proofs of theorems, (3) solving applied math problems. To build competence at these problems, you need to exercise the basic skills, of which computational skills are a big part. If you don't exercise your computational skills, you can't become good at mathematics in the sense of (1), (2), or (3). The same way you can't get to be good at boxing without doing a lot of pushups. Now, not all teachers are good. Many certainly try to force students to do too much computational practice, compared to their level. But both as a student and later as a prof, I discovered that there is value is doing really dumb and complicated calculations by hand sometimes during your education. The reason is that many times when people your computational algorithm fails, you have to do the dirty thing by hand to figure out what went wrong. And depending on what you do, computing the 3rd derivative of a random trig function manually correctly with nuances is exactly what you need to do [1]. Well, not everyone. Many/most people do jobs that only require low levels of maths, and they might never need to do any of that. But a math course taught to everyone has to take a cookie cutter approach and that usually requires setting a high enough bar, so people who have to do the hard math at their job, have those skills. This is an unfortunate fact of resource optimization. [1] I am an industry researcher now, and have to do all kinds of these crappy calculations.