4 ms·
I don't see any mention of automatic differentiation which I thought was the dominant technique in this area. Is there an advantage to the method described in O
by jjoonathan 11y ago
I don't see any mention of automatic differentiation which I thought was the dominant technique in this area. Is there an advantage to the method described in OP (other than mathematical cuteness)?
- jacobolus 11y agoHe did link to a book of papers largely about automatic differentiation, http://amzn.com/0898713854 http://amzn.com/0898713854 (table of contents here http://bt.pa.msu.edu/cgi-bin/display.pl?name=sfbook http://bt.pa.msu.edu/cgi-bin/display.pl?name=sfbook)
- jjoonathan 11y agoI already know that a book on numerical analysis is likely to contain the answer I seek. The point of asking was to avoid a trip to the library since the working memory of someone who has been investigating differentiation techniques also likely contains the answer I seek.
- jacobolus 11y agoI don’t quite understand what your question is, so I’m going to assume you were trying to ask, “When would you not want to use automatic differentiation?” I think the main time you can’t get much out of automatic differentiation is when your function is a black box or approximation, e.g. it comes from measuring physical data, or is the output of some kind of simulation, etc. I’m not an expert though. You can get a lot of hits if you google related keywords (e.g. a query with both terms "numerical differentiation" "automatic differentiation" in quotation marks). But here’s a start, http://alexey.radul.name/ideas/2013/introduction-to-automatic-differentiation/#approximate-the-derivative_-dont_-differentiate-the-approximation http://alexey.radul.name/ideas/2013/introduction-to-automati... You can probably get useful advice if you ask on http://scicomp.stackexchange.com http://scicomp.stackexchange.com or similar.
- jjoonathan 11y agoSomeone who knows to bring automatic differentiation into a discussion about complex-step differentiation with the implication that they have already compared them and found one wanting (while implying that they may have missed something and are receptive to contrary opinions) already knows how CSD and AD work. This person also likely know how to use google, the library / amazon, and stack exchange (at the very least) but upon running a few simple queries has determined that the approximate effort to find an answer exceeds their interest in the subject. Unless someone else can use their own greater familiarity with the subject to provide an opinion or a pointer to a specific easily accessible resource (example: lloda's pdf link, counterexample: your book link) the line of inquiry will fizzle out due to the tyranny of cost/benefit analysis and having better things to do. I appreciate that you were trying to be helpful, which is why I'm trying to be gentle (and explanatory) as I tell you that you weren't.
- lloda 11y agoNo. Automatic differentiation is superior to the complex step method in every way. The only reason to use the complex step method is if you're using legacy languages where it's difficult to implement dual numbers but you have good support for complex numbers. I don't think anybody should be using the complex step method in new applications. Some reading ~ http://aero-comlab.stanford.edu/Papers/martins.aiaa.01-0921.pdf http://aero-comlab.stanford.edu/Papers/martins.aiaa.01-0921....
- jordigh 11y agowtf Automatic differentiation only works for the simplest functions for which you already know what the Taylor series looks like. For those cases, you might as well just hardcode derivative functions and the basic derivative rules (linearity and chain rule). It is not a general-purpose method. For functions that you can't even express by a simple formula, you still have to rely on finite differencing. Don't call "legacy" anything that you don't understand.
- jjoonathan 11y agoCan you comment on the space of functions which you know to be complex differentiable but for which you can't analytically evaluate/approximate the real derivative? If so, I look forward to learning about it. If not, you were throwing stones in a glass house.
- jordigh 11y agoSure, a simple example is the derivative of Riemann zeta for Re s < 1. Granted, usually you work with the logarithmic derivative which is a lot more tractable, but if for whatever reason you need the actual derivative, you're gonna have a lot of trouble with automatic differentiation, as the typical analytic continuation formulas involve some complicated improper integrals. There are also functions that you only know by sampling (e.g. ocean temperatures) for which you assume smoothness. You need to pick an interpolation method, but sometimes you do not interpolate beyond the sampling points, because that's just making up numbers. When you're limited by your original sampling step size, you have little recurse but to compute derivatives by some finite differencing scheme.