4 ms·
> If the derivative is abnormal in a way such that the normal truncated calculative formulas would conceal it, then AD would fail to give an accurate result as
by ogogmad 4y ago
> If the derivative is abnormal in a way such that the normal truncated calculative formulas would conceal it, then AD would fail to give an accurate result as well.
The following is proof of otherwise. First implement the dual numbers by borrowing the code from here [1]. Then define the sine function in a crude way:
from math import factorial
def sin(x):
return sum((-1)**i * x**(2*i + 1) / factorial(i) for i in range(100))
We then define f:
def f(x, a=1e-36):
return 1 + a * sin(x / a)
and use the dual numbers to estimate the derivative at 0:
In [49]: print(f(Dual(0,{'epsilon':1})))
f = 1.0
fepsilon = 1.0
We get 1 as the estimate, which is correct! For comparison, we may try using finite differencing to estimate the derivative at 0:
In [50]: h = 1e-36; (f(h) - f(0))/h
Out[50]: 0.0
The estimate for the derivative is 0 here, which is wrong. The finite differencing cannot be improved using stencil theory because the estimate of f(x) is always exactly 1 under floating point, and therefore the estimate for f'(x) is always exactly 0. Dual numbers win.
[1] - https://github.com/ujjwalkhandelwal/Dual-numbers-and-automatic-differentiation-using-Python/blob/main/dual/ad_dual.py#L28 https://github.com/ujjwalkhandelwal/Dual-numbers-and-automat...