4 ms·
Here's the (pedantically-worked) example with reverse-mode: Program: y = g(x) = x^3 z = f(y) = cuberoot(y) Want: dz/dx Reverse-mode autodiff: // Forw
by FooBarBizBazz 7y ago
Here's the (pedantically-worked) example with reverse-mode:
Program:
y = g(x) = x^3
z = f(y) = cuberoot(y)
Want: dz/dx
Reverse-mode autodiff:
// Forward pass
y = g(0) = 0^3 = 0 g'(0) = 3 x^2 = 0
z = f(0) = cuberoot(0) = 0 f'(0) = (1/3) / cuberoot(y^2) = (1/3)/0 = Inf
// Reverse pass
dz/dy = f'(0) = Inf
dz/dx = dz/dy * g'(0) = Inf * 0 = NaN