3 ms·
You don't always need the input to compute the gradient. For example the gradient of a sum function doesn't require the original input, it just sets all of the
by wsmoses 6y ago
You don't always need the input to compute the gradient. For example the gradient of a sum function doesn't require the original input, it just sets all of the derivative(input)'s to 1.
- shoyer 6y agoTo be more precise, in backwards mode auto-diff, inputs only need to be saved if they are used in a non-linear way.