13 ms·
Much discussion of backprop could be avoided by recalling that the bit that does the work is the chain rule from calculus. Error terms represent a sum and prod
by conjectures 10y ago
Much discussion of backprop could be avoided by recalling that the bit that does the work is the chain rule from calculus.
Error terms represent a sum and product of derivatives. The product of a bunch of terms will tend to get really big or really small.
The rest is detail: are the terms in some interval? Which? how many are we multiplying? how many are we summing over? do we doctor the sum after we get it?