3 ms·
Isn't gradient descent basically PID over parameters? And tricks like momentum basically a low-pass filter integrated in the PID loop? It's quite weird how not
by mordae 4y ago
Isn't gradient descent basically PID over parameters?
And tricks like momentum basically a low-pass filter integrated in the PID loop? It's quite weird how not that many concepts from analog electronics domain have gotten carried over to ML.