3 ms·
I assume backprop is still most efficient for the neural net itself, rather than one of these algorithms? I am doing a course and I am aware that there are othe
by quickthrower2 3y ago
I assume backprop is still most efficient for the neural net itself, rather than one of these algorithms? I am doing a course and I am aware that there are other algorithms than gradient descent but haven’t seen the details yet on that.
I know things like cross entropy loss are designed to be easy to differentiate and probably more efficient to do that than something that approximates it.
- efavdb 3y agoYeah, backprop gives efficient gradient calcs to train at a given net architecture. To find the best architecture though people try various other methods, eg random search or Gaussian processes, which don’t evaluate derivatives wrt architecture Params.