3 ms·
I’ve been slowing crunching through Math for Deep Learning, so spent a fair amount of time looking at Hessian matrices + second order optimisation. I’ve been sl
by dandelionv1bes 9mo ago
I’ve been slowing crunching through Math for Deep Learning, so spent a fair amount of time looking at Hessian matrices + second order optimisation. I’ve been slowly reading this book for a year, so stopping to do most of the math by hand each time. One chapter to go!
Then I was sick all last week, so ended up down a rabbit hole about the current card collecting bubble (right word?). Super interesting.
- dapperdrake 9mo agoWhere do Hessians come into play for neural networks? It seems like they just use auto-diff to compute the Jacobian or the gradient for backpropagation. The theoretical results sometime look at the second order derivative.