3 ms·
Are you familiar with what the 2nd derivative of a function is? The Hessian is just that, when you have a function with multiple inputs. What's the second deriv
by mehrdadn 8y ago
Are you familiar with what the 2nd derivative of a function is? The Hessian is just that, when you have a function with multiple inputs. What's the second derivative of f(x,y) = x/y? Well there are four of them depending on the order in which you differentiate: f_xx = 0, f_xy = -1/y^2, f_yx = -1/y^2, f_yy = 2x/y^3. You just put these in a nice matrix and call it the Hessian matrix. So for functions of N input variables, there are N^2 second-derivatives, so the Hessian is the matrix of all those second derivatives.
I'm not sure how you saw it used for fitting a logistic model, but in general knowing the concavity of a function is useful for minimizing or maximizing it (e.g. by repeatedly approximating it as a 2nd-order polynomial and minimizing that), so maybe that's what you saw.
- thomasahle 8y ago> I'm not sure how you saw [the Hessian] used for fitting a logistic model Probably some 2nd derivative version of gradient descent or Newton's method. https://en.wikipedia.org/wiki/Newton%27s_method_in_optimization#Higher_dimensions https://en.wikipedia.org/wiki/Newton%27s_method_in_optimizat...
- mehrdadn 8y agoYeah that's what I was referring to when I said "approximating it as a 2nd-order polynomial and minimizing that".