It's the optimization strategy. Optimizers for neural nets are designed to perform well for non-linear problems, which comes at the expense of not providing exa

It's the optimization strategy. Optimizers for neural nets are designed to perform well for non-linear problems, which comes at the expense of not providing exact solutions for linear regressions. A general exact solution is not possible for nonlinear, over-specified problems (which is what neural nets are good at), so strategies different than those used in linear regression are necessary.