4 ms·
1. Gradient descent almost always finds a non optimum local min (it is not guaranteed to find a global min).
by aabaker99 5y ago
1. Gradient descent almost always finds a non optimum local min (it is not guaranteed to find a global min).
- agnosticmantis 5y agoIsn’t the current best practice to train highly over-parametrized models to zero training error? That’d be a global optima, no? Unless we’re talking about the optima of test error.
- aabaker99 5y agoIf you find a zero in a non negative function, I would call that a global minima, yes.
- aledalgrande 5y agoYeah but depending on the data you might have even worse results, selecting the right subset to be representative is really important.
- jhgb 5y agoWould a random sample be representative? Statistically this seems to be the case for any large N. In fact it's not clear to me that any other sample would be more representative.
- aledalgrande 5y agoMany public datasets have skewed classes so if you take a random approach you're not gonna have a good result. And N might not be big enough anyway.