5 ms·
> This would mean that overall geometry is not a (big) factor when it comes to getting stuck on local optima. I thought the results only applied to convex shap
by scythmic_waves 6y ago
> This would mean that overall geometry is not a (big) factor when it comes to getting stuck on local optima.
I thought the results only applied to convex shapes. Search spaces in ML need not be convex, right? Or am I missing something?
- tovej 6y agoI assume they are convex, typically cuboid. This is the type of space you get when each parameter is searched for a certain range. Surely there could be search spaces that aren't convex. In that case the range of a variable would depend on the values of other variables. If you have an example of such a case I'd be interested in knowing about it.
- Bootvis 6y agoIf you have linear constraints the search space will be convex.
- scythmic_waves 6y agoAh I was confusing the loss landscape with the parameter search space because you said "local optima". Yep, I imagine most parameter search spaces are convex.