3 ms·
But if the definition of "optimisation" is achieving the best solution for the problem, then regularization isn't constraining optimisation, it's part of gettin
by mdiesel 6y ago
But if the definition of "optimisation" is achieving the best solution for the problem, then regularization isn't constraining optimisation, it's part of getting to the best solution and so becomes a part of the better optimiser.
I agree, any sufficiently complex problem is beyond what we can define as a concrete objective function and any of the solutions we have. So instead the best we can hope for is that today is closer to being optimal than yesterday, and tomorrow will be better still.
- ssivark 6y agoRegularization might give better solutions, but it’s no more optimizing the stated objective, and often runs counter. In some cases, regularization is explicit and by choice, but it’s also often implicit and not under control.
- mdiesel 6y agoAgain, "stated objective" isn't the original objective but the approximation we've had to use instead. You say that it gets a "better" solution, so the against some indescribable original objective there's a difference.