You are not guaranteed to find optimal solutions (although you might find local minima) For constrained optimisation ideally you should be using Lagrange multi

You are not guaranteed to find optimal solutions (although you might find local minima) For constrained optimisation ideally you should be using Lagrange multipliers. But even then there are `abnormal` cases where the Lagrange multiplier fails to give you an optimal solution. Roughly your constraint does not have enough information to give you a unique solution (dg insufficient rank to find solutions in df = \lambda dg). In infinite dimensions (minimising a functional) a penalty metric is an even worse an idea. My favorite example is sub-Riemannian geometry [0]. People wanted to find sub-Riemannian geodesics - special curves that could only move in restricted directions. Early on people tried to use the penalty metric idea (making it really expensive to move in certain directions), but Montgomery [0] proved that this does not necessarily give optimal solutions. The limit of an equation is not necessarily the same as the limit of the solutions. In a sense they used Lagrange multipliers to examine the situation. Looking at abnormal optimisation problems is an interesting area of research [1]. However if a local solution is "good enough" maybe you can get away with a penalty metric. [0] https://www.ams.org/books/surv/091/ https://www.ams.org/books/surv/091/ [1] https://books.google.com.au/books?isbn=9401594384 https://books.google.com.au/books?isbn=9401594384