I have never dabbled with constraining the parameter values themselves. Ive mostly put the constraints into the loss function. This works well when working with

I have never dabbled with constraining the parameter values themselves. Ive mostly put the constraints into the loss function. This works well when working with regression, a super simple constraint that adds penalty when the regression goes outside of the possible solution space has been extremely helpful in our work. Heuristically, I think it’s more useful during the first few iterations to find the correct local minima. If you were already finding the “correct” local minima this might be less important, but if you’ve ever dealt with convolutional artifacts (boxes, lines, edge effects) in your predictions, well informed constraints tend to help avoid these, as they are a symptom of being in an incorrect local minima.