It’s interesting that your experience has focused on polynomial regression. My experience is that “nonlinear” regression more means adding interaction terms, te

It’s interesting that your experience has focused on polynomial regression. My experience is that “nonlinear” regression more means adding interaction terms, testing for discontinuities, and allowing for partially pooled separate linear functions based on strata of the data, like hierarchical regression. For anything truly needing to model nonlinearities in the manifold of the input data, I’d instead use tree based regression, SVM regression with an appropriate kernel, Gaussian Processes /krieging, etc. I’ve never experienced pragmatically useful results with polynomial regression that wouldn’t have been more easily obtained from more appropriate nonlinear models, and wasn’t aware anyone would ever actually try this in practice.