4 ms·
In general, it's OK to use more parameters than data points even, if you properly use regularization (such as weight decay), for example. Other times, even sign
by nearestneighbor 17y ago
In general, it's OK to use more parameters than data points even, if you properly use regularization (such as weight decay), for example. Other times, even significantly fewer parameters than data points can be wrong.
- jey 17y agoThat sounds pretty ad-hoc. Sure, you can throw an L1 or L2 regularizer on your objective function, but it should be well-motivated. Should probably just use Gaussian process regression if you want to do inference over the space of all[1] functions in a principled (i.e. Bayesian) manner. 1. (or the space of all polynomial functions or something. I forget)
- _delirium 17y agoI don't think there's any consensus in the statistics community on which of the many ways to do inference over "nearly all" functions is the better one; nonparametric regression is basically a whole field, and pretty in flux over the past 10 years.