4 ms·
Isn't training literally an optimization of a loss function over training data?
by taeric 5y ago
Isn't training literally an optimization of a loss function over training data?
- diffeomorphism 5y agoTechnically yes, but practically no. This is discussed in depth in section 11.
- srean 5y agoIt is and that is why I find the title a clickbait. The article isnt bad, but the title is certainly being coyly disingenuous. In problems like this there are two aspects, (i) designing or specifying the search space of functions (ii) choosing the best function within the search space. The opposite extremes are a) the search space contains only one function, the right function. In this case the training/optimization is moot. The other extreme is to have a very wide search space, say all smooth functions. In that case searching/training/optimizing is more challenging. The more reasonable example is one uses domain knowledge to design a much more restricted search space (for example, one may encode that the function is periodic with a known period) making the next step easier.