3 ms·
> For low training set sample sizes it looks like the simpler method (just picking the top 10 and using a linear model) slightly outperforms the more complicate
by minimaxir 9y ago
> For low training set sample sizes it looks like the simpler method (just picking the top 10 and using a linear model) slightly outperforms the more complicated methods.
This is a very bad argument for the given clickbaity headline. A methodology that works well for one dataset with few observations will not necessarily work well for another dataset.
You can do almost whatever you want with small datasets, it's just harder than with big data (and is necessary if obtaining data is expensive, e.g. medical trials). Specifically, you'll want to do bootstrapping to simulate additional data and reduce the uncertainty due to a low amount of data.
The "almost" is that you can't have hundreds of features if you have a small dataset (Curse of Dimensionality: https://en.wikipedia.org/wiki/Curse_of_dimensionality https://en.wikipedia.org/wiki/Curse_of_dimensionality)