3 ms·
My experience with real world (but still academic) data has been that there is lots of magic---feature selection to be specific. (I'm not disagreeing, just ref
by tel 13y ago
My experience with real world (but still academic) data has been that there is lots of magic---feature selection to be specific.
(I'm not disagreeing, just referring to a different kind of "magic")
Everything else matters, but when your ML doesn't work it's 100% a feature selection problem. Which usually means it's 99% a problem of getting lots of domain expertise jammed up against a lot of ML experience and mathematical understanding. It's also a bear.
- linux_devil 13y agoAgree I too found feature extraction and selection one of the vital components that helps to improve scores in lot of ML competitions hosted at Kaggle
- nabla9 13y agoThe way 80% of real real world (non academic) data mining problems are solved: 1. Feature selection. 2. intelligent data massage. Real world data has usually noise that humans can easily identify as irrelevant or erroneous. 3. logit regression. Starting with simple, well understood algorithms first should be the second lesson after knowing about validation sets. In those cases where they are not enough, they set the baseline for comparison against other algorithms.
- tlarkworthy 13y agoYeah I agree both with you and the ancestor. To get good results you basically have to provide the solution to the algorithm through great features. Then the choice of algorithm is not so important, as all the hard work is done in representation. So the algorithms are not magic, you have to drop the solution in front of them and all they do is find the optimal parameters for some trivial low dimensional separation manifold.
- larrydag 13y agoI would add 4. Ensemble methods. Having a few models helps to generalize the data fairly well.
- tel 13y agoThat's a good point.