3 ms·

So I'm not ML guru or anything but what I learned was that if you have m features on n samples you want n > m to prevent over-fitting, no? Also, with so few sa

by cletus 7y ago

So I'm not ML guru or anything but what I learned was that if you have m features on n samples you want n > m to prevent over-fitting, no? Also, with so few samples, how do you do your hyperparameter tuning and validation? I mean you could eliminate certain features in isolation but that doesn't capture dependent features. And how would you do dimensionality reduction?

ruborcalor 7y ago
Yes I agree you want number of samples to be greater than number of features. This is why the number of features were reduced from over 400,000 to 25. After this reduction the number of features is less than the number of samples (~700). Honestly I didn't prioritize hyperparameter tuning enough. I pretty much went with one of the first models I identified. Could you elaborate on the idea of not capturing dependent features please?