3 ms·
many competition solutions use stacking, where you use the probabilities of several classifiers are used instead of their predictions. Since naive bayes makes
by autokad 9y ago
many competition solutions use stacking, where you use the probabilities of several classifiers are used instead of their predictions.
Since naive bayes makes an independence assumption, and most competitions have features that are highly correlated, its probability / confidence of its prediction is probably a lot more useless than other classifiers.
besides that, nb doesn't do feature selection, regularization, and interactions like many tree based algorithms (such as XGBoost) do out of the box.
interactions is a big one for me. With NB, you have to add an interaction feature by multiplying two together, but in creating the feature you must steal some probability away from the other stuff, and finding a good interaction among many features is hard. The tree structure of random forests naturally captures that
edit: to your last point, nb does worse on larger data sets. when you dont have a lot of data, you have to make larger assumptions. NB makes a big independence assumption, thus it does well on very small data sets but falls short on large ones