4 ms·
The (conditional) independence assumptions it makes are pretty strong and usually inaccurate. In the text example, consider "Steph Curry" and "Warriors." These
by imh 9y ago
The (conditional) independence assumptions it makes are pretty strong and usually inaccurate. In the text example, consider "Steph Curry" and "Warriors." These are going to show up very frequently together, and both provide strong evidence that the topic is sports. However, they don't provide strongly independent evidence of sports. Supposing Curry announced he was making an investment in a local restaurant, the article would probably mention the Warriors just because it's Curry.
The problem is that seeing "Warriors" when you see "Steph Curry" shouldn't make you that much more confident that it's sports than just seeing "Steph Curry" alone. Just like seeing "University" with "Stanford" shouldn't change any inferences much from seeing "Stanford" alone. In naive bayes models, these are treated as independent pieces of evidence and that can lead to overconfidence and errors.
Naive bayes is a generative model, so in flipping bayes rule around to discriminate classes, you have to be sure your probability model is decent. In a discriminative model, you just go straight to learning the p(class | observations) and have no requirement for a decent model of p(observations). p(observations) is the kind of model that would have to know about "university" and "stanford" being likely to be observed together. Often, it's better to go straight to the discriminative model. In this case, logistic regression.