3 ms·
Assumptions such as independence are always dutifully mentioned when discussing statistical methods, but I never see a discussion about in which sort of practic
by andreasvc 12y ago
Assumptions such as independence are always dutifully mentioned when discussing statistical methods, but I never see a discussion about in which sort of practical problems such assumptions hold. In the case of independence and text classification, it seems to me that it doesn't hold, certainly not on the scale of a paragraph. As the blog post mentions, naive Bayes performs well despite violating the independence assumption; I'm curious to understand why that is.
- rasbt 12y agoIn practice, I would say that it is a very challenging task to determine whether variables are totally independent or not: It is more about the strength of independence. Every time something is "estimated", you'd introduce an additional source of error, and in cases of weak independence the violation may be negligible. Modeling the conditional dependences is basically the idea of Bayesian Belief Networks, and you might find the paper "Comparing Bayesian Network Classifiers" interesting in this context (includes comparisons to Naive Bayes) http://link.springer.com/article/10.1023/A:1007465528199 http://link.springer.com/article/10.1023/A:1007465528199