4 ms·
Re: "Empirical studies showed that the mutli-variate Bernoulli model is inferior to the multinomial Bayes model, and the latter has been shown to reduce the err
by dj-wonk 12y ago
Re: "Empirical studies showed that the mutli-variate Bernoulli model is inferior to the multinomial Bayes model, and the latter has been shown to reduce the error by 27% on average [13]."
I remain skeptical of statements that claim that one kind of model is categorically inferior. Also, I have a hard time believing this is a fair summary of the literature. First, one study is cited, but the sentence implies many.
Second, the citation in the original post is not a fair summary of the cited research. The abstract of "A Comparison of Event Models for Naive Bayes Text Classification" says: "This paper aims to clarify the confusion by describing
the differences and details of these two models, and by
empirically comparing their classification performance
on five text corpora. We find that the multi-variate
Bernoulli performs well with small vocabulary sizes,
but that the multinomial performs usually performs
even better at larger vocabulary sizes—providing on
average a 27% reduction in error over the multi-variate
Bernoulli model at any vocabulary size."
- rasbt 12y agoThanks for the comment, and I agree! I rephrased it a little bit to "Empirical comparisons provide evidence that the multinomial model tends to outperform the multi-variate Bernoulli model if the vocabulary size is relatively large [13]. However, the performance of machine learning algorithms is highly dependent on the appropriate choice of features. In the case of naive Bayes classifiers and text classification, large differences in performance can be attributed to the choices of stop word removal, stemming, and token-length [14]. In practice, it is recommended that the choice between a multi-variate Bernoulli or multinomial model for text classification should precede comparative studies including different combinations of feature extraction and selection steps."
- dj-wonk 12y agoThanks for the updates.