4 ms·
The examples other people are using are fairly narrow. I would like to substantiate that text categorization via naive bayes classifier is surprisingly accurate
by mooman219 9y ago
The examples other people are using are fairly narrow. I would like to substantiate that text categorization via naive bayes classifier is surprisingly accurate and simple. This paper[1] uses ngrams and a simple out of place measure to compare articles against different verticals and often sees greater than 99% accuracy for relatively small blocks of text. The out of place measure also adds a penalty to features not found in the document, which helps establish the individuality for the category classification. Raw matching performance is also fairly impressive; A less naive implementation is also highly parallelizable.
[1] http://odur.let.rug.nl/~vannoord/TextCat/textcat.pdf http://odur.let.rug.nl/~vannoord/TextCat/textcat.pdf
- abhgh 9y agoFrom experience, I suspect NB does well on text primarily for two reasons: (1) High dimensionality - this is also the reason why SVMs with linear kernels do so well on text (NBs find a linear boundary too - but a different linear boundary than linear-kernel SVMs) (2) this reason applies to certain cases only: text where "token" based estimates are sufficiently discriminatory. For ex take a dataset that has two kinds of documents - one talking about product A and the other talking about product B. And you want to label the documents based on which product they're talking about. Here, just noticing if the term "A" or "B" appears in the document is good enough for classification. You don't have to have powerful models that infer "connotations" of words based on context. NB will do well here, esp if bundled with a feature selection technique that weeds out noisy features (like sthg based on Normalized Mutual Information)