3 ms·

Very cool. That seemed to work better than I'd have expected. The explanation of the 'naive' part of Naive Bayes isn't quite right, though. Throwing out the

Very cool. That seemed to work better than I'd have expected. The explanation of the 'naive' part of Naive Bayes isn't quite right, though. Throwing out the possibility of the animal being human based on the datapoint "four legs" is orthogonal to naivety. A more sophisticated Naive Bayes system could reject classifying an animal as human based on having four legs, and conversely a non-naive system i.e. one that used joint probabilities of the features, might not be more likely to. I guess the most rational way to do that would be to express and calculate the conditional probabilities with some statistical distance, like the # of standard deviations. So 100k examples of humans without a single one having the "four legs" feature would make that a very strong indicator of being non-human. And that'd work just as well with a naive algorithm as any other. Is there a name for this?

alextp 16y ago
You seem to be talking about smoothing: how many non-human four-legged animals and non-four-legged humans have you got to see before you can estimate that humans are not likely to have four legs? This is reasonably well-solved; and you either use dirichlet (laplace, add-one) smoothing or something more closely corresponding to your domain, like good-turing smoothing. This reweights the probabilities in your classifier to make sure that (by analogy) if you see a talking, thinking, four-legged banker, you will probably classify him as a human (since the other features overwhelmingly point in that direction). They all assume some measure of confidence in the features, and make sure that a feature that has been observed once or twice will not move the class boundaries too much.