3 ms·
Bayes is very simple, easily understood by a 6 year old, if explained properly. There are scary looking integral equations used to express it, but ultimately if
by jackhack 7y ago
Bayes is very simple, easily understood by a 6 year old, if explained properly. There are scary looking integral equations used to express it, but ultimately if you can count you can implement this.
Let's build a bayesian spam engine:
Start with a known-good email (not-spam) and a solicitation (spam). For each word in the not-spam document, count the number of times each word appears. That's your not-spam "corpus". Repeat for the known-spam document. Now you have a list of words+frequency (dictionary of counts). There will be many words which appear in both lists. This is normal.
Now, when you have an unclassified document, for each word in the document check both lists (spam & not-spam). In which list does this word appear more frequently? if the count for that word is higher in the spam list, then this is a "spam" word, therefore this document is +1 spam. If a not-spam word, then this document is +1 not-spam. Repeat for all words. At the end, which count (spam or not-spam) has the higher total? There's your answer. Spam or not-spam. Done.
Simple counting and comparison.
Seriously, that's the algorithm. It's that simple.
If the system miscategorizes a document, move it to the appropriate group yourself and count all the words again. (retrain). The system just "learned" from the mistake. It got "smarter".
You can easily add more categories, other than "spam" and "not spam" -- "questions from customers", "family & friends", etc. It works the same. The group with the highest total "wins" the classification.
extra credit:
Now here's the interesting part. The corpus doesn't have to be words. It could be a switch set to "on" or "off." Joint angles on a robot. Ping time of a packet. Temperature from a thermistor.
And the classification doesn't need to be spam/not-spam, it could be "friend vs foe", upright/inverted, safe/not-safe, or anything else. The algorithm doesn't care, it's looking for probability of being in a classification group.
There you go. Have fun!
- baron_harkonnen 7y ago>Seriously, that's the algorithm. It's that simple. It's funny because when I saw that line I thought I would be pedantic and double check to see if you got the laplacian smoothing correct (because in practice it's never "that simple" for any numeric algorithm) and then realized that you don't understand how to implement Naive Bayes' at all. For starters you aren't using probability. If you want to put all of the words in each document together into a two dictionaries of counts, then for each word in the unclassified document you want to look at the product of the probability of those words appearing in the spam corpus vs the non-spam corpus. That probability is n_word/total_words the corpus. This is where you need to do some smoothing because if a word does not appear in the one of the corpora then you will get a probability of 0 for that class. Smoothing just adds 1 to the numerator and N_classes to the denominator. It is the equivalent of assuming a weakly informative uniform prior.
- jackhack 7y ago>That probability is n_word/total_words the corpus. Thanks. It's been 15 years since I implemented it. Guess I failed the interview.
- pokernaming 7y ago> That probability is n_word/total_words the corpus. In this case, wouldn't it actually be better to just drop the denominator, because it will be the same for both (spam & not spam).
- deleted 7y ago[deleted]
- tomrod 7y ago> For starters you aren't using probability. I never met a measure space I didn't like!
- windsignaling 7y agoImplementation of NB can be much more interesting than that: - try different models for the likelihood (the author mentioned Gaussian, Bernoulli, and Multinomial which are part of scikit but didn't bother to implement them) - understand how the "naive" assumption makes implementation easier but less expressive (author mentions it in words but pictures are critical here, e.g. the Gaussian case) - using log probabilities for numerical stability - reframing the model as a linear model
- random314 7y agoYour algorithm is incorrect.