3 ms·
Define those three terms and I'll give you a cookie. I'm writing an app. In the app the user categorizes things and says whether he likes them or not. So he mi
by DanielBMarkham 17y ago
Define those three terms and I'll give you a cookie.
I'm writing an app. In the app the user categorizes things and says whether he likes them or not. So he might categorize one thing as "blue" and "heavy" and say he doesn't like it.
Now in the future when a blue or heavy thing comes along, I'd like the machine to try to predict whether he'll like it or not.
The thing that's making my head twist around a bit is that there are about 200 tags, and the user can choose 1 or a dozen. So it seems like I'm going to end up with this hugely complex nest of cartesian products from all the tags.
I think.
- mattmcknight 17y agoIt sounds like you want a recommender and maybe a bayesian classifier to determine if something is blue and/or heavy. I recommend reviewing Apache/Mahout Taste source code. http://lucene.apache.org/mahout/taste.html#overview http://lucene.apache.org/mahout/taste.html#overview There's even a book coming out on it. http://manning.com/owen/ http://manning.com/owen/
- physcab 17y agoWhat you can do is have a classifier for each tag. You can use a naive bayes technique, which is in PG's essay that I posted to above or you can check out the chapter in Collective Intelligence. You would need to know how much the tag appears for the user given all the other tags, how much the tag appears in the whole document, and then maybe an initial guess of what you think the tag's distribution is for every user. To do prediction, you would take that tag's posterior probabilities and multiply it by the user preferences. Then you can just recommend the ones that are the highest. You can do this with any language as it just revolves around counting and such. For extremely large datasets you can write a MapReduce script, but that probably isn't your usage case.
- DanielBMarkham 17y agoSo. I have 200 tags. For each tag in the system I need CountOfThisTagInAllDocs CountOfThisTagInTheCurrentDoc ---- Then I need ---- CountOfAllTagsInAllDocs and CountOfAllTagsInThisDoc ---- Then For The User ---- CountOfAllTagsInAllDocs CountOfThisTagInAllDocs ?? CountOfThisTagInAllFavoredDocs Which somehow gets me to likelihood that given the current tags the current doc will be favored by the user. It's like I have all the pieces and still can't see how the puzzle fits together. Sorry so slow here. It seems like for each user I need to have some sort of cross lookup for each tag. Either that or compute it all on the fly.
- physcab 17y ago> Now in the future when a blue or heavy thing comes along, I'd like the machine to try to predict whether he'll like it or not. There seems to be two separate steps. First you need to classify the thing a person rates. I'm guessing for each item, its not going to be a simple "blue" classification because this is user generated data so it might have for example 40% "blue" and 60% "heavy". Bayesian theory will do those classifications. p (tag given thing) = p (thing given tag) * p(guess tag) Then you just set a threshold and anything above it is the class for that item. Second, to do recommendations, you're taking into account the user preferences or whether they say they like something or not. You can do this a number of ways, but perhaps the one that is the easiest is just to calculate the similarities between items. For example you would find maybe "blue" and "red" things are highly similar, so you'd recommend "red" items to the user that loved the "blue" items. You can use Pearson Correlation to do that. Check out Collective Intelligence. It's got lots of (python) code and gives a great practical, introductory explanation to these algorithms.
- Eliezer 17y agoYeah, so, you got four choices here. First, you can use something off-the-shelf. Two, you can read up on Naive Bayes, a very simple way of solving the problem which sometimes works and sometimes doesn't. The reason it's called "naive" is that it assumes that each tag, for each user, makes the same contribution to the likelihood of the user liking or disliking something, independently of any other tags assigned. Like it says on the label, sometimes that works, sometimes it doesn't. Three, you can get a copy of "Elements of Statistical Learning" and read it, after which you will actually understand all this stuff. Four, if your problem is actually important, hire a professional machine learning person.
- DanielBMarkham 17y agoI'll take door #2, Naive Bayes, perhaps combined with a bit of cluster analysis. It's for a startup, so the job is to get something that partially works up and running. Worst-case I can tweak it later.
- Eliezer 17y agoOK. Naive Bayes is ridiculously simple. [EDIT: Oops, it looks like I myself managed to get it backward on the first write! Whaps self in forehead. Rewritten now.] Let's say your user likes 40 items and hates 60 items. So the prior odds for likes/hates are: 4:6 Then it turns out that, of the 40 items the user liked, 4 of these or 10% are tagged "green", and of the 60 items the user hated, 12 of these or 20% are marked "green". So the likelihood ratio for "green" is: 0.1:0.2 (Given that an item is hated, the user is twice as likely to tag it "green" than if the item is liked.) Then it turns out that of the 40 items the user liked, 8 or 1/5 are tagged "red", and of the 60 items the user hated, 3 or 1/20 are tagged "red. So the likelihood ratio for "red" is: 4:1 (Given that an item is liked, the user is four times as likely to tag it red.) Consider a new item tagged "red" and "green". The posterior odds equals the prior odds times the likelihood ratios: 16:12 57% probability of liking it, claims Naive Bayes. Sometimes it works, sometimes it doesn't. If multiplying the prior odds by the likelihood ratio doesn't seem justified as a way of getting the answer, see http://yudkowsky.net/rational/bayes http://yudkowsky.net/rational/bayes - there ought to be a shorter version of this, but I haven't gotten around to writing it.