2 ms·

What you can do is have a classifier for each tag. You can use a naive bayes technique, which is in PG's essay that I posted to above or you can check out the

by physcab 17y ago

What you can do is have a classifier for each tag. You can use a naive bayes technique, which is in PG's essay that I posted to above or you can check out the chapter in Collective Intelligence. You would need to know how much the tag appears for the user given all the other tags, how much the tag appears in the whole document, and then maybe an initial guess of what you think the tag's distribution is for every user. To do prediction, you would take that tag's posterior probabilities and multiply it by the user preferences. Then you can just recommend the ones that are the highest. You can do this with any language as it just revolves around counting and such. For extremely large datasets you can write a MapReduce script, but that probably isn't your usage case.

DanielBMarkham 17y ago
So. I have 200 tags. For each tag in the system I need CountOfThisTagInAllDocs CountOfThisTagInTheCurrentDoc ---- Then I need ---- CountOfAllTagsInAllDocs and CountOfAllTagsInThisDoc ---- Then For The User ---- CountOfAllTagsInAllDocs CountOfThisTagInAllDocs ?? CountOfThisTagInAllFavoredDocs Which somehow gets me to likelihood that given the current tags the current doc will be favored by the user. It's like I have all the pieces and still can't see how the puzzle fits together. Sorry so slow here. It seems like for each user I need to have some sort of cross lookup for each tag. Either that or compute it all on the fly.
- physcab 17y ago
  > Now in the future when a blue or heavy thing comes along, I'd like the machine to try to predict whether he'll like it or not. There seems to be two separate steps. First you need to classify the thing a person rates. I'm guessing for each item, its not going to be a simple "blue" classification because this is user generated data so it might have for example 40% "blue" and 60% "heavy". Bayesian theory will do those classifications. p (tag given thing) = p (thing given tag) * p(guess tag) Then you just set a threshold and anything above it is the class for that item. Second, to do recommendations, you're taking into account the user preferences or whether they say they like something or not. You can do this a number of ways, but perhaps the one that is the easiest is just to calculate the similarities between items. For example you would find maybe "blue" and "red" things are highly similar, so you'd recommend "red" items to the user that loved the "blue" items. You can use Pearson Correlation to do that. Check out Collective Intelligence. It's got lots of (python) code and gives a great practical, introductory explanation to these algorithms.

by physcab 17y ago

DanielBMarkham 17y ago

So. I have 200 tags. For each tag in the system I need CountOfThisTagInAllDocs CountOfThisTagInTheCurrentDoc ---- Then I need ---- CountOfAllTagsInAllDocs and CountOfAllTagsInThisDoc ---- Then For The User ---- CountOfAllTagsInAllDocs CountOfThisTagInAllDocs ?? CountOfThisTagInAllFavoredDocs Which somehow gets me to likelihood that given the current tags the current doc will be favored by the user. It's like I have all the pieces and still can't see how the puzzle fits together. Sorry so slow here. It seems like for each user I need to have some sort of cross lookup for each tag. Either that or compute it all on the fly.

physcab 17y ago
> Now in the future when a blue or heavy thing comes along, I'd like the machine to try to predict whether he'll like it or not. There seems to be two separate steps. First you need to classify the thing a person rates. I'm guessing for each item, its not going to be a simple "blue" classification because this is user generated data so it might have for example 40% "blue" and 60% "heavy". Bayesian theory will do those classifications. p (tag given thing) = p (thing given tag) * p(guess tag) Then you just set a threshold and anything above it is the class for that item. Second, to do recommendations, you're taking into account the user preferences or whether they say they like something or not. You can do this a number of ways, but perhaps the one that is the easiest is just to calculate the similarities between items. For example you would find maybe "blue" and "red" things are highly similar, so you'd recommend "red" items to the user that loved the "blue" items. You can use Pearson Correlation to do that. Check out Collective Intelligence. It's got lots of (python) code and gives a great practical, introductory explanation to these algorithms.