4 ms·

Is it just me or does this sound oddly familiar to machine learning recommendation systems? ie: show an item based on the average of all the other items that e

• deleted 11y ago
  [deleted]
• casperc 11y ago
  Yeah, I was thinking of the same thing. It's not my field professionally though, so I am not sure what the relevant conclusion would be for machine learning though (or if the comparison holds at all).
  • BillinghamJ 11y ago
    I don't think that's quite correct. Recommendation engines normally work by taking the things you favor, then looking at who else favors those things, and deducting that you're all likely to have shared interests.
    • masklinn 11y ago
      Yeah I'd assume the simplest recommendation engines work on correlation. A recommendation engine working on average would be complete garbage, you'd be better off picking recommendation at random.
      • yeukhon 11y ago
        Quote emcq's response above: > If you're thinking about the cold start problem when you dont have any information about a user, yes it's possible that your overall statistics is a combination of many subpopulations that doesn't really fit anyone very accurately but there are ways around this as well. I'd say a better approach (but may have user experience penalty) would don't even decide for the user, ASK. For example, take Flipboard / Quora as an example, you are asked to choose some topics to follow at the beginning. Assume research/data show 80% of the users are software engineers and 90% of them always pick "technology" as a topic they want to follow, would you rather show technology as one of the top five in the list of topics to choose? There's actually a lot of experiments you can do from a simple selection/survey process. I personally can't stand at going through pages to find something relevant, but I am also surprised to find things I never thought would be interesting to follow if I weren't present the options at all / or earlier.
• emcq 11y ago
  Perhaps for a naive recommendation system, but quality ones don't do this. For example if you employ 1-nearest neighbors, you will actually return a sample that is guaranteed to reproduce another user's preferences. You could of course sample over more neighbors, smoothing results, at the cost of more lookups and at some point in the limit you do see reduced performance due to too many points becoming similar and having this average of everyone being bad for everyone sort of thing. Most ML approaches account for this and will only look at user/item clusters where the assumption is that a user can be sampled from a user cluster and an item can be sampled from an item cluster. Locally you may have error incorrectly capturing the clusters but in practice this works fairly well and you dont have a one size fit all model. If you're thinking about the cold start problem when you dont have any information about a user, yes it's possible that your overall statistics is a combination of many subpopulations that doesn't really fit anyone very accurately but there are ways around this as well. So in short - a naive ML recommender that perhaps you get out of a textbook would certainly face these issues but the are addressable in production with an "industry" grade recommender system and will lead to better performance. I don't have firm numbers but in A/B/C tests compared to other black boxes I've seen 20-30% better performance when you start to address some of these issues.