Based on the difficulties I had implementing recommender systems, I think it boils down to two main problems: 1. Evaluating recommender systems is REALLY hard.

Based on the difficulties I had implementing recommender systems, I think it boils down to two main problems: 1. Evaluating recommender systems is REALLY hard. When it comes to evaluation, recommender systems are very different from other machine learning tasks: Let's assume the classic methodology of collecting a dataset of events, splitting it at a certain point in time to get a training and test set, and checking the precision and recall - If your recommender system works as expected, you will influence what your users do in the future, so your test set is probably not going to represent what your users would actually do. - Having a precision and recall of 1 is actually bad. That means your recommender system was perfect, but also useless (you only recommended stuff that the user was already going to pick anyway). One way to address this is to just use A/B tests and try to optimize some business metric (e.g. number of purchases). This is usually "good enough", but this will make your recommender focus on sales, not on user satisfaction. There are also some other metrics that can be used[1], but there are so many of them and some are not very practical to implement, so I guess that everyone just goes with the A/B testing approach. 2. Recommendations require an explanation A lot of recommender algorithms are black boxes. Sure, you can write "this was recommended based on users similar to you" when you use a collaborative filtering algorithm, but that doesn't help much. Recommendations without an explanation are not that useful. When a friend recommends you a movie, he'll also tell you "why". Otherwise, it's really hard to make users trust your recommendations (especially if the recommender system recommends something outside of the user's "comfort zone") I've noticed that a lot of sites have improved on this front though, and I do enjoy those recommendations a lot more. [1]: Herlocker et al., "Evaluating collaborative filtering recommender systems" http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.97.5270&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.97....