4 ms·

I've built a recommender system for my movie database app "Coollector Movie Database". It's based on Collaborative filtering and it took me 2 years to implement

by prades 8y ago

I've built a recommender system for my movie database app "Coollector Movie Database". It's based on Collaborative filtering and it took me 2 years to implement. I built it from scratch and it's unique in several ways (for example, you can view the reliability of each recommendation). The technical difficulty is to crunch fast enough a huge quantity of data. I've had to apply all the optimizations that I could think of. https://www.coollector.com/help.html#recommendations https://www.coollector.com/help.html#recommendations

superasn 8y ago
That is a pretty amazing site! Can't download anything right now which makes me think It would make it a lot easier if instead of a downloadable software it was being offered as a saas site.
- prades 8y ago
  Thank you! It could not be free if it was a website because calculating the recommendations uses quite a lot of CPU. It's fine when run on each user's computer, but it would require expensive servers to make all the calculations for everyone, with a risk of congestion. There's a website named criticker.com which gives great movie recommendations, but you'll see that they have problems handling all the calculations. You'll literally see the recommendations being slowly generated, and updates are a problem when you rate more movies.
eggie5 8y ago
How would you generalize the method you are using?
- prades 8y ago
  As long as the users rate how much they like something, my method could easily work with songs or books or anything.
- prades 8y ago
  I don't like ML frameworks (Tensorflow, etc...), maybe it's because I haven't tried them. My understanding is that they're like a magic black box: you input some data, you adjust some settings, and you wish for the results to be good. Instead, I've taken a direct approach to the collaborative filtering problem, the difficulty being to correlate a huge amount of data. Some said that only quantum computers would one day be fast enough to solve the recommendation problem, until recently a student demonstrated that it could be solved with classical computers. https://www.quantamagazine.org/teenager-finds-classical-alternative-to-quantum-recommendation-algorithm-20180731/ https://www.quantamagazine.org/teenager-finds-classical-alte... This student's algorithm is quite different from mine, but I suppose that my algorithm is yet another example of solving the recommendation problem with classical computers.

by prades 8y ago

superasn 8y ago

That is a pretty amazing site! Can't download anything right now which makes me think It would make it a lot easier if instead of a downloadable software it was being offered as a saas site.

prades 8y ago
Thank you! It could not be free if it was a website because calculating the recommendations uses quite a lot of CPU. It's fine when run on each user's computer, but it would require expensive servers to make all the calculations for everyone, with a risk of congestion. There's a website named criticker.com which gives great movie recommendations, but you'll see that they have problems handling all the calculations. You'll literally see the recommendations being slowly generated, and updates are a problem when you rate more movies.

eggie5 8y ago

How would you generalize the method you are using?

prades 8y ago
As long as the users rate how much they like something, my method could easily work with songs or books or anything.
prades 8y ago
I don't like ML frameworks (Tensorflow, etc...), maybe it's because I haven't tried them. My understanding is that they're like a magic black box: you input some data, you adjust some settings, and you wish for the results to be good. Instead, I've taken a direct approach to the collaborative filtering problem, the difficulty being to correlate a huge amount of data. Some said that only quantum computers would one day be fast enough to solve the recommendation problem, until recently a student demonstrated that it could be solved with classical computers. https://www.quantamagazine.org/teenager-finds-classical-alternative-to-quantum-recommendation-algorithm-20180731/ https://www.quantamagazine.org/teenager-finds-classical-alte... This student's algorithm is quite different from mine, but I suppose that my algorithm is yet another example of solving the recommendation problem with classical computers.