5 ms·
What reddit really needs is a ranking engine recommending users, submissions and comments to you based on the users you've friended and the commen
by rorrr 15y ago
What reddit really needs is a ranking engine
recommending users, submissions and comments
to you based on the users you've friended and
the comments and submissions you've upvoted.
Do you realize the insane amount of calculations that would take?
For each pageview you would have to find all stories in that subreddit, find all the authors, find all stories you ever voted on, find all the users you are friends with, and somehow rank all this shit.
It will cost millions of dollars a month in CPU time just to run something on the scale of reddit.
- ak217 15y agoSounds like a challenge :) First, this doesn't have to be for each pageview. Crudely, you want every story to have a vector in some giant hyperspace characterizing it, and every user to have a vector precomputed offline based on their previously shown affinities, and take dot products. This is not necessarily easy, or computationally very cheap, but the payoff can be pretty big. I believe LinkedIn runs something similar to this every night.
- rorrr 15y agoI'm not sure what you mean exactly, but hey, if you can pull it off, it might be successful. Just think of the number of comments up/down voted every millisecond, and that's just one the dimensions. The scale of that matrix would be enormous.
- overcyn 15y agohow is it much different from the ranking engines netflix and amazon use?
- noglorp 15y agoIn those cases, users purely receive recommendations for media/products. In this case, users receive recommendations for other users to follow as well.
- rorrr 15y agoYou can calculate the "weight" of the movie for a given user once. The number of movies is very low, compared to the number of stories on reddit. Whatever that guy is proposing is an insanely complex dynamic system. I'm not saying it's not doable, I'm just saying it's very expensive.
- a3camero 15y agoIt might be less complicated then you'd expect (at least one way of doing it): http://en.wikipedia.org/wiki/Collaborative_filtering http://en.wikipedia.org/wiki/Collaborative_filtering The core idea is linear regression. Here's Google's service: http://code.google.com/apis/predict/ http://code.google.com/apis/predict/. Here's a commercial API service: http://www.directededge.com/ http://www.directededge.com/.