6 ms·
Netflix Progress Prize Awarded
- madmanslitany 18y agoI started working on this about a month and a half ago. My RMSE is pathetic thus far(just started tweaking a kNN with Pearson correlation as a distance metric), but I don't really care. It's a great way to brush up on a lot of CompSci concepts at once you may be rusty on. Besides obviously machine learning, when you're trying to process 2 gb of training files on your personal computer, O(n) time/space complexity REALLY starts to matter, as does choice of implementation language...using mostly Python with C extensions for the heavy math right now...
- FiReaNG3L 18y agoAs per the prize rule, they are under the obligation to describe and publish their mode (as they did last time). I really wish someone would release an open source recommender package, coded in C, with PHP bindings, based on the latest / greatest algorithms, or any algorithm (just plain SVD if anything). Interesting links (from the winning team(s) websites about their latest research) : http://public.research.att.com/~volinsky/netflix/kdd08koren.pdf http://public.research.att.com/~volinsky/netflix/kdd08koren.... http://www.commendo.at/index.php?lang=0&_0=2&_1=3 http://www.commendo.at/index.php?lang=0&_0=2&_1=3
- icefox 18y agoWell I am not going to do everything for you, but here is something to start with: http://benjamin-meyer.blogspot.com/2006/10/netflix-prize-contest.html http://benjamin-meyer.blogspot.com/2006/10/netflix-prize-con... It is a little framework I put together to play with the netflix prize framework. I implemented SVD as one of the examples. On a side note I did think about the idea of putting together a small company that would sell recommendation systems to companies for their website. The value would not be in the algorithm, but in the ability to integrate with their software.
- FiReaNG3L 18y agoVery nice, I saw many Netflix Prize derived frameworks but I somehow missed this one. Did you test RMSE of your SVD on Netflix Dataset?
- icefox 18y agoNope, it is just a stock SVD, it wont win any awards, just get the job done and be an example. There are plenty of people who have tweaked SVD up and down so I didn't spent time doing it too.
- jmtame 18y ago2 years in and they still haven't given out the prize money. How's that crowd sourcing working out for you NetFlix?
- boredguy8 18y agoA 9% improvement represents a substantial increase in quality of recommendations. If I remember correctly, a 10% improvement meant the difference between a complete star rating, on average. (There was a fairly substantial write-up about what the 10% meant last year.) I'd say for the people-hours put in to the project, they've gotten more than their money's worth.
- jmtame 18y agoOh I bet they have. Have you calculated the number of hours the 35,000 teams put in relative to the amount of pay they received? I bet they've gotten more than their money's worth.
- mhb 18y agoAt what rate should we value their hours? Do you charge more or less for doing something you like? Have you considered that voluntary trade is not zero sum?
- jmtame 18y agoVoluntary as in they don't get paid? Because I'm sure that every single person has no intention of receiving any of the prize money. None whatsoever, it's purely voluntary, and nobody deserves any money for making Netflix more profitable. Wow you guys really want to bring it out of me on this thread. That's alright though, I dig everyone's energy (and I don't mind taking a bit of a karma beating over this either). Actually, at the time of writing this, as a result of my discussion, other people have been awarded 28 karma points (where I have suffered a loss of a few). Outside discussion accounts for 35 karma points. So relative to the overall karma, I'm helping a lot of people out by being the grumpy, grouchy, mean old Mr. Scrooge in defense of the researcher. I do it for you guys! :D
- aswanson 18y agoIt may be simply mathematically impossible to get to 10 percent given the observable variables. Maybe netflix knows this and just gets synthesized research on the cheap.
- pg 18y agoI'd guess the machine learning guys by this point understand the data set at least as well as Netflix does.
- aswanson 18y agoWouldn't argue with that, but they may have run up against a bound, multiplied it by a constant, and said to themselves "highly unlikely that anyone can beat this, so lets put it out there and see if someone out there can beat it." So far no one has and they have gotten a ton of research on the cheap, like the current team's for 50 k USD. Smart.
- jderick 18y agoThe machine learning guys have only a subset of the full dataset to work with. They have to send their algorithms to Netflix for testing on the full dataset. So it may be possible Netflix knows something they don't. It does seem quite strange how close and yet how far that 10% mark has been.
- Retric 18y agoBecause they are testing it vs a fixed data set teams can gain information about that data set based on their submissions. Which is why several teams mix and mach several different approaches. They can increase the weight of each approach over time and game the system. Edit: I am not saying they are intentionally cheating rather it's one of the reasons that approach works.
- pg 18y agoHmm. What could Netflix know, though? Is it even possible to quantify how hard it is to improve a recommendation algorithm by a certain amount?
- staunch 18y agoI keep hoping someone will swoop down with a crazy solution and snatch the prize from the people trying to solve it conventionally. So much more romantic than this slog!
- tocomment 18y agoI'd love to win just so I could settle arguments by saying "anyone who has won a million dollar data mining competition raise your hand"
- deleted 18y ago[deleted]