5 ms·
Netflix Recommendations: Beyond the 5 stars
- rmord 15y agoI remember seeing Xavier's talk at Strata 2012 (Santa Clara) on this, but not sure if it is up on the web.
- sigil 15y agoGreat read! Isn't this a bummer though? -- "On the topic of friends, we recently released our Facebook connect feature in 46 out of the 47 countries we operate – all but the US because of concerns with the VPPA law." (I wonder to what extent it also constrains recommendations based on user similarity.)
- stephengillie 15y agoWhat's this VPPA? http://en.wikipedia.org/wiki/Video_Privacy_Protection_Act http://en.wikipedia.org/wiki/Video_Privacy_Protection_Act The Video Privacy Protection Act (VPPA) was a bill passed by the United States Congress in 1988. Congress passed the VPPA after Robert Bork's video rental history was published during his Supreme Court nomination. It makes any "video tape service provider" that discloses rental information outside the ordinary course of business liable for up to $2500 in actual damages. (Covers DVD rentals too, and apparently paid-for streaming content.) Wow, never heard of that law before...
- waterlesscloud 15y agoIt sucks. Prevents a lot of interesting possibilities. I see that it was written with good intentions, but it's a great example of unintentional consequences when it comes to legislation.
- ralfd 15y agoInteresting story. Robert Bork said that the Constitution does not contain a general "right to privacy", which somehow is related to abortion (sometimes I think all american politics is related to Roe vs Wade) and then a Journalist just visited Bork's videothek and asked for his record. http://www.theamericanporch.com/bork4.htm http://www.theamericanporch.com/bork4.htm -------------------------------------------------------- When I arrived that day I told the assistant manager I was thinking of writing about Judge Bork’s video tastes. “Cool,” the assistant manager said. “I’ll look.” While I stewed in a sudden outbreak of conscience – what if Robert Bork only rented homosexual porn…or slasher flicks…or (the…horror…) Disney? – she went upstairs to eyeball the records, returning a little glum. “There sure are a lot of them,” she said. “Is it okay if I make a Xerox copy?” Next day, while visiting Jack Shafer, I asked if an article on Judge Bork’s video rentals would interest him. “You can’t do that, “ he said. “That’s illegal.” “No, it isn’t,” I said. “Judge Bork said so.” Okay, Shafer said. Write the story. -------------------------------------------------------- Crazy!
- bostonvaulter2 15y agoThis is the first I've ever heard about the VPPA law. I'll have go look it up later when I have time.
- DanBC 15y agoIt's baffling that the US lacks proper privacy laws, apart from one obscure and weird law about video rental. Compare to the EU laws: (http://en.wikipedia.org/wiki/Data_Protection_Directive http://en.wikipedia.org/wiki/Data_Protection_Directive)
- deleted 15y ago[deleted]
- underwater 15y ago"Of course, when we say “you”, we really mean everyone in your household." It's interesting that they've made a conscious decision to go that route. My wife and I share a streaming account which does not allow for separate instant queues. We have different tastes so I often see prompts to rate or recommendations to watch films and shows I dislike. If I'm honest in my rating it seems like I'll skew the recommendation engine away from shows my wife would like but if I'm not I'll end up seeing only recommendations i done like. It's a catch-22 situation and makes me distrust pretty much anything that is recommended.
- buro9 15y agoPlus if you ever split up... who gets the Netflix account and does it represent you? Personalisation is weird when they're not even dealing with people but households. Householdalisation?
- mark_l_watson 15y agoMy wife and I have separate accounts because we don't want our profiles for recommending new movies to influence each other. We try to find a common movie, and if nothing jumps out as being a good match for both of us, we simply sit next to each other with our laptops and use noise canceling headphones. We have been married for almost 30 years, so this is not as isolating as you might think.
- novakinblood 15y agoI also have kids and that really screws up my "suggested" queue. Hmmm, should I watch "Inspector Gadget" or "The Girl with the Dragon Tattoo" tonight?
- frossie 15y agoYeah - instead of putting their brainpower in finessing recommendation, I wish they would put it into improving the user experience. The web UI has hardly changed since they launched. It does not support notes (Joe recommended this movie), it does not support filtering of long queues (show me all the movies in my queue with predicted 4+ star rating that were added more than a year ago"), search is not great and lacks features (show me only PG movies), the site works poorly on the iPad, and yeah - a single streaming queue and the assumption that everybody in the household has the same tastes? Ai.
- Metapony 15y agoScrew netflix -- they stopped trying to even upsell you to dvd rentals if you have instant. So if you search for something, and you have instant, they no longer tell you that you could rent the dvd. It's just plain dumb, but then so was their last 'de-design'. Also the instant que is not in the order we added the items in, and seems to be arbitrary. It's just a terrible service with stale content. We'll be cancelling soon.
- DiabloD3 15y agoThe reason you are getting downvoted is because you phrased your comment badly. HOWEVER, I do agree with you. I habitually do not use the web UI and only use Netflix through my two Rokus. The web UI will tell you its on DVD, my Roku won't. Problem is, my Roku used to, so clearly the functionality exists, they just shut it off. I'd like it to be put back, honestly.
- noelwelsh 15y agoGood to see the Netflix prize paid off for Netflix. In pure terms of hourly rate, Netflix managed to get some of the smartest people in the world to work for less than chump change. In those terms alone it was a huge success, but the Netflix prize also pushed forward the field so really no-one was exploited. The same idea is been commercialised by Kaggle (http://www.kaggle.com/ http://www.kaggle.com/) but there are several issues. Of course there is less up-take as the idea is no longer novel and the prizes are less. More than that, I think people are realising that winner-takes-all sucks, and the winning entries tend to combine so many different techniques that, as Netflix found, putting them into production is difficult. There is some interesting work on a better model here: http://arxiv.org/abs/1111.2664 http://arxiv.org/abs/1111.2664
- wslh 15y agoAnd, if you look at the winners of the netflix prize they are people who were working in heavy research laboratories (Yahoo, AT&T, etc). Very few people can really compete.
- CWuestefeld 15y agoThe Netflix prize's criteria -- accurately rate a pile of movies -- is a red herring, though. It may help some, but the broad accuracy comes at the expense of an optimal algorithm for what's really important. What I want is a recommendation of what I'll probably like. It is absolutely irrelevant if mid-range movies sore a 2 or a 3. If Netflix can pick out a list of movies that I would rate a 5 (and maybe even 4), they've got the holy grail. Nothing else matters. So why optimize your algorithm to capture the 2s and 3s as well? For bonus point, it might be nice to be able to pick out the real dogs. If it could warn me that I'm about to rent a 1 or 2, that would be cool. But it doesn't matter if they can tell me which of 1 or 2 it is. The precision is irrelevant, just tell me I won't like it. (If I've said this once, I've said it a hundred times. But I guess I'll keep on like a broken record as long as Netflix keeps trumpeting what an achievement the Prize's algorithm was.)
- noelwelsh 15y agoYou're not alone. I don't know if you have a ML background, but it has been somewhat widely discussed in the community that the way Netflix scored the competition -- RMSE -- isn't the best way. E.g. http://hunch.net/?p=949 http://hunch.net/?p=949 and http://andrewgelman.com/2008/11/netflix_prize_s/ http://andrewgelman.com/2008/11/netflix_prize_s/ In the OPs article they mention they monitor if a movie is watched to completion, which gives them a much better metric to optimise. The other issue is that this is really a sequential decision making problem. Recommending a movie has an opportunity cost -- there are other movies you don't recommend -- and the recommendation is an ongoing process, so it is probably best to spend some time exploring the user's taste on the assumption this will let you make better recommendations in the future. Accounting for these issues is much harder in a competition format.
- WiseWeasel 15y agoI'm actually mocking up a recommendation engine at the moment, and this certainly gives me a lot to think about. For example, I should probably be planning for more future dimensions (ratings, metadata, etc.) so I can manipulate their interactions in a much more elegant way as we add new data, rather than trying to manually flatten everything out (combining metadata and ratings into a score for each item, then calculating compatibility) for every possible combination, which seems like a much less computationally expensive approach. I probably just need to always keep every scrap of data we get, then find the right ways to combine the data into a score with one routine, and compare the scores to calculate compatibility with another. The hard part is figuring out how the score should be calculated (I'll get to that in the morning). Seems like this rabbit hole goes pretty deep...
- pagetruif 15y agoFor people interested in the algorithms that won the Netflix Prize (including the Matrix Factorization and Restricted Boltzmann Machine approaches mentioned in the first section), there's a summary here: http://blog.echen.me/2011/10/24/winning-the-netflix-prize-a-summary/ http://blog.echen.me/2011/10/24/winning-the-netflix-prize-a-...
- mirsadm 15y agoI often wondered if this amount of effort is truly required and if it actually provides results people can relate to. I used Netflix for a while and I find the recommendations to be pretty average for me. Quite possibly it is because I don't tend to rate most movies I watch. I wonder how often that is the case for others though. More importantly I just don't think you can predict what I want to watch based on factors such as history, ratings, watch times etc. External factors such as mood, recommendation from friend (not on Netflix) and curiosity almost always play a large factor in choose the thing I watch next. These things are not known to Netflix. As a semi-educated guess I would say that simply providing a listing of popular movies based on ratings/watch times per genre as well as similarity would be enough for most people. It is easier to understand and how many people want to watch something just because it is popular anyway?
- mau 15y agoThat's why they show different rows from different sources. In my country Netflix is not available yet so I'm not sure about what I'm going to say, but I'm quite confident that one of those rows is just the top-rated/popular movies of the site.
- speedracr 15y agoTo me, the Netflix price is as much about marketing as it is about the quality of the algorithm: Netflix was ahead of the pack (i.e. Blockbuster) even before, but after spreading the word that it takes rocket scientists to even slightly improve the existing algorithm, who would even think twice about testing a DVD/ streaming provider other than Netflix?
- flipside 15y ago"Another important element in Netflix’ personalization is awareness. We want members to be aware of how we are adapting to their tastes. This not only promotes trust in the system, but encourages members to give feedback that will result in better recommendations. A different way of promoting trust with the personalization component is to provide explanations as to why we decide to recommend a given movie or show. We are not recommending it because it suits our business needs, but because it matches the information we have from you: your explicit taste preferences and ratings, your viewing history, or even your friends’ recommendations." Well, at least they're trying to demonstrate awareness and explain their recommendations, even if they come up short. Awareness? Am I missing something here because most people I've talked to (several hundred for the record) feel like Netflix recommendations are hit or miss, a black box that plateaus after a while. If anyone can explain to me how Netflix conveys their awareness I'd love to hear it. Speaking of explanations, sure, showing that I'm being recommended a movie because of some other movies is sometimes helpful, but the rest of the time it just reveals how poorly they understand me. It's can be pretty obvious that the algorithms have no clue why I actually like certain movies, after all, how could they? There are tons of things to like/dislike about any given movie, but when that gets boiled down to 5-stars all that context gets lost. Diminishing returns and noise will keep them from really understanding my tastes with their current system. Why am I ranting? Because I want to be understood but Netflix and other personalization services still feel so damn impersonal. It's frustrating. Anyways, we're working on a solution, sign up for our alpha at http://tagbax.com http://tagbax.com or comment/vote to tell me how right/wrong I am. Thanks for reading.
- mgkimsal 15y agoIt seems to me they'd be able to get a lot more metadata for their recommendation engine if they worked on allowing multiple personas per household account. Analyzing the viewing behaviour of people within the same household would give them a treasure trove of more data to make better recommendations for other people in similar households. "You watched and liked XYZ, your 38 year old wife hates ABC, and you've got 3 daughters 8/10/12? They'll probably like these shows ...." It seems they missed out on years of awesome data by not allowing for segmenting by persona from the early days.
- ralfd 15y agoBut making a multiuser-interface complicates a design and ease of use tremendously. Think about password protection (and recovery). And you don't want to administer another computer system, you just want to watch TV.
- mgkimsal 15y agoWouldn't necessarily need password protection. There's an assumption that you're all living in the same house. But a 'who are you?' at the top, letting you type your name in, and 'switch to', shouldn't be too hard. I agree, it does complicate things some, but consider A) not every account would need it. B) not everyone offered it would use it C) the accounts that used it would probably be a bit savvier to start with D) they'd get a lot of extra valuable data, not least of which how people use that feature, and could make it easier to use over time.
- jonathanwallace 15y agoI wonder if grouping everyone in the same household is a simple way to improve diversity and create an affordance in the mind of the user. "Ah, there's my kids Dora selections again.. where's my recommendations?"
- dailyllama 15y agoThey actually have some support for this (or at least they used to, and grandfathered some of us in). Unfortunately, it's limited to the website, and doesn't carry over to Roku/AppleTV/etc.
- ck2 15y agoInstantwatcher and streamingsoon are far better than any netflix recommendations in my experience.
- damon_c 15y agoIf you want to know more about the technical details... http://www.machinelearning.org/proceedings/icml2007/papers/407.pdf http://www.machinelearning.org/proceedings/icml2007/papers/4...
- vph 15y agorecommendation is a good concept if you have things to recommend. Netflix current catalog is seriously lacking. We were in the middle of watching the TV show Spartacus when it was taken off (due to licensing problems).
- vph 15y agorecommendation is a good concept if you have things to recommend. Netflix current catalog is seriously lacking. We were in the middle of watching the TV show Spartacus when it was taken off (due to licensing problems).
- mark_l_watson 15y agoI wish that they used a 10 star metric. Often I'll want to give a 3.5 or 4.5 rating - not possible. They need to use it a finer scale so customers get the impression, at least, of finer control of how they rate movies.
- StuffMaster 15y agoI wish there was at least one more - "OK". As in, I didn't "like" it, but I don't regret watching it.