4 ms·

It's a hard problem but Netflix's model represents the state of the art in machine learning for recommendations. Still scared of the singularity? :)

nicoburns 9y ago
Really? Because in terms of actual usefulness, I find youtube's suggestions to better...
- godelski 9y ago
  Youtube does better than Netflix for me, but they still suggest stuff that I have already watched. Sometimes stuff I literally watched an hour ago. And I watched one Joe Rogan episode and it will not stop suggesting it to me but fails to notify me on things that I watch every episode on; like 3Blue1Brown, Robert Miles, or Rare Earth.
- factsaresacred 9y ago
  Spotify is king IMO. Their 'discover weekly' playlist turns up gems every time. But that's because they dive into other user playlists that contain the same music you play. Pretty simple. I assume Youtube do somthing similar. If Netflix had playlists or a 'want to watch' feature I bet their recommendations would improve.
- scoggs 9y ago
  I tend to agree with you. The only thing I dislike about it is when I happen to open a link I'll randomly be sent or notice in an article that's something unlike what I normally enjoy or something I close right away and then for the next few days, or until I watch a bunch more of what I usually enjoy, the entire suggestion list is only things to do with that one random link.
  - zrobotics 9y ago
    I don't think it's possible on a phone, but in a browser you can remove videos from your watch history. Click the three dots next to the video for options, and select not interested. Google actually seems to take these into account, I had watched an Alex Jones video that was linked from elsewhere, and while it is good to not have too much of a bubble there's just certain things I don't need to see more of. Or, just use a separate browser. I typically only use opera for YouTube & anything where I don't mind Google tracking me, but if I open a YouTube link in Firefox I'm not logged in (and w/ opera's VPN enabled it doesn't appear to affect YouTube recommendations). I'm sure Google does correlate traffic between the two to some extent, but this seems like the only useful way to use operas integrated VPN.
dazc 9y ago
How hard can it be? Any film that I didn't watch all the way to end ought to be a strong signal that I didn't enjoy it. Not that I want to watch other movies just like it.
- danso 9y ago
  OTOH, that you even chose to watch a movie is a sign that you are interested in the genre. Just because I abruptly stopped watching “Star Trek IX”, doesn’t mean that I have abandoned the sci-fi genre or even Star Trek
  - dazc 9y ago
    I have a friend who shares my account since I stay with him when I am in the UK. It isn't worth the effort to have two profiles. So, he likes horror movies and will watch any horror regardless of any signal that it's going to be poor. You can look at the viewing history and see he rarely goes beyond 5 minutes of watching any of them. I watch Netflix regularly throughout the year, he watches in phases that last a week or two and then nothing at all for months at a time (for reasons that should seem obvious by now). As a non-horror movie aficionado, I can say with certainty that only 2% of horror movies are ever worth watching and only 50% of these are any good. As a consequence, my personal viewing history includes almost no movies in this genre. My favoured genre is drama and I normally watch all the way through. You should be able to guess by now that I should rarely be recommended horror movies but, alas, Netflix thinks otherwise. Btw. I also rate movies I watch - my friend doesn't.
    - danso 9y ago
      I’m confused — you honestly expect Netflix’s model to have figured out that your profile is actually 2 people based on what you believe to be regular and obvious cyclical patterns? I would have to imagine that this is a relative edge case for Netflix, and there is no obvious answer for what to do with someone who mostly likes drama but for some reason, goes on a periodic horror binge. I assume that Netflix’s model has the premise that profiles aren’t in fact, very easy to create. I have separate profiles for my parents, as well as a test profile to see what happens when a user only seems to like the “Human Centipede” trilogy.
acdha 9y ago
There’s a bug confound with the catalog: Netflix’s streaming catalog is much smaller than their DVD selection was and it changes as licensing deals expire. No model can make up for that entirely.
- emodendroket 9y ago
  I use the DVD one and I do find the suggestions are mostly pretty reasonable ones.
godelski 9y ago
Personally, I used to get substantially better results. It would suggest to me a movie I'd never seen before with a score above 90, I'd watch, and enjoy. Now I'm getting things like kids shows suggested to me. I double checked by history to make sure no one watched one on my account, but they keep popping up with high percentage. I also really like horror and get weird suggestions [1] for similar movies. After I watched Alien it suggested the Great British Baking show. I literally have 0 trust in the new ranking system. They push certain shows way too hard and TV shows I'm watching show up on page 3 of "continue watching". This is far from my previous stance of "Huh, movie I've never seen before and in a different language? It's 95% so yeah, I'll give it a go." And before someone says anything, I do vote up and down quite frequently. But can't help but notice that my suggestions were better when I had the more nuanced star system. [1] https://i.imgur.com/MOt3XlL.png https://i.imgur.com/MOt3XlL.png [1.1] How I'd rate these. Cars 3: not really interested. Liked the first though. Stitches: idk, doesn't look appealing. Teeth: Classic cult film but yeah... Big Mouth: I have ZERO interest in watching this, please stop suggesting. I have downvoted this! Waterboy: I like it, but far from 95%. I'll give it like an 80.
- dazc 9y ago
  Since switching from 1-5 start ratings to like or dislike it's gotten much worse. I honestly thought the 1-5 ratings were not useful since I either like or don't like movies, I am not interested in nuances. But, it's not working out as I expected.
- sdfmxiid1 9y ago
  The coverage of the Netflix ratings format switch has always been frustrating to me because what people are complaining about (you're not the only one; https://www.polygon.com/2017/4/7/15212718/netflix-rating-system-disapproval https://www.polygon.com/2017/4/7/15212718/netflix-rating-sys...) was predictable from the beginning. I do research in this area and it's fairly well established that when you go from something like five points to two points with ratings, you throw away tons of information. There's diminishing returns with numbers of points, but as you go lower you lose information. The "ratings don't matter because what you want is implicit signals from peoples' actual behavior" is also disingenous because the rating behavior is a behavior that's directly tied to the stimulus in question. Not saying that indirect behavioral correlates aren't useful, only that the rating is a very powerful, direct correlate that tends to be very specific. Going back to the topic of the thread, sure, all those Facebook likes are going to be useful in predicting how much you like a candidate, but you're sure as hell going to get a lot of information by just asking them "on a scale of 1 to 5, how much do you approve of X?"
pps43 9y ago
Yes, it's a hard problem, but no, Netflix recommendations are nowhere near state of the art. They were at the time of Netflix prize, but since then the field has advanced a lot, while Netflix only dumbed down their recommendation models.
cirgue 9y ago
There's a problem with all ML-based content recommendation, whether it's advertising, movies, whatever: the people who make the content have strong opinions about what should get shown to whom and when. If those people have negotiating leverage over your organization, your business team will compromise your recommendations to placate those people. That means that after you've build this beautiful model that minimizes whatever cost function you've chosen, there will be a bevy of business rules and content-specific score adjustments that will be overlaid. Over time, this shaggy, bad-assumption-laden system will dominate the user experience, and unless your management has a vested interest in maintaining the integrity of the recommendation system, your model will eventually be lost in a sea of human-generated noise. Not that I'm bitter about this or anything.

nicoburns 9y ago

Really? Because in terms of actual usefulness, I find youtube's suggestions to better...

godelski 9y ago
Youtube does better than Netflix for me, but they still suggest stuff that I have already watched. Sometimes stuff I literally watched an hour ago. And I watched one Joe Rogan episode and it will not stop suggesting it to me but fails to notify me on things that I watch every episode on; like 3Blue1Brown, Robert Miles, or Rare Earth.
factsaresacred 9y ago
Spotify is king IMO. Their 'discover weekly' playlist turns up gems every time. But that's because they dive into other user playlists that contain the same music you play. Pretty simple. I assume Youtube do somthing similar. If Netflix had playlists or a 'want to watch' feature I bet their recommendations would improve.
scoggs 9y ago
I tend to agree with you. The only thing I dislike about it is when I happen to open a link I'll randomly be sent or notice in an article that's something unlike what I normally enjoy or something I close right away and then for the next few days, or until I watch a bunch more of what I usually enjoy, the entire suggestion list is only things to do with that one random link.
- zrobotics 9y ago
  I don't think it's possible on a phone, but in a browser you can remove videos from your watch history. Click the three dots next to the video for options, and select not interested. Google actually seems to take these into account, I had watched an Alex Jones video that was linked from elsewhere, and while it is good to not have too much of a bubble there's just certain things I don't need to see more of. Or, just use a separate browser. I typically only use opera for YouTube & anything where I don't mind Google tracking me, but if I open a YouTube link in Firefox I'm not logged in (and w/ opera's VPN enabled it doesn't appear to affect YouTube recommendations). I'm sure Google does correlate traffic between the two to some extent, but this seems like the only useful way to use operas integrated VPN.

dazc 9y ago

How hard can it be? Any film that I didn't watch all the way to end ought to be a strong signal that I didn't enjoy it. Not that I want to watch other movies just like it.

danso 9y ago
OTOH, that you even chose to watch a movie is a sign that you are interested in the genre. Just because I abruptly stopped watching “Star Trek IX”, doesn’t mean that I have abandoned the sci-fi genre or even Star Trek
- dazc 9y ago
  I have a friend who shares my account since I stay with him when I am in the UK. It isn't worth the effort to have two profiles. So, he likes horror movies and will watch any horror regardless of any signal that it's going to be poor. You can look at the viewing history and see he rarely goes beyond 5 minutes of watching any of them. I watch Netflix regularly throughout the year, he watches in phases that last a week or two and then nothing at all for months at a time (for reasons that should seem obvious by now). As a non-horror movie aficionado, I can say with certainty that only 2% of horror movies are ever worth watching and only 50% of these are any good. As a consequence, my personal viewing history includes almost no movies in this genre. My favoured genre is drama and I normally watch all the way through. You should be able to guess by now that I should rarely be recommended horror movies but, alas, Netflix thinks otherwise. Btw. I also rate movies I watch - my friend doesn't.
  - danso 9y ago
    I’m confused — you honestly expect Netflix’s model to have figured out that your profile is actually 2 people based on what you believe to be regular and obvious cyclical patterns? I would have to imagine that this is a relative edge case for Netflix, and there is no obvious answer for what to do with someone who mostly likes drama but for some reason, goes on a periodic horror binge. I assume that Netflix’s model has the premise that profiles aren’t in fact, very easy to create. I have separate profiles for my parents, as well as a test profile to see what happens when a user only seems to like the “Human Centipede” trilogy.

acdha 9y ago

There’s a bug confound with the catalog: Netflix’s streaming catalog is much smaller than their DVD selection was and it changes as licensing deals expire. No model can make up for that entirely.

emodendroket 9y ago
I use the DVD one and I do find the suggestions are mostly pretty reasonable ones.

godelski 9y ago

Personally, I used to get substantially better results. It would suggest to me a movie I'd never seen before with a score above 90, I'd watch, and enjoy. Now I'm getting things like kids shows suggested to me. I double checked by history to make sure no one watched one on my account, but they keep popping up with high percentage. I also really like horror and get weird suggestions [1] for similar movies. After I watched Alien it suggested the Great British Baking show. I literally have 0 trust in the new ranking system. They push certain shows way too hard and TV shows I'm watching show up on page 3 of "continue watching". This is far from my previous stance of "Huh, movie I've never seen before and in a different language? It's 95% so yeah, I'll give it a go." And before someone says anything, I do vote up and down quite frequently. But can't help but notice that my suggestions were better when I had the more nuanced star system. [1] https://i.imgur.com/MOt3XlL.png https://i.imgur.com/MOt3XlL.png [1.1] How I'd rate these. Cars 3: not really interested. Liked the first though. Stitches: idk, doesn't look appealing. Teeth: Classic cult film but yeah... Big Mouth: I have ZERO interest in watching this, please stop suggesting. I have downvoted this! Waterboy: I like it, but far from 95%. I'll give it like an 80.

dazc 9y ago
Since switching from 1-5 start ratings to like or dislike it's gotten much worse. I honestly thought the 1-5 ratings were not useful since I either like or don't like movies, I am not interested in nuances. But, it's not working out as I expected.
sdfmxiid1 9y ago
The coverage of the Netflix ratings format switch has always been frustrating to me because what people are complaining about (you're not the only one; https://www.polygon.com/2017/4/7/15212718/netflix-rating-system-disapproval https://www.polygon.com/2017/4/7/15212718/netflix-rating-sys...) was predictable from the beginning. I do research in this area and it's fairly well established that when you go from something like five points to two points with ratings, you throw away tons of information. There's diminishing returns with numbers of points, but as you go lower you lose information. The "ratings don't matter because what you want is implicit signals from peoples' actual behavior" is also disingenous because the rating behavior is a behavior that's directly tied to the stimulus in question. Not saying that indirect behavioral correlates aren't useful, only that the rating is a very powerful, direct correlate that tends to be very specific. Going back to the topic of the thread, sure, all those Facebook likes are going to be useful in predicting how much you like a candidate, but you're sure as hell going to get a lot of information by just asking them "on a scale of 1 to 5, how much do you approve of X?"

pps43 9y ago

Yes, it's a hard problem, but no, Netflix recommendations are nowhere near state of the art. They were at the time of Netflix prize, but since then the field has advanced a lot, while Netflix only dumbed down their recommendation models.

cirgue 9y ago

There's a problem with all ML-based content recommendation, whether it's advertising, movies, whatever: the people who make the content have strong opinions about what should get shown to whom and when. If those people have negotiating leverage over your organization, your business team will compromise your recommendations to placate those people. That means that after you've build this beautiful model that minimizes whatever cost function you've chosen, there will be a bevy of business rules and content-specific score adjustments that will be overlaid. Over time, this shaggy, bad-assumption-laden system will dominate the user experience, and unless your management has a vested interest in maintaining the integrity of the recommendation system, your model will eventually be lost in a sea of human-generated noise. Not that I'm bitter about this or anything.