6 ms·

I'm the dev that built Netflix's autoplay of the next episode. We built it first on the web player because it is easy to A/B test new features there. We called

I'm the dev that built Netflix's autoplay of the next episode. We built it first on the web player because it is easy to A/B test new features there. We called it "post-play" at the time. When I worked there the product team at Netflix had two KPIs all new features were tested against: hours watched and retention. We would come up with all sorts of ideas to try out, and release them to small user populations of about 100,000 or so. It was great because you didn't have to debate much about whether a new feature was a good idea or not, you just built it and tested it. If the feature didn't increase hours watched or retention in a statistically significant way, the feature was removed. Autoplay massively increased hours watched. I can't remember the exact numbers, but it was by far the biggest increase in the hours watched KPI of any feature we ever tested. There was some skepticism about whether the number was inflated by Netflix continuing to play when the user left the room. As part of the autoplay test, we tested how long the countdown should be between episodes. 5 seconds, 10 seconds or 15 seconds. 10 seconds caused the biggest increase in hours watched. We thought that it gave people time to digest what they had just watched, but wasn't too fast (5 seconds) where it became jarring. Interestingly, Netflix recently changed the countdown between episodes to 5 seconds. That means they tested it out and found that people watch more if with a shorter countdown. This didn't use to be the case. Netflix user have become conditioned to expect autoplay. So yes, Netflix wants you to spend more hours watching Netflix and the product team is scientifically engineering the product to make it more addictive. But...the product team at Doritos does the same thing.

Shorel 7y ago
The time in seconds should be also user definable. Too much of A/B testing is good only for people who never want to change defaults.
digitalsushi 7y ago
I've never been able to understand why netflix benefits from autoplaying - you're not selling anything, you're just delivering content - so why does autoplaying, which presumably has some measurable, additional cost for bandwidth, end up benefiting the company? Does the perceived value of netflix increase when autoplay is enabled? I really want to understand this, I've been strung up on this for a while.
- orev 7y ago
  When you start measuring something, that thing tends to become what you optimize for, regardless of if it actually makes sense. Hours spent watching is some kind of indicator of whether people enjoy using the service, and is also a nice clean metric to be able to report on.
- mbreese 7y ago
  When you have to choose between "watch the next episode" or "watch something else", Netflix benefits when the default is "watch the next episode". When you have to consciously choose to watch the next episode, you can also make the choice to turn off Netflix and do something else. The more you watch Netflix, the more likely you are to keep paying for it. When you actively choose to do something else, the less likely you are to keep paying for Netflix. It's not the perceived value, it's the choice. When you have to make the choice, you could choose to spend more time with something else (like a video game, a Hulu show, etc...). If you choose to do that, you might get sucked into that game or show and decide to not come back to Netflix. Netflix would rather spend the few cents it costs to stream you a new video (which is probably already cached at your ISP) than make your choice to do something else explicit. That said, I think it's a great feature that makes it easy to binge through a series. Its beneficial to both parties in that way -- I get to not move from my couch and they get to send me more entertainment (and keep me paying them). Also -- the more you watch of a series, the more you'll talk about it to your friends. And the more likely they are to watch it. If you get stuck on Episode 2 of 10, you're not as likely to talk about it. But if you keep going to episode 3, 4, 5, then you are more likely to finish the series. And if you finish, you'll probably talk about it, and then the cycle continues...
- jerf 7y ago
  The probability of you cancelling Netflix in a given month is inversely proportional to the amount of Netflix you or your household watched [1]. The expense of a marginal delivery of a particular episode's worth of resources is a great deal less for Netflix than the revenue of your subscription. [1]: To a first approximation, anyhow. I bet there's a bump at zero, where people just entirely forget Netflix exists but they are still subscribed. If someone watches a single thing a month they're probably more likely to eventually realize they're not using it enough.
deleted 7y ago
[deleted]
deleted 7y ago
[deleted]
jsilence 7y ago
Yeah, ever so often when I want to continue watching the current series I realize that autowatch made me watch the whole season while I was asleep on the couch. Really wish autoplay would stop every three or four episodes.
- vtkacenko 7y ago
  It does stop for me.
- RandallBrown 7y ago
  It does stop after 3 or 4 episodes.
- clinta 7y ago
  I've been annoyed with it on series that have unique end credit songs per episode. They put that song in there and I'd like to hear it, but autoplay cuts it off and just starts the next episode.
orev 7y ago
Serious question: how do you feel about having been involved in that feature, knowing that you were creating an addictive product and trying to get people more hooked on it? You might try to say that people have free will and can do what they want, but you know that the reason it’s done is exactly because it’s addictive.
- rsweeney21 7y ago
  I left Netflix mostly because of this. I've always believed that too much TV was bad for you and I didn't like thinking about how to get people to spend more time on the couch. It doesn't feel great.
  - RangerScience 7y ago
    Well, if it makes you feel a bit better, I love autoplay because most of the time I'm watching Netflix I'm cooking/cleaning/etc. I definitely clean less when there's nothing left that I want to watch, and autoplay means I don't have to break that flow.
  - Rooster61 7y ago
    To me, Netflix is probably the most benign out of FAANG. I wonder how many people there are at the other companies wringing their hands over the effects of their work. Facebook in particular, what with the exacerbation of vitriol in the country, as well as potential for foreign actor misuse. I respect you for leaving. I decided a long time ago that the boost in resume for working for one of these companies really wouldn't be worth the moral baggage (Netflix being a possible exception for the reason I stated above).
  - ghostbrainalpha 7y ago
    Did being responsible for a feature so important to the growth of Netflix result in a reward or promotion outside of your expected compensation? And as a father with 3 young children, you don't know how much I appreciate what you built. Not having to click next between episodes of "word party" for my toddlers is sanity saving. Also to the people without kids who will be judging this comment, you may not realize that many children's programs are not always 30 minutes long. Episodes of Peppa Pig can be ~ 8 minutes. And on iTunes you cannot make a playlist or autoplay episodes, making Peppa Pig during car rides on the iPad an experience akin to visiting Hell.
djaychela 7y ago
>But...the product team at Doritos does the same thing. True, but they can't make the end of the bag of Doritos lead you to the counter to get another one in 5 seconds. (yet!)
- rsweeney21 7y ago
  Yeah, but they make HUGE bags of Doritos now! :D
saurik 7y ago
> When I worked there the product team at Netflix had two KPIs all new features were tested against: hours watched and retention. So, I totally understand retention: that one makes sense to me, as that's revenue for Netflix; if I'm more likely to renew my account, then Netflix makes more money. But... I honestly don't understand why Netflix is optimizing for hours watched, and in fact I had assumed it would be the opposite (leading to Netflix actually making it kind of annoyingly difficult to watch the full credits on movies, for example). > But...the product team at Doritos does the same thing. This makes sense to me: the more Doritos I eat, the more money Doritos makes. This isn't true of Netflix: every time I watch something, it costs Netflix money. I would, naively, expect that what Netflix would want to optimize for is some weird balance of retention (at high priority) and least hours watched (at a much lower priority). > So yes, Netflix wants you to spend more hours watching Netflix and the product team is scientifically engineering the product to make it more addictive. I also appreciate that the more hours I watch of Netflix, the more I might be "addicted" to doing so... but the reason I'd expect Netflix to care about that is because it means higher retention, which is already being directly measured. Any other aspects of my addiction don't seem to benefit Netflix: it costs Netflix money every time I stream something from them... ...and, it would seem to me crucially, it means that Netflix is "on the hook" for more and more content. If, on a relatively short-term basis (such as during an experiment), my measured retention is the same, but my hours watched is higher, then it means I'm burning through content I like to watch at a faster rate, and so Netflix needs to spend more money to rotate their catalog and create more originals. It just seems so weird that Netflix would be optimizing for "hours watched", and I'd love to learn from someone more on the inside who was involved in that why they would do that. I understand why YouTube and Facebook optimize for "engagement" (even though I also think it is abhorrent) as 1) their revenue is based on advertisements and 2) they spread using network effects; neither of these are true for Netflix.
- rsweeney21 7y ago
  Hours watched is a leading indicator of retention. The more people watch, the more likely they are to remain a customer.
  - 7y ago
littlestymaar 7y ago
> But...the product team at Doritos does the same thing. This is a good comparison because we know it's a public health disaster. Unfortunately he junk food industry was powerful enough to repeal this kind of rules in their domain, and I'd actually be surprised if this one was enacted.
dantheman 7y ago
I love this feature :) So thank you.
Rooster61 7y ago
You wouldn't happen to know the guy/gal that implemented the automatic preview when hovering over a selection, would you? Curious as to how they felt about it, and how much that really affected retention (it bugs the everloving shit out of me, I try to use my old smart tv with an outdated version as it lacks this "feature").
- squidi 7y ago
  I mute my TV when navigating through Netflix because of auto previews. I bet that doesn’t show up in their analytics!
  - veidr 7y ago
    I do that too! And so does every person that I personally know well enough to have seen them navigate Netflix (which admittedly is only like 5).
jancsika 7y ago
> But...the product team at Doritos does the same thing. I get awfully suspicious any time someone so clearly on today's side of the digital divide feels the need to hop the fence back to yesterday to elucidate an ethical argument. (And esp. on HN where you'd be hopping away from the place we're all standing.) What's the threshold of addiction beyond which a digital streaming service becomes unethical in your opinion? How about dangerous? I ask because you unequivocally refer to the product you worked on as "addictive." I can't imagine working on such a product without a clear sense of what the boundaries are.
cableshaft 7y ago
If I'm watching Netflix, I am fine with autoplay. Youtube too, while we're at it. I.e. if I'm there specifically to watch videos, and you want to autoplay the next video after I'm done, that's fine. What I'm not fine with is places that autoplay that have no business doing it, especially if they do that specifically to brag about how many autoplays it has (i.e. CNN). CNN in particular is my main annoyance, as when I go there I just want to read their news article, NOT watch video, and yet I have to scramble to hit the pause button on their video the moment I launch one of their pages (even by accident, because I'm just clicking a google link or a Hacker News link). So long story short; Netflix and Youtube or sites that are just about serving video (and nothing else), autoplay to the next video is fine and expected. EVERYTHING ELSE, DON'T AUTOPLAY.
- wolco 7y ago
  Autoplaying a series makes sense. Autoplaying videos as one scrolls down the cnn website causing the site to take hours to load is foolish.
jcranberry 7y ago
Funny, for my account the timer is 20 seconds. I wonder if it differs per user.
tinus_hn 7y ago
At least I can turn this ‘feature’ off, unlike the obnoxious auto playing trailers.

Shorel 7y ago

The time in seconds should be also user definable. Too much of A/B testing is good only for people who never want to change defaults.

digitalsushi 7y ago

I've never been able to understand why netflix benefits from autoplaying - you're not selling anything, you're just delivering content - so why does autoplaying, which presumably has some measurable, additional cost for bandwidth, end up benefiting the company? Does the perceived value of netflix increase when autoplay is enabled? I really want to understand this, I've been strung up on this for a while.

orev 7y ago
When you start measuring something, that thing tends to become what you optimize for, regardless of if it actually makes sense. Hours spent watching is some kind of indicator of whether people enjoy using the service, and is also a nice clean metric to be able to report on.
mbreese 7y ago
When you have to choose between "watch the next episode" or "watch something else", Netflix benefits when the default is "watch the next episode". When you have to consciously choose to watch the next episode, you can also make the choice to turn off Netflix and do something else. The more you watch Netflix, the more likely you are to keep paying for it. When you actively choose to do something else, the less likely you are to keep paying for Netflix. It's not the perceived value, it's the choice. When you have to make the choice, you could choose to spend more time with something else (like a video game, a Hulu show, etc...). If you choose to do that, you might get sucked into that game or show and decide to not come back to Netflix. Netflix would rather spend the few cents it costs to stream you a new video (which is probably already cached at your ISP) than make your choice to do something else explicit. That said, I think it's a great feature that makes it easy to binge through a series. Its beneficial to both parties in that way -- I get to not move from my couch and they get to send me more entertainment (and keep me paying them). Also -- the more you watch of a series, the more you'll talk about it to your friends. And the more likely they are to watch it. If you get stuck on Episode 2 of 10, you're not as likely to talk about it. But if you keep going to episode 3, 4, 5, then you are more likely to finish the series. And if you finish, you'll probably talk about it, and then the cycle continues...
jerf 7y ago
The probability of you cancelling Netflix in a given month is inversely proportional to the amount of Netflix you or your household watched [1]. The expense of a marginal delivery of a particular episode's worth of resources is a great deal less for Netflix than the revenue of your subscription. [1]: To a first approximation, anyhow. I bet there's a bump at zero, where people just entirely forget Netflix exists but they are still subscribed. If someone watches a single thing a month they're probably more likely to eventually realize they're not using it enough.

deleted 7y ago

[deleted]

deleted 7y ago

[deleted]

jsilence 7y ago

Yeah, ever so often when I want to continue watching the current series I realize that autowatch made me watch the whole season while I was asleep on the couch. Really wish autoplay would stop every three or four episodes.

vtkacenko 7y ago
It does stop for me.
RandallBrown 7y ago
It does stop after 3 or 4 episodes.
clinta 7y ago
I've been annoyed with it on series that have unique end credit songs per episode. They put that song in there and I'd like to hear it, but autoplay cuts it off and just starts the next episode.

orev 7y ago

Serious question: how do you feel about having been involved in that feature, knowing that you were creating an addictive product and trying to get people more hooked on it? You might try to say that people have free will and can do what they want, but you know that the reason it’s done is exactly because it’s addictive.

rsweeney21 7y ago
I left Netflix mostly because of this. I've always believed that too much TV was bad for you and I didn't like thinking about how to get people to spend more time on the couch. It doesn't feel great.
- RangerScience 7y ago
  Well, if it makes you feel a bit better, I love autoplay because most of the time I'm watching Netflix I'm cooking/cleaning/etc. I definitely clean less when there's nothing left that I want to watch, and autoplay means I don't have to break that flow.
- Rooster61 7y ago
  To me, Netflix is probably the most benign out of FAANG. I wonder how many people there are at the other companies wringing their hands over the effects of their work. Facebook in particular, what with the exacerbation of vitriol in the country, as well as potential for foreign actor misuse. I respect you for leaving. I decided a long time ago that the boost in resume for working for one of these companies really wouldn't be worth the moral baggage (Netflix being a possible exception for the reason I stated above).
- ghostbrainalpha 7y ago
  Did being responsible for a feature so important to the growth of Netflix result in a reward or promotion outside of your expected compensation? And as a father with 3 young children, you don't know how much I appreciate what you built. Not having to click next between episodes of "word party" for my toddlers is sanity saving. Also to the people without kids who will be judging this comment, you may not realize that many children's programs are not always 30 minutes long. Episodes of Peppa Pig can be ~ 8 minutes. And on iTunes you cannot make a playlist or autoplay episodes, making Peppa Pig during car rides on the iPad an experience akin to visiting Hell.

djaychela 7y ago

>But...the product team at Doritos does the same thing. True, but they can't make the end of the bag of Doritos lead you to the counter to get another one in 5 seconds. (yet!)

rsweeney21 7y ago
Yeah, but they make HUGE bags of Doritos now! :D

saurik 7y ago

> When I worked there the product team at Netflix had two KPIs all new features were tested against: hours watched and retention. So, I totally understand retention: that one makes sense to me, as that's revenue for Netflix; if I'm more likely to renew my account, then Netflix makes more money. But... I honestly don't understand why Netflix is optimizing for hours watched, and in fact I had assumed it would be the opposite (leading to Netflix actually making it kind of annoyingly difficult to watch the full credits on movies, for example). > But...the product team at Doritos does the same thing. This makes sense to me: the more Doritos I eat, the more money Doritos makes. This isn't true of Netflix: every time I watch something, it costs Netflix money. I would, naively, expect that what Netflix would want to optimize for is some weird balance of retention (at high priority) and least hours watched (at a much lower priority). > So yes, Netflix wants you to spend more hours watching Netflix and the product team is scientifically engineering the product to make it more addictive. I also appreciate that the more hours I watch of Netflix, the more I might be "addicted" to doing so... but the reason I'd expect Netflix to care about that is because it means higher retention, which is already being directly measured. Any other aspects of my addiction don't seem to benefit Netflix: it costs Netflix money every time I stream something from them... ...and, it would seem to me crucially, it means that Netflix is "on the hook" for more and more content. If, on a relatively short-term basis (such as during an experiment), my measured retention is the same, but my hours watched is higher, then it means I'm burning through content I like to watch at a faster rate, and so Netflix needs to spend more money to rotate their catalog and create more originals. It just seems so weird that Netflix would be optimizing for "hours watched", and I'd love to learn from someone more on the inside who was involved in that why they would do that. I understand why YouTube and Facebook optimize for "engagement" (even though I also think it is abhorrent) as 1) their revenue is based on advertisements and 2) they spread using network effects; neither of these are true for Netflix.

rsweeney21 7y ago
Hours watched is a leading indicator of retention. The more people watch, the more likely they are to remain a customer.
- 7y ago

littlestymaar 7y ago

> But...the product team at Doritos does the same thing. This is a good comparison because we know it's a public health disaster. Unfortunately he junk food industry was powerful enough to repeal this kind of rules in their domain, and I'd actually be surprised if this one was enacted.

dantheman 7y ago

I love this feature :) So thank you.

Rooster61 7y ago

You wouldn't happen to know the guy/gal that implemented the automatic preview when hovering over a selection, would you? Curious as to how they felt about it, and how much that really affected retention (it bugs the everloving shit out of me, I try to use my old smart tv with an outdated version as it lacks this "feature").

squidi 7y ago
I mute my TV when navigating through Netflix because of auto previews. I bet that doesn’t show up in their analytics!
- veidr 7y ago
  I do that too! And so does every person that I personally know well enough to have seen them navigate Netflix (which admittedly is only like 5).

jancsika 7y ago

> But...the product team at Doritos does the same thing. I get awfully suspicious any time someone so clearly on today's side of the digital divide feels the need to hop the fence back to yesterday to elucidate an ethical argument. (And esp. on HN where you'd be hopping away from the place we're all standing.) What's the threshold of addiction beyond which a digital streaming service becomes unethical in your opinion? How about dangerous? I ask because you unequivocally refer to the product you worked on as "addictive." I can't imagine working on such a product without a clear sense of what the boundaries are.

cableshaft 7y ago

If I'm watching Netflix, I am fine with autoplay. Youtube too, while we're at it. I.e. if I'm there specifically to watch videos, and you want to autoplay the next video after I'm done, that's fine. What I'm not fine with is places that autoplay that have no business doing it, especially if they do that specifically to brag about how many autoplays it has (i.e. CNN). CNN in particular is my main annoyance, as when I go there I just want to read their news article, NOT watch video, and yet I have to scramble to hit the pause button on their video the moment I launch one of their pages (even by accident, because I'm just clicking a google link or a Hacker News link). So long story short; Netflix and Youtube or sites that are just about serving video (and nothing else), autoplay to the next video is fine and expected. EVERYTHING ELSE, DON'T AUTOPLAY.

wolco 7y ago
Autoplaying a series makes sense. Autoplaying videos as one scrolls down the cnn website causing the site to take hours to load is foolish.

jcranberry 7y ago

Funny, for my account the timer is 20 seconds. I wonder if it differs per user.

tinus_hn 7y ago

At least I can turn this ‘feature’ off, unlike the obnoxious auto playing trailers.