4 ms·

Not really. With the amount of users these companies have, you can get statistically valid results very fast (and you can run hundreds of tests in parallel).

blobster 8y ago
Maybe, but I'm talking about getting to that level in the first place.
- nostrademons 8y ago
  Startups that are still searching for product/market fit don't really do much A/B testing - the user populations are too small for anything statistically relevant. They rely more on intuition, or as some would call it, blind luck. In some ways, what an entrepreneur does is very coarse-grained A/B testing anyways. If you launch 50 different startup ideas and they all get zero traffic, and then you launch the 51st and it has 100 users sign up over a weekend, then there's a good indication that you should go with the 51st idea.
  - quanticle 8y ago
    It's also what venture capitalists do at the level of companies. Fund a hundred companies, and then watch them. Pour more money into the ones who appear to be succeeding.
deleted 8y ago
[deleted]
masonic 8y ago
Then why is their search such utter crap? When I search on a specific title, I get anything that roughly matches any keywords in combination, then also any title that remotely associates (even conceptually):in any possible way with any individual keyword. But what it won't tell me is "we don't have the title you want". Specific example from earlier: look for the 2007 film "Sunshine". Then, go plug "sunshine film" into duckduckgo and see what comes up as the very first item.
- homero 8y ago
  That upsets me too. They hope I'll forget what i wanted and use a recommendation but I'm left confused and leave the app.
- mongol 8y ago
  Let me guess: It has proved to work well for retention.
  - masonic 8y ago
    Probably not so much toward retention (I'd have quit over this were I not getting it free) as toward redirecting you to content they already control at little or no variable cost.
- groestl 8y ago
  Probably causes some customers to watch into some of the crappy search result titles, to see if it's the thing they wanted. Hence, the "hours watched" metric goes up.
- shalmanese 8y ago
  I think the way they do it is very elegant. They surmise the film you want and if they don't have it, they put it in the top bar as "find films related to X". That way, they can push you into consuming something else even if they don't have what you were originally looking for.
  - masonic 8y ago
    "find films related to X" But my point is that most of the results are in no way related to X.
- dewey 8y ago
  Do you really think that’s not on purpose or surprising? If you show them recommendations instead of an empty search result they might want to watch something else or get inspired to use other search terms to find some other result. That may sound like a bad search for engineers but for the user it’s probably what they want and what Netflix measured.

blobster 8y ago

Maybe, but I'm talking about getting to that level in the first place.

nostrademons 8y ago
Startups that are still searching for product/market fit don't really do much A/B testing - the user populations are too small for anything statistically relevant. They rely more on intuition, or as some would call it, blind luck. In some ways, what an entrepreneur does is very coarse-grained A/B testing anyways. If you launch 50 different startup ideas and they all get zero traffic, and then you launch the 51st and it has 100 users sign up over a weekend, then there's a good indication that you should go with the 51st idea.
- quanticle 8y ago
  It's also what venture capitalists do at the level of companies. Fund a hundred companies, and then watch them. Pour more money into the ones who appear to be succeeding.

deleted 8y ago

[deleted]

masonic 8y ago

Then why is their search such utter crap? When I search on a specific title, I get anything that roughly matches any keywords in combination, then also any title that remotely associates (even conceptually):in any possible way with any individual keyword. But what it won't tell me is "we don't have the title you want". Specific example from earlier: look for the 2007 film "Sunshine". Then, go plug "sunshine film" into duckduckgo and see what comes up as the very first item.

homero 8y ago
That upsets me too. They hope I'll forget what i wanted and use a recommendation but I'm left confused and leave the app.
mongol 8y ago
Let me guess: It has proved to work well for retention.
- masonic 8y ago
  Probably not so much toward retention (I'd have quit over this were I not getting it free) as toward redirecting you to content they already control at little or no variable cost.
groestl 8y ago
Probably causes some customers to watch into some of the crappy search result titles, to see if it's the thing they wanted. Hence, the "hours watched" metric goes up.
shalmanese 8y ago
I think the way they do it is very elegant. They surmise the film you want and if they don't have it, they put it in the top bar as "find films related to X". That way, they can push you into consuming something else even if they don't have what you were originally looking for.
- masonic 8y ago
  "find films related to X" But my point is that most of the results are in no way related to X.
dewey 8y ago
Do you really think that’s not on purpose or surprising? If you show them recommendations instead of an empty search result they might want to watch something else or get inspired to use other search terms to find some other result. That may sound like a bad search for engineers but for the user it’s probably what they want and what Netflix measured.