3 ms·

I have a question to the experienced PMs out there. If Lenny now time slipped back to the early discussions when they were rolling out the counter and the refe

I have a question to the experienced PMs out there. If Lenny now time slipped back to the early discussions when they were rolling out the counter and the referral program, would he then vehemently oppose the ideas, citing all that he's learned? "Duolingo isn't like a match-3 game" "Language learners aren't referring discounted rides" Maybe he would? Maybe he should, because he's learned these lessons. But was younger Lenny's job to be the whippersnapper who says with conviction "you don't know if you don't try, let's give it a shot!" And so in a sense you must have these two types of people, and once you've been through the gauntlet, you either make it, and have been right all along, or you become wise, and your job is now to be the naysayer. Or, if you're an experienced PM, are you instead thinking, this isn't perfect hindsight, but simply a largely avoidable detour before he arrived at the metrics that mattered? Maybe a critique of the actual journey is even more educational.

acjacobson 4y ago
In my opinion and experience it's better to be the younger version who runs lots of experiments. While the older version may feel more wise, and may have taken some lessons from the past that helps him be aware of pitfalls in the future, there's always the danger of hindsight bias. The older version has ideas about why referrals didn't work, or the counter didn't work, but they might not be right (and I suspect they're not exactly right as Duolingo still contains different versions of affiliate referrals and counters). A better strategy is to think about it in terms of tradeoffs - if I build an affiliate program, what does that mean I won't build? I got the sense from the article that they explored one growth idea at a time, as opposed to evaluating a group of options from the beginning. All things being equal it's probably best to spend a bit of time thinking about several opportunities before going off and building the first one. Previous product experience won't guarantee you can call what will work for the future, but it can help you weight options vs one another more effectively. You're much more likely to have a nuanced view of what's possible or what might go wrong.

acjacobson 4y ago

In my opinion and experience it's better to be the younger version who runs lots of experiments. While the older version may feel more wise, and may have taken some lessons from the past that helps him be aware of pitfalls in the future, there's always the danger of hindsight bias. The older version has ideas about why referrals didn't work, or the counter didn't work, but they might not be right (and I suspect they're not exactly right as Duolingo still contains different versions of affiliate referrals and counters). A better strategy is to think about it in terms of tradeoffs - if I build an affiliate program, what does that mean I won't build? I got the sense from the article that they explored one growth idea at a time, as opposed to evaluating a group of options from the beginning. All things being equal it's probably best to spend a bit of time thinking about several opportunities before going off and building the first one. Previous product experience won't guarantee you can call what will work for the future, but it can help you weight options vs one another more effectively. You're much more likely to have a nuanced view of what's possible or what might go wrong.