4 ms·
How do you know that it wasn't caused simply by selecting for more serious trialers? What makes you think the cause was "committing", and not a selection thing
by edmundsauto 6y ago
How do you know that it wasn't caused simply by selecting for more serious trialers? What makes you think the cause was "committing", and not a selection thing?
- teej 6y ago> We AB tested this
- snowwrestler 6y ago"More serious trialers" will be more likely to commit by entering an email; people who are not willing to enter an email are obviously not very serious trialers. It's just two different ways of saying the same thing.
- black_puppydog 6y agoNo, one implies a causality that the other doesn't.
- edoceo 6y agoIt's not a causality, it's a pre-filter. So, of the people who passed filter A more are likely to pass filter B. Today's XKCD was basically about this (in the Alt text) https://m.xkcd.com/2357/ https://m.xkcd.com/2357/
- dependenttypes 6y agodesktop version: https://xkcd.com/2357/ https://xkcd.com/2357/
- snowwrestler 6y ago"More serious trialer" is not a fixed attribute or category that you can select for. A person fits this description based solely on their behavior; they become a more serious trialer by the act of putting in their email address. From that point the causality is the same.
- edmundsauto 6y agoYes - my point was that the narrative of "why" isn't useful. A/B tests tell you what happened. They do not provide insight into why. Yet everyone has a story they tell to fill in the why, and these stories are harmful to making good decisions.
- chrisdbanks 6y agoBecause the conversion rate was higher from no sign-up to paid. So we were probably selecting more serious trailers, but they were then more likely to pay as well. If it was just the conversion rate from sign-up to paid then you would be right, but we looked at the conversion across the entire funnel.