2 ms·

Sure they do, they're just very product-specific and an unsexy slog for the reasons you're highlighting. It is definitely a valid optimization to not even try,

Sure they do, they're just very product-specific and an unsexy slog for the reasons you're highlighting. It is definitely a valid optimization to not even try, and just stick to very granular trailing metrics. But that doesn't mean there is never any way to look at how people are using a product over time and draw valid conclusions that can be used as input into good decisions.

marcosdumay 4y ago
> way to look at how people are using a product over time and draw valid conclusions Well, that exists. Metrics that summarize those conclusions is what don't exist. Metrics are a different thing from information. You can discover if people like your product, it just takes effort and isn't accessible to people without any knowledge on the subject.
- sanderjd 4y ago
  Let me give a concrete example. Say it's a subscription service. Say you track three metrics: - Revenue. - Subscription renewal / cancellation by monthly cohort. - Signups. Say the second metric shows that you have a meaningful subscription drop-off after 18 months. By comparing that metric with the signup metric, you might be able to successfully foresee a revenue drop-off months before it materializes in the revenue metric alone, buying valuable time to figure out what to do. Tracking a revenue metric alone always runs the risk of surprising you in real time when it ticks down and you have no idea why.
  - marcosdumay 4y ago
    Hum... That metric doesn't tell whether the users are having a good experience. If that was your point, I misunderstood your comment. Yes, there are metrics that will help you predict problems. My comment wasn't about this.
    - sanderjd 4y ago
      Ah I see. Yep that other comment I made was uselessly hand-wavy. The point I was trying to make, starting with my initial comment in the thread, is that there is a middle ground between "we're spending way too much time devising and implementing really fancy fine grained metrics" and "we only look at revenue". My concrete example makes this point much better, and it seems like we're in agreement, so carry on :)

marcosdumay 4y ago

> way to look at how people are using a product over time and draw valid conclusions Well, that exists. Metrics that summarize those conclusions is what don't exist. Metrics are a different thing from information. You can discover if people like your product, it just takes effort and isn't accessible to people without any knowledge on the subject.

sanderjd 4y ago
Let me give a concrete example. Say it's a subscription service. Say you track three metrics: - Revenue. - Subscription renewal / cancellation by monthly cohort. - Signups. Say the second metric shows that you have a meaningful subscription drop-off after 18 months. By comparing that metric with the signup metric, you might be able to successfully foresee a revenue drop-off months before it materializes in the revenue metric alone, buying valuable time to figure out what to do. Tracking a revenue metric alone always runs the risk of surprising you in real time when it ticks down and you have no idea why.
- marcosdumay 4y ago
  Hum... That metric doesn't tell whether the users are having a good experience. If that was your point, I misunderstood your comment. Yes, there are metrics that will help you predict problems. My comment wasn't about this.
  - sanderjd 4y ago
    Ah I see. Yep that other comment I made was uselessly hand-wavy. The point I was trying to make, starting with my initial comment in the thread, is that there is a middle ground between "we're spending way too much time devising and implementing really fancy fine grained metrics" and "we only look at revenue". My concrete example makes this point much better, and it seems like we're in agreement, so carry on :)