4 ms·

The point of NPS vs free cash flow or operational metrics is that it is in theory a leading indicator instead of a trailing indicator. It's nice (but a bit too

by yuliyp 5y ago

The point of NPS vs free cash flow or operational metrics is that it is in theory a leading indicator instead of a trailing indicator. It's nice (but a bit too late) to know that your business is screwed because your customers left, it's another to know that your business is about to be screwed because your customers are on the verge of leaving if an opportunity appears. The criticisms of NPS pointed out (that it's a measure of a high-variance metric) are fair, but the conclusion is not.

keiran_cull 5y ago
What exactly would you suggest it's a leading indicator of? Churn? Ultimately if it matters, it should have to be an input to the cash flow time series -- mediated by churn -- no? (Article means to frame free cash flow as a series rather than a point-in-time value). If you're interested in forecasting, as a company, you should know the things that mean your customer is not doing well within the specific context of your specific business. E.g. in software, you should have a customer health score that's built up from product data. Or you could ask simple questions that are easily interpreted, e.g. asking "how would you rate X's value for money," or "how satisfied are you with X," or even "have you recommended X in the last Y months." These things have a cleaner relationship to future business metrics and a tidier interpretation.
- andrewingram 5y ago
  My own understanding is that it’s supposed to be a prediction input organic growth and nothing more. All it’s really telling you is whether a customer is likely to provide you with another customer, which is why only really high values answers matter, because “hell yeahs” are all that really matter. I’m skeptical as to its usefulness even for this, but most companies I’ve worked for use it as a general KPI which is even more aggravating.
  - marcosdumay 5y ago
    If you go into the trouble of collecting it, it's reasonable to use as a KPI, as it carries useful information. I imagine you are annoyed that they optimize for it. And yes, it's not reasonable to optimize for it. Why do people optimize every KPI?
    - Talanes 5y ago
      >Why do people optimize every KPI? Because if it's not worth optimizing around, how key can it be?
      - marcosdumay 5y ago
        You can do a lot of things with a number that are not optimizing. For example, you can satisfy it, you can alert on behavior, or you can use it as a control (but ok, maybe this one makes a two number KPI instead of two KPIs).
- lmeyerov 5y ago
  Hopefully, it should be a leading indicator for negative churn -- per-account and new account growth / virality -- and for more complicated products, a way to slice that across different personas and features. ('bakers at high-end restaurants have high NPS for the new precision thermometer tracker while those at regular shops ignore them and have low NPS.') Esp in early days of B2B products, it's hard to get that because you don't have the volumes and velocity of b2c nor a good way to detect and attribute viral activation. If you are in a startup and not riding the channel of some megacorp, even more so. Alternatives like signups or other activation checkpoints, or say qualitative interviews, are also interesting, but even more spotty. (Ex: startups raising based on GitHub stars.) We don't do NPS as we have our plate full with known funnel holes through less annoying data collection methods, but as soon as we are happy with the baseline funnel, that's the simple next step.
- yuliyp 5y ago
  I'd expect it to be a leading indicator of user growth / churn, yes.
hakfoo 5y ago
One of the most common contexts for NPS scoring is giving them to direct retail customers as a phone/email followup survey. This is the most dangerous context for it, because a customer who isn't familiar with the system, is sort of primed to have the wrong expectations. They're thinking in terms of "how satisfied was I", which is a different scale entirely with "would you recommend the product". But the NPS question, even when explicitly explained, reads close enough that they'll answer it with the "how satisfied am I" answer, NPS might have worked in a context of a professionally proctored focus group where everyone understood the question and discussed it, but I'm worried you're losing a lot of information when you turn it to one digit on a keypad.

by yuliyp 5y ago

keiran_cull 5y ago

What exactly would you suggest it's a leading indicator of? Churn? Ultimately if it matters, it should have to be an input to the cash flow time series -- mediated by churn -- no? (Article means to frame free cash flow as a series rather than a point-in-time value). If you're interested in forecasting, as a company, you should know the things that mean your customer is not doing well within the specific context of your specific business. E.g. in software, you should have a customer health score that's built up from product data. Or you could ask simple questions that are easily interpreted, e.g. asking "how would you rate X's value for money," or "how satisfied are you with X," or even "have you recommended X in the last Y months." These things have a cleaner relationship to future business metrics and a tidier interpretation.

andrewingram 5y ago
My own understanding is that it’s supposed to be a prediction input organic growth and nothing more. All it’s really telling you is whether a customer is likely to provide you with another customer, which is why only really high values answers matter, because “hell yeahs” are all that really matter. I’m skeptical as to its usefulness even for this, but most companies I’ve worked for use it as a general KPI which is even more aggravating.
- marcosdumay 5y ago
  If you go into the trouble of collecting it, it's reasonable to use as a KPI, as it carries useful information. I imagine you are annoyed that they optimize for it. And yes, it's not reasonable to optimize for it. Why do people optimize every KPI?
  - Talanes 5y ago
    >Why do people optimize every KPI? Because if it's not worth optimizing around, how key can it be?
    - marcosdumay 5y ago
      You can do a lot of things with a number that are not optimizing. For example, you can satisfy it, you can alert on behavior, or you can use it as a control (but ok, maybe this one makes a two number KPI instead of two KPIs).
lmeyerov 5y ago
Hopefully, it should be a leading indicator for negative churn -- per-account and new account growth / virality -- and for more complicated products, a way to slice that across different personas and features. ('bakers at high-end restaurants have high NPS for the new precision thermometer tracker while those at regular shops ignore them and have low NPS.') Esp in early days of B2B products, it's hard to get that because you don't have the volumes and velocity of b2c nor a good way to detect and attribute viral activation. If you are in a startup and not riding the channel of some megacorp, even more so. Alternatives like signups or other activation checkpoints, or say qualitative interviews, are also interesting, but even more spotty. (Ex: startups raising based on GitHub stars.) We don't do NPS as we have our plate full with known funnel holes through less annoying data collection methods, but as soon as we are happy with the baseline funnel, that's the simple next step.
yuliyp 5y ago
I'd expect it to be a leading indicator of user growth / churn, yes.

hakfoo 5y ago

One of the most common contexts for NPS scoring is giving them to direct retail customers as a phone/email followup survey. This is the most dangerous context for it, because a customer who isn't familiar with the system, is sort of primed to have the wrong expectations. They're thinking in terms of "how satisfied was I", which is a different scale entirely with "would you recommend the product". But the NPS question, even when explicitly explained, reads close enough that they'll answer it with the "how satisfied am I" answer, NPS might have worked in a context of a professionally proctored focus group where everyone understood the question and discussed it, but I'm worried you're losing a lot of information when you turn it to one digit on a keypad.