6 ms·

Metrics-driven organization (aka KPIs or what the author calls Spreadsheet mentality) is what I do for a living: I’ve seen companies die by it, and companies su

by bertil 4y ago

Metrics-driven organization (aka KPIs or what the author calls Spreadsheet mentality) is what I do for a living: I’ve seen companies die by it, and companies succeed thanks to it. It’s meant to do one thing: clarify everyone’s role in a large, confusing, often political organization where success wouldn't otherwise be clear. If that’s not a large, systemic issue that even the best would-be cult leader among your CxOs can’t address, you need to have KPIs for things to be clear (on top of the best would-be cult leader among your CxOs to sell the idea, the hard choices in there and the vision unpinning it). Imagine you work for the customers' support team: you unblock the client, the client is happy, and you also fix the public doc to unblock anyone after you… That’s all great. But at large organizations (namely: Google, Facebook, and LinkedIn) people ask: should we have a CS team at all? Isn’t that objective unscalable and trying to satisfy people a feature we shouldn’t have because every solution should be a product, not a process? Unless you are working at companies where that question is legitimate, you don’t need KPIs formalized into a tree structure: you need objectives, clear goals, praises for unexpected great work, and support to make sure that people who are asked to do things can. The MECE tree is useful if you don’t want to miss something. For smaller structures, the gaps are a feature: you can’t do it all.

hef19898 4y ago
If you are in operations, regardless of what kind exactly, a proper set of metrics is a basic necessity to succeed. Without data, and proper metrics built around them, you are running blind and every single decision, no matter how well informed you think it is, is just based on gut feeling. Granted that gut feeling can be good (I dare to think mine isn't to bad) but there is no way to tell whether your gut is right without metrics and data to back it up. Pretending otherwise is just lying to yourself. It is very easy so, to convince yourself you actually have good metrics and data. That could be called "Spreadsheet Mentality", even I think actual spreadsheets are way better than highly sophisticated data analysis tools.
- bertil 4y ago
  That’s the distinction that I make between having a complex set of KPIs that cover all the organization (which tend to alienate any challenge, what the author laments) and simple team objectives (that are only relevant for the team, and intentionally miss a lot of the complexity in the rest of the organization).
  - hef19898 4y ago
    If you metrics system fails to connect goal and performance measurement on the team level witu the bigger goals, I'd say the metrics systems is badly degined and implemented.
    - avereveard 4y ago
      You also need to make sure the process has the tool to improve in the metric it's measured, otherwise it's just going to be generating churn Like I've seen dev getting measured on sales goals. Sure the bigger goal is important, but devs what are supposed to do with that?
      - hef19898 4y ago
        Great example of a bad metric. Even if the dev would work for sales, he would at max drive sales indirectly. And if something is only influenced indirectly it is a bad metric use case.
    - michaelt 4y ago
      Connecting things it's possible to usefully measure with the bigger goals is an insurmountable problem in almost every business, because some things are much easier to measure than others. For example, it's very easy to measure if the call centre answered a highly efficient N support calls per worker-hour. And it's very difficult to measure if people who deal with the call centre end up badmouthing your company to their friends, because your useless call centre always just tells them to reboot and never does anything to resolve the underlying issue.
- Archelaos 4y ago
  I think what the author meant by "spreadsheet mentality" is not the belief that metrics may be necessary, but that they are (more or less?) sufficient.
- cupofpython 4y ago
  In addition to this, metrics do not need targets. repeating for emphasis: metrics do not need targets. You absolutely need measurements and data, and to highlight trends.. but target metrics are almost always bullshit. It reminds me of a psychology study done on kids that I wish I had the source for. IIRC when asked to perform a task for no reward (like, how many simple math problems can you solve in a minute?), many performed it well. then they gave the good performers gold stars. Immediately, the kids not getting stars started performing worse than originally. Then they took away the gold stars. Now the top performers started doing worse as well. People just want to be seen and heard. Measure the things that are important WITHOUT deciding in advance what is considered a "good job". this way you can do your 1:1 on the whole picture. "you're slow, but your quality is exceptional" The only exception to this is if the work is simple enough that you can measure it directly against your $$$, and you have a target that ensures you are green. Like maybe a strictly data entry position for a very consistent source of data. In this case, there arent 1:1's to be expected with your employees.. they either hit target or get fired.
  - hef19898 4y ago
    True, if a metric has a target the metric is the target. If there ever was an incentive to game a KPI, that is it. SPC is tricky case in that regard. Because it, kind of, makes the trend and long term development of metric the target. Edit: You had me thinking. Settng a target, or rather target tolerances, for a metric can be necessary if that metric is directly measuring the output of one process that is used by another process. E.g. forecast accuracy, if downstream ops are expected to cope with, say, a deviation of 20 %, that tolerance kind of becomes the goal for the forecast accuracy. This is not some arbritary goal so. For me, setting limits on certain metrics is something different from a target. It is a thin line between those.
    - cupofpython 4y ago
      > if downstream ops are expected to cope with, say, a deviation of 20 %, that tolerance kind of becomes the goal for the forecast accuracy. I think a forecasting target of 20% shouldnt be necessary unless you want to be around 19% deviation. Which might be the case if trying to reach 0% has diminishing returns and isnt actually cost effective. In other words, targets make sense when you are literally targeting a sweet-spot. If 0% deviation is desired, then the directive is still the same - try to be as accurate as possible. If "as accurate as possible" is consistently falling below 20% (which is the importance of measuring and data) then you need to address that. Maybe forecasting is broken or infeasible. On ops side, they might not be expected to handle deviations over 20%, but the directive is probably the same either way "do the best you can to manage the deviation". >For me, setting limits on certain metrics is something different from a target. It is a thin line between those. I agree with this in general. It can be tricky to communicate "here is a boundary that absolutely cannot be crossed, but keep in mind you should be nowhere near that boundary anyway" without it impacting peoples personal optimization strategies. You really have to weigh the consequences of what actually happens when that limit is reached, and if it's worth a possible depression of performance from the team.
- choeger 4y ago
  I would argue that software maintenance would also benefit from metrics. Heck, even development would. Unfortunately, maintenance is hardly ever done at all and sensible metrics for development are really hard to come up with.
- iso1631 4y ago
  I work in operations. Of course we have systems metrics, we know if our systems are playing up or not. We don't do metrics for people though.
  - Karrot_Kream 4y ago
    +1. Metrics for systems have very clear benefits. It lets you know what's wrong and because systems are technical things, they (generally unless you run up against a work political boundary) have clear solution paths. People are nothing like technical systems.
phh 4y ago
> Imagine you work for the customers' support team: you unblock the client, the client is happy, and you also fix the public doc to unblock anyone after you… That’s all great. But at large organizations (namely: Google, Facebook, and LinkedIn) people ask: should we have a CS team at all? Isn’t that objective unscalable and trying to satisfy people a feature we shouldn’t have because every solution should be a product, not a process? First on the "scalable": CS is most definitely scalable, the question is how much. Without using any tooling whatsoever I dare say it scales in O(N*(log N)^2). I can understand the will to optimize it, and with good tooling, I have no doubt it can become sub-linear, which is more than enough. Anything sub-linear wins money with growth, so no need to "make it a product" (which I read as: Put a team of O(1) to replace those people, but maybe I'm misunderstanding) Anyway on to my point: Your Google example proves TFA's point: Google having broken CS is one of the major pain point against Google most people here have. Another Google pain point can probably be explained by misuse of KPIs: Google products that keep getting killed, and replaced, again and again. Even the best people are subject to "the spreadsheet mentality". Even when they are aware of it! My employer's founder bought back the company to stop focusing on the company's share price. Musk would like to take Tesla private. France renationalized EDF which was already >80% own by the state. etc.
- someguydave 4y ago
  I suspect The Spreadsheet Mentality (when it comes to people management) is reinforced by the radical blank-slate/egalitarian civil rights law and enforcement culture relating to employment in the US. There are many legal landmines around Officially Noticing and Considering various differences between individuals, so thinking of people as interchangeable cogs is the most legally defensible approach.
  - mmierz 4y ago
    You’re using super obfuscated language so it’s hard to follow what you’re actually saying. It sounds like you’re saying because you can’t evaluate people with lazy mental shortcuts like race or religion, which would be illegal, you have to try and determine their actual merit, which is harder, and that is the source of corporate spreadsheet hell. Consequently, businesses would spend less time in spreadsheet hell if they could just go back to discriminating against classes of people that are now protected by law. Is that really what you mean?
kalimanzaro 4y ago
Great points but may I suggest "C?Os"? CxO reads like "Chiefs & Officers" to me
- indymike 4y ago
  CxO / CXO is just a way to say C-Level Officer and has been that way for at least a decade or two.
  - kalimanzaro 4y ago
    https://en.m.wikipedia.org/wiki/Chief_experience_officer https://en.m.wikipedia.org/wiki/Chief_experience_officer
  - deleted 4y ago
    [deleted]
- pintxo 4y ago
  Let's make it /^C[A-Z]O$/
Spooky23 4y ago
Spreadsheets are only as good as the understanding of problem domain and scope. I betcha that decision cost Google Cloud a lot of money in RFP evaluations to AWS.
jkukul 4y ago
> But at large organizations (namely: Google, Facebook, and LinkedIn) people ask: should we have a CS team at all? I'd risk stating a thesis that in metrics-driven organisations these kind of questions are actually not asked. Such organisations get very busy defining tree-based KPIs (OKRs or whatever you want to call them) and even busier tracking them. A lot of effort goes into maintaining the KPI driven culture and in the end there's no one left to ask the question like "should we have a CS team at all?".
antonvs 4y ago
> should we have a CS team at all? And that's why no-one uses Google Cloud (comparatively speaking)

by bertil 4y ago

hef19898 4y ago

If you are in operations, regardless of what kind exactly, a proper set of metrics is a basic necessity to succeed. Without data, and proper metrics built around them, you are running blind and every single decision, no matter how well informed you think it is, is just based on gut feeling. Granted that gut feeling can be good (I dare to think mine isn't to bad) but there is no way to tell whether your gut is right without metrics and data to back it up. Pretending otherwise is just lying to yourself. It is very easy so, to convince yourself you actually have good metrics and data. That could be called "Spreadsheet Mentality", even I think actual spreadsheets are way better than highly sophisticated data analysis tools.

bertil 4y ago
That’s the distinction that I make between having a complex set of KPIs that cover all the organization (which tend to alienate any challenge, what the author laments) and simple team objectives (that are only relevant for the team, and intentionally miss a lot of the complexity in the rest of the organization).
- hef19898 4y ago
  If you metrics system fails to connect goal and performance measurement on the team level witu the bigger goals, I'd say the metrics systems is badly degined and implemented.
  - avereveard 4y ago
    You also need to make sure the process has the tool to improve in the metric it's measured, otherwise it's just going to be generating churn Like I've seen dev getting measured on sales goals. Sure the bigger goal is important, but devs what are supposed to do with that?
    - hef19898 4y ago
      Great example of a bad metric. Even if the dev would work for sales, he would at max drive sales indirectly. And if something is only influenced indirectly it is a bad metric use case.
  - michaelt 4y ago
    Connecting things it's possible to usefully measure with the bigger goals is an insurmountable problem in almost every business, because some things are much easier to measure than others. For example, it's very easy to measure if the call centre answered a highly efficient N support calls per worker-hour. And it's very difficult to measure if people who deal with the call centre end up badmouthing your company to their friends, because your useless call centre always just tells them to reboot and never does anything to resolve the underlying issue.
Archelaos 4y ago
I think what the author meant by "spreadsheet mentality" is not the belief that metrics may be necessary, but that they are (more or less?) sufficient.
cupofpython 4y ago
In addition to this, metrics do not need targets. repeating for emphasis: metrics do not need targets. You absolutely need measurements and data, and to highlight trends.. but target metrics are almost always bullshit. It reminds me of a psychology study done on kids that I wish I had the source for. IIRC when asked to perform a task for no reward (like, how many simple math problems can you solve in a minute?), many performed it well. then they gave the good performers gold stars. Immediately, the kids not getting stars started performing worse than originally. Then they took away the gold stars. Now the top performers started doing worse as well. People just want to be seen and heard. Measure the things that are important WITHOUT deciding in advance what is considered a "good job". this way you can do your 1:1 on the whole picture. "you're slow, but your quality is exceptional" The only exception to this is if the work is simple enough that you can measure it directly against your $$$, and you have a target that ensures you are green. Like maybe a strictly data entry position for a very consistent source of data. In this case, there arent 1:1's to be expected with your employees.. they either hit target or get fired.
- hef19898 4y ago
  True, if a metric has a target the metric is the target. If there ever was an incentive to game a KPI, that is it. SPC is tricky case in that regard. Because it, kind of, makes the trend and long term development of metric the target. Edit: You had me thinking. Settng a target, or rather target tolerances, for a metric can be necessary if that metric is directly measuring the output of one process that is used by another process. E.g. forecast accuracy, if downstream ops are expected to cope with, say, a deviation of 20 %, that tolerance kind of becomes the goal for the forecast accuracy. This is not some arbritary goal so. For me, setting limits on certain metrics is something different from a target. It is a thin line between those.
  - cupofpython 4y ago
    > if downstream ops are expected to cope with, say, a deviation of 20 %, that tolerance kind of becomes the goal for the forecast accuracy. I think a forecasting target of 20% shouldnt be necessary unless you want to be around 19% deviation. Which might be the case if trying to reach 0% has diminishing returns and isnt actually cost effective. In other words, targets make sense when you are literally targeting a sweet-spot. If 0% deviation is desired, then the directive is still the same - try to be as accurate as possible. If "as accurate as possible" is consistently falling below 20% (which is the importance of measuring and data) then you need to address that. Maybe forecasting is broken or infeasible. On ops side, they might not be expected to handle deviations over 20%, but the directive is probably the same either way "do the best you can to manage the deviation". >For me, setting limits on certain metrics is something different from a target. It is a thin line between those. I agree with this in general. It can be tricky to communicate "here is a boundary that absolutely cannot be crossed, but keep in mind you should be nowhere near that boundary anyway" without it impacting peoples personal optimization strategies. You really have to weigh the consequences of what actually happens when that limit is reached, and if it's worth a possible depression of performance from the team.
choeger 4y ago
I would argue that software maintenance would also benefit from metrics. Heck, even development would. Unfortunately, maintenance is hardly ever done at all and sensible metrics for development are really hard to come up with.
iso1631 4y ago
I work in operations. Of course we have systems metrics, we know if our systems are playing up or not. We don't do metrics for people though.
- Karrot_Kream 4y ago
  +1. Metrics for systems have very clear benefits. It lets you know what's wrong and because systems are technical things, they (generally unless you run up against a work political boundary) have clear solution paths. People are nothing like technical systems.

phh 4y ago

> Imagine you work for the customers' support team: you unblock the client, the client is happy, and you also fix the public doc to unblock anyone after you… That’s all great. But at large organizations (namely: Google, Facebook, and LinkedIn) people ask: should we have a CS team at all? Isn’t that objective unscalable and trying to satisfy people a feature we shouldn’t have because every solution should be a product, not a process? First on the "scalable": CS is most definitely scalable, the question is how much. Without using any tooling whatsoever I dare say it scales in O(N*(log N)^2). I can understand the will to optimize it, and with good tooling, I have no doubt it can become sub-linear, which is more than enough. Anything sub-linear wins money with growth, so no need to "make it a product" (which I read as: Put a team of O(1) to replace those people, but maybe I'm misunderstanding) Anyway on to my point: Your Google example proves TFA's point: Google having broken CS is one of the major pain point against Google most people here have. Another Google pain point can probably be explained by misuse of KPIs: Google products that keep getting killed, and replaced, again and again. Even the best people are subject to "the spreadsheet mentality". Even when they are aware of it! My employer's founder bought back the company to stop focusing on the company's share price. Musk would like to take Tesla private. France renationalized EDF which was already >80% own by the state. etc.

someguydave 4y ago
I suspect The Spreadsheet Mentality (when it comes to people management) is reinforced by the radical blank-slate/egalitarian civil rights law and enforcement culture relating to employment in the US. There are many legal landmines around Officially Noticing and Considering various differences between individuals, so thinking of people as interchangeable cogs is the most legally defensible approach.
- mmierz 4y ago
  You’re using super obfuscated language so it’s hard to follow what you’re actually saying. It sounds like you’re saying because you can’t evaluate people with lazy mental shortcuts like race or religion, which would be illegal, you have to try and determine their actual merit, which is harder, and that is the source of corporate spreadsheet hell. Consequently, businesses would spend less time in spreadsheet hell if they could just go back to discriminating against classes of people that are now protected by law. Is that really what you mean?

kalimanzaro 4y ago

Great points but may I suggest "C?Os"? CxO reads like "Chiefs & Officers" to me

indymike 4y ago
CxO / CXO is just a way to say C-Level Officer and has been that way for at least a decade or two.
- kalimanzaro 4y ago
  https://en.m.wikipedia.org/wiki/Chief_experience_officer https://en.m.wikipedia.org/wiki/Chief_experience_officer
- deleted 4y ago
  [deleted]
pintxo 4y ago
Let's make it /^C[A-Z]O$/

Spooky23 4y ago

Spreadsheets are only as good as the understanding of problem domain and scope. I betcha that decision cost Google Cloud a lot of money in RFP evaluations to AWS.

jkukul 4y ago

> But at large organizations (namely: Google, Facebook, and LinkedIn) people ask: should we have a CS team at all? I'd risk stating a thesis that in metrics-driven organisations these kind of questions are actually not asked. Such organisations get very busy defining tree-based KPIs (OKRs or whatever you want to call them) and even busier tracking them. A lot of effort goes into maintaining the KPI driven culture and in the end there's no one left to ask the question like "should we have a CS team at all?".

antonvs 4y ago

> should we have a CS team at all? And that's why no-one uses Google Cloud (comparatively speaking)