5 ms·

Yeah this also sounds like part of the "MBA-ification" of business. I think a lot of this comes from a reasonable goal to have: if can find appropriate metrics

by skohan 5y ago

Yeah this also sounds like part of the "MBA-ification" of business. I think a lot of this comes from a reasonable goal to have: if can find appropriate metrics which represent the work your company is doing, this should give you a reasonably objective way of measuring how different interventions move you closer to or farther from your goal. But I think the problem is that actually finding and implementing those metrics is a bit of a pipe dream in most cases. A lot of these metrics which people come up with to try to measure productivity - story points, ticket flow, time-on-task - are all fairly subjective, and not very good proxies for measuring actual value creation. I think it gets worse in orgs where there's a disconnect between management and the workforce which is actually getting things done. Often times, upper management only has these artificial metrics to evaluate the performance of a team. A lot of company cultures put management in a bit of a bubble, so middle managers have more of a focus on conversations with other people in that management layer, who are metrics focused, rather than dealing with development teams, who are actually engaged with the product. As a result, the "ground truth" for management becomes the metrics, and you start to see really silly things happen because the metrics matter more than the product.

Jtsummers 5y ago
You Can't Improve What You Can't Measure Disease YCIWYCM is a great way to kill productivity and performance improvement efforts when taken too far. Sometimes, you've just got to go with your gut. The amount of effort needed to make measurements, especially for less tangible knowledge work, can kill productivity overall. Often, the measurable parts are at the conclusion of a particular effort, the total time it took to get there and the value of the output. But it's not necessarily possible to meaningfully measure the internal process of reaching it when (again, knowledge work specific) half the time may be spent thinking about the thing. How do you measure that? How do you indicate progress when you're one "aha" moment away from success but can't force one to occur? Synthesis, what most programmers do, the combining of ideas to form a new theory [Naur1985] is hard to do mechanically. It can be done, for a lot of work, but not necessarily for everything. But because it's not a purely mechanical process (in contrast to a lot of other business processes or physical manufacturing work where you have more mechanical work and less synthesis) it's hard to identify useful metrics and collect them. When the work is more mechanical ("We've done this before, but...") then you can start finding more places to measure and gain useful insights. But the more novelty that occurs, the more you need to collaborate with each other or spend time in thought, the harder it is to find useful measurements. [Naur1985] https://pages.cs.wisc.edu/~remzi/Naur.pdf https://pages.cs.wisc.edu/~remzi/Naur.pdf
- vp8989 5y ago
  This empiricism theater that so many engage in these days is just another form of CYA.
knightofmars 5y ago
I witnessed what you are describing first-hand. Some years ago, in a job far away, the overall account manager for my region managed everything through a single budget spreadsheet. Decisions were made entirely based on cost without any actual thought to the outcome of the decision. While in the middle of cutting cost, the company failed to deliver some important contracts that led to the company being penalized (to the tune of tens of millions of dollars). The irony, if they'd just spent a fraction of the penalty amount hiring people, they would have been able to deliver on the contracts and avoided the penalty.
sodapopcan 5y ago
I feel a metrics are only valuable to the team itself (if they find them helpful). No one else should be looking at them because as you touch on, they are meaningless to anyone else. My team points stories and we just that as an idea of how much work to take on for the week. If we under or overshoot, it's not a big a deal, but story points help us relay externally what will likely get done this week. No one is going to be able to look at how many points we did in a week be able to get anywhere close to an accurate representation of how much work we did. Even looking at how often we hit our mark is going to tell them nothing about our productivity.
fghorow 5y ago
SAP. Enough said.
synquid 5y ago
Sounds like a case of Goodhart's law, "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes."

by skohan 5y ago

Jtsummers 5y ago

You Can't Improve What You Can't Measure Disease YCIWYCM is a great way to kill productivity and performance improvement efforts when taken too far. Sometimes, you've just got to go with your gut. The amount of effort needed to make measurements, especially for less tangible knowledge work, can kill productivity overall. Often, the measurable parts are at the conclusion of a particular effort, the total time it took to get there and the value of the output. But it's not necessarily possible to meaningfully measure the internal process of reaching it when (again, knowledge work specific) half the time may be spent thinking about the thing. How do you measure that? How do you indicate progress when you're one "aha" moment away from success but can't force one to occur? Synthesis, what most programmers do, the combining of ideas to form a new theory [Naur1985] is hard to do mechanically. It can be done, for a lot of work, but not necessarily for everything. But because it's not a purely mechanical process (in contrast to a lot of other business processes or physical manufacturing work where you have more mechanical work and less synthesis) it's hard to identify useful metrics and collect them. When the work is more mechanical ("We've done this before, but...") then you can start finding more places to measure and gain useful insights. But the more novelty that occurs, the more you need to collaborate with each other or spend time in thought, the harder it is to find useful measurements. [Naur1985] https://pages.cs.wisc.edu/~remzi/Naur.pdf https://pages.cs.wisc.edu/~remzi/Naur.pdf

vp8989 5y ago
This empiricism theater that so many engage in these days is just another form of CYA.

knightofmars 5y ago

I witnessed what you are describing first-hand. Some years ago, in a job far away, the overall account manager for my region managed everything through a single budget spreadsheet. Decisions were made entirely based on cost without any actual thought to the outcome of the decision. While in the middle of cutting cost, the company failed to deliver some important contracts that led to the company being penalized (to the tune of tens of millions of dollars). The irony, if they'd just spent a fraction of the penalty amount hiring people, they would have been able to deliver on the contracts and avoided the penalty.

sodapopcan 5y ago

I feel a metrics are only valuable to the team itself (if they find them helpful). No one else should be looking at them because as you touch on, they are meaningless to anyone else. My team points stories and we just that as an idea of how much work to take on for the week. If we under or overshoot, it's not a big a deal, but story points help us relay externally what will likely get done this week. No one is going to be able to look at how many points we did in a week be able to get anywhere close to an accurate representation of how much work we did. Even looking at how often we hit our mark is going to tell them nothing about our productivity.

fghorow 5y ago

SAP. Enough said.

synquid 5y ago

Sounds like a case of Goodhart's law, "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes."