4 ms·

> trying to collect and compare precise numbers that show how they have become more efficient at a task that is in essence not really quantifiable For me this

> trying to collect and compare precise numbers that show how they have become more efficient at a task that is in essence not really quantifiable For me this is particularly maddening as I run into it frequently. Managers/bureaucrats want to make decisions based on 'metrics', primarily because it gives them cover if it happens to be the wrong decision. But, many many issues where decisions are required are more subjective than objective. Meaning there really aren't any metrics that can reliably 'measure' them. Instead of just acknowledging this plain and simple fact, they just create more metrics and/or define the criteria with even more detail. It just makes more work but adds no further insight because ultimately, the issue is subjective and no volume of measurements will help. Think about coming up with a formula about how 'good' art is by measuring how many colors it has, how many curves, right angles, etc. It just doesn't work that way and but they keep trying.

Schroedingersat 4y ago
> But, many many issues where decisions are required are more subjective than objective. They're still objective, it's still objectively wrong to order your employees into a battle royale to the death or paint everything they can find teal as a response to pretty much any problem even when the$e are multiple right answers with a human component. It's just that measuring can cost more than the benefit the task brings, and change what is being measured to destroy anyvalue produced.
- Eddy_Viscosity2 4y ago
  The belief that all problems can become objective provided enough data is measured is the problem. It's not anywhere close to practically true, and I doubt its even theoretically true even with all the assumptions of a frictionless spherical cow in vacuum type scenarios. This belief is what is driving the both the wasted resources in pointless data gathering, but also a paralysis of decision making by those in positions to make them. The key above is 'all'. There are many, a majority even, of problems for which data gathering and analysis is absolutely the best the approach. These would be the objective problems. Not all problems fall into this category.
  - Schroedingersat 4y ago
    > The belief that all problems can become objective provided enough data is measured is the problem. Reality exists. There are objective answers to management problems. There may not be a unique right answer, and which answers are objectively correct depends on the people involved far more than any oversimplified metric, but that does not make them subjective. It's objectively true that in any long term problem with difficult to measure outcomes giving people agency, allowing them to own their work, respecting them, and communicating in whichever way is appropriate to the particular situation is more effective than trying to treat them like a pocket calculator and maximizing some kpi that isn't even related to the output you want. The term you are searching for is solvable by an algorithm or measurable, which they are objectively not. I'm fond of Goodheart's law as a rule of thumb as well as keeping in mind the maxim that any measurement both has a cost and is an interaction with the system being measured and thus must change it.
    - Eddy_Viscosity2 4y ago
      > which answers are objectively correct depends on the people involved This is the almost the very definition of subjective. However, I do take your point about problems that are objective but that are not measurable or solvable by metrics and algorithms. I would add that to my list of problems for which managers will use the metrics tool, even though it won't and can't work and causing frustration for everyone who is relying on them and/or have to collect the metrics data.
      - Schroedingersat 4y ago
        > This is the almost the very definition of subjective. The implication is that the manager is the subject if the object is their interaction with the employee. Thus the answer would be fully internal to the manager if it were subjective. Of course which answer is correct depends on which problem you are solving. By that metric every problem is subjective.
    - deterministic 4y ago
      > There are objective answers to management problems That is something you believe to be true. And something I believe to not be true. Have you ever managed large teams of developers? I have. For many years. And I was very successful doing it. However there was no “objective” way for me to manage the teams. Humans are complex. And the interaction of team members even more so. I have never seen an “objective” metric applied to a team that actually worked without unexpected negative consequences and/or without being gamed by smart team members. If you have an example of an objective metric that has actually worked in reality without negative side effects and without being gamed I would like to hear it.
Helmut10001 4y ago
The reason is responsibility, and not one wants to take it, especially not politicians or people working for the goverment. That is why they need metrics, so they cannot be taken responsible for the consequences. "It was not my fault. The numbers said it."
- Mezzie 4y ago
  I've seen this in academia, for-profit, and non-profit sectors as well. It's what happens when responsibility is not married with reward, or worse, is punished by those above you. Why take responsibility when all responsibility gets you is fired, scolded, blame, or a bunch of additional work but no accolades, pay raises, promotions, etc?
  - MichaelZuo 4y ago
    The only way to get around this, that has actually been proven to work so far, is to give a single body nearly unchallengeable power. Such that even if some mega-disaster happened because of a lack of quantification, nobody would ever dare punish, simultaneously, a super-majority of said body. i.e. central committees of ruling communist parties. Of course the downsides of such a system are familiar to many.
PaulDavisThe1st 4y ago
> Managers/bureaucrats want to make decisions based on 'metrics', primarily because it gives them cover if it happens to be the wrong decision. That might be true, but "making decisions based on metrics" is also the core idea of the meritocracy/technocracy. Rather than claiming that a person has special insight or superior intuition, and should thus control the decision, there's this contemporary (post-Enlightenment?) idea that we could instead systematize decision making and thus avoid or at least reduce cognitive bias, corruption and nepotism.
- gopher_space 4y ago
  "Metrics" as a concept is not under attack just because someone ginned up a false one to save money.
jollybean 4y ago
" Managers/bureaucrats want to make decisions based on 'metrics', primarily because it gives them cover if it happens to be the wrong decision" My word, no, they want to make decisions based on 'metrics' because it validates the 'do or not decision'. We make hardware based products and have a very real need to model component prices, fluctuations, trade/political risks etc.. Most things are not 'art'. If someone wants metrics on something so fuzzy, put the 'wide error' bars on the estimation. Better yet - don't even talk about measurements, make it a 'risk assessment' exercise and identify the key issues that go into defining the unknown. Ontario government is spending a huge amount of money doing 'environmental assessment' for high speed rail - it's better to do that than to start building and hope they don't come across some 'magical wetland' etc. Granted - they are spending too much money on the assessment ...
- Eddy_Viscosity2 4y ago
  I am in no way advocating for total `gut-based' decision making, because that would be an obvious disaster. I think gathering data, analyzing that data, and making a decision based on the results is a very good and important thing. My issue is that not ALL issues can be decided in this way. So while most things are not 'art', as you've said, some very much are like art and its for these types of issues the metrics model breaks down. It's just the wrong tool for the job. The `risk assessment' approach, I agree can also be a useful one in many cases but not all. I think one example is when trying to measure, or more likely, rank, people for suitability for something. This can be more of an art. There are metrics that can be useful, but then Goodhart's Law [1] can kick in. And even when the metrics are useful, they simply can't account for, and will never be able to account for, the shear complexity and adaptability of actual people. So yes, metrics can be good for some thing, risk assessment good for others, but that still leaves some problems on the table for which neither are sufficient. [1] https://www.formpl.us/blog/goodharts-law#:~:text=Goodhart's%20Law%20is%20the%20idea,they're%20trying%20to%20improve https://www.formpl.us/blog/goodharts-law#:~:text=Goodhart's%....

Schroedingersat 4y ago

> But, many many issues where decisions are required are more subjective than objective. They're still objective, it's still objectively wrong to order your employees into a battle royale to the death or paint everything they can find teal as a response to pretty much any problem even when the$e are multiple right answers with a human component. It's just that measuring can cost more than the benefit the task brings, and change what is being measured to destroy anyvalue produced.

Eddy_Viscosity2 4y ago
The belief that all problems can become objective provided enough data is measured is the problem. It's not anywhere close to practically true, and I doubt its even theoretically true even with all the assumptions of a frictionless spherical cow in vacuum type scenarios. This belief is what is driving the both the wasted resources in pointless data gathering, but also a paralysis of decision making by those in positions to make them. The key above is 'all'. There are many, a majority even, of problems for which data gathering and analysis is absolutely the best the approach. These would be the objective problems. Not all problems fall into this category.
- Schroedingersat 4y ago
  > The belief that all problems can become objective provided enough data is measured is the problem. Reality exists. There are objective answers to management problems. There may not be a unique right answer, and which answers are objectively correct depends on the people involved far more than any oversimplified metric, but that does not make them subjective. It's objectively true that in any long term problem with difficult to measure outcomes giving people agency, allowing them to own their work, respecting them, and communicating in whichever way is appropriate to the particular situation is more effective than trying to treat them like a pocket calculator and maximizing some kpi that isn't even related to the output you want. The term you are searching for is solvable by an algorithm or measurable, which they are objectively not. I'm fond of Goodheart's law as a rule of thumb as well as keeping in mind the maxim that any measurement both has a cost and is an interaction with the system being measured and thus must change it.
  - Eddy_Viscosity2 4y ago
    > which answers are objectively correct depends on the people involved This is the almost the very definition of subjective. However, I do take your point about problems that are objective but that are not measurable or solvable by metrics and algorithms. I would add that to my list of problems for which managers will use the metrics tool, even though it won't and can't work and causing frustration for everyone who is relying on them and/or have to collect the metrics data.
    - Schroedingersat 4y ago
      > This is the almost the very definition of subjective. The implication is that the manager is the subject if the object is their interaction with the employee. Thus the answer would be fully internal to the manager if it were subjective. Of course which answer is correct depends on which problem you are solving. By that metric every problem is subjective.
  - deterministic 4y ago
    > There are objective answers to management problems That is something you believe to be true. And something I believe to not be true. Have you ever managed large teams of developers? I have. For many years. And I was very successful doing it. However there was no “objective” way for me to manage the teams. Humans are complex. And the interaction of team members even more so. I have never seen an “objective” metric applied to a team that actually worked without unexpected negative consequences and/or without being gamed by smart team members. If you have an example of an objective metric that has actually worked in reality without negative side effects and without being gamed I would like to hear it.

Helmut10001 4y ago

The reason is responsibility, and not one wants to take it, especially not politicians or people working for the goverment. That is why they need metrics, so they cannot be taken responsible for the consequences. "It was not my fault. The numbers said it."

Mezzie 4y ago
I've seen this in academia, for-profit, and non-profit sectors as well. It's what happens when responsibility is not married with reward, or worse, is punished by those above you. Why take responsibility when all responsibility gets you is fired, scolded, blame, or a bunch of additional work but no accolades, pay raises, promotions, etc?
- MichaelZuo 4y ago
  The only way to get around this, that has actually been proven to work so far, is to give a single body nearly unchallengeable power. Such that even if some mega-disaster happened because of a lack of quantification, nobody would ever dare punish, simultaneously, a super-majority of said body. i.e. central committees of ruling communist parties. Of course the downsides of such a system are familiar to many.

PaulDavisThe1st 4y ago

> Managers/bureaucrats want to make decisions based on 'metrics', primarily because it gives them cover if it happens to be the wrong decision. That might be true, but "making decisions based on metrics" is also the core idea of the meritocracy/technocracy. Rather than claiming that a person has special insight or superior intuition, and should thus control the decision, there's this contemporary (post-Enlightenment?) idea that we could instead systematize decision making and thus avoid or at least reduce cognitive bias, corruption and nepotism.

gopher_space 4y ago
"Metrics" as a concept is not under attack just because someone ginned up a false one to save money.

jollybean 4y ago

" Managers/bureaucrats want to make decisions based on 'metrics', primarily because it gives them cover if it happens to be the wrong decision" My word, no, they want to make decisions based on 'metrics' because it validates the 'do or not decision'. We make hardware based products and have a very real need to model component prices, fluctuations, trade/political risks etc.. Most things are not 'art'. If someone wants metrics on something so fuzzy, put the 'wide error' bars on the estimation. Better yet - don't even talk about measurements, make it a 'risk assessment' exercise and identify the key issues that go into defining the unknown. Ontario government is spending a huge amount of money doing 'environmental assessment' for high speed rail - it's better to do that than to start building and hope they don't come across some 'magical wetland' etc. Granted - they are spending too much money on the assessment ...

Eddy_Viscosity2 4y ago
I am in no way advocating for total `gut-based' decision making, because that would be an obvious disaster. I think gathering data, analyzing that data, and making a decision based on the results is a very good and important thing. My issue is that not ALL issues can be decided in this way. So while most things are not 'art', as you've said, some very much are like art and its for these types of issues the metrics model breaks down. It's just the wrong tool for the job. The `risk assessment' approach, I agree can also be a useful one in many cases but not all. I think one example is when trying to measure, or more likely, rank, people for suitability for something. This can be more of an art. There are metrics that can be useful, but then Goodhart's Law [1] can kick in. And even when the metrics are useful, they simply can't account for, and will never be able to account for, the shear complexity and adaptability of actual people. So yes, metrics can be good for some thing, risk assessment good for others, but that still leaves some problems on the table for which neither are sufficient. [1] https://www.formpl.us/blog/goodharts-law#:~:text=Goodhart's%20Law%20is%20the%20idea,they're%20trying%20to%20improve https://www.formpl.us/blog/goodharts-law#:~:text=Goodhart's%....