6 ms·

I wish more companies would be able to work with KPIs as _one_ part of the equation, rather than the be all end all. I haven't seen a lot of that :( In my expe

by fhd2 3y ago

I wish more companies would be able to work with KPIs as _one_ part of the equation, rather than the be all end all. I haven't seen a lot of that :( In my experience, data driveness quickly leads to an unhealthy focus on short term results - cause that's usually what's most straightforward to measure and impact. What you get is teams hunting local maxima. What's missing is the guts to follow a vision/intuition regardless of what the numbers say, at least some of the time. It seems hard to accept that we can't measure everything that impacts the long term success of a company, but I don't see why that typically translates to not caring about it altogether.

tomrod 3y ago
The general approach is: performance, when measured, improves; performance, when measured and reported, improves exponentially. KPIs might be defined incorrectly, but their use is simple and straightforward for people to understand. This gives rise to Goodhart's Law, "when a measure becomes a target, it ceases to be a good measure." The best KPI is something that can't be gamed by those whose performance is being measured. This is probably impossible in practice.
pipo234 3y ago
Managers like dashboards. KPIs are dashboards. Your car dashboard is great for staying within speed limit, but now try driving without taking your eyes from the speed dial. :-)
- midasuni 3y ago
  My car has a HUD which puts the current limit and the speed I’m doing on the windscreen
- stefanka 3y ago
  great analogy!
  - phalf 3y ago
    It's even great in the sense that a speedometer tends to make people aim for "I need to go 65 mph" instead of "I should not go above 65 mph" which a speed limit is actually intended to mean. The speed limit became a target instead of being a limit.
    - Arch-TK 3y ago
      The speed limit being a target is how it is treated in driver education and tests in the UK. You can fail a driving test if you spend too much time driving more than ~5 miles per hour below the speed limit in safe conditions.
      - tristor 3y ago
        This is largely because speed limits are too low. In safe conditions, most roadways are built in such a way that a modern vehicle can safely go 10-20mph over the speed limit without difficulty or increased risk. There are several reasons for this. The point though is that timidity in a driver is actually more dangerous than exceeding the speed limit. Being consistently below the speed limit is a strong sign of timidity.
        panragon 3y ago
        I don't think it's so much about the timidity of the driver, it's the effect on driving slow has on people behind you. Driving below the speed limit increases the amount of drivers who need to do overtaking, which are moments substantially more dangerous than just driving straight. If it weren't for forcing other people to overtake I don't think we'd really care, even if it was a sign of timidity. Very common to see older people drive slow on the highway because going 100KM/h seems scary to them. This is, of course, absurd, since driving 80KM/hs on a 100KM/h road causes way more moments for bad things to happen because you're forcing everyone to merge into the speed lane, including trucks with bigger blind spots, rather then only the drivers choosing to go above the speed limit. If you do this you should absolutely be fined, and that's not because it's a good metric for timidity (which I agree can be bad for drivers in general), but rather that the action itself causes harm.
- jollyllama 3y ago
  Indeed. There's no substitute for vision.
- zerkten 3y ago
  Do managers actually like dashboards and KPIs? Not all managers report directly into the CEO, so suffer from the problems as much as their underlings. Management want indicators of certainty and progress with the least effort possible. Superficially, dashboards composed of KPIs give them what they want. Good CEOs are unlikely to rely on these, but instead think of them as an alignment and investigative tool. Each layer of the organisation aligns itself around some KPIs with lower levels often feeding upwards through the hierarchy. As you further down the hierarchy things get messier as they are further from the executives, may have less experience/skill with using tools like KPIs in the way the CEO wants, and there many more teams to align across with problems like duplication, etc. The majority of engineers are at the bottom of the pyramid. Their management is coming from their own ranks with little support, so it's not a surprise there is misunderstanding and misuse. In programming, when the career ladder often involves stopping dev work and becoming a manager, it only makes things worse. When I see peers promoted into management, I always hope it's the ones that have capacity for leadership versus experience because they are going to be the ones that adapt KPIs to match the internal and external needs of the company.
passwordoops 3y ago
>data driveness quickly leads to an unhealthy focus on short term results In my experience it's not data, but when finance and/or sales call the shots. Their incentives are quarterly so without good engineering pushback you get a lack of long term vision

by fhd2 3y ago

tomrod 3y ago

The general approach is: performance, when measured, improves; performance, when measured and reported, improves exponentially. KPIs might be defined incorrectly, but their use is simple and straightforward for people to understand. This gives rise to Goodhart's Law, "when a measure becomes a target, it ceases to be a good measure." The best KPI is something that can't be gamed by those whose performance is being measured. This is probably impossible in practice.

pipo234 3y ago

Managers like dashboards. KPIs are dashboards. Your car dashboard is great for staying within speed limit, but now try driving without taking your eyes from the speed dial. :-)

midasuni 3y ago
My car has a HUD which puts the current limit and the speed I’m doing on the windscreen
stefanka 3y ago
great analogy!
- phalf 3y ago
  It's even great in the sense that a speedometer tends to make people aim for "I need to go 65 mph" instead of "I should not go above 65 mph" which a speed limit is actually intended to mean. The speed limit became a target instead of being a limit.
  - Arch-TK 3y ago
    The speed limit being a target is how it is treated in driver education and tests in the UK. You can fail a driving test if you spend too much time driving more than ~5 miles per hour below the speed limit in safe conditions.
    - tristor 3y ago
      This is largely because speed limits are too low. In safe conditions, most roadways are built in such a way that a modern vehicle can safely go 10-20mph over the speed limit without difficulty or increased risk. There are several reasons for this. The point though is that timidity in a driver is actually more dangerous than exceeding the speed limit. Being consistently below the speed limit is a strong sign of timidity.
      - panragon 3y ago
        I don't think it's so much about the timidity of the driver, it's the effect on driving slow has on people behind you. Driving below the speed limit increases the amount of drivers who need to do overtaking, which are moments substantially more dangerous than just driving straight. If it weren't for forcing other people to overtake I don't think we'd really care, even if it was a sign of timidity. Very common to see older people drive slow on the highway because going 100KM/h seems scary to them. This is, of course, absurd, since driving 80KM/hs on a 100KM/h road causes way more moments for bad things to happen because you're forcing everyone to merge into the speed lane, including trucks with bigger blind spots, rather then only the drivers choosing to go above the speed limit. If you do this you should absolutely be fined, and that's not because it's a good metric for timidity (which I agree can be bad for drivers in general), but rather that the action itself causes harm.
jollyllama 3y ago
Indeed. There's no substitute for vision.
zerkten 3y ago
Do managers actually like dashboards and KPIs? Not all managers report directly into the CEO, so suffer from the problems as much as their underlings. Management want indicators of certainty and progress with the least effort possible. Superficially, dashboards composed of KPIs give them what they want. Good CEOs are unlikely to rely on these, but instead think of them as an alignment and investigative tool. Each layer of the organisation aligns itself around some KPIs with lower levels often feeding upwards through the hierarchy. As you further down the hierarchy things get messier as they are further from the executives, may have less experience/skill with using tools like KPIs in the way the CEO wants, and there many more teams to align across with problems like duplication, etc. The majority of engineers are at the bottom of the pyramid. Their management is coming from their own ranks with little support, so it's not a surprise there is misunderstanding and misuse. In programming, when the career ladder often involves stopping dev work and becoming a manager, it only makes things worse. When I see peers promoted into management, I always hope it's the ones that have capacity for leadership versus experience because they are going to be the ones that adapt KPIs to match the internal and external needs of the company.

passwordoops 3y ago

>data driveness quickly leads to an unhealthy focus on short term results In my experience it's not data, but when finance and/or sales call the shots. Their incentives are quarterly so without good engineering pushback you get a lack of long term vision