3 ms·

What a long article to ultimately get to the realization (but fail to state it): If you pick a quantitative measure and optimize for it as a target, that's what

What a long article to ultimately get to the realization (but fail to state it): If you pick a quantitative measure and optimize for it as a target, that's what you get. If you optimize for "velocity" (# of changes, here) then you get velocity, a lot of changes. If you optimize for "high impact" changes then you get high impact changes. This is not shocking. What also shouldn't be shocking is that if the two are only poorly correlated then you'll end up not getting meaningful improvements in the other measure, and may in fact reduce the other if there is a conflict between them. The bigger problem is that most organizations don't know what they want to optimize or how to measure the things they want to optimize. So they find proxies like "velocity" or "impact" and end up optimizing for them instead of what they actually want. In other words, Goodhart's Law.

marcosdumay 4y ago
It's very likely that velocity is very strongly correlated with impact, that's not exactly the problem. The thing that people mostly don't expect is that if you completely optimize for some metric, you will harm any metric that is not perfectly aligned with it. It doesn't matter if the correlation is very good, as long as it's not perfect it will be harmed.
- Jtsummers 4y ago
  Right. I was more referring to the article's definition of velocity which was number of changes ("launching more, faster"). It's easy for that to be misaligned with anything useful. You can make a lot of small things with little "impact" or value if your goal is just the number of changes. If you have a saner definition of velocity which is based on the size or scope and not just the raw number of changes, then you can pair that with prioritization based on expected value and your measures will be better aligned.
  - marcosdumay 4y ago
    But Agile's velocity is basically that. It's a measure of how long it will take you to change course if you discover your current design is bad and you should be working on something slightly different. Again, this is very likely very correlated with impact. But you seem to want it to be a measure of productivity; it's not.
    - Jtsummers 4y ago
      I gave two definitions of velocity and you just used "that". I have no idea if we're in agreement or disagreement now. If you mean that Scrum uses sizes, then yeah. It tends to be better aligned with what you want because it uses sizes and priorities. It doesn't just count the raw number of changes as the target, even for its velocity measure (which isn't the target, but a guide, at least in theory). The target in Scrum is getting what the customer (again, in theory) wants as quickly as possible which means focusing on priorities. Velocity is there to see how well you're operating rather than being a goal. If you mean Agile (which version?) doesn't use sizes but just a raw count of changes for velocity, then we're in disagreement. I mean, there probably are idiots out there who do that. But that's not what's encouraged by anyone that cares about getting shit done. It's a useless measure especially if it becomes your target for optimization. Anyone can get high velocity if all changes count the same. Submit 500 change requests to add 500 logging statements. Wow they're fast! Also they're gaming the system.

marcosdumay 4y ago

It's very likely that velocity is very strongly correlated with impact, that's not exactly the problem. The thing that people mostly don't expect is that if you completely optimize for some metric, you will harm any metric that is not perfectly aligned with it. It doesn't matter if the correlation is very good, as long as it's not perfect it will be harmed.

Jtsummers 4y ago
Right. I was more referring to the article's definition of velocity which was number of changes ("launching more, faster"). It's easy for that to be misaligned with anything useful. You can make a lot of small things with little "impact" or value if your goal is just the number of changes. If you have a saner definition of velocity which is based on the size or scope and not just the raw number of changes, then you can pair that with prioritization based on expected value and your measures will be better aligned.
- marcosdumay 4y ago
  But Agile's velocity is basically that. It's a measure of how long it will take you to change course if you discover your current design is bad and you should be working on something slightly different. Again, this is very likely very correlated with impact. But you seem to want it to be a measure of productivity; it's not.
  - Jtsummers 4y ago
    I gave two definitions of velocity and you just used "that". I have no idea if we're in agreement or disagreement now. If you mean that Scrum uses sizes, then yeah. It tends to be better aligned with what you want because it uses sizes and priorities. It doesn't just count the raw number of changes as the target, even for its velocity measure (which isn't the target, but a guide, at least in theory). The target in Scrum is getting what the customer (again, in theory) wants as quickly as possible which means focusing on priorities. Velocity is there to see how well you're operating rather than being a goal. If you mean Agile (which version?) doesn't use sizes but just a raw count of changes for velocity, then we're in disagreement. I mean, there probably are idiots out there who do that. But that's not what's encouraged by anyone that cares about getting shit done. It's a useless measure especially if it becomes your target for optimization. Anyone can get high velocity if all changes count the same. Submit 500 change requests to add 500 logging statements. Wow they're fast! Also they're gaming the system.