11 ms·

This is why when leading an engineering org I narrowed down to one metric: Committed to Completed Ratio It encourages completing things and only committing to

This is why when leading an engineering org I narrowed down to one metric: Committed to Completed Ratio It encourages completing things and only committing to what you can complete. It's legible, easy to understand by engineering, product, and management, pointless to game, and positively reinforces completion. It's quite difficult to accurately predict when a complete feature will ship, let alone a whole product. But, if one breaks down a feature into its legible constituent components and then commits to completing just what is within that (which also necessitates any required communication to arrive at understanding the requirements) then over time you can get quite good at predicting what you will be able to get done in a sprint and better at only committing to what you believe you can actually complete.

nickreese 4y ago
Sounds like a book in the making. Would love to read this.
- mirekrusin 4y ago
  The essence is right there in the comment, no? Why the whole book about single idea?
- coderintherye 4y ago
  There's been some talk in Agile about such a metric: https://gorillalogic.com/blog/completion-against-commitment-cac-a-useful-metric-for-the-agile-team/ https://gorillalogic.com/blog/completion-against-commitment-... Changing to it was inspired by a few things one of which was: https://www.strategy-business.com/article/10204 https://www.strategy-business.com/article/10204 As well as a talk at CTO club which unfortunately is not shareable but can share the footnote listing: [1] 2014 State of DevOps Report https://devops-research.com/research.html https://devops-research.com/research.html [2] 2016 State of DevOps Report https://devops-research.com/research.html https://devops-research.com/research.html [3] 2015 State of DevOps Report https://devops-research.com/research.html https://devops-research.com/research.html [4] Westrum, R. “A Typology of Organizational Cultures”, http://bmj.co/1BRGh5q http://bmj.co/1BRGh5q [5] Shook, J. “How to Change a Culture: Lessons from NUMMI” http://sloanreview.mit.edu/article/how-to-change-a-culture-lessons-from-nummi/ http://sloanreview.mit.edu/article/how-to-change-a-culture-l... [6] https://www.slideshare.net/reed2001/culture-1798664/93-Highly_Aligned_Loosely_Coupled_Highly https://www.slideshare.net/reed2001/culture-1798664/93-Highl... Though if you are interested in a good book on loosely related subject, highly recommend "Principals of Product Development Flow" https://amzn.to/3Ox4PgB https://amzn.to/3Ox4PgB
sebcat 4y ago
You get a pretty good ratio by doing very little using only that metric.
- coderintherye 4y ago
  That's a fair criticism for certain contexts. Most people I've worked with desire to do well and to feel good about their accomplishments. In those sorts of teams, this works. It would likely not work as well in an environment where people were just hoping to do the minimum possible.
- llampx 4y ago
  That's a problem with pretty much any ratio
- von_lohengramm 4y ago
  I think the purpose of this ratio is that a predictable low is better than an unpredictable high. It's sort of the same idea as padding out an ECD. Better to sometimes impress than sometimes disappoint.
  - morelisp 4y ago
    It is better. If you have a predictable low rate other people can still plan their schedules around you, while you figure out how to speed up / scale up. If you have an unpredictable rate, you block everyone else (and often yourself).
- ChadNauseam 4y ago
  A slightly better one: (committed + 1)/(completed + 2)
  - vitus 4y ago
    Wouldn't you prefer (committed + 1) / completed? Or (committed + 2) / (completed + 1)? In the limit, you want this to approach 1 (namely, you complete more or less everything you commit). A bad situation is where you're overcommitted and have many committed things, but few completed ones (e.g. 10/2), which results in a high ratio. But, with your proposed metric, you'd achieve the asymptotically ideal ratio with 2 committed projects and only 1 completed one, and in fact the global optimum is "I promised nothing and did nothing, yet achieved a ratio of 50%".
kqr 4y ago
Don't people start to go, "oh, we're definitely going to do this, we just haven't committed yet"?
- rgoulter 4y ago
  I liked the ideas I read about 'kanban'. If there's a feeling that work flows well, then there's less concern about when things will be done by. If you can maximize flow, or minimize work in progress.. a kanban board should try to be a visual aid to support that. -- "commited / completed" sounds along the same lines to me. In that sense, I think "it's desirable, but not in progress" isn't necessarily bad in itself. I think it would be better phrased as "this isn't a priority for us right now".
- grayclhn 4y ago
  That can be good. The number of times I’ve actually wound up doing something I was “definitely going to do” is way less than 100%.
Archelaos 4y ago
But how do you encourage people for difficult tasks? It is easy to commit myself to correct a particular spelling in a lable by tomorrow. But to fix the indeterministic bug that crashes our top client's server every now and then?
- rightbyte 4y ago
  It is like metric driven development is a bad idea ... As soon as any "oh this wont be used for your yearly review" metrics are made up. You need to drop everything and play by the metrics. I guess a commited to completed ratio encourages padding out the estimate on easy tickets and punnishes doing hard tickets.
- throw1234651234 4y ago
  Nothing. There is also nothing to encourage people from not taking forever. This metric worked well on my team until I got people who game it by pulling easier tickets and then sitting on them for a week. We still use the approach though. No approach survives a bad team. I could implement draconian rules, but that would just hamper the productive people.
- knicholes 4y ago
  You could commit to something like, "Investigate the logs for two hours" or "inspect memory dump" for an hour. From there, one can add more detailed tasks.
d0mine 4y ago
It is a rookie mistake to think that programming is splitting a big task into smaller tasks that are splitted into even smaller tasks, and that once the process is done, you can use mindless drones to complete these microtasks. It assumes the perfect knowledge, the fixed work volume. In practice, you don't know how much work might be required (a small task may blow up). It is inevitable that sometimes you have to drop features to ship in time. Programming is an iterative probabilistic process.
- morelisp 4y ago
  It is also a rookie mistake to think that just because some tasks might blow up, you can't accurately estimate a large class of them to within a reasonable confidence window (say, within one day 99% of the time), and likewise identify those tasks with a large chance of blowing up. Breaking down tasks into smaller tasks seems to help build this skill faster in some people. If you don't want to be a mindless drone, you should also see the need to practice skills beyond just cranking out code.
  - rekrsiv 4y ago
    It may also be a mistake to assume that it's always possible to break down a system into logical parts and simplify that system until it fits your complexity budget prior to development without knowing the upper bound of your problem's complexity. Some things still have to be discovered before they can be measured.
    - morelisp 4y ago
      But it doesn’t need to be possible all the time, just often enough to set bounds acceptable to the rest of the organization. And how are you going to know unless you’ve already done it a hundred times, and/or try?
      - rekrsiv 4y ago
        The problem with acceptable bounds is when they're applied uniformly to all your code monkeys without adjusting for experience and mental state at any point in time and we can't sample those values in real-time. Performance metrics are often too noisy to be useful so proper bounds are too difficult to set without the "experience" of seeing the team work over a very long period of time. Predicting the future of a high entropy system is predicted to always be hard if you're aiming for 99% accuracy. Solving for hard problems AND solving for this particularly hard problem on a daily basis is less energy efficient than if you stop trying to predict the future at every standup and trust that your average best guess while ignoring the future is good enough to keep you going until the end of this task. TL;DR: Spend less energy sampling the efficiency of human creativity and spend more (but still not too much) on removing barriers that limit creativity.

nickreese 4y ago

Sounds like a book in the making. Would love to read this.

mirekrusin 4y ago
The essence is right there in the comment, no? Why the whole book about single idea?
coderintherye 4y ago
There's been some talk in Agile about such a metric: https://gorillalogic.com/blog/completion-against-commitment-cac-a-useful-metric-for-the-agile-team/ https://gorillalogic.com/blog/completion-against-commitment-... Changing to it was inspired by a few things one of which was: https://www.strategy-business.com/article/10204 https://www.strategy-business.com/article/10204 As well as a talk at CTO club which unfortunately is not shareable but can share the footnote listing: [1] 2014 State of DevOps Report https://devops-research.com/research.html https://devops-research.com/research.html [2] 2016 State of DevOps Report https://devops-research.com/research.html https://devops-research.com/research.html [3] 2015 State of DevOps Report https://devops-research.com/research.html https://devops-research.com/research.html [4] Westrum, R. “A Typology of Organizational Cultures”, http://bmj.co/1BRGh5q http://bmj.co/1BRGh5q [5] Shook, J. “How to Change a Culture: Lessons from NUMMI” http://sloanreview.mit.edu/article/how-to-change-a-culture-lessons-from-nummi/ http://sloanreview.mit.edu/article/how-to-change-a-culture-l... [6] https://www.slideshare.net/reed2001/culture-1798664/93-Highly_Aligned_Loosely_Coupled_Highly https://www.slideshare.net/reed2001/culture-1798664/93-Highl... Though if you are interested in a good book on loosely related subject, highly recommend "Principals of Product Development Flow" https://amzn.to/3Ox4PgB https://amzn.to/3Ox4PgB

sebcat 4y ago

You get a pretty good ratio by doing very little using only that metric.

coderintherye 4y ago
That's a fair criticism for certain contexts. Most people I've worked with desire to do well and to feel good about their accomplishments. In those sorts of teams, this works. It would likely not work as well in an environment where people were just hoping to do the minimum possible.
llampx 4y ago
That's a problem with pretty much any ratio
von_lohengramm 4y ago
I think the purpose of this ratio is that a predictable low is better than an unpredictable high. It's sort of the same idea as padding out an ECD. Better to sometimes impress than sometimes disappoint.
- morelisp 4y ago
  It is better. If you have a predictable low rate other people can still plan their schedules around you, while you figure out how to speed up / scale up. If you have an unpredictable rate, you block everyone else (and often yourself).
ChadNauseam 4y ago
A slightly better one: (committed + 1)/(completed + 2)
- vitus 4y ago
  Wouldn't you prefer (committed + 1) / completed? Or (committed + 2) / (completed + 1)? In the limit, you want this to approach 1 (namely, you complete more or less everything you commit). A bad situation is where you're overcommitted and have many committed things, but few completed ones (e.g. 10/2), which results in a high ratio. But, with your proposed metric, you'd achieve the asymptotically ideal ratio with 2 committed projects and only 1 completed one, and in fact the global optimum is "I promised nothing and did nothing, yet achieved a ratio of 50%".

kqr 4y ago

Don't people start to go, "oh, we're definitely going to do this, we just haven't committed yet"?

rgoulter 4y ago
I liked the ideas I read about 'kanban'. If there's a feeling that work flows well, then there's less concern about when things will be done by. If you can maximize flow, or minimize work in progress.. a kanban board should try to be a visual aid to support that. -- "commited / completed" sounds along the same lines to me. In that sense, I think "it's desirable, but not in progress" isn't necessarily bad in itself. I think it would be better phrased as "this isn't a priority for us right now".
grayclhn 4y ago
That can be good. The number of times I’ve actually wound up doing something I was “definitely going to do” is way less than 100%.

Archelaos 4y ago

But how do you encourage people for difficult tasks? It is easy to commit myself to correct a particular spelling in a lable by tomorrow. But to fix the indeterministic bug that crashes our top client's server every now and then?

rightbyte 4y ago
It is like metric driven development is a bad idea ... As soon as any "oh this wont be used for your yearly review" metrics are made up. You need to drop everything and play by the metrics. I guess a commited to completed ratio encourages padding out the estimate on easy tickets and punnishes doing hard tickets.
throw1234651234 4y ago
Nothing. There is also nothing to encourage people from not taking forever. This metric worked well on my team until I got people who game it by pulling easier tickets and then sitting on them for a week. We still use the approach though. No approach survives a bad team. I could implement draconian rules, but that would just hamper the productive people.
knicholes 4y ago
You could commit to something like, "Investigate the logs for two hours" or "inspect memory dump" for an hour. From there, one can add more detailed tasks.

d0mine 4y ago

It is a rookie mistake to think that programming is splitting a big task into smaller tasks that are splitted into even smaller tasks, and that once the process is done, you can use mindless drones to complete these microtasks. It assumes the perfect knowledge, the fixed work volume. In practice, you don't know how much work might be required (a small task may blow up). It is inevitable that sometimes you have to drop features to ship in time. Programming is an iterative probabilistic process.

morelisp 4y ago
It is also a rookie mistake to think that just because some tasks might blow up, you can't accurately estimate a large class of them to within a reasonable confidence window (say, within one day 99% of the time), and likewise identify those tasks with a large chance of blowing up. Breaking down tasks into smaller tasks seems to help build this skill faster in some people. If you don't want to be a mindless drone, you should also see the need to practice skills beyond just cranking out code.
- rekrsiv 4y ago
  It may also be a mistake to assume that it's always possible to break down a system into logical parts and simplify that system until it fits your complexity budget prior to development without knowing the upper bound of your problem's complexity. Some things still have to be discovered before they can be measured.
  - morelisp 4y ago
    But it doesn’t need to be possible all the time, just often enough to set bounds acceptable to the rest of the organization. And how are you going to know unless you’ve already done it a hundred times, and/or try?
    - rekrsiv 4y ago
      The problem with acceptable bounds is when they're applied uniformly to all your code monkeys without adjusting for experience and mental state at any point in time and we can't sample those values in real-time. Performance metrics are often too noisy to be useful so proper bounds are too difficult to set without the "experience" of seeing the team work over a very long period of time. Predicting the future of a high entropy system is predicted to always be hard if you're aiming for 99% accuracy. Solving for hard problems AND solving for this particularly hard problem on a daily basis is less energy efficient than if you stop trying to predict the future at every standup and trust that your average best guess while ignoring the future is good enough to keep you going until the end of this task. TL;DR: Spend less energy sampling the efficiency of human creativity and spend more (but still not too much) on removing barriers that limit creativity.