44 ms·

Metrics-Driven Development

41 points by soofaloofa 10y ago

c0l0 10y ago
I, for one, think we're rapidly approaching Peak Bullshit.
- collyw 10y ago
  I thought the same when I saw yet another "driven development".
coldcode 10y ago
Every time I read one of these utopian posts I think of the fortune 50 company I work for and laugh for about ten minutes. I'd rather read a post on bullshit-driven-development.
maxxxxx 10y ago
Oh my. We need more common sense development practices bundled as new idea. Reminds me of the old saying "If you want to be a guru read some 20 year old papers and republish them as novelty.". This probably works for systems where it's easy to have meaningful metrics. There are plenty of scenarios where metrics are not that easy unless you come up with some bullshit metrics.
- doug1001 10y ago
  yep--but in this instance, the renewal cycle isn't 20 but 50 years. MDD re-packages a 50-year old paradigm--the OODA Loop by USAF Col John Boyd. The wikipedia page on OODA gives the use cases that led to OODA's conception, eg, "Consider a fighter pilot being scrambled to shoot down an enemy aircraft." pass
  - maxxxxx 10y ago
    You are wrong about the renewal cycle because it has been recycled already 3 times in the meantime and forgotten :-)
- dbcurtis 10y ago
  > There are plenty of scenarios where metrics are not that easy unless you come up with some bullshit metrics. And... You get what you measure for. Metrics always skew operations towards what is being measured, so don't leave out anything important, or it will be optimized away in favor of things that are measurable. If something important is difficult to measure, it is a recipe for creating an organization that is very efficient at doing the wrong thing. (Queue up: Steven Covey story about the difference between being efficient and being effective.)
  - sevensor 10y ago
    > And... You get what you measure for. Having been subjected to metric-of-the-month management, I've seen bad metrics drive bad engineering practices. Everybody who invents a new metric thinks it's fair and comprehensive. It's not. If you tighten the screws and tell people to lower metric X or else, they're going to do whatever they have to, from short-term fixes that cause trouble in the long run, to gaming the numbers.
ctrl-j 10y ago
Everywhere I've worked has been in reactive mode. We never have time to slow down and adjust. Never given any time to work on the infrastructure projects that will give us the tools to do our jobs better in the long run. This looks like the epitome of reactionary development. Plus it doesn't lend you any real insight into the "why" of a given problem. In the included example, latency is up on a given method. Lets say it turns out that particular process calls three different "manager" classes, and has another inline sql query. So, you do the right thing and spend a bunch of time cleaning it up and getting things looking more right (with no help from MDD, since there's nothing about development practices in this outline.) Turns out, it wasn't the code after all. Someone had designed autocomplete to work off of a sql table, and someone else disabled the job that re-indexed the search column. * Long story short, TDD and BDD give us development tools * MDD appears to give us... help making minor decisions on potential focus areas? Maybe ammunition to tell our PMs what we should work on? Anyone else see what the big draw for this would be?
- pionar 10y ago
  >Everywhere I've worked has been in reactive mode. We never have time to slow down and adjust. Never given any time to work on the infrastructure projects that will give us the tools to do our jobs better in the long run. >This looks like the epitome of reactionary development. Plus it doesn't lend you any real insight into the "why" of a given problem. How do you know those infrastructure projects are the right projects to do? Will those bring better business value than something else (bug fixes, etc.)? Noting that developer happiness is a measure of value, maybe they are. The only way to really know is to measure it. Put telemetry in there that tells you how your current infrastructure is affecting your business. Does the poor infrastructure lead to more bugs, costing the business more in developer time that could be spent on new feature development? Does the poor infrastructure drive down customer retention or new customer acquisitions? I think the author's point is that if you don't know what people actually do with the code you write, you can't prove that making changes will provide business value. Being proactive instead of reactive is very difficult if your product managers don't know what your customers find useful. They can't know what your customers actually use your product for without metrics.
dmreedy 10y ago
I am curious how a framework like this would detect and handle something like a pivot. It does seem like the logical conclusion of the agile-mentalities that have been developing and the iteration logic embodied in Boyd's Law (which is, of course, cited). But I've watched several software projects with a lot of promise iterate themselves into evolutionary dead-ends by paying attention to nothing other than metrics, and rapidly founding themselves oscillating in a local maximum. Granted, that may just be my bias towards some manner of top-down vision/guidance speaking. It may be that the global maximum I was imagining never actually existed. I do like the emphasis on single point of truth when it comes to metrics. I think that idea is integral to any software development process that wants to have a bus factor > 1
projektfu 10y ago
Deming actually wrote the opposite in The New Economics. https://blog.deming.org/2015/08/myth-if-you-cant-measure-it-you-cant-manage-it/ https://blog.deming.org/2015/08/myth-if-you-cant-measure-it-...
henrik_w 10y ago
Here's a similar post that I thought was quite good: "If the system is not easy to monitor, it is not well designed" http://benjiweber.co.uk/blog/2015/03/02/monitoring-check-smells/ http://benjiweber.co.uk/blog/2015/03/02/monitoring-check-sme...
4ndrewl 10y ago
isn't this just a tldr; of Lean UX? http://shop.oreilly.com/product/0636920021827.do http://shop.oreilly.com/product/0636920021827.do
olkid 10y ago
Similar to what I am reading in Eric Ries' "The Lean Startup". As Ries points out, you have to be careful which metrics are driving development. Only watch metrics that can deliver meaningful insight to help make good business decisions. Has the "Build, Measure, Learn" development cycle fallen out of favour? Or perhaps, it just applies to a subset of business models.

41 points by soofaloofa 10y ago

c0l0 10y ago

I, for one, think we're rapidly approaching Peak Bullshit.

collyw 10y ago
I thought the same when I saw yet another "driven development".

coldcode 10y ago

Every time I read one of these utopian posts I think of the fortune 50 company I work for and laugh for about ten minutes. I'd rather read a post on bullshit-driven-development.

maxxxxx 10y ago

Oh my. We need more common sense development practices bundled as new idea. Reminds me of the old saying "If you want to be a guru read some 20 year old papers and republish them as novelty.". This probably works for systems where it's easy to have meaningful metrics. There are plenty of scenarios where metrics are not that easy unless you come up with some bullshit metrics.

doug1001 10y ago
yep--but in this instance, the renewal cycle isn't 20 but 50 years. MDD re-packages a 50-year old paradigm--the OODA Loop by USAF Col John Boyd. The wikipedia page on OODA gives the use cases that led to OODA's conception, eg, "Consider a fighter pilot being scrambled to shoot down an enemy aircraft." pass
- maxxxxx 10y ago
  You are wrong about the renewal cycle because it has been recycled already 3 times in the meantime and forgotten :-)
dbcurtis 10y ago
> There are plenty of scenarios where metrics are not that easy unless you come up with some bullshit metrics. And... You get what you measure for. Metrics always skew operations towards what is being measured, so don't leave out anything important, or it will be optimized away in favor of things that are measurable. If something important is difficult to measure, it is a recipe for creating an organization that is very efficient at doing the wrong thing. (Queue up: Steven Covey story about the difference between being efficient and being effective.)
- sevensor 10y ago
  > And... You get what you measure for. Having been subjected to metric-of-the-month management, I've seen bad metrics drive bad engineering practices. Everybody who invents a new metric thinks it's fair and comprehensive. It's not. If you tighten the screws and tell people to lower metric X or else, they're going to do whatever they have to, from short-term fixes that cause trouble in the long run, to gaming the numbers.

ctrl-j 10y ago

Everywhere I've worked has been in reactive mode. We never have time to slow down and adjust. Never given any time to work on the infrastructure projects that will give us the tools to do our jobs better in the long run. This looks like the epitome of reactionary development. Plus it doesn't lend you any real insight into the "why" of a given problem. In the included example, latency is up on a given method. Lets say it turns out that particular process calls three different "manager" classes, and has another inline sql query. So, you do the right thing and spend a bunch of time cleaning it up and getting things looking more right (with no help from MDD, since there's nothing about development practices in this outline.) Turns out, it wasn't the code after all. Someone had designed autocomplete to work off of a sql table, and someone else disabled the job that re-indexed the search column. * Long story short, TDD and BDD give us development tools * MDD appears to give us... help making minor decisions on potential focus areas? Maybe ammunition to tell our PMs what we should work on? Anyone else see what the big draw for this would be?

pionar 10y ago
>Everywhere I've worked has been in reactive mode. We never have time to slow down and adjust. Never given any time to work on the infrastructure projects that will give us the tools to do our jobs better in the long run. >This looks like the epitome of reactionary development. Plus it doesn't lend you any real insight into the "why" of a given problem. How do you know those infrastructure projects are the right projects to do? Will those bring better business value than something else (bug fixes, etc.)? Noting that developer happiness is a measure of value, maybe they are. The only way to really know is to measure it. Put telemetry in there that tells you how your current infrastructure is affecting your business. Does the poor infrastructure lead to more bugs, costing the business more in developer time that could be spent on new feature development? Does the poor infrastructure drive down customer retention or new customer acquisitions? I think the author's point is that if you don't know what people actually do with the code you write, you can't prove that making changes will provide business value. Being proactive instead of reactive is very difficult if your product managers don't know what your customers find useful. They can't know what your customers actually use your product for without metrics.

dmreedy 10y ago

I am curious how a framework like this would detect and handle something like a pivot. It does seem like the logical conclusion of the agile-mentalities that have been developing and the iteration logic embodied in Boyd's Law (which is, of course, cited). But I've watched several software projects with a lot of promise iterate themselves into evolutionary dead-ends by paying attention to nothing other than metrics, and rapidly founding themselves oscillating in a local maximum. Granted, that may just be my bias towards some manner of top-down vision/guidance speaking. It may be that the global maximum I was imagining never actually existed. I do like the emphasis on single point of truth when it comes to metrics. I think that idea is integral to any software development process that wants to have a bus factor > 1

projektfu 10y ago

Deming actually wrote the opposite in The New Economics. https://blog.deming.org/2015/08/myth-if-you-cant-measure-it-you-cant-manage-it/ https://blog.deming.org/2015/08/myth-if-you-cant-measure-it-...

henrik_w 10y ago

Here's a similar post that I thought was quite good: "If the system is not easy to monitor, it is not well designed" http://benjiweber.co.uk/blog/2015/03/02/monitoring-check-smells/ http://benjiweber.co.uk/blog/2015/03/02/monitoring-check-sme...

4ndrewl 10y ago

isn't this just a tldr; of Lean UX? http://shop.oreilly.com/product/0636920021827.do http://shop.oreilly.com/product/0636920021827.do

olkid 10y ago

Similar to what I am reading in Eric Ries' "The Lean Startup". As Ries points out, you have to be careful which metrics are driving development. Only watch metrics that can deliver meaningful insight to help make good business decisions. Has the "Build, Measure, Learn" development cycle fallen out of favour? Or perhaps, it just applies to a subset of business models.