3 ms·

Every measurement you make that is advertised will change behaviour. It is very important when making measurements to take that into consideration. Similarly,

Every measurement you make that is advertised will change behaviour. It is very important when making measurements to take that into consideration. Similarly, there is a difference between measurements and metrics. A measurement is simply a measure of something. A metric is a measurement that you make that is fed back into the process improvement cycle: in other words it is a measurement you make to drive changes to the official process. Things like targets or published measurements that have no particular purpose in your process improvement cycle are risky because they will change behaviour in undefined ways. You should always avoid them if you can. Measurements should be taken in order to answer a question. You should avoid taking measurements if you have no questions. Similarly, you should try to measure the minimum that you can that will answer your question. This will minimise the change in behaviour. Before you can have metrics you need a defined process (because by definition a metric is a measurement that is used to change your defined process). Avoid taking metrics when you are unsure of your existing process. When you change your process as a result of a metric, use measurements to answer the question, "Did we affect things we didn't expect to affect". If the answer is "Yes", then take more measurements to answer the question, "Is this change something we need to do something about". In this way you will be able to control your process. One last thing. Do not attempt to protect against gaming other than to not measure things. People naturally optimise and will do it whether they are aware of it or not. Simply observe the effect of your measurements and decide if they are what you want. If not, then remove the measurement.

renholder 8y ago
>Things like targets or published measurements that have no particular purpose in your process improvement cycle are risky because they will change behaviour in undefined ways. You should always avoid them if you can. The problem is that a lot of businesses focus hard on metrics and this doesn't translate to a process improvement cycle, rather, it's on a "person improvement" cycle - making raises, bonuses, promotions, etc. dependent on that notion. A perfect example how this is problematic is in areas like Customer Support, where a number of handled tickets doesn't equate to a good value of the product provided. In a posit, let's consider there's two Support Agents working on supporting 'X' product for 'Y' company, over the course of a quarter. The product and the company have been around and established for quite some time, so there's no need for semantics on availability pool in this posit. Support Agent 1 works a case that takes 3 months but is a relevantly large product bug that impacts millions of users around the world via the fix. Support Agent 1's workload, of course, has a considerably smaller queue size due to this. Now, Support Agent 2 keeps picking-up the low-hanging fruit because it's a numbers game and they know this. What should happen is that management sees the disproportionate numbers and attempts to find out why that is (and/or attempts to automate the low-hanging fruit out of the equation). They should also look at the impact that both of their work had, in overall user-base. Instead, what more than likely happens, is because Support Agent 1's numbers were so low, this makes Support Agent's Manager's numbers look bad and that goes up the chain. Upper Management, oft, doesn't care for the semantics around numbers, so shit rolls down-hill (as the colloquialism goes) and, now, Support Agent 1 is on "Unsatisfactory/Needs Improvement" for their review cycle, losing raises, bonuses, promotions, etc. I think that would disenfranchise Support Agent 1. Since this is a majority American board, consider the wonderful hell that is known as Wal-Mart. From what I understand, it also utilises metrics across the board. If you think on it, can you name a person "happy" to work there? I'd reckon you couldn't. (Again, this is my understanding of it and not anedotal, first-hand experience with the company or anyone who works for it. Aside from Asda, in the UK, Wal-Mart isn't a "thing" here.) So, more than just being fruitless, if it doesn't go into the process improvement cycle, I'd say it rolls pretty high numbers in damaging the overall morale.
- mikekchar 8y ago
  That example is a measurement, not a metric (even though it is mistakenly called a metric by people who don't study this stuff). It is also a bad measurement to take, as you point out. This gets into very complicated discussions. How do you measure productivity? How do you reward good performance? Can you use a reward for good (measured) performance as extrinsic motivation? Anybody who tells you they have a good way of doing this that will work across a wide range of different tasks is either deluded or lying (or both). Lot's of successful businesses are run by deluded/lying people.

renholder 8y ago

>Things like targets or published measurements that have no particular purpose in your process improvement cycle are risky because they will change behaviour in undefined ways. You should always avoid them if you can. The problem is that a lot of businesses focus hard on metrics and this doesn't translate to a process improvement cycle, rather, it's on a "person improvement" cycle - making raises, bonuses, promotions, etc. dependent on that notion. A perfect example how this is problematic is in areas like Customer Support, where a number of handled tickets doesn't equate to a good value of the product provided. In a posit, let's consider there's two Support Agents working on supporting 'X' product for 'Y' company, over the course of a quarter. The product and the company have been around and established for quite some time, so there's no need for semantics on availability pool in this posit. Support Agent 1 works a case that takes 3 months but is a relevantly large product bug that impacts millions of users around the world via the fix. Support Agent 1's workload, of course, has a considerably smaller queue size due to this. Now, Support Agent 2 keeps picking-up the low-hanging fruit because it's a numbers game and they know this. What should happen is that management sees the disproportionate numbers and attempts to find out why that is (and/or attempts to automate the low-hanging fruit out of the equation). They should also look at the impact that both of their work had, in overall user-base. Instead, what more than likely happens, is because Support Agent 1's numbers were so low, this makes Support Agent's Manager's numbers look bad and that goes up the chain. Upper Management, oft, doesn't care for the semantics around numbers, so shit rolls down-hill (as the colloquialism goes) and, now, Support Agent 1 is on "Unsatisfactory/Needs Improvement" for their review cycle, losing raises, bonuses, promotions, etc. I think that would disenfranchise Support Agent 1. Since this is a majority American board, consider the wonderful hell that is known as Wal-Mart. From what I understand, it also utilises metrics across the board. If you think on it, can you name a person "happy" to work there? I'd reckon you couldn't. (Again, this is my understanding of it and not anedotal, first-hand experience with the company or anyone who works for it. Aside from Asda, in the UK, Wal-Mart isn't a "thing" here.) So, more than just being fruitless, if it doesn't go into the process improvement cycle, I'd say it rolls pretty high numbers in damaging the overall morale.

mikekchar 8y ago
That example is a measurement, not a metric (even though it is mistakenly called a metric by people who don't study this stuff). It is also a bad measurement to take, as you point out. This gets into very complicated discussions. How do you measure productivity? How do you reward good performance? Can you use a reward for good (measured) performance as extrinsic motivation? Anybody who tells you they have a good way of doing this that will work across a wide range of different tasks is either deluded or lying (or both). Lot's of successful businesses are run by deluded/lying people.