3 ms·
>>When quantifying things, people naturally focus on things that can easily be measured. Measuring the final result doesn’t provide enough quantitative data, so
by toss1 5y ago
>>When quantifying things, people naturally focus on things that can easily be measured. Measuring the final result doesn’t provide enough quantitative data, so it’s tempting to include the data from intermediate steps. This is an attempt to shorten the feedback loop, and trying to shorten feedback loops is very dangerous in complex systems.
This is a really important point that the author sort of glosses over -- adding measurements, or tighter measurements must be done extremely carefully for two reasons: 1) the risk is 'looking for the key under the streetlamp because it'd better light', and 2) unintended side effects.
The first will cause data to just be basically irrelevant or even inverse to the desired signal, but people will chase it anyway because it is easy to measure. The second taps into the management phenomena of "you get the results that you measure", and create perverse incentives to do the wrong thing, either self-managing, learning, or managing an org.
Good article, much more worthwhile than I expected!
- lanstin 5y agoAlso in many processes, there are things going on at various time scales. Tight feed back loops are good for higher frequency phenomena. Ignoring the slower phenomena can make you like the kids who see the water retreating, build a sandbox, and then come back in the evening to see it all underwater. So many places have all the log data, terabytes, maybe petabytes, but just for three months. What happened last year? When can we expect traffic to break this scaling assumption and need a refactor, this year or next? Impossible to know, unless you make your own copies of summary data and stay around for a few years.
- toss1 5y agoExcellent points! I'm actually thinking that this is part of what is going on with all the cries of "Inflation!!" - compared to last years' numbers, when everything was running at about 40%, of course prices are rebounding, and even overcompensating on the high side - the real question is for how long (there was also a massive hole blown in the money supply, which needs to be backfilled).