3 ms·
Great article. My take on it: - Metrics give everyone a false sense of objectivity. If you actually look at any stats in depth, you will end up making decision
by lordnacho 6y ago
Great article. My take on it:
- Metrics give everyone a false sense of objectivity. If you actually look at any stats in depth, you will end up making decisions about what to measure, and what the numbers mean. These words are actually the most important part of any study, but are always left out in favor of the graph. Want to know how many people died of covid? You need to consider things like population adjusted deaths. And adjustment for age. And a a whole host of other things to do with how the data was collected, and how it was presented. The economic side is even more messy. How much would GDP have been if we'd done this or that? There's no number that's the answer, but we do need numbers.
- You'll always care about more metrics than one. There's no generally objective way to decide how to weight those. You might have a utility function that decides this, thus reducing the problem to a simple optimization, but the guns/butter balance has still got to come from some kind of value that you express.
- The big question is rules vs discretion. You set up an elaborate measuring system (eg GDP) and you imagine that you have certain levers that you can move in response to whatever situation occurs. Part of this is that actors expect you to behave a certain way. But what happens if something you thought would work mechanically doesn't behave like you thought? Should you move your levers contrary to your previous commitment?
- Finally, there are the unmeasurable sensations. You have a team, it feels wrong somehow. You talk to people, get opinions, you get a feel for what is wrong. None of it is something you found in your LOC/user summary, or CI pipeline fails, or whatever.