4 ms·

Huh. This is actually quite insightful to me. I wonder if this applies more broadly? What from your own life makes you say that?

by ketzo 6y ago

InitialLastName 6y ago
Not GP, but that's a formation of Goodhart's law [0], which comes up in a lot of situations. Notable potential Goodhart's law situations: - Testing in education - GDP (or a number of other economic indicators) as a measure of economic performance/target for economic performance - Healthcare outcomes along a number of axes [0] https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law
banana_giraffe 6y ago
It's a variant of Goodhart's law. Can't speak for the person you're replying to, but absent good management, I've seen review goals turn into nonsense. "You had a goal to fix 5 bugs this quarter, and only fixed 4", "but one was a massive bug that would have cost us $x!", "but you didn't fix 5 bugs, so no bonus" You see it happen on different levels a surprising number of times.
- indigochill 6y ago
  Anecdotally, when I worked in customer support, we were explicitly told that we had to hit a certain number of replies to customers in our ticket system. So I played by the rules: I very clearly grabbed all the easy-to-reply tickets, sent the easy reply, and dropped them if they got hard. Appropriately, a manager spoke to me telling me this was unacceptable (though he did concede the goal was stupid), and gratifyingly the official goals soon changed to reflect an appropriate balance of consistency with a customer as well as some degree of overall throughput. The drawback there is that accurately reflecting what they want in hard metrics might not really be possible because it does come down to a judgment call where the exact balance between support quality and throughput lies (and it's often a shifting balance as well, depending how much support load there is). Which is probably why managers fall back on stupid but easy-to-measure metrics.
deleted 6y ago
[deleted]
dkn775 6y ago
It does in my experience, I work for a gov public safety agency and the numbers they show you are one part of the story. In public policy we learned how you determine what the thresholds are (When does good become excellent?), as well as things like what does improvement mean (a ten percent decrease in raw number? ten percent decrease from last years average?) mean that the framing of metrics must always be examined. Not to mention the litany of inside baseball stuff regarding how various things like 911 calls, car crashes, crimes, and population figures are captured, analyzed, and understood. Outside of numbers. The same logic applies to fact checking sites which is why I always question why so many people use them to refute arguements. The authors decided what was deemed a good source. A dirty secret I’ve noticed of the most controversial issues at the end of the day can be argued successfully both ways, so the fact checkers effectively choose the side they agree with. For example, should Texas be demonized for removing their mask mandates? Lot of areas in Texas are very remote. Texas is more insular of a state than say New Jersey. Federalism and US legal history typically affords states the latitude to make decisions for themselves based on their populations desires. To what extent does interstate commerce (flying out of Texas for example) justify outside/Federal force to upend Texas sovereignty? Lot of people try to argue Texas is just wrong but it’s deeper than that.

Huh. This is actually quite insightful to me. I wonder if this applies more broadly? What from your own life makes you say that?

by ketzo 6y ago

Huh. This is actually quite insightful to me. I wonder if this applies more broadly? What from your own life makes you say that?

InitialLastName 6y ago

Not GP, but that's a formation of Goodhart's law [0], which comes up in a lot of situations. Notable potential Goodhart's law situations: - Testing in education - GDP (or a number of other economic indicators) as a measure of economic performance/target for economic performance - Healthcare outcomes along a number of axes [0] https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law

banana_giraffe 6y ago

It's a variant of Goodhart's law. Can't speak for the person you're replying to, but absent good management, I've seen review goals turn into nonsense. "You had a goal to fix 5 bugs this quarter, and only fixed 4", "but one was a massive bug that would have cost us $x!", "but you didn't fix 5 bugs, so no bonus" You see it happen on different levels a surprising number of times.

indigochill 6y ago
Anecdotally, when I worked in customer support, we were explicitly told that we had to hit a certain number of replies to customers in our ticket system. So I played by the rules: I very clearly grabbed all the easy-to-reply tickets, sent the easy reply, and dropped them if they got hard. Appropriately, a manager spoke to me telling me this was unacceptable (though he did concede the goal was stupid), and gratifyingly the official goals soon changed to reflect an appropriate balance of consistency with a customer as well as some degree of overall throughput. The drawback there is that accurately reflecting what they want in hard metrics might not really be possible because it does come down to a judgment call where the exact balance between support quality and throughput lies (and it's often a shifting balance as well, depending how much support load there is). Which is probably why managers fall back on stupid but easy-to-measure metrics.

deleted 6y ago

[deleted]

dkn775 6y ago

It does in my experience, I work for a gov public safety agency and the numbers they show you are one part of the story. In public policy we learned how you determine what the thresholds are (When does good become excellent?), as well as things like what does improvement mean (a ten percent decrease in raw number? ten percent decrease from last years average?) mean that the framing of metrics must always be examined. Not to mention the litany of inside baseball stuff regarding how various things like 911 calls, car crashes, crimes, and population figures are captured, analyzed, and understood. Outside of numbers. The same logic applies to fact checking sites which is why I always question why so many people use them to refute arguements. The authors decided what was deemed a good source. A dirty secret I’ve noticed of the most controversial issues at the end of the day can be argued successfully both ways, so the fact checkers effectively choose the side they agree with. For example, should Texas be demonized for removing their mask mandates? Lot of areas in Texas are very remote. Texas is more insular of a state than say New Jersey. Federalism and US legal history typically affords states the latitude to make decisions for themselves based on their populations desires. To what extent does interstate commerce (flying out of Texas for example) justify outside/Federal force to upend Texas sovereignty? Lot of people try to argue Texas is just wrong but it’s deeper than that.