5 ms·
Goodhart's Law as generalized by Marilyn Strathern: When a measure becomes a target, it ceases to be a good measure Campbell's law: The more any quantitative s
by cratermoon 5y ago
Goodhart's Law as generalized by Marilyn Strathern: When a measure becomes a target, it ceases to be a good measure
Campbell's law: The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.
- zamalek 5y agoFor many reasons, I typically do better with qualitative performance reviews, and I'm usually more critical of myself than my manager. I have first-hand experience with exactly the kind of point I believe you might be trying to make, because I've been lucky enough to be managed by people who are genuinely passionate about managing people (and have therefore used qualitative reviews). The problem is when people disagree on performance, in isolation ("I don't deserve a 3 this quarter") or in relation ("why did X get a promotion instead of me"). Quantitative reviews are essentially your employer covering their ass. People might game the system, but there's a paper trail to back up their actions: "we hired Y, but they had a worse quantitative review than everyone else on the team and so was fired." 'Y' has no reasonable recourse, to sue or whatever. Quantitative performance reviews are too valuable for labor relations.
- dbt00 5y agoI like the quantitative review for one reason -- people need to know if they're a 2 or not. Too many managers soft pedal concerns about perf and then people get fired 6 months later not ever knowing they were at risk.
- lumost 5y agoThe problem with this approach is that for any individual who says "I don't deserve a 3 this quarter" there is someone who says "I'm a 4 this quarter or I'll look elsewhere". Managers optimize for the (polite) squeaky wheel and have finite promotions/resources too allocate in a given quarter.
- zamalek 5y agoIt seems as though, at Amazon, managers are optimizing their own quantitative performance reviews.
- nebulous1 5y agoI was surprised by how quickly the author seemed to confuse measurement and goal > First, however, it's worth mentioning that having a goal for attrition isn't inherently bad. In some ways, it's simply acknowledging reality and placing a measurement on what is already going to happen in a healthy environment.
- chipotle_coyote 5y agoThat pairing stuck out to me as well. I think the article is generally on point, but I would have rewritten the first sentence to say "it's worth mentioning that having a metric for attrition isn't inherently bad." But having a goal for attrition is exactly the problem the article's describing.
- nebulous1 5y agoI thought they should have discussed why a company might want to incentivize managers to not be under a URA goal. They discuss why having a URA is "normal" and why it might be bad to incentivize managers to meet a certain URA, but not why a company might choose to so anyway.
- asdfman123 5y agoasdfman123's corollary: the key to success is finding out which metrics upper management is secretly using as targets
- bb88 5y agobb88's conjecture: close personal relationships trump management metrics.
- weeblewobble 5y agoweeblewobble's hypothesis: as an IC, your job is not to hit your deliverables but to allow your manager to hit their deliverables
- asdfman123 5y agoI don't know who needs to hear this, but in management's eyes, it's your job to make them look good. If you want to be rewarded, make them look good, and if you want to be punished, make them look bad. That's it. Fight losing self-destructive battles if you feel you have to, but never forget it.
- sib 5y agosib's lemma: Not just as an IC (but as a manager, executive, senior exec, ...).
- alchemism 5y agoI thought The Sims University modeled it brilliantly, as one could succeed in classes without attending them by hosting study sessions with classmates and becoming a close friend to the instructors.
- ekianjo 5y ago> Goodhart's Law as generalized by Marilyn Strathern: When a measure becomes a target, it ceases to be a good measure Except if you keep that metric unknown to whoever is working on it. The main problem with measures is when they are widely communicated.
- chipotle_coyote 5y agoI strongly disagree with this. I don't entirely agree with Strathern, for that matter. I would argue that (a) you need to calculate metrics that are important to your business, (b) some metrics really are important as targets, and (c) you must communicate those targets to the people whose job it is to help the business meet them. The problem with "hire to fire" as described here is that it fails those criteria (the meta-metrics, if you will). Knowing your company's attrition/turnover rate is good; communicating that metric, at least to managers, is (I would argue) also good. But using it as a target that your managers must not go under is toxic to work culture and employee morale.
- Retric 5y agoThere are zero relevant metrics a company can use that don’t create negative incentives. Maximizing profit, cut R&D. Business is simply complex, you can always sacrifice an intangible like brand perception to tweak some metric. But, it’s the along term that represents the vast majority of a companies value. At least assuming the company is likely to survive the next 2 years.
- chipotle_coyote 5y agoOne of the very first companies I worked at was a local exchange carrier, back in the days those were a thing, and there was supposed to be a certain maximum number of days between the time an order was placed for a new line and the time the line was operational. (These were lines like frame relay drops, fractional or full T1s, etc., so fulfilling the order often required coordinating with other providers, colocation facilities, and so on.) But nobody was actually formally measuring this. I created a spreadsheet which actually calculated the metrics by querying a database for recent orders, showing which ones were still open and how long they'd been open for, and what the distributions of placement-to-fulfillment times were, and it turned out we were only hitting our own internal targets about half of the time. They didn't know that. There was a vague sense they weren't hitting all the targets, but there was no sense they were missing half the targets, in some cases by a lot. Maybe you can make a case that it was a very bad no good horrible thing for me to have done, but I'm going to respond that if you want your company to survive the next two years, you need to be know whether your company is doing the thing it is in business to do. Sure, it's really easy to use metrics to create negative incentives. That's not an argument against using metrics. It's an argument against using metrics badly.
- Uehreka 5y agoI do agree with both of these, but then the question becomes “so then what DO we do?” The options would seem to be: - Openly state what the goals and metrics are, know that they’ll be gamed, shrug and just keep going. - Don’t tell anyone your goals and metrics so they can’t become gamified, breed a culture of distrust where people are afraid they aren’t doing the right thing because they don’t know how they’re being evaluated. Of course, people could always guess what the metrics are and then all the upside goes away. - Set qualitative goals that are evaluated on an “I mean what I mean, you know what I mean, right?” kind of basis. This can work optimally when a team is small, driven and tight-knit, but usually explodes when you try to scale it. It’s not the responsibility of these philosophers to provide solutions, merely pointing out flaws is important enough. But for those of us trying to learn from those philosophies and then build things, we then have to figure out something to do.
- smaddox 5y agoHow about: - give employees and managers enough context to understand the business goals and empower them to make intelligent decisions and affect positive change