29 ms·

A good measurement culture where numbers don’t replace common sense

• ianai 3y ago
  I’m reminded of the philosophy of science, I think. Math in general is very powerful and effective at modeling the phenomena of the universe. But it’s generally not enough to simply show math notation to demonstrate knowledge nor come up with scientific hypotheses. Proofs can’t be written solely in the language of the system. There’s got to be some base intuition to a hypothesis stemming from some prior experience of the phenomenon under consideration. (Yes, much of the standard model and modern computational science muddies this point. That’s illustrative and worth contemplation. Obviously, such are only as good as their data and calculations agree with reality.)
  • opensource4ever 3y ago
    in academia that is divided into quantitative vs qualitative. While most of my earlier few decades of my live have being focused on quantitative part, more and more as I age do I begin to value the qualitative part more and more. Often performances of company, organizations and individuals are not measurable in quantitative ways.
• mrweasel 3y ago
  The best framing for this is still: "You get what you measure". So if your have a KPI that says 20% growth in user base per year, you make it easy to signup, hard to leave, but frequently forget to measure activity, user satisfaction or profitability. I worked for a company that wanted some growth in business, so they started to sign contracts for jobs for which we had little to no qualification. We got new customers, more business, more work, but employees started to leave or burnout.
  • hackerman_uwu 3y ago
    Along very similar lines… There is a saying in BI that goes: “What you measure grows.” So let’s say you measure MAU, the infamous vanity metric. And let’s say your MAU increases with each dollar you funnel toward advertising, the thing you’re actually growing there is your dollar spend and not the growth of the user base. Thus it is your advertising dollar spend that will grow and not the size of your user base.
    • phalf 3y ago
      Your comment would be significantly more accessible to the general HN crowd if you explained what BI and MAU stand for.
      • swores 3y ago
        Business Intelligence and Monthly Active Users
      • deleted 3y ago
        [deleted]
• nelsonic 3y ago
  Goodhart's Law: “when a measure becomes a target, it ceases to be a good measure.” https://en.m.wikipedia.org/wiki/Goodhart%27s_law https://en.m.wikipedia.org/wiki/Goodhart%27s_law
  • dpq 3y ago
    It sounds like a profound wisdom and on a very surface level it does make sense, but think of this: if you can assess that a measure is a bad one, this means that you have your own intrinsic preferences, otherwise you wouldn't be able to tell that! Therefore, if you are unhappy with a measure, it means solely that it doesn't capture all of your preferences properly. Which is a technical problem rather than a philosophical one.
    • OJFord 3y ago
      It ceases to be a good measure: it captured your preferences properly prior to becoming a target.
    • digitalsushi 3y ago
      good == honest in this context
    • constantly 3y ago
      What your saying sounds like profound next level wisdom and on a very surface level does make sense, but think of this: an org can create a measure that captures the relevant preferences properly, and everyone is quite happy with the results because the team continues or slightly modifies what they’re doing and lifts up the product to lift up the measurement and life is good. And then over time they start realizing they don’t have to lift up the whole product, but just a small piece to increase the measurement. So they do that and the measurement goes up, but the product doesn’t get better because they’ve found the path of least resistance to raising the measure. This is really the underlying crux of Goodharts Law. So it was a good measure probably until it became a target. So what is a manager to do? “Capture all [the] preferences properly” as you put it? Probably not because that quickly devolves from measurements to long form status reports, not even measurements, because it’s impossible to capture all dimensions of this with measurements, so one has to reduce the dimensionality a little. This is a philosophical problem not a technical one. Though your point does seem superficially correct, in practice with real teams the second and third order effects from the measure becoming a target dominate.
• klooney 3y ago
  Seeing Like A State is the most important business book.
  • datadrivenangel 3y ago
    Legibility even at the cost of the things you want!
  • philipodonnell 3y ago
    The forestry allegory is so relevant to what we go through in enterprise data. I recommend this book all the time to understand the motivations behind it.
• hliyan 3y ago
  In the last engineering team I managed (by far the most successful one up to date), we discarded most velocity measures in favour of a simple solution: every Friday, each team sends a brief "here's what we delivered this week" email to the whole company. It contains some screenshots and instructions like "agents can now update the foobar rate from their mobile app without having to call in". After a month or two of these going out like clockwork, it gave both management and business stakeholders a level of comfort that no amount of KPI dashboards ever could. KPIs compress away details that stakeholders need to form an understanding. A full blast from the task tracker is overkill. This solution worked for us. But of course, required a solid feature flag practice so that half-built features (story slices) could be released weekly for testing without affecting production. That said, we did maintain a quarter-level sum-of-t-shirt-size based KPI.
  • cheschire 3y ago
    Was your sum KPI burned down on a report during the quarter or was it only reviewed once the quarter was complete?
    • hliyan 3y ago
      I did keep an eye on it because it was a good indicator of problems. But I never drove the team based on it, any more than I would try to increase a car's speed by grabbing the speedometer needle.
  • pydry 3y ago
    I like how the most convincing examples of "agile transformations" almost never use the word agile.
    • lloydatkinson 3y ago
      In those cases (the majority) the word Agile is used to describe it. Not actually agile, just whatever the hell Agile is. https://www.youtube.com/watch?v=a-BOSpxYJ9M https://www.youtube.com/watch?v=a-BOSpxYJ9M
  • 3y ago
• mkl95 3y ago
  > Now your course of action will not be optimization of server-side processing, but something else (e.g. deploying in that region or on-premise for those customers). And to empower your engineers to achieve that, you should - Train them so they gain specific knowledge on whatever tech they are going to use. Most likely one of the big three cloud providers. Some companies avoid this step because they want to invest the time and money required. Often with severe consequences when engineers deliver a botched solution. - Make it a realistic goal e.g. you may need to create a new team with a smaller scope, so those people are efficient and not distracted by other stuff. If Mary from accounts is pinging your engineers about some legacy system that keeps failing, do not be shocked when "your estimates were wildly inaccurate".
• xg15 3y ago
  Is time-to-last-byte really defined as the moment the initial response has been successfully received? Or is it defined more vaguely as "all responses needed to render the page have been received "? I'd understand "time-to-first-byte" as something that could at least give you a reliable lower bound for how long users have to wait, but I don't see how "time the initial response has been received" is any kind of meaningful indicator of UX in modern javascript-driven websites.
• Ecstatify 3y ago
  Original Title: How to avoid KPI psychosis in your organization?
• culebron21 3y ago
  Having economics degree, I figured this out of evolutionary dynamics theory -- a more sophisticated creature will outplay a less sophisticated (in this case it's an employee with creativity against a manager that just uses KPI). Later I found out this rule was articulated in mid-XX-century in sociology -- the Goodhart's law: "a metric stops being a good metric as soon as it is used to make decisions".
• fhd2 3y ago
  I wish more companies would be able to work with KPIs as _one_ part of the equation, rather than the be all end all. I haven't seen a lot of that :( In my experience, data driveness quickly leads to an unhealthy focus on short term results - cause that's usually what's most straightforward to measure and impact. What you get is teams hunting local maxima. What's missing is the guts to follow a vision/intuition regardless of what the numbers say, at least some of the time. It seems hard to accept that we can't measure everything that impacts the long term success of a company, but I don't see why that typically translates to not caring about it altogether.
  • tomrod 3y ago
    The general approach is: performance, when measured, improves; performance, when measured and reported, improves exponentially. KPIs might be defined incorrectly, but their use is simple and straightforward for people to understand. This gives rise to Goodhart's Law, "when a measure becomes a target, it ceases to be a good measure." The best KPI is something that can't be gamed by those whose performance is being measured. This is probably impossible in practice.
  • pipo234 3y ago
    Managers like dashboards. KPIs are dashboards. Your car dashboard is great for staying within speed limit, but now try driving without taking your eyes from the speed dial. :-)
    • midasuni 3y ago
      My car has a HUD which puts the current limit and the speed I’m doing on the windscreen
    • stefanka 3y ago
      great analogy!
      • phalf 3y ago
        It's even great in the sense that a speedometer tends to make people aim for "I need to go 65 mph" instead of "I should not go above 65 mph" which a speed limit is actually intended to mean. The speed limit became a target instead of being a limit.
• not_the_fda 3y ago
  Do people still try to do KPI? I haven't worked in an organization that has done that in ages. Its so well known to not work and backfire, its a giant red flag if an organization implements it.
  • kaoD 3y ago
    What I noticed is fashionable now are OKRs. Never seen them work correctly though.
  • sarchertech 3y ago
    Oh it’s still filtering its way down through the big companies. Most startups eventually hire someone who had experience with metrics driven management at bigco who starts selling it as a panacea.
  • gardenhedge 3y ago
    Agree with the red flag. I'd hesitate to join a place led by KPIs
  • cableshaft 3y ago
    From my limited experience they seem to have shifted to OKRs, which are pretty much the same thing with a different name, at least for the companies I've worked for. I hate it just as much. Currently two of my OKRs for this quarter based on what I thought a project was going to be and have since discovered I was wrong, so they don't actually matter and I'll have limited opportunity to achieve them. Great, so now they get a fun data point can point to in order to give me a smaller bonus next year.
• nine_zeros 3y ago
  Managers lurve to have standard KPIs that they could apply to all engineers (for the purpose of fairness of course) and then be able to rank engineers. At the same time, they forget that not everyone, not every team is working on the same thing. So the KPI (whatever it is defined to be) is not measuring anything of value. But a manager will never admit that they are measuring the wrong things. So engineers are stuck with poor practices. Go improve your number of commits even if is meaningless.
  • malfist 3y ago
    One of the things Amazon measures for engineers is revisions to PRs. This makes since at a very superficial level. If you're a better engineer, you make the PR right the first time. Except what happens is either engineers game it by just canceling PRs, making the requested change and resubmitting it. Or fighting tooth and nail for everything. No I won't change the naming on this variable because then I might get out on PIP. Or even more insidious, the team just doesn't do real code reviews, they just rubber stamp each others work in a "you scratch my back, I'll scratch yours"
    • nine_zeros 3y ago
      An Amazon engineer (also other FAANG engineers) spends an insane amount of time on gaming an adversarial system. If it were not for the unrealistic anti-competitive moats of these companies, these management practices would have burned them to the ground. Startups that have copied these practices without giving FAANG compensation have faded away in history.
  • quickthrower2 3y ago
    The problem with capitalism is it works too well: a company doing well can subsidise a lot of bad management practice and still do great. And because it is doing great they believe what they are doing is great. But it is largely a snowball effect. Talking about larger companies more than startups here.
  • marcinzm 3y ago
    If a company wants to rank engineers then there is always a metric to rank engineers. That metric may simply be how much the manager or director likes each engineer. That's still a metric and it still can be gamed (ie: classic kiss ass culture).
• NTDF9 3y ago
  Leaving this not-so-humorous-post https://medium.com/@NTDF9/if-soccer-managers-did-performance-management-like-tech-companies-69c54b587386 https://medium.com/@NTDF9/if-soccer-managers-did-performance... Full disclosure, it is my own rant about performance management.
• LaundroMat 3y ago
  The most vivid (and horrific) example of what relying too much on metrics can lead to are Robert McNamara's policies during the Vietnam War. [1] Also, because defenders of KPI-driven management love to use the saying that "you can't manage what you can't measure", it's worth repeating the original quote where this is lifted from: > "It is wrong to suppose that if you can’t measure it, you can’t manage it – a costly myth." (W. Edwards Deming) [1] https://en.m.wikipedia.org/wiki/McNamara_fallacy https://en.m.wikipedia.org/wiki/McNamara_fallacy
  • infinite8s 3y ago
    That's the original quote? That's the opposite of "you can't manage what you can't measure."
    • LaundroMat 3y ago
      Exactly. Most people I witnessed saying this never even heard of Deming.
    • lcnPylGDnU4H9OF 3y ago
      I haven’t heard either quote but a quick search yielded this: https://deming.org/myth-if-you-cant-measure-it-you-cant-manage-it/ https://deming.org/myth-if-you-cant-measure-it-you-cant-mana....
    • marcosdumay 3y ago
      It's certainly not the origin of "you can't manage what you can't measure". For a start, Deming is almost a century too young for that.
      • LaundroMat 3y ago
        Ah, that's interesting. I always thought it was a selective quote lifted from Deming. Can you tell where it originated? Taylor, or even before?
  • Spooky23 3y ago
• steveBK123 3y ago
  I hate this stuff. I generally work on internal platforms within tech orgs, and we periodically get this shoved down our throats. Tech management wants us to come up with KPIs, despite not having any of their own for us to use for alignment. Our product guy who drives our roadmap doesn't want to put any work into KPIs either. So instead some second-in-command on the team has to come up with some KPIs that they will have no control over addressing (they aren't team lead, they aren't product, they don't own the backlog or roadmap). Inevitably there is a bunch of angst from management that the KPIs are too tech/engineering centric (well..) while again providing no actual good KPIs themselves, and then they all get missed anyway since no one is incentivized to address them.
  • malfist 3y ago
    Oh it's even worse when senior leadership gives you KPIs. I remember when I was a fresh face college grad working at Dell Services. One of my KPIs was about selling more headcount to the client. None of the KPIs that came down from on high made any sense. I could be the best junior engineer in the company and it would have no impact on my performance review.
    • steveBK123 3y ago
      Irrelevant management KPIs at least tell you what management cares about. "I'll know it when I see it" iterations of bringing different "no not that" KPIs to management is frustrating in the opposite direction. Ultimately management wants to set the tone in both scenarios, at least when giving you dumb KPIs they have done some work and volunteered information.
• malfist 3y ago
  I really think their cartoon illustrated the point of the article quite well. For those visually impaired or who didn't read the article it was a two panel saying: --- Boss: John, your commit frequency dropped 25% last week. Are you quiet quitting? John: Sorry boss, I was on sick leave. I'll be more active. --- Before you cry "this is a contrived example!" Look at what Amazon just did. They sent out an email threatening folks who weren't in the office 3 times a week in the previous two weeks. I did not take into account sick leave, vacation, FMLA, corporate travel, anything. You take a vacation? You and your manager got a threatening email about your KPI you're not meeting.
  • datadrivenangel 3y ago
    I know multiple people who have worked at companies where KPIs were not adjusted for paid time off... 2 week vacation means your numbers are going to tank, which will reduce bonuses and opportunities for promotion.
    • malfist 3y ago
      I had a friend that worked at deloitte and that's exactly how they managed it. You had to bill so many hours a month and if you took time off, you had to make it up when you got back (or before you left). He was absolutely miserable there.
• drawkbox 3y ago
  Metrics don't tell the full story and should only be a read, not the direction. Project management and customer needs or product needs should be the driver. We are at the point now where the KPIs have the wheel. If they were just called metrics in most companies and not Key Performance Indicators (KPI), then they would get less focus. KPIs has been a heavy consultant pushed term but they are just metrics, they should never been seen as how to create value and develop a product. People love to say "KPI" as much as "Agile" and "velocity" nowadays, both have been horribly twisted and ruin agility and value creation over value extraction. They need to be generic again... metrics, agility, delivery and usability. When metrics are all you measure, you end up with problems [1] [1] https://en.wikipedia.org/wiki/Performance_indicator#Problems https://en.wikipedia.org/wiki/Performance_indicator#Problems
• elric 3y ago
  I would suggest that the problem is not KPIs (or Agile, or whatever else is ruining your particular company). It's shitty management. KPIs (and Agile, and $buzzword) can be useful, when done right. If you get hit in the face with a hammer, is the hammer the problem? Or the idiot who swung the hammer at you? Time and time again I've seen managers come up with a Great Idea(tm). It works for a little while. And then it turns into a shitshow because it goes from being a useful idea to being dogma.
  • rightbyte 3y ago
    Before Agile management didn't have a hammer to swing at you in the first place. Micromanagement required actual work. Agile has automated micromanagement for management ...
    • elric 3y ago
      Hmm, you might have a point there. Still, it's just a hammer. It's blameless. I guess these management fads just make it easier for lousy managers to do damage?
      • jq-r 3y ago
        Exactly. I would even say that hammer is a too nice of a word for what I would call a whip. Or even a rope. Lousy management gives you a rope which you can hang yourself with. That way they don't have to bother to get their hands dirty. For the life of me I couldn't really understand why 9 months ago when a new CEO came to our company and first thing he said "measure everything, every team should have public OKRs", everyone from my team jumped on that OKR/"metrics" train and was hyped to have it. I was the only one against it, but that didn't make any difference. Two quarters later it was pretty obvious nobody gave a shit about those (we had much more important stuff to do) but our manager had to apologize to upper management quarterly for "poor performance" of our team. And informally, nobody was taking any of those OKRs seriously, and colleagues thought of it as a bad joke. I have no idea why the same team members didn't voice their opinion immediately and shot that idea down while it didn't take root yet. Now half of the company is getting laid off behind a thinly veiled excuse of a RTO (from a company which was remote first since the beginning, and much longer than covid years) and everyone is a surprised pikachu. I'm not saying this is solely because of the OKRs, there were more red flags besides that one. I'm actually surprised that very few saw it coming. That the CEO is trying to emulate Musk and do a Twitter speedrun, but with someone else's money. And blame engineers who actually made the company as known as it is.
• costanzaDynasty 3y ago
  I worked at a giant mega corp as low rung salesman and the major KPI was sales numbers. Until about June when if sales were looking bad region wide it turned to waste(returns) as the major KPI. If you knew how to straddle the line, by about May you could pivot to the KPI of the moment. Every year some new hire would go out and sale 15% of sales plan and would be praise as a superstar. The next year they'd be written up for some bs reason and after a few years they would be flushed out. What did they do wrong? They out performed sales plan in the first year by 15% and that meant that now every other year their plan for the year would add 5% growth on top of the 15%. Since the initial 15% growth was unsustainable the waste grew and magically they were now failing at the 2 major KPI's.
  • mmcconnell1618 3y ago
    I've witnessed this far too often in sales. First, a team lands a giant deal that is far above plan and celebrates with Club status and a trip to a tropical Island. The next year the algorithm decides they need to get that deal + 30% more out of the same customer. There is literally no new workload to sell and the team underperforms plan despite a massive run rate from the original deal.
• neom 3y ago
  This is a great article. I've consulted with many startups that are really scared of OKRs and KPIs and have some PTSD around them, but they need not be scary! Here is a scenario: It's January and Sales is getting feedback from customers that if the business has a managed k8s offering, they would not only buy it but expand their use of the platform generally. Sales passes this information to product and product does some research, finding that indeed there would be a market for a managed k8s offering. They pull together a report and start to shop it around to get feedback and buy in from around the company, consulting with different orgs. After a while, a business case is flushed out and the management green light an MVP k8s offering that will generate revenue by the end of the year. An objective and key result for the business becomes: this year launch a managed k8s offering so a small number of customers with an eye for $50,000 MRR. Every org should then have some performance indicators that indicate that the business is tracking towards the release of this product this year, it doesn't not need to be complex just as the OKR should not and does not need to be complex, it's simply an indicator that we're on track (and it's ok if you're not, maybe the OKR was unrealistic, that's ok!) Eng: KPI: stand up k8s - KPI: make k8s stable - KPI: make k8s more stable - KPI: tie k8s into the rest of the stack - etc Product: KPI: Research how k8s should be be exposed - KPI: Research how k8s service should be billed - KPI: MVP product design and test internally - etc Support: KPI: understand staff to support a new offering - KPI: Training and hiring for new staff to support the offering - etc Marketing: KPI: understand the message to the market - KPI: understand how competitors are marketing their k8s offering - KPI: understand how to advertise new offering - etc Etc - etc. OKRs and KPIs do not have to be a whole song and dance, they are simply a way to keep a company of organizations aligned and moving in the same direction regarding agreed upon objectives for the business, folks get super hung up on granularity. Doing this well helps tie together some of the most important aspects of a healthy and successful go to market: Vision, Mission and Purpose. I understand the vision of the business, the things we will make true in the future. I understand the mission of my team and organization on how we will realize this vision. I understand my purpose within the vision and mission and why I am an important component. I firmly believe it's only with this as part of your foundation can achieve employee satisfaction, have directionally forward teams, and become a "high performing business".
  • candiddevmike 3y ago
• mathattack 3y ago
  KPIs without judgment are bad because you can’t model everything. Judgment without measurement is also bad because we all have human bias. It’s a matter of where you lean first. (Do you check your model, or check your judgment?)
• 6stringmerc 3y ago
  KPIs are useful in context - to echo other valid points given thus far. In a large scale business plan or proposal typically an understanding of what is to be performed can be measured. SLAs for uptime for you cloud dwellers. Are SLA metrics everything? Also, when KPIs were hit in a time frame, who talks about 20% of certain teams not being there anymore? I do agree past performance is not a guarantee. It’s kind of a dirty secret that yes this sales document is truthful but it represents conditions not current. What is really destroying business in US corporations is woeful understaffing in “gets things done” positions. Mostly due to Wall St metrics. Book the sale then revenue and bonus…then get out the kitty litter. It’s been a race to the bottom for 10 years in professional services and lower tier positions by the dozen. Of course the new fad of “AI will make this more efficient with fewer people!” appeals more than “wow we need bodies to move figurative data boxes around…why don’t they want shit wages limited healthcare and terrible hours?” when talking to investors. This is simply a problem of engineering capital to Human Resources but the invisible hand has been cuffed for so long there really is no middle class anymore. The rich don’t spend. The poor spend all they have and because milk is fucking $5 a gallon, gas is $3 here in Texas, and bills ain’t getting cheaper during 112 degree days - wanna know why the economy sucks? When in lockdown basic income to not die or be evicted didn’t ruin society. Now that we’re out of the pandemic it’s forgotten. I’m sure lobbyists will somehow change the lesson to be “we need to buy guns from companies and give them out free that will solve poverty” and Ted Cruz will be right there eating bacon off their… Businesses are self destructing. Not my monkey not my circus. Cassandra out.
• wyuenho 3y ago
  It appears to me this is another case of the slow evolution of European businesses and management practices. There exists a practice called OKR. Intel has been using it since the early 70s, Google since year 2, and pretty much everyone in SV has switched over since around the time that John Doerr book came out. It’s designed to solve exactly all the problems of KPI. You shouldn’t spend much time figuring out what to measure and whether you are measuring the right thing or how to put what you measured into context. You should be defining simple objectives and key results what will give you a binary answer at regular intervals. Practicing KPI solely is putting the cart in front of the horse. You should know your objectives and milestones before you figure out how to measure them.
  • AlexandrB 3y ago
    This "just use OKRs" comment is funny when contrasted with another comment that described how OKRs were "gamed" at his company: https://news.ycombinator.com/item?id=37221787 https://news.ycombinator.com/item?id=37221787
    • wyuenho 3y ago
      It’s even funnier when it’s apparent that company isn’t really doing OKR. First of all, the Os are supposed to be somewhat resistant to change in direction. Second of all, you shouldn’t be doing KPI on top of OKR. If you set the OKRs right, you’ll set stretch goals so ambitious, even if you game the completion percentage, the simple answer to whether you are doing the right things or performing should still be immediately answerable.
      • andrekandre 3y ago
        > set stretch goals so ambitious, even if you game the completion percentage, the simple answer to whether you are doing the right things or performing should still be immediately answerable. any good examples of this in practice?
• kriro 3y ago
  Clickbaity title submission. The actual title of the blog post is "How to avoid KPI psychosis in your organization?" and it talks about why blindly following KPI can be bad, that humans are good at hacking numbers etc. and basically says "don't use them blindly". It's also framing it in the context of tech businesses not all businesses in general. So not sure how to jump from that post to the submitted title of "Why KPIs are destroying businesses". KPI can be very useful as sanity checks and it also varies quite a bit by industry if and when it makes sense to lean more of them. Anything involving physical goods/supply chains usually benefits greatly from having good KPI in place. Same for financial information. The "I" stands for indicator, I think it's usually a good idea to set up some KPI with boundaries and alerts to resteer the ship if anything goes badly off the path (cash flow indicators for finance for example).
  • _8j50 3y ago
    Basically McNamara fallacy but more subject specific: https://en.wikipedia.org/wiki/McNamara_fallacy https://en.wikipedia.org/wiki/McNamara_fallacy The KPI is a measurement tool not a goal. If you run a race, your goal should not be to beat ussain bolt's record, your goal should be winning with your best possible time which could or could not be much better than usain's. It is your coach who is supposed to measure you and observe how you are running and training and help you adjust that based on his/her expertise on the subject that is supposed to use usain's record as a measuing stick as they help you adjust and adapt to break the record.
    • Karellen 3y ago
      Cool. I'd not seen that one before. I was going to link to Goodhart's Law, but the McNamara fallacy seems broader and more interesting. Thanks.
    • staplung 3y ago
      This. Step 1: measure only that which can be easily measured Step 2: ignore what can't be easily measured Step 3: assume that which cannot be easily measured is unimportant Step 4: assume that which cannot be easily measured is not even real Plus yeah, also Goodwin's Law.
• nprateem 3y ago
  Business strategy -> objectives -> KPIs. Don't start at the wrong end.
• shortrounddev2 3y ago
  I saw this at my last company. We had to set 3 OKRs (goals) per quarter, and each OKR had to be measured by 3 KPIs. Changes to these after the first week of the quarter had to be approved by our managers. Our CEO had the philosophy that if you achieved 100% of your goals every quarter, you weren't being ambitious enough. As a result, we would stop logging progress at ~80% of our goals, which was the threshold to qualify for your bonus However, scope on projects would change during a quarter, and sometimes the VP would become distracted by another shiny new thing and make us start on his new pet project. As a result, some OKRs were abandoned and would remain at 0% by the end of the quarter. Now, no manager (especially an engineering manager) wants to be a bad guy with his employees, so my boss often just changed our OKRs (or, if we only achieved 50% of our KPI, would lower the KPI's target) on the last day of the quarter so that we would all qualify for a bonus. The company was losing 18mil a year and was sold to our competitor at a 100mil loss for the parent company
  • itsoktocry 3y ago
    >As a result, we would stop logging progress at ~80% of our goals, which was the threshold to qualify for your bonus I feel like you're stating this as "Look how silly KPIs are!", but your CEO is correct. The issue isn't the KPIs; it's the team members who are willing to intentionally game a system in conflict with overarching company goals. There's nothing magical about KPIs. It's data. How you apply it matters. If your team wants to take advantage of it for selfish reasons, there's nothing stopping them.
    • bicijay 3y ago
      Someone gamed a system created to be gamed? :pikachu_face:
      • bluGill 3y ago
        The real failure is the CEO didn't make a system where by gaming it you made things better for the company. This is a hard problem - but CEOs are paid big bucks to solve hard problems like this.
• anoy8888 3y ago
  Another hard question is how do measure people performance since kpi is not the best tool ?
• dalbasal 3y ago
  Everything about KPIs in this blog, Goodhart's law, the comments... it's all true. That said, I'm not sure KPIs themselves are the centre of this problem. A national olympic team (or any competitive sports team) using KPIs probably would not experience these problems. They might get over-exuberant and over optimise slightly. Athletes might game the system slightly. But, it's unlikely that a professional sports team degenerates to levels of KPI affliction like an average corporation. No matter the corruption, olympic teams are made up of people who really want to win medals. Athletes, Coaches. Dieticians. Managers. Administrators. Etc. They're not angels, they're just not as focused on appeasing their boss, managing up, and such. Group size plays a role. C-level distortion fields play a role. The orgs age. etc. Legibility. "Market" feedback loops. It all ads up to an organisation that can take any management tool and make it pathological. A commercial example of resilience to bullshit is a classic factory, that ideal "Firm" we based out economic models on. "The job" of manufacturing objects is legible, can be effectively reduced to component parts, measured and optimised. The feedback loop any significant successes and failures is effective. A young startup with a "lets make it work together and we'll all be rich" vibe can probably use KPIs effectively. There are lots of example, and we intuitively know how this is going to go. I think it's time to look at some microeconomic questions with the reverse of conventional logic. Why does the financial services sector have sop many employees? Not because of the marginal profit value of the least employee. Because of the revenue. A company with $Xbn and a fat profit margin is (often) going to employ lots of people because it can. At this point, all you need is to lower efficiency to achieve the required output.
  • ElevenLathe 3y ago
    The sports team is an interesting case. Many fans (I know the details for MLB, being a fan myself, but the same is true of other sports) feel, essentially, that teams have optimized the wrong KPI: pressure to win has made them emphasize pitching to the point where there is very nearly a pitching change every inning in most games. Meanwhile offense values home run hitters more than ever. The result is more competitive teams but less entertaining games: much of the drama of the game is close plays on fielded balls, while managers, scouts, players, and back office analysts are doing whatever they can to avoid plays like that. Yes, the team can make the playoffs, but ultimately baseball is an entertainment product -- the goal should be for the games to be fun to watch!
• switch007 3y ago
  I wish CEOs would stop flying long haul just so they stop having the time to read the management book du jour and inflict this silly things on the entire company.
• HughParry 3y ago
  Newest KPI is number of KPIs per month (lower is better). Lets circle back to that discussion later in our OKR chat this Friday
• thesuperbigfrog 3y ago
  People will find ways to game any system to maximize their score for what is being measured, particularly when there are attractive rewards for high scores. This leads to perverse incentives where what the organization's leadership really wanted is set aside in favor of maximizing scores by whatever means necessary: https://courses.cs.duke.edu/fall01/cps108/code/jpixmap/images/dilbert/bug.gif https://courses.cs.duke.edu/fall01/cps108/code/jpixmap/image...
• octacat 3y ago
  KPI/OKR stop working once workers starting to optimize for KPIs, ignoring stuff like quality of code or the mood in the team (is it supportive or is it till burnout?). Especially cool when companies put generalised goals, and the product development has different goals. For example, if we wanna improve the loading time of the site overall, we could just completely ban the slowest clients (problem "solved", KPI improved).
  • ethanbond 3y ago
    This is why experienced managers should consider, “what’s a way we might superficially achieve this while yielding neutral or negative actual results?” and then set up more KPIs to detect that happening. It’ll never be perfect, but it’s probably better than nothing. Company-destructive assholes and metric-gamers can run their playbooks with or without KPIs or OKR:
    • octacat 3y ago
      Who has said metric-gamers would be so obvious? "Add more KPIs" - yea, only benefits the metric-gamers actually. The regular workers would be more annoyed and distracted. Do we focus on the sprint and tasks in it? Or do we focus on 10 new KPIs?
      • ethanbond 3y ago
        Well if you want to just complete tasks then go ahead and do that. If you want your business to succeed it’s pretty important that the task-doing is producing positive results for the business overall (which is what KPIs imperfectly try to measure). “Ha, you want a brake and a steering wheel? Those are annoying and distracting, do we push the gas pedal or not?” Yeah it’s complicated but all these components are necessary.
• joshstrange 3y ago
  KPIs are such an incredible waste of time. At my last job out team was tasked with coming up with 5 KPIs, that alone took many hours of discussion (times 6+ people) and then to implement the KPI reporting we would have to start logging/recording a ton more data (manually, in JIRA). On top of that we had to write code to pull that data from JIRA so it could spit out the KPI number. After all of this I once again pointed out how useless and stupid the whole process was and how easy to game it was. We could easily lie in our scripts if we wanted to, it wasn't like management was capable of checking, and it was going to encourage reclassifying/hiding certain bugs (hard issues) and pretending other things were bugs (small changes/minor feature requests) to juice the numbers. The whole processes took a _ton_ of time and in the end I never heard another department's KPI numbers (we were told every dept was doing it and we would all report monthly for the whole company to see) and I'm fairly certain it fizzled out though I left shortly after that.
  • itsoktocry 3y ago
    >We could easily lie in our scripts if we wanted to, it wasn't like management was capable of checking, and it was going to encourage reclassifying/hiding certain bugs (hard issues) and pretending other things were bugs (small changes/minor feature requests) to juice the numbers. But why would you do this??? Holy cow, some of these comments. As someone fixing bugs, why would you not be interested in the classification, status and tracking of bug-related data? Why would your first instinct be to lie about what's happening with the business? To what end? Honestly, this thread has me wondering if the average worker does so little that they see KPIs as a way of being accountable, and don't want that in any manner.
    • polonbike 3y ago
      I think op is saying that he is gaming the stupid system, to be a bit less stupid and unfair. Not that he wants to cheat or not fix issues. Classifications of issues is indeed important, but if management is not playing a fair game, then neither am I, while still doing my job, tracking and fixing bugs efficiently
    • mholm 3y ago
• RecycledEle 3y ago
  Why not make the KPI mirror what you want from the employee? Here is a book that makes that quite clear. Adam Smith published it in 1776. This is not a new idea. https://www.gutenberg.org/ebooks/3300 https://www.gutenberg.org/ebooks/3300 I remember a recent story about Ziff Davis killing old web pages. If that is KPI psychosis, the fault lies with the person who set the KPI to be average page hits, not total page hits across their web footprint.
  • bluGill 3y ago
    The problem is what you really care about is the bottom line. For just-in-time manufacturing this isn't too hard as you make it and sell it at about the same time. However engineering often takes years of work before you can sell anything, and until you do you have no idea how successful you will be, except for proxy measures which might or might not actually line up with long term success. Of course there is a lot of in between.
• fanf2 3y ago
  I like this quip, tho QI says it is due to William Bruce Cameron: “Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted.” https://quoteinvestigator.com/2010/05/26/everything-counts-einstein/ https://quoteinvestigator.com/2010/05/26/everything-counts-e...
• m3kw9 3y ago
  KPI's help PM's stay relevant easily by having them talk sht about KPI's that no one understands but them.
• siva7 3y ago
  KPIs have always been there. They just had before another name. Businesses are still alive.
  • the_af 3y ago
    Mind you, the article never states that "KPIs are destroying businesses". That's just the clickbait title HN for some reason chose for TFA, not the actual title.
• SilverBirch 3y ago
  I'll bang my old drum once again: There's nothing wrong with KPIs, the problem is with the organisation that introduces KPIs. Companies that are doing well and succeeding rarely bring in KPIs, because they don't need them, they already have a system that works. It does sometimes happen when some ambitious young fellow decides he wants to be a manager or be known for IMPROVING PROCESSES! But really, mostly KPIs are introduced because the business side of the organisation isn't happy with what the engineering side of the organisation produces, so they start to try and come up with ways to measure engineering's failure. It's manifestation of a struggle between two parts of the business.
• bregma 3y ago
  This is a corollary of the phenomenon I've dubbed "spreadsheet reality". I have noticed that ever since the advent of the spreadsheet the numbers in the pivot table are more real to decision makers than what's going on outside their office door. It's just so easy to tweak a few numbers in a cell in VisiCalc or its descendents to get the results you're looking for on the bottom line. So what if that means destroying the lives of a few thousand (now former) employees? The development of the spreadsheet was the beginning of the end.
• the_af 3y ago
  The title HN chose is puzzling. It's not at all the title of the article, which is far less negative about KPIs: "How to avoid KPI psychosis in your organization?". And, in fact, the article talks about how to use KPIs with common sense, and never claims they are destroying businesses. I wonder what's the reason for this clickbait title?
• drewcoo 3y ago
  > KPIs should be used in combination with human intuition And that's where everything goes from wibbly-wobbly to "I've fallen and I can't get up!" What is "human intuition?" Do we all have it? Do we have the same "it" or is this just something we accuse other people of not having but that we can't quite define, like "common sense?" This statement may as well be "ideally, we should use KPIs with ample hand-waving, for some unnamed value of 'ample.'"
• divan 3y ago
  One of the best articles on this subject - Goodhart's Law Isn't as Useful as You Might Think by Common Cog. Good overview of Goodhart's effects and particulary effective practice of fighting it at Amazon from "Working Backwards" book. https://commoncog.com/goodharts-law-not-useful/ https://commoncog.com/goodharts-law-not-useful/
• mhb 3y ago
  For the handful of dullards like me: KPI = Key Performance Indicator https://en.wikipedia.org/wiki/Performance_indicator https://en.wikipedia.org/wiki/Performance_indicator
• rdsubhas 3y ago
  I would add a root cause to this: KPIs are fine when treated as health indicators. You can deliver a feature, and then watch over a good period of time whether the health improves. This "good period of time" can be months, or for something that's intuitive – e.g. you're launching a feature to be on parity or enter a new market or catch up with a competitor – this can take YEARS for your customers to understand and move closer to your product. What backfires is, using KPIs as quarterly goals. Imagine this, you launch a feature and expect within the quarter that this feature is going to grab X% of your competitor market share (beautifully expressed as "increases X business by Y%"). This is where it starts to fall apart. Things that are human intuitive for the product (common sense market features, foundational work, new market launches) – will never get prioritized because they won't move a number in that quarter. Using KPIs as development cycle goals are destroying businesses. KPIs are fine as health indicators. The problem is, using KPIs as health indicators – works only with the kind of environment where: Teams after healthy discussions can trust and commit to delivering features – and somebody takes accountability for the KPIs asynchronously behind the scenes. Where a shepherd steps up and says, "I approved the features last year, I take responsibility if they all didn't change our trajectory, some did, some didn't". Today's adoption of KPIs is an "empowerment" trend, where managers figured out – teams are delivering features and I'm answering KPIs, why do this roundabout, let me just pass on the entire KPIs to the team, problem solved! "I forwarded the KPIs to my team and they committed to doing these features, I did my job, the team is not performing". The path to hell is paved with good intentions, but there are also nefarious habits.
  • jandrese 3y ago
    “When a metric becomes a target it fails to be a good metric” https://www.mrowe.co.za/blog/2019/11/when-a-metric-becomes-a-target-it-fails-to-be-a-good-metric/ https://www.mrowe.co.za/blog/2019/11/when-a-metric-becomes-a...
• PaulHoule 3y ago
  I worked at a place where I thought OKRs and KPIs were a big problem, a much bigger problem than this article describes. This was a startup that was struggling to find product-market fit. Things were generally disorganized and there wasn't a lot of discipline. I couldn't get the data scientists to use the same version of Python or use version control in a disciplined way. The engineering manager told me about a large number of practices that we allegedly used in our code and also told me with did code reviews but when I got involved with that code I found those practices were not being applied and I can't imagine we were really doing code reviews or we would have known. The most challenging thing I noticed was that we were doing a lot of zigging and zagging to keep up with our enterprise customers who had very different projects. Some of that was intrinsic to what we were doing but it also meant we'd spend a lot of time developing something and then abandon the work which had us spinning our wheels. What we did need, in my mind, was the discipline to deal with that situation, when I am working on my own things like that don't bother me much because I take good news and usually do a good job of restarting projects that are interrupted but my team was not so good at it. My impression was that the investors really liked the CEO and believed in his vision but didn't trust him 100% so they released the money in dribs and drabs which didn't really help. We had some consultants tell us all to do OKRs and we were all expected to make up 10-20 quantifiable goals and it struck me (1) a personal distraction, (2) a game that a narcissist can play to make management think his glass is half full and yours is half empty, (3) a distraction for management from "the goal" of finding product-market fit.
• PCalvanelli 3y ago
  KPIs aren't the issue; it's how businesses use them. Think of it like flying a plane without gauges—it's risky and uncertain. The risk of stalling or getting lost increases. Good KPIs are like helpful plane instruments. Some may seem unnecessary, but the focus should be on the essential ones. KPIs are just measurements. The problem is businesses not being explicit in their direction and selecting the right measurements to reach those goals. You rely on measurements every day, don't stop now.
  • snarf21 3y ago
    I think KPIs are (one of) the issue(s) .. They will always fail because of Goodhart's Law .. Measurements are fine, but good luck finding any leadership that won't turn them into targets
• jgeada 3y ago
  Professional managers, particularly American MBA types is what destroys businesses. Not KPIs, Agile, Scrum or whatever the fad of the moment to blame may be. Management is a service task, it does not produce value itself. Good management is important to prioritize, allocate resources and resolve conflicts. But that is not how MBAs seem themselves, they see themselves as the most important thing around, they're the ones in charge after all. And damn physics and reality, the spreadsheets say you can get get a baby out in a month if you use 9 women to do it. Soon management captures all the attention and sucks away all the money, and shortly thereafter end up with a moribund shell of the former company. You can tell how close to moribund a company is by looking at the manager to worker ratio. Get out of there once it climbs much above 1 to 10. There are good managers around, but they're almost as rare as unicorns.
  • ldjkfkdsjnv 3y ago
    I've worked at many top big tech firms. Encountered very few "American MBAs", at this point it feels like a mythical unicorn. Alot/Most of tech companies are ran by ex-engineer H1Bs
    • alecco 3y ago
      I wonder how many of those went to get an MBA. That's the point of this masters, giving management knowledge to people with non-management degree. Like engineering + MBA -> promotion. I've worked for companies who pay MBAs for employees who are on-track to management.
    • jgeada 3y ago
      Most managers I encountered sooner or later would get an MBA, particularly as they made it further up the corporate ladder. Yes, it makes sense to get the specialized training being management needs, but an MBA almost never is that training. And re ex-engineer H1Bs: just because you used to be an individual contributor before becoming management does not imply you retain your individual contributor skills nor that you'll be a good manager. Most people across my career that became managers did so because the pay was better, workload was lower and promotion ladder more straightforward. Worst managers were usually the type that expected and demanded to be management after X years as an individual contributor; this is particularly true of people from some countries, where it is culturally expected that you'll be a manager within at most 10 years of starting. Most companies do treat moving to the management ladder as a promotion, which create all sorts of perverse incentives for many to become managers. Reminds me of a blog from a few years back: https://charity.wtf/2020/09/06/if-management-isnt-a-promotion-then-engineering-isnt-a-demotion/ https://charity.wtf/2020/09/06/if-management-isnt-a-promotio...
• kazanz 3y ago
  The comic in the article is the literal purpose of a KPI. The comic: "John your commit frequency dropped 25%, are you quiet quitting?" "Sorry boss, I was on sick leave. Next time I'll be more active." If we had the next box of the comic it would say: "You're good John, you don't have to be more active, and I'm glad you are feeling better. If you need anything let me know." The KPI is a leading indicator of something potentially going wrong (or right!) and a signal for management to look deeper. A good manager would connect with the human-being behind the metric, get the full picture, then help. If management is treating a KPI as a source of truth and ignoring the complex people behind them, then it's a manager problem. KPIs are not destroying business, bad managers are.
• adamc 3y ago
  [flagged]
• 23B1 3y ago
  KPIs are indicators, not measurements.
• ericls 3y ago
  I think KPI on itself is fine, it's basically the goal of the game you are playing, defining it clearly should help. However, it never make sense to to have multiple KPIs, everything should be derived from a single KPI. This actually give KPI some weight, because it literally defines what you do. Whenever you define you KPI, you'd think, "okay, is this all that I'm doing now?". And whenever you change your KPI, you'd be thinking, "okay, should I abandon the last game I played and play a new one?".
• sarbee 3y ago
  I like the concept of kpi psychosis but his solve for it is not helpful IMO. Its not actionable. “Use intuition” On the macro level it’s about incentives and leadership ( on the org level). If workers incentives are based on (faulty) KPIs there is going to be a perverse incentive even if members within the org know it is not optimal. On the micro level it’s about gathering other forms of data, basically qualitative data to inform the behavior that is observed in the KPIs or quantitative metrics. This could be accomplished with user research or speaking to those who have direct interaction with users - sales and support people.
• msvana 3y ago
  This is a difficult topic especially when it comes to public finance. An example: I live in Czechia and our universities are mostly publicly funded. In an ideal scenario resources should be allocated to achieve some noble but vague goal - to have a have population as well educated as possible, while also doing it efficiently in terms of money and time. But how should we divide the money available between different universities to achieve this goal? One of the most important factors currently used is the number of students studying at each university. But this indicator leads to weird behavior. The universities start to favor quantity over quality. They might, for example make the exams easier to keep weaker students around. Or they might admit more students than teachers can reasonably handle. You could choose a different metric and solve this issue, but most likely some other damaging behavior will replace the previous one. At the same time, relying on intuition might not play well with the current push for governmental transparency. It would require an enlightened incorruptible leader to make the decisions. And the public might demand objectivity. It's a difficult problem.
• 38 3y ago
  please don't write an entire article with even saying what the initial stands for: Key Performance Indicators
• ChrisCinelli 3y ago
  While working at a big corporation we had a velocity initiative supposedly aimed to lead the company toward continuous integration. "How long a PR stays open" was one of the KPI in a dashboard. I said: "Be careful with that!" People started to close PRs and reopen new PRs with the same code. Middle managers and sometimes the person in the division that was the point of contact for the velocity initiative were asking to do that. The script measuring this KPI was improved to look at the branch and the diff of the code. Result? people changing branch and EOL encoding in the new PR Learnings? B and C players with questionable ethics screw companies quite rapidly. If you do not have an outstanding company culture, KPI and aligning them with company values is futile.
• Alex_Bell 3y ago
  I agree completely that discussing the size of KPIs without considering the company's strategy and its Balanced Scorecard (BSC) is meaningless. KPIs cannot be isolated metrics; they must be directly related to overall goals and strategic directions of the company. Without understanding what specific aspects of the business and tasks these indicators reflect, discussing their size becomes a futile activity. Balanced Scorecards provide a holistic view of how different aspects of a company's activities relate to each other and to the overall strategy. They help identify causal relationships between different indicators and goals. It's important for leaders not only to define KPIs but also to clearly explain the logic behind their formation to KPI owners, and to demonstrate how these metrics fit into the context of the strategy and the BSC. Unfortunately, a lack of information and understanding of this logic can lead to misinterpreting KPIs and, consequently, to misaligned behavior or a focus on irrelevant metrics. Therefore, it's crucial for leadership to actively educate KPI owners and help them see the bigger picture — how their work contributes to the achievement of the company's overall objectives.
• deeviant 3y ago
  These musing miss the key failing of KPI, Goodhart's Law, "When a measure becomes a target, it ceases to be a good measure". There are multiple reasons why Goodhart's law tends to be true, but the most obvious one is as soon as people understand X is important, they do or inflate X. If they successful do X, it's often at the cost of other equally or more important but untracked activities, if they inflate it, it's obvious just as bad.
• gitfan86 3y ago
  I was hired at a pretty big startup because I came from a small startup where I had delivered close to 100% uptime. I pretty quickly realized that their outages were due to their KPIs being based on jira and GitHub statistics only. The hilarious thing was that after 6 months they started to complain about my GitHub stats. And everytime I asked why they are not tracking downtime since that is why they hired me in the first place. And they basically said it was too hard to track.
• gregshap 3y ago
  Leadership and management are crafts just like engineering. Goals and KPIs are tools. Most of the time when I've experienced bad Goals or KPIs, the root problem was one of the following. 1)Lack of clear mission/direction at the top level of the organization. This is a 100% necessary condition for good goals & kpis throughout the rest of the org. Otherwise it feels like political battles and favoritism between different people or departments with conflicting goals. 2) Goals & KPIs that are not well mapped from the org's mission or roadmap down to the individuals. Now all the individuals can hit their goals but org fails, or vice versa. This feels like "doing what's right for the company is bad for my performance review." 3) Confusing health KPIs with outcomes or goals. Something like commits per day/month is a great health check, but bad goal, and the benchmark for healthy is totally context dependent. This feels like "She solves the hardest problems but the solutions are very few LOC." Or "he has the most commits because his code is so buggy".
• jewelry 3y ago
  isn't it of a bad KPI that's destroying business? for KLOC committed and sick leave for example, isn't it suppose to be "KLOC per active days" that fix the issue?
• flappyeagle 3y ago
  KPIs are a tool and like every tool they can be misused by unskilled practitioners. HR KPI — hire 20 engineers: you can hire 20 morons. Sales KPI — close 100k of business: you sold the wrong use case and all of those customers churn Eng KPI — reduce bug frequency: you make reporting bugs too onerous to do Bad execution isn’t fixed by KPIs, it’s fixed by good execution. Good execution benefits from KPIs.
• Hasz 3y ago
  Show me the incentives and I'll show you the results. It's my favorite Munger quote, and it's a really good one. So often businesses invest so much time, effort, and money into everything but incentives. No matter how much you spend on culture, perks, diversity, swag etc, if the incentives are not aligned to some over-arching goal internally, you will not get the results you're looking for. Let me ground this in a concrete example. If your KPI is a certain number of calls per day from a sales person, here's how the most Machiavellian caller will hit it: 1. Spend little time with good prospects (they're paid by call, not by conversion!) 2. Call old numbers/disconnected/rapid disconnects 3. Call multiple numbers at once, reduce call quality etc. For any other metric you pick (conversion rate, call time, etc) there are similar schemes that have undesirable side effects. Either you very carefully build a composite framework of incentives to use, or you take the index fund approach and compensate on overall company performance (like a stock bonus, profit bonuses, etc). Personally, I think the best option is stock bonuses, but you may disagree. Either way, getting those incentives right is absolutely key to performance.
  • louwrentius 3y ago
    Stock bonus means: 1. I have less control over my bonus 2. My bonus depends on the performance of others 3. I can just do the bare minimum and still get a good chunk due to others
    • Hasz 3y ago
      Correct, certainly not perfect. I actually view 1,2 as features, and 3 as preferable to some of the active harm that comes with other incentives. You could reject that and optimize on something else. The goal is really to understand the impacts of whatever incentives you choose. I don't think there is a side-effect free incentive.
    • opportune 3y ago
      Sure, but it has plenty of other desirable outcomes (some of which are more for your employer than you): 1. There is a risk that the stock loses value but stocks generally have positive EV. 2. Since stocks vest over time there is some intrinsic value in being able to simply walk away if the stock crashes (dropping your comp going forward) but being able to stay if the stock surged (greatly increasing your comp compared to your market rate). It’s kind of like having a long-dated option. 3. It generally takes months to a couple years for stocks to change in value enough for it to put you over market rate. It takes similar amounts of time for you to fully ramp up and start adding value to the company. Generally you tend to get into golden handcuff range right around the time you start settling in. Getting paid above-market in a role where you’re doing well is great. 4. You may not have enough influence to really move the stock up but you often do have enough influence to move it down (like being sloppy or doing something destructive like leaking). 5. From the employer perspective, aligning your financial interests with the board/founders/investors/execs probably means you’re unlikely to try to unionize or get angry at the company when it gives you only a 5% salary bump in a year where the stock increased 100%. Generally speaking the worker:exec relationship is less combative since there’s less of a conflict of interests. As an employee there are nice aspects of this, like the whole company having a more collegiate or team-like atmosphere.
• lamontcg 3y ago
  McNamara fallacy: > The McNamara fallacy (also known as the quantitative fallacy),[1] named for Robert McNamara, the US Secretary of Defense from 1961 to 1968, involves making a decision based solely on quantitative observations (or metrics) and ignoring all others. The reason given is often that these other observations cannot be proven. > US Air Force Brigadier General Edward Lansdale reportedly told McNamara,[4] who was trying to develop a list of metrics to allow him to scientifically follow the progress of the war, that he was not considering the feelings of the common rural Vietnamese people. McNamara wrote it down on his list in pencil, then erased it and told Lansdale that he could not measure it, so it must not be important. https://en.wikipedia.org/wiki/McNamara_fallacy https://en.wikipedia.org/wiki/McNamara_fallacy
  • nxobject 3y ago
    Ooh, good example! McNamara's Vietnam War-era career is, in general, a long-ass list of well-meaning, manufacturing-inspired data-driven decisions and efficiency drives (after all, he was previously president of Ford) that horifically backfired. I wonder whether there's a "Mythical Man-Month" style book about that out there.
• gdubs 3y ago
  Sadly I've been in many organizations that have a kind of "vibes for me, data for thee" approach to things. Meaning, "data drives decisions" is the official philosophy ... until it isn't. It takes a certain amount of rigor and product maturity to make metric-based decisions work — and some types of applications are better suited to this than others.
• koliber 3y ago
  KPIs can be used to measure and direct attention to problems or successes. KPIs can also be used to drive change. KPIs are rarely used for much more than vanity metrics. The article describes the situation nicely. Like any system, KPIs can be tweaked and made to work for you, to accomplish your goals. The key is to start by asking why you want the metric. Sometimes the answer “to appease management” is the right answer. Marketing does matter! More often though you can come up with a better goal for your metrics and it will make your team and company more successful.
• dragonwriter 3y ago
  The level of certainty you need to have that a measure captures all and only what is important varies with how much weight is given to it. When it becomes a key and decisive factor in evaluating individual and/or unit performance, tied to reward and punishment, the required certainty is near unity, because you are very strongly creating an incentive structure which will exhibit Goodhart's Law if it isn't perfectly aligned with your actual intent. Things that make tolerable broad status indicators make horrible KPIs for that reason.
• dboreham 3y ago
  KPIs are very useful: if you find yourself working for an organization that begins to them, they indicate that you should leave. If you interview at an organization that uses them, you know not to take a job there.
• steno132 3y ago
  Hard disagree. KPIs are invaluable for any modern tech business because they provide clarity on what the organization's goals are, and how to achieve them. Let's say you have a goal to increase revenue by 50%. You come up with a plan, and assign different sub-goals to departments like Marketing, Sales, etc. Each person then gets a sub-goal down to the lowest levels of the org. KPIs provide incredible org-wide alignment which no other management technique before, or after, them produces nearly as well.
  • wetpaws 3y ago
    [dead]
• Helmut10001 3y ago
  Wow, the new title is so much clearer than the business speak that was before it. Thank you, HN.
• MPlus88 3y ago
  [dead]
• opportune 3y ago
  I have a more pessimistic take on this KPI stuff. You absolutely can get great outcomes without putting KPIs or metrics on a pedestal. And for sure, metrics should work for the employees; employees shouldn’t work for the metrics. The problem is that this kind of falls apart in large organizations because, from the perspective of CEOs, execs, and upper management, leaving wiggle room requires placing a high degree of trust in line teams or unscaleable micromanagement. And you can argue that the high degree of trust should be given, but bluntly, unless your employees are very well coordinated and your management team is really on top of their game, a lot of teams will flounder without clear goals. Now as an IC that sounds like bullshit, but let me ask you: have you ever seen a team that seems like they basically didn’t do anything? What did that team have in commmon? Usually it’s at least two of 1. a lack of clear goals 2. low to average levels of motivation and personal ownership 3. low to average levels of skill. If you’re on this website discussing software and KPIs in your free time you’re probably above average in motivation and skill. But I’m sure you know that without clear goals, people just working to put food on the table tend not to have the business interest and drive to create actually-valuable work. If you can keep your business small enough that pockets of low-productivity can’t fly under the radar, or somehow hire only motivated, skilled people that stay motivated and skilled years after they’re hired, you probably do need thing like KPIs to hold teams accountable. Not your team necessarily, but if say 10-20% of teams will flounder without KPIs, it’s probably just good policy to have it, even when it annoys the other 80% of teams, as long as you don’t add pathological processes around the KPIs.
• godelski 3y ago
  This really is just "Goodhart's Law" in action. We have a saying "all models are wrong" but I think we also need to make the phrase "all metrics are wrong" more common. Metrics are simply guides. It is also why we say that all rules are mean to be broken. If you rely too heavily on a metric you will only get yourself burned. Unfortunately no matter how good our algorithms are, they won't be any more than a guide. You still unfortunately have to use your brain.
• osigurdson 3y ago
  Human intuition is based on raw data - photons, pressure waves and so on. Extracting meaning from this raw data is complex. Maybe, one day a machine will be able to look at various performance metrics (time to last byte, load times, commits per day, etc) and give us a concrete plan of action. However, we don’t have that today so should probably learn to trust the associative cortices since it is the best machine that we currently have.