6 ms·

Why is it so hard to measure productivity?

• svsaraf 3y ago
  Where is the Moneyball for knowledge workers?
  • deleted 3y ago
    [deleted]
  • kelseyfrog 3y ago
    Whats the incentive to share it? Disclosure is an alpha-erasing move.
    • quickthrower2 3y ago
      Not always in the startup world. http://www.paulgraham.com/newideas.html http://www.paulgraham.com/newideas.html
  • nine_zeros 3y ago
    Oh it exists. Lines of code, number of commits, number of documents and other such BS uncorrelated with actual work. Imagine how poor management must be to think more lines of code == more revenue.
    • wilde 3y ago
      They are correlated until you admit to using them in the performance process. Then they’re far too easy to game and become useless. What we haven’t found is an adversary-resistant measure of productivity.
      • nine_zeros 3y ago
        They are absolutely not correlated. Engineers often spend some quarters investigating and fixing scaling issues. This work does not need to a high number of locs. If your work is merely adding some feature factory or bug fixes, maybe there is a correlation.
• ndjdbdbnms 3y ago
  [flagged]
  • tracer4201 3y ago
    > It doesn’t work so we keep doing it because we can’t try anything different because what we are doing doesn’t work. Saying it “doesn’t work” is a stretch, no? The large tech companies all give similar technical interviews for software engineers. Today, most people hired through this process are probably “good enough”. And by what metric do I mean good enough? These companies are mostly demonstrating some acceptable level of growth. At the end of the day, that’s all their shareholders care about. We won’t have meaningful change unless there’s an incentive. You think something is broken, but the key decision makers don’t agree. Source: 10+ years in big tech
    • jdidhebhdf 3y ago
      [flagged]
  • iraqmtpizza 3y ago
    Of course there is a better interviewing process that we know of. The US military has been using it for like a century.
    • tuna74 3y ago
      Could you describe (or provide a link to an explanation) that process?
      • deanmen 3y ago
        I think he means aptitude tests like the ASVAB.
    • nradov 3y ago
      If the US military has such a good interviewing process then how do we keep ending up with generals who lose wars? Back during WW2 they fired or promoted officers based primarily on battlefield performance, which eventually worked pretty well for eliminating the incompetent.
• frnkng 3y ago
  Currently I’m following the idea that it starts with good work priorization. With other words: What work shall an organization/team/individual actually execute? Which tickets shall be picked from the infinite stream of tickets? How do you know that the right tickets are picked? Unless this questions can be answered, measuring the individual performance is completely impossible in the short term. One can „work hard“ and produce a lot of code, write many tests, do many tickets with low ROI and still achieve subpar results. Thus the measure of productivity must truly capture the added value of one’s work. And there is no „one size fits all“ solution. Long term the differences in productivity are clearly visible.
• esafak 3y ago
  Because it's not observable; it has to be inferred. Every employee affects the finished product through a complex web of interactions. As a latent variable, we can only hope to gain a noisy estimate. Companies don't bother estimating worker productivity themselves. Rather they let employees make their own case, based on their work, using whatever data they can scrounge.
• rhelz 3y ago
  It’s easy to measure productivity. Take all the profits and divide by all the hours worked. What is HARD is to come up with endless rationals for why some people get way more that others.
  • esafak 3y ago
    We all have the same productivity? Nice!
  • latency-guy2 3y ago
    If that's how we're getting paid, I think its time to fire all junior devs at the least, and then get onto the slackers and people who don't really do their job
  • quickthrower2 3y ago
    Owning a share is infinitely productive.
    • rhelz 3y ago
      Or infinitly exploitative, depending on how comfortable you are doing all the work while someone else take the lion's share of what you've produced.
• galenmarchetti 3y ago
  Creativity follows a different pattern than highly repeatable line work. Software engineering is an inherently creative pursuit, especially when paired closely with a dynamic product function. Definitely recommend Rick Rubin's book The Creative Act for a detailed glance into how wildly off the attempt to systematize and measure productivity is, in a creative context. https://www.amazon.com/Creative-Act-Way-Being/dp/0593652886 https://www.amazon.com/Creative-Act-Way-Being/dp/0593652886
• Guvante 3y ago
  > Interviews will transition to be more real-world scenarios, perhaps within the customer's actual codebase, addressing a genuine problem the customer faces—possibly even compensating the interviewee for their time The mechanics of this idea suck for everybody. Sometimes you have a candidate who is up to speed on your stack or one of its dependencies and can go from idea to implementation in a short time period but otherwise the math doesn't work. A week of time with a mentor for one day over that time is probably enough to do a 2 day task if there are no difficulties when you consider lack of knowledge of the code base and interfacing with a new code environment. So best case scenario you pay 3x for a feature whose mental state won't stick around and the interviewer has to find time to do a week of work while juggling their existing job (which could be job searching if nothing else). You could say "but you would hire them after" but if we are talking about that late in the process probationary periods make more sense. If you aren't sure they are a good fit (say 50% certainty) overpaying for a potential solution is bad and then wasting that much time on a solution also doesn't make sense. That is why most practical things are less than a day of effort and not impactful on the product to avoid weirdness around NDAs/Copyright/proper payment. And if you think meaningful work happens more often than every two days I wonder what software company you are working for... Breaking down tasks to 1 hour slots isn't what I would call a good use of resources...
• pcmaffey 3y ago
  Why do we need to measure productivity? No one seems to ask that question. To me it seems that trying to measure the individual (people) parts of a system in some quasi-reproducable and transitive way does more harm to the total output of the system than good. Do we measure trust (upon which all business is based)? If we relinquish individual productivity from our “measure all the things so we can improve them” obsession to the soft realm of the immeasurable, we might magically stop wasting so much time on stupid shit and accomplish a lot more meaningful work.
  • aeternum 3y ago
    So would you just fire the entire team if a product does poorly? Many individuals don't like startups because of this risk, most startup only get 1-2 chances to get productivity right. Very few waste time measuring productivity.
    • gabrielso 3y ago
      Start-ups don't get 1-2 chances to get productivity right. They get 1-2 chances to get PRODUCT right. If they don't get productivity right from the get-go, they only get 1 chance instead of the "1-2 chances". That's the risk most individuals don't like about startups, if productivity is not a centerpiece of their operation, the leadership pushes for the only other solution they believe: long hours in inhumane working conditions. I don't believe there's causation between a start-up productivity and their product success, but I would love to be proven wrong.
    • quickthrower2 3y ago
      Option 1 - Don't measure and hope Option 2 - Measure Option 3 - Use judgement We undervalue qualitative measurements when talking about this stuff. But in real life it is all there is. If you did everything with a measuring tape, you could do nothing (not even walk to your car!). If you care about nothing and close your eyes, things will get disorganized and chaotic and out of control. This doesn't mean measure nothing, but don't try to get some perfect measure of productivity. At least not in creative work like software development.
  • 3y ago
• pizzafeelsright 3y ago
  Measuring productivity is silly. Measure results. Here's my desired review: Entire team goes around and answers the question: who do you want to leave? Two strikes get another chance. Three strikes and they're gone. Same with the boss. Keep their job or replace em? More than three say replace the level up does a panel with the three. During our last layoff the question for the managers was "if you could drop any headcount who would it be?" We lost about 8% from that. Everyone I talked to said they were happy that the layoff got rid of the bad apples they knew.
  • aeternum 3y ago
    >Entire team goes around and answers the question: who do you want to leave? This sounds like it might work as a one-time thing, but just imagine the second-order effects. Your annual reviews literally become Survivor, how could you ever prevent backroom alliances?
    • wilde 3y ago
      This is effectively Meta’s perf process in portions of the company.
• rhelz 3y ago
  From the OP: “every potential metric..is woefully inadequate”. How many years do we keep trying before realize there is no such thing as individual productivity. My point is more radical than “we all have the same productivity.” The very concept of individual productivity is incoherent.
• Hermitian909 3y ago
  Lot of poorly thought out stuff here, I suspect the author is young and lacks real world experience running a business. > If 10x engineers truly exist, why do pay scales intra company not cover a 10x spectrum? The value you produce presents a cap on how much you can earn, not a floor. If you want to capture that value you need leverage, high end SWEs usually don't have that leverage. The statement is wrong anyway, at companies like Google, pay ranges from 200k-10million+. > Interviews will transition to be more real-world scenarios, perhaps within the customer's actual codebase Many good companies already do this, it doesn't actually change the interviews that much. All interview questions my company has are taken straight from the codebase - the end result is a standard system design + leetcode style interview because we actually need to solve these problems somewhat regularly. Other options listed that lengthen the interview process from the current 6-8 hour standard are off the table at any companies targeting top of market talent who won't put up with longer than standard interviews (makes it hard to get companies to compete for the interviewee). Companies not targeting top of market talent will almost always ape top of market won't adapt because they are not generally tech first and thus simply want to adopt "industry best practices". The organizational incentives to do this are incredibly strong. Small tech-first startups are usually the only ones who can muster the organization will to do something more original recruiting wise but will necessarily stay a small part of the market. > Interview performance (at least for those hired) definitely does not correlate with on-the-job performance Citation needed, every company I've seen try to do a large scale assessment of this has found that interview performance is positively correlated with on-the job performance[0]. (I've seen some actual data + studies, but not sure any of them are fully public). What does the author have to offer against this? I'd encourage the author to think much more about the incentive structures of the actors involved here and try coming to some new (more interesting) conclusions. [0] https://rework.withgoogle.com/guides/hiring-use-structured-interviewing/steps/read-googles-internal-research/#:~:text=Structured%20interviews%20are%20better%20at,hires%20across%20functions%20and%20levels https://rework.withgoogle.com/guides/hiring-use-structured-i....
  • wilde 3y ago
    Your citation is structured vs unstructured interviews. Not overall. Ugh I can’t find the original source but this concept comes from a 2013 study from Google? Finding old stuff on the internet is hard. https://www.quora.com/Is-there-a-link-between-job-interview-performance-and-job-performance https://www.quora.com/Is-there-a-link-between-job-interview-...
• deterministic 3y ago
  Don’t. Measure (income - cost) instead. You are running a business after all.
• Nasrudith 3y ago
  Putting aside the obvious Goodhart's Law implication productivity isn't just based off of the "components" be they human or otherwise but also the fundamental goals which control the end product. If I was an utterly mad billionaire who believed that I could hire top software talent and have them attempt to store physical objects in data. Not 'encode the schematics' or even 'encode the complete physical details' but such that it was possible to generate some hidden combination. Such an endevour would not be productive but the fault doesn't lie in the employees.
• sarabande 3y ago
  I like the article's premise. I would like some argument for why we'd suddenly get better at measuring something that has escaped us for at least 80 years when knowledge work became more common. > My suspicion is that [...] there will be a surge in our capacity to measure and evaluate real-life work performance.
• crvdgc 3y ago
  > recursion is extremely rare in shipped code but extremely common in interviews Though not quite important to the main theme of the article, I wonder what area the author is referring to.
  • kelahcim 3y ago
    I guess to all the whiteboard coding with the recursion (e.g. Fibonacci sequence). On a contrary, I think it is present in many places where you have to dive into any sensible, tree like, structure. Then, it's a mess once it turns out person has no idea what recursion is.
• bmitc 3y ago
  What even is productivity? I think two questions are only needed: are people, teams, and organizations working on what they are supposed to be working on on the scale of weeks to months? Are they hitting acceptable milestones? And when I say milestones, I don't mean time-based ones, which are more often than not arbitrary.
  • johnnyanmac 3y ago
    >when I say milestones, I don't mean time-based ones, which are more often than not arbitrary. artbitrary, yes. But also what the entire business world revolves around. quarterly earning calls, monthly finances, yearly budgets. It's arbituary, but those are period they pick to determine a bunch of stuff above the product itself.
• imtringued 3y ago
  It is quite easy. Just apply the income = productivity hypothesis that economists seem to love. Whoever you pay the most, is the most productive person in the company. Yes, that means you, the CEO, should pay yourself the most because you are the most productive employee...