5 ms·

Can you think of a transparent way to measure developer productivity?

by zepto 5y ago

onlyrealcuzzo 5y ago
One possible option is to make ratings and pay public. If you think your rating doesn't make sense, or your rating does make sense, but your pay is lower than people with much lower ratings - then you have a case to argue. Of course - this could be just as nepotistic as the current system. But I think transparency is bad for nepotism long-term. Usually the most corrupt governments and organizations are the most opaque. Developers aren't really that different from bakers. You could bake 45 pies in 1 hour, and they could all be terrible pies that can't be sold, and you could burn down half the bakery in the process. In the same way, a developer could ship a bunch of features, but they could all be "half-baked" and a nightmare to maintain. You could also try to define metrics for what quality and productivity are. But people will just game that. Ratings can also easily be gamed or nepotistic.
- zepto 5y ago
  I don’t see you answering the question. You don’t propose any mechanism for rating productivity, and you come up with some potential problems. > Developers aren’t really that different from bakers. I suspect very few people would agree with this. Only with an answer to this question does what you have argued so far apply to knowledge work.
  - joshuamorton 5y ago
    > I don’t see you answering the question. You don’t propose any mechanism for rating productivity, and you come up with some potential problems. They don't need to. As long as the system is transparent and everyone agrees to it, what it is doesn't matter. Performance based on lines of code? Based on peer evaluations? Based on users acquired? Based on meeting arbitrary goals your director sets that year? It doesn't matter as long as the evaluation criteria are predefined and available. Companies don't even need to use the same criteria, you can choose a company whose process you vibe with.
    - onlyrealcuzzo 5y ago
      I don't think you need metrics like lines of code. > Based on peer evaluations? Based on their interpretation of how you hit metrics? Why not just look at the metrics directly? I think it's okay if ratings are subjective based on loosely defined job responsibilities. That's how it works at FAANG. Nothing is going to be fair. But I think it's better than just counting how many bugs you fix or how many lines of code you review or how many users your code acquires (which should be mostly determined by your product and design team's decisions).
      - joshuamorton 5y ago
        I'm making a slightly higher level point: If some companies choose to make things strictly LoC based, that's fine (even though I think its a dumb measure). If they're transparent about that, and you prefer a FAANG-like "subjective peer eval based on loosely defined job responsibilities" approach, you can avoid the LoC based companies. There doesn't need to an objective, or even a best subjective, measure of productivity, as long as the company's chosen measure is transparent and preregistered.
        zepto 5y ago
        How is it fine to make the measure LoC based? Being open about it doesn’t make it any less of a terrible thing to incentivize.
        joshuamorton 5y ago
        Because if you don't like the measure, you can go work somewhere else. Exactly the same as if you don't like how much you're being compensated today, you can go work somewhere else. It's just that with a transparent evaluation process, you know why that is and can make more informed decisions than having your manager say "no don't worry, you're on track for a promotion next year" every year. To put it a different way: how do you know that you aren't being compensated based on LoC output today, if you don't have some amount of transparency in evaluation criteria (and, importantly, some amount of transparency in pay, so you can verify what the company says)?
  - onlyrealcuzzo 5y ago
    The only thing different about knowledge work is that you're not producing a commodity. In this way - you can't just simply count how many pies a SWE coded. But you still get paid to produce "something". Is it even relevant what that is? Should you get paid less at Google if you work on a new product and you absolutely crush your job vs you work on optimizing ads and aren't very good at your job - but regardless are able to generate a lot more profits? Maybe? I don't have a good answer for this. I'm assuming "productivity" is somehow tied to "profit". No one cares if you work really hard and get nothing done. But if you work really hard and do your job well, but your boss told you to work on something unproductive - what should happen? You and your boss are considered unproductive? Just you? Just your boss? I don't know. I think the easiest fairest system would be a mix of levels like FAANG has along with ratings (which FAANG also has) along with transparent pay (which almost no one has). From a business perspective - you're currently paid the least possible amount it costs to employ you or replace you. If you have a valuable and rare skill set, you'll get paid a lot more, even if you aren't particularly good at your job. I don't see this ever going away.
    - zepto 5y ago
      > The only thing different about knowledge work is that you're not producing a commodity. That’s a categorical difference. Not some minor detail. It’s also not the only difference. Knowledge workers contributions are necessarily much more interdependent than the contributions of people who make commodities. But see that if you think of knowledge work as little different from manufacturing commodities, your arguments and conclusions makes sense. I just think your assumptions about the similarities are not correct.
- chongli 5y ago
  You could bake 45 pies in 1 hour, and they could all be terrible pies that can't be sold, and you could burn down half the bakery in the process. In the same way, a developer could ship a bunch of features, but they could all be "half-baked" and a nightmare to maintain. You can inspect the pies made by your staff. Terrible pies that can’t be sold either look bad (misshapen, burnt, leaking) or taste bad (too sweet, not sweet enough, salty, over/undercooked). It’s not difficult to figure out. On the other hand, evaluating a developer’s productivity and quality of work would require a lot of code review which takes time and skill and also may require the developer to be present, wasting their time and lowering productivity. Push the review and eval process far enough and you really risk bringing down that developer to an unacceptably low level of productivity.

Can you think of a transparent way to measure developer productivity?

by zepto 5y ago

Can you think of a transparent way to measure developer productivity?

One possible option is to make ratings and pay public. If you think your rating doesn't make sense, or your rating does make sense, but your pay is lower than people with much lower ratings - then you have a case to argue. Of course - this could be just as nepotistic as the current system. But I think transparency is bad for nepotism long-term. Usually the most corrupt governments and organizations are the most opaque. Developers aren't really that different from bakers. You could bake 45 pies in 1 hour, and they could all be terrible pies that can't be sold, and you could burn down half the bakery in the process. In the same way, a developer could ship a bunch of features, but they could all be "half-baked" and a nightmare to maintain. You could also try to define metrics for what quality and productivity are. But people will just game that. Ratings can also easily be gamed or nepotistic.

zepto 5y ago
I don’t see you answering the question. You don’t propose any mechanism for rating productivity, and you come up with some potential problems. > Developers aren’t really that different from bakers. I suspect very few people would agree with this. Only with an answer to this question does what you have argued so far apply to knowledge work.
- joshuamorton 5y ago
  > I don’t see you answering the question. You don’t propose any mechanism for rating productivity, and you come up with some potential problems. They don't need to. As long as the system is transparent and everyone agrees to it, what it is doesn't matter. Performance based on lines of code? Based on peer evaluations? Based on users acquired? Based on meeting arbitrary goals your director sets that year? It doesn't matter as long as the evaluation criteria are predefined and available. Companies don't even need to use the same criteria, you can choose a company whose process you vibe with.
  - onlyrealcuzzo 5y ago
    I don't think you need metrics like lines of code. > Based on peer evaluations? Based on their interpretation of how you hit metrics? Why not just look at the metrics directly? I think it's okay if ratings are subjective based on loosely defined job responsibilities. That's how it works at FAANG. Nothing is going to be fair. But I think it's better than just counting how many bugs you fix or how many lines of code you review or how many users your code acquires (which should be mostly determined by your product and design team's decisions).
    - joshuamorton 5y ago
      I'm making a slightly higher level point: If some companies choose to make things strictly LoC based, that's fine (even though I think its a dumb measure). If they're transparent about that, and you prefer a FAANG-like "subjective peer eval based on loosely defined job responsibilities" approach, you can avoid the LoC based companies. There doesn't need to an objective, or even a best subjective, measure of productivity, as long as the company's chosen measure is transparent and preregistered.
      - zepto 5y ago
        How is it fine to make the measure LoC based? Being open about it doesn’t make it any less of a terrible thing to incentivize.
        joshuamorton 5y ago
        Because if you don't like the measure, you can go work somewhere else. Exactly the same as if you don't like how much you're being compensated today, you can go work somewhere else. It's just that with a transparent evaluation process, you know why that is and can make more informed decisions than having your manager say "no don't worry, you're on track for a promotion next year" every year. To put it a different way: how do you know that you aren't being compensated based on LoC output today, if you don't have some amount of transparency in evaluation criteria (and, importantly, some amount of transparency in pay, so you can verify what the company says)?
- onlyrealcuzzo 5y ago
  The only thing different about knowledge work is that you're not producing a commodity. In this way - you can't just simply count how many pies a SWE coded. But you still get paid to produce "something". Is it even relevant what that is? Should you get paid less at Google if you work on a new product and you absolutely crush your job vs you work on optimizing ads and aren't very good at your job - but regardless are able to generate a lot more profits? Maybe? I don't have a good answer for this. I'm assuming "productivity" is somehow tied to "profit". No one cares if you work really hard and get nothing done. But if you work really hard and do your job well, but your boss told you to work on something unproductive - what should happen? You and your boss are considered unproductive? Just you? Just your boss? I don't know. I think the easiest fairest system would be a mix of levels like FAANG has along with ratings (which FAANG also has) along with transparent pay (which almost no one has). From a business perspective - you're currently paid the least possible amount it costs to employ you or replace you. If you have a valuable and rare skill set, you'll get paid a lot more, even if you aren't particularly good at your job. I don't see this ever going away.
  - zepto 5y ago
    > The only thing different about knowledge work is that you're not producing a commodity. That’s a categorical difference. Not some minor detail. It’s also not the only difference. Knowledge workers contributions are necessarily much more interdependent than the contributions of people who make commodities. But see that if you think of knowledge work as little different from manufacturing commodities, your arguments and conclusions makes sense. I just think your assumptions about the similarities are not correct.
chongli 5y ago
You could bake 45 pies in 1 hour, and they could all be terrible pies that can't be sold, and you could burn down half the bakery in the process. In the same way, a developer could ship a bunch of features, but they could all be "half-baked" and a nightmare to maintain. You can inspect the pies made by your staff. Terrible pies that can’t be sold either look bad (misshapen, burnt, leaking) or taste bad (too sweet, not sweet enough, salty, over/undercooked). It’s not difficult to figure out. On the other hand, evaluating a developer’s productivity and quality of work would require a lot of code review which takes time and skill and also may require the developer to be present, wasting their time and lowering productivity. Push the review and eval process far enough and you really risk bringing down that developer to an unacceptably low level of productivity.