4 ms·

I don’t see you answering the question. You don’t propose any mechanism for rating productivity, and you come up with some potential problems. > Developers ar

by zepto 5y ago

I don’t see you answering the question. You don’t propose any mechanism for rating productivity, and you come up with some potential problems. > Developers aren’t really that different from bakers. I suspect very few people would agree with this. Only with an answer to this question does what you have argued so far apply to knowledge work.

joshuamorton 5y ago
> I don’t see you answering the question. You don’t propose any mechanism for rating productivity, and you come up with some potential problems. They don't need to. As long as the system is transparent and everyone agrees to it, what it is doesn't matter. Performance based on lines of code? Based on peer evaluations? Based on users acquired? Based on meeting arbitrary goals your director sets that year? It doesn't matter as long as the evaluation criteria are predefined and available. Companies don't even need to use the same criteria, you can choose a company whose process you vibe with.
- onlyrealcuzzo 5y ago
  I don't think you need metrics like lines of code. > Based on peer evaluations? Based on their interpretation of how you hit metrics? Why not just look at the metrics directly? I think it's okay if ratings are subjective based on loosely defined job responsibilities. That's how it works at FAANG. Nothing is going to be fair. But I think it's better than just counting how many bugs you fix or how many lines of code you review or how many users your code acquires (which should be mostly determined by your product and design team's decisions).
  - joshuamorton 5y ago
    I'm making a slightly higher level point: If some companies choose to make things strictly LoC based, that's fine (even though I think its a dumb measure). If they're transparent about that, and you prefer a FAANG-like "subjective peer eval based on loosely defined job responsibilities" approach, you can avoid the LoC based companies. There doesn't need to an objective, or even a best subjective, measure of productivity, as long as the company's chosen measure is transparent and preregistered.
    - zepto 5y ago
      How is it fine to make the measure LoC based? Being open about it doesn’t make it any less of a terrible thing to incentivize.
      - joshuamorton 5y ago
        Because if you don't like the measure, you can go work somewhere else. Exactly the same as if you don't like how much you're being compensated today, you can go work somewhere else. It's just that with a transparent evaluation process, you know why that is and can make more informed decisions than having your manager say "no don't worry, you're on track for a promotion next year" every year. To put it a different way: how do you know that you aren't being compensated based on LoC output today, if you don't have some amount of transparency in evaluation criteria (and, importantly, some amount of transparency in pay, so you can verify what the company says)?
        zepto 5y ago
        What if the better criteria for picking a good place to work is good leadership, and not a rule set?
- zepto 5y ago
  The rating system matters because if it affects pay then it sets the incentives. If you use lines of code, you incentivize inefficient boilerplate. If you use peer evaluations, you incentivize positive social behaviors and favors, but not necessarily productivity or good engineering. What you measure becomes the goal. Being predefined and available only amplifies the effect.
  - joshuamorton 5y ago
    Yes exactly. With transparency and predefined incentives, you can now make informed rational decisions about what company you want to work for based on how you feel their values (and yes, what you're describing are essentially values) align with your own and with where you feel you'll be successful. If you think about this over a long enough timescale, and believe in something like the efficient market hypothesis, companies will converge on near-optimal structures that pay for the performance that is valued in their niche, and individual will be able to make informed decisions to work with companies that will pay them based on their strengths. Note that I'm not saying that a company can't ever change its compensation and performance evaluation, so if LoC is a bad metric, over time I'd expect companies to move away from it as they find it to be gamed. But like, I'm not going to stop a company from picking a bad metric, or claim that transparency is bad because companies can make bad decisions. Companies can make equally bad decisions today, and you won't know.
    - zepto 5y ago
      You are making the assumption that an easily written set of fixed rules will outperform the more fluid and less easily quantified but nonetheless real heuristic of “anticipate what leadership needs”. I’d be very surprised if the game like companies you describe can outperform a well led organization.
      - joshuamorton 5y ago
        Addressing the other thread first: > What if the better criteria for picking a good place to work is good leadership, and not a rule set? I think an attribute of good leadership is being able to clearly communicate goals and priorities. If you can clearly communicate those goals and priorities in advance, you can evaluate people based on how well they perform against those goals and priorities. If I'm forced to "anticipate what leadership needs", without any guidance on how or what leadership values, I'm doing the same job as "leadership", in that I'm defining organizational values, goals, and priorities. That's bad, because it dilutes leadership. > You are making the assumption that an easily written set of fixed rules will outperform the more fluid and less easily quantified but nonetheless real heuristic of “anticipate what leadership needs”. > I’d be very surprised if the game like companies you describe can outperform a well led organization. I don't think I am. I don't think such heuristics are particularly easy to quantify or easy to write down, nor do I think they're particularly fixed. I do however think that most successful organizations do something like what I'm doing. As far as I know, Google, Microsoft, Amazon, and Facebook all provide more concrete performance expectations than "anticipate leadership needs". It would be hard to describe all of those as not "well led" or underperforming. In fact, most modern companies with more than a few hundred engineers develop some form of leveling system and rubric. The challenges with these are that they often remain vague, since they need to apply reasonably well to every new-grad engineer working at the company, from someone working on client features in javascript, to someone optimizing the C++ compiler, to everything in between. But even still, you can usually get better, pre-registered answers from your manager or director that are more relevant and local: "If you complete this project on time, that should prove performance at rating X or at level +". How exactly this works depends a bit on the company (e.x. at Google, since peer reviews are important, having a senior peer or two whoa aren't your manager clearly state that something is L+1 work is probably more valuable than your manager stating it, and good managers communicate that). This of course doesn't mean that you have no autonomy. Goals can be vague and high level. "Ship Frobulators by the end of the year" is a goal, and "we value end-user experience over implementation cost" is a value. If you know your company (or perhaps at a large company, your organization) has those goals and values, and you know the method by which they convert achievement of goals and values into compensation, you can better do what they want. The alternatives are that: 1. Leadership is unable to express their goals and values in a way that clearly communicate what they want from employees. This appears to be what you're getting at with the example of LoC. 2. Leadership is unable to maintain consistent goals and values 3. Leadership doesn't want to reward people based on their stated goals and values etc. etc. etc. In the first case, they're incompetent. In the second case, they're unreliable. In the third, they're duplicitous and trying to compensate you based on some secret, second set of values, as opposed to the ones that they're claiming to share. None of those alternative are things that inspire confidence in leadership.
onlyrealcuzzo 5y ago
The only thing different about knowledge work is that you're not producing a commodity. In this way - you can't just simply count how many pies a SWE coded. But you still get paid to produce "something". Is it even relevant what that is? Should you get paid less at Google if you work on a new product and you absolutely crush your job vs you work on optimizing ads and aren't very good at your job - but regardless are able to generate a lot more profits? Maybe? I don't have a good answer for this. I'm assuming "productivity" is somehow tied to "profit". No one cares if you work really hard and get nothing done. But if you work really hard and do your job well, but your boss told you to work on something unproductive - what should happen? You and your boss are considered unproductive? Just you? Just your boss? I don't know. I think the easiest fairest system would be a mix of levels like FAANG has along with ratings (which FAANG also has) along with transparent pay (which almost no one has). From a business perspective - you're currently paid the least possible amount it costs to employ you or replace you. If you have a valuable and rare skill set, you'll get paid a lot more, even if you aren't particularly good at your job. I don't see this ever going away.
- zepto 5y ago
  > The only thing different about knowledge work is that you're not producing a commodity. That’s a categorical difference. Not some minor detail. It’s also not the only difference. Knowledge workers contributions are necessarily much more interdependent than the contributions of people who make commodities. But see that if you think of knowledge work as little different from manufacturing commodities, your arguments and conclusions makes sense. I just think your assumptions about the similarities are not correct.

by zepto 5y ago

joshuamorton 5y ago

> I don’t see you answering the question. You don’t propose any mechanism for rating productivity, and you come up with some potential problems. They don't need to. As long as the system is transparent and everyone agrees to it, what it is doesn't matter. Performance based on lines of code? Based on peer evaluations? Based on users acquired? Based on meeting arbitrary goals your director sets that year? It doesn't matter as long as the evaluation criteria are predefined and available. Companies don't even need to use the same criteria, you can choose a company whose process you vibe with.

onlyrealcuzzo 5y ago
I don't think you need metrics like lines of code. > Based on peer evaluations? Based on their interpretation of how you hit metrics? Why not just look at the metrics directly? I think it's okay if ratings are subjective based on loosely defined job responsibilities. That's how it works at FAANG. Nothing is going to be fair. But I think it's better than just counting how many bugs you fix or how many lines of code you review or how many users your code acquires (which should be mostly determined by your product and design team's decisions).
- joshuamorton 5y ago
  I'm making a slightly higher level point: If some companies choose to make things strictly LoC based, that's fine (even though I think its a dumb measure). If they're transparent about that, and you prefer a FAANG-like "subjective peer eval based on loosely defined job responsibilities" approach, you can avoid the LoC based companies. There doesn't need to an objective, or even a best subjective, measure of productivity, as long as the company's chosen measure is transparent and preregistered.
  - zepto 5y ago
    How is it fine to make the measure LoC based? Being open about it doesn’t make it any less of a terrible thing to incentivize.
    - joshuamorton 5y ago
      Because if you don't like the measure, you can go work somewhere else. Exactly the same as if you don't like how much you're being compensated today, you can go work somewhere else. It's just that with a transparent evaluation process, you know why that is and can make more informed decisions than having your manager say "no don't worry, you're on track for a promotion next year" every year. To put it a different way: how do you know that you aren't being compensated based on LoC output today, if you don't have some amount of transparency in evaluation criteria (and, importantly, some amount of transparency in pay, so you can verify what the company says)?
      - zepto 5y ago
        What if the better criteria for picking a good place to work is good leadership, and not a rule set?
zepto 5y ago
The rating system matters because if it affects pay then it sets the incentives. If you use lines of code, you incentivize inefficient boilerplate. If you use peer evaluations, you incentivize positive social behaviors and favors, but not necessarily productivity or good engineering. What you measure becomes the goal. Being predefined and available only amplifies the effect.
- joshuamorton 5y ago
  Yes exactly. With transparency and predefined incentives, you can now make informed rational decisions about what company you want to work for based on how you feel their values (and yes, what you're describing are essentially values) align with your own and with where you feel you'll be successful. If you think about this over a long enough timescale, and believe in something like the efficient market hypothesis, companies will converge on near-optimal structures that pay for the performance that is valued in their niche, and individual will be able to make informed decisions to work with companies that will pay them based on their strengths. Note that I'm not saying that a company can't ever change its compensation and performance evaluation, so if LoC is a bad metric, over time I'd expect companies to move away from it as they find it to be gamed. But like, I'm not going to stop a company from picking a bad metric, or claim that transparency is bad because companies can make bad decisions. Companies can make equally bad decisions today, and you won't know.
  - zepto 5y ago
    You are making the assumption that an easily written set of fixed rules will outperform the more fluid and less easily quantified but nonetheless real heuristic of “anticipate what leadership needs”. I’d be very surprised if the game like companies you describe can outperform a well led organization.
    - joshuamorton 5y ago
      Addressing the other thread first: > What if the better criteria for picking a good place to work is good leadership, and not a rule set? I think an attribute of good leadership is being able to clearly communicate goals and priorities. If you can clearly communicate those goals and priorities in advance, you can evaluate people based on how well they perform against those goals and priorities. If I'm forced to "anticipate what leadership needs", without any guidance on how or what leadership values, I'm doing the same job as "leadership", in that I'm defining organizational values, goals, and priorities. That's bad, because it dilutes leadership. > You are making the assumption that an easily written set of fixed rules will outperform the more fluid and less easily quantified but nonetheless real heuristic of “anticipate what leadership needs”. > I’d be very surprised if the game like companies you describe can outperform a well led organization. I don't think I am. I don't think such heuristics are particularly easy to quantify or easy to write down, nor do I think they're particularly fixed. I do however think that most successful organizations do something like what I'm doing. As far as I know, Google, Microsoft, Amazon, and Facebook all provide more concrete performance expectations than "anticipate leadership needs". It would be hard to describe all of those as not "well led" or underperforming. In fact, most modern companies with more than a few hundred engineers develop some form of leveling system and rubric. The challenges with these are that they often remain vague, since they need to apply reasonably well to every new-grad engineer working at the company, from someone working on client features in javascript, to someone optimizing the C++ compiler, to everything in between. But even still, you can usually get better, pre-registered answers from your manager or director that are more relevant and local: "If you complete this project on time, that should prove performance at rating X or at level +". How exactly this works depends a bit on the company (e.x. at Google, since peer reviews are important, having a senior peer or two whoa aren't your manager clearly state that something is L+1 work is probably more valuable than your manager stating it, and good managers communicate that). This of course doesn't mean that you have no autonomy. Goals can be vague and high level. "Ship Frobulators by the end of the year" is a goal, and "we value end-user experience over implementation cost" is a value. If you know your company (or perhaps at a large company, your organization) has those goals and values, and you know the method by which they convert achievement of goals and values into compensation, you can better do what they want. The alternatives are that: 1. Leadership is unable to express their goals and values in a way that clearly communicate what they want from employees. This appears to be what you're getting at with the example of LoC. 2. Leadership is unable to maintain consistent goals and values 3. Leadership doesn't want to reward people based on their stated goals and values etc. etc. etc. In the first case, they're incompetent. In the second case, they're unreliable. In the third, they're duplicitous and trying to compensate you based on some secret, second set of values, as opposed to the ones that they're claiming to share. None of those alternative are things that inspire confidence in leadership.

onlyrealcuzzo 5y ago

The only thing different about knowledge work is that you're not producing a commodity. In this way - you can't just simply count how many pies a SWE coded. But you still get paid to produce "something". Is it even relevant what that is? Should you get paid less at Google if you work on a new product and you absolutely crush your job vs you work on optimizing ads and aren't very good at your job - but regardless are able to generate a lot more profits? Maybe? I don't have a good answer for this. I'm assuming "productivity" is somehow tied to "profit". No one cares if you work really hard and get nothing done. But if you work really hard and do your job well, but your boss told you to work on something unproductive - what should happen? You and your boss are considered unproductive? Just you? Just your boss? I don't know. I think the easiest fairest system would be a mix of levels like FAANG has along with ratings (which FAANG also has) along with transparent pay (which almost no one has). From a business perspective - you're currently paid the least possible amount it costs to employ you or replace you. If you have a valuable and rare skill set, you'll get paid a lot more, even if you aren't particularly good at your job. I don't see this ever going away.

zepto 5y ago
> The only thing different about knowledge work is that you're not producing a commodity. That’s a categorical difference. Not some minor detail. It’s also not the only difference. Knowledge workers contributions are necessarily much more interdependent than the contributions of people who make commodities. But see that if you think of knowledge work as little different from manufacturing commodities, your arguments and conclusions makes sense. I just think your assumptions about the similarities are not correct.