4 ms·

There’s a weird disconnect because on the one hand I agree you can’t measure productivity and on the other hand we all know that some engineers are vastly more

There’s a weird disconnect because on the one hand I agree you can’t measure productivity and on the other hand we all know that some engineers are vastly more productive than others. So what gives?

throw5959 2y ago
You can measure productivity by measuring the success, but that's kinda useless for day to day software engineering management.
- Trasmatta 2y ago
  How do you define success? If a product bombs, is that because of the engineering or the product design?
  - throw5959 2y ago
    I don't think it's possible to answer generally. Track what matters for your business.
  - skeeter2020 2y ago
    if it's successful, it's because of sales. If it fails, engineering didn't build the right thing / was too slow - it really doesn't matter.
- ChrisMarshallNY 2y ago
  I tend to go by results, and for me, "results" means shipped* code that is used and accepted by end users**, can be maintained and extended***, and doesn't generate trouble tickets. * MVP doesn't count. ** Can include users inside the organization. *** It's OK if it requires senior-level ongoing support. I think expecting it to be maintained by monkeys is a bad idea.
  - pinkmuffinere 2y ago
    To me, "MVP doesn't count" feels like a crazy take -- in many roles, the _only_ ask is to produce a series of different MVP's. I guess maybe the definition of "MVP" is a bit squishy, and these people-who-ship-MVPs themselves make MVP-MVP's, which shouldn't count as shipped?
    - ChrisMarshallNY 2y ago
      I spent most of my career, shipping finished product, which, in many cases, probably could have benefitted from an MVP-like "tuning phase," but we called that "beta." I think MVP generates more useful feedback, but I really don't like thinking of an MVP as "shipping software." I also worked for hardware companies, where shipping stuff had some pretty serious stakes, and learned how to make sure we got it as good as possible, before getting it out the door. I like the idea of evolutionary design, and "tuning," but I think it's a bad idea (for me) to deliberately ship bad software as an end-product. (Also, MVP, by definition, generates lots of trouble tickets. I am allergic to trouble tickets. It's totally a personal thing, but I live by it).
  - skeeter2020 2y ago
    saying "MVP doesn't count" implies that you throw it away and then right "the perfect system" at some point. If you've ever had an MVP land you know that's not how it happens.
    - ChrisMarshallNY 2y ago
      I write "as close to perfect" as I can get. I know that "The Perfect is the enemy of the good" is a popular meme, but I have found that "The perfect is something to strive for" has been useful, for me. In fact, my way has been working for me, for decades. I'm quite aware that many folks do it differently, and that's one reason that I try to "keep it in the I," and write about how I do it, and talk about the bar that I set, for myself. Most of the software I write, is free software that Serves a pretty small demographic. It can have a fairly outsize influence on the lives of the people that use my software, and I really care about the end-users of my work, so I tend to set a pretty high personal bar. I'm quite aware that I don't have many of the stressors that beset commercial software houses, so I sincerely don't feel "snooty." In fact, I feel profoundly grateful to be in a position, where I can follow my muse. I really would like it if folks wrote better stuff, but I am also aware of the culture, and how that's next to impossible, these days.
jprete 2y ago
Those are two different concepts hiding in similar words. You can't [numerically or precisely] measure productivity, but some engineers are vastly more productive [such that you can easily tell the difference without a formal measurement].
- fshafique 2y ago
  Gut feeling uses all your internal predispositions and biases.
  - skeeter2020 2y ago
    you don't need to rely on gut feeling and risk bias. You can stop looking for the "productivity metric" and instead bet on <some> measure, then track the change over time. It's the only thing that's ever worked for me.
    - noirbot 2y ago
      Sure, but which measure you pick is itself a gut feeling and biased. It's the easy way to miss people doing whatever sort of work that you aren't measuring that may be what's letting everyone else excel at what you are measuring.
Trasmatta 2y ago
We all "know" that, but there are also some engineers that only give a very strong illusion of being more productive. Maybe engineer #1 is constantly pushing up code. In the time it takes them to merge 15 PRs, engineer #2 opens only 1 - but maybe they thought really deeply about the problem, and their approach actually saves the team hundreds or thousands of hours of future development work vs how engineer #1 would have solved the problem. Part of what makes this so hard to measure is the long tail effects of development decisions. (Incidentally, that's also a source of burnout for me - the constant mental overhead of worrying about the long term implications of what I'm doing, and particularly how they effect other people. It's very challenging.)
- ebiester 2y ago
  The problem is that the vast majority of code will not have long term implications so long as it reaches a minimum of design, performance, and does its job without bugs. Consistency of patterns is more important than the optimal pattern for most decisions. There are some core areas of the application that are much more important, but they are often the earliest data structures and built before the problem is known. You will not know how your code will change, so make it as consistent as possible with the rest of the system until you know more.
Sevii 2y ago
You can measure productivity with correlated metrics. The issue has always been that the metrics which are easy to track don't line up incentives with the actual business goals. A group of 10 people who write 200k loc per year are probably more productive than a group of 10 people who write 10k loc per year. If you took those metrics and then did an investigation of the people in your company writing 10k loc you might find that they are slackers or that they write assembly. The issue is when metrics are used to stack rank teams with no thinking put into it. You can't treat correlated metrics like direct metrics. A logger might be evaluated based on how many trees he cut down in a day. There is no comparable way to pay software engineers piecemeal. Metrics are good, but people want to use them without thinking or taking context into consideration.
- bigs 2y ago
  Or you may find they write higher quality code - less bugs, more performant code, or so on.
llm_trw 2y ago
The engineers are only productive because they have the support structure in place. The most productive fpga engineer I ever hired was so hopeless with git that I had to hire a second software engineer to babysit him. After I left both of them got fired and the product they were ahead of schedule on when I left had slipped 2 years behind before it finally got cancelled three years later.
daz0007 2y ago
a weird disconnect... of any true innovation or even reality... such vague objectional blandness... "They'd beg to work for us" - what the f8ck.... if they were the best they would not beg anyone how degrading...They would be there for a mission or wanting to improve something about themseves or other parts of the world. There's nothing here apart from Agile coach wanting to get some more work. 1984 was released in 1949, if anyone thinks these words / values really mean what is writen wow. People, Internal Quality, Lovability, Visibility, Agility, Profitability...
Salgat 2y ago
It's an extremely complex mixture of many factors (which can vary wildly between two different productive engineers), and trying to make that into some magical formula ends up creating a system that can be gamed to superficially appear productive to managers.
skeeter2020 2y ago
I have an incredibly productive staff developer. Not only does he work a lot, he also produces, and it's very high quality. He also does a relatively poor job of upskilling his teammates, and is a little rough when mentoring. This is not intentional (i.e. he's not a jerk). Overall I don't know if, in the context of a staff developer, he's vastly more productive than say, another dev I have who produces less but levels-up his team better than almost anybody I've ever seen.
- ggregoryarms 2y ago
  Maybe that other dev has a unique ability you should reward. Sound awesome. Focus on that.

throw5959 2y ago

You can measure productivity by measuring the success, but that's kinda useless for day to day software engineering management.

Trasmatta 2y ago
How do you define success? If a product bombs, is that because of the engineering or the product design?
- throw5959 2y ago
  I don't think it's possible to answer generally. Track what matters for your business.
- skeeter2020 2y ago
  if it's successful, it's because of sales. If it fails, engineering didn't build the right thing / was too slow - it really doesn't matter.
ChrisMarshallNY 2y ago
I tend to go by results, and for me, "results" means shipped* code that is used and accepted by end users**, can be maintained and extended***, and doesn't generate trouble tickets. * MVP doesn't count. ** Can include users inside the organization. *** It's OK if it requires senior-level ongoing support. I think expecting it to be maintained by monkeys is a bad idea.
- pinkmuffinere 2y ago
  To me, "MVP doesn't count" feels like a crazy take -- in many roles, the _only_ ask is to produce a series of different MVP's. I guess maybe the definition of "MVP" is a bit squishy, and these people-who-ship-MVPs themselves make MVP-MVP's, which shouldn't count as shipped?
  - ChrisMarshallNY 2y ago
    I spent most of my career, shipping finished product, which, in many cases, probably could have benefitted from an MVP-like "tuning phase," but we called that "beta." I think MVP generates more useful feedback, but I really don't like thinking of an MVP as "shipping software." I also worked for hardware companies, where shipping stuff had some pretty serious stakes, and learned how to make sure we got it as good as possible, before getting it out the door. I like the idea of evolutionary design, and "tuning," but I think it's a bad idea (for me) to deliberately ship bad software as an end-product. (Also, MVP, by definition, generates lots of trouble tickets. I am allergic to trouble tickets. It's totally a personal thing, but I live by it).
- skeeter2020 2y ago
  saying "MVP doesn't count" implies that you throw it away and then right "the perfect system" at some point. If you've ever had an MVP land you know that's not how it happens.
  - ChrisMarshallNY 2y ago
    I write "as close to perfect" as I can get. I know that "The Perfect is the enemy of the good" is a popular meme, but I have found that "The perfect is something to strive for" has been useful, for me. In fact, my way has been working for me, for decades. I'm quite aware that many folks do it differently, and that's one reason that I try to "keep it in the I," and write about how I do it, and talk about the bar that I set, for myself. Most of the software I write, is free software that Serves a pretty small demographic. It can have a fairly outsize influence on the lives of the people that use my software, and I really care about the end-users of my work, so I tend to set a pretty high personal bar. I'm quite aware that I don't have many of the stressors that beset commercial software houses, so I sincerely don't feel "snooty." In fact, I feel profoundly grateful to be in a position, where I can follow my muse. I really would like it if folks wrote better stuff, but I am also aware of the culture, and how that's next to impossible, these days.

jprete 2y ago

Those are two different concepts hiding in similar words. You can't [numerically or precisely] measure productivity, but some engineers are vastly more productive [such that you can easily tell the difference without a formal measurement].

fshafique 2y ago
Gut feeling uses all your internal predispositions and biases.
- skeeter2020 2y ago
  you don't need to rely on gut feeling and risk bias. You can stop looking for the "productivity metric" and instead bet on <some> measure, then track the change over time. It's the only thing that's ever worked for me.
  - noirbot 2y ago
    Sure, but which measure you pick is itself a gut feeling and biased. It's the easy way to miss people doing whatever sort of work that you aren't measuring that may be what's letting everyone else excel at what you are measuring.

Trasmatta 2y ago

We all "know" that, but there are also some engineers that only give a very strong illusion of being more productive. Maybe engineer #1 is constantly pushing up code. In the time it takes them to merge 15 PRs, engineer #2 opens only 1 - but maybe they thought really deeply about the problem, and their approach actually saves the team hundreds or thousands of hours of future development work vs how engineer #1 would have solved the problem. Part of what makes this so hard to measure is the long tail effects of development decisions. (Incidentally, that's also a source of burnout for me - the constant mental overhead of worrying about the long term implications of what I'm doing, and particularly how they effect other people. It's very challenging.)

ebiester 2y ago
The problem is that the vast majority of code will not have long term implications so long as it reaches a minimum of design, performance, and does its job without bugs. Consistency of patterns is more important than the optimal pattern for most decisions. There are some core areas of the application that are much more important, but they are often the earliest data structures and built before the problem is known. You will not know how your code will change, so make it as consistent as possible with the rest of the system until you know more.

Sevii 2y ago

You can measure productivity with correlated metrics. The issue has always been that the metrics which are easy to track don't line up incentives with the actual business goals. A group of 10 people who write 200k loc per year are probably more productive than a group of 10 people who write 10k loc per year. If you took those metrics and then did an investigation of the people in your company writing 10k loc you might find that they are slackers or that they write assembly. The issue is when metrics are used to stack rank teams with no thinking put into it. You can't treat correlated metrics like direct metrics. A logger might be evaluated based on how many trees he cut down in a day. There is no comparable way to pay software engineers piecemeal. Metrics are good, but people want to use them without thinking or taking context into consideration.

bigs 2y ago
Or you may find they write higher quality code - less bugs, more performant code, or so on.

llm_trw 2y ago

The engineers are only productive because they have the support structure in place. The most productive fpga engineer I ever hired was so hopeless with git that I had to hire a second software engineer to babysit him. After I left both of them got fired and the product they were ahead of schedule on when I left had slipped 2 years behind before it finally got cancelled three years later.

daz0007 2y ago

a weird disconnect... of any true innovation or even reality... such vague objectional blandness... "They'd beg to work for us" - what the f8ck.... if they were the best they would not beg anyone how degrading...They would be there for a mission or wanting to improve something about themseves or other parts of the world. There's nothing here apart from Agile coach wanting to get some more work. 1984 was released in 1949, if anyone thinks these words / values really mean what is writen wow. People, Internal Quality, Lovability, Visibility, Agility, Profitability...

Salgat 2y ago

It's an extremely complex mixture of many factors (which can vary wildly between two different productive engineers), and trying to make that into some magical formula ends up creating a system that can be gamed to superficially appear productive to managers.

skeeter2020 2y ago

I have an incredibly productive staff developer. Not only does he work a lot, he also produces, and it's very high quality. He also does a relatively poor job of upskilling his teammates, and is a little rough when mentoring. This is not intentional (i.e. he's not a jerk). Overall I don't know if, in the context of a staff developer, he's vastly more productive than say, another dev I have who produces less but levels-up his team better than almost anybody I've ever seen.

ggregoryarms 2y ago
Maybe that other dev has a unique ability you should reward. Sound awesome. Focus on that.