4 ms·

Cannot Measure Productivity (2003)

34 points by aard 2y ago

swiftcoder 2y ago
> economists are now seeing productivity increases in business due to the computer investments in the nineties. The point is that the improvements lag the investments Aye. And one rarely sees a software team stay intact long enough after shipping software to make a fair evaluation of what they actually delivered. Most of the time the incentive structures cause everyone who didn't get a promotion off the back of shipping to bail, and corporate then scavenges the remainder to staff other projects...
- andrelaszlo 2y ago
  Other things that make it difficult, some mentioned in the article: - It's a strange activity. The output is non-linear, non-repeatable, and basically chaotic in some ways. - You almost never build the same thing twice. Even if you do, productivity is affected (positively or negatively) just by the fact that you already did it before. - The value of what you produce is unknown or fluctuating wildly. You might be a unicorn one day and bankrupt (even personally liable!) the next. Less extreme examples are interesting too, maybe your software is a plugin for a product that stops supporting third party plugins. I think, like the article says, that pretending to be able to measure output just because we have something that might look superficially like actual output in other activities (e.g. number of units produced in a factory) is fundamentally misguided. It's a cop-out since what's really missing a lot of the time is (good) leadership.
mkesper 2y ago
(2003)
oldpersonintx 2y ago
[dead]
d--b 2y ago
And yet, people talk about productivity all the time because they can feel it. People say: "oh I am much more productive with C# than I am with Python". Or: "This guy is one of the most productive engineer I know". But the issue here is that value is subjective. But is that really a problem? "Value" is not better defined than "urgency" or "criticality", yet we prioritize tasks all the time based on the perceived urgency/criticality of task A vs task B. So in the same vein, we could assign a "value score" of project X, that would amount to something like "usefulness * difficulty * quality" (to be discussed), and define productivity as that over time spent.
- andrelaszlo 2y ago
  I think that's a good point. Say it's your company, then you need to decide if you want to build the product in C# or Python. It's going to affect productivity, but it's very difficult to say how. If you pick Brainf* of course most people can tell you productivity will suffer, but in the C#/Python example you might start building in C# since it's what you're familiar with, then have problems recruiting developers in your area a few years down the line, since most people are now doing Python (say). Technically, your original choice might be "correct" in some ways, but who knows? Perhaps the discussion should revolve around how we make decisions better in complex, rapidly changing environments, with limited information? It feels like we're clinging to the need for things to be predictable and measurable even when they're not, and we end up in more or less delusional discussions about which of two almost identical programming languages is better, or even tabs vs spaces :D
  - neonsunset 2y ago
    Perhaps this specific comparison is flawed. It's the same as if comparing "Do we build it in Rust or TypeScript?" (not in terms of hiring pool but in terms of specific language used - picking either Python or C# will result in products with massively different grade of performance and quality).
- Juliate 2y ago
  The hint that productivity is perceived (subjective, qualitative), but not measured (quantitative), is good. Productivity is argued, in the end, not demonstrated; or only within a very specific, limited framework (a very productive endeavour locally may reveal to be counter-productive on a higher or later scale). For instance, heavily relying on fossile fuel while pushing CO2 in the atmosphere has proved very successful in a specific time frame, the past century, let's say. But if (if) that means that Earth becomes inhabitable by current life forms within the next century, from an external observer, it was rather an error, or mismanaged, so not so productive.
svilen_dobrev 2y ago
in software making, what's Doable, Sellable, Wanted, and Needed, rarely overlap.. When they do, there's a dream come true. But - how to measure that? Is that productivity? or acumen? or luck? Which may hint at Exupery.. How do you measure love?
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]
thread_id 2y ago
The best measure of productivity for a software implementation team is the quality of the product as measured by latent defects and technical debt. The team that remains to support the system will get to spend some or all of their time dealing with these. For a year or a few years or in perpetuity. Their productivity is directly related to how much time they spend unburdening themselves from that hidden cost.
wodenokoto 2y ago
While I don’t disagree that in the end, the only meaningful measurement is business value, I’m not sure it’s fair to put the value of John’s implemented features against the value of mine. Neither me, nor John had very much say in which feature we get to implement (assuming we are both, say frontend developers. Might make more sense if John is in devops) Those decisions are usually decided from higher up than developers. Maybe John got the job of fixing the advertising embed code in the html template, because he’s a junior and its basically just copy paste from the documentation and since all revenue come from ads, he is now responsible for the most valuable feature.
- Juliate 2y ago
  > the only meaningful measurement is business value That's true for the business. But the business is not always the unique and only incentive in work, generally speaking. > Those decisions are usually decided from higher up than developers. This depends a lot on the team structure and culture and size you are in. My first 5 years as a junior developer in a ~100 people software company, a LOT was up to developers/sysadmin left alone to design, triage, prioritise and deliver too without the supervision of a product owner or manager (which was the CTO, the role/concept of architect/PM/PO did not really exist in that environment at the time - circa 2004). I have never felt as productive and fulfilled, neither have been in a similarly fluid and flowing project since then. Sure we delivered. The company still went down for unrelated reasons. So was it productive really? In retrospect, 20 years later, what would have been really more productive (both tech-wise and business-wise)? Spinning off 3/4 internal technologies into collaborative, but dedicated small startups.

34 points by aard 2y ago

> economists are now seeing productivity increases in business due to the computer investments in the nineties. The point is that the improvements lag the investments Aye. And one rarely sees a software team stay intact long enough after shipping software to make a fair evaluation of what they actually delivered. Most of the time the incentive structures cause everyone who didn't get a promotion off the back of shipping to bail, and corporate then scavenges the remainder to staff other projects...

andrelaszlo 2y ago
Other things that make it difficult, some mentioned in the article: - It's a strange activity. The output is non-linear, non-repeatable, and basically chaotic in some ways. - You almost never build the same thing twice. Even if you do, productivity is affected (positively or negatively) just by the fact that you already did it before. - The value of what you produce is unknown or fluctuating wildly. You might be a unicorn one day and bankrupt (even personally liable!) the next. Less extreme examples are interesting too, maybe your software is a plugin for a product that stops supporting third party plugins. I think, like the article says, that pretending to be able to measure output just because we have something that might look superficially like actual output in other activities (e.g. number of units produced in a factory) is fundamentally misguided. It's a cop-out since what's really missing a lot of the time is (good) leadership.

mkesper 2y ago

(2003)

oldpersonintx 2y ago

[dead]

d--b 2y ago

And yet, people talk about productivity all the time because they can feel it. People say: "oh I am much more productive with C# than I am with Python". Or: "This guy is one of the most productive engineer I know". But the issue here is that value is subjective. But is that really a problem? "Value" is not better defined than "urgency" or "criticality", yet we prioritize tasks all the time based on the perceived urgency/criticality of task A vs task B. So in the same vein, we could assign a "value score" of project X, that would amount to something like "usefulness * difficulty * quality" (to be discussed), and define productivity as that over time spent.

andrelaszlo 2y ago
I think that's a good point. Say it's your company, then you need to decide if you want to build the product in C# or Python. It's going to affect productivity, but it's very difficult to say how. If you pick Brainf* of course most people can tell you productivity will suffer, but in the C#/Python example you might start building in C# since it's what you're familiar with, then have problems recruiting developers in your area a few years down the line, since most people are now doing Python (say). Technically, your original choice might be "correct" in some ways, but who knows? Perhaps the discussion should revolve around how we make decisions better in complex, rapidly changing environments, with limited information? It feels like we're clinging to the need for things to be predictable and measurable even when they're not, and we end up in more or less delusional discussions about which of two almost identical programming languages is better, or even tabs vs spaces :D
- neonsunset 2y ago
  Perhaps this specific comparison is flawed. It's the same as if comparing "Do we build it in Rust or TypeScript?" (not in terms of hiring pool but in terms of specific language used - picking either Python or C# will result in products with massively different grade of performance and quality).
Juliate 2y ago
The hint that productivity is perceived (subjective, qualitative), but not measured (quantitative), is good. Productivity is argued, in the end, not demonstrated; or only within a very specific, limited framework (a very productive endeavour locally may reveal to be counter-productive on a higher or later scale). For instance, heavily relying on fossile fuel while pushing CO2 in the atmosphere has proved very successful in a specific time frame, the past century, let's say. But if (if) that means that Earth becomes inhabitable by current life forms within the next century, from an external observer, it was rather an error, or mismanaged, so not so productive.

svilen_dobrev 2y ago

in software making, what's Doable, Sellable, Wanted, and Needed, rarely overlap.. When they do, there's a dream come true. But - how to measure that? Is that productivity? or acumen? or luck? Which may hint at Exupery.. How do you measure love?

deleted 2y ago

[deleted]

deleted 2y ago

[deleted]

thread_id 2y ago

The best measure of productivity for a software implementation team is the quality of the product as measured by latent defects and technical debt. The team that remains to support the system will get to spend some or all of their time dealing with these. For a year or a few years or in perpetuity. Their productivity is directly related to how much time they spend unburdening themselves from that hidden cost.

wodenokoto 2y ago

While I don’t disagree that in the end, the only meaningful measurement is business value, I’m not sure it’s fair to put the value of John’s implemented features against the value of mine. Neither me, nor John had very much say in which feature we get to implement (assuming we are both, say frontend developers. Might make more sense if John is in devops) Those decisions are usually decided from higher up than developers. Maybe John got the job of fixing the advertising embed code in the html template, because he’s a junior and its basically just copy paste from the documentation and since all revenue come from ads, he is now responsible for the most valuable feature.

Juliate 2y ago
> the only meaningful measurement is business value That's true for the business. But the business is not always the unique and only incentive in work, generally speaking. > Those decisions are usually decided from higher up than developers. This depends a lot on the team structure and culture and size you are in. My first 5 years as a junior developer in a ~100 people software company, a LOT was up to developers/sysadmin left alone to design, triage, prioritise and deliver too without the supervision of a product owner or manager (which was the CTO, the role/concept of architect/PM/PO did not really exist in that environment at the time - circa 2004). I have never felt as productive and fulfilled, neither have been in a similarly fluid and flowing project since then. Sure we delivered. The company still went down for unrelated reasons. So was it productive really? In retrospect, 20 years later, what would have been really more productive (both tech-wise and business-wise)? Spinning off 3/4 internal technologies into collaborative, but dedicated small startups.