7 ms·

Developer Productivity for Humans: Software Quality

133 points by saeedesmaili 3y ago

techdmn 3y ago
In my experience few things sap developer motivation quite the same way as being forced to support, but not allowed to fix(!), a low quality product. The business rationalization makes a certain sort of sense. If we burn, say, 4 hours a week manually cleaning up after bug X, but it would take 80 hours to fix it, maybe the return on investment doesn't compete well against other initiatives this quarter. But in my experience, it often sends a message that we don't care about our users or our products. That isn't something engineers can be proud of at the end of the day, and it makes it difficult to care about your work. This can obviously be overdone in the other direction as well, but I often find the analysis to be penny wise but pound foolish.
- allsunny 3y ago
  The inconvenient truth is that the business exists not to provide engineers with work they’re happy with or proud of but to make investors money. The good news is developer happiness is associated with increased productivity and reduced attrition which usually factors into the business calculus in a much bigger way.
  - klabb3 3y ago
    > The inconvenient truth is that the business exists not to provide engineers with work they’re happy with or proud of but to make investors money. First that’s not always technically true because not all businesses have investors. But assuming that the primary purpose is to make money, even that’s not precise since there’s a massive difference on optimizing for ROI tomorrow vs next quarter vs when you retire and hand your business over to others. Or different metrics altogether like market share or “engagement”. Those are all conflicting goals and have changed a lot just in a decade. But even if you decide precisely which metric to aim for, there’s still a giant difference between primary purpose and only purpose. Simple things like a circuit breaker or a nail might be single purpose. But as soon as you step up almost everything is multi-purpose. A car, a house, even a hat have many degrees of freedom. A business has many unquantifiable components. Anyway, the point is that giving carte blanche to decisions labeled with “business reasons” is at best lazy, and at worst legitimizes a sort of pseudo religious cargo cult worship of the “business truths” of the current year. The truth is that nobody sits on magical predictive abilities, neither the 10x engineers nor the MBA grand poobahs. But it’s also clear that the companies with technical leadership and/or a high degree of engineering agency have been so successful in the last two decades, to the point of redefining the nature of business itself.
    - sokoloff 3y ago
      > not all businesses have investors. If I form a business and take no outside money, I am the investor, particularly in the context of GP’s point that “[businesses exist]…to make investors money.”
      - kdfjgbdfkjgb 3y ago
        what about nonprofits?
        Davidbrcz 3y ago
        Nonprofits still need to break even or raise funds (membership fees, donations).
        musicale 3y ago
        What part of that involves making money for investors?' How much profit are nonprofits expected to make?
        deleted 3y ago
        [deleted]
      - 6gvONxR4sf7o 3y ago
        So in that case, you're investing your time in return for money/happiness? Sounds like the same proposition as employees.
        sokoloff 3y ago
        And money to get the company started. Even before a dollar of revenue comes in, lawyers, accountants, and the state need to be paid, and someone has to front or pay for the real estate. And employees are guaranteed a wage, unlike founders. These differences are often overlooked, seemingly out of convenience for the argument.
      - musicale 3y ago
- linuxlizard 3y ago
  I know it's ridiculous, but I will use my own time to work on some problems that officially we're not allowed to fix. I make that issue my hobby project and use my time to learn some new tool/technology (that looks good on a resume). For example, right now I'm learning Rust to fix [a hairy build problem]. Taking on the problem gives me back some that developer motivation. Using it as a learning opportunity makes me feel like I'm personally growing.
  - deleted 3y ago
    [deleted]
  - pdimitar 3y ago
    It's not at all ridiculous but at the end of the day it turns out that our excitement for our craft is turned against us (as in making us work more hours for no extra pay or anything at all really except a little better resume as you said; and the cases where the resume gets enriched are IMO super rare).
- ravenstine 3y ago
  Yet if you make that argument to the business itself, they'll say "but we're making record profits and customer satisfaction is up!", confusing the quality and longevity of the product for their ability to market said product and craft customer surveys.
  - devoutsalsa 3y ago
    If raise your prices, cut R&D, and all your disgruntled customers leave, customer satisfaction & profits will go up in the short term!
- ponector 3y ago
  After 10+ years of working as a QA engineer I accepted that every project, every piece of software is a crap, is full of bugs, many of which are not going to be fixed. The difference is only how shitty it is. Internal software of huge corporations are total crap. Software which is facing paying customers is better.
- whstl 3y ago
  > "being forced to support, but not allowed to fix" IMO that's a super important point. Having accountability without control is probably the fastest way to burnout and dissatisfaction in my experience. It doesn't matter the amount of hours you put in, it doesn't matter how much vacation you have. It's the difference between a feeling of accomplishment vs the learned helplessness and Sisyphean feeling of endlessly pushing a stone uphill. And productivity suffers the most. This situation negatively affects the whole system, naturally.
  - nine_zeros 3y ago
    > Having accountability without control My manager: I want this in X days Me: It is unlikely to finish in X days given x,y,z. It is also likely to cause outages. My manager: If it causes outages, we will throw you on a PIP. Me gets burned out. Lesson learned. If a manager forces me ever again, I am going to let the project and the manager fail.
    - sam_bristow 3y ago
      It sucks when you're in that situation, but sometimes a paper-trail and a dose of failure is the only way people learn.
      - nine_zeros 3y ago
        The real problem is too much power to management without accountability. If the team suffers due to management decisions, how about we yank the management first?
        whstl 3y ago
        But also: There is often zero practical need for rushing things, or for having some complicated vanity feature. This way there is a lot of risk and accountability as far as the team is concerned, but there was nobody up the chain asking for it. The game of Chinese Whispers of features (and deadlines!) is probably the most inefficient part of software development.
- deleted 3y ago
  [deleted]
0xbadcafebee 3y ago
If you're going to worry about software quality, you need to stop talking about developer productivity, and start talking about business productivity. You can't objectively measure the productivity of a worker in isolation. You have to measure whether the productivity of a worker increases the overall value of the business, or whatever goals the business has. It goes like this: 1. "We need to improve how fast developers release features". Ok, so we throw out tests. Now they're shipping fast! 2. "We need to improve the quality of developers' code". Ok, so we add quality gates. Now they're shipping better code! 3. "The quality gates are slowing down productivity." Ok, so we'll introduce them at earlier stages, shifting left. Now they're shipping better code faster! 4. "The software works, but the customer is unhappy with the product." Ok, we'll focus on giving the customer regular updates and validating the results are what they expect. Now they're shipping better code faster that the customer likes! 5. "The software is now much more expensive than it used to be." Ok, so we... ...and on, and on. Each time, because nobody is focusing on the entire value chain, only a single thing is getting fixed, and it ends up having knock-on effects. It's not enough to perfect one link in the chain. The whole chain needs to be strong, and communication needs to go both directions, constantly.
- galaxyLogic 3y ago
  I find the concept of "productivity" fascinating. It must be measured in money because (positive) productivity means someone is paying for what you are doing. And in that sense you cannot measure the productivity of a single person alone, you must measure what people are willing to pay for their output. If you produce exactly the same as you did last month but nobody is buying it this month (or ever again) it means your productivity is zero. Is that true? Or does it mean your productivity is negative? Or is it the case that when you work for someone else your productivity can be measured by the salary they are paying you? That would mean that "productivity is in the eye of the beholder". Your productivity as seen by you is different than your productivity measured by your employer.
  - krapp 3y ago
    >If you produce exactly the same as you did last month but nobody is buying it this month (or ever again) it means your productivity is zero. Is that true? Or does it mean your productivity is negative? No, your productivity has no connection to sales unless your job is in sales. And productivity for individual employees is measured all the time, across all industries. Your time and output is being paid for by your employer, not the customer (except where they are the same person,) therefore productivity is loosely a measure of the value you create for your employer over time.
crabbone 3y ago
Here are some comments in no specific order: * A sample of 9 developers, especially all working for the same company (in similar conditions) isn't much... but this seems to be a sore spot for most research on software development: very small, bordering on unrepresentative samples. Very hard to control for bias. Very hard to establish whether experiment subjects even have the relevant knowledge. * I would very much prefer if code quality wasn't about people's feelings. There's nothing wrong with people's feelings... except I want a metric. It seems like the authors either don't believe it's possible to capture code quality "impartially" or that we are long way away from being able to do so. * In the end of the day, the research behind the article doesn't have "action items". Suppose the reader didn't know about the complexities of measuring quality -- then they would've learned how hard it is. But the article doesn't as much as suggest what needs to be done to measure the quality (better). It's hard to fault the authors for not suggesting anything in this regard, and by their own admission, the metrics that they've found so far are kind of a snake oil... But, I really want a metric. That's why others keep pumping that snake oil. Anyways, I still find the article useful in case anyone needs a brief explanation of the difficulties of assessing software quality.
- loup-vaillant 3y ago
  Here's the beginning of a metric: https://loup-vaillant.fr/articles/source-of-readability https://loup-vaillant.fr/articles/source-of-readability
  - whstl 3y ago
    Wow, that's a good list. I agree with all of those, and am bookmarking this to show to others. On the subject of deep modules, as also recommended by John Ousterhout, I also enjoy a side effect of this approach: flatter dependency trees (internal dependencies). If you have deeper modules, you stop needing 10, 20 levels of modules to accomplish things. Not needing this makes it easier to debug and understand the big picture. IMO, "big picture code readability" is something of an afterthought. Things like Clean Code only care about the "small picture", individual classes and methods. Things that are rarely a problem in practice IMO. Flatter hierarchies also reduces the number of things to maintain/understand: in the frontend, for example, a flat component hierarchy removes the need for components far away in the hierarchy to communicate, so no need for Redux (also no prop-drilling). It also makes things like DI containers not strictly necessary, as you can easily do it manually, if you have fewer layers. Another important thing I realized is flatter hierarchies allow making that dependency graph closer to a tree, rather than a cyclic graph. This reduces cross-cutting concerns and minimizes incidences of "surprise code" that are often super deep into the dependency tree and a common source of bugs. All IMO and IME, of course. But I'm curious if you also share my experience.
danielovichdk 3y ago
"...or that reliability may not even fully depend on code quality at all (“I’ve seen lots of reliable code that’s poor quality.”) They indicated that while these are related, they are not the same concept as “code quality.”" Something being more reliable than something else is an absolute quality measure.
- prmph 3y ago
  Code quality is mostly related to maintainability. A piece of code may be reliable currently, but hard to modify.
bhupesh 3y ago
Off-topic, but was anyone able to locate Part 3 of this series?
- grork 3y ago
  https://ieeexplore.ieee.org/document/10109339 https://ieeexplore.ieee.org/document/10109339 appears to be part 3, based on the authors (collin) LinkedIn publications list.
genr8 3y ago
I liked this article a lot, it really helped to nail down what EXACTLY people mean when it comes to metrics.
deleted 3y ago
[deleted]

133 points by saeedesmaili 3y ago

In my experience few things sap developer motivation quite the same way as being forced to support, but not allowed to fix(!), a low quality product. The business rationalization makes a certain sort of sense. If we burn, say, 4 hours a week manually cleaning up after bug X, but it would take 80 hours to fix it, maybe the return on investment doesn't compete well against other initiatives this quarter. But in my experience, it often sends a message that we don't care about our users or our products. That isn't something engineers can be proud of at the end of the day, and it makes it difficult to care about your work. This can obviously be overdone in the other direction as well, but I often find the analysis to be penny wise but pound foolish.

allsunny 3y ago
The inconvenient truth is that the business exists not to provide engineers with work they’re happy with or proud of but to make investors money. The good news is developer happiness is associated with increased productivity and reduced attrition which usually factors into the business calculus in a much bigger way.
- klabb3 3y ago
  > The inconvenient truth is that the business exists not to provide engineers with work they’re happy with or proud of but to make investors money. First that’s not always technically true because not all businesses have investors. But assuming that the primary purpose is to make money, even that’s not precise since there’s a massive difference on optimizing for ROI tomorrow vs next quarter vs when you retire and hand your business over to others. Or different metrics altogether like market share or “engagement”. Those are all conflicting goals and have changed a lot just in a decade. But even if you decide precisely which metric to aim for, there’s still a giant difference between primary purpose and only purpose. Simple things like a circuit breaker or a nail might be single purpose. But as soon as you step up almost everything is multi-purpose. A car, a house, even a hat have many degrees of freedom. A business has many unquantifiable components. Anyway, the point is that giving carte blanche to decisions labeled with “business reasons” is at best lazy, and at worst legitimizes a sort of pseudo religious cargo cult worship of the “business truths” of the current year. The truth is that nobody sits on magical predictive abilities, neither the 10x engineers nor the MBA grand poobahs. But it’s also clear that the companies with technical leadership and/or a high degree of engineering agency have been so successful in the last two decades, to the point of redefining the nature of business itself.
  - sokoloff 3y ago
    > not all businesses have investors. If I form a business and take no outside money, I am the investor, particularly in the context of GP’s point that “[businesses exist]…to make investors money.”
    - kdfjgbdfkjgb 3y ago
      what about nonprofits?
      - Davidbrcz 3y ago
        Nonprofits still need to break even or raise funds (membership fees, donations).
        musicale 3y ago
        What part of that involves making money for investors?' How much profit are nonprofits expected to make?
        deleted 3y ago
        [deleted]
    - 6gvONxR4sf7o 3y ago
      So in that case, you're investing your time in return for money/happiness? Sounds like the same proposition as employees.
      - sokoloff 3y ago
        And money to get the company started. Even before a dollar of revenue comes in, lawyers, accountants, and the state need to be paid, and someone has to front or pay for the real estate. And employees are guaranteed a wage, unlike founders. These differences are often overlooked, seemingly out of convenience for the argument.
    - musicale 3y ago
linuxlizard 3y ago
I know it's ridiculous, but I will use my own time to work on some problems that officially we're not allowed to fix. I make that issue my hobby project and use my time to learn some new tool/technology (that looks good on a resume). For example, right now I'm learning Rust to fix [a hairy build problem]. Taking on the problem gives me back some that developer motivation. Using it as a learning opportunity makes me feel like I'm personally growing.
- deleted 3y ago
  [deleted]
- pdimitar 3y ago
  It's not at all ridiculous but at the end of the day it turns out that our excitement for our craft is turned against us (as in making us work more hours for no extra pay or anything at all really except a little better resume as you said; and the cases where the resume gets enriched are IMO super rare).
ravenstine 3y ago
Yet if you make that argument to the business itself, they'll say "but we're making record profits and customer satisfaction is up!", confusing the quality and longevity of the product for their ability to market said product and craft customer surveys.
- devoutsalsa 3y ago
  If raise your prices, cut R&D, and all your disgruntled customers leave, customer satisfaction & profits will go up in the short term!
ponector 3y ago
After 10+ years of working as a QA engineer I accepted that every project, every piece of software is a crap, is full of bugs, many of which are not going to be fixed. The difference is only how shitty it is. Internal software of huge corporations are total crap. Software which is facing paying customers is better.
whstl 3y ago
> "being forced to support, but not allowed to fix" IMO that's a super important point. Having accountability without control is probably the fastest way to burnout and dissatisfaction in my experience. It doesn't matter the amount of hours you put in, it doesn't matter how much vacation you have. It's the difference between a feeling of accomplishment vs the learned helplessness and Sisyphean feeling of endlessly pushing a stone uphill. And productivity suffers the most. This situation negatively affects the whole system, naturally.
- nine_zeros 3y ago
  > Having accountability without control My manager: I want this in X days Me: It is unlikely to finish in X days given x,y,z. It is also likely to cause outages. My manager: If it causes outages, we will throw you on a PIP. Me gets burned out. Lesson learned. If a manager forces me ever again, I am going to let the project and the manager fail.
  - sam_bristow 3y ago
    It sucks when you're in that situation, but sometimes a paper-trail and a dose of failure is the only way people learn.
    - nine_zeros 3y ago
      The real problem is too much power to management without accountability. If the team suffers due to management decisions, how about we yank the management first?
      - whstl 3y ago
        But also: There is often zero practical need for rushing things, or for having some complicated vanity feature. This way there is a lot of risk and accountability as far as the team is concerned, but there was nobody up the chain asking for it. The game of Chinese Whispers of features (and deadlines!) is probably the most inefficient part of software development.
deleted 3y ago
[deleted]

0xbadcafebee 3y ago

If you're going to worry about software quality, you need to stop talking about developer productivity, and start talking about business productivity. You can't objectively measure the productivity of a worker in isolation. You have to measure whether the productivity of a worker increases the overall value of the business, or whatever goals the business has. It goes like this: 1. "We need to improve how fast developers release features". Ok, so we throw out tests. Now they're shipping fast! 2. "We need to improve the quality of developers' code". Ok, so we add quality gates. Now they're shipping better code! 3. "The quality gates are slowing down productivity." Ok, so we'll introduce them at earlier stages, shifting left. Now they're shipping better code faster! 4. "The software works, but the customer is unhappy with the product." Ok, we'll focus on giving the customer regular updates and validating the results are what they expect. Now they're shipping better code faster that the customer likes! 5. "The software is now much more expensive than it used to be." Ok, so we... ...and on, and on. Each time, because nobody is focusing on the entire value chain, only a single thing is getting fixed, and it ends up having knock-on effects. It's not enough to perfect one link in the chain. The whole chain needs to be strong, and communication needs to go both directions, constantly.

galaxyLogic 3y ago
I find the concept of "productivity" fascinating. It must be measured in money because (positive) productivity means someone is paying for what you are doing. And in that sense you cannot measure the productivity of a single person alone, you must measure what people are willing to pay for their output. If you produce exactly the same as you did last month but nobody is buying it this month (or ever again) it means your productivity is zero. Is that true? Or does it mean your productivity is negative? Or is it the case that when you work for someone else your productivity can be measured by the salary they are paying you? That would mean that "productivity is in the eye of the beholder". Your productivity as seen by you is different than your productivity measured by your employer.
- krapp 3y ago
  >If you produce exactly the same as you did last month but nobody is buying it this month (or ever again) it means your productivity is zero. Is that true? Or does it mean your productivity is negative? No, your productivity has no connection to sales unless your job is in sales. And productivity for individual employees is measured all the time, across all industries. Your time and output is being paid for by your employer, not the customer (except where they are the same person,) therefore productivity is loosely a measure of the value you create for your employer over time.

crabbone 3y ago

Here are some comments in no specific order: * A sample of 9 developers, especially all working for the same company (in similar conditions) isn't much... but this seems to be a sore spot for most research on software development: very small, bordering on unrepresentative samples. Very hard to control for bias. Very hard to establish whether experiment subjects even have the relevant knowledge. * I would very much prefer if code quality wasn't about people's feelings. There's nothing wrong with people's feelings... except I want a metric. It seems like the authors either don't believe it's possible to capture code quality "impartially" or that we are long way away from being able to do so. * In the end of the day, the research behind the article doesn't have "action items". Suppose the reader didn't know about the complexities of measuring quality -- then they would've learned how hard it is. But the article doesn't as much as suggest what needs to be done to measure the quality (better). It's hard to fault the authors for not suggesting anything in this regard, and by their own admission, the metrics that they've found so far are kind of a snake oil... But, I really want a metric. That's why others keep pumping that snake oil. Anyways, I still find the article useful in case anyone needs a brief explanation of the difficulties of assessing software quality.

loup-vaillant 3y ago
Here's the beginning of a metric: https://loup-vaillant.fr/articles/source-of-readability https://loup-vaillant.fr/articles/source-of-readability
- whstl 3y ago
  Wow, that's a good list. I agree with all of those, and am bookmarking this to show to others. On the subject of deep modules, as also recommended by John Ousterhout, I also enjoy a side effect of this approach: flatter dependency trees (internal dependencies). If you have deeper modules, you stop needing 10, 20 levels of modules to accomplish things. Not needing this makes it easier to debug and understand the big picture. IMO, "big picture code readability" is something of an afterthought. Things like Clean Code only care about the "small picture", individual classes and methods. Things that are rarely a problem in practice IMO. Flatter hierarchies also reduces the number of things to maintain/understand: in the frontend, for example, a flat component hierarchy removes the need for components far away in the hierarchy to communicate, so no need for Redux (also no prop-drilling). It also makes things like DI containers not strictly necessary, as you can easily do it manually, if you have fewer layers. Another important thing I realized is flatter hierarchies allow making that dependency graph closer to a tree, rather than a cyclic graph. This reduces cross-cutting concerns and minimizes incidences of "surprise code" that are often super deep into the dependency tree and a common source of bugs. All IMO and IME, of course. But I'm curious if you also share my experience.

danielovichdk 3y ago

"...or that reliability may not even fully depend on code quality at all (“I’ve seen lots of reliable code that’s poor quality.”) They indicated that while these are related, they are not the same concept as “code quality.”" Something being more reliable than something else is an absolute quality measure.

prmph 3y ago
Code quality is mostly related to maintainability. A piece of code may be reliable currently, but hard to modify.

bhupesh 3y ago

Off-topic, but was anyone able to locate Part 3 of this series?

grork 3y ago
https://ieeexplore.ieee.org/document/10109339 https://ieeexplore.ieee.org/document/10109339 appears to be part 3, based on the authors (collin) LinkedIn publications list.

genr8 3y ago

I liked this article a lot, it really helped to nail down what EXACTLY people mean when it comes to metrics.

deleted 3y ago

[deleted]