7 ms·

Quality and Effort (2018)

125 points by absolute100 5y ago

absolute100 5y ago
Seth Godin reminds us that the organization is part of the system that builds systems: "We need to put care into our systems. We need to build checklists and peer review and resilience into the way we express our carefulness.[...] If it matters enough to be careful, it matters enough to build a system around it."
- starkd 5y ago
  All well and good. Unless the system is so good it puts everyone to sleep.
iamgopal 5y ago
Poka Yoke.
- Jtsummers 5y ago
  https://en.wikipedia.org/wiki/Poka-yoke https://en.wikipedia.org/wiki/Poka-yoke Fool-proofing, in essence (though not the literal translation). For those, like me, who knew they knew the term from somewhere but couldn't place it, or who just don't know it. Related idea in software is to make invalid states unrepresentable in the system. It's also a central theme in systems safety.
ChrisMarshallNY 5y ago
I've always enjoyed the gems that people mine from his blog. I'm big on Quality. Damn near obsessive about it. I worked for a company that was renown for Quality. They were (and will be, for a while), one of those brands that is basically synonymous for "Quality." They got there by a combination of really heavy-duty, enforced process, combined with some awesome "tribal knowledge." They have been doing it for over a century. But it was not a good fit for software development (they make hardware). Way too inflexible. Since leaving that company, I've been trying out some ideas that I had, as I worked there. For the most part, these ideas have been working -quite well; but I haven't really had a chance to scale them. But software quality, these days, sucks like a supermassive, galaxy-core, singularity. It's reached the point where our entire culture is acclimated to crap software, and we even punish efforts to improve it. Something needs to be done, and I've found that small, commonsense habits can go a long way towards that. If we watch a craftsman at work, we will see them do lots of little things, that we barely notice (and that they, themselves, barely notice, as they are habit). These little things are what makes their work "Quality." When they have an apprentice, they are forced to inventory their habits, and articulate them. That's one reason why martial arts schools make their senior belts train junior belts. It's also why I think that today's engineering culture of "24 months, max" at corporations is pretty corrosive to Quality. I've found that writing my code, as if I am going to be using it to train someone else, and doing it long enough to develop Quality habits, helps me to do a better job on it.
- keeganpoppen 5y ago
  i'm assuming from the subject matter and the capitalization of 'Q' in "Quality", that you have read Zen and the Art of Motorcycle Maintenance? (if not... let's just say you are in for a treat :^)
  - ChrisMarshallNY 5y ago
    No, but I think I should get around to it. Been hearing that, for ages. I capitalize “Quality,” because I consider it a Core Tenet of my personal philosophy. I picked that up from that company I mentioned. I often wanted to strangle the inflexible, stubborn bastards, but I have always been in awe of their singleminded focus on Quality.
- themodelplumber 5y ago
  Quality, aka other naughty things like "subjectivity" and "introversion" (Jungian). Seth is also breezy and appropriately shallow in the quantitative sense. He strikes a nice balance.
- vbtemp 5y ago
  What you wrote is really interesting. I too was in a software group at a "hardcore" engineering institution for a while and took just a bit of that rigor with me throughout my career - and have often been seen as the "quality" guy because of it. > It's also why I think that today's engineering culture of "24 months, max" at corporations is pretty corrosive to Quality. This is something I think a lot about. The problem is this is the incentive gradient that is an emergent effect. Obviously, if it was generally in organizations' interest to retain people, there would be some kind of incentive sink to keep them there (i.e., everyone would stop finding more competitive compensation at other companies).
  - ChrisMarshallNY 5y ago
    > The problem is this is the incentive gradient that is an emergent effect. Obviously, if it was generally in organizations' interest to retain people, there would be some kind of incentive sink to keep them there (i.e., everyone would stop finding more competitive compensation at other companies). It's cultural. The tech scene has turned into a "gimmegimmegimme" culture. People have been incentivized to live a lifestyle, and hold values, that are commensurate with significant wealth. My secret is that I've lived like a pauper, my whole life. It has earned me many sneers, but I don't really care. That would not be the case for a lot of people. I'm kinda weird. In my case, I have found that shallow excess is not actually very satisfying, but no one wants to hear that. I'm told that "I'm just jealous, because I don't have it." There's also tremendous cynicism; for good reason. Corporations try to sell asceticism, so they don't need to pay their developers (so their corporate officers and shareholders can live large). I know that there are "compensation consultants" that specialize in synthesizing low salary scales, so corporations can feel justified in offering a pittance. There was that collusion thing, in SV, where all the big companies colluded to suppress salaries. That goes a long way towards earning respect and loyalty. </s> Until there is less of a cultural "pull" to excess, there will always be people using money to "keep score." The recording industry figured this out, long ago. Most of the excess that you see musicians (at least, ones getting started) living, is actually owned by their recording company. If they leave the recording company, they lose the crib and the ride. Sort of a seven-figure "company store." I think that fostering an atmosphere of simple, basic respect would go a long ways towards retaining talent. True Respect; not "pretend" respect. Most folks that run companies have very little respect for their employees, and that makes it easy for employees and applicants to justify holding their feet to the fire, compensation-wise. It's really the only leverage that a lot of folks have. They hate me anyway, so why shouldn't I make them pay for it? I think this is obvious in the ridiculous hazing rituals that masquerade as job interviews. Corporations go to drastic lengths to make it plain that they are the ones in charge of the relationship, and that applicants are, in reality, supplicants. This results in cynical people that learn to manipulate the hazing process, and extract what they want; with absolutely no respect or loyalty to the people that deliberately humiliated them, on the way in. I can't say that I blame them. No pride of ownership or craftsmanship. No loyalty. No incentive to leave a legacy. Just "smash and grab." I'll be gone in a year, so why bother writing software that's maintainable? It won't be my problem. It's a very different world, from when I had been doing the interview circuit. Also, as long as customers (including B2B customers) are happy with dreck, then dreck, they get. Why bother putting in the considerable extra cost for Quality, when the suc- um, customers, will be happy with garbage?
ren_engineer 5y ago
sounds like this article might have been inspired by William Edwards Deming who was ignored in the US but Japan adopted his methods and that was a huge factor in their manufacturing success post war. Eventually all the big US car manufacturers basically begged him for help once they started losing market share quality and reliability almost always comes down to the system, not heroic efforts of individual employees https://en.wikipedia.org/wiki/W._Edwards_Deming https://en.wikipedia.org/wiki/W._Edwards_Deming https://deming.org/explore/red-bead-experiment/ https://deming.org/explore/red-bead-experiment/
- Jtsummers 5y ago
  Out of the Crisis was simultaneously one of the best books I've read on the topic of systems, longterm versus short-term thinking, and quality, and one of the most depressing. A very worthwhile book to get and read through, though not in one sitting. It's worth pondering on what he writes and considering how (or if) it applies to your organization as you go through it.
dperfect 5y ago
> They got a bonus of $20 for every question they found where their answer was more correct than the original. > We need to put care into our systems. We need to build checklists and peer review and resilience into the way we express our carefulness. Yes – the systems help, but proper incentives are also super important. Granted, designing a system wherein the incentives are properly aligned with the goal (and not easily manipulated; shortcuts can and will be exploited by humans and ML algorithms alike) is no easy task.
- vxNsr 5y ago
  Right, I think making the right incentives is part of building a better system.
  - kqr 5y ago
    Indeed a fairly profound part of the system. In Donatella Meadows' leverage points, incentives fall under "Rules of the system", i.e. pretty far out! https://miro.medium.com/max/1280/1*f60gJ4MELQIdOoLwTilk-w.png https://miro.medium.com/max/1280/1*f60gJ4MELQIdOoLwTilk-w.pn...
sethammons 5y ago
Off topic, but as a seth myself, i shook my fist at the air when he snagged that domain. In retrospect, he has done much, much more than i ever would
- josephg 5y ago
  Hah most of my friends know me as Seph, and your comment inspired me to register sephs.blog - which was available!
themodelplumber 5y ago
Systems design meta-advice from the writer of a blog known for bite-sized insights. Hmm! I would really like to read more of his takes on systems design, in part because I like bite-sized stuff, but also in part because I want to see aspects like complexity and modeling either embraced or ignored by someone like this. I want to watch that particular bout and see what happens.
tehnub 5y ago
I always recommend the “Field Guide to Understanding Human Error” [0], which discusses a lot of similar ideas. Really helped refine my thinking about systems and errors. [0]: https://www.oreilly.com/library/view/the-field-guide/9781317031833/ https://www.oreilly.com/library/view/the-field-guide/9781317...
closeparen 5y ago
I used to think that quality was about personal traits and state of mind, getting everything right the first time. Over time I've come to realize that quality is iterative. Everything has problems the first time. What separates low and high quality practice in software engineering is the degree to which you follow up on the issues you notice over time. Will you do it now? If not, will you create a ticket? If that ticket is created, will it be prioritized? Low quality is simply loss in that pipeline. High quality is simply being engaged enough with the project to constantly be generating these insights, and systematically running them down, until you don't have any more.
- deleted 5y ago
  [deleted]
- jpittis 5y ago
  Quality isn't only sacrificed due to mismanaged loss in the pipeline. It's also at odds with delivery speed, feature breadth, cost, and itself. Quality is at odds with quality, because quality is not single dimensional!
  - kqr 5y ago
    Quality is not at odds with cost and delivery speed. That's a common misconception, but empirically all sorts of people (perhaps starting with Shewhart) have discovered that a good process leads to lower cost, higher quality, faster delivery. If anything, low quality is what increases cost and delivery time, by adding expensive inspection, re-work, warehousing of defective product, missed returns on development time, customer disappointment, loss of feedback. Quality achieved by exhortation ("Try harder!") is at odds with all sorts of things. Quality achieved by straightening out a process designed to produce defects in large numbers -- not at odds with anything, except perhaps office politics.
caramelcream 5y ago
In case anyone is wondering what "Prodigy" (mentioned in the article) was: https://en.wikipedia.org/wiki/Prodigy_(online_service) https://en.wikipedia.org/wiki/Prodigy_(online_service)
forgingahead 5y ago
This is basically the idea espoused by Scott Adams in his "Systems vs. Goals" post years ago: https://www.scottadamssays.com/2013/11/18/goals-vs-systems/ https://www.scottadamssays.com/2013/11/18/goals-vs-systems/ It's a very successful life hack - system-based approaches can be less prone to errors because it relies less on willpower and individual conscientiousness, both of which can stumble at random points.
sitkack 5y ago
> In school, we harangue kids to be more careful, and spend approximately zero time teaching them to build better systems instead. We ignore checklists and processes because we’ve been taught that they’re beneath us. Feedback. Signals. Systems theory.
kqr 5y ago
Along these lines, the CAST Handbook by Leveson completely changed the way I look at failure, and set me off on a long journey toward higher quality, cheaper development. It is available for free: http://psas.scripts.mit.edu/home/get_file4.php?name=CAST_handbook.pdf http://psas.scripts.mit.edu/home/get_file4.php?name=CAST_han... I also stumbled over a slide deck that may or may not be an understandable summary for one not well versed in the concepts of system safety: http://sunnyday.mit.edu/workshop2019/CAST-Tutorial2019.pdf http://sunnyday.mit.edu/workshop2019/CAST-Tutorial2019.pdf
antipaul 5y ago
Does any large corporation produce quality software, at least most of the time?
ram_rar 5y ago
Over the years, the complexity of our collective knowledge vastly exceeds capacity of any individual to get everything right. This reminds me of the book checklist manifesto [1] Even simple tools like checklist can substantially decrease errors and spare much of our cognitive capacity in doing quality work. [1] https://en.wikipedia.org/wiki/The_Checklist_Manifesto https://en.wikipedia.org/wiki/The_Checklist_Manifesto

125 points by absolute100 5y ago

Seth Godin reminds us that the organization is part of the system that builds systems: "We need to put care into our systems. We need to build checklists and peer review and resilience into the way we express our carefulness.[...] If it matters enough to be careful, it matters enough to build a system around it."

starkd 5y ago
All well and good. Unless the system is so good it puts everyone to sleep.

iamgopal 5y ago

Poka Yoke.

Jtsummers 5y ago
https://en.wikipedia.org/wiki/Poka-yoke https://en.wikipedia.org/wiki/Poka-yoke Fool-proofing, in essence (though not the literal translation). For those, like me, who knew they knew the term from somewhere but couldn't place it, or who just don't know it. Related idea in software is to make invalid states unrepresentable in the system. It's also a central theme in systems safety.

ChrisMarshallNY 5y ago

I've always enjoyed the gems that people mine from his blog. I'm big on Quality. Damn near obsessive about it. I worked for a company that was renown for Quality. They were (and will be, for a while), one of those brands that is basically synonymous for "Quality." They got there by a combination of really heavy-duty, enforced process, combined with some awesome "tribal knowledge." They have been doing it for over a century. But it was not a good fit for software development (they make hardware). Way too inflexible. Since leaving that company, I've been trying out some ideas that I had, as I worked there. For the most part, these ideas have been working -quite well; but I haven't really had a chance to scale them. But software quality, these days, sucks like a supermassive, galaxy-core, singularity. It's reached the point where our entire culture is acclimated to crap software, and we even punish efforts to improve it. Something needs to be done, and I've found that small, commonsense habits can go a long way towards that. If we watch a craftsman at work, we will see them do lots of little things, that we barely notice (and that they, themselves, barely notice, as they are habit). These little things are what makes their work "Quality." When they have an apprentice, they are forced to inventory their habits, and articulate them. That's one reason why martial arts schools make their senior belts train junior belts. It's also why I think that today's engineering culture of "24 months, max" at corporations is pretty corrosive to Quality. I've found that writing my code, as if I am going to be using it to train someone else, and doing it long enough to develop Quality habits, helps me to do a better job on it.

keeganpoppen 5y ago
i'm assuming from the subject matter and the capitalization of 'Q' in "Quality", that you have read Zen and the Art of Motorcycle Maintenance? (if not... let's just say you are in for a treat :^)
- ChrisMarshallNY 5y ago
  No, but I think I should get around to it. Been hearing that, for ages. I capitalize “Quality,” because I consider it a Core Tenet of my personal philosophy. I picked that up from that company I mentioned. I often wanted to strangle the inflexible, stubborn bastards, but I have always been in awe of their singleminded focus on Quality.
themodelplumber 5y ago
Quality, aka other naughty things like "subjectivity" and "introversion" (Jungian). Seth is also breezy and appropriately shallow in the quantitative sense. He strikes a nice balance.
vbtemp 5y ago
What you wrote is really interesting. I too was in a software group at a "hardcore" engineering institution for a while and took just a bit of that rigor with me throughout my career - and have often been seen as the "quality" guy because of it. > It's also why I think that today's engineering culture of "24 months, max" at corporations is pretty corrosive to Quality. This is something I think a lot about. The problem is this is the incentive gradient that is an emergent effect. Obviously, if it was generally in organizations' interest to retain people, there would be some kind of incentive sink to keep them there (i.e., everyone would stop finding more competitive compensation at other companies).
- ChrisMarshallNY 5y ago
  > The problem is this is the incentive gradient that is an emergent effect. Obviously, if it was generally in organizations' interest to retain people, there would be some kind of incentive sink to keep them there (i.e., everyone would stop finding more competitive compensation at other companies). It's cultural. The tech scene has turned into a "gimmegimmegimme" culture. People have been incentivized to live a lifestyle, and hold values, that are commensurate with significant wealth. My secret is that I've lived like a pauper, my whole life. It has earned me many sneers, but I don't really care. That would not be the case for a lot of people. I'm kinda weird. In my case, I have found that shallow excess is not actually very satisfying, but no one wants to hear that. I'm told that "I'm just jealous, because I don't have it." There's also tremendous cynicism; for good reason. Corporations try to sell asceticism, so they don't need to pay their developers (so their corporate officers and shareholders can live large). I know that there are "compensation consultants" that specialize in synthesizing low salary scales, so corporations can feel justified in offering a pittance. There was that collusion thing, in SV, where all the big companies colluded to suppress salaries. That goes a long way towards earning respect and loyalty. </s> Until there is less of a cultural "pull" to excess, there will always be people using money to "keep score." The recording industry figured this out, long ago. Most of the excess that you see musicians (at least, ones getting started) living, is actually owned by their recording company. If they leave the recording company, they lose the crib and the ride. Sort of a seven-figure "company store." I think that fostering an atmosphere of simple, basic respect would go a long ways towards retaining talent. True Respect; not "pretend" respect. Most folks that run companies have very little respect for their employees, and that makes it easy for employees and applicants to justify holding their feet to the fire, compensation-wise. It's really the only leverage that a lot of folks have. They hate me anyway, so why shouldn't I make them pay for it? I think this is obvious in the ridiculous hazing rituals that masquerade as job interviews. Corporations go to drastic lengths to make it plain that they are the ones in charge of the relationship, and that applicants are, in reality, supplicants. This results in cynical people that learn to manipulate the hazing process, and extract what they want; with absolutely no respect or loyalty to the people that deliberately humiliated them, on the way in. I can't say that I blame them. No pride of ownership or craftsmanship. No loyalty. No incentive to leave a legacy. Just "smash and grab." I'll be gone in a year, so why bother writing software that's maintainable? It won't be my problem. It's a very different world, from when I had been doing the interview circuit. Also, as long as customers (including B2B customers) are happy with dreck, then dreck, they get. Why bother putting in the considerable extra cost for Quality, when the suc- um, customers, will be happy with garbage?

ren_engineer 5y ago

sounds like this article might have been inspired by William Edwards Deming who was ignored in the US but Japan adopted his methods and that was a huge factor in their manufacturing success post war. Eventually all the big US car manufacturers basically begged him for help once they started losing market share quality and reliability almost always comes down to the system, not heroic efforts of individual employees https://en.wikipedia.org/wiki/W._Edwards_Deming https://en.wikipedia.org/wiki/W._Edwards_Deming https://deming.org/explore/red-bead-experiment/ https://deming.org/explore/red-bead-experiment/

Jtsummers 5y ago
Out of the Crisis was simultaneously one of the best books I've read on the topic of systems, longterm versus short-term thinking, and quality, and one of the most depressing. A very worthwhile book to get and read through, though not in one sitting. It's worth pondering on what he writes and considering how (or if) it applies to your organization as you go through it.

dperfect 5y ago

> They got a bonus of $20 for every question they found where their answer was more correct than the original. > We need to put care into our systems. We need to build checklists and peer review and resilience into the way we express our carefulness. Yes – the systems help, but proper incentives are also super important. Granted, designing a system wherein the incentives are properly aligned with the goal (and not easily manipulated; shortcuts can and will be exploited by humans and ML algorithms alike) is no easy task.

vxNsr 5y ago
Right, I think making the right incentives is part of building a better system.
- kqr 5y ago
  Indeed a fairly profound part of the system. In Donatella Meadows' leverage points, incentives fall under "Rules of the system", i.e. pretty far out! https://miro.medium.com/max/1280/1*f60gJ4MELQIdOoLwTilk-w.png https://miro.medium.com/max/1280/1*f60gJ4MELQIdOoLwTilk-w.pn...

sethammons 5y ago

Off topic, but as a seth myself, i shook my fist at the air when he snagged that domain. In retrospect, he has done much, much more than i ever would

josephg 5y ago
Hah most of my friends know me as Seph, and your comment inspired me to register sephs.blog - which was available!

themodelplumber 5y ago

Systems design meta-advice from the writer of a blog known for bite-sized insights. Hmm! I would really like to read more of his takes on systems design, in part because I like bite-sized stuff, but also in part because I want to see aspects like complexity and modeling either embraced or ignored by someone like this. I want to watch that particular bout and see what happens.

tehnub 5y ago

I always recommend the “Field Guide to Understanding Human Error” [0], which discusses a lot of similar ideas. Really helped refine my thinking about systems and errors. [0]: https://www.oreilly.com/library/view/the-field-guide/9781317031833/ https://www.oreilly.com/library/view/the-field-guide/9781317...

closeparen 5y ago

I used to think that quality was about personal traits and state of mind, getting everything right the first time. Over time I've come to realize that quality is iterative. Everything has problems the first time. What separates low and high quality practice in software engineering is the degree to which you follow up on the issues you notice over time. Will you do it now? If not, will you create a ticket? If that ticket is created, will it be prioritized? Low quality is simply loss in that pipeline. High quality is simply being engaged enough with the project to constantly be generating these insights, and systematically running them down, until you don't have any more.

deleted 5y ago
[deleted]
jpittis 5y ago
Quality isn't only sacrificed due to mismanaged loss in the pipeline. It's also at odds with delivery speed, feature breadth, cost, and itself. Quality is at odds with quality, because quality is not single dimensional!
- kqr 5y ago
  Quality is not at odds with cost and delivery speed. That's a common misconception, but empirically all sorts of people (perhaps starting with Shewhart) have discovered that a good process leads to lower cost, higher quality, faster delivery. If anything, low quality is what increases cost and delivery time, by adding expensive inspection, re-work, warehousing of defective product, missed returns on development time, customer disappointment, loss of feedback. Quality achieved by exhortation ("Try harder!") is at odds with all sorts of things. Quality achieved by straightening out a process designed to produce defects in large numbers -- not at odds with anything, except perhaps office politics.

caramelcream 5y ago

In case anyone is wondering what "Prodigy" (mentioned in the article) was: https://en.wikipedia.org/wiki/Prodigy_(online_service) https://en.wikipedia.org/wiki/Prodigy_(online_service)

forgingahead 5y ago

This is basically the idea espoused by Scott Adams in his "Systems vs. Goals" post years ago: https://www.scottadamssays.com/2013/11/18/goals-vs-systems/ https://www.scottadamssays.com/2013/11/18/goals-vs-systems/ It's a very successful life hack - system-based approaches can be less prone to errors because it relies less on willpower and individual conscientiousness, both of which can stumble at random points.

sitkack 5y ago

> In school, we harangue kids to be more careful, and spend approximately zero time teaching them to build better systems instead. We ignore checklists and processes because we’ve been taught that they’re beneath us. Feedback. Signals. Systems theory.

kqr 5y ago

Along these lines, the CAST Handbook by Leveson completely changed the way I look at failure, and set me off on a long journey toward higher quality, cheaper development. It is available for free: http://psas.scripts.mit.edu/home/get_file4.php?name=CAST_handbook.pdf http://psas.scripts.mit.edu/home/get_file4.php?name=CAST_han... I also stumbled over a slide deck that may or may not be an understandable summary for one not well versed in the concepts of system safety: http://sunnyday.mit.edu/workshop2019/CAST-Tutorial2019.pdf http://sunnyday.mit.edu/workshop2019/CAST-Tutorial2019.pdf

antipaul 5y ago

Does any large corporation produce quality software, at least most of the time?

ram_rar 5y ago

Over the years, the complexity of our collective knowledge vastly exceeds capacity of any individual to get everything right. This reminds me of the book checklist manifesto [1] Even simple tools like checklist can substantially decrease errors and spare much of our cognitive capacity in doing quality work. [1] https://en.wikipedia.org/wiki/The_Checklist_Manifesto https://en.wikipedia.org/wiki/The_Checklist_Manifesto