6 ms·

That's a highly simplistic and incorrect viewpoint—and the correctness of a model is not dependent on popular vote. When looking at increasing quality (essenti

That's a highly simplistic and incorrect viewpoint—and the correctness of a model is not dependent on popular vote. When looking at increasing quality (essentially, scope) in isolation of systems and the environment in which that quality is created, then the only way to increase quality is to increase cost. That much is true, but it's also incomplete. When looking at and eventually optimizing the whole system—from the customer and their needs, to the leadership of the company, to the clarity of purpose, to values and culture, to the intrinsic motivation of your employees, to methods of management, to your learning and education processes, to your processes of production, to your communication links, to your output and how it's measured, and how each is controlled and understood in great detail—you can and will improve quality much more than you could by simply increasing cost in isolation—and at the same time, you'll decrease your overall costs in the long term, you'll speed up production, and you'll increase sales and customer satisfaction. The end result is a better product, for lower cost, with more predictability, proud happy employees, and faster to boot. That's what Quality really means. Looking at quality in terms of cost alone is the naive mistake. Look at the whole system. And for some evidence, look at Japan circa 1948. These exact methods of systemic quality, "total quality control" and statistical quality control taught to Japan by W. Edwards Deming turned them from the ruins of World War II into the second largest economy in the world, to the point of such success that Japanese companies were putting American ones to shame by the 70's. They did this by following exactly the model I've described above, decreasing total costs, increasing quality, and improving time to market, all by focusing on their total systems across the organization. The proof is in the success of an entire country's economy, I don't know how it can get much clearer than that. It is not my fault that everyone in our industry has forgotten these lessons on running companies, but the prevailing methods remain incorrect, as does your original statement.

kjksf 11y ago
Either you're wrong or literally every software company on the planet is wrong. I'm betting it's you. Please do prove us all wrong by starting up a software company that produces bug free software faster and cheaper than every other software company on the planet. There's a big pot of gold at the end of that journey. Go and claim it. If everyone else is as incompetent as you claim, it's going to be a walk in the park.
- calinet6 11y ago
  Not only software companies, but most companies in the US are indeed wrong about the prevailing methods and styles of management and leadership. Is this really that surprising to you? Do companies you work with appear chaotic yet surprisingly successful? Do the same things keep going wrong? Do they get in cycles of cultural upheaval? And do we accept these things as "normal business" and go on with out lives, powerless to control them? And I'm not saying it's easy. Not at all. It might be the hardest problem in all of business—combining the true systems of work into a cohesive philosophy: systems, statistics, people, and knowledge. I'm just saying (and I'm just repeating the words of some great thinkers in the quality space, mind you) that if we change the way we think about quality and companies in general, we'll get a lot of gigantic unrealized gains in productivity. This happens naturally in many companies in small pockets, but human nature and psychology tends to break it down eventually. If I'm wrong, then a great many other people are also wrong, primarily W. Edwards Deming, who, as I noted, single-handedly turned around the entire economy of Japan using these exact methods and ideas. Here's a great place to start: http://www.amazon.com/Leaders-Handbook-Making-Things-Getting/dp/0070580286 http://www.amazon.com/Leaders-Handbook-Making-Things-Getting...
kabdib 11y ago
If true, this is truly amazing. So here's a quick test: 1. Write a letter outlining the above to a VC. Explain how this works in somewhat more detail; 10-15 pages should get it across. Sure it's work. But you seem resolved, and the reward is great. If you pull this off, not only are you gonna get a Turing award, they're going to mint a whole new award and name it after you. Anyway, give it a decent treatment and mail it off. 2. Tell us how that went over. Hundreds of thousands, heck, millions of pretty smart people have been thinking of ways to solve "the software problem" for over 60 years. Many of those people are smarter than I am, and probably smarter than most of the readership on HN. The actual improvements have been incremental. I'm not saying there hasn't been improvement. For instance, we tend to write more secure systems these days, but security remains really, really hard. Saying "you're doing it all wrong" to the incredibly smart and driven people I know who are experts in security is pretty bold. So, I urge you to make a proposal and report back.
- calinet6 11y ago
  Interesting sarcastic, insulting proposal. I'm tackling just that issue within my own company now, and it's incredibly difficult. I'll explain why. The main issue is that once you exceed a certain number of developers, the problems become more human than software. It's not the software problem we need to solve, and our solution to the human problem thus far is highly individual and anti-system, riddled with ineffective processes, "communication problems," and "culture issues" that we believe are someone else's problem that we can do very little to fix even as leaders. These are the root cause of quality issues, and without looking at the whole system—and indeed, creating a whole organization capable of looking at the whole system—the quality issues will continue dependably. The final issue is that when it comes to people problems, everyone has their own beliefs. Ideas are easy—ideas can be explained, processes implemented, software designed—but beliefs are hard. If I were to write to a VC outlining these ideas (which are W. Edwards Deming's ideas circa 1950, Joseph Juran's ideas circa 1980, and Peter Sholtes's ideas continuing into the 90's, and the foundation of the current Lean movement in software, education, manufacturing, and healthcare)‚ what would really change? I can make a convincing argument, but I would be in for a 1-3 year conversation about beliefs about management and people, depending on the person and their adaptability and openness to new ideas. It's much easier to simply do it, and I'm trying to do as much as I can in the context of a growing company at present. Plus, there are companies who are already into this stuff. Pluralsight integrates Deming's philosophy into their management and onboarding. Check them out. Someday I will start my own company, and these concepts will be at the foundation. For sure. Until then, I'll try my best to shift some perspectives.
EdSharkey 11y ago
For all you detractors, here's my story, maybe it will help you believe that quality is possible in our industry. I have been doing professional software development since 1997, and from then until two years ago, I have been a developer on waterfall and scrummerfall projects. There have been deathmarches aplenty and lots of failures small and large with some heroic successes here and there. Overall, I'd say the performance of the teams I had been on has been fair to poor, and the code quality has been generally poor because it was always rushed. Then two years ago, my manager got on a quality kick and the team was largely with him because we were frustrated with some boneheaded defects that had escaped to production. First, we started to require some unit testing here and there, and code quality started to improve, at least to those of us on the team who noticed such things. Next, we all decided on a new greenfield project that we would enforce a test-first/TDD rule. Our code coverage was suddenly obscene! Months later, we had no escape defects when we delivered to production on-time. No escape defects was unheard of in our division, and a point of pride for the manager. The TDD we were doing was largely just a gentlemen's agreement, and there was little pairing. Mainly we all still worked in our little heads-down solo developer modes. So, manager decided to amp up the agile more. We got TDD training ("obey the testing goat" video series) and we became much more open to the idea of pairing. The QA team was fired, we ended up with one floating QE engineer. He occasionally pairs with us, developing for real stories. An agile coach came to visit us and gave some tips. Got scrummaster training, bleh ... On the next project, also a greenfield, we were much more confident. Pairing happened very dependably whenever we did any new development. We were less intent on metrics such as coverage, and more intent on writing our unit tests like they were a software spec. Spock Framework for Java is a really nice system for doing this, our tests read almost like english paragraphs. Our defect rate is similarly low like it was on the previous project, but now we have a much better comprehension of the code. I feel like the whole team is communicating better and can understand the subtle design direction changes that naturally happen throughout the course of a project. We just completed our first story mapping exercise for this project, it's really neat to see our plans all laid out on the board like that and this has been very helpful for our product owner. There's a lot wrong with our situation, but we're impatient to fix things since we don't have defects to mentally burden us constantly. A one-click build would be really nice, and our overall system design handed down to us from on-high is heavy-handed. We're so agile now we can usually devise a long-term plan to address our issues that everyone is comfortable with. Our product owner is struggling to get better definition around aspects of the project, but we make it possible for him to shift directions - again because the codebase is well-factored and largely free from bugs. My main observation looking back on my 18 years of coding is that software takes a long time to write. It's really excruciating. Trying to turbo boost a project by overcommitting resources and doing death marches usually doesn't shorten the actual time to delivery all that much in my estimation, but it DOES drive up the risk because people are going to fail on you left and right. This excruciating slowness to develop software has only become more evident to me now that I'm on a team that is genuinely trying to be agile. Every little feature and every task on the board is SOOOO drawn out and painstakingly developed by pairs of developers. DevOps is a big deal that saps tons of the team's time, but it is important for us to own this. I just never thought I would see proper resource allocation to software projects, but here it is. My company is spending the money it needs to AND the management, product owner, and developers on the team are all willing to work this way. We took no shortcuts and focus on code quality. Effort put into documentation is fairly lite, unless it is intended for the user. I can count on one hand the number of days I was asked to work overtime in the last 2 years. The question I have is, if high quality software development is possible, is it a lot more expensive than the old death march/waterfall/lie-to-ourselves-constantly style of development? My team no longer has show-stopper production environment defects nor a QA department anymore. Surely that has to cut our costs a lot! We do have a dependable cadence, and have recently been seeing our velocity accelerate. While we do meet our product owner's goals, but I'd say our progress feels achingly slow at times. Maybe I'm just impatient or restless. I'll need to ask my manager if he can gauge whether we're more or less expensive to run than we were in the past.
p0nce 11y ago
It's amazing you have to justify this view point. There are books precisely documenting the correlation between quality, speed of execution and lower costs. http://ptgmedia.pearsoncmg.com/images/9780132582209/samplepages/0132582201.pdf http://ptgmedia.pearsoncmg.com/images/9780132582209/samplepa...
- calinet6 11y ago
  Exactly. The resistance to change is stunning. It's an ingrained psychology of individuality continuing to dig our management hole for us. Ignore the system at your peril.
  - lucozade 11y ago
    > The resistance to change is stunning If you ask a bunch of people who are analytically trained to ignore all the evidence in front of them you should expect this reaction. Especially if you tell them, to paraphrase, all they need to do is change their culture and all processes and everything will be rosy. Pointing to books written by quality consultants isn't going to help your case much either I'm afraid. Which is a shame. If you've ever been in an organisation that has this deep-seated view of quality it is quite an eye-opener. Of those that I am familiar with, only GE really were able to take it beyond manufacturing and definitely the only one where I saw it applied to software. It felt a little "religious cult"y at first but was pretty impressive nonetheless. Your focus on Edwards Deming is a little unfortunate too, as I would expect that most people with an engineering bent will look at his work on SPC and, rightly, consider it inapplicable to software. It requires a high level of repetition to be valuable and software development doesn't have that. His work on organisations could be applicable but it would require most companies to fundamentally change the way they approach almost everything. On the basis that it's not necessary to do this, I can only see it occurring if it became necessary. A special set of circumstances brought it to prevalence in manufacturing and by no means universally. If it happens to software I think it would need an equivalent set.
    - calinet6 11y ago
      Yes. Good feedback, thanks. Deming himself said that any company can grow in a favorable market, so it is difficult to argue for better when indeed even running a company in a mediocre fashion can be highly successful as compared to the baseline. What Deming's organizational view promises, instead, is joy in work. Very simply, that's what I'm really after.

kjksf 11y ago

Either you're wrong or literally every software company on the planet is wrong. I'm betting it's you. Please do prove us all wrong by starting up a software company that produces bug free software faster and cheaper than every other software company on the planet. There's a big pot of gold at the end of that journey. Go and claim it. If everyone else is as incompetent as you claim, it's going to be a walk in the park.

calinet6 11y ago
Not only software companies, but most companies in the US are indeed wrong about the prevailing methods and styles of management and leadership. Is this really that surprising to you? Do companies you work with appear chaotic yet surprisingly successful? Do the same things keep going wrong? Do they get in cycles of cultural upheaval? And do we accept these things as "normal business" and go on with out lives, powerless to control them? And I'm not saying it's easy. Not at all. It might be the hardest problem in all of business—combining the true systems of work into a cohesive philosophy: systems, statistics, people, and knowledge. I'm just saying (and I'm just repeating the words of some great thinkers in the quality space, mind you) that if we change the way we think about quality and companies in general, we'll get a lot of gigantic unrealized gains in productivity. This happens naturally in many companies in small pockets, but human nature and psychology tends to break it down eventually. If I'm wrong, then a great many other people are also wrong, primarily W. Edwards Deming, who, as I noted, single-handedly turned around the entire economy of Japan using these exact methods and ideas. Here's a great place to start: http://www.amazon.com/Leaders-Handbook-Making-Things-Getting/dp/0070580286 http://www.amazon.com/Leaders-Handbook-Making-Things-Getting...

kabdib 11y ago

If true, this is truly amazing. So here's a quick test: 1. Write a letter outlining the above to a VC. Explain how this works in somewhat more detail; 10-15 pages should get it across. Sure it's work. But you seem resolved, and the reward is great. If you pull this off, not only are you gonna get a Turing award, they're going to mint a whole new award and name it after you. Anyway, give it a decent treatment and mail it off. 2. Tell us how that went over. Hundreds of thousands, heck, millions of pretty smart people have been thinking of ways to solve "the software problem" for over 60 years. Many of those people are smarter than I am, and probably smarter than most of the readership on HN. The actual improvements have been incremental. I'm not saying there hasn't been improvement. For instance, we tend to write more secure systems these days, but security remains really, really hard. Saying "you're doing it all wrong" to the incredibly smart and driven people I know who are experts in security is pretty bold. So, I urge you to make a proposal and report back.

calinet6 11y ago
Interesting sarcastic, insulting proposal. I'm tackling just that issue within my own company now, and it's incredibly difficult. I'll explain why. The main issue is that once you exceed a certain number of developers, the problems become more human than software. It's not the software problem we need to solve, and our solution to the human problem thus far is highly individual and anti-system, riddled with ineffective processes, "communication problems," and "culture issues" that we believe are someone else's problem that we can do very little to fix even as leaders. These are the root cause of quality issues, and without looking at the whole system—and indeed, creating a whole organization capable of looking at the whole system—the quality issues will continue dependably. The final issue is that when it comes to people problems, everyone has their own beliefs. Ideas are easy—ideas can be explained, processes implemented, software designed—but beliefs are hard. If I were to write to a VC outlining these ideas (which are W. Edwards Deming's ideas circa 1950, Joseph Juran's ideas circa 1980, and Peter Sholtes's ideas continuing into the 90's, and the foundation of the current Lean movement in software, education, manufacturing, and healthcare)‚ what would really change? I can make a convincing argument, but I would be in for a 1-3 year conversation about beliefs about management and people, depending on the person and their adaptability and openness to new ideas. It's much easier to simply do it, and I'm trying to do as much as I can in the context of a growing company at present. Plus, there are companies who are already into this stuff. Pluralsight integrates Deming's philosophy into their management and onboarding. Check them out. Someday I will start my own company, and these concepts will be at the foundation. For sure. Until then, I'll try my best to shift some perspectives.

EdSharkey 11y ago

For all you detractors, here's my story, maybe it will help you believe that quality is possible in our industry. I have been doing professional software development since 1997, and from then until two years ago, I have been a developer on waterfall and scrummerfall projects. There have been deathmarches aplenty and lots of failures small and large with some heroic successes here and there. Overall, I'd say the performance of the teams I had been on has been fair to poor, and the code quality has been generally poor because it was always rushed. Then two years ago, my manager got on a quality kick and the team was largely with him because we were frustrated with some boneheaded defects that had escaped to production. First, we started to require some unit testing here and there, and code quality started to improve, at least to those of us on the team who noticed such things. Next, we all decided on a new greenfield project that we would enforce a test-first/TDD rule. Our code coverage was suddenly obscene! Months later, we had no escape defects when we delivered to production on-time. No escape defects was unheard of in our division, and a point of pride for the manager. The TDD we were doing was largely just a gentlemen's agreement, and there was little pairing. Mainly we all still worked in our little heads-down solo developer modes. So, manager decided to amp up the agile more. We got TDD training ("obey the testing goat" video series) and we became much more open to the idea of pairing. The QA team was fired, we ended up with one floating QE engineer. He occasionally pairs with us, developing for real stories. An agile coach came to visit us and gave some tips. Got scrummaster training, bleh ... On the next project, also a greenfield, we were much more confident. Pairing happened very dependably whenever we did any new development. We were less intent on metrics such as coverage, and more intent on writing our unit tests like they were a software spec. Spock Framework for Java is a really nice system for doing this, our tests read almost like english paragraphs. Our defect rate is similarly low like it was on the previous project, but now we have a much better comprehension of the code. I feel like the whole team is communicating better and can understand the subtle design direction changes that naturally happen throughout the course of a project. We just completed our first story mapping exercise for this project, it's really neat to see our plans all laid out on the board like that and this has been very helpful for our product owner. There's a lot wrong with our situation, but we're impatient to fix things since we don't have defects to mentally burden us constantly. A one-click build would be really nice, and our overall system design handed down to us from on-high is heavy-handed. We're so agile now we can usually devise a long-term plan to address our issues that everyone is comfortable with. Our product owner is struggling to get better definition around aspects of the project, but we make it possible for him to shift directions - again because the codebase is well-factored and largely free from bugs. My main observation looking back on my 18 years of coding is that software takes a long time to write. It's really excruciating. Trying to turbo boost a project by overcommitting resources and doing death marches usually doesn't shorten the actual time to delivery all that much in my estimation, but it DOES drive up the risk because people are going to fail on you left and right. This excruciating slowness to develop software has only become more evident to me now that I'm on a team that is genuinely trying to be agile. Every little feature and every task on the board is SOOOO drawn out and painstakingly developed by pairs of developers. DevOps is a big deal that saps tons of the team's time, but it is important for us to own this. I just never thought I would see proper resource allocation to software projects, but here it is. My company is spending the money it needs to AND the management, product owner, and developers on the team are all willing to work this way. We took no shortcuts and focus on code quality. Effort put into documentation is fairly lite, unless it is intended for the user. I can count on one hand the number of days I was asked to work overtime in the last 2 years. The question I have is, if high quality software development is possible, is it a lot more expensive than the old death march/waterfall/lie-to-ourselves-constantly style of development? My team no longer has show-stopper production environment defects nor a QA department anymore. Surely that has to cut our costs a lot! We do have a dependable cadence, and have recently been seeing our velocity accelerate. While we do meet our product owner's goals, but I'd say our progress feels achingly slow at times. Maybe I'm just impatient or restless. I'll need to ask my manager if he can gauge whether we're more or less expensive to run than we were in the past.

p0nce 11y ago

It's amazing you have to justify this view point. There are books precisely documenting the correlation between quality, speed of execution and lower costs. http://ptgmedia.pearsoncmg.com/images/9780132582209/samplepages/0132582201.pdf http://ptgmedia.pearsoncmg.com/images/9780132582209/samplepa...

calinet6 11y ago
Exactly. The resistance to change is stunning. It's an ingrained psychology of individuality continuing to dig our management hole for us. Ignore the system at your peril.
- lucozade 11y ago
  > The resistance to change is stunning If you ask a bunch of people who are analytically trained to ignore all the evidence in front of them you should expect this reaction. Especially if you tell them, to paraphrase, all they need to do is change their culture and all processes and everything will be rosy. Pointing to books written by quality consultants isn't going to help your case much either I'm afraid. Which is a shame. If you've ever been in an organisation that has this deep-seated view of quality it is quite an eye-opener. Of those that I am familiar with, only GE really were able to take it beyond manufacturing and definitely the only one where I saw it applied to software. It felt a little "religious cult"y at first but was pretty impressive nonetheless. Your focus on Edwards Deming is a little unfortunate too, as I would expect that most people with an engineering bent will look at his work on SPC and, rightly, consider it inapplicable to software. It requires a high level of repetition to be valuable and software development doesn't have that. His work on organisations could be applicable but it would require most companies to fundamentally change the way they approach almost everything. On the basis that it's not necessary to do this, I can only see it occurring if it became necessary. A special set of circumstances brought it to prevalence in manufacturing and by no means universally. If it happens to software I think it would need an equivalent set.
  - calinet6 11y ago
    Yes. Good feedback, thanks. Deming himself said that any company can grow in a favorable market, so it is difficult to argue for better when indeed even running a company in a mediocre fashion can be highly successful as compared to the baseline. What Deming's organizational view promises, instead, is joy in work. Very simply, that's what I'm really after.