5 ms·

Investigating Higher Quality Software

16 points by blrice 12y ago

tribaal 12y ago
Well, it seems like a relatively surprise-free article... Write explicit code, write tests, have QA staff and procedures, build what the stakeholders what. I guess the biggest problem with software quality is not that we don't know how to do it, but that it's expensive.
- cema 12y ago
  Poor software quality is also expensive. I would like to see a study about the balance points and how to achieve them.
  - collyw 12y ago
    Not always.
  - tribaal 12y ago
    Sure, what I meant is most companies will chose cheap-fast over good-fast, in my experience. Flinging crap out fast is sometimes seen as a valid business model (unfortunately): that crap is only expensive to you if you have to maintain it, but that's not your problem if all you're trying to do is to "exit". On the other end of the spectrum (like where I work at now for example), we take a lot of effort and time to make sure our code is not crap - but that is an expensive decision both in terms of money and time.
    - blrice 12y ago
      This. I think there's a big space between cheap-fast (crap) and good-expensive (or just "good"). In my experience, we seem to be way more toward the crap side of things. I think we can make a correction without going too far the other way. You can make sacrifices for speed that don't always mean crap in the end. I'm willing to sacrifice perfect architecture in order to get something out the door, for instance.
panic 12y ago
Deploying 100% bug-free code is the Holy Grail of software development. Is this really true? Judging by their decisions, most people's Holy Grail is more like something reasonably good and shipped on time. 100% bug-free code is already possible, but few are willing to pay for it.
- collyw 12y ago
  Absolutely. University ideals, versus the real world.
- sbov 12y ago
  Is anyone willing to pay for it? Is there any non-trivial code that's been deployed that has been bug free? I mean, supposedly NASA paid $1,000 per line of code and still got 0.11 defects per 1,000 lines. By that metric, windows would cost $40 billion to develop, but it would still have 4,400 defects in it.
  - DanWaterworth 12y ago
    compcert, sel4, etc.
  - panic 12y ago
    Yes, as DanWaterworth has pointed out, people have written large projects with entirely bug-free code (in the sense that it verifiably implements a formal specification). The way these projects are developed is very different from the process described by this article -- a lot of time is spent writing formally-verifiable proofs of correctness.
    - blrice 12y ago
      I'm definitely not advocating throwing out agile and embracing waterfall. I like the ability to iterate quickly. I just think we seem to be taking "move fast and break things" a little too seriously in terms of breaking all the things. Everything in moderation.
      - 12y ago
DanWaterworth 12y ago
The non-negotiable part is that there should be tests, regardless of when they were written. Those tests will be the record of truth for future development. They are proof that the proper specification was implemented, regardless of methodology. I really disagree with this. Asides from the fact that tests can't prove the absence of bugs, it's very easy to write a test suite that avoids the unhandled edge-cases; most people do without thinking about it. Your test suite is the tool you have created to search for bugs and should be written in that adversarial mindset rather than as a certification that you actually did your job.
- BmoreDaniel 12y ago
  Formal verification can prove the absence of bugs.
  - DanWaterworth 12y ago
    I didn't say it couldn't; I didn't say anything about formal verification.

16 points by blrice 12y ago

Well, it seems like a relatively surprise-free article... Write explicit code, write tests, have QA staff and procedures, build what the stakeholders what. I guess the biggest problem with software quality is not that we don't know how to do it, but that it's expensive.

cema 12y ago
Poor software quality is also expensive. I would like to see a study about the balance points and how to achieve them.
- collyw 12y ago
  Not always.
- tribaal 12y ago
  Sure, what I meant is most companies will chose cheap-fast over good-fast, in my experience. Flinging crap out fast is sometimes seen as a valid business model (unfortunately): that crap is only expensive to you if you have to maintain it, but that's not your problem if all you're trying to do is to "exit". On the other end of the spectrum (like where I work at now for example), we take a lot of effort and time to make sure our code is not crap - but that is an expensive decision both in terms of money and time.
  - blrice 12y ago
    This. I think there's a big space between cheap-fast (crap) and good-expensive (or just "good"). In my experience, we seem to be way more toward the crap side of things. I think we can make a correction without going too far the other way. You can make sacrifices for speed that don't always mean crap in the end. I'm willing to sacrifice perfect architecture in order to get something out the door, for instance.

panic 12y ago

Deploying 100% bug-free code is the Holy Grail of software development. Is this really true? Judging by their decisions, most people's Holy Grail is more like something reasonably good and shipped on time. 100% bug-free code is already possible, but few are willing to pay for it.

collyw 12y ago
Absolutely. University ideals, versus the real world.
sbov 12y ago
Is anyone willing to pay for it? Is there any non-trivial code that's been deployed that has been bug free? I mean, supposedly NASA paid $1,000 per line of code and still got 0.11 defects per 1,000 lines. By that metric, windows would cost $40 billion to develop, but it would still have 4,400 defects in it.
- DanWaterworth 12y ago
  compcert, sel4, etc.
- panic 12y ago
  Yes, as DanWaterworth has pointed out, people have written large projects with entirely bug-free code (in the sense that it verifiably implements a formal specification). The way these projects are developed is very different from the process described by this article -- a lot of time is spent writing formally-verifiable proofs of correctness.
  - blrice 12y ago
    I'm definitely not advocating throwing out agile and embracing waterfall. I like the ability to iterate quickly. I just think we seem to be taking "move fast and break things" a little too seriously in terms of breaking all the things. Everything in moderation.
    - 12y ago

DanWaterworth 12y ago

The non-negotiable part is that there should be tests, regardless of when they were written. Those tests will be the record of truth for future development. They are proof that the proper specification was implemented, regardless of methodology. I really disagree with this. Asides from the fact that tests can't prove the absence of bugs, it's very easy to write a test suite that avoids the unhandled edge-cases; most people do without thinking about it. Your test suite is the tool you have created to search for bugs and should be written in that adversarial mindset rather than as a certification that you actually did your job.

BmoreDaniel 12y ago
Formal verification can prove the absence of bugs.
- DanWaterworth 12y ago
  I didn't say it couldn't; I didn't say anything about formal verification.