3 ms·
I think an important thing that we're currently missing is some kind of quantitative assessment what influence various measures have on software quality. All we
by adrianN 11y ago
I think an important thing that we're currently missing is some kind of quantitative assessment what influence various measures have on software quality. All we have are "best practices" that all sound like common sense, but without much data backing them.
How much do formal methods really buy us it terms of quality? What's the defect rate if you use language X vs language Y? Is it better to spend lots of time gathering requirements and then produce something that exactly matches the requirements or is it better to iterate as quickly as possible and gather requirements as you go? What's the impact of the team as compared to the methodology?
I suspect the answers to these questions are really murky and its not clear whether there is a single methodology that produces the best quality for the money invested.
- TheOtherHobbes 11y agoActually all we have is faddy unrigorous hobbyisms, mediocre tools, and poor management. If everyone did what Praxis does software would work more reliably, but there would be far fewer programmers and - crucially - far fewer projects of all kinds. We have the software industry we have because businesses don't want to pay for a better industry. It's the classic "worse is better" dilemma. If software were rigorous, the initial cost of many projects would make them look prohibitively expensive. You can try to argue that total costs, including fixes and maintenance, are much lower, but not many managers are going to pay n times more for a project up front in the hope the cost will be n times less over its lifetime - even if there's a serious risk of paying millions or billions and literally getting nothing of use, as regularly happens on very big contracts. Eventually quality would get cheaper. But for now it's as if the entire field is drowning in technical debt, buried under me-too frameworks and languages that do very little to genuinely improve productivity or reliability, supported by "Take the money, promise everything, and hope for the best" commercial attitudes. Unfortunately you'd have to wipe out so many languages, techniques, ideas, systems, practices, and management attitudes, and put so many people out of work - or force them into retraining - that the kind of bug rates Praxis produce are never going to be possible across the industry as a whole.
- HeyLaughingBoy 11y agoOh, we have those. The metrics gathered by a team doing TSP (https://en.wikipedia.org/wiki/Personal_software_process https://en.wikipedia.org/wiki/Personal_software_process) are useful since the SEI (https://en.wikipedia.org/wiki/Software_Engineering_Institute https://en.wikipedia.org/wiki/Software_Engineering_Institute) has data on thousands of projects and the metrics do provide good guidance when used appropriately. It is possible to consistently achieve < 0.1 defects/kloc using TSP. The problem is, having spent seven years on a PSP/TSP team, that it greatly slows the development process so it's really only useful for those few types of projects where the cost of quality is worth it. For the vast majority of software, I'd say that it's better to accept a small number of minor defects in order to ship faster.