3 ms·
> attack someone personally for dishonesty Hackers hate ad hominem, and with good reason. I too subscribe to the school of "harsh on the problem, soft on the p
by Morendil 14y ago
> attack someone personally for dishonesty
Hackers hate ad hominem, and with good reason. I too subscribe to the school of "harsh on the problem, soft on the person". On the other hand, it makes no sense not to call out things that keep us in the swamp, or to tiptoe around important epistemic issues just to spare hurt feelings.
One problem we have is that few people are willing to go to great lengths to check out the available evidence; instead the pattern is to repeat claims (and associated citations) made by people who sound authoritative, accepting them essentially on faith. This has the unfortunate side-effect of magnifying the mistakes of people who have become authorities.
In "Leprechauns" and elsewhere, my focus isn't on what any particular person says, but on specific claims. "The cost of fixing defects rises exponentially as a function of SDLC phase" isn't tied to a particular person - it originated with Boehm but many others have propagated it. My method has been to look up the evidence and to see if it held up. Also to think hard about why studies may have failed to show conclusively what they set out to prove, and how to overcome these challenges.
I'm doing the same kind of thing in my own area, i.e. I'm collecting all available evidence, pro or con, about whether various Agile practices "work".
> is there any finding in the software engineering literature that you think holds up
There are many good ideas and rules of thumb, but when it comes to very general "laws", solidly established - that's harder. I've been asking that question over and over again, hoping to get a convincing answer. Still haven't got one.
I've read a bunch of supposedly solid references, e.g. Pressman, or the "Handbook of Software and Systems Engineering" and have been underwhelmed. (The first "law" proposed in the Handbook: "Requirement deficiences are the prime source of project failures," based on evidence like the Chaos Reports. My rebuttal: http://lesswrong.com/lw/amt/causal_diagrams_and_software_engineering/ http://lesswrong.com/lw/amt/causal_diagrams_and_software_eng... )
- gruseom 14y agoThe thing about "many good ideas and rules of thumb" is, I've got a few dozen of those of my own! Most of us do. It would be interesting if there were decisive evidence against any of them, but even when I read studies whose conclusions contradict my beliefs, the studies are so flimsy that I find it easy to keep my beliefs. There does seem to be a recent wave of software engineering literature, exemplified by http://www.amazon.com/Making-Software-Really-Works-Believe/dp/0596808321 http://www.amazon.com/Making-Software-Really-Works-Believe/d... (which I haven't read). Are you familiar with this more recent stuff? Does it represent new research or merely new reporting on old research? If the former, are the standards higher?
- mcguire 14y agoI haven't read all of Making Software yet myself, but you would be interested in one of the first few chapters. I don't remember who wrote it offhand, but as I recall, in discussing the standards of evidence needed for software engineering the author concluded, and I am paraphrasing here, that hard numbers were difficult to get and came with many, many conditions; as a result anecdotes were likely the best you could do and were perfectly acceptable. (Was that enough disclaimers?) You might be able to tell why I lost my enthusiasm for the book.
- gruseom 14y agoThanks. My enthusiasm mostly consists of trying to get other people to read this stuff and tell me what it says :) I think that's the argument for junking the SE literature. If it can't do any better than anecdote, well, to quote Monty Python, we've already got some they're very nice.