4 ms·

"The fact that no studies have produced findings that contradict the 10x claim provides even more confidence in the 10x claim. When I consider the number of stu

by Maxious 14y ago

"The fact that no studies have produced findings that contradict the 10x claim provides even more confidence in the 10x claim. When I consider the number of studies that have been done, in aggregate I find the research to be not only suggestive, but conclusive—which is rare in software engineering research." Steve McConnell http://forums.construx.com/blogs/stevemcc/archive/2011/01/09/origins-of-10x-how-valid-is-the-underlying-research.aspx http://forums.construx.com/blogs/stevemcc/archive/2011/01/09... http://news.ycombinator.com/item?id=4117417 http://news.ycombinator.com/item?id=4117417

Morendil 14y ago
The linked post isn't about the 10x claim, it's about the unrelated "defect cost increase" claim.
deleted 14y ago
[deleted]
nateberkopec 14y ago
Yikes, that doesn't strike me as very intellectually sound. "The fact that no studies have produced findings that contradict the existence of a teapot orbiting Jupiter..."
- deleted 14y ago
  [deleted]
- deleted 14y ago
  [deleted]
- gruseom 14y ago
  I think if you read the article you'll find that that's unfair. His argument is that there are many studies that support it and none that contradict it, and that such consistency is unusual in the literature. That may be wrong but it's not trivially stupid as you make it sound.
gruseom 14y ago
Someone needs to defend McConnell here. I'll do it. He's one of the most meticulous people to work with this material. That comes through pretty clearly in the post you linked to. The problem is not McConnell, it's – and this is the dirty secret of software engineering – that the research literature itself is so weak and spotty. Most studies consist of arbitrary metrics on small data sets with no replication, rarely (ever?) making the actual data available. Massive assumptions and confounding factors are typically obvious and ignored (things like: how familiar were the programmers with the tools they were using). The overwhelming factor is the interpretive preferences of the authors, who knit their conclusions out of this dodgy material. As science, it not only does not hold up, it's a joke. On the other hand, what they are trying to study is incredibly complex - not just people, not just software, but people working on software, and even social systems of people working on software. How do you even begin to reliably measure that? Psychology looks like physics by comparison. Yet the resources available to do these studies are a drop in the bucket. This is not something society values – it's not even something our own industry values. So it's unsurprising that what we have so far is at best a smattering. You can argue that these toy studies ought not to be dressed up in formal scientific wear, and I don't disagree – it gives an altogether misleading impression of how solid their conclusions are. But no doubt that's true of a lot of papers. (Oh and on top of all that, much of the literature is hard to track down. Didn't anyone notice that the OP wasn't able to even find half of the citations he was trying to check? Based on that alone he shouldn't be claiming that "the table is completely false", only that he was unable to rebuild it from data.) The real question is whether we should throw it all out as junk or try to make something of it. I think there's a good argument for junking. But it's also reasonable to say: hold on, for all their weaknesses these studies are all we have and they're not nothing. At least someone is trying to work with actual data. So let's consider them, but carefully. Like walking on a swamp. McConnell is one of the few people who have made serious efforts to walk the swamp and filter it somewhat and convey its findings to professional programmers. Of all these people (that I've read), he actually stands out as the most meticulous. To fire cowboy accusations of dishonesty at him personally is likely unfair and misses the point. I do think McConnell overestimates how solid the swamp is, but he's not ignorant about it (read that whole post) and the claim that he outright falsifies citations, or anything like that, demands a high burden of proof. I haven't read Bossavit's critique (should I?) but anyone taking that position had better first make sure he's not standing on a swamp too. That's harder than it sounds.

by Maxious 14y ago

Morendil 14y ago

The linked post isn't about the 10x claim, it's about the unrelated "defect cost increase" claim.

deleted 14y ago

[deleted]

nateberkopec 14y ago

Yikes, that doesn't strike me as very intellectually sound. "The fact that no studies have produced findings that contradict the existence of a teapot orbiting Jupiter..."

deleted 14y ago
[deleted]
deleted 14y ago
[deleted]
gruseom 14y ago
I think if you read the article you'll find that that's unfair. His argument is that there are many studies that support it and none that contradict it, and that such consistency is unusual in the literature. That may be wrong but it's not trivially stupid as you make it sound.

gruseom 14y ago

Someone needs to defend McConnell here. I'll do it. He's one of the most meticulous people to work with this material. That comes through pretty clearly in the post you linked to. The problem is not McConnell, it's – and this is the dirty secret of software engineering – that the research literature itself is so weak and spotty. Most studies consist of arbitrary metrics on small data sets with no replication, rarely (ever?) making the actual data available. Massive assumptions and confounding factors are typically obvious and ignored (things like: how familiar were the programmers with the tools they were using). The overwhelming factor is the interpretive preferences of the authors, who knit their conclusions out of this dodgy material. As science, it not only does not hold up, it's a joke. On the other hand, what they are trying to study is incredibly complex - not just people, not just software, but people working on software, and even social systems of people working on software. How do you even begin to reliably measure that? Psychology looks like physics by comparison. Yet the resources available to do these studies are a drop in the bucket. This is not something society values – it's not even something our own industry values. So it's unsurprising that what we have so far is at best a smattering. You can argue that these toy studies ought not to be dressed up in formal scientific wear, and I don't disagree – it gives an altogether misleading impression of how solid their conclusions are. But no doubt that's true of a lot of papers. (Oh and on top of all that, much of the literature is hard to track down. Didn't anyone notice that the OP wasn't able to even find half of the citations he was trying to check? Based on that alone he shouldn't be claiming that "the table is completely false", only that he was unable to rebuild it from data.) The real question is whether we should throw it all out as junk or try to make something of it. I think there's a good argument for junking. But it's also reasonable to say: hold on, for all their weaknesses these studies are all we have and they're not nothing. At least someone is trying to work with actual data. So let's consider them, but carefully. Like walking on a swamp. McConnell is one of the few people who have made serious efforts to walk the swamp and filter it somewhat and convey its findings to professional programmers. Of all these people (that I've read), he actually stands out as the most meticulous. To fire cowboy accusations of dishonesty at him personally is likely unfair and misses the point. I do think McConnell overestimates how solid the swamp is, but he's not ignorant about it (read that whole post) and the claim that he outright falsifies citations, or anything like that, demands a high burden of proof. I haven't read Bossavit's critique (should I?) but anyone taking that position had better first make sure he's not standing on a swamp too. That's harder than it sounds.