5 ms·

I'm disappointed to see the 10x programmer meme hit a publication like the Economist. There's no scientific evidence of 10x differences in programmer productivi

I'm disappointed to see the 10x programmer meme hit a publication like the Economist. There's no scientific evidence of 10x differences in programmer productivity. To quote Laurent Bossavit [1], who's investigated the studies people like McConnell use: "How strong is the support conferred to the 10x claim by the best-reputed list of references, for a reader persistent enough to follow the chain of citations back to primary sources? "Based on our close reading of the “10x files”, we can now answer: quite weak. "Not a single one of the references is to a replication, in the scientific sense of the term, of the original exploratory experiment. "The empirical data is in general quite old, most if not all of it predating widespread use of the Internet - which we can safely expect to have wrought major changes in programming practice. "None of the studies address the question of construct validity, that is, how meaningful it is to speak of an individual programmer’s productivity, and if it is meaningful, whether the experimental measurements line up adequately with that meaning." Don't get me wrong--I've certainly seen 10x (and greater) variations in productivity. But they've been due to factors other than inherent individual capability. They're things like * technical debt * familiarity with the codebase and problem domain * interruptions, distractions, multi-tasking and other office environment issues * choice of language - but this is a distant fourth compared to other the other three On top of all this, the Economist article is a thinly-veiled press release for tenXer, which is itself nothing more than an exercise in testerone-fueled egotism. tenXer measures activity, not productivity [2], and I expect the primary result of increasing your tenXer metrics will be less time thinking, more time hacking, and insufferable smugness. [1] The Leprechauns of Software Engineering explores what science is and how we distinguish between fact and folklore in software engineering. It specifically explores the 10x claim, and determines that it's folklore. http://leanpub.com/leprechauns http://leanpub.com/leprechauns [2] Productivity is defined as output over input, and the output of programming is software capability. No one's come up with a good measure of that yet. So any time someone claims they can prove increased productivity, ask them how they measure it. Chances are good you'll get a bogus value that measures activity instead, such as "lines of code." http://martinfowler.com/bliki/CannotMeasureProductivity.html http://martinfowler.com/bliki/CannotMeasureProductivity.html

dkersten 14y ago
From what I heard the 10x meme came from a study back in the 60's where they timed something like ten programmers programming on punch cards for a couple of hours. Now, I didn't check the facts behind this myself, but if thats the case its hardly scientifically rigorous and hardly relevant at all today.
- dalke 14y ago
  What you heard is incorrect. See http://forums.construx.com/blogs/stevemcc/archive/2011/01/09/origins-of-10x-how-valid-is-the-underlying-research.aspx http://forums.construx.com/blogs/stevemcc/archive/2011/01/09... for an incomplete list of some of the research which have been done, and which conclude that a 5x-25x variation exists.
Retric 14y ago
Wait, so you have seen 10x variations in productivity but assume because 'bob' is say better able to avoid interruptions that does not mean he is a 10x better programmer? Programming is not just a question of time spent in front of a computer soft skills are important. If someone looks at inconsistent requirements and get's clarification rather than jump into a quagmire without end they are being more productive even if there not writing code at the time. I think the 10-50x observed increase in productivity is task specific. But, people also deal with a finite list of specific problems so what if bob is 2x as productive on random tasks if there is 10 years worth of work where he can be 10x as productive he is worth a lot more than the average programmer.
barrkel 14y ago
I've seen developers who create, rather than reduce technical debt; who take longer to get familiar with codebases; have a hard time focusing in the presence of distractions; and who seem to have difficulty ramping up solution domain knowledge (i.e. programming / software engineering techniques). And all of these things seem to be down to individual ability or motivation, i.e. something about the person, rather than external factors. Most developers suck at programming (and that's actually OK). 90% of the code they write is copy and pasted from elsewhere in the codebase or from the web, and then tweaked until it does something approximately correct. They usually need help when solving a new pattern of problem. What they do often have is domain knowledge not related to programming, and produce just about acceptable results out of the machine in a labour-intensive way that doesn't really scale over time or problem size, but they muddle through with determination. I don't know about a 10x productivity differences though. I'm not really arguing in favour of that hypothesis. Rather, if anything, I've seen step differences in potential; that is, there are developers that can do things other developers can never do, not in 10x the time, not in 1000x the time.

dkersten 14y ago

From what I heard the 10x meme came from a study back in the 60's where they timed something like ten programmers programming on punch cards for a couple of hours. Now, I didn't check the facts behind this myself, but if thats the case its hardly scientifically rigorous and hardly relevant at all today.

dalke 14y ago
What you heard is incorrect. See http://forums.construx.com/blogs/stevemcc/archive/2011/01/09/origins-of-10x-how-valid-is-the-underlying-research.aspx http://forums.construx.com/blogs/stevemcc/archive/2011/01/09... for an incomplete list of some of the research which have been done, and which conclude that a 5x-25x variation exists.

Retric 14y ago

Wait, so you have seen 10x variations in productivity but assume because 'bob' is say better able to avoid interruptions that does not mean he is a 10x better programmer? Programming is not just a question of time spent in front of a computer soft skills are important. If someone looks at inconsistent requirements and get's clarification rather than jump into a quagmire without end they are being more productive even if there not writing code at the time. I think the 10-50x observed increase in productivity is task specific. But, people also deal with a finite list of specific problems so what if bob is 2x as productive on random tasks if there is 10 years worth of work where he can be 10x as productive he is worth a lot more than the average programmer.

barrkel 14y ago

I've seen developers who create, rather than reduce technical debt; who take longer to get familiar with codebases; have a hard time focusing in the presence of distractions; and who seem to have difficulty ramping up solution domain knowledge (i.e. programming / software engineering techniques). And all of these things seem to be down to individual ability or motivation, i.e. something about the person, rather than external factors. Most developers suck at programming (and that's actually OK). 90% of the code they write is copy and pasted from elsewhere in the codebase or from the web, and then tweaked until it does something approximately correct. They usually need help when solving a new pattern of problem. What they do often have is domain knowledge not related to programming, and produce just about acceptable results out of the machine in a labour-intensive way that doesn't really scale over time or problem size, but they muddle through with determination. I don't know about a 10x productivity differences though. I'm not really arguing in favour of that hypothesis. Rather, if anything, I've seen step differences in potential; that is, there are developers that can do things other developers can never do, not in 10x the time, not in 1000x the time.