3 ms·

[...] They studied professional programmers with an average of 7 years' experience and found that the ratio of initial coding time between the best and worst pr

[...] They studied professional programmers with an average of 7 years' experience and found that the ratio of initial coding time between the best and worst programmers was about 20 to 1 I thought the perceived idea about the 10x developer was that he/she is 10x better than the average programmer, not the worst. So even this scientific study (from the 60s) says that there aren't 10x developers according to this definition: the ratio goes down to sqrt(20), using the geometric mean, if a good-enough sample was chosen. Program size and program execution speed would put that ratio to sqrt(5) and sqrt(10), respectively.

andmarios 9y ago
Also I think in the 60s there were analysts, the people who solved the problem and programmers, the people who converted the solution to code. Anyway, in the 60s the profession was in its infancy, so I'm not sure how much studies from this era matter today.
- andrewstuart 9y ago
  I vaguely recall some of those big statements about programmer productivity came from observations of the relative performance of programmers developing IBM mainframe operating systems in assembly language. I bet you'd surely find 10 to 1 or 100 to 1 differences in programming productivity on a project like that following waterfall methodology in the 1960's.
- nnfy 9y ago
  I also wonder how the productivity gap has been affected by modern development technology, or even the languages themselves.
  - acdha 9y ago
    It definitely affects your ability to compare studies over a long period of time – e.g. there was a huge shift in what programming work looked like during the progression from submitting code as batch jobs with a turnaround time measured in hours, to having a terminal on a shared system (where compilation was still a slow process), to individual workstations. That changes some of the skills significantly – i.e. reasoning about code on a whiteboard isn't useless but it's not a critical skill the way it was before everyone had an editor running code checks continuously, interactive debugger, etc. I think Fred Brooks was right to argue[1] that we're not going to see dramatic further improvements because we're at the point of diminishing returns on overhead and most of the remaining work is actually inherent to whatever problem is being solved. The closest we've probably come to a change on that scale was the rise of cloud technology shrinking the feedback cycle for testing but I don't think that's been an order-of-magnitude change for most people. 1. https://en.wikipedia.org/wiki/No_Silver_Bullet https://en.wikipedia.org/wiki/No_Silver_Bullet
EliRivers 9y ago
I thought the perceived idea about the 10x developer was that he/she is 10x better than the average programmer, not the worst That could well be what people perceive. It's not what the studies said. While the article has some good references, I might as well quote Peopleware again just for the fun of it. I churn this out every so often: A 2nd edition of Peopleware summarises it; the 10x programmer is not a myth, but it's comparing the best to the worst; NOT best to median. It's also not about programming specifically; it's simply a common distribution in many metrics of performance. The rule of thumb Peopleware states is that you can rely on the best outperforming the worst by a factor of 10, and you can rely on the best outperforming the median by a factor of 2.5. This of course indicates that a median developer, middle of the pack, is a 4x developer. Obviously, this is a statistical rule, and if you've got a tiny sample size or some kind of singular outlier or other such; well, we're all adults and we understand how statistics and distributions work. Peopleware uses Boehn (1981), Sackman (1968), Augustine (1979) and Lawrence (1981) as its sources. [ "Peopleware", DeMarco and Lister, 1987, p45 ]
- michaelmrose 9y ago
  10x the worst literally means nothing at all if you find a sufficiently bad programmer you can have an infinity times programmer and the 10x developer as it is normally expressed is 10x average. Further author even punts at the end of article by claiming that its not needed to measure difficult to measure things as we can instead reason our way to the right answer which is practically nonsensical. Measuring is step one in actually reasoning about things. In short author may be a good developer but he is certainly not a 10x author.
  - deleted 9y ago
    [deleted]
- kybernetikos 9y ago
  > you can rely on the best outperforming the median by a factor of 2.5 This is roughly what I estimate the productivity improvement a good developer gets when they're working on a task that fascinates them and that they choose for themselves, and without too many dependencies on productivity killing things (like builds and flaky external libraries and services). The most productive people I knew tended to the people who didn't do as they were told. They were amazingly productive at producing the things that fascinated them, much less productive at the drudge that the organisation demanded of them. Sometimes the organisation was able to recognise that they were doing useful stuff (despite the fact that they weren't always pulling their weight on the things they were supposed to be doing) and they became rockstars, sometimes it wasn't and they became pariahs.
mvindahl 9y ago
I thought the same thing. Maybe because it's the only definition that makes sense. If you measure everyone against the department Wally (the lazy guy from Dilbert), you'll probably find plenty of 10x programmers. Not that it would mean anything. Also, sample size would matter. Assume that there really is a Superwally out there who has worked in programming for seven years for some company without accomplishing anything at all. And I imagine that there is. It would matter a lot if he is part of the sample. In fact, the only sample that would make sense would be all programmers in the world, and now, thanks to Superwally, we all look pretty good. And of course, there is probably some Hyperwally as well who can smugly boast seven years of negative productivity. Deleting random stuff, breaking functionality, leaving messy code for others to untangle. Adding Hyperwally to the equation and the relative productivity of everyone else would approach singularity levels.
ShannonAlther 9y ago
The trouble with comparing the best to the worst is that there's really no floor for how bad a programmer can be. If an average programmer takes ten minutes to do a task, and a great one takes four, how long does an incompetent one take? Twenty minutes, thirty, an hour or two, never...?
- Chyzwar 9y ago
  Incompetent programmer will finish task in the same time as 10x but at the same time it will break production for 5h, losing 100h for company and another 5h for someone to fix it.

andmarios 9y ago

Also I think in the 60s there were analysts, the people who solved the problem and programmers, the people who converted the solution to code. Anyway, in the 60s the profession was in its infancy, so I'm not sure how much studies from this era matter today.

andrewstuart 9y ago
I vaguely recall some of those big statements about programmer productivity came from observations of the relative performance of programmers developing IBM mainframe operating systems in assembly language. I bet you'd surely find 10 to 1 or 100 to 1 differences in programming productivity on a project like that following waterfall methodology in the 1960's.
nnfy 9y ago
I also wonder how the productivity gap has been affected by modern development technology, or even the languages themselves.
- acdha 9y ago
  It definitely affects your ability to compare studies over a long period of time – e.g. there was a huge shift in what programming work looked like during the progression from submitting code as batch jobs with a turnaround time measured in hours, to having a terminal on a shared system (where compilation was still a slow process), to individual workstations. That changes some of the skills significantly – i.e. reasoning about code on a whiteboard isn't useless but it's not a critical skill the way it was before everyone had an editor running code checks continuously, interactive debugger, etc. I think Fred Brooks was right to argue[1] that we're not going to see dramatic further improvements because we're at the point of diminishing returns on overhead and most of the remaining work is actually inherent to whatever problem is being solved. The closest we've probably come to a change on that scale was the rise of cloud technology shrinking the feedback cycle for testing but I don't think that's been an order-of-magnitude change for most people. 1. https://en.wikipedia.org/wiki/No_Silver_Bullet https://en.wikipedia.org/wiki/No_Silver_Bullet

EliRivers 9y ago

I thought the perceived idea about the 10x developer was that he/she is 10x better than the average programmer, not the worst That could well be what people perceive. It's not what the studies said. While the article has some good references, I might as well quote Peopleware again just for the fun of it. I churn this out every so often: A 2nd edition of Peopleware summarises it; the 10x programmer is not a myth, but it's comparing the best to the worst; NOT best to median. It's also not about programming specifically; it's simply a common distribution in many metrics of performance. The rule of thumb Peopleware states is that you can rely on the best outperforming the worst by a factor of 10, and you can rely on the best outperforming the median by a factor of 2.5. This of course indicates that a median developer, middle of the pack, is a 4x developer. Obviously, this is a statistical rule, and if you've got a tiny sample size or some kind of singular outlier or other such; well, we're all adults and we understand how statistics and distributions work. Peopleware uses Boehn (1981), Sackman (1968), Augustine (1979) and Lawrence (1981) as its sources. [ "Peopleware", DeMarco and Lister, 1987, p45 ]

michaelmrose 9y ago
10x the worst literally means nothing at all if you find a sufficiently bad programmer you can have an infinity times programmer and the 10x developer as it is normally expressed is 10x average. Further author even punts at the end of article by claiming that its not needed to measure difficult to measure things as we can instead reason our way to the right answer which is practically nonsensical. Measuring is step one in actually reasoning about things. In short author may be a good developer but he is certainly not a 10x author.
- deleted 9y ago
  [deleted]
kybernetikos 9y ago
> you can rely on the best outperforming the median by a factor of 2.5 This is roughly what I estimate the productivity improvement a good developer gets when they're working on a task that fascinates them and that they choose for themselves, and without too many dependencies on productivity killing things (like builds and flaky external libraries and services). The most productive people I knew tended to the people who didn't do as they were told. They were amazingly productive at producing the things that fascinated them, much less productive at the drudge that the organisation demanded of them. Sometimes the organisation was able to recognise that they were doing useful stuff (despite the fact that they weren't always pulling their weight on the things they were supposed to be doing) and they became rockstars, sometimes it wasn't and they became pariahs.

mvindahl 9y ago

I thought the same thing. Maybe because it's the only definition that makes sense. If you measure everyone against the department Wally (the lazy guy from Dilbert), you'll probably find plenty of 10x programmers. Not that it would mean anything. Also, sample size would matter. Assume that there really is a Superwally out there who has worked in programming for seven years for some company without accomplishing anything at all. And I imagine that there is. It would matter a lot if he is part of the sample. In fact, the only sample that would make sense would be all programmers in the world, and now, thanks to Superwally, we all look pretty good. And of course, there is probably some Hyperwally as well who can smugly boast seven years of negative productivity. Deleting random stuff, breaking functionality, leaving messy code for others to untangle. Adding Hyperwally to the equation and the relative productivity of everyone else would approach singularity levels.

ShannonAlther 9y ago

The trouble with comparing the best to the worst is that there's really no floor for how bad a programmer can be. If an average programmer takes ten minutes to do a task, and a great one takes four, how long does an incompetent one take? Twenty minutes, thirty, an hour or two, never...?

Chyzwar 9y ago
Incompetent programmer will finish task in the same time as 10x but at the same time it will break production for 5h, losing 100h for company and another 5h for someone to fix it.