3 ms·

Depending on the field, it could be as much as 40x. We had a firmware engineer, who in 3 months, single handedly wrote an entire FHSS MAC layer for an RF appli

Depending on the field, it could be as much as 40x. We had a firmware engineer, who in 3 months, single handedly wrote an entire FHSS MAC layer for an RF application, with a high level of quality and hundreds of functions, while simultaneously laying down the architecture for it's growth over the next seven years. He was extraordinarily capable, intelligent, and had worked on the identical project in the previous 5-6 years, and so was able to anticipate all the blind alleys to avoid. I'm reasonably confident that a low-performing engineer with little or no experience in this space would have taken seven-ten years to accomplish the same feat at the same level of quality and completeness. (Of course, it could be argued that our engineer had taken seven years to accomplish this task, it's just that they had worked six of them at a prior company) The challenge in these types of tests, of course is (A) who defines/measures quality, (B) How do you know which functions were essential? Programmer performance is not a function of how fast you can complete coding challenges, but how well you can architect, document, communicate, collaborate, and all the other peripheral functions not associated with cranking out algorithms.

ExpiredLink 12y ago
> Depending on the field, it could be as much as 40x The best dentist I've met easily was 40x better than the worst. But what does this tell us about dentists?
awjr 12y ago
I'm not saying it happened but if I had a developer who had worked for 5-6 years on an identical project, I would be very careful that your codebase does not include any of that project. I do feel you are taking somebody who had an immense understanding of the domain, who had delivered an identical project over the last 5-6 years (and probably had that codebase to refer to) and trying to compare it to an engineer with no domain experience and no identical project to refer to.
- ghshephard 12y ago
  No codebase to refer to, everything was recreated from scratch; but the concepts (basing a hopping sequence on MAC address, agressive discovery of neighbors on power up, sending a single data/ack packet, instead of a complete POLL-ACK-DATA-ACK for small packets, tracking information for all adjacent nodes in a nodeq, keeping track of both RSSI and info success, etc...) were concepts that he had learned/developed previously (thankfully many of them in early 1990s so patent issues were probably not an issue either) And you are correct - I am taking somebody who had an immense understanding of the domain, and testing them on a (close to) identical project and I am trying to compare it to an engineer with no domain experience - that was kind of my point. He was still an extraordinary engineer - but, moved into a new domain, maybe a 2x, possibly 3x engineer. (i.e. he could write code in a single month that would take a low level engineer three months to write to similar quality.) But, if you are looking to build a new product - isn't the idea to hire extraordinary engineers with 7-10 years of experience in that field to get the huge boost in productivity? And, if you really could get 20x-40x performance boost in quality and performance - what would it be worth to hire those employees away from their current companies to lead your team, if only for a few years? And, if these people really do exist - doesn't it make sense that they should be working for only one-two years at companies, where their skills are precisely matched to requirements, and get the huge boost in compensation for that time? The only thing that is missing, is some mechanisms, system, that could match up these "world class engineers with extraordinarily specific skill" to companies that need those specific skills.
onion2k 12y ago
A brilliant engineer: ...had worked on the identical project in the previous 5-6 years, and so was able to anticipate all the blind alleys to avoid. A low-performing engineer: ...with little or no experience in this space would have taken seven-ten years to accomplish the same feat If the brilliant engineer spent 5 to 6 years working on identical apps before doing a good job for you, and the low-performing engineer would take 7 to 10 years to do a good job, doesn't that imply a good engineer is only 1.5* better, but any engineer needs 5 years experience before they're capable?
- ghshephard 12y ago
  That's a pretty good summary. In reality, though, what happens is that engineers with no experience first get an education. For example, in RF - getting your EE degree will quickly boost you up from a 1x to 5x engineer. Then, they go work for companies that do have experienced engineers, and they learn hundreds of skillsets, and develop deep domain knowledge in the ones that they are interested in, so my "engineer with little or no experience" in this space probably doesn't exist - a typical EE who came to work out of school for an RF company, probably learned a bunch of different concepts (PHY, MAC, Routing, DFM, RF Immunity, Antenna Design) from the 40x engineers already there with 10-20 years of relevant experience - and so, within a five years, would already be a 10x engineer in those field versus one that had no experience/education in them, and would be 20x in the projects that they had direct experience developing. I would suggest that 40x performance increase comes from a combination of innate skill (you are already extraordinarily gifted and capable), Direct domain experience, a leadership role in which you were required to develop/invent/create and then document/x-train/and educate others on the technologies that you have mastery on, and, perhaps most importantly - passion, energy, and devotion to the field. The caveat to all of this is that with projects of enough duration, everyone gets a chance to pick up experience, and knowledge, and so will increase their capability in a particular domain. The other caveat, is the vast amount of engineering is not simply repeating the same task over and over again - unless you are making cell phones for Samsung.
svisser 12y ago
How does that qualify as 40x? It's not reasonable to compare an engineer who has worked for 5-6 years on an identical project to an engineer that hasn't or has less work experience. For this particular task he may have been 40x more effective than anyone else but only because he walked the path before.
- geocar 12y ago
  Do you pay programmers with 5-6 years experience in a domain 40x junior programmers?
  - alkonaut 12y ago
    Productivity as an inherent property of a developer has to be for "unseen" tasks. If someone started working with me today I'd be more than 10x as productive as them tomorrow and maybe all of next week. That is relevant for a company, but not really interesting.
    - geocar 12y ago
      That only means instantaneous productivity isn't useful. Fortunately most employers know this: The employer pays for a mean-productivity-over-time. If your productivity falls below that value you will get fired, and if it exceeds some other bound you may be bonus'd or promoted. However if the productivity exceeds this line by 40x, then the question remains: Do you pay your programmers 40x?
- ghshephard 12y ago
  That was the point I was trying to make - there is more than one way to get a boost in performance. Innate Skill is one of them, but another is to do it through experience.
duncanawoods 12y ago
>> it could be argued that our engineer had taken >> seven years to accomplish this task, it's just >> that they had worked six of them at a prior company Thats a great line - I think it reveals a lot about the impact relevant experience has on productivity. Its truly awesome when it works out but change the problem or the technology and the 40x engineer becomes a 0.2 engineer in the short/medium and sadly sometimes the long term too. I see two main problems that prevents experience always compounding positively - technology churn whereby a master carpenter has to switch to metal every two years and the unspeakable variety and flexibility of the software medium. I see software engineering as a collection of 1000 niches rather than a homogenous whole. RT embedded vs. data center vs. client-side vs. 3d vs. mass consumer and hitting market first vs. 10 year continuous improvement vs. safety critical etc. there is only so much overlap between them. In one niche, killer productivity might be unlocked by leveraging libraries, in another by reducing dependencies. In one niche highly defensive programming may be the winner, in another rapid iteration might be the secret sauce. In one you need to master the theory, in another you need to abandon it. We too easily learn lessons in one niche and don't realise they are not generally applicable. This is also why comparing two engineers is pretty tough. You can test them on the same problem but this will only ever be a narrow view of their comparative strengths.

ExpiredLink 12y ago

> Depending on the field, it could be as much as 40x The best dentist I've met easily was 40x better than the worst. But what does this tell us about dentists?

awjr 12y ago

I'm not saying it happened but if I had a developer who had worked for 5-6 years on an identical project, I would be very careful that your codebase does not include any of that project. I do feel you are taking somebody who had an immense understanding of the domain, who had delivered an identical project over the last 5-6 years (and probably had that codebase to refer to) and trying to compare it to an engineer with no domain experience and no identical project to refer to.

ghshephard 12y ago
No codebase to refer to, everything was recreated from scratch; but the concepts (basing a hopping sequence on MAC address, agressive discovery of neighbors on power up, sending a single data/ack packet, instead of a complete POLL-ACK-DATA-ACK for small packets, tracking information for all adjacent nodes in a nodeq, keeping track of both RSSI and info success, etc...) were concepts that he had learned/developed previously (thankfully many of them in early 1990s so patent issues were probably not an issue either) And you are correct - I am taking somebody who had an immense understanding of the domain, and testing them on a (close to) identical project and I am trying to compare it to an engineer with no domain experience - that was kind of my point. He was still an extraordinary engineer - but, moved into a new domain, maybe a 2x, possibly 3x engineer. (i.e. he could write code in a single month that would take a low level engineer three months to write to similar quality.) But, if you are looking to build a new product - isn't the idea to hire extraordinary engineers with 7-10 years of experience in that field to get the huge boost in productivity? And, if you really could get 20x-40x performance boost in quality and performance - what would it be worth to hire those employees away from their current companies to lead your team, if only for a few years? And, if these people really do exist - doesn't it make sense that they should be working for only one-two years at companies, where their skills are precisely matched to requirements, and get the huge boost in compensation for that time? The only thing that is missing, is some mechanisms, system, that could match up these "world class engineers with extraordinarily specific skill" to companies that need those specific skills.

onion2k 12y ago

A brilliant engineer: ...had worked on the identical project in the previous 5-6 years, and so was able to anticipate all the blind alleys to avoid. A low-performing engineer: ...with little or no experience in this space would have taken seven-ten years to accomplish the same feat If the brilliant engineer spent 5 to 6 years working on identical apps before doing a good job for you, and the low-performing engineer would take 7 to 10 years to do a good job, doesn't that imply a good engineer is only 1.5* better, but any engineer needs 5 years experience before they're capable?

ghshephard 12y ago
That's a pretty good summary. In reality, though, what happens is that engineers with no experience first get an education. For example, in RF - getting your EE degree will quickly boost you up from a 1x to 5x engineer. Then, they go work for companies that do have experienced engineers, and they learn hundreds of skillsets, and develop deep domain knowledge in the ones that they are interested in, so my "engineer with little or no experience" in this space probably doesn't exist - a typical EE who came to work out of school for an RF company, probably learned a bunch of different concepts (PHY, MAC, Routing, DFM, RF Immunity, Antenna Design) from the 40x engineers already there with 10-20 years of relevant experience - and so, within a five years, would already be a 10x engineer in those field versus one that had no experience/education in them, and would be 20x in the projects that they had direct experience developing. I would suggest that 40x performance increase comes from a combination of innate skill (you are already extraordinarily gifted and capable), Direct domain experience, a leadership role in which you were required to develop/invent/create and then document/x-train/and educate others on the technologies that you have mastery on, and, perhaps most importantly - passion, energy, and devotion to the field. The caveat to all of this is that with projects of enough duration, everyone gets a chance to pick up experience, and knowledge, and so will increase their capability in a particular domain. The other caveat, is the vast amount of engineering is not simply repeating the same task over and over again - unless you are making cell phones for Samsung.

svisser 12y ago

How does that qualify as 40x? It's not reasonable to compare an engineer who has worked for 5-6 years on an identical project to an engineer that hasn't or has less work experience. For this particular task he may have been 40x more effective than anyone else but only because he walked the path before.

geocar 12y ago
Do you pay programmers with 5-6 years experience in a domain 40x junior programmers?
- alkonaut 12y ago
  Productivity as an inherent property of a developer has to be for "unseen" tasks. If someone started working with me today I'd be more than 10x as productive as them tomorrow and maybe all of next week. That is relevant for a company, but not really interesting.
  - geocar 12y ago
    That only means instantaneous productivity isn't useful. Fortunately most employers know this: The employer pays for a mean-productivity-over-time. If your productivity falls below that value you will get fired, and if it exceeds some other bound you may be bonus'd or promoted. However if the productivity exceeds this line by 40x, then the question remains: Do you pay your programmers 40x?
ghshephard 12y ago
That was the point I was trying to make - there is more than one way to get a boost in performance. Innate Skill is one of them, but another is to do it through experience.

duncanawoods 12y ago

>> it could be argued that our engineer had taken >> seven years to accomplish this task, it's just >> that they had worked six of them at a prior company Thats a great line - I think it reveals a lot about the impact relevant experience has on productivity. Its truly awesome when it works out but change the problem or the technology and the 40x engineer becomes a 0.2 engineer in the short/medium and sadly sometimes the long term too. I see two main problems that prevents experience always compounding positively - technology churn whereby a master carpenter has to switch to metal every two years and the unspeakable variety and flexibility of the software medium. I see software engineering as a collection of 1000 niches rather than a homogenous whole. RT embedded vs. data center vs. client-side vs. 3d vs. mass consumer and hitting market first vs. 10 year continuous improvement vs. safety critical etc. there is only so much overlap between them. In one niche, killer productivity might be unlocked by leveraging libraries, in another by reducing dependencies. In one niche highly defensive programming may be the winner, in another rapid iteration might be the secret sauce. In one you need to master the theory, in another you need to abandon it. We too easily learn lessons in one niche and don't realise they are not generally applicable. This is also why comparing two engineers is pretty tough. You can test them on the same problem but this will only ever be a narrow view of their comparative strengths.