3 ms·

> For example, an individual contributor software engineer s competency matrix may include a row for coding/feature output velocity. In a Level 1 to Level 5 sys

by dberg 3y ago

> For example, an individual contributor software engineer s competency matrix may include a row for coding/feature output velocity. In a Level 1 to Level 5 system, the matrix would specify that for a Level 1 engineer, expectations on code velocity are X pull requests per week I mean. No.

ekms 3y ago
What are you quoting? I don't see that sentence anywhere in parent's linked twitter thread
GordonS 3y ago
It really is a ridiculous metric. So ridiculous that you'd think it would never be used in reality... and yet, I have in the past actually had the CTO of a large engineering firm insist upon it.
synergy20 3y ago
“If you can't measure it, you can't manage it”,do you have better ideas that also work in daily operations and practical? is there a way at all to measure software engineer or no?
- bcantrill 3y ago
  "If you can't measure it, you can't manage it" is absolutely terrible leadership advice. Or to phrase it another way, if you could magically transport yourself to work on any team in history, which would be and why? And how did those teams run? How did the leadership function? How did the team think of its own performance? In this regard, I feel history is a much better guide than middle management claptrap admonishing us to all kneel before the false god of Measure Everything.
  - synergy20 3y ago
    I don't buy it fully either, but, I could not figure out what's a better alternative, not in theory, but in practice. Software projects are notoriously hard to manage indeed, but it has to be managed somehow, Agile approach is a nice try, we had CMMI as well. Even a neurosurgeon can be measured, why can't software developers, why are software engineers so special? Maybe AI programmer is the way out.
    - bcantrill 3y ago
      So, interesting example: how much do you know about surgery? The best surgeons care about things that are not, in fact measured -- things that amount to the quality of the craft that very much correlate to patient outcomes, but are also hard to measure. Indeed, if you really get a surgeon going, they will complain about the things that are measured -- namely, the number of surgeries that they perform. I know of more than one surgeon who has left their craft because this quantification was essentially forcing them to perform unnecessary procedures! In general, quantification of human endeavor is fraught with peril -- even in those domains where it would seem to be entirely non-controversial like professional athletics.
    - josephg 3y ago
      The alternative to measuring everything seems obvious. Don’t measure everything. Why not measure everything? Because measurement has a cost. And some things cost more to measure than you gain from the measurement. I was on a team once with a product owner who insisted on absolutely everything being in jira with story point estimations. Well, that’s fine and good but one day I sat down with the designer and walked through the product. We made a list of about 30 things that needed fixing - almost all of them trivial. “This is the wrong shade of blue”. “There is too much spacing here”, etc. Well, I made a list in notepad and got going but I got in trouble for not putting it in jira. Only, it would have taken longer to write up these issues than it would have taken to fix them. To say nothing of wasting everyone’s time doing story point estimation. In this case, measuring our work would have cost significantly more than doing the work! We argued back and forth and eventually he admitted he wanted things in jira to appease upper management, who I suppose wanted to know how busy we were by looking at a number on a spreadsheet. (He ended up writing up all my points in jira himself, guessing costs and ticking them all off as “done”. What a waste of time.) And no, software isn’t special. Teaching isn’t measured (except maybe with a student survey at the end of the year). Science is measured in citations per lifetime, not micropapers/hr. And you don’t quantitatively measure your spouse, your friends or your enjoyment of mum’s cooking. Measurement is a lovely tool. But don’t make a religion out of it. I heard a story from a leadership summit recently. A young CEO got up and said “But there’s no way I’d make hiring and firing decisions by gut instinct!”. A bunch of the older people there disagreed immediately, and said that’s exactly what you should do. Train your instincts then trust them to do their job.
- ericjohannsen 3y ago
  Measuring software engineers productivity has been a meme and the butt of jokes for decades. However, I think there are things worth measuring that do help manage success and are not intrusive. For example, giant pull requests or pull requests that go into production with no/minimal review are likely to create problems. We use LinearB to point this sort of thing out automatically. Code with high cognitive complexity is likely to generate bugs (and be hard to maintain). There are many other similar issues that can be automatically detected. We use SonarCloud for this. Asking an engineer "How do you feel about work overall", "your personal well being", "career growth", "work relationships", and "impact & productivity" on a scale of horrible to outstanding once a week prior to 1:1 meetings allows each engineer to tell you their story in numbers. If you have built a culture where people trust that you care about them, you will get honest answers. This helps discuss and correct things that bother your team members, and in aggregate shows whether you have a problem with leadership and/or culture that needs to be corrected. Those are the numbers I gather, and it takes a lot of the gut feel and guesswork out of management.
- danmaz74 3y ago
  It's incredibly difficult to measure the work of a software engineer with metrics which are useful AND objective. You can have many metrics which are useful, but not objective. And you can have many metrics which are objective, but not useful (and often detrimental, like "number of merge requests" per week).
somsak2 3y ago
The thing is, for a junior engineer, this is not that bad of an idea. I don't think that it should be part of a formal performance process, but if you're green and aren't putting out enough PRs, it suggests you're either struggling to contribute, or putting out PRs that are far too large and need to be broken up to be easier to review & safer to deploy. Think of it as a threshold. A high number doesn't suggest you're great. But a low number can often be a symptom of a greater problem.
- pcthrowaway 3y ago
  I assume this suggestion only applies to big orgs where juniors can get a steady stream of small, well-scoped items to tackle. At a lot of smaller orgs I assume it's harder to do this? Where I've worked, juniors often receive exploratory items that are lower-priority, and off the critical path, but they're expected to have reduced output, as well as asking for guidance as needed from more senior members. Perhaps this is a failure of planning (which would have been my own failing as a lead at times), as well as management
  - jiveturkey 3y ago
    > steady stream of small, well-scoped items to tackle Yes. The part of the quote parent left out is > or the ability to close Y story points per sprint With that in mind, it makes sense and the metric is dependent on the entire org functioning well. So to the extent it does so, this is an objective, very quantifiable, metric that all junior engineers can be compared with. That said, this is a guide for startups. So I do believe this metric is garbaj. Junior engineers will be unfairly held to task for org dysfunction.
withinboredom 3y ago
I was just thinking the other day about one of the issues I have working in this industry. You are expected to solve some hard problems, within a couple of days, usually. If you fail to do this satisfactorily, people will wonder what you're doing; if you're smart enough. And if you can't solve those problems fast enough, the "right" way, you are let go for not "performing." In reality, you don't run into these kinds of problems often. Usually, the problems we're solving are relatively straightforward. However, if you accidentally pick up too many of these problems in a short span of time ... you can be screwed.
rmbyrro 3y ago
I mean, if you want the # of PRs to skyrocket to an impossible to manage level, that's great advice!

by dberg 3y ago

ekms 3y ago

What are you quoting? I don't see that sentence anywhere in parent's linked twitter thread

GordonS 3y ago

It really is a ridiculous metric. So ridiculous that you'd think it would never be used in reality... and yet, I have in the past actually had the CTO of a large engineering firm insist upon it.

synergy20 3y ago

“If you can't measure it, you can't manage it”,do you have better ideas that also work in daily operations and practical? is there a way at all to measure software engineer or no?

bcantrill 3y ago
"If you can't measure it, you can't manage it" is absolutely terrible leadership advice. Or to phrase it another way, if you could magically transport yourself to work on any team in history, which would be and why? And how did those teams run? How did the leadership function? How did the team think of its own performance? In this regard, I feel history is a much better guide than middle management claptrap admonishing us to all kneel before the false god of Measure Everything.
- synergy20 3y ago
  I don't buy it fully either, but, I could not figure out what's a better alternative, not in theory, but in practice. Software projects are notoriously hard to manage indeed, but it has to be managed somehow, Agile approach is a nice try, we had CMMI as well. Even a neurosurgeon can be measured, why can't software developers, why are software engineers so special? Maybe AI programmer is the way out.
  - bcantrill 3y ago
    So, interesting example: how much do you know about surgery? The best surgeons care about things that are not, in fact measured -- things that amount to the quality of the craft that very much correlate to patient outcomes, but are also hard to measure. Indeed, if you really get a surgeon going, they will complain about the things that are measured -- namely, the number of surgeries that they perform. I know of more than one surgeon who has left their craft because this quantification was essentially forcing them to perform unnecessary procedures! In general, quantification of human endeavor is fraught with peril -- even in those domains where it would seem to be entirely non-controversial like professional athletics.
  - josephg 3y ago
    The alternative to measuring everything seems obvious. Don’t measure everything. Why not measure everything? Because measurement has a cost. And some things cost more to measure than you gain from the measurement. I was on a team once with a product owner who insisted on absolutely everything being in jira with story point estimations. Well, that’s fine and good but one day I sat down with the designer and walked through the product. We made a list of about 30 things that needed fixing - almost all of them trivial. “This is the wrong shade of blue”. “There is too much spacing here”, etc. Well, I made a list in notepad and got going but I got in trouble for not putting it in jira. Only, it would have taken longer to write up these issues than it would have taken to fix them. To say nothing of wasting everyone’s time doing story point estimation. In this case, measuring our work would have cost significantly more than doing the work! We argued back and forth and eventually he admitted he wanted things in jira to appease upper management, who I suppose wanted to know how busy we were by looking at a number on a spreadsheet. (He ended up writing up all my points in jira himself, guessing costs and ticking them all off as “done”. What a waste of time.) And no, software isn’t special. Teaching isn’t measured (except maybe with a student survey at the end of the year). Science is measured in citations per lifetime, not micropapers/hr. And you don’t quantitatively measure your spouse, your friends or your enjoyment of mum’s cooking. Measurement is a lovely tool. But don’t make a religion out of it. I heard a story from a leadership summit recently. A young CEO got up and said “But there’s no way I’d make hiring and firing decisions by gut instinct!”. A bunch of the older people there disagreed immediately, and said that’s exactly what you should do. Train your instincts then trust them to do their job.
ericjohannsen 3y ago
Measuring software engineers productivity has been a meme and the butt of jokes for decades. However, I think there are things worth measuring that do help manage success and are not intrusive. For example, giant pull requests or pull requests that go into production with no/minimal review are likely to create problems. We use LinearB to point this sort of thing out automatically. Code with high cognitive complexity is likely to generate bugs (and be hard to maintain). There are many other similar issues that can be automatically detected. We use SonarCloud for this. Asking an engineer "How do you feel about work overall", "your personal well being", "career growth", "work relationships", and "impact & productivity" on a scale of horrible to outstanding once a week prior to 1:1 meetings allows each engineer to tell you their story in numbers. If you have built a culture where people trust that you care about them, you will get honest answers. This helps discuss and correct things that bother your team members, and in aggregate shows whether you have a problem with leadership and/or culture that needs to be corrected. Those are the numbers I gather, and it takes a lot of the gut feel and guesswork out of management.
danmaz74 3y ago
It's incredibly difficult to measure the work of a software engineer with metrics which are useful AND objective. You can have many metrics which are useful, but not objective. And you can have many metrics which are objective, but not useful (and often detrimental, like "number of merge requests" per week).

somsak2 3y ago

The thing is, for a junior engineer, this is not that bad of an idea. I don't think that it should be part of a formal performance process, but if you're green and aren't putting out enough PRs, it suggests you're either struggling to contribute, or putting out PRs that are far too large and need to be broken up to be easier to review & safer to deploy. Think of it as a threshold. A high number doesn't suggest you're great. But a low number can often be a symptom of a greater problem.

pcthrowaway 3y ago
I assume this suggestion only applies to big orgs where juniors can get a steady stream of small, well-scoped items to tackle. At a lot of smaller orgs I assume it's harder to do this? Where I've worked, juniors often receive exploratory items that are lower-priority, and off the critical path, but they're expected to have reduced output, as well as asking for guidance as needed from more senior members. Perhaps this is a failure of planning (which would have been my own failing as a lead at times), as well as management
- jiveturkey 3y ago
  > steady stream of small, well-scoped items to tackle Yes. The part of the quote parent left out is > or the ability to close Y story points per sprint With that in mind, it makes sense and the metric is dependent on the entire org functioning well. So to the extent it does so, this is an objective, very quantifiable, metric that all junior engineers can be compared with. That said, this is a guide for startups. So I do believe this metric is garbaj. Junior engineers will be unfairly held to task for org dysfunction.

withinboredom 3y ago

I was just thinking the other day about one of the issues I have working in this industry. You are expected to solve some hard problems, within a couple of days, usually. If you fail to do this satisfactorily, people will wonder what you're doing; if you're smart enough. And if you can't solve those problems fast enough, the "right" way, you are let go for not "performing." In reality, you don't run into these kinds of problems often. Usually, the problems we're solving are relatively straightforward. However, if you accidentally pick up too many of these problems in a short span of time ... you can be screwed.

rmbyrro 3y ago

I mean, if you want the # of PRs to skyrocket to an impossible to manage level, that's great advice!