4 ms·

> VP of Engineering, due to his lack of ability to directly evaluate work, he would base performance reviews on a weighted average of your teammates' assessmen

> VP of Engineering, due to his lack of ability to directly evaluate work, he would base performance reviews on a weighted average of your teammates' assessment of your work I know this is an old hat discussion, and this argument was lost ages ago, (and I don't know if you are specifically referring to this) but that situations like this happen at all really bothers me. Assuming the "VP of Engineering" means "VP of Software Engineering" (or the "CTO" of a software company, or equivalent). If he/she can't truly evaluate the work, he/she is not qualified to be in that role, full stop, no exceptions. No one expects them to read/write code all day (often, they might spend no time at all reading/writing code anymore). But they have to be able to do so, or they have no place in that role. In the same way that the Chief Nursing Officer at a Hospital has to have been a qualified licensed RN, and be able to be a qualified licensed RN, even if they don't actually work as a nurse on the floor anymore.

sulam 8y ago
Let’s test this assertion. Does it hold in the extreme? For instance, why then does the person that VP of Engineering reports to not need to be a qualified engineer? If this is the CEO then it’s clearly impossible for them to be qualified at all of the functions they are ultimately responsible for. If we accept that the CEO is essentially incompetent at their job the way it’s defined here, why are we unwilling to accept it one node down? Is the function of that node to somehow insert all the competence the node above them lacks? We will shortly see that this can’t possibly be the case. Another example — suppose this person has held management positions for the last 15 years. In a large organization this is likely to be true. This person last coded before some of our most popular languages even existed, and in the Node case they have missed umpteen million iterations on what a good webdev stack looks like. They also have never done modern mobile development and probably couldn’t gradient descent themselves out of a paper bag. They wouldn’t get through a phone screen, so how are you so sure their technical skills matter? Another issue: assuming they somehow find time to keep up to date on some aspect of software engineering, how likely is it that the one area where they are a qualified developer is the only thing they manage? Or is it more likely that they manage distributed systems engineers next to machine learning specialists next to security engineers next to site reliability, system engineers in test, and Bluetooth LE on Android specialists (that’s the reality where I work)? How can they possibly be an expert on all of this? Given that they lack true expertise, why would we value faux expertise? My simplifying assumption is that skill at a job they do not do is actually close to meaningless and in fact what really matters is their ability to be effective at the thing they spend 10 hours a day on — communication, setting directions, intently listening, resolving conflicts, and integrating the demands of the business, customers, and the people that have entrusted them with a portion of their career.
- deleted 8y ago
  [deleted]
- twic 8y ago
  > For instance, why then does the person that VP of Engineering reports to not need to be a qualified engineer? Because that person isn't evaluating their direct reports on the basis of their engineering ability, or even their engineering leadership ability, but their management ability. Unlike the crummy boss in the parent comment.
  - sulam 8y ago
    Crummy bosses are crummy. Can't really draw a lot of conclusions from bad.
joshuamorton 8y ago
There are two possible answers here. One is that sure, a VPE/CTO who cannot evaluate technical contributions in a vacuum is unfit. I agree. However, a VPE who can't evaluate any specific technical contributions in their organization may not be failing, and in fact probably isn't. To evaluate someone's technical contributions, there are at least two things you want to look at (there may be more, but there are at least two): resultant impact, and inherent complexity[1] of the problem. A VPE should almost always be able to evaluate the first. They're the one setting goals. They can evaluate a person's contributions on whether or not those helped meet or met those goals. They won't however be able to know the inherent problem space complexity. That requires a level of in-the-weeds familiarity with the space that you don't get as a non-individual contributor. It's practically impossible to accurately judge how difficult a problem is to solve without either having attempted to solve it yourself, or seeing how many people perform solving it, and most of the time, you don't have many people attempt to solve the same business problem. You also don't want to solely trust the people working on the project, they have reasons to inflate the complexity, both conscious and unconscious. So you use a lot of things to gain partial signal: - what they say: "this was hard" - what other people who had views of their actual contributions say: "their problem space looked difficult and I couldn't find any better solutions" - what people with historic knowledge of the component/space say: "Their solution managed to clean up years of technical debt that I accrued when designing this system" - Smaller impact metrics: developer velocity improved when making additional changes to the new system, etc. But its often not possible for a VPE to directly assess those things, they don't have the context to do that for every employee. [1]: Note that this is complexity of the problem space and not complexity of the solution. Simple solutions to hard problems are good things, complex solutions to complex problems are acceptable, and complicated solutions to simple problems should be a negative. And yes, you need to account for problem space complexity, otherwise you encourage infighting since performance = impact, then higher impact per unit difficulty will result in better performance per unit time.

sulam 8y ago

Let’s test this assertion. Does it hold in the extreme? For instance, why then does the person that VP of Engineering reports to not need to be a qualified engineer? If this is the CEO then it’s clearly impossible for them to be qualified at all of the functions they are ultimately responsible for. If we accept that the CEO is essentially incompetent at their job the way it’s defined here, why are we unwilling to accept it one node down? Is the function of that node to somehow insert all the competence the node above them lacks? We will shortly see that this can’t possibly be the case. Another example — suppose this person has held management positions for the last 15 years. In a large organization this is likely to be true. This person last coded before some of our most popular languages even existed, and in the Node case they have missed umpteen million iterations on what a good webdev stack looks like. They also have never done modern mobile development and probably couldn’t gradient descent themselves out of a paper bag. They wouldn’t get through a phone screen, so how are you so sure their technical skills matter? Another issue: assuming they somehow find time to keep up to date on some aspect of software engineering, how likely is it that the one area where they are a qualified developer is the only thing they manage? Or is it more likely that they manage distributed systems engineers next to machine learning specialists next to security engineers next to site reliability, system engineers in test, and Bluetooth LE on Android specialists (that’s the reality where I work)? How can they possibly be an expert on all of this? Given that they lack true expertise, why would we value faux expertise? My simplifying assumption is that skill at a job they do not do is actually close to meaningless and in fact what really matters is their ability to be effective at the thing they spend 10 hours a day on — communication, setting directions, intently listening, resolving conflicts, and integrating the demands of the business, customers, and the people that have entrusted them with a portion of their career.

deleted 8y ago
[deleted]
twic 8y ago
> For instance, why then does the person that VP of Engineering reports to not need to be a qualified engineer? Because that person isn't evaluating their direct reports on the basis of their engineering ability, or even their engineering leadership ability, but their management ability. Unlike the crummy boss in the parent comment.
- sulam 8y ago
  Crummy bosses are crummy. Can't really draw a lot of conclusions from bad.

joshuamorton 8y ago

There are two possible answers here. One is that sure, a VPE/CTO who cannot evaluate technical contributions in a vacuum is unfit. I agree. However, a VPE who can't evaluate any specific technical contributions in their organization may not be failing, and in fact probably isn't. To evaluate someone's technical contributions, there are at least two things you want to look at (there may be more, but there are at least two): resultant impact, and inherent complexity[1] of the problem. A VPE should almost always be able to evaluate the first. They're the one setting goals. They can evaluate a person's contributions on whether or not those helped meet or met those goals. They won't however be able to know the inherent problem space complexity. That requires a level of in-the-weeds familiarity with the space that you don't get as a non-individual contributor. It's practically impossible to accurately judge how difficult a problem is to solve without either having attempted to solve it yourself, or seeing how many people perform solving it, and most of the time, you don't have many people attempt to solve the same business problem. You also don't want to solely trust the people working on the project, they have reasons to inflate the complexity, both conscious and unconscious. So you use a lot of things to gain partial signal: - what they say: "this was hard" - what other people who had views of their actual contributions say: "their problem space looked difficult and I couldn't find any better solutions" - what people with historic knowledge of the component/space say: "Their solution managed to clean up years of technical debt that I accrued when designing this system" - Smaller impact metrics: developer velocity improved when making additional changes to the new system, etc. But its often not possible for a VPE to directly assess those things, they don't have the context to do that for every employee. [1]: Note that this is complexity of the problem space and not complexity of the solution. Simple solutions to hard problems are good things, complex solutions to complex problems are acceptable, and complicated solutions to simple problems should be a negative. And yes, you need to account for problem space complexity, otherwise you encourage infighting since performance = impact, then higher impact per unit difficulty will result in better performance per unit time.