7 ms·

I don't know what to make of this. Looks like a potential headache. As an engineer in an interview, we never get told what we're being scored on in advance (nam

I don't know what to make of this. Looks like a potential headache. As an engineer in an interview, we never get told what we're being scored on in advance (naming, speed, coming up with wonky edge-cases, scalability to 1million times workload, versatility, mentioning a bunch of relevant buzzwords (bloom filters!), confidence?). This is one reason why most interviewing is a subjective clusterfuck (not just software, I suspect). I'd bet really good money that numerous companies would reject Linus, Carmack, etc. Hence the giving of graphs and other media that are meant to project an idea of objectivity while reducing an engineer to a few quantities turns my stomach.

spyspy 9y ago
> giving of graphs and other media that are meant to project an idea of objectivity Agile has the same issues.
- kyberias 9y ago
  Would you like to elaborate? What "agile graphs" fail and why?
  - lloeki 9y ago
    Burndown charts, planning poker, difficulty weights, priority tiers, team velocity over time, milestone progress bars. I’ve yet to see someone tell me with a straight face that they hold any value like what we want them to be.
    - ashark 9y ago
      Their value is that they make the client comfortable with estimates that are explicitly fuzzy BS, which is vital since there's no way it makes sense for the client to pay for the time it'd take to give them a non-fuzzy, non-BS estimate. It also deflects them from attempting to get non-fuzzy, non-BS estimates out of you in a fraction of the time it'd otherwise take—the reputation and authority of the Agile brand is what gets them to do this, not any actual utility it has. That agile estimation is fuzzy BS is a certainty. But it's a nice comforting blanket to help clients accept "we're just gonna start working on this, if you decide you don't want us to work on it anymore tell us to stop," which is important because that's the most cost-effective way for the client to spend their money, in many cases. It's nowhere near as accurate as anything resembling an actual engineering estimation process, but ain't no-one got the time/money for that. It's probably a little better than the seat-of-pants "just give me a ballpark right now (but I'm totally gonna hold you to it, gee isn't it weird how we're always fighting fires and in crunch mode?)" that it's replacing as far as accuracy, but more importantly it makes that crap not happen (assuming it's functioning correctly).
      - lloeki 9y ago
        Precisely, IOW they're just for show and are basically GIGO when you're completely honest, but they're a psychological device required both externally (as leverage one way or another towards clients) and internally (it's motivating to see something go forward, even when you know it's completely bollocks†). Just to say that this definitely subscribes to the basic idea that they "are meant to project an idea of objectivity". BTW, you forgot the part about giving improving metrics to satisfy management and maybe get a raise (vs 100% not getting one), or a nice story line about how you managed to improve team velocity YOY by 50% through magic on your resume, which in the same way is "reducing an engineer to a few quantities" and equally "turns my stomach" as for alexandercrohde. Most of the things I've done to introduce improvements are immeasurable because they're changes that are mindset shifting or technological leverage therefore there is no metric associated because of the pervasiveness of their effects. † if you're doing, say, 10 push-ups, instead of "1, 2, 3" try counting each number twice, like "1, 1, 2, 2, 3, 3". Magic, you've done 20. You know it's a mind trick, but it'll still work. †† Going from email+phone+hallway communication, silo'd knowledge and SPOF Bus Factor to a serious issue tracking, knowledge sharing, and automation on GitLab. Going from manual machine building to fully automated system provisioning and one liner deployment. Any associated metric would be either completely immaterial (e.g confidence and well being) or produce an off-the chart ratio (just computed one just for fun, that's a ~1800% improvement in human cycles — not even counting failure modes — it's so beyond ridiculous it doesn't even mean anything).
      - megy 9y ago
        That is a bit ridiculous. It shows everyone who the project is tracking, and how inline with reality estimates are. I am not sure why you guys find it so hard to comprehend why that is useful?
  - WaxProlix 9y ago
    Burndown charts and the like, one imagines.
d4mi3n 9y ago
100% agree with this. At any given company I've worked at, the recruiting/evaluation strategy has been different--largely because they _are_ looking for different things in candidates. Some places value communication skills over technical skills. Others will value experience over credentials. I think it's unrealistic to expect that there is "one true way" to perform an evaluation (technical or otherwise). It all needs to be in the context of the team/department/organization you're hiring for.
rafiki6 9y ago
Totally agree. Classic fallacy of test confidence. The question is, why are we confident in the test to begin with...
ng12 9y ago
> This is one reason why most interviewing is a subjective clusterfuck (not just software, I suspect). This is why I get a laugh out of people complaining about whiteboard-style interviews. Far and away tech still has one of the most intensely meritocratic hiring processes. Not perfect, sure, but better than almost every other industry. The majority of high-skilled jobs rely heavily on pedigree, education, social standing, references, sociability, etc. I'd much rather reject Linus for flubbing a BFS traversal than hire him for attending my Alma Matter and knowing a lot about golf.
- dragonwriter 9y ago
  > Almost every other industry relies heavily on pedigree, education, social standing, references, sociability, etc. Almost every industry, including tech, relies heavily on those things as pre-interview filters. Tech is not a meritocratic exception here.
  - ng12 9y ago
    How many traders at Goldman Sachs don't have degrees from Penn/Harvard/Yale/etc? I bet not very many. Yet some of the best developers I know are self-taught or went to no-name schools. My SO works in media and I can assure you the hiring process there is completely, thoroughly, perniciously subjective. Experiencing her job search really gave me appreciation for the fact that taking a test is 80% of my interview process.
    - vonmoltke 9y ago
      > How many traders at Goldman Sachs don't have degrees from Penn/Harvard/Yale/etc? I just did a quick check in LinkedIn, choosing four people at GS with "Trader" in their title. The four schools I got: UNC Chapel Hill; Duke; Williams College; Amherst College. I find that distribution hard to correlate with your statement.
      - southphillyman 9y ago
        Aren't these all expensive private "elite non Ivy" schools? Maybe OP was being too restrictive with the 3 listed schools but I think the general point being implied is correct.
blktiger 9y ago
To be fair, I'd reject Linus. I wouldn't want to work with someone like him even if he is a well known programmer with leading a well known open source project very successfully. I don't think he'd make a good cultural fit at most of the places I've worked.
- dominotw 9y ago
  I think you missed the point.
  - deleted 9y ago
    [deleted]
koala_man 9y ago
> we never get told what we're being scored on in advance I make a point of not telling you. I want you to demonstrate that you're able to work out the requirements when you're given a task. Some people will take a task, go off for a few weeks, develop ill-fitting solutions, and then say "well, you didn't say you wanted unit tests" or "you didn't tell me it had to support files that can't fit in memory". Others will try to find out what's needed, with simple questions like "Roughly how big are these files?" and "Does this actually need to run in line with the front page? What if we load it asynchronously?" I'm looking for the latter.
- jnordwick 9y ago
  I generally give all the important parts of the question up front. Having to fish for requirement isn't the point.
  - heartbreak 9y ago
    I like to leave at least something small ambiguous to see if clarification is requested. My clients rarely give unambiguous requirements, so I feel it’s pertinent to the job.
  - nnd 9y ago
    Depends on the role, if it requires high levels of autonomy and working with highly ambiguous product definitions (e.g. most freelancing jobs) - then it's a part of your role to constantly fish for requirements or even suggest your own requirements.
megy 9y ago
> This is one reason why most interviewing is a subjective clusterfuck (not just software, I suspect). This is the thing, all OTHER interviewing doesn't do this, they do it by talking to the candidates. Damn, I am going to get out of development.
- pseudonom- 9y ago
  FYI, most evidence suggests unstructured interviews (talking to the candidates) have very low predictive power about job performance and that work sample tests (take home problems and on-site coding) fare much better in that regard.
  - Redsquare 9y ago
    Where is the evidence?
    - pseudonom- 9y ago
      Table 1 here: http://www.flume.com.br/pdf/Scmidt_The_validity_and_utility.pdf http://www.flume.com.br/pdf/Scmidt_The_validity_and_utility....

spyspy 9y ago

> giving of graphs and other media that are meant to project an idea of objectivity Agile has the same issues.

kyberias 9y ago
Would you like to elaborate? What "agile graphs" fail and why?
- lloeki 9y ago
  Burndown charts, planning poker, difficulty weights, priority tiers, team velocity over time, milestone progress bars. I’ve yet to see someone tell me with a straight face that they hold any value like what we want them to be.
  - ashark 9y ago
    Their value is that they make the client comfortable with estimates that are explicitly fuzzy BS, which is vital since there's no way it makes sense for the client to pay for the time it'd take to give them a non-fuzzy, non-BS estimate. It also deflects them from attempting to get non-fuzzy, non-BS estimates out of you in a fraction of the time it'd otherwise take—the reputation and authority of the Agile brand is what gets them to do this, not any actual utility it has. That agile estimation is fuzzy BS is a certainty. But it's a nice comforting blanket to help clients accept "we're just gonna start working on this, if you decide you don't want us to work on it anymore tell us to stop," which is important because that's the most cost-effective way for the client to spend their money, in many cases. It's nowhere near as accurate as anything resembling an actual engineering estimation process, but ain't no-one got the time/money for that. It's probably a little better than the seat-of-pants "just give me a ballpark right now (but I'm totally gonna hold you to it, gee isn't it weird how we're always fighting fires and in crunch mode?)" that it's replacing as far as accuracy, but more importantly it makes that crap not happen (assuming it's functioning correctly).
    - lloeki 9y ago
      Precisely, IOW they're just for show and are basically GIGO when you're completely honest, but they're a psychological device required both externally (as leverage one way or another towards clients) and internally (it's motivating to see something go forward, even when you know it's completely bollocks†). Just to say that this definitely subscribes to the basic idea that they "are meant to project an idea of objectivity". BTW, you forgot the part about giving improving metrics to satisfy management and maybe get a raise (vs 100% not getting one), or a nice story line about how you managed to improve team velocity YOY by 50% through magic on your resume, which in the same way is "reducing an engineer to a few quantities" and equally "turns my stomach" as for alexandercrohde. Most of the things I've done to introduce improvements are immeasurable because they're changes that are mindset shifting or technological leverage therefore there is no metric associated because of the pervasiveness of their effects. † if you're doing, say, 10 push-ups, instead of "1, 2, 3" try counting each number twice, like "1, 1, 2, 2, 3, 3". Magic, you've done 20. You know it's a mind trick, but it'll still work. †† Going from email+phone+hallway communication, silo'd knowledge and SPOF Bus Factor to a serious issue tracking, knowledge sharing, and automation on GitLab. Going from manual machine building to fully automated system provisioning and one liner deployment. Any associated metric would be either completely immaterial (e.g confidence and well being) or produce an off-the chart ratio (just computed one just for fun, that's a ~1800% improvement in human cycles — not even counting failure modes — it's so beyond ridiculous it doesn't even mean anything).
    - megy 9y ago
      That is a bit ridiculous. It shows everyone who the project is tracking, and how inline with reality estimates are. I am not sure why you guys find it so hard to comprehend why that is useful?
- WaxProlix 9y ago
  Burndown charts and the like, one imagines.

d4mi3n 9y ago

100% agree with this. At any given company I've worked at, the recruiting/evaluation strategy has been different--largely because they _are_ looking for different things in candidates. Some places value communication skills over technical skills. Others will value experience over credentials. I think it's unrealistic to expect that there is "one true way" to perform an evaluation (technical or otherwise). It all needs to be in the context of the team/department/organization you're hiring for.

rafiki6 9y ago

Totally agree. Classic fallacy of test confidence. The question is, why are we confident in the test to begin with...

ng12 9y ago

> This is one reason why most interviewing is a subjective clusterfuck (not just software, I suspect). This is why I get a laugh out of people complaining about whiteboard-style interviews. Far and away tech still has one of the most intensely meritocratic hiring processes. Not perfect, sure, but better than almost every other industry. The majority of high-skilled jobs rely heavily on pedigree, education, social standing, references, sociability, etc. I'd much rather reject Linus for flubbing a BFS traversal than hire him for attending my Alma Matter and knowing a lot about golf.

dragonwriter 9y ago
> Almost every other industry relies heavily on pedigree, education, social standing, references, sociability, etc. Almost every industry, including tech, relies heavily on those things as pre-interview filters. Tech is not a meritocratic exception here.
- ng12 9y ago
  How many traders at Goldman Sachs don't have degrees from Penn/Harvard/Yale/etc? I bet not very many. Yet some of the best developers I know are self-taught or went to no-name schools. My SO works in media and I can assure you the hiring process there is completely, thoroughly, perniciously subjective. Experiencing her job search really gave me appreciation for the fact that taking a test is 80% of my interview process.
  - vonmoltke 9y ago
    > How many traders at Goldman Sachs don't have degrees from Penn/Harvard/Yale/etc? I just did a quick check in LinkedIn, choosing four people at GS with "Trader" in their title. The four schools I got: UNC Chapel Hill; Duke; Williams College; Amherst College. I find that distribution hard to correlate with your statement.
    - southphillyman 9y ago
      Aren't these all expensive private "elite non Ivy" schools? Maybe OP was being too restrictive with the 3 listed schools but I think the general point being implied is correct.

blktiger 9y ago

To be fair, I'd reject Linus. I wouldn't want to work with someone like him even if he is a well known programmer with leading a well known open source project very successfully. I don't think he'd make a good cultural fit at most of the places I've worked.

dominotw 9y ago
I think you missed the point.
- deleted 9y ago
  [deleted]

koala_man 9y ago

> we never get told what we're being scored on in advance I make a point of not telling you. I want you to demonstrate that you're able to work out the requirements when you're given a task. Some people will take a task, go off for a few weeks, develop ill-fitting solutions, and then say "well, you didn't say you wanted unit tests" or "you didn't tell me it had to support files that can't fit in memory". Others will try to find out what's needed, with simple questions like "Roughly how big are these files?" and "Does this actually need to run in line with the front page? What if we load it asynchronously?" I'm looking for the latter.

jnordwick 9y ago
I generally give all the important parts of the question up front. Having to fish for requirement isn't the point.
- heartbreak 9y ago
  I like to leave at least something small ambiguous to see if clarification is requested. My clients rarely give unambiguous requirements, so I feel it’s pertinent to the job.
- nnd 9y ago
  Depends on the role, if it requires high levels of autonomy and working with highly ambiguous product definitions (e.g. most freelancing jobs) - then it's a part of your role to constantly fish for requirements or even suggest your own requirements.

megy 9y ago

> This is one reason why most interviewing is a subjective clusterfuck (not just software, I suspect). This is the thing, all OTHER interviewing doesn't do this, they do it by talking to the candidates. Damn, I am going to get out of development.

pseudonom- 9y ago
FYI, most evidence suggests unstructured interviews (talking to the candidates) have very low predictive power about job performance and that work sample tests (take home problems and on-site coding) fare much better in that regard.
- Redsquare 9y ago
  Where is the evidence?
  - pseudonom- 9y ago
    Table 1 here: http://www.flume.com.br/pdf/Scmidt_The_validity_and_utility.pdf http://www.flume.com.br/pdf/Scmidt_The_validity_and_utility....