4 ms·

Problem is, company A doesn't need an engineer to solve those interview questions but real problems.

okdood64 2y ago
This is the answer. Let's not pretend 95% of companies are asking asinine interview questions (though I understand the reasons why) that LLMs can easily solve.
- nottorp 2y ago
  Let's go one step further: LLMs can't solve anything, but most interview questions are covered so much online that they'll parrot a passable answer.
  - Jensson 2y ago
    Yeah, what you want is a General Intelligence that has learned the topic you care about. Google search returning an algorithm when you ask it doesn't mean that you shouldn't test candidates on that algorithm, since you still need a General Intelligence that knows it and not just the algorithm itself.
placardloop 2y ago
“Real problems” aren’t something that can be effectively discussed in the time span of an interview, so companies concoct unreal problems that are meant to be good indicators.
- 0_____0 2y ago
  Really? How short are your interviews, and how big are these Real Problems such that you can't get a sense of how your candidate would start to tackle them?
  - placardloop 2y ago
    The “real problems” most companies want people to help solve involve the evolution of products that last for years, involve repeated design discussions, in depth research, and applying retrospective learning. I don’t need someone that can just glue a Rails API together. If I did, I can literally just download that from the internet for free. If my problems could be solved in the time span of an interview, why would I waste my time doing that interview instead of just solving it?
    - 0_____0 2y ago
      I don't see the issue here. Nobody expects candidates to build actual product during the interview. Having a (targeted, scope and time-limited) design discussion or giving your candidate some made-up context around an engineering cycle and then doing a retrospective with them are practical and useful ways to interview a candidate. I'm also not sure what the alternative is? Just not hiring?
      - placardloop 2y ago
        > Having a (targeted, scope and time-limited) design discussion or giving your candidate some made-up context around an engineering cycle and then doing a retrospective with them You just described a contrived, “unreal” problem. > I'm also not sure what the alternative is? Just not hiring? The alternative is to come up with questions that are representative of skills related to “real problems”, as you just did, and use those instead. Unfortunately candidates consistently complain that such questions aren’t realistic.
        0_____0 2y ago
        OK I think we're on the same page, and just had a semantic issue.
        nottorp 2y ago
        I've had some success with just describing what we're doing and seeing what the candidates ask. Mind, I work in very small companies and never had to give input for filling 10 positions at once... just one at a time.
- 542354234235 2y ago
  On that, these unreal questions/problems are decent proxies for general knowledge for humans, but not for AI. Humans don't have encyclopedic knowledge, so questions on a topic can do a decent job of indicating a person has the broader depth of knowledge in that topic and could bring that to bear in a job. An AI can answer all the questions but can't bring that to bear in a job. WE saw this last year with all the "AI can now pass the bar exam" articles, but that doesn't lead to them being able to do anything approaching practicing law, because AI failure modes are not the same as humans and can't be tested the same way.

This is the answer. Let's not pretend 95% of companies are asking asinine interview questions (though I understand the reasons why) that LLMs can easily solve.

nottorp 2y ago
Let's go one step further: LLMs can't solve anything, but most interview questions are covered so much online that they'll parrot a passable answer.
- Jensson 2y ago
  Yeah, what you want is a General Intelligence that has learned the topic you care about. Google search returning an algorithm when you ask it doesn't mean that you shouldn't test candidates on that algorithm, since you still need a General Intelligence that knows it and not just the algorithm itself.

placardloop 2y ago

“Real problems” aren’t something that can be effectively discussed in the time span of an interview, so companies concoct unreal problems that are meant to be good indicators.

0_____0 2y ago
Really? How short are your interviews, and how big are these Real Problems such that you can't get a sense of how your candidate would start to tackle them?
- placardloop 2y ago
  The “real problems” most companies want people to help solve involve the evolution of products that last for years, involve repeated design discussions, in depth research, and applying retrospective learning. I don’t need someone that can just glue a Rails API together. If I did, I can literally just download that from the internet for free. If my problems could be solved in the time span of an interview, why would I waste my time doing that interview instead of just solving it?
  - 0_____0 2y ago
    I don't see the issue here. Nobody expects candidates to build actual product during the interview. Having a (targeted, scope and time-limited) design discussion or giving your candidate some made-up context around an engineering cycle and then doing a retrospective with them are practical and useful ways to interview a candidate. I'm also not sure what the alternative is? Just not hiring?
    - placardloop 2y ago
      > Having a (targeted, scope and time-limited) design discussion or giving your candidate some made-up context around an engineering cycle and then doing a retrospective with them You just described a contrived, “unreal” problem. > I'm also not sure what the alternative is? Just not hiring? The alternative is to come up with questions that are representative of skills related to “real problems”, as you just did, and use those instead. Unfortunately candidates consistently complain that such questions aren’t realistic.
      - 0_____0 2y ago
        OK I think we're on the same page, and just had a semantic issue.
      - nottorp 2y ago
        I've had some success with just describing what we're doing and seeing what the candidates ask. Mind, I work in very small companies and never had to give input for filling 10 positions at once... just one at a time.
542354234235 2y ago
On that, these unreal questions/problems are decent proxies for general knowledge for humans, but not for AI. Humans don't have encyclopedic knowledge, so questions on a topic can do a decent job of indicating a person has the broader depth of knowledge in that topic and could bring that to bear in a job. An AI can answer all the questions but can't bring that to bear in a job. WE saw this last year with all the "AI can now pass the bar exam" articles, but that doesn't lead to them being able to do anything approaching practicing law, because AI failure modes are not the same as humans and can't be tested the same way.