8 ms·

The best interview process I've ever been a part of involved pair programming with the person for a couple hours, after doing the tech screening having a phone

The best interview process I've ever been a part of involved pair programming with the person for a couple hours, after doing the tech screening having a phone call with a member of the team. You never failed to know within a few minutes whether the person could do the job, and be a good coworker. This process worked so well, it created the best team, most productive team I've worked on in 20+ years in the industry, despite that company's other dysfunctions. The problem with it is the same curse that has rotted so much of software culture—the need for a scalable process with high throughput. "We need to run through hundreds of candidates per position, not a half dozen, are you crazy? It doesn't matter if the net result is better, it's the metrics along the way that matter!"

chiph 2y ago
Was this an in-person pair programming session or remote via shared desktop? Because the latter is garbage. You can't see the other person or read their body language. It also gives off "proctored test" vibes. Or worse than vibes - where you're expected to clear your room, swing your camera around, and never look away.
ukd1 2y ago
Pairing on something close to whatever real work they'd be doing, but familiar to the applicant is my favorite way to evaluate someone (e.g. choose a side project, pre agree adding a feature). I don't care if someone uses modern tools (like AI assists), google, etc - e.g. "open book" - as that's the how they want to work. Evaluating their style of thinking / solving problems, comms, and output is the win.
agustechbro 2y ago
Beautifully articulated truth
m11a 2y ago
I dislike pair programming interviews - as they currently exist - because they usually feel like a time-crunched exam. You don't realistically have the freedom to actually think as you would in actual pair programming. i.e. if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview, but it's pretty realistic of real life work and entirely a non-problem. It's probably even good to test for at interview: how does a person work when they aren't working with an oracle that already knows the answer (ie: the interviewer)? Pair programming with the person for a couple hours, maybe even on an actual feature, would probably work, assuming the candidate is compensated for their time. I can imagine it'd especially work for teams working on open source projects (Sentry, Zed, etc). Might not be as workable for companies whose work is entirely closed source. Indeed, the other problem is what you mention: it doesn't scale to high throughput.
- karaterobot 2y ago
  That's definitely up to the interviewer, in which a lot of discretion and trust has been placed. I think a lot of it also comes down to the culture of the company—whether they're cutthroat or supportive. As you get better people into the company, hopefully this improves over time. I know that when we did it, it was never about nailing it on the first try, it was literally about proving you knew how to program and were not an asshole. So, not the equivalent of reversing a binary tree on a whiteboard. The kinds of problems we worked on in the interviews weren't leetcode type problems, they were real tickets from our current project. Sometimes it was just doing stuff like making a new component or closing a bug, but those were the things we really did, so it felt like a better test.
- barbazoo 2y ago
  > i.e. if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview That’s an assumption. Perhaps following a dead end for a while, realizing it, pivoting, etc is a valuable, positive, signal?
  - m11a 2y ago
    I agree. But what I mean is: that's not how it's perceived in the current interview structure, which lasts maybe 45 minutes or so. Ultimately, going down a dead end means you'd now have 30 minutes to find the right solution and code it up. So the oracle (the interviewer) would probably help you realise sooner that it's a bad idea, so you don't waste your time. That's assuming they know the problem and solution well; if they don't, you'll just lose them and burn through your time. In a 2 hour pair programming session on an 'unsolved' problem (like an open issue / minor bug / minor feature in a public repo), yes, it will likely not matter if you tried a bad idea, and would both be more realistic and a positive signal.
    - dingnuts 2y ago
      this is an interviewer problem, unless the candidate is totally silent. A candidate that can't ask questions isn't going to succeed, anyway. If the candidate will communicate with me, I will offer them a LOT of guidance. It is still very, very easy to tell who knows what they are doing and who does not. You give them a basic but domain-specific task, you give them whatever extra context they need to do it, and you watch them hammer out the code. It should be a task that is sufficiently familiar to the person applying to the role that they do not need to do -much- looking at docs, and as the interviewer you should be prepared to help them quickly find the docs you already know they will need -- you designed the task after all -- so that they don't waste time with that. What's important is that they ask for docs when they need them, and that they can understand them, quickly, and use them. It will be obvious if they are using AI because of how long things will take. It will be obvious if they don't know when to reach for documentation, and it will be obvious if they cannot understand the documentation. Then, they should write a test for their solution. This weeds out 95% of candidates. Talk to the other 5% and you'll find someone who can both actually write code and also discuss design.
hylaride 2y ago
Similarly, in general the best interviews I've ever been part of (either giving or being) turn into discussions where people's experience, opinions, and stories get aired (going both ways). You eventually get a good sense of each other and things get more relaxed when you both realize that you know what you're talking about (this is harder for Jr roles, though). Being peppered with questions very rarely gives any insight.
- lanstin 2y ago
  For junior roles, you want to interview for intelligence and shall we say an interest in learning rather than specific skills. Even for senior roles, that's what I want to interview for, although it is true at times a business case can be made for someone that is good at some specific complex skill and doesn't need to listen to other people to do ok work.
joshdavham 2y ago
I've (unfortunately) been interviewing the last two months and the main pattern that I've noticed is that a) big companies have terrible interview processes while b) small companies and startups are great at interviewing. Big companies need to hire tons of people and interview even more so they need some sort of scalable process for it. An early stage startup can just ask you about your past projects and pair program with you for an hour.
- solarmist 2y ago
  I hear this all the time, but I have yet to experience it. It may be because the small companies that I interview with are all startups, but I have yet to be able to get a call back from any other kind of small company. And the startups I do interview with have a full FAANG interview loops. There seems to be a weird selection bias that if you're FAANG or FAANG adjacent these small companies aren't interested.
  - codr7 2y ago
    Yeah, been there, done that; wannabe FAANGs are the worst.
    - drdaeman 2y ago
      At least those are typically honest about what they try to be and give a very clear signal, right at the interview time. It's much worse when the interview gives you different vibes (and expectations) than the actual day-to-day work.
      - solarmist 2y ago
        I've had both I've had ones that only tell you what the next interview looks like. Unless you specifically prior it out of the recruiter's hands.
      - codr7 2y ago
        The only way to test actual work is to do actual work, should be obvious but here we are.
Attummm 2y ago
I've been a proponent of pair programming since the early days of Agile, when it was still seen as part of extreme programming. Unfortunately, it’s not often employed in workplace settings. With that said, would your perception of the interview remain positive if the outcome had been negative? A common challenge across all interviews is a mismatch in personal dynamics, which can significantly impact the experience for both participants. Consider a scenario where a senior developer, who prefers simplicity, is paired with a mid-level developer who is still captivated by complexity.
- lanstin 2y ago
  Or a "just start typing" person is with a "mull it over first" person. By the time I am typing code, I want to have 90% of it already completely worked out (at least till I type a "c.Lock()" and suddenly realize I hadn't considered thus and so synchronization issue.
tristramb 2y ago
"We need to run through hundreds of candidates per position, not a half dozen" But you don't! You only need to find the first person who is good enough to do the job. You do not need to find the best person.
- dowager_dan99 2y ago
  You need to run through hundreds of candidates to find someone marginally qualified. I am not exaggerating.
  - samr71 2y ago
    Do we have different definitions of "marginally qualified"? Idk, I feel I'm a decent engineer - I can certainly do whatever leetcode medium they throw at me, as much as this counts of anything - and can actually code, but I still get maybe 1 callback per 50 applications. Does "marginally qualified" mean "Ivy League Competitive Programmer PhD" or something?
    - marcosdumay 2y ago
      The GP is still passing through hundreds of people, dozens of them capable, until he reaches somebody that convinces him of their competence. You were passed down because you weren't convincing enough. Or maybe he is getting resumes from a channel that has been victim of machine-gun filling, and there indeed thousands of incompetent people posting resumes into every channel and just half a dozen real applicants. TBF, I have no idea how to fix either one of those problems. Hiring is just completely broken.
      - samr71 2y ago
        I have only a resume to convince them. I have job experience at major companies, with examples of what I've built when there, personal projects I've made, a github link, a great GPA from a good school. And I know similar Junior-mid people in the same boat. We can all do Fizzbuzz, we've all built things, and somehow we're not getting interviews, but people that can't do Fizzbuzz are. Do thousands of incompetents also machine-gun apply to, say, mechanical engineering, accounting, marketing, HR, or finance gigs? Is it just tech? Something isn't adding up.
superb_dev 2y ago
This is how my team hires and it’s incredible. I think what makes it work is that our code pair is pretty low stakes. I was told that I didn’t have to finish the problem and I was free to use whatever tools or language I needed. They just wanted to see how I work and collaborate.
slg 2y ago
>The best interview process I've ever been a part of involved pair programming with the person for a couple hours... You never failed to know within a few minutes whether the person could do the job There is something funny about the "best interview process" taking "a couple hours" despite giving you the answer "within a few minutes". Seems like even the best process is a little broken.
- alwa 2y ago
  Lightly ironic indeed! Though I’m not sure “broken” is exactly the word I’d choose. I can only speak for myself, but I imagine myself as a candidate approaching a “couple of hours” project or relationship differently than I would a “few minutes” speed round. For that matter I can think of people I know professionally who I only know through their behavior “on stage” in structured and stylized meetings of a half hour or an hour—and I don’t feel like I have any sense at all of how they would be as day-to-day coworkers. If we sat down to work together, you’d probably have a sense in the first few minutes of whether or not we were going to work out—but that would be contingent on us going into it with the attitude that we were working together as colleagues toward a goal.
- karaterobot 2y ago
  That's mainly because there were multiple pairing sessions, and even if you knew the person was going to pass, there are still a couple more people who need to meet them, and a schedule to make sure they're available to do that. Plus due diligence, etc. Nor am I saying it was a perfect system, just the best I've seen in terms of results.
andyjohnson0 2y ago
Some of us find the prospect of pairing with an unknown person in an unknown environment, and against the clock, to be very stressful.
- andyjohnson0 2y ago
  Anecdote: I've been interviewing recently and got through to the last round (of five...) with an interesting company. I knew the interview involved pairing, but I didn't expect: two people sitting behind me while I sat on a wobbly stool at a standing desk, trying to use a sticky wired mouse and a non-UK keyboard, and a very bright monitor (I normally use dark mode). They had a screen recorder running, and a radio was on in the next room. I totally bombed it. I suspect just the two people behind me would have been enough though.
- hirvi74 2y ago
  I would find trying to solve such problems with known people in known environments to be somewhat stressful too.
- stcroixx 2y ago
  I've had a great career and very positive feedback from every employer. I would do absolutely terrible in this kind of interview.
lifeisstillgood 2y ago
This is the exact time to use phrase “A people hire other A people, B people hire C people” Additionally it’s rarely the hiring that makes a great team - it’s the long term commitment and investment in training.
dowager_dan99 2y ago
Very few people doing this sort of interview (they tend to be our best, most empathetic developers) are likely to cut a multi-hour planned process short after a few minutes. It will eat at least an hour of their (very expensive & valuable) time. Also how am I supposed to filter the 100's of AI-completed assessments? Who gets this opportunity?
- samr71 2y ago
  Use another AI, of course! Idk if I'm even being sarcastic here.
- karaterobot 2y ago
  We didn't do assessments (if by that you mean take home assignments). This was partly a solution to that, since nobody thought they were a good idea. If you mean the phone screen, I think that would be a problem, yep, but it wasn't an issue back in 2016. Having them pair would weed out cheaters, but we would have to figure out a way to weed them out during the screening, I agree. We also did not require the employers doing the interview to be our most senior team members. They probably did it more often than most people, but often because they volunteered to do it. Anyone on the team would be part of the loop, which helped with scheduling. And, remember, we were working on actual tickets, so in a lot of cases it actually helped having the candidate there as a pairing partner. For a little extra detail, the way we actually did it was to have 2-3 pairing sessions of up to 2 hours apiece. At the end of the day, all the team members who paired with the candidate had to give them the thumbs up.
jitix 2y ago
I used to love getting to know the interviewer and doing things like that but IMO the market has shifted fundamentally on both ends for this to be effective anymore for most SaaS roles. This is anecdotal for US/Canada tech market over the past 10 years so YMMV. Developers Side: Since developers don't have job security anymore (at least for those who work on common languages like Go, Python, Java and Typescript) they are better off learning and keeping in touch with leetcode and system design questions, looking for new opportunities and interviewing in "batch mode" when looking for a job. The idea is to clear as many interviews as possible using the same concepts, get in and make money asap before you get laid off. No incentive for collaboration or for fulfilling but esoteric stuff like Haskell and Scala. Career security > Job security. Companies Side: On the other end software companies have less trust in developers staying long term so they want to make the interview process as quick and risk free as possible. In essence they are betting that by perusing 100s of resumes and hiring someone who seemingly knows CS concepts they can get some value out of them before they leave. Standardized tests/vetting > team fit. TLDR; The art is gone from this job, its become akin to management consulting or investment banking. Quality and UX seems to be regressing across the board as a result.
- no_wizard 2y ago
  >its become akin to management consulting or investment banking Not sure how those are similar.
  - jitix 2y ago
    I meant the “grind”, short term profit mentality of SWE market has become similar to professionals in those fields, not that any of these fields are similar.
hassleblad23 2y ago
Pair programming on what problem though? I dont think many companies would want an outsider to work on their codebase.
- hassleblad23 2y ago
  And devs wouldn't want to spend substantial time working on stuff unrelated to work just for the interview process.
whiplash451 2y ago
It's a super interesting approach, and if you put a strong filter before it (e.g. intense non-BS Q&A), the whole thing could be high-throughput.
shihab 2y ago
The biggest victims of these non-scalable process is people without a good network. As an intl PhD student, I am that person. So now I have this weird dynamic: I get interview calls only from FAANG companies, the ones with the manpower to do your so called "cursed" scalable interviews. But the smaller companies or startups, ones who are a far better fit for my specialized kills, never call me. You need to either "know someone" or be from a big school, or there is zero chance.
namuol 2y ago
Does your company operate the same way? I.e. is most, or at least a large chunk of engineering done as pair-programming?
qudat 2y ago
> You never failed to know within a few minutes whether the person could do the job Then why spend a couple hours?
0x457 2y ago
> person for a couple hours, >You never failed to know within a few minutes whether the person could do the job Did a misunderstood something or your best interview process is to multiple hours from someone when you've decided within minutes?
reverendsteveii 2y ago
This. Interviewing for a sr dev position with a web app, backend stack is the bog standard java, spring, SQL abstracted away via JPA. We did a first screen, then the tech interview was two of their senior devs shoulder surfing me as I built a simple API. We chatted, I built, they asked questions, I defended my decisions (sometimes successfully, sometimes gracefully conceding defeat), they left knowing that I was who my resume said I was and the reminder that popped up in the middle of the interview to feed my sourdough starter showed them that I'm a culture fit. I think you're onto something with that last paragraph but I want to try being a bit more generous with why things are the way they are. The question seems to be "When there are hundreds of applicants how do we give everyone a fair shake without hiring an entire team of devs who do nothing but interview?" From that perspective the intentionality is different and even sensible but the end product is likely to be the same. Even when someone is chasing a metric it's because someone else wants what's best and has decided that metric is a sensible way to make that happen. At the end of the day they really do want to hire the best candidate out of a pool whose size is extremely variable and that's challenging.
christkv 2y ago
Thats what we did, pair program on some real production code and tickets. This way the person could get a feel about what they potentially were walking into and you get a good idea of how they think and approach problems.
zensavona 2y ago
I work at a company which has 11 engineers and competes with companies with 100s. The hiring process was a screening call with the CTO to not waste the prospective team's time, then a call with 2 of my prospective colleagues to gauge competence and cultural fit. Since then I have been involved in hiring most of the team I work with now. The CTO is one of the most competent engineers I have ever met and he designed this process. He also has very high EQ. One of the points I sell to prospective hires is him as a person to work with, as well as our team. He has also flatly denied people I suggested before and that's fine. I have been here 5 years now and I'm working with the most competent team I have ever worked with. My take away from this is that hiring doesn't need to be commoditised and scale, it just needs to find good people and give them an opportunity to show you that you do or don't want to work with them.
bberenberg 2y ago
This seems like a great idea, and similar to my old approach of “we pay you for 10-20 hours of work across a few days to see how you work with us”. The downside we found was inconsistency. Candidate 1 and 2 get variable difficulty work. How do we decide who did better?

chiph 2y ago

Was this an in-person pair programming session or remote via shared desktop? Because the latter is garbage. You can't see the other person or read their body language. It also gives off "proctored test" vibes. Or worse than vibes - where you're expected to clear your room, swing your camera around, and never look away.

ukd1 2y ago

Pairing on something close to whatever real work they'd be doing, but familiar to the applicant is my favorite way to evaluate someone (e.g. choose a side project, pre agree adding a feature). I don't care if someone uses modern tools (like AI assists), google, etc - e.g. "open book" - as that's the how they want to work. Evaluating their style of thinking / solving problems, comms, and output is the win.

agustechbro 2y ago

Beautifully articulated truth

m11a 2y ago

I dislike pair programming interviews - as they currently exist - because they usually feel like a time-crunched exam. You don't realistically have the freedom to actually think as you would in actual pair programming. i.e. if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview, but it's pretty realistic of real life work and entirely a non-problem. It's probably even good to test for at interview: how does a person work when they aren't working with an oracle that already knows the answer (ie: the interviewer)? Pair programming with the person for a couple hours, maybe even on an actual feature, would probably work, assuming the candidate is compensated for their time. I can imagine it'd especially work for teams working on open source projects (Sentry, Zed, etc). Might not be as workable for companies whose work is entirely closed source. Indeed, the other problem is what you mention: it doesn't scale to high throughput.

karaterobot 2y ago
That's definitely up to the interviewer, in which a lot of discretion and trust has been placed. I think a lot of it also comes down to the culture of the company—whether they're cutthroat or supportive. As you get better people into the company, hopefully this improves over time. I know that when we did it, it was never about nailing it on the first try, it was literally about proving you knew how to program and were not an asshole. So, not the equivalent of reversing a binary tree on a whiteboard. The kinds of problems we worked on in the interviews weren't leetcode type problems, they were real tickets from our current project. Sometimes it was just doing stuff like making a new component or closing a bug, but those were the things we really did, so it felt like a better test.
barbazoo 2y ago
> i.e. if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview That’s an assumption. Perhaps following a dead end for a while, realizing it, pivoting, etc is a valuable, positive, signal?
- m11a 2y ago
  I agree. But what I mean is: that's not how it's perceived in the current interview structure, which lasts maybe 45 minutes or so. Ultimately, going down a dead end means you'd now have 30 minutes to find the right solution and code it up. So the oracle (the interviewer) would probably help you realise sooner that it's a bad idea, so you don't waste your time. That's assuming they know the problem and solution well; if they don't, you'll just lose them and burn through your time. In a 2 hour pair programming session on an 'unsolved' problem (like an open issue / minor bug / minor feature in a public repo), yes, it will likely not matter if you tried a bad idea, and would both be more realistic and a positive signal.
  - dingnuts 2y ago
    this is an interviewer problem, unless the candidate is totally silent. A candidate that can't ask questions isn't going to succeed, anyway. If the candidate will communicate with me, I will offer them a LOT of guidance. It is still very, very easy to tell who knows what they are doing and who does not. You give them a basic but domain-specific task, you give them whatever extra context they need to do it, and you watch them hammer out the code. It should be a task that is sufficiently familiar to the person applying to the role that they do not need to do -much- looking at docs, and as the interviewer you should be prepared to help them quickly find the docs you already know they will need -- you designed the task after all -- so that they don't waste time with that. What's important is that they ask for docs when they need them, and that they can understand them, quickly, and use them. It will be obvious if they are using AI because of how long things will take. It will be obvious if they don't know when to reach for documentation, and it will be obvious if they cannot understand the documentation. Then, they should write a test for their solution. This weeds out 95% of candidates. Talk to the other 5% and you'll find someone who can both actually write code and also discuss design.

hylaride 2y ago

Similarly, in general the best interviews I've ever been part of (either giving or being) turn into discussions where people's experience, opinions, and stories get aired (going both ways). You eventually get a good sense of each other and things get more relaxed when you both realize that you know what you're talking about (this is harder for Jr roles, though). Being peppered with questions very rarely gives any insight.

lanstin 2y ago
For junior roles, you want to interview for intelligence and shall we say an interest in learning rather than specific skills. Even for senior roles, that's what I want to interview for, although it is true at times a business case can be made for someone that is good at some specific complex skill and doesn't need to listen to other people to do ok work.

joshdavham 2y ago

I've (unfortunately) been interviewing the last two months and the main pattern that I've noticed is that a) big companies have terrible interview processes while b) small companies and startups are great at interviewing. Big companies need to hire tons of people and interview even more so they need some sort of scalable process for it. An early stage startup can just ask you about your past projects and pair program with you for an hour.

solarmist 2y ago
I hear this all the time, but I have yet to experience it. It may be because the small companies that I interview with are all startups, but I have yet to be able to get a call back from any other kind of small company. And the startups I do interview with have a full FAANG interview loops. There seems to be a weird selection bias that if you're FAANG or FAANG adjacent these small companies aren't interested.
- codr7 2y ago
  Yeah, been there, done that; wannabe FAANGs are the worst.
  - drdaeman 2y ago
    At least those are typically honest about what they try to be and give a very clear signal, right at the interview time. It's much worse when the interview gives you different vibes (and expectations) than the actual day-to-day work.
    - solarmist 2y ago
      I've had both I've had ones that only tell you what the next interview looks like. Unless you specifically prior it out of the recruiter's hands.
    - codr7 2y ago
      The only way to test actual work is to do actual work, should be obvious but here we are.

Attummm 2y ago

I've been a proponent of pair programming since the early days of Agile, when it was still seen as part of extreme programming. Unfortunately, it’s not often employed in workplace settings. With that said, would your perception of the interview remain positive if the outcome had been negative? A common challenge across all interviews is a mismatch in personal dynamics, which can significantly impact the experience for both participants. Consider a scenario where a senior developer, who prefers simplicity, is paired with a mid-level developer who is still captivated by complexity.

lanstin 2y ago
Or a "just start typing" person is with a "mull it over first" person. By the time I am typing code, I want to have 90% of it already completely worked out (at least till I type a "c.Lock()" and suddenly realize I hadn't considered thus and so synchronization issue.

tristramb 2y ago

"We need to run through hundreds of candidates per position, not a half dozen" But you don't! You only need to find the first person who is good enough to do the job. You do not need to find the best person.

dowager_dan99 2y ago
You need to run through hundreds of candidates to find someone marginally qualified. I am not exaggerating.
- samr71 2y ago
  Do we have different definitions of "marginally qualified"? Idk, I feel I'm a decent engineer - I can certainly do whatever leetcode medium they throw at me, as much as this counts of anything - and can actually code, but I still get maybe 1 callback per 50 applications. Does "marginally qualified" mean "Ivy League Competitive Programmer PhD" or something?
  - marcosdumay 2y ago
    The GP is still passing through hundreds of people, dozens of them capable, until he reaches somebody that convinces him of their competence. You were passed down because you weren't convincing enough. Or maybe he is getting resumes from a channel that has been victim of machine-gun filling, and there indeed thousands of incompetent people posting resumes into every channel and just half a dozen real applicants. TBF, I have no idea how to fix either one of those problems. Hiring is just completely broken.
    - samr71 2y ago
      I have only a resume to convince them. I have job experience at major companies, with examples of what I've built when there, personal projects I've made, a github link, a great GPA from a good school. And I know similar Junior-mid people in the same boat. We can all do Fizzbuzz, we've all built things, and somehow we're not getting interviews, but people that can't do Fizzbuzz are. Do thousands of incompetents also machine-gun apply to, say, mechanical engineering, accounting, marketing, HR, or finance gigs? Is it just tech? Something isn't adding up.

superb_dev 2y ago

This is how my team hires and it’s incredible. I think what makes it work is that our code pair is pretty low stakes. I was told that I didn’t have to finish the problem and I was free to use whatever tools or language I needed. They just wanted to see how I work and collaborate.

slg 2y ago

>The best interview process I've ever been a part of involved pair programming with the person for a couple hours... You never failed to know within a few minutes whether the person could do the job There is something funny about the "best interview process" taking "a couple hours" despite giving you the answer "within a few minutes". Seems like even the best process is a little broken.

alwa 2y ago
Lightly ironic indeed! Though I’m not sure “broken” is exactly the word I’d choose. I can only speak for myself, but I imagine myself as a candidate approaching a “couple of hours” project or relationship differently than I would a “few minutes” speed round. For that matter I can think of people I know professionally who I only know through their behavior “on stage” in structured and stylized meetings of a half hour or an hour—and I don’t feel like I have any sense at all of how they would be as day-to-day coworkers. If we sat down to work together, you’d probably have a sense in the first few minutes of whether or not we were going to work out—but that would be contingent on us going into it with the attitude that we were working together as colleagues toward a goal.
karaterobot 2y ago
That's mainly because there were multiple pairing sessions, and even if you knew the person was going to pass, there are still a couple more people who need to meet them, and a schedule to make sure they're available to do that. Plus due diligence, etc. Nor am I saying it was a perfect system, just the best I've seen in terms of results.

andyjohnson0 2y ago

Some of us find the prospect of pairing with an unknown person in an unknown environment, and against the clock, to be very stressful.

andyjohnson0 2y ago
Anecdote: I've been interviewing recently and got through to the last round (of five...) with an interesting company. I knew the interview involved pairing, but I didn't expect: two people sitting behind me while I sat on a wobbly stool at a standing desk, trying to use a sticky wired mouse and a non-UK keyboard, and a very bright monitor (I normally use dark mode). They had a screen recorder running, and a radio was on in the next room. I totally bombed it. I suspect just the two people behind me would have been enough though.
hirvi74 2y ago
I would find trying to solve such problems with known people in known environments to be somewhat stressful too.
stcroixx 2y ago
I've had a great career and very positive feedback from every employer. I would do absolutely terrible in this kind of interview.

lifeisstillgood 2y ago

This is the exact time to use phrase “A people hire other A people, B people hire C people” Additionally it’s rarely the hiring that makes a great team - it’s the long term commitment and investment in training.

dowager_dan99 2y ago

Very few people doing this sort of interview (they tend to be our best, most empathetic developers) are likely to cut a multi-hour planned process short after a few minutes. It will eat at least an hour of their (very expensive & valuable) time. Also how am I supposed to filter the 100's of AI-completed assessments? Who gets this opportunity?

samr71 2y ago
Use another AI, of course! Idk if I'm even being sarcastic here.
karaterobot 2y ago
We didn't do assessments (if by that you mean take home assignments). This was partly a solution to that, since nobody thought they were a good idea. If you mean the phone screen, I think that would be a problem, yep, but it wasn't an issue back in 2016. Having them pair would weed out cheaters, but we would have to figure out a way to weed them out during the screening, I agree. We also did not require the employers doing the interview to be our most senior team members. They probably did it more often than most people, but often because they volunteered to do it. Anyone on the team would be part of the loop, which helped with scheduling. And, remember, we were working on actual tickets, so in a lot of cases it actually helped having the candidate there as a pairing partner. For a little extra detail, the way we actually did it was to have 2-3 pairing sessions of up to 2 hours apiece. At the end of the day, all the team members who paired with the candidate had to give them the thumbs up.

jitix 2y ago

I used to love getting to know the interviewer and doing things like that but IMO the market has shifted fundamentally on both ends for this to be effective anymore for most SaaS roles. This is anecdotal for US/Canada tech market over the past 10 years so YMMV. Developers Side: Since developers don't have job security anymore (at least for those who work on common languages like Go, Python, Java and Typescript) they are better off learning and keeping in touch with leetcode and system design questions, looking for new opportunities and interviewing in "batch mode" when looking for a job. The idea is to clear as many interviews as possible using the same concepts, get in and make money asap before you get laid off. No incentive for collaboration or for fulfilling but esoteric stuff like Haskell and Scala. Career security > Job security. Companies Side: On the other end software companies have less trust in developers staying long term so they want to make the interview process as quick and risk free as possible. In essence they are betting that by perusing 100s of resumes and hiring someone who seemingly knows CS concepts they can get some value out of them before they leave. Standardized tests/vetting > team fit. TLDR; The art is gone from this job, its become akin to management consulting or investment banking. Quality and UX seems to be regressing across the board as a result.

no_wizard 2y ago
>its become akin to management consulting or investment banking Not sure how those are similar.
- jitix 2y ago
  I meant the “grind”, short term profit mentality of SWE market has become similar to professionals in those fields, not that any of these fields are similar.

hassleblad23 2y ago

Pair programming on what problem though? I dont think many companies would want an outsider to work on their codebase.

hassleblad23 2y ago
And devs wouldn't want to spend substantial time working on stuff unrelated to work just for the interview process.

whiplash451 2y ago

It's a super interesting approach, and if you put a strong filter before it (e.g. intense non-BS Q&A), the whole thing could be high-throughput.

shihab 2y ago

The biggest victims of these non-scalable process is people without a good network. As an intl PhD student, I am that person. So now I have this weird dynamic: I get interview calls only from FAANG companies, the ones with the manpower to do your so called "cursed" scalable interviews. But the smaller companies or startups, ones who are a far better fit for my specialized kills, never call me. You need to either "know someone" or be from a big school, or there is zero chance.

namuol 2y ago

Does your company operate the same way? I.e. is most, or at least a large chunk of engineering done as pair-programming?

qudat 2y ago

> You never failed to know within a few minutes whether the person could do the job Then why spend a couple hours?

0x457 2y ago

> person for a couple hours, >You never failed to know within a few minutes whether the person could do the job Did a misunderstood something or your best interview process is to multiple hours from someone when you've decided within minutes?

reverendsteveii 2y ago

This. Interviewing for a sr dev position with a web app, backend stack is the bog standard java, spring, SQL abstracted away via JPA. We did a first screen, then the tech interview was two of their senior devs shoulder surfing me as I built a simple API. We chatted, I built, they asked questions, I defended my decisions (sometimes successfully, sometimes gracefully conceding defeat), they left knowing that I was who my resume said I was and the reminder that popped up in the middle of the interview to feed my sourdough starter showed them that I'm a culture fit. I think you're onto something with that last paragraph but I want to try being a bit more generous with why things are the way they are. The question seems to be "When there are hundreds of applicants how do we give everyone a fair shake without hiring an entire team of devs who do nothing but interview?" From that perspective the intentionality is different and even sensible but the end product is likely to be the same. Even when someone is chasing a metric it's because someone else wants what's best and has decided that metric is a sensible way to make that happen. At the end of the day they really do want to hire the best candidate out of a pool whose size is extremely variable and that's challenging.

christkv 2y ago

Thats what we did, pair program on some real production code and tickets. This way the person could get a feel about what they potentially were walking into and you get a good idea of how they think and approach problems.

zensavona 2y ago

I work at a company which has 11 engineers and competes with companies with 100s. The hiring process was a screening call with the CTO to not waste the prospective team's time, then a call with 2 of my prospective colleagues to gauge competence and cultural fit. Since then I have been involved in hiring most of the team I work with now. The CTO is one of the most competent engineers I have ever met and he designed this process. He also has very high EQ. One of the points I sell to prospective hires is him as a person to work with, as well as our team. He has also flatly denied people I suggested before and that's fine. I have been here 5 years now and I'm working with the most competent team I have ever worked with. My take away from this is that hiring doesn't need to be commoditised and scale, it just needs to find good people and give them an opportunity to show you that you do or don't want to work with them.

bberenberg 2y ago

This seems like a great idea, and similar to my old approach of “we pay you for 10-20 hours of work across a few days to see how you work with us”. The downside we found was inconsistency. Candidate 1 and 2 get variable difficulty work. How do we decide who did better?