7 ms·

At my old company, we never did take home assignments or whiteboard interviews. We just sat candidates down and pair programmed with them for a couple hours on

At my old company, we never did take home assignments or whiteboard interviews. We just sat candidates down and pair programmed with them for a couple hours on whatever we were working on at the time. We paid them the same hourly rate we were making, (but didn't charge the clients for it). If the candidate obviously didn't know what they were doing, we ended it early. Usually you knew within 15 minutes. One candidate hated the interview process, and walked out. One candidate, likely a future business leader, argued vociferously for a higher hourly rate during the interview, and later with the founder. But I would describe our success rate for finding good programmers who were easy to work with as "extremely high", and our rate for false positives as "I can't remember ever having one in about 4 years of working there". The overhead for the engineering team to run these was not zero, and not everybody enjoyed doing it. But some people did, so we had them do it more often. I have no idea why this isn't a more common practice, I would recommend it to anybody.

_448 4y ago
This! > I have no idea why this isn't a more common practice My maths teacher once said in class during lecture, "Kids, when you venture into the outside world, you will realise 'common-sense is not a common thing!'".
activitypea 4y ago
Can you go a bit more in depth about the pairing session? Most companies I've worked at, even if the person coming in was familiar all technologies we were using, I can't imagine them grokking nor just "skipping over" the datamodel and system architecture/design. All non-trivial projects I've worked on had a "bad" codebase as a result of deadlines, pivots and misunderstandings. The code is always steadily improving, but never good enough that someone can just waltz in and get to hacking on it.
- kornork 4y ago
  I can't answer for the parent, but I've done similar style interviews. Because the candidate is not familiar with your codebase, you must be the expert. Your job is to let them drive as much as possible, while you "unstick" them when they trip over what they can't possibly know. It is a collaboration, which is part of what makes this a good interview style IMO. Sometimes a candidate will offer an otherwise good solution, but it won't work well within your codebase -- in this case you explain why, and then you iterate.
- karaterobot 4y ago
  Yeah, usually we would give a little bit of context, and then look at the sprint tickets in Jira, and take one. The candidate was not expected to be familiar with the codebase or product at all beforehand, and they weren't expected to become an expert on it during the course of the interview. But coming up to speed quickly is one of the skills we liked to see, since that was part of the job. They should be able to 'hang', in the sense of being familiar with how we approach problem solving as programmers. They should ask questions when needed, and answer questions as they arose. The goal was to just figure out what level they were at in general, and what it would be like to work with them. They weren't expected to come up with the best possible solution, because usually there wasn't a single right solution. We would ask them questions like "well, you saw the ticket, how do you think we should approach this?" or even just natural things like "this function we just wrote isn't working, what's going on here?". The point is that they are working on real problems — literally real problems — and not abstract CS exercises. Since we mostly did pair programming every day, the other thing you'd get a sense of is how easy someone is to work with. Not everybody had experience pairing, which is to be expected. You found out quickly who took to it, and who wanted to be a lone wolf. We didn't not hire someone because they weren't good at pairing, but we factored it in. Some people just did not communicate very well, which is a problem. (This also gave them a good sense of what it would be like to work with us, and sometimes people decided not to take an offer because they didn't like the way we worked: good outcome for everybody) They would usually do two rounds of this, both on the same day. Morning session, then an afternoon session. We offered to take them to lunch in between, which exposed them to more of the team, and let us talk to them in a more casual setting. Part of the interview would also be a private talk with one of the founders, and an HR person (in a different session). The people the candidate paired with would provide input like: "I would recommend hiring this person" or not, but didn't make the decision themselves. Here, I should admit that we did have a set of artificial problems we'd give candidates if we were hiring for a skillset that did not have a real project to test them on. We did both web and mobile contracts at this (consulting) company, so if we were hiring, e.g., an Android dev but didn't have an Android project for them to work on, we'd have them check out a repository and work on a fake ticket instead. Usually this was with a developer beside them, but I can remember times when the candidate had to do it solo.
  - 4y ago
colinmhayes 4y ago
This is great for small companies, but large companies are interested in standardizing the interview process. Leetcode is great for that since the outcome is pretty binary.
- karaterobot 4y ago
  Agreed, but I see the impulse to standardize as the source of a lot of the problems people have — by people, I mean both good engineers, and the companies who can't hire them.
  - angarg12 4y ago
    Honest question, how would you handle hiring at FAANG scale without standard processes? For reference we had an opening in my team and 1600 people applied.
    - b20000 4y ago
      go on linkedin and message the people you think can do the job based on prior work and experience or degrees. trash the rest.
    - karaterobot 4y ago
      This process still relied on screening, including technical screens. The biggest difference is that it substituted a day of whiteboard interviews with a day of pairing, which we found to produce better results, and candidates really liked too.
    - hcrean 4y ago
      Having 1600 applicants for a SE role is not a unique, but definitely a very niche, problem to have...
    - animalhot 4y ago
      Apple used to have a per team interview process(don’t know if it has changed). While the interview experience can vary a lot, there are certain pockets/teams that are spectacularly brilliant and have the freedom to maintain the bar & conduct targeted hiring.
gopher_space 4y ago
> I have no idea why this isn't a more common practice It used to just be “the practice”. The folks who don’t do this are trying to automate hiring.
- _448 4y ago
  > The folks who don’t do this are trying to automate hiring. Or they are trying to emulate Google and similar companies without understanding why they do it.
kushan2020 4y ago
If this interview process is so golden (based on your metrics of success) why isn’t every one doing it?
- caffeine 4y ago
  Not OP but here are a couple reasons: * It doesn’t scale well to huge organisations. * It’s not standardised “repeatable” in the sense that you cant generate datasets from this that let you correlate interviews with subsequent outcomes and improve hiring metrics company wide * It requires expertise in pair programming, which most companies don’t have Basically, OP’s hiring process is an excellent example of “doing things that don’t scale” to gain an asymmetric advantage as a small organisation. They don’t need to derive metrics, they can invest the time, they have a team who are skilled at pair programming - so they can have this hiring process and other people can’t.
  - r0b05 4y ago
    Brilliant.
- f9weew 4y ago
jacknews 4y ago
This is exactly the right way to do it. Not only do you find out if the candidate can program, but how well they can pick up unfamiliar code, tools, etc, can they communicate, can they work with others, etc, etc, etc. Leetcode bears little resemblance to everyday programming. Of course it's sometimes necessary to understand big-O, and that various data structures and algorithms exist, and which ones you might apply to certain problems, but coding them from scratch, from memory, within a fixed time? Pretty much never.
- heleninboodler 4y ago
  > but coding them from scratch, from memory, within a fixed time? Pretty much never This is definitely true, but having to do it quickly and on-the-spot is one way of measuring just how comfortable you are with it. You might never have to do it in an hour, but being able to is a proxy, however weak and indirect, for your competence, and guess what, I only have an hour to talk to you. I love and hate leetcode interviews. I enjoy the challenge, and I hate being put on the spot and knowing that poor performance on one specific problem might result in an unwarranted thumbs down. I sincerely hope any interviewer whose chair I'm sitting in treats them the way I do, which is that it isn't important that you come up with the optimal solution; it's important that you can think through the problem and come up with any solution, however inefficient, and then honestly evaluate this solution (including big O) and point out how it could be improved. You probably don't have time to implement that improvement, but you've demonstrated competence in both thinking and coding along the way. Edit to add: When I'm preparing to interview, I practice leetcode-style problems, but not so much to memorize all the questions and their answers as much to just practice my quickness for thinking and coding on that type of thing. It feels kinda silly because, as you point out, that's pretty much not a part of the job at all, but it's very very helpful to exercise your mental muscles for that sort of thing so you don't stumble over the simple mechanics of the problem. It feels absolutely great when you get a question you've never heard before but you've put yourself in the mental state to solve it and you kill it. And like it or not, I've personally found that people who enjoy this type of challenge are often the exact types of coders I want to work with.
  - jacknews 4y ago
f9weew 4y ago
gwnywg 4y ago
I'm 15+ years of experience, I funded a few high level projects for corps and a few startups including hiring my teams. I was always conducting my interviews by making some work with the candidate and I found it best approach. And quite recently I sat on another end of hiring table. I'm good at algorithmic challenges like letcode or hackerrank, I always liked puzzles like this. And yet, last interview I failed. Purely because I got a bit nervous and did not manage to deliver fully working solution in 15 minutes in front of interviewer. And that after passing their hackerranks and answering their general system questions. I guess their approach was all or nothing. Is it good or bad, I don't know. But from hearing their problems I knew their problems is something I eat for breakfast :)
b20000 4y ago
the goal of the leetcode interviews is to get into who is the smartest mode, so the cost of hiring can be driven down. those who conduct the interviews are unaware of this. the process you describe would result in paying people more which is not what FAANGs and those imitating them want.

_448 4y ago

This! > I have no idea why this isn't a more common practice My maths teacher once said in class during lecture, "Kids, when you venture into the outside world, you will realise 'common-sense is not a common thing!'".

activitypea 4y ago

Can you go a bit more in depth about the pairing session? Most companies I've worked at, even if the person coming in was familiar all technologies we were using, I can't imagine them grokking nor just "skipping over" the datamodel and system architecture/design. All non-trivial projects I've worked on had a "bad" codebase as a result of deadlines, pivots and misunderstandings. The code is always steadily improving, but never good enough that someone can just waltz in and get to hacking on it.

kornork 4y ago
I can't answer for the parent, but I've done similar style interviews. Because the candidate is not familiar with your codebase, you must be the expert. Your job is to let them drive as much as possible, while you "unstick" them when they trip over what they can't possibly know. It is a collaboration, which is part of what makes this a good interview style IMO. Sometimes a candidate will offer an otherwise good solution, but it won't work well within your codebase -- in this case you explain why, and then you iterate.
karaterobot 4y ago
Yeah, usually we would give a little bit of context, and then look at the sprint tickets in Jira, and take one. The candidate was not expected to be familiar with the codebase or product at all beforehand, and they weren't expected to become an expert on it during the course of the interview. But coming up to speed quickly is one of the skills we liked to see, since that was part of the job. They should be able to 'hang', in the sense of being familiar with how we approach problem solving as programmers. They should ask questions when needed, and answer questions as they arose. The goal was to just figure out what level they were at in general, and what it would be like to work with them. They weren't expected to come up with the best possible solution, because usually there wasn't a single right solution. We would ask them questions like "well, you saw the ticket, how do you think we should approach this?" or even just natural things like "this function we just wrote isn't working, what's going on here?". The point is that they are working on real problems — literally real problems — and not abstract CS exercises. Since we mostly did pair programming every day, the other thing you'd get a sense of is how easy someone is to work with. Not everybody had experience pairing, which is to be expected. You found out quickly who took to it, and who wanted to be a lone wolf. We didn't not hire someone because they weren't good at pairing, but we factored it in. Some people just did not communicate very well, which is a problem. (This also gave them a good sense of what it would be like to work with us, and sometimes people decided not to take an offer because they didn't like the way we worked: good outcome for everybody) They would usually do two rounds of this, both on the same day. Morning session, then an afternoon session. We offered to take them to lunch in between, which exposed them to more of the team, and let us talk to them in a more casual setting. Part of the interview would also be a private talk with one of the founders, and an HR person (in a different session). The people the candidate paired with would provide input like: "I would recommend hiring this person" or not, but didn't make the decision themselves. Here, I should admit that we did have a set of artificial problems we'd give candidates if we were hiring for a skillset that did not have a real project to test them on. We did both web and mobile contracts at this (consulting) company, so if we were hiring, e.g., an Android dev but didn't have an Android project for them to work on, we'd have them check out a repository and work on a fake ticket instead. Usually this was with a developer beside them, but I can remember times when the candidate had to do it solo.
- 4y ago

colinmhayes 4y ago

This is great for small companies, but large companies are interested in standardizing the interview process. Leetcode is great for that since the outcome is pretty binary.

karaterobot 4y ago
Agreed, but I see the impulse to standardize as the source of a lot of the problems people have — by people, I mean both good engineers, and the companies who can't hire them.
- angarg12 4y ago
  Honest question, how would you handle hiring at FAANG scale without standard processes? For reference we had an opening in my team and 1600 people applied.
  - b20000 4y ago
    go on linkedin and message the people you think can do the job based on prior work and experience or degrees. trash the rest.
  - karaterobot 4y ago
    This process still relied on screening, including technical screens. The biggest difference is that it substituted a day of whiteboard interviews with a day of pairing, which we found to produce better results, and candidates really liked too.
  - hcrean 4y ago
    Having 1600 applicants for a SE role is not a unique, but definitely a very niche, problem to have...
  - animalhot 4y ago
    Apple used to have a per team interview process(don’t know if it has changed). While the interview experience can vary a lot, there are certain pockets/teams that are spectacularly brilliant and have the freedom to maintain the bar & conduct targeted hiring.

gopher_space 4y ago

> I have no idea why this isn't a more common practice It used to just be “the practice”. The folks who don’t do this are trying to automate hiring.

_448 4y ago
> The folks who don’t do this are trying to automate hiring. Or they are trying to emulate Google and similar companies without understanding why they do it.

kushan2020 4y ago

If this interview process is so golden (based on your metrics of success) why isn’t every one doing it?

caffeine 4y ago
Not OP but here are a couple reasons: * It doesn’t scale well to huge organisations. * It’s not standardised “repeatable” in the sense that you cant generate datasets from this that let you correlate interviews with subsequent outcomes and improve hiring metrics company wide * It requires expertise in pair programming, which most companies don’t have Basically, OP’s hiring process is an excellent example of “doing things that don’t scale” to gain an asymmetric advantage as a small organisation. They don’t need to derive metrics, they can invest the time, they have a team who are skilled at pair programming - so they can have this hiring process and other people can’t.
- r0b05 4y ago
  Brilliant.
f9weew 4y ago

jacknews 4y ago

This is exactly the right way to do it. Not only do you find out if the candidate can program, but how well they can pick up unfamiliar code, tools, etc, can they communicate, can they work with others, etc, etc, etc. Leetcode bears little resemblance to everyday programming. Of course it's sometimes necessary to understand big-O, and that various data structures and algorithms exist, and which ones you might apply to certain problems, but coding them from scratch, from memory, within a fixed time? Pretty much never.

heleninboodler 4y ago
> but coding them from scratch, from memory, within a fixed time? Pretty much never This is definitely true, but having to do it quickly and on-the-spot is one way of measuring just how comfortable you are with it. You might never have to do it in an hour, but being able to is a proxy, however weak and indirect, for your competence, and guess what, I only have an hour to talk to you. I love and hate leetcode interviews. I enjoy the challenge, and I hate being put on the spot and knowing that poor performance on one specific problem might result in an unwarranted thumbs down. I sincerely hope any interviewer whose chair I'm sitting in treats them the way I do, which is that it isn't important that you come up with the optimal solution; it's important that you can think through the problem and come up with any solution, however inefficient, and then honestly evaluate this solution (including big O) and point out how it could be improved. You probably don't have time to implement that improvement, but you've demonstrated competence in both thinking and coding along the way. Edit to add: When I'm preparing to interview, I practice leetcode-style problems, but not so much to memorize all the questions and their answers as much to just practice my quickness for thinking and coding on that type of thing. It feels kinda silly because, as you point out, that's pretty much not a part of the job at all, but it's very very helpful to exercise your mental muscles for that sort of thing so you don't stumble over the simple mechanics of the problem. It feels absolutely great when you get a question you've never heard before but you've put yourself in the mental state to solve it and you kill it. And like it or not, I've personally found that people who enjoy this type of challenge are often the exact types of coders I want to work with.
- jacknews 4y ago

f9weew 4y ago

gwnywg 4y ago

I'm 15+ years of experience, I funded a few high level projects for corps and a few startups including hiring my teams. I was always conducting my interviews by making some work with the candidate and I found it best approach. And quite recently I sat on another end of hiring table. I'm good at algorithmic challenges like letcode or hackerrank, I always liked puzzles like this. And yet, last interview I failed. Purely because I got a bit nervous and did not manage to deliver fully working solution in 15 minutes in front of interviewer. And that after passing their hackerranks and answering their general system questions. I guess their approach was all or nothing. Is it good or bad, I don't know. But from hearing their problems I knew their problems is something I eat for breakfast :)

b20000 4y ago

the goal of the leetcode interviews is to get into who is the smartest mode, so the cost of hiring can be driven down. those who conduct the interviews are unaware of this. the process you describe would result in paying people more which is not what FAANGs and those imitating them want.