3 ms·

I try to point out at the beginning that we understand it's manufactured pressure, and try to ease their minds if we can before getting started. In particular,

I try to point out at the beginning that we understand it's manufactured pressure, and try to ease their minds if we can before getting started. In particular, we ask people to solve the problems in whatever language they are most comfortable with, and that it's an open book test; we're not testing you on whether you've memorized the Python standard library. This can certainly make things interesting! I've spent most of my career with perl, python, c#, and java, so the guy deciding to do his challenge in clojure was perhaps more of a challenge for me than it was for him. I don't think there's a universal definition of 'high pressure,' so I am not sure what else to say about this. It's certainly not a fishbowl experience where you're surrounded by 20 people yelling and pleading for you to save their lives by writing a function to reverse a string in place. And while I've read that this happens elsewhere, we're not critiquing you on the spot and arguing with you about why using this technique causes the interpreter to do some less efficient thing and how could you be so stupid. We try to coach and offer suggestions though, to help a candidate get to the right answer. That doesn't mean they necessarily passed - if we have to basically give you the answer, then you didn't solve the problem, and the coding interview is one of several inputs into a hiring decision - but I don't like to let candidates leave feeling like they failed if I can help it. I like to find at least one thing to praise about the solutions they turn in and tell them why I liked what they did. It is not the point of these interviews to ruin someone's day or make them feel worthless. All that said, we can only do so much. Ultimately whether you feel an environment is high pressure is on you, and how you handle it is also on you. We would regret missing out on a good candidate but I'm sure it happens. There are other candidates, and other opportunities for those we've missed out on.

haswell 4y ago
> Ultimately whether you feel an environment is high pressure is on you, and how you handle it is also on you. Obviously that is subjective, but this is on everyone involved, especially the company creating said environment. Twitter 2.0 is an obvious (if extreme) example. The bottom line is that the whole approach is manufactured, and not indicative of real world problem solving skills or what the candidate will be dealing with on the team. > We would regret missing out on a good candidate but I'm sure it happens. It absolutely happens, and I’d bet good money that it happens often. Whether they realize it or not, most teams that take this approach end up creating an environment that is skewed towards a particular kind of personality, and that is ultimately a net negative in the long run. Back to my personal anecdote (which I’ve observed over and over throughout my career), the teams that rely too heavily on this approach end up rather dysfunctional and unbalanced over time. The team that turned me down was disbanded after some horrific security issues came to the surface. They were so focused on whether candidates could solve certain algorithms that they apparently didn’t verify that these devs understood the fundamentals of client-side vs. server-side validation and why they’re important (and not the same thing). But those devs could bang out a chess algorithm on a whiteboard with the best of them. I’m not saying that’s how you’re running things, but in my experience, teams that overemphasize this are doing so at the expense of far more important indicators. It takes a lot of discipline and care to not let this become a crutch. I’m also not saying it isn’t important to verify that a candidate has the necessary skills, but having a conversation about person’s learning style, how they got into development, how they go about taking on new tasks, etc can go a long way. I care more about mindset and approach to information acquisition than I do about what a candidate can whip up in a hour. Sitting down together and going over some existing code (after giving them enough time to absorb it) while having a discussion about the nuances of that code is far more collaborative, and in my experience far more likely to indicate whether or not the candidate knows what’s going on. I’ve spoken to many like minded and high performing colleagues who will not consider companies that are overly focused on this. Another colleague was recently offered a position after going through such a process, but bailed when they failed to follow up with enough meaningful conversation. Her conclusion: if that’s all they care about, I don’t trust that they’ve built a good team. There’s a growing contingent of the types of people who you’d want the most, but will not consider your team for doing this.
- saxonww 4y ago
  > I’m not saying that’s how you’re running things Good, because we're not. The main takeaway here, which I think we agree on, is that a good hiring decision requires more than one input. We happen to use a coding challenge as one input in our decisions, but I agree that a collaborative discussion could work too. In particular, we do ask about background and learning style, among other things, in subsequent interviews. Overall, if I had to attach a percentage to it, I'd say the coding challenge is no more than 25% of the overall decision, with the provision that if you clearly just don't know what you're doing at all - which has happened just a handful of times that I can remember - we're not going to invest more time in you as a candidate. > There’s a growing contingent of the types of people who you’d want the most, but will not consider your team for doing this. All I can say to this is that you're probably right, but also that almost no one is as uniquely qualified as they think they are. There are millions of candidates, and especially now that more software companies are OK with remote work, we can consider more of them than before. This is really good and really bad, the latter because employers are even more likely to :shrug: at this sort of thing and move to an equivalently-capable candidate who doesn't complain about having a one hour coding test.
  - haswell 4y ago
    Yeah, I think we’re on the same page regarding the importance of multiple inputs. > In particular, we do ask about background and learning style, among other things, in subsequent interviews. In the past, I’ve done the opposite, and started with these questions up front. In my mind, they’re the most important thing to learn about a candidate, but I realize not everyone goes about this the same way. It does require more time investment up front though, and I get that. > I'd say the coding challenge is no more than 25% of the overall decision, with the provision that if you clearly just don't know what you're doing at all - which has happened just a handful of times that I can remember - we're not going to invest more time… If this is the case, wouldn’t that be evidence that this process should be de-emphasized even further, or at least moved to a later stage? Correct me if I’m misreading this, but if almost everyone coming through has had some level of success with the coding stage, it’s not really screening many candidates out, but still could be changing the candidate’s view of your team and company. If it’s the first thing they encounter, it’s a big part of the first impression the candidate forms. In a worst case scenario, you may drive someone good away with minimal benefit to the hiring process, and in a best case scenario, you’ve only saved a few conversations given the low failure rate. > but also that almost no one is as uniquely qualified as they think they are. I completely agree with you here, I just don’t believe an artificial coding test is the antidote to that. The more experience and confidence someone has (warranted or not), the less they feel they need to put themselves through pain for their next role. Really not trying to be difficult here, but trying to understand the mindset and strategy behind this style of hiring, and I’m not quite seeing where the value is coming from based on what you’ve shared. I will say that what you describe is one of the more reasonable stances I’ve come across, but still seems potentially counterproductive. I do hear you regarding the access to millions of candidates. It’s a great problem to have, I just wonder if the old ways are still relevant. Thanks for being willing to share your thought processes on this - it’s what I love about this site.
quickthrower2 4y ago
High pressure to me, in this case, is because the stakes are high. And if not enough time is given that makes it pressure. The whole way I think in these circumstances is different to on the job. There is more shortcutting, looking for artificial tricks and so on. In the real job more deep thinking/strategic/ looking for real world murphys law type things that’ll bite you. But interviewing, i.e. determining if you want 5k hours work from someone in less than 5 hours, is going to be heuristic and imperfect. Always.

haswell 4y ago

> Ultimately whether you feel an environment is high pressure is on you, and how you handle it is also on you. Obviously that is subjective, but this is on everyone involved, especially the company creating said environment. Twitter 2.0 is an obvious (if extreme) example. The bottom line is that the whole approach is manufactured, and not indicative of real world problem solving skills or what the candidate will be dealing with on the team. > We would regret missing out on a good candidate but I'm sure it happens. It absolutely happens, and I’d bet good money that it happens often. Whether they realize it or not, most teams that take this approach end up creating an environment that is skewed towards a particular kind of personality, and that is ultimately a net negative in the long run. Back to my personal anecdote (which I’ve observed over and over throughout my career), the teams that rely too heavily on this approach end up rather dysfunctional and unbalanced over time. The team that turned me down was disbanded after some horrific security issues came to the surface. They were so focused on whether candidates could solve certain algorithms that they apparently didn’t verify that these devs understood the fundamentals of client-side vs. server-side validation and why they’re important (and not the same thing). But those devs could bang out a chess algorithm on a whiteboard with the best of them. I’m not saying that’s how you’re running things, but in my experience, teams that overemphasize this are doing so at the expense of far more important indicators. It takes a lot of discipline and care to not let this become a crutch. I’m also not saying it isn’t important to verify that a candidate has the necessary skills, but having a conversation about person’s learning style, how they got into development, how they go about taking on new tasks, etc can go a long way. I care more about mindset and approach to information acquisition than I do about what a candidate can whip up in a hour. Sitting down together and going over some existing code (after giving them enough time to absorb it) while having a discussion about the nuances of that code is far more collaborative, and in my experience far more likely to indicate whether or not the candidate knows what’s going on. I’ve spoken to many like minded and high performing colleagues who will not consider companies that are overly focused on this. Another colleague was recently offered a position after going through such a process, but bailed when they failed to follow up with enough meaningful conversation. Her conclusion: if that’s all they care about, I don’t trust that they’ve built a good team. There’s a growing contingent of the types of people who you’d want the most, but will not consider your team for doing this.

saxonww 4y ago
> I’m not saying that’s how you’re running things Good, because we're not. The main takeaway here, which I think we agree on, is that a good hiring decision requires more than one input. We happen to use a coding challenge as one input in our decisions, but I agree that a collaborative discussion could work too. In particular, we do ask about background and learning style, among other things, in subsequent interviews. Overall, if I had to attach a percentage to it, I'd say the coding challenge is no more than 25% of the overall decision, with the provision that if you clearly just don't know what you're doing at all - which has happened just a handful of times that I can remember - we're not going to invest more time in you as a candidate. > There’s a growing contingent of the types of people who you’d want the most, but will not consider your team for doing this. All I can say to this is that you're probably right, but also that almost no one is as uniquely qualified as they think they are. There are millions of candidates, and especially now that more software companies are OK with remote work, we can consider more of them than before. This is really good and really bad, the latter because employers are even more likely to :shrug: at this sort of thing and move to an equivalently-capable candidate who doesn't complain about having a one hour coding test.
- haswell 4y ago
  Yeah, I think we’re on the same page regarding the importance of multiple inputs. > In particular, we do ask about background and learning style, among other things, in subsequent interviews. In the past, I’ve done the opposite, and started with these questions up front. In my mind, they’re the most important thing to learn about a candidate, but I realize not everyone goes about this the same way. It does require more time investment up front though, and I get that. > I'd say the coding challenge is no more than 25% of the overall decision, with the provision that if you clearly just don't know what you're doing at all - which has happened just a handful of times that I can remember - we're not going to invest more time… If this is the case, wouldn’t that be evidence that this process should be de-emphasized even further, or at least moved to a later stage? Correct me if I’m misreading this, but if almost everyone coming through has had some level of success with the coding stage, it’s not really screening many candidates out, but still could be changing the candidate’s view of your team and company. If it’s the first thing they encounter, it’s a big part of the first impression the candidate forms. In a worst case scenario, you may drive someone good away with minimal benefit to the hiring process, and in a best case scenario, you’ve only saved a few conversations given the low failure rate. > but also that almost no one is as uniquely qualified as they think they are. I completely agree with you here, I just don’t believe an artificial coding test is the antidote to that. The more experience and confidence someone has (warranted or not), the less they feel they need to put themselves through pain for their next role. Really not trying to be difficult here, but trying to understand the mindset and strategy behind this style of hiring, and I’m not quite seeing where the value is coming from based on what you’ve shared. I will say that what you describe is one of the more reasonable stances I’ve come across, but still seems potentially counterproductive. I do hear you regarding the access to millions of candidates. It’s a great problem to have, I just wonder if the old ways are still relevant. Thanks for being willing to share your thought processes on this - it’s what I love about this site.

quickthrower2 4y ago

High pressure to me, in this case, is because the stakes are high. And if not enough time is given that makes it pressure. The whole way I think in these circumstances is different to on the job. There is more shortcutting, looking for artificial tricks and so on. In the real job more deep thinking/strategic/ looking for real world murphys law type things that’ll bite you. But interviewing, i.e. determining if you want 5k hours work from someone in less than 5 hours, is going to be heuristic and imperfect. Always.