3 ms·

These aren't just wacky, they are stupid. "How many ping-pong balls can fit in a VW bus" is just about the worst question I've ever gotten. Little or no relatio

These aren't just wacky, they are stupid. "How many ping-pong balls can fit in a VW bus" is just about the worst question I've ever gotten. Little or no relation to software development. Here's what we do. Interested in feedback, please! Via the phone: * A few background questions (why are you looking for a job, what sounds interesting about what we are doing) (20-30 minutes) * A bunch of topgrading questions http://www.smarttopgrading.com/ http://www.smarttopgrading.com/ (maybe an hour) In person: * A few programming challenges (of increasing difficulty) that involve stuff the candiate is not expected to know (some algorithmic, most problem-solving type thing), and has to solve it via Google and Stack Overflow or whatnot, to make sure they can work outside of their comfort zone (I've had a lot of people freeze up at "Write a program to get Time from outside of the system", for example) (couple of hours til lunchtime) * Then pair programming for the rest of the day to make sure we can all get along with this person (after lunch, and next day if possible) If reasonable, we put a new hire on a 1 month contract before we make a final offer (we give a tentative salary offer at day one so they know approximately what it'll be but cannot judge equity grants until we have more time with them).

finnw 14y ago
Are you hiring now? :-)
- MattRogish 14y ago
  Sure: https://www.fundinggates.com/jobs/ https://www.fundinggates.com/jobs/ :D
  - 31reasons 14y ago
    Interesting name of the company. Whats the story behind it ?
IvyMike 14y ago
> "How many ping-pong balls can fit in a VW bus" is just about the worst question I've ever gotten. The point of questions like this is "Does this guy have any common sense?" Some of my least favorite hires were people who were whip-sharp at the details of their chosen programming language, but then when hired, would go off and do crazy and impractical stuff with it. Whether or not the question accomplishes a good evaluation of someone's common sense is up for debate, but unless you're hiring drones, you do have to ask some questions that get a broader picture of the way the person thinks. (BTW, your "pair programming for the rest of the day" is a great way of getting that. But pair programming doesn't/wouldn't work at any place I've ever been, for a variety of reasons.)
- columbo 14y ago
  > The point of questions like this is "Does this guy have any common sense?" GROOOOOAAAAN Maybe twenty years ago, and maybe if you were the individual to think up the question. Honestly I'm so goddamned bored with these unoriginal blither-blather questions I consider them a strike against the interviewer and the company.
  - IvyMike 14y ago
    I actually have never asked an estimation question, but I understand the motivation for asking this kind of question. I'm curious--what do you ask in interviews to get at a candidate's common sense? It's not very easy to do. Or do you think you don't need to assess common sense at all? Maybe it's not, but like I said, a lot of the hires I later regretted came down to something like that.
    - columbo 14y ago
      > Or do you think you don't need to assess common sense at all? shrug If we're talking developers... The ability to drive to the location on time, answer questions without going blank are good indicators of common sense for me. Extracurricular activities, anything from volunteering at an animal shelter, to making civil war models to home brewing beer, are huge in my book because they are a good indicator of a grounded individual. That mixed with some pair-programming should give more than enough information. YMMV.
- MattRogish 14y ago
  I think the "real world, open-book" problems have to be crafted well, but I think they give a good sense of "Common sense". They are set up that you can absolutely prematurely optimize or go down really crazy paths (and I've seen them happen). You don't need another developer to pair program to get that - essentially you set up a clean computer with their editor on it and your testing/interview environment, and they can hack away while you watch. But yes, if there's some way to get a candidate to pair program on actual, real code with real developers, it's invaluable. Just make sure they sign a contract stating they are giving you the rights to the code, and that they are adequately compensated for their time at some reasonable hourly rate.
fishtoaster 14y ago
I see a little value in "Fermi problems" like ping pongs in a VW bus. I use one like it on a candidate's second call (first call with an engineer; takes place after our on-site recruiter has screened tehem). The point is to see how they handle a relatively intractable problem. The only real wrong answer is either a refusal to come up with an answer or an off-the-cuff guess with no explanation. If they at least try to extrapolate an answer from reasonable assumptions, that's fine. To your process, my thoughts: - Open-book challenges like you describe (in which you can use stack overflow and whatnot) are awesome. However, why do you have them do stuff they're not expected to know? From what I've read, the only thing hiring studies seem to agree on is that the questions which best predict a candidate's long-term success are those that simulate what the candidate will actually be working on. Getting them to learn outside their comfort zone is good, but maybe you'd be better off doing that by finding problems you've actually had in the course of your work that are outside of their comfort zone. - Short-term contracts are tricky. On one hand, I think they're the best way to test how well a candidate will work out. On the other hand, switching jobs is a lot of work and stress. If I had a choice between two pretty good companies, one of which was a 1-month contract and one of which was a normal job, I'd definitely take the latter. If I had to move to get your job, it'd make rejecting such a proposal even more of a no-brainer. If it works out for you, that's great, but I'd be really worried about losing excellent candidates. - It seems like your remote screening is lighter than what I'm used to. At my current gig, we send candidates a challenge to be completed in their own time (3-9 hours of work)- we've found it's a great prelude to the phone screen with an engineer. It's pretty much a code sample for which we have an excellent reference (all the previous submissions). We've found it improves the quality of candidates that get to our on-site.
- MattRogish 14y ago
  Re: open book, what I meant was "not expected to know" means something that they can't do blindfolded. For a Rails person making a contrived app from scratch doesn't tell me anything other than they could type `rails foo` and run a few generators. What I mean is that, I put together a few challenges with APIs we work with (or gems we've written, that sort of thing) and have them solve a problem that will require them to use their innate ability, drive, and Google/StackOverflow/etc. to solve the problem. It's not something that they would be able to do without the internet. Does that make sense? The contract thing is tough, although at least in NYC the market is so tight that there's really no risk of unemployment, but yeah, we wouldn't do relo + contract. We'd start them off as remote and see if that worked. If so, then relo; it's risky risky for both parties (especially in NYC) that it's kinda crazy to expect either party to agree to it. The candidate has no idea what NYC is like, and the organization may be reluctant to fire a bad hire who just isn't fitting in because they feel bad. At least with a remote contract, no one has to move and we can see if we both like each other enough to get married. Then would come the offer + relo., or if it's not a good fit, end the contract (with 2 weeks notice at least). Finally, I'd say the initial screen is a bit light. We used to have about a 1-2 hr. technical Q&A followed by Fizzbuzz-style questions, but that had a low false-positive rate so we got rid of it. That is to say, the background work I did (googling them, stack overflow, github, twitter, their resume, etc.) was a pretty good filter at keeping out the obviously underqualified. However, I'm considering putting it back as it helped give me a more nuanced picture of the candidates that I end up doing as part of the contract, and it might help to have some first-impressions ahead of time (but I am always worried about false first impressions clouding my judgement).

finnw 14y ago

Are you hiring now? :-)

MattRogish 14y ago
Sure: https://www.fundinggates.com/jobs/ https://www.fundinggates.com/jobs/ :D
- 31reasons 14y ago
  Interesting name of the company. Whats the story behind it ?

IvyMike 14y ago

> "How many ping-pong balls can fit in a VW bus" is just about the worst question I've ever gotten. The point of questions like this is "Does this guy have any common sense?" Some of my least favorite hires were people who were whip-sharp at the details of their chosen programming language, but then when hired, would go off and do crazy and impractical stuff with it. Whether or not the question accomplishes a good evaluation of someone's common sense is up for debate, but unless you're hiring drones, you do have to ask some questions that get a broader picture of the way the person thinks. (BTW, your "pair programming for the rest of the day" is a great way of getting that. But pair programming doesn't/wouldn't work at any place I've ever been, for a variety of reasons.)

columbo 14y ago
> The point of questions like this is "Does this guy have any common sense?" GROOOOOAAAAN Maybe twenty years ago, and maybe if you were the individual to think up the question. Honestly I'm so goddamned bored with these unoriginal blither-blather questions I consider them a strike against the interviewer and the company.
- IvyMike 14y ago
  I actually have never asked an estimation question, but I understand the motivation for asking this kind of question. I'm curious--what do you ask in interviews to get at a candidate's common sense? It's not very easy to do. Or do you think you don't need to assess common sense at all? Maybe it's not, but like I said, a lot of the hires I later regretted came down to something like that.
  - columbo 14y ago
    > Or do you think you don't need to assess common sense at all? shrug If we're talking developers... The ability to drive to the location on time, answer questions without going blank are good indicators of common sense for me. Extracurricular activities, anything from volunteering at an animal shelter, to making civil war models to home brewing beer, are huge in my book because they are a good indicator of a grounded individual. That mixed with some pair-programming should give more than enough information. YMMV.
MattRogish 14y ago
I think the "real world, open-book" problems have to be crafted well, but I think they give a good sense of "Common sense". They are set up that you can absolutely prematurely optimize or go down really crazy paths (and I've seen them happen). You don't need another developer to pair program to get that - essentially you set up a clean computer with their editor on it and your testing/interview environment, and they can hack away while you watch. But yes, if there's some way to get a candidate to pair program on actual, real code with real developers, it's invaluable. Just make sure they sign a contract stating they are giving you the rights to the code, and that they are adequately compensated for their time at some reasonable hourly rate.

fishtoaster 14y ago

I see a little value in "Fermi problems" like ping pongs in a VW bus. I use one like it on a candidate's second call (first call with an engineer; takes place after our on-site recruiter has screened tehem). The point is to see how they handle a relatively intractable problem. The only real wrong answer is either a refusal to come up with an answer or an off-the-cuff guess with no explanation. If they at least try to extrapolate an answer from reasonable assumptions, that's fine. To your process, my thoughts: - Open-book challenges like you describe (in which you can use stack overflow and whatnot) are awesome. However, why do you have them do stuff they're not expected to know? From what I've read, the only thing hiring studies seem to agree on is that the questions which best predict a candidate's long-term success are those that simulate what the candidate will actually be working on. Getting them to learn outside their comfort zone is good, but maybe you'd be better off doing that by finding problems you've actually had in the course of your work that are outside of their comfort zone. - Short-term contracts are tricky. On one hand, I think they're the best way to test how well a candidate will work out. On the other hand, switching jobs is a lot of work and stress. If I had a choice between two pretty good companies, one of which was a 1-month contract and one of which was a normal job, I'd definitely take the latter. If I had to move to get your job, it'd make rejecting such a proposal even more of a no-brainer. If it works out for you, that's great, but I'd be really worried about losing excellent candidates. - It seems like your remote screening is lighter than what I'm used to. At my current gig, we send candidates a challenge to be completed in their own time (3-9 hours of work)- we've found it's a great prelude to the phone screen with an engineer. It's pretty much a code sample for which we have an excellent reference (all the previous submissions). We've found it improves the quality of candidates that get to our on-site.

MattRogish 14y ago
Re: open book, what I meant was "not expected to know" means something that they can't do blindfolded. For a Rails person making a contrived app from scratch doesn't tell me anything other than they could type `rails foo` and run a few generators. What I mean is that, I put together a few challenges with APIs we work with (or gems we've written, that sort of thing) and have them solve a problem that will require them to use their innate ability, drive, and Google/StackOverflow/etc. to solve the problem. It's not something that they would be able to do without the internet. Does that make sense? The contract thing is tough, although at least in NYC the market is so tight that there's really no risk of unemployment, but yeah, we wouldn't do relo + contract. We'd start them off as remote and see if that worked. If so, then relo; it's risky risky for both parties (especially in NYC) that it's kinda crazy to expect either party to agree to it. The candidate has no idea what NYC is like, and the organization may be reluctant to fire a bad hire who just isn't fitting in because they feel bad. At least with a remote contract, no one has to move and we can see if we both like each other enough to get married. Then would come the offer + relo., or if it's not a good fit, end the contract (with 2 weeks notice at least). Finally, I'd say the initial screen is a bit light. We used to have about a 1-2 hr. technical Q&A followed by Fizzbuzz-style questions, but that had a low false-positive rate so we got rid of it. That is to say, the background work I did (googling them, stack overflow, github, twitter, their resume, etc.) was a pretty good filter at keeping out the obviously underqualified. However, I'm considering putting it back as it helped give me a more nuanced picture of the candidates that I end up doing as part of the contract, and it might help to have some first-impressions ahead of time (but I am always worried about false first impressions clouding my judgement).