8 ms·

Whiteboard exercises are a terrible way to investigate coding ability (2012)

83 points by tiger_dioxide 7y ago

codingslave 7y ago
Being good at coding can be predicted by IQ. Being good at white boarding puzzles require IQ. -----> White board puzzles filter for good coders. Its not perfect, but its the best way to scale it. Sorry everyone, this isn't going away
- thaumasiotes 7y ago
  > Its not perfect, but its the best way to scale it. IQ tests scale a lot better than one-to-one whiteboard interviews do. They are specifically designed for that purpose. How is an expensive, labor-intensive, never-validated approximation of an IQ test "better scaled" than a cheap, easy-to-administer, thoroughly validated IQ test?
  - codingslave 7y ago
    Right, but IQ tests are illegal.
    - thaumasiotes 7y ago
      There are legal barriers, but no, they're not illegal. GE uses one. The NFL uses one. McKinsey uses one.
  - jldugger 7y ago
    Standard IQ tests are basically offlimits in the US as they feature a well known, well studied disparate impact. But I think your correspondent is underselling work sample tests. They are not perfectly correlated with IQ testing, and do in fact measure something beyond IQ; on it's own work sample tests are only slightly worse than IQ testing, and combining IQ testing and work sample testing improves predictions of hiring outcomes vs just IQ testing. That said, very few workplaces have well calibrated whiteboard interviews, or take home interviews, or pair programming screenings.
marcosdumay 7y ago
Programming on a projector is the stuff nightmares are made of... but it still beats a whiteboard. Anyway, I don't think those boards are there to investigate ability. They are basically useless for that. If they have a rational motive at all, it's probably because the most strict schools teach the "assemble everything in your head, then write it" method, so they accidentally select people with good credentials.
- mjevans 7y ago
  How about: bring a laptop with an HDMI port (or adapter) and a dev environment for "whiteboard" and "napkin" problems. I'd be OK with an Interview like that, just don't laugh about the fact that my laptop is a still functional but otherwise aging smart terminal for securely connecting to my real workstations when I'm somewhere I don't trust.
  - marcosdumay 7y ago
    It's not the setup that is bad, it's how there are a lot of people focused on your code, noticing your typos before you do. It's bad enough when you are in a position of power like in a presentation or a class. It's nearly unthinkable when you are in a position of weakness.
- decebalus1 7y ago
  > Programming on a projector is the stuff nightmares are made of... but it still beats a whiteboard. It depends. I recently had an onsite interview where I was handed a laptop to write code on, which was great. What was not great was that the interviewer was shadowed by 2 other developers and had to write the code while screensharing to the meeting room's screen. It was basically a meeting where people were looking at me go. It was by far the weirdest experience ever (and I've been doing this for decades). I kind of missed the 1:1 with the interviewer and a whiteboard.
Areading314 7y ago
I don't see the problem with whiteboard as long as its emphasized that you are looking for an algorithmic solution, not a perfectly running code snippet in a specific language. Whiteboard is great for sketching out ideas and solving problems collaboratively, which is a good indicator of team fit.
- grecy 7y ago
  Exactly right. I think a lot of people stress that a whiteboard exercise is not a great way to ascertain coding skill.... and they're right. The part they're missing is the whiteboard exercise isn't intended to only test coding skill. It's about how well you ask questions, how well you listen to instructions, how well you work with others and interpret the specs and information you're given. Then it's about how well you adapt to changing or incomplete requirements, and how well you can explain yourself and your thought process. Like so many tests in life it's not about the "answer", it's about the process you used to get there.
  - Aeolun 7y ago
    The thing is, I can do all of those things very well when I’m not standing in front of a whiteboard. When someone asks me to do then on a whiteboard in an interview setting they might as well dismiss me immediately.
    - Ididntdothis 7y ago
      When I interview I usually offer a whiteboard or paper and pen. Or they can talk through the problem. I just want to see that somebody is capable of understanding a problem and has a way to work towards a solution. Maybe I should offer a text editor too in the future.
  - pmiller2 7y ago
    In practice, facing down some 25 year old who’s been at the company for all of 9 months, it is all about “the answer.” In some cases, it’s about 3 answers. Often, it’s actually about their answer — you must do it their way, otherwise you are wrong. This is not the case 100% of the time, but it is often enough that if you’re interviewing in SF, particularly at startups, you will see it at least every few interviews. It’s ridiculous.
maerF0x0 7y ago
White/Chalk board coding makes sense in a computational world that uses punch cards and has a 24 hour turnaround on output.
threatofrain 7y ago
The article says that whiteboard tests are about whiteboard tests. Ultimately, pen and paper tests are about pen and paper tests. Take-home tests are about take-home tests. Algorithm tests are about algorithm tests, and chess tests are about chess tests. But among these, I would argue that the whiteboard is better at facilitating general communication. If you have anything to say about anything, whiteboards are the most flexible way to say it. What ends up on the whiteboard doesn't have to be some toy problem, it can be about anything. It's probably one of the most flexible programming languages in the world, because you use a combination of words, arrows, and drawings to communicate however you want, and your ideas don't even have to make sense.
- nsxwolf 7y ago
  It's great if that's how you're being scored, but you never know how you're being scored. One interviewer might be happy with a conceptual approach, another one might reject you for leaving off a curly brace.
  - deleted 7y ago
    [deleted]
  - bootlooped 7y ago
    In my experience some interviewers were not happy that I worked on a brute force solution first, with the aim to optimize it after I got it working. They wanted to see the efficient, optimized solution first. That is counter to everything I read before doing these interviews, and it was, I think, a bit unreasonable.
    - Apocryphon 7y ago
      As much of a cesspool of anonymous gossip as places like Glassdoor might be, reporting this sort of bad behavior might be useful, if to at least force companies that engage in unreasonable behavior to grapple with these poor policies. Or to warn other candidates of what they’re in for.
- pauljurczak 7y ago
seanmcdirmid 7y ago
Try doing whiteboard exercises over video conferencing for even more fun.
- adamredwoods 7y ago
  Amazon does this. My experience was terrible. I don't mind whiteboarding if they don't care about specifics, and are open to talking about process and ideas, versus strict syntax.
  - seanmcdirmid 7y ago
    It can work if both participants have styluses with their tablets (iPad Pro is best) and they use something like google Jamboard. For coding exercises, use alongside code road running on your laptop.
christiansakai 7y ago
I'm an anomaly. I like whiteboard algorithm tests. I do still think it is kinda a waste of time doing Leetcode over and over like what I'm doing right now, but hear me out. An even more waste of time is actually learning framework A, B, C, D, database A, B, C, D, bundler A, B, C, D. The more and more I think about it, as a fullstack developer, I continuously forget documentations and how a specific technology works, but I was always able to solve problems using those in the past. I did Angular, Vue, jQuery, React, Webpack 1, 2, 3, 4, Grunt, Gulp. I did MongoDB, MySQL, S3, DynamoDB, Go, Python. Those things change all the time and I always forget how they worked. Because I genuinely like fullstack development I always switch back and forth between different technologies and I can't remember them all. Whenever a company interview me, I'd really appreciate it if they just ask whiteboard algorithm questions instead, at the very least I don't have to scramble brushing up X if I got interviewed as an X developer. At least whiteboard algorithm and data structures questions make me stronger fundamentally as an all around developer and I just need to grind those. And after a while, grinding those aren't that difficult anymore. P.S. I still failed FAANG interview, but I still keep grinding Leetcode. I am definitely a believer in whiteboard algorithm and data structure questions. It is simple, to the point, saves everyone time. If I were an interviewer I would also ask data structures and algorithm questions instead of nitty bitty of React hooks or Javascript promises or Golang's concurrency pattern. Those things can be learned easily if you are a decent all around developer. I'm an average developer. I don't have the creative capability or the insights that 10x programmer has. Those people are not only good with programming, but also at mentoring, at business, at finance, at swimming, at hunting bears, at predicting the future, etc. So, in my resume I often don't have those quantitative accomplishment "implement X in Y minutes that resulted in Z performance and A savings for the company". So, whiteboard algorithm questions fits me better.
- BlargMcLarg 7y ago
  I had this problem with switching contexts. For work, we use a combination of old Java, Tomcat, JS and the likes. For my hobby, I mainly program games in C#. Two entirely different infrastructures. That said, I can keep both up decently well due to the sheer volume of practice. A few months ago I had my first go at ASP.NET Core, specifically Razor. Hadn't touched it for months, now I'm trying it again and I already forget intricacies of the syntax (when to put curly braces etc.), all stuff you only really remember through continued practice. Most of all: its just knowledge you can find on the internet. The case isn't unique enough to not have a creative solution or find a source on. In fact, the information is so easy to find, its almost a waste of brain space to remember rather than figuring out the pattern to uncover the solution next time. I also find webdev to be more and more complex by the year. Not because of the frameworks introduced, but due to the increasing demands from users, the competition and security issues, while there's still very little to streamline the process for the developer. It takes a lot of steps to get to the point where your site is online, with certified security, connected to a domain instead of an IPaddress, coming as someone who primarily did desktop development.
LordOfWolves 7y ago
I organically discovered this many years ago when attempting to learn my first recursive algorithm by writing out its final result step-by-step on a whiteboard. Definitely went through more than one marker.
dntbnmpls 7y ago
Whiteboard exercises are there to show how well you understood topics, how well you can think on your feet and how well you can communicate. Most whiteboard exercises generally deal with pseudocode. Haven't seen a whiteboard that can compile and run the code you write on it to test your code or your coding ability.
mrkeen 7y ago
I like doing real work on a whiteboard. Especially if there's two or more devs involved. It's a great way to see the whole problem at once, and it's so much more productive than all sitting at a keyboard together. So a whiteboard is a pretty natural medium for interviewing for me.
Veen 7y ago
Surely this can be tested fairly easily by seeing how well whiteboard test results correlate with coding ability or developer effectiveness more generally. I'd assume businesses that hire a lot of developers have the data to know whether whiteboard tests are useful for achieving their hiring goals.
cosmiccatnap 7y ago
I'm a much bigger fan of telling someone they have 24h to build me a toy deliverable to test how well they can work under pressure and scrape code together. That gives you a better understanding of how well they can do that on the spot just make sure to have then walk through the code the next day so you know they actually wrote it

83 points by tiger_dioxide 7y ago

Being good at coding can be predicted by IQ. Being good at white boarding puzzles require IQ. -----> White board puzzles filter for good coders. Its not perfect, but its the best way to scale it. Sorry everyone, this isn't going away

thaumasiotes 7y ago
> Its not perfect, but its the best way to scale it. IQ tests scale a lot better than one-to-one whiteboard interviews do. They are specifically designed for that purpose. How is an expensive, labor-intensive, never-validated approximation of an IQ test "better scaled" than a cheap, easy-to-administer, thoroughly validated IQ test?
- codingslave 7y ago
  Right, but IQ tests are illegal.
  - thaumasiotes 7y ago
    There are legal barriers, but no, they're not illegal. GE uses one. The NFL uses one. McKinsey uses one.
- jldugger 7y ago
  Standard IQ tests are basically offlimits in the US as they feature a well known, well studied disparate impact. But I think your correspondent is underselling work sample tests. They are not perfectly correlated with IQ testing, and do in fact measure something beyond IQ; on it's own work sample tests are only slightly worse than IQ testing, and combining IQ testing and work sample testing improves predictions of hiring outcomes vs just IQ testing. That said, very few workplaces have well calibrated whiteboard interviews, or take home interviews, or pair programming screenings.

marcosdumay 7y ago

Programming on a projector is the stuff nightmares are made of... but it still beats a whiteboard. Anyway, I don't think those boards are there to investigate ability. They are basically useless for that. If they have a rational motive at all, it's probably because the most strict schools teach the "assemble everything in your head, then write it" method, so they accidentally select people with good credentials.

mjevans 7y ago
How about: bring a laptop with an HDMI port (or adapter) and a dev environment for "whiteboard" and "napkin" problems. I'd be OK with an Interview like that, just don't laugh about the fact that my laptop is a still functional but otherwise aging smart terminal for securely connecting to my real workstations when I'm somewhere I don't trust.
- marcosdumay 7y ago
  It's not the setup that is bad, it's how there are a lot of people focused on your code, noticing your typos before you do. It's bad enough when you are in a position of power like in a presentation or a class. It's nearly unthinkable when you are in a position of weakness.
decebalus1 7y ago
> Programming on a projector is the stuff nightmares are made of... but it still beats a whiteboard. It depends. I recently had an onsite interview where I was handed a laptop to write code on, which was great. What was not great was that the interviewer was shadowed by 2 other developers and had to write the code while screensharing to the meeting room's screen. It was basically a meeting where people were looking at me go. It was by far the weirdest experience ever (and I've been doing this for decades). I kind of missed the 1:1 with the interviewer and a whiteboard.

Areading314 7y ago

I don't see the problem with whiteboard as long as its emphasized that you are looking for an algorithmic solution, not a perfectly running code snippet in a specific language. Whiteboard is great for sketching out ideas and solving problems collaboratively, which is a good indicator of team fit.

grecy 7y ago
Exactly right. I think a lot of people stress that a whiteboard exercise is not a great way to ascertain coding skill.... and they're right. The part they're missing is the whiteboard exercise isn't intended to only test coding skill. It's about how well you ask questions, how well you listen to instructions, how well you work with others and interpret the specs and information you're given. Then it's about how well you adapt to changing or incomplete requirements, and how well you can explain yourself and your thought process. Like so many tests in life it's not about the "answer", it's about the process you used to get there.
- Aeolun 7y ago
  The thing is, I can do all of those things very well when I’m not standing in front of a whiteboard. When someone asks me to do then on a whiteboard in an interview setting they might as well dismiss me immediately.
  - Ididntdothis 7y ago
    When I interview I usually offer a whiteboard or paper and pen. Or they can talk through the problem. I just want to see that somebody is capable of understanding a problem and has a way to work towards a solution. Maybe I should offer a text editor too in the future.
- pmiller2 7y ago
  In practice, facing down some 25 year old who’s been at the company for all of 9 months, it is all about “the answer.” In some cases, it’s about 3 answers. Often, it’s actually about their answer — you must do it their way, otherwise you are wrong. This is not the case 100% of the time, but it is often enough that if you’re interviewing in SF, particularly at startups, you will see it at least every few interviews. It’s ridiculous.

maerF0x0 7y ago

White/Chalk board coding makes sense in a computational world that uses punch cards and has a 24 hour turnaround on output.

threatofrain 7y ago

The article says that whiteboard tests are about whiteboard tests. Ultimately, pen and paper tests are about pen and paper tests. Take-home tests are about take-home tests. Algorithm tests are about algorithm tests, and chess tests are about chess tests. But among these, I would argue that the whiteboard is better at facilitating general communication. If you have anything to say about anything, whiteboards are the most flexible way to say it. What ends up on the whiteboard doesn't have to be some toy problem, it can be about anything. It's probably one of the most flexible programming languages in the world, because you use a combination of words, arrows, and drawings to communicate however you want, and your ideas don't even have to make sense.

nsxwolf 7y ago
It's great if that's how you're being scored, but you never know how you're being scored. One interviewer might be happy with a conceptual approach, another one might reject you for leaving off a curly brace.
- deleted 7y ago
  [deleted]
- bootlooped 7y ago
  In my experience some interviewers were not happy that I worked on a brute force solution first, with the aim to optimize it after I got it working. They wanted to see the efficient, optimized solution first. That is counter to everything I read before doing these interviews, and it was, I think, a bit unreasonable.
  - Apocryphon 7y ago
    As much of a cesspool of anonymous gossip as places like Glassdoor might be, reporting this sort of bad behavior might be useful, if to at least force companies that engage in unreasonable behavior to grapple with these poor policies. Or to warn other candidates of what they’re in for.
pauljurczak 7y ago

seanmcdirmid 7y ago

Try doing whiteboard exercises over video conferencing for even more fun.

adamredwoods 7y ago
Amazon does this. My experience was terrible. I don't mind whiteboarding if they don't care about specifics, and are open to talking about process and ideas, versus strict syntax.
- seanmcdirmid 7y ago
  It can work if both participants have styluses with their tablets (iPad Pro is best) and they use something like google Jamboard. For coding exercises, use alongside code road running on your laptop.

christiansakai 7y ago

I'm an anomaly. I like whiteboard algorithm tests. I do still think it is kinda a waste of time doing Leetcode over and over like what I'm doing right now, but hear me out. An even more waste of time is actually learning framework A, B, C, D, database A, B, C, D, bundler A, B, C, D. The more and more I think about it, as a fullstack developer, I continuously forget documentations and how a specific technology works, but I was always able to solve problems using those in the past. I did Angular, Vue, jQuery, React, Webpack 1, 2, 3, 4, Grunt, Gulp. I did MongoDB, MySQL, S3, DynamoDB, Go, Python. Those things change all the time and I always forget how they worked. Because I genuinely like fullstack development I always switch back and forth between different technologies and I can't remember them all. Whenever a company interview me, I'd really appreciate it if they just ask whiteboard algorithm questions instead, at the very least I don't have to scramble brushing up X if I got interviewed as an X developer. At least whiteboard algorithm and data structures questions make me stronger fundamentally as an all around developer and I just need to grind those. And after a while, grinding those aren't that difficult anymore. P.S. I still failed FAANG interview, but I still keep grinding Leetcode. I am definitely a believer in whiteboard algorithm and data structure questions. It is simple, to the point, saves everyone time. If I were an interviewer I would also ask data structures and algorithm questions instead of nitty bitty of React hooks or Javascript promises or Golang's concurrency pattern. Those things can be learned easily if you are a decent all around developer. I'm an average developer. I don't have the creative capability or the insights that 10x programmer has. Those people are not only good with programming, but also at mentoring, at business, at finance, at swimming, at hunting bears, at predicting the future, etc. So, in my resume I often don't have those quantitative accomplishment "implement X in Y minutes that resulted in Z performance and A savings for the company". So, whiteboard algorithm questions fits me better.

BlargMcLarg 7y ago
I had this problem with switching contexts. For work, we use a combination of old Java, Tomcat, JS and the likes. For my hobby, I mainly program games in C#. Two entirely different infrastructures. That said, I can keep both up decently well due to the sheer volume of practice. A few months ago I had my first go at ASP.NET Core, specifically Razor. Hadn't touched it for months, now I'm trying it again and I already forget intricacies of the syntax (when to put curly braces etc.), all stuff you only really remember through continued practice. Most of all: its just knowledge you can find on the internet. The case isn't unique enough to not have a creative solution or find a source on. In fact, the information is so easy to find, its almost a waste of brain space to remember rather than figuring out the pattern to uncover the solution next time. I also find webdev to be more and more complex by the year. Not because of the frameworks introduced, but due to the increasing demands from users, the competition and security issues, while there's still very little to streamline the process for the developer. It takes a lot of steps to get to the point where your site is online, with certified security, connected to a domain instead of an IPaddress, coming as someone who primarily did desktop development.

LordOfWolves 7y ago

I organically discovered this many years ago when attempting to learn my first recursive algorithm by writing out its final result step-by-step on a whiteboard. Definitely went through more than one marker.

dntbnmpls 7y ago

Whiteboard exercises are there to show how well you understood topics, how well you can think on your feet and how well you can communicate. Most whiteboard exercises generally deal with pseudocode. Haven't seen a whiteboard that can compile and run the code you write on it to test your code or your coding ability.

mrkeen 7y ago

I like doing real work on a whiteboard. Especially if there's two or more devs involved. It's a great way to see the whole problem at once, and it's so much more productive than all sitting at a keyboard together. So a whiteboard is a pretty natural medium for interviewing for me.

Veen 7y ago

Surely this can be tested fairly easily by seeing how well whiteboard test results correlate with coding ability or developer effectiveness more generally. I'd assume businesses that hire a lot of developers have the data to know whether whiteboard tests are useful for achieving their hiring goals.

cosmiccatnap 7y ago

I'm a much bigger fan of telling someone they have 24h to build me a toy deliverable to test how well they can work under pressure and scrape code together. That gives you a better understanding of how well they can do that on the spot just make sure to have then walk through the code the next day so you know they actually wrote it