5 ms·

Thank you for writing this post. It was informative. A few comments from a fellow software developer who is approaching 50... I don't think the coding test isn

by geebee 8y ago

Thank you for writing this post. It was informative. A few comments from a fellow software developer who is approaching 50... I don't think the coding test isn't there because people think you're lying, it's there because we have no industry wide, respected entrance exam. Actuarial interviews don't (to my knowledge) contain a whiteboard vector calculus exam, but this isn't because people just sort of believe actuaries, it's because you have to pass a rigorous exam on these topics to become an actuary. I sometimes wish we had something like this in our field (not necessarily a legal requirement, just something that is widely recognized as legit). I'd be happy to put the time in and really study for a proper exam, but I'm not as interested in doing this over and over for companies that may administer and conduct these exams frivolously and in secret. Also - I actually do agree that it's bad if someone can't write fizz buzz, but every interview exam I've taken has been vastly more difficult than fizz buzz. It's "find all subsets of a set of integers that is divisible by the sum of a different subset of integers". At a whiteboard, in 45 minutes. These aren't impossible problems, but they're much tougher than fizzbuzz. I am saddened, angry, and relieved to hear of your problems with homework. The response you got back makes me realize that the two times I've done a homework assignment were probably pretty typical. I won't do this anymore, and I do accept that this will limit my career options. It does anger me, though, when the companies who do this complain about a shortage of engineers. True story: I did an on line exam and take home homework assignment, waited over a month (crickets chirping) to hear back, pinged the recruiter politely now and then, and finally got a one liner "we've decided not to continue at this time.' A few weeks later, an article with came out with a picture of the CEO of this company standing next to President Obama, who nodded gravely as the CEO talked about the desperate shortage of software developers.

octorian 8y ago
> ...who nodded gravely as the CEO talked about the desperate shortage of software developers. This comment is exactly why reading about "interview gymnastics" like this really pisses me off. Its also why I wonder how many big tech companies are somehow able to hire so many people... With processes like these, you'd think that they either have a "secret backdoor" or never managed to actually hire more than a dozen or so.
- s2g 8y ago
  Volume. Huge volume. Google has a completely broken interview process, to the point where they pretty much brag about how ridiculous their false negative rate is, but they make up for it on volume. When people will throw themselves at you two or three times in hope of getting that lucky set of algorithm wankery questions that they can answer, then you will keep interviewing like that.
  - d1zzy 8y ago
    How do you define broken? It definitely sounds like it's working for Google so going by the "is the process working for the employer" definition it isn't broken.
    - talmand 8y ago
      >> in hope of getting that lucky set of algorithm wankery questions that they can answer If true, hiring based on luck seems broken to me. Like the joke of the hiring manager throwing away half the resumes and not hiring "unlucky" people.
  - munchbunny 8y ago
    I'm not so certain that Google's process is more broken than anyone else's. The thing the giant companies have going for them is that they tend to be more consistent, so at least you can get a good sense of what they'll ask, how to prepare, and how they're measuring you. With small companies, it often comes down to "were you charming enough?" They won't admit it, but it's a clear subconscious bias.
  - ljw1001 8y ago
    Actually, I suspect Google makes up for it on acquisitions. When I was there that's how most people got into the company.
- mixmastamyk 8y ago
  Definitely a back door, friends on the inside to bypass the BS.
s2g 8y ago
> it's there because we have no industry wide, respected entrance exam So you think the person with 30 years experience might be, whats the word... oh right. Lying. You think they are lying. Quit trying to dress it up. Coding tests are exactly you saying you don't believe the experience written on the resume.
- zip1234 8y ago
  I use a coding test not only to see if someone can write basic code, but also to assess general intelligence and see/hear their problem solving process. A resume definitely doesn't tell you all you need to know and experience does not necessarily mean effectiveness.
  - geebee 8y ago
    I actually do understand why people administer these tests in our field. There's so little to guide you otherwise. The problem is that it's kind of awful, and I personally really do believe that it's driving people away from the field. Here's my take (I've posted this on HN a few times). Many people find in person, at the whiteboard exams quite stressful. This is fairly common, many people describe exams to be among the more stressful events in their lives. Over time, I believe that institutions that conduct exams evolved a bill or rights between institution and applicant. Exams are used, but they are administered consistently and fairly, the topic and subject matter will not be a huge surprise (questions are kept secret of course, but their nature should be predictable). People who pass or fail will receive an answer, and often get a score and feedback to know how they did. The people who grade the exams are provided training to ensure they evaluate people fairly, consistently, and without bias, and are expert in their field. Now, I know that universities, medical boards, bar exam reviewers, don't always live up to this, but it is the intended goal. I believe that our field, software, subjects people to exams repeatedly, but without any of the bill or rights I descried above. For example, how do you grade? Are you qualified to assess general intelligence? Can you understand that someone might not be eager to submit to your general assessment of their intelligence? Now, if you want to say nobody is entitled to a job, I certainly agree. But the industry seems to think it's entitled to workers, and they just seem utterly blind to how off putting their practices are. And in an industry with concerns about gender, racial, and age discrimination, double secret assessments of people's general intelligence by interviewers of unknown qualifications is about the worst thing you can do.
mixmastamyk 8y ago
Last paragraph is a hidden gem, should be pinned at the top of the page. Perhaps a blog post? :D
- liaocs2008 8y ago
  Agree. The true story is actually sad.
- muddyrivers 8y ago
  I agree. On the other hand, we engineers need to look at ourselves in the mirror. After all, we engineers conduct the technical interview. I was involved in hundreds of tech interviews in the past several years. So many times in discussing a candidate's performance, I found some of the interviewers' questions and comments were unrealistic, even preposterous. In one case, an interviewer gave a hard dynamic programming question. We were a tech company and had very good engineers. Our products and our technologies, however, rarely need dynamic programming skills of this difficult level. I couldn't bear it. So I asked, "who think he can solve this problem on the whiteboard in 40 minutes right now?" Dead silence.
solatic 8y ago
> ... because we have no industry wide, respected entrance exam. This. In fact there needs to be one exam, with subsections and subscores per subsection, along with an overall score. Being able to solve an algorithmic challenge on a whiteboard does not mean that you can: a) write readable and maintainable code b) effectively communicate requirements to whoever is actually running your code in production c) know about application-level security vulnerabilities like SQL injections and know not to mindlessly insert them into your code d) instrument your code for metrics and logging in ways that are more or less standard and expected for production monitoring e) engage in effective review of others' code f) use typical API handling techniques which are production-ready, e.g. exponential backoff and circuit breakers. All of which are testable in a certifiable way. Simple truth of the matter is, the vast majority of companies that hire 10 developers who studied efficient algorithms for their interviews, are better served by hiring 9 engineers who are aware of how production and organizational realities affect their work, plus one "performance hacker" who may or may not have a master's degree in CS, whose job is to find the worst real-world production bottlenecks and replace the implementations with algorithmically more efficient implementations (plus stricter performance tests so that the more efficient code doesn't get ripped out later).
- irundebian 8y ago
  That's an interesting conclusion and it makes pretty sense to me. I've heard that most projects fail due to organizational failures rather than development performance failures. It's probably better to invest in process/quality-aware developers than in rockstar programmers. But I've never read studies about that.
- aeorgnoieang 8y ago
  How would you test, "in a certifiable way", just the first of your examples "write readable and maintainable code"?
  - solatic 8y ago
    The SAT has a writing section - there is no reason you can't write code on a standardized test. The idea is to assign low points to people who write a long function, and high points to people who write many small, self-documenting functions. Such an exam could also ask questions like "what is the cyclometric complexity of the following code?", which while not an indicator that the test taker will always write low-complexity code in a professional environment, at least indicates that the test taker is aware of maintainability metrics like cyclometric complexity - much more than can be said of programmers who mindlessly turn in 400 line functions with deeply nested conditionals.
spirodonfl 8y ago
I like the idea of a proper and respected exam of some kind to test you on a variety of skills that are actually valuable in general and in specific. I've attempted to gather data from colleagues around me but the data is limited. Posting online doesn't get much attention either. But I am trying to gather enough data to see if it would even be possible to create such an exam. If a bunch of developers took a few minutes of their time to answer questions regarding what kinds of things should go into an exam and why (or heck, even a better teaching curriculum than what we have now) I think we could easily come up with something pretty awesome. But, again, gathering that kind of attention and asking developers to provide you with insight is hard. I'd love to tackle it if others were willing to help.

by geebee 8y ago

octorian 8y ago

> ...who nodded gravely as the CEO talked about the desperate shortage of software developers. This comment is exactly why reading about "interview gymnastics" like this really pisses me off. Its also why I wonder how many big tech companies are somehow able to hire so many people... With processes like these, you'd think that they either have a "secret backdoor" or never managed to actually hire more than a dozen or so.

s2g 8y ago
Volume. Huge volume. Google has a completely broken interview process, to the point where they pretty much brag about how ridiculous their false negative rate is, but they make up for it on volume. When people will throw themselves at you two or three times in hope of getting that lucky set of algorithm wankery questions that they can answer, then you will keep interviewing like that.
- d1zzy 8y ago
  How do you define broken? It definitely sounds like it's working for Google so going by the "is the process working for the employer" definition it isn't broken.
  - talmand 8y ago
    >> in hope of getting that lucky set of algorithm wankery questions that they can answer If true, hiring based on luck seems broken to me. Like the joke of the hiring manager throwing away half the resumes and not hiring "unlucky" people.
- munchbunny 8y ago
  I'm not so certain that Google's process is more broken than anyone else's. The thing the giant companies have going for them is that they tend to be more consistent, so at least you can get a good sense of what they'll ask, how to prepare, and how they're measuring you. With small companies, it often comes down to "were you charming enough?" They won't admit it, but it's a clear subconscious bias.
- ljw1001 8y ago
  Actually, I suspect Google makes up for it on acquisitions. When I was there that's how most people got into the company.
mixmastamyk 8y ago
Definitely a back door, friends on the inside to bypass the BS.

s2g 8y ago

> it's there because we have no industry wide, respected entrance exam So you think the person with 30 years experience might be, whats the word... oh right. Lying. You think they are lying. Quit trying to dress it up. Coding tests are exactly you saying you don't believe the experience written on the resume.

zip1234 8y ago
I use a coding test not only to see if someone can write basic code, but also to assess general intelligence and see/hear their problem solving process. A resume definitely doesn't tell you all you need to know and experience does not necessarily mean effectiveness.
- geebee 8y ago
  I actually do understand why people administer these tests in our field. There's so little to guide you otherwise. The problem is that it's kind of awful, and I personally really do believe that it's driving people away from the field. Here's my take (I've posted this on HN a few times). Many people find in person, at the whiteboard exams quite stressful. This is fairly common, many people describe exams to be among the more stressful events in their lives. Over time, I believe that institutions that conduct exams evolved a bill or rights between institution and applicant. Exams are used, but they are administered consistently and fairly, the topic and subject matter will not be a huge surprise (questions are kept secret of course, but their nature should be predictable). People who pass or fail will receive an answer, and often get a score and feedback to know how they did. The people who grade the exams are provided training to ensure they evaluate people fairly, consistently, and without bias, and are expert in their field. Now, I know that universities, medical boards, bar exam reviewers, don't always live up to this, but it is the intended goal. I believe that our field, software, subjects people to exams repeatedly, but without any of the bill or rights I descried above. For example, how do you grade? Are you qualified to assess general intelligence? Can you understand that someone might not be eager to submit to your general assessment of their intelligence? Now, if you want to say nobody is entitled to a job, I certainly agree. But the industry seems to think it's entitled to workers, and they just seem utterly blind to how off putting their practices are. And in an industry with concerns about gender, racial, and age discrimination, double secret assessments of people's general intelligence by interviewers of unknown qualifications is about the worst thing you can do.

mixmastamyk 8y ago

Last paragraph is a hidden gem, should be pinned at the top of the page. Perhaps a blog post? :D

liaocs2008 8y ago
Agree. The true story is actually sad.
muddyrivers 8y ago
I agree. On the other hand, we engineers need to look at ourselves in the mirror. After all, we engineers conduct the technical interview. I was involved in hundreds of tech interviews in the past several years. So many times in discussing a candidate's performance, I found some of the interviewers' questions and comments were unrealistic, even preposterous. In one case, an interviewer gave a hard dynamic programming question. We were a tech company and had very good engineers. Our products and our technologies, however, rarely need dynamic programming skills of this difficult level. I couldn't bear it. So I asked, "who think he can solve this problem on the whiteboard in 40 minutes right now?" Dead silence.

solatic 8y ago

> ... because we have no industry wide, respected entrance exam. This. In fact there needs to be one exam, with subsections and subscores per subsection, along with an overall score. Being able to solve an algorithmic challenge on a whiteboard does not mean that you can: a) write readable and maintainable code b) effectively communicate requirements to whoever is actually running your code in production c) know about application-level security vulnerabilities like SQL injections and know not to mindlessly insert them into your code d) instrument your code for metrics and logging in ways that are more or less standard and expected for production monitoring e) engage in effective review of others' code f) use typical API handling techniques which are production-ready, e.g. exponential backoff and circuit breakers. All of which are testable in a certifiable way. Simple truth of the matter is, the vast majority of companies that hire 10 developers who studied efficient algorithms for their interviews, are better served by hiring 9 engineers who are aware of how production and organizational realities affect their work, plus one "performance hacker" who may or may not have a master's degree in CS, whose job is to find the worst real-world production bottlenecks and replace the implementations with algorithmically more efficient implementations (plus stricter performance tests so that the more efficient code doesn't get ripped out later).

irundebian 8y ago
That's an interesting conclusion and it makes pretty sense to me. I've heard that most projects fail due to organizational failures rather than development performance failures. It's probably better to invest in process/quality-aware developers than in rockstar programmers. But I've never read studies about that.
aeorgnoieang 8y ago
How would you test, "in a certifiable way", just the first of your examples "write readable and maintainable code"?
- solatic 8y ago
  The SAT has a writing section - there is no reason you can't write code on a standardized test. The idea is to assign low points to people who write a long function, and high points to people who write many small, self-documenting functions. Such an exam could also ask questions like "what is the cyclometric complexity of the following code?", which while not an indicator that the test taker will always write low-complexity code in a professional environment, at least indicates that the test taker is aware of maintainability metrics like cyclometric complexity - much more than can be said of programmers who mindlessly turn in 400 line functions with deeply nested conditionals.

spirodonfl 8y ago

I like the idea of a proper and respected exam of some kind to test you on a variety of skills that are actually valuable in general and in specific. I've attempted to gather data from colleagues around me but the data is limited. Posting online doesn't get much attention either. But I am trying to gather enough data to see if it would even be possible to create such an exam. If a bunch of developers took a few minutes of their time to answer questions regarding what kinds of things should go into an exam and why (or heck, even a better teaching curriculum than what we have now) I think we could easily come up with something pretty awesome. But, again, gathering that kind of attention and asking developers to provide you with insight is hard. I'd love to tackle it if others were willing to help.