6 ms·

<<Most damningly, he says the questions posed by HackerRank and similar tests aren’t necessarily relevant to the work actually done in finance technology roles:

by carlmr 9y ago

<<Most damningly, he says the questions posed by HackerRank and similar tests aren’t necessarily relevant to the work actually done in finance technology roles: “They’re the sort of problems you’re posed at university. – There’s a focus on data structures and algorithms, but this is less relevant when your success as a developer is in building complex enterprise systems.”>> Exactly on point. Also one gripe I have with hackerrank (I did some of the challenges to brush up on some language knowledge), is that a lot of the questions are poorly designed and from obvious amateurs. I still think that neither whiteboarding nor hackerrank selects for good developers. In the end you get people that studied hard to solve a certain class of problem that you'll never need in the real world. It's wasting candidates time, and it still shows nothing, except that your candidates know something unrelated to the job.

jordanpg 9y ago
Agree, but what is the alternative from the standpoint of the interviewer? These types of challenges are simple to state, self-contained, easy to timebox. For the interviewer for a say intermediate-level enterprise software backend position who needs to quickly sift through 50 applicants, what is the better option by way of code challenges?
- pizzaman09 9y ago
  Credentialing. Seriously. Have a Bar Association for developers. When anyone can do a few weeks of Googling and claim to be a software developer, you're going to get mostly crap candidates. Offload that process to a formal body, like lawyers, doctors, and engineers do. You can hire unlicensed developers, but you do so at your own risk. Then once you're talking to someone licensed in an interview, you can focus on just getting to know them and talking about their past work. Edit: wow, I've clearly touched a nerve. Good thing your doctor is licensed to prescribe something for it instead of a rockstar named Chad who read about MongoDB once but can basically build Uber by this point.
  - icebraining 9y ago
    doctors Because the healthcare job market is such a success! Remember those accusations against tech companies for colluding to depress wages? Now imagine the same, but with State immunity! The anti-trust class-action lawsuit Jung v. AAMC alleged collusion to prevent American trainee doctors from negotiating for better working conditions. The working conditions of medical residents often involved 80- to 100-hour workweeks. The suit had some early success, but failed when the U.S. Congress enacted a statute exempting matching programs from federal anti-trust laws.
    - pizzaman09 9y ago
      I never said "let's make software licensure exactly like healthcare licensure". There are parts of healthcare licensure we could also criticize, but that's not my point. If you'd like to discuss specific parts of softwares licensure, I'm more than willing, but don't engage in faulty reasoning.
      - icebraining 9y ago
        You don't get to decide how it'll shape up; public choice and markets don't work like that. The decision to mandate licensure unleashes forces that very often lead to the kinds of problems that I described above. Unless you have a very good reason why it won't in this case, you're just playing with fire.
  - djhworld 9y ago
    For the professions this is quite simple as there are standards, Doctors, Accountants etc are all trained to the standard qualifications that are often internationally recognised. How could you do that for software engineers? There are so many programming languages, frameworks out there that are constantly evolving and companies have different demands. I think it would be a really hard problem to generalise.
    - wolfgke 9y ago
      > There are so many programming languages, frameworks out there that are constantly evolving and companies have different demands. An idea that I have thought of is to standardize on some programming language/frameworks/demands that are known to be supported for a very long time (15 years?). If you need something from this buffet, you are fine and have the advantages that this provides. If you have special demands, you are on you own - which is not a problem per se, but as all special demands this simply needs to increased risk or cost.
- wolfgke 9y ago
  > For the interviewer for a say intermediate-level enterprise software backend position who needs to quickly sift through 50 applicants, what is the better option by way of code challenges? Patrick McKenzie (patio11) created a lecture about this topic: > http://businessofsoftware.org/2016/07/hiring-at-scale-patrick-mckenzie-starfighter-bos-usa-2015/ http://businessofsoftware.org/2016/07/hiring-at-scale-patric... If you are short of time, here is a summary: > https://42hire.com/hiring-at-scale-c99f0665b893 https://42hire.com/hiring-at-scale-c99f0665b893
- jbob2000 9y ago
  > For the interviewer for a say intermediate-level enterprise software backend position who needs to quickly sift through 50 applicants, what is the better option by way of code challenges? Code challenges are a dead end. You can have someone who passes every code challenge imaginable, but is absolutely awful to work with. Software development is more about communication and problem solving than anything else, so focus on those things first. Technical challenges are easier to overcome than personal ones! Here's my strategy: - Resume smell test to filter out 90% of the applicants (spelling errors, run on sentences, poor formatting, poor experience, etc. If the applicant doesn't care or know enough to fix these on their resume, chances are they will do the same to your code base) - 20min In person or on-camera interview smell test to filter out another 5% (punctuality, check for communication and personality problems, smoke test of technical knowledge) - 1 hour+ in person interview to pick your candidate (deep dives into past projects, talk through or whiteboard a common problem in your industry, meet'n'greet with other team members) - 3 month probation period to fully vet the new candidate and build trust. There's no way that, out of 50 applicants, all of them are going to be equal, don't treat them that way. You need to fast track ones that are very promising and get them to meet with the team asap.
- noxToken 9y ago
  Take some Real World™ problems that are applicable to the positions are create your own. I don't mean come up with something completely from scratch; literally comb through your code base to see what kinds of design patterns and algorithms you're using. Then generalize it. It could be as simple as: Given a list L of accounts, return only accounts from L that match key K. That's programming 101 type stuff that is applicable to the job. Or it could be more abstract and open ended: Given some user-submitted form data, what steps would you take to move the data from the browser's front-end to the database. Assume the database has been created. Tailor the question to the technologies on used on the job (Spring, .NET). This type of question gives you a whole host of information about the candidate such as their overall knowledge of the framework, their areas of expertise, potential weaknesses, as well as the ability to nudge the interviewee in the correct direction if they're stuck. Interview questions don't have to be clever. They don't need to have obscure edge cases to trip people up. I firmly believe that interviews should be about gauging candidate ability to be functional on the job rather than solve the riddle (how many piano tuners are in Chicago?) or non-applicable CS (create a script/program that can output the nth row of Pascal's Triangle [ignoring int overflow]).
on_and_off 9y ago
I recently had a round of interviews for a senior engineering position. Luckily it was on hackerrank, but only as a way to store the exercise instructions with a timer. The test itself was not terribly difficult, build a small mobile app (I am a mobile dev). It was still enough to demonstrate how you can architecture an app, good design patterns, clean code, etc After that I had another round of interview with that same company. I feared that ridiculous whiteboard was on the menu. One of the engineers reassured me when I inquired about the content of this interview : "we think that wb are bullshit, we just want to discuss". In the end I was offered the job, with a really nice compensation package. I should mention that I live on another continent and did all these interviews remotely. Is this a general trend in the silicon valley area ? or was I just lucky to stumble on companies that don't believe in whiteboard questions ?
perlpimp 9y ago
I get idea that these questions are exactly same from that you get from clients and incompetent bosses, where you expected by other side to fill the missing bits and show your brilliant intuition to solve the right problem. they are entertaining to play with(solve), though i haven't encountered any serious algo problems during my line of work in 15 years. my friend, game coder, always does.
pc86 9y ago
Then the issue isn't with the tests necessarily, it's with employers using a test of Skillset A to screen candidates for a job that is 80% Skillset B and 20% Skillset C. That doesn't make the tests ageist (which I suspect was included mostly as click bait), it just makes the hiring managers at these companies bad at their jobs. Raise your hand if hiring managers being bad at screening technical candidates for highly technical roles is surprising to you.
shados 9y ago
> similar tests aren’t necessarily relevant to the work actually done in finance technology roles IMO, what's relevant for a role is the skill set most people currently doing that role have and that's not always a good thing. I've been the first shouting on top of a roof that we shouldn't ask people to know stupid little algorithm trivias to get a job. But the narative for that quickly devolved to "data structures and algorithms are not important for day to day work". To which I honestly say bullshit. The industry is managing without it: between people who forgot everything from school, bootcamps, self taughts, and people with alternate degree, people who can do actual computer science on the job are a tiny, tiny minority. So the solutions those people come up with to day to day problems are considered a speciality (eg: "machine learning!", "data science!"). But if doing that shit was just second nature to everyone, how would it change the way we do things? What if category theory wasn't so scary? What if people weren't scared of threads? What if hashmaps were not magical black boxes people use because the person who wrote the code before them did? The foundation is becoming pretty darn weak if you ask me. I totally agree shit like HackerRank is poorly done. But I'm not sold that the skillset it tries to assess (poorly!) is that useless. We, as a community and an industry, essentially did everything we could to MAKE IT useless. That's different I think.
- Joe-Z 9y ago
  I consider it from a problem-solving perspective: I don't need to know the intricate details of dozens of sorting algorithms or data structures. What I need to be able to do is solve the problems that get thrown at me in daily work. So if I get assigned, say, a performance problem I might, after some research, come up with the idea to use a different data structure to solve it. Or something entirely else! Depends on the problem and where my research leads me. As a software developer I see myself mainly as someone who solves problems to help others do their work better. So, while I agree that having a strong CS background is definitely a benefit, I don't think it matters that much if you can't get binary trees just right after not needing to implement them for years.
  - shados 9y ago
    > So if I get assigned, say, a performance problem I might, after some research, come up with the idea to use a different data structure to solve it. Yup. And my feel on that is that (assuming the perf issue is related to a core data structure), that -particular- perf issue shouldn't even have happened in the first place. It should have been obvious given the right background and you could have spent that time doing something less fundamental.

by carlmr 9y ago

jordanpg 9y ago

Agree, but what is the alternative from the standpoint of the interviewer? These types of challenges are simple to state, self-contained, easy to timebox. For the interviewer for a say intermediate-level enterprise software backend position who needs to quickly sift through 50 applicants, what is the better option by way of code challenges?

pizzaman09 9y ago
Credentialing. Seriously. Have a Bar Association for developers. When anyone can do a few weeks of Googling and claim to be a software developer, you're going to get mostly crap candidates. Offload that process to a formal body, like lawyers, doctors, and engineers do. You can hire unlicensed developers, but you do so at your own risk. Then once you're talking to someone licensed in an interview, you can focus on just getting to know them and talking about their past work. Edit: wow, I've clearly touched a nerve. Good thing your doctor is licensed to prescribe something for it instead of a rockstar named Chad who read about MongoDB once but can basically build Uber by this point.
- icebraining 9y ago
  doctors Because the healthcare job market is such a success! Remember those accusations against tech companies for colluding to depress wages? Now imagine the same, but with State immunity! The anti-trust class-action lawsuit Jung v. AAMC alleged collusion to prevent American trainee doctors from negotiating for better working conditions. The working conditions of medical residents often involved 80- to 100-hour workweeks. The suit had some early success, but failed when the U.S. Congress enacted a statute exempting matching programs from federal anti-trust laws.
  - pizzaman09 9y ago
    I never said "let's make software licensure exactly like healthcare licensure". There are parts of healthcare licensure we could also criticize, but that's not my point. If you'd like to discuss specific parts of softwares licensure, I'm more than willing, but don't engage in faulty reasoning.
    - icebraining 9y ago
      You don't get to decide how it'll shape up; public choice and markets don't work like that. The decision to mandate licensure unleashes forces that very often lead to the kinds of problems that I described above. Unless you have a very good reason why it won't in this case, you're just playing with fire.
- djhworld 9y ago
  For the professions this is quite simple as there are standards, Doctors, Accountants etc are all trained to the standard qualifications that are often internationally recognised. How could you do that for software engineers? There are so many programming languages, frameworks out there that are constantly evolving and companies have different demands. I think it would be a really hard problem to generalise.
  - wolfgke 9y ago
    > There are so many programming languages, frameworks out there that are constantly evolving and companies have different demands. An idea that I have thought of is to standardize on some programming language/frameworks/demands that are known to be supported for a very long time (15 years?). If you need something from this buffet, you are fine and have the advantages that this provides. If you have special demands, you are on you own - which is not a problem per se, but as all special demands this simply needs to increased risk or cost.
wolfgke 9y ago
> For the interviewer for a say intermediate-level enterprise software backend position who needs to quickly sift through 50 applicants, what is the better option by way of code challenges? Patrick McKenzie (patio11) created a lecture about this topic: > http://businessofsoftware.org/2016/07/hiring-at-scale-patrick-mckenzie-starfighter-bos-usa-2015/ http://businessofsoftware.org/2016/07/hiring-at-scale-patric... If you are short of time, here is a summary: > https://42hire.com/hiring-at-scale-c99f0665b893 https://42hire.com/hiring-at-scale-c99f0665b893
jbob2000 9y ago
> For the interviewer for a say intermediate-level enterprise software backend position who needs to quickly sift through 50 applicants, what is the better option by way of code challenges? Code challenges are a dead end. You can have someone who passes every code challenge imaginable, but is absolutely awful to work with. Software development is more about communication and problem solving than anything else, so focus on those things first. Technical challenges are easier to overcome than personal ones! Here's my strategy: - Resume smell test to filter out 90% of the applicants (spelling errors, run on sentences, poor formatting, poor experience, etc. If the applicant doesn't care or know enough to fix these on their resume, chances are they will do the same to your code base) - 20min In person or on-camera interview smell test to filter out another 5% (punctuality, check for communication and personality problems, smoke test of technical knowledge) - 1 hour+ in person interview to pick your candidate (deep dives into past projects, talk through or whiteboard a common problem in your industry, meet'n'greet with other team members) - 3 month probation period to fully vet the new candidate and build trust. There's no way that, out of 50 applicants, all of them are going to be equal, don't treat them that way. You need to fast track ones that are very promising and get them to meet with the team asap.
noxToken 9y ago
Take some Real World™ problems that are applicable to the positions are create your own. I don't mean come up with something completely from scratch; literally comb through your code base to see what kinds of design patterns and algorithms you're using. Then generalize it. It could be as simple as: Given a list L of accounts, return only accounts from L that match key K. That's programming 101 type stuff that is applicable to the job. Or it could be more abstract and open ended: Given some user-submitted form data, what steps would you take to move the data from the browser's front-end to the database. Assume the database has been created. Tailor the question to the technologies on used on the job (Spring, .NET). This type of question gives you a whole host of information about the candidate such as their overall knowledge of the framework, their areas of expertise, potential weaknesses, as well as the ability to nudge the interviewee in the correct direction if they're stuck. Interview questions don't have to be clever. They don't need to have obscure edge cases to trip people up. I firmly believe that interviews should be about gauging candidate ability to be functional on the job rather than solve the riddle (how many piano tuners are in Chicago?) or non-applicable CS (create a script/program that can output the nth row of Pascal's Triangle [ignoring int overflow]).

on_and_off 9y ago

I recently had a round of interviews for a senior engineering position. Luckily it was on hackerrank, but only as a way to store the exercise instructions with a timer. The test itself was not terribly difficult, build a small mobile app (I am a mobile dev). It was still enough to demonstrate how you can architecture an app, good design patterns, clean code, etc After that I had another round of interview with that same company. I feared that ridiculous whiteboard was on the menu. One of the engineers reassured me when I inquired about the content of this interview : "we think that wb are bullshit, we just want to discuss". In the end I was offered the job, with a really nice compensation package. I should mention that I live on another continent and did all these interviews remotely. Is this a general trend in the silicon valley area ? or was I just lucky to stumble on companies that don't believe in whiteboard questions ?

perlpimp 9y ago

I get idea that these questions are exactly same from that you get from clients and incompetent bosses, where you expected by other side to fill the missing bits and show your brilliant intuition to solve the right problem. they are entertaining to play with(solve), though i haven't encountered any serious algo problems during my line of work in 15 years. my friend, game coder, always does.

pc86 9y ago

Then the issue isn't with the tests necessarily, it's with employers using a test of Skillset A to screen candidates for a job that is 80% Skillset B and 20% Skillset C. That doesn't make the tests ageist (which I suspect was included mostly as click bait), it just makes the hiring managers at these companies bad at their jobs. Raise your hand if hiring managers being bad at screening technical candidates for highly technical roles is surprising to you.

shados 9y ago

> similar tests aren’t necessarily relevant to the work actually done in finance technology roles IMO, what's relevant for a role is the skill set most people currently doing that role have and that's not always a good thing. I've been the first shouting on top of a roof that we shouldn't ask people to know stupid little algorithm trivias to get a job. But the narative for that quickly devolved to "data structures and algorithms are not important for day to day work". To which I honestly say bullshit. The industry is managing without it: between people who forgot everything from school, bootcamps, self taughts, and people with alternate degree, people who can do actual computer science on the job are a tiny, tiny minority. So the solutions those people come up with to day to day problems are considered a speciality (eg: "machine learning!", "data science!"). But if doing that shit was just second nature to everyone, how would it change the way we do things? What if category theory wasn't so scary? What if people weren't scared of threads? What if hashmaps were not magical black boxes people use because the person who wrote the code before them did? The foundation is becoming pretty darn weak if you ask me. I totally agree shit like HackerRank is poorly done. But I'm not sold that the skillset it tries to assess (poorly!) is that useless. We, as a community and an industry, essentially did everything we could to MAKE IT useless. That's different I think.

Joe-Z 9y ago
I consider it from a problem-solving perspective: I don't need to know the intricate details of dozens of sorting algorithms or data structures. What I need to be able to do is solve the problems that get thrown at me in daily work. So if I get assigned, say, a performance problem I might, after some research, come up with the idea to use a different data structure to solve it. Or something entirely else! Depends on the problem and where my research leads me. As a software developer I see myself mainly as someone who solves problems to help others do their work better. So, while I agree that having a strong CS background is definitely a benefit, I don't think it matters that much if you can't get binary trees just right after not needing to implement them for years.
- shados 9y ago
  > So if I get assigned, say, a performance problem I might, after some research, come up with the idea to use a different data structure to solve it. Yup. And my feel on that is that (assuming the perf issue is related to a core data structure), that -particular- perf issue shouldn't even have happened in the first place. It should have been obvious given the right background and you could have spent that time doing something less fundamental.