7 ms·

I'm sure anyone determined to do so can act unfairly regardless of what process is in place, but the fact that there is a standardized test in my mind does the

I'm sure anyone determined to do so can act unfairly regardless of what process is in place, but the fact that there is a standardized test in my mind does the opposite and makes the process much fairer. Assuming a fair-minded interviewer, the process gives a chance to a candidate whose resume may have less vaunted names on it to demonstrate their skill. I'm quite sure that I'd never have had some of the opportunities I have, not having a CS degree, if it weren't for whiteboarding interviews. I can't imagine any possible process that would thwart interviewers intentionally subverting it to hire their friends.

Benjammer 2y ago
So are we just going with a base assumption that interviewers can NEVER be trusted with anti-bias training and learning how evaluate people fairly? The examples mentioned in this comment section are all blatantly intentional biases that people are choosing to use. The amazing part is that all the “standard test eliminates bias” people seem to the most ignorant to where bias helps THEM. Forcing people to study for two months is blatantly discriminatory to age and family status, at a systemic level. While “this white straight guy might explicitly choose to give the other white straight guy an easy question,” is very subjective and intentional on the individual level. Like, employees can always choose to do bad things, in any situation. That’s why we have at-will employment… Is the culture just so broken at these companies that it’s hopeless to expect people NOT to blatantly exploit the system for their friends? Why don’t people get fired for doing that?
- Aunche 2y ago
  > Forcing people to study for two months is blatantly discriminatory to age and family status, at a systemic level. How is this any less discriminatory than any other assessment based interview where you need to prepare? Non-assessment based interviews end up being vibes based which is much more discriminatory.
  - angoragoats 2y ago
    You’ve set up a false dichotomy here; for any given position, there is typically a way to conduct an assessment-based interview that doesn’t require too much preparation for qualified candidates. For example, at my current job, we hire web developers with Rails experience. Our technical interview process consists of either a pair programming session or an async/take-home task (candidate’s choice) which requires the candidate to implement a small feature in a Rails codebase. We do have some room to improve on objective evaluation of the candidate’s performance, but there is a test suite and a rubric which we use to evaluate their work. None of this should require that the candidate study, unless they’re coming in to the interview without Rails experience.
    - Aunche 2y ago
      This may work out great if you happen to have worked on Rails for your last job. However, I doubt that everyone you interview is actually that familiar with Rails but rather is pursuing any sort of opportunity that they can get. In that case, they would actually more time to brush up multiple different tech stacks than simply on algorithms.
      - angoragoats 2y ago
        In that case, the interview question is working as intended. For most of our roles, we want several years of Rails experience, and we are clear about that fact in the job listing. If someone applies without Rails experience, they either didn't read the job listing, or are desperate to find any job. While I empathize with folks in the latter situation, our positions really do require the experience, and the job market isn't so bad right now that a smart candidate should be going a long time without finding something. If you happen to have Rails experience but it was several jobs ago, the task we give you is basic enough that you should be able to Google what you need during the task to refresh your memory fairly quickly. In fact, I did this when I applied, having not worked with Rails in a number of years. Edit: My main point is, even if you technically do have to "study" (really, just Google a few basic Rails concepts) if you're rusty, everything you do is preparing you for the actual job. Studying how to implement a hashmap, or computing the longest palindrome from a string of characters, or whatever other harebrained problem FAANG etc want to ask, is 99% of the time not really helping you prepare for those jobs.
- johnnyanmac 2y ago
  >are we just going with a base assumption that interviewers can NEVER be trusted with anti-bias training and learning how evaluate people fairly? Yes. Because interviewing is 1. hard, but no company has proper full time proctors. So "expert interviewers" are a rarity 2. not standardized in the slightest. So your performance varies entirely by the interviewer, their style, and their mood that day. 3. some weird blind test where you hope you studied the right topics. You're not getting the best out of a candidate, you're basically hoping they read your mind and know exactly what you want them to say. >Is the culture just so broken at these companies that it’s hopeless to expect people NOT to blatantly exploit the system for their friends? Why don’t people get fired for doing that? Sure. but that's a universal problem. "It's who you know, not what you know" is advice that has spanned centuries. I can't even blame the modern tech industry for that one. That + the above issues with interviewing mean you're always going to go with a referral over a random applicant.
  - bluGill 2y ago
    > 2. not standardized in the slightest. So your performance varies entirely by the interviewer, their style, and their mood that day. That isn't always true. When I interview HR gives me the exact questions I'm allowed to ask. These are vetted both to prevent me from asking something illegal, and also by research to get the type of things useful for interviews. Sometimes it is annoying - you can easially finish the interview with a great score but I have no clue if you can write code or not. However we are carefully trained on how to ask the questions and how to grade them.
    - johnnyanmac 2y ago
      That makes sense for soft questions. But I doubt it's HR devevloping a dynamic programming problem and writing a rubric for how good a score you can give based on a response. I was mostly referring to technical tests, but I understand there are definitely some set of questions you need to ask no matter what. I don't really knock recruiters too much for repeating the usual "are you authorized to work in the US" kinda stuff even though it is the first question on their job application.
  - 2y ago
- emodendroket 2y ago
  > Is the culture just so broken at these companies that it’s hopeless to expect people NOT to blatantly exploit the system for their friends? Why don’t people get fired for doing that? How do you catch them? Unless they’re idiots they’ll write facially plausible explanations of their hire/no-hire decisions.
369548684892826 2y ago
> the fact that there is a standardized test in my mind does the opposite and makes the process much fairer The problem isn't standardized tests, it's that leetcode questions are about having the time to have learnt the answer beforehand, rather than raw ability for problem solving.
- jonkho 2y ago
  That’s not true. In any discipline when confronted with a test there are two strategies: brute memorization of the question/answers, or developing the skills to tackle the problem dynamically. You cannot categorically claim that LC tests are largely memorization tests rather than raw problem-solving skills. That is just the approach you are capable of taking. Not being able to see up the mountain doesn’t imply there are no climbers above you.
  - Tade0 2y ago
    That's exactly true. LC tests typically copy problems from the university the interviewer graduated from. College programs differ, so this is really a case of what you were introduced to. There's a fairly popular online LC test company in my corner of the world which was formed by graduates and lecturers from a certain university and they started out by just giving the problems from the curriculum. Result was heavy bias in favour of students and alumni of that university.
  - KaiserPro 2y ago
    > You cannot categorically claim that LC tests are largely memorization tests rather than raw problem-solving skills If that were the case then a normal, well accomplished software engineer shouldn't need to "grind" leetcode to pass an interview. its just a cargo cult. Getting someone to do a code review is a much much better test of skill: Do they ask questions? are they kind in their assertions? At what point do they go "I don't know"? Do they concentrate on style or substance? do they raise an issue with a lack of comments? do they ask why the description of the PR is so vague? When they get a push back, are they aggressive? All of those are much better tests than "rewrite a thing that a library would do for you already"
scott_w 2y ago
> the fact that there is a standardized test in my mind does the opposite and makes the process much fairer. It can make the process fairer but it's not a given. You can do the classic "no dogs, no blacks no Irish," and that was a standard across pubs in England. It certainly wasn't fair, unless you were racist. If you're committed to fairness, then a standard will help. It gives you a clear point to fix and improve things and something you can use to measure if you're achieving your stated goals. That's definitely something you can't do if you're just making things up as you go along. And yes, I made a deliberately provocative statement. I'm obviously not saying whiteboard tests are the equivalent of segregation.
jkukul 2y ago
A leet-code test would be much more standardized if candidates could solve it at home. Just send me a link to the quiz and let me solve it within a specified time frame. I've done tests like this for some companies. It felt a lot fairer and more closely resembling the actual work environment than live leet-code interviews, with biased interviewer(s) and a stress factor that's not a part of the actual job.
- skeeter2020 2y ago
  As a hiring manager I HATE leet-code tests, and they do nothing to differentiate candidates, but a take home in the era where people run chatGPT beside the interview window, or have someone else do the interview for them? Not a chance. You are 100% correct that it is way more representative, but the prevalence of cheating is ridiculous.
  - oytis 2y ago
    Take home is fine if you discuss it later in the interview. But also there should be some pre-screening to keep the number of interviewees reasonable.
  - jkukul 2y ago
    I totally understand you, but want to offer a different perspective. They will also be able to use ChatGPT on the job. And StackOverflow. And Google. If they know how to use tools available to solve a problem, that will benefit them on the job. If you're testing them for what ChatGPT can already solve, then are the skills being tested worth anything, in this day and age? Take-home LeetCode, even with cheating will still filter out a good chunk of candidates. Those who are not motivated enough or those who don't even know how to use the available help. You'll still be able to rank those who solved the task. You'll still see the produced code and be able to judge it. Like other commenter points out, you can always follow up the take-home LeetCode. Usually, it becomes apparent really quickly if a candidate solved it on their own.
  - CoastalCoder 2y ago
    This does seem like a vexing problem, especially when interviews are conducted remotely. I wonder if either of the following could be cost-effective: (a) Fly the candidate to a company office, where their compute usage could be casually monitored by an employee. (b) Use high-quality proctoring services that are nearby to the candidate. E.g., give them 1-2 days in a coworking space, and hire a proctor to verify that thy're not egregiously using tools like ChatGPT. Or alternatively, would it suffice to just have a long conversation with the candidate about their solution? E.g. what design trade-offs they considered, or how might they adapt their solution to certain changes in the requirements.
bossyTeacher 2y ago
> I'm sure anyone determined to do so can act unfairly regardless of what process is in place, but the fact that there is a standardized test in my mind does the opposite and makes the process much fairer. This is what privilege looks like. The inability to see barriers that affect others in a worse position. Standards do not imply fairness only consistency. You got a test that filters out people who lack the time to train for these tests. Basically, devs with a life (see family)
- CoastalCoder 2y ago
  IIUC, in your view using standard tests is a damned if you do, damn't if you don't scenario. Is there a solution that you'd recommend?
- emodendroket 2y ago
  OK, but I have a two-year-old, so consider my privilege sufficiently "checked" before having written the post. Anyway, what would you suggest that would be more fair?
civilized 2y ago
> the fact that there is a standardized test ...a standardized test? No. There are tests. They sure as heck aren't standardized. Maybe they should be, since everyone seems to be doing the same thing.
- emodendroket 2y ago
  Within one organization the form is standardized, and across companies it's very similar. But sure, maybe not the right word. I'm comparing it to "just letting the interviewers chat and decide based on whatever" which is typical in industries without a skills test.
  - civilized 2y ago
    Leetcode-style tech interviews fall far short of standardized tests. In real standardized tests, the same questions are given to everyone, and effort is made to develop new questions not known to the test takers. They're more like a standardized test than a pure shoot-the-shit interview is, but still pretty far away.
    - emodendroket 2y ago
      OK, I grant the point but it doesn't fundamentally change my argument.

Benjammer 2y ago

So are we just going with a base assumption that interviewers can NEVER be trusted with anti-bias training and learning how evaluate people fairly? The examples mentioned in this comment section are all blatantly intentional biases that people are choosing to use. The amazing part is that all the “standard test eliminates bias” people seem to the most ignorant to where bias helps THEM. Forcing people to study for two months is blatantly discriminatory to age and family status, at a systemic level. While “this white straight guy might explicitly choose to give the other white straight guy an easy question,” is very subjective and intentional on the individual level. Like, employees can always choose to do bad things, in any situation. That’s why we have at-will employment… Is the culture just so broken at these companies that it’s hopeless to expect people NOT to blatantly exploit the system for their friends? Why don’t people get fired for doing that?

Aunche 2y ago
> Forcing people to study for two months is blatantly discriminatory to age and family status, at a systemic level. How is this any less discriminatory than any other assessment based interview where you need to prepare? Non-assessment based interviews end up being vibes based which is much more discriminatory.
- angoragoats 2y ago
  You’ve set up a false dichotomy here; for any given position, there is typically a way to conduct an assessment-based interview that doesn’t require too much preparation for qualified candidates. For example, at my current job, we hire web developers with Rails experience. Our technical interview process consists of either a pair programming session or an async/take-home task (candidate’s choice) which requires the candidate to implement a small feature in a Rails codebase. We do have some room to improve on objective evaluation of the candidate’s performance, but there is a test suite and a rubric which we use to evaluate their work. None of this should require that the candidate study, unless they’re coming in to the interview without Rails experience.
  - Aunche 2y ago
    This may work out great if you happen to have worked on Rails for your last job. However, I doubt that everyone you interview is actually that familiar with Rails but rather is pursuing any sort of opportunity that they can get. In that case, they would actually more time to brush up multiple different tech stacks than simply on algorithms.
    - angoragoats 2y ago
      In that case, the interview question is working as intended. For most of our roles, we want several years of Rails experience, and we are clear about that fact in the job listing. If someone applies without Rails experience, they either didn't read the job listing, or are desperate to find any job. While I empathize with folks in the latter situation, our positions really do require the experience, and the job market isn't so bad right now that a smart candidate should be going a long time without finding something. If you happen to have Rails experience but it was several jobs ago, the task we give you is basic enough that you should be able to Google what you need during the task to refresh your memory fairly quickly. In fact, I did this when I applied, having not worked with Rails in a number of years. Edit: My main point is, even if you technically do have to "study" (really, just Google a few basic Rails concepts) if you're rusty, everything you do is preparing you for the actual job. Studying how to implement a hashmap, or computing the longest palindrome from a string of characters, or whatever other harebrained problem FAANG etc want to ask, is 99% of the time not really helping you prepare for those jobs.
johnnyanmac 2y ago
>are we just going with a base assumption that interviewers can NEVER be trusted with anti-bias training and learning how evaluate people fairly? Yes. Because interviewing is 1. hard, but no company has proper full time proctors. So "expert interviewers" are a rarity 2. not standardized in the slightest. So your performance varies entirely by the interviewer, their style, and their mood that day. 3. some weird blind test where you hope you studied the right topics. You're not getting the best out of a candidate, you're basically hoping they read your mind and know exactly what you want them to say. >Is the culture just so broken at these companies that it’s hopeless to expect people NOT to blatantly exploit the system for their friends? Why don’t people get fired for doing that? Sure. but that's a universal problem. "It's who you know, not what you know" is advice that has spanned centuries. I can't even blame the modern tech industry for that one. That + the above issues with interviewing mean you're always going to go with a referral over a random applicant.
- bluGill 2y ago
  > 2. not standardized in the slightest. So your performance varies entirely by the interviewer, their style, and their mood that day. That isn't always true. When I interview HR gives me the exact questions I'm allowed to ask. These are vetted both to prevent me from asking something illegal, and also by research to get the type of things useful for interviews. Sometimes it is annoying - you can easially finish the interview with a great score but I have no clue if you can write code or not. However we are carefully trained on how to ask the questions and how to grade them.
  - johnnyanmac 2y ago
    That makes sense for soft questions. But I doubt it's HR devevloping a dynamic programming problem and writing a rubric for how good a score you can give based on a response. I was mostly referring to technical tests, but I understand there are definitely some set of questions you need to ask no matter what. I don't really knock recruiters too much for repeating the usual "are you authorized to work in the US" kinda stuff even though it is the first question on their job application.
- 2y ago
emodendroket 2y ago
> Is the culture just so broken at these companies that it’s hopeless to expect people NOT to blatantly exploit the system for their friends? Why don’t people get fired for doing that? How do you catch them? Unless they’re idiots they’ll write facially plausible explanations of their hire/no-hire decisions.

369548684892826 2y ago

> the fact that there is a standardized test in my mind does the opposite and makes the process much fairer The problem isn't standardized tests, it's that leetcode questions are about having the time to have learnt the answer beforehand, rather than raw ability for problem solving.

jonkho 2y ago
That’s not true. In any discipline when confronted with a test there are two strategies: brute memorization of the question/answers, or developing the skills to tackle the problem dynamically. You cannot categorically claim that LC tests are largely memorization tests rather than raw problem-solving skills. That is just the approach you are capable of taking. Not being able to see up the mountain doesn’t imply there are no climbers above you.
- Tade0 2y ago
  That's exactly true. LC tests typically copy problems from the university the interviewer graduated from. College programs differ, so this is really a case of what you were introduced to. There's a fairly popular online LC test company in my corner of the world which was formed by graduates and lecturers from a certain university and they started out by just giving the problems from the curriculum. Result was heavy bias in favour of students and alumni of that university.
- KaiserPro 2y ago
  > You cannot categorically claim that LC tests are largely memorization tests rather than raw problem-solving skills If that were the case then a normal, well accomplished software engineer shouldn't need to "grind" leetcode to pass an interview. its just a cargo cult. Getting someone to do a code review is a much much better test of skill: Do they ask questions? are they kind in their assertions? At what point do they go "I don't know"? Do they concentrate on style or substance? do they raise an issue with a lack of comments? do they ask why the description of the PR is so vague? When they get a push back, are they aggressive? All of those are much better tests than "rewrite a thing that a library would do for you already"

scott_w 2y ago

> the fact that there is a standardized test in my mind does the opposite and makes the process much fairer. It can make the process fairer but it's not a given. You can do the classic "no dogs, no blacks no Irish," and that was a standard across pubs in England. It certainly wasn't fair, unless you were racist. If you're committed to fairness, then a standard will help. It gives you a clear point to fix and improve things and something you can use to measure if you're achieving your stated goals. That's definitely something you can't do if you're just making things up as you go along. And yes, I made a deliberately provocative statement. I'm obviously not saying whiteboard tests are the equivalent of segregation.

jkukul 2y ago

A leet-code test would be much more standardized if candidates could solve it at home. Just send me a link to the quiz and let me solve it within a specified time frame. I've done tests like this for some companies. It felt a lot fairer and more closely resembling the actual work environment than live leet-code interviews, with biased interviewer(s) and a stress factor that's not a part of the actual job.

skeeter2020 2y ago
As a hiring manager I HATE leet-code tests, and they do nothing to differentiate candidates, but a take home in the era where people run chatGPT beside the interview window, or have someone else do the interview for them? Not a chance. You are 100% correct that it is way more representative, but the prevalence of cheating is ridiculous.
- oytis 2y ago
  Take home is fine if you discuss it later in the interview. But also there should be some pre-screening to keep the number of interviewees reasonable.
- jkukul 2y ago
  I totally understand you, but want to offer a different perspective. They will also be able to use ChatGPT on the job. And StackOverflow. And Google. If they know how to use tools available to solve a problem, that will benefit them on the job. If you're testing them for what ChatGPT can already solve, then are the skills being tested worth anything, in this day and age? Take-home LeetCode, even with cheating will still filter out a good chunk of candidates. Those who are not motivated enough or those who don't even know how to use the available help. You'll still be able to rank those who solved the task. You'll still see the produced code and be able to judge it. Like other commenter points out, you can always follow up the take-home LeetCode. Usually, it becomes apparent really quickly if a candidate solved it on their own.
- CoastalCoder 2y ago
  This does seem like a vexing problem, especially when interviews are conducted remotely. I wonder if either of the following could be cost-effective: (a) Fly the candidate to a company office, where their compute usage could be casually monitored by an employee. (b) Use high-quality proctoring services that are nearby to the candidate. E.g., give them 1-2 days in a coworking space, and hire a proctor to verify that thy're not egregiously using tools like ChatGPT. Or alternatively, would it suffice to just have a long conversation with the candidate about their solution? E.g. what design trade-offs they considered, or how might they adapt their solution to certain changes in the requirements.

bossyTeacher 2y ago

> I'm sure anyone determined to do so can act unfairly regardless of what process is in place, but the fact that there is a standardized test in my mind does the opposite and makes the process much fairer. This is what privilege looks like. The inability to see barriers that affect others in a worse position. Standards do not imply fairness only consistency. You got a test that filters out people who lack the time to train for these tests. Basically, devs with a life (see family)

CoastalCoder 2y ago
IIUC, in your view using standard tests is a damned if you do, damn't if you don't scenario. Is there a solution that you'd recommend?
emodendroket 2y ago
OK, but I have a two-year-old, so consider my privilege sufficiently "checked" before having written the post. Anyway, what would you suggest that would be more fair?

civilized 2y ago

> the fact that there is a standardized test ...a standardized test? No. There are tests. They sure as heck aren't standardized. Maybe they should be, since everyone seems to be doing the same thing.

emodendroket 2y ago
Within one organization the form is standardized, and across companies it's very similar. But sure, maybe not the right word. I'm comparing it to "just letting the interviewers chat and decide based on whatever" which is typical in industries without a skills test.
- civilized 2y ago
  Leetcode-style tech interviews fall far short of standardized tests. In real standardized tests, the same questions are given to everyone, and effort is made to develop new questions not known to the test takers. They're more like a standardized test than a pure shoot-the-shit interview is, but still pretty far away.
  - emodendroket 2y ago
    OK, I grant the point but it doesn't fundamentally change my argument.